< back to list

RFC 3: Tilde Center Specification

Author: Austin Ewens <aewens@tilde.center>

Status: Accepted


Abstract

This document outlines the core philosophy and components that the Tilde Center project is composed of, as well as laying out the fundation to bootstrap the rest of the project for future modifications and expansion to create a sustainable, self-reliant, decentralized network of tilde servers.

Introduction

The Tilde Center (herein referred to as ~center) project was created by Austin Ewens (better known in the community as ~aewens), in December 2018. The goal of the project is to create a decentralized and federated server architecture built upon home-brewed open source projects (herein referred to as HBOSP) made by and maintained by its user base.

Terminology

To assist in clarifying the intentions of the project, this section will outline the meanings behind the terminology used thought the document:

Clarifications

Before going any further into the specifics of the ~center project, some clarifications should be made.

Firstly, the Tildeverse and Tilde Center are two distinctly different organizations (for lack of a better word). While the initial ~center server created by the project's founder, ~aewens, the other peer servers do not need to be members of the Tildeverse to be a part of the ~center project.

Secondly, the peer servers in the ~center project are not clones of each other. They are unique servers managed by their own system administrator (aka sysadmin), but they share the user accounts along with select data and services with other servers to create a decentralized network of services for its users.

Peer Servers

To better understand the peer servers in the Tilde Center Network (herein referred to as TCN) it helps to define what they will and will not do:

While this list may grow and/or change in future revisions of this document, in general a server in the TCN will share fundamental components with its peers while still having the ability to have unique characteristics (i.e. the peers will not be direct clones of one another).

The next few sections will outline some more of the specifics for the points listed above.

Prerequisites

Due to the nature of the project, it would preferable to have more rather than less peer servers in the TCN for the sake of decentralization (the more peers in the network, the more resilient it is). For this reason, it is in the best interest of the project to minimize any factors that would inhibit potential servers from becoming peers in the TCN. This is one of the reasons why peer servers are not just clones of each other because this would at the very least require all peers in the network to upgrade the hardware for their storage to meet the demands of new users joining the network.

With this in mind, all current and future services that will be federated across the TCN should aim to minimize the data that actually needs to be stored and processed by all of its peers to make it easier for new peer servers to join the network.

Center Directory

On all peer servers there will be a /center directory that will contain the bulk of the ~center related files. Within this directory will be the following sub-directories:

Aside from /center/home, all other sub-directories under /center will be synchronized with the other peer servers.

Dispatcher

Arguably one of the most important components of the ~center server components, the dispatcher will be in charge of federating data across the TCN and handling the logistics of any mention of syncing data in this document. This will be accomplished by having a socket server and client combo that can read and execute a set of instructions sent from other servers. To reduce the possibilities of this becoming a security vulnerability for remote code execution (RCE), it is important that the dispatcher only receive requests and deliver them to the appropriate targets without ever directly executing the instructions.

Communication Model

For this to work, dispatcher will use a publish-subscribe communication model where services can subscribe to specific events from the dispatcher and, if approved by the dispatcher (which will most likely be handled via an action from the sysadmin of the peer server), will receive those events upon being received. The messages the dispatcher will handle will be JSON in the following format:


    {
        "name": "name-of-event",
        "meta": {
            "from": "uuid-of-sender",
            "to": "uuid-of-receiver",
            "iat": "timestamp-when-message-sent",
            "exp": "timestamp-when-message-expires"
        },
        "data": {
            ...
        }
    }

Along with this message will be a GPG signature of the message prefixed with the code "GPG+" and the message itself will be encrypted with a key made from the Double Ratchet algorithm (the logistics of which will be explained in a moment). The message format provides the name of the event being sent, where it came from, who the message was meant for, when it was sent, when it should expire, and all the data that should be passed on to the subscribers. Since the messages are sent and received over a socket, the dispatcher will already know the IP address of the sender to verify the signature of the message and know which key to use for decrypting before parsing its contents, but should still verify that the metadata in the message is valid before making any additional actions on the rest of the message.

Stored Information

For the dispatcher to properly handle communication with the other peers, it will need to maintain a list of all known peer servers via their last known IP address, their universally unique identifier (aka UUID), the GPG public key of the server, and the shared secret key used with that peer server. While the shared secret key chain should be stored safely and securely, the IP addresses and UUIDs of the peer servers should be stored in a distributed hash table (aka DHT), specifically a Kademlia DHT, to optimize the fact that the TCN is a decentralized network with the dispatcher communicating in a peer-to-peer (aka P2P) system.

As well, the dispatcher will locally need to maintain a list of services subscribed to its feed, what events they are allowed to receive, and what events they are allowed to send. Each of these will need to be approved by the sysadmin to ensure only valid services are listening to the dispatcher, only receiving appropriate events, and only sending out appropriate events to the peer servers.

Adding New Peers

To facilitate the process of authenticating messages from server-to-server, when a new peer server is created it will generate a UUID and GPG key pair for itself followed by sending out a message in the following format to one or more existing peer servers:


    {
        "name": "new-peer-server",
        "meta": {
            "from": "uuid-of-sender",
            "to": "uuid-of-receiver",
            "iat": "timestamp-when-message-sent",
            "exp": "timestamp-when-message-expires"
        },
        "data": {
            "admin": "username",
            "server": "subdomain.domain.tld:ip-address",
            "secret_file": "secret_file_name",
            "dh_pub_key": "diffie-hellman-public-key",
            "gpg_pub_key": "gpg-public-key",
            "signature": "gpg-made-signature"
        }
    }

The message will be sent in plain-text and in place of the signature it will instead send a string of 64 zeros (i.e. an empty SHA256) with the prefix code "NEW" so that the dispatcher knows to handle the message as a new peer server. Additional logistics to how the on-boarding process behind new servers joining the TCN can be found at Joining The Network but for the sake of this section the "admin" field is the username of the sysadmin on the peer server the message was sent to, "server" is the domain name and IP address of the server sending the message, "secret_file" is the name of a file under the $HOME directory of the username specified by "admin", "dh_pub_key" is the public key used in the Diffie-Hellman key exchange process (this should be different for each peer server the message is sent to), and "gpg_pub_key" is the public GPG key generated earlier, and "signature" is the phrase contents of the "secret_file" signed using the GPG private key. The secret file will be readable only to the "admin" user as well as contain a value known ahead of time by the peer server sysadmin through prior communications and will be used to validate that the sysadmin of the new server is indeed the same user the sysadmin of the existing peer server expects. If there is any doubt that the sender of the message is not from the expected user / server, this process can be repeated using different secret files to employ a zero knowledge proof (aka ZKP) to raise the confidence that the sender is who they claim to be. Once the sysadmin of the peer server trusts the new server, it will send out its a message in the following form:


    {
        "name": "add-peer-server",
        "meta": {
            "from": "uuid-of-sender",
            "to": "uuid-of-receiver",
            "iat": "timestamp-when-message-sent",
            "exp": "timestamp-when-message-expires"
        },
        "data": {
            "dh_pub_key": "diffie-hellman-public-key"
        }
    }

With the signature used being the message encrypted and signed using the peer server's private GPG key with the prefix code "GPG". Since the new server joining should have the GPG public key ahead of time for the peer server, the dispatcher can use this to verify the authenticity of the peer server. After this message, both servers now can use the Diffie-Hellman public key to of the other and can now create a shared key to communicate with each other. All responses proceeding between the two servers will be signed using the key created through the Double Ratchet algorithm and the servers can now add the other to their roster of peers. Once the peer server receives back a response from the new server using the GPG+ method, it will send to the new server its DHT of peers to the new server along with sending out a message containing the domain name, IP address, UUID, and GPG public key of the new server to its own peers so that they can issue their own "add-peer-server" event to the new server to establish communication with its dispatcher. From which, the new server will reply with its own "add-peer-server" message and will add the peer server if it responds with a valid GPG+ message.

It is, of course, up to the sysadmins of the other peer servers whether they send the "add-peer-server" message to the new server, but since these servers will now have the information about the new server already the new server can always issue its own "add-peer-server" or "new-peer-server" message to the other peers manually if it wants to proactively establish communication with the other peers. However, the dispatcher must communicate the new peer server to the rest of its peers so that other peer servers in the TCN can receive dispatcher messages from the new peer server.

User Accounts

When a user joins the TCN, an account is created on the peer server the joined from, which is then subsequently created on all other peer servers. Also, when a new peer server joins the TCN, all existing user accounts will need to be added to their server prior to becoming a full-fledged peer server (this will of course be an automated process).

The user accounts themselves on the peer servers will be stored in an LDAP database and their SSH keys will be stored inside their shared directory. To preserve security of the LDAP database, it will use TLS from SSL certificates signed by a customer certificate authority (thus isolating the usage of the LDAP database to just the server itself). Additionally, the ppolicy in LDAP should also be applied to lock the account after a certain amount of failed authentication attempts to prevent any brute force attacks by the servers own users. For supported distributions, this setup will of course be automated.

The rationale for using LDAP is that it provides a resource that users can take advantage of when creating services on the ~center platform. There would be no need for users to create an account for every service on the server, but rather they can simply use their existing account by having the service just authenticate against the LDAP database. However, should a user create an alternative to LDAP to provide the same features utilized here, it can be put to a vote to use this service instead in favor of the HBOSP philosophy of the ~center project.

Additionally, during the process of new users being added into the TCN the peer server the user joined from along with the date they were added should be logged within the ~center database on the peer node. This can be useful for both observing how the TCN is growing along with help to investigate or protect against any rogue peer servers.

Shared Directory

When a user gets an account on a peer server, they will be provided with a size limited directory to store any files they wish to be available across the other peer servers. The reason the directory is size limited is to keep with minimizing the prerequisites for peer servers as the number of users in the TCN grows. The size of the shared directory will initially be five megabytes per user with the potential to grow if decided later on by the user base (see the later section on Governance). This size was chosen because it is large enough to hold dotfiles, emails, and a few images while still scaling relatively well as more users join the network (e.g. even if the TCN reaches 1000 users, only ~5GB of space will be needed to provide this feature).

The actual mechanism to employ this feature can be achieved using file system quotas. Like the other features outlined in the ~center project, this will of course be automated to simplify the deployment process as new users join the network. Additionally, the contents of these directories will sync all changes made to the rest of the peer servers (as opposed to periodically copying the directory as a whole which would quickly congest the network as users / peer servers join the network).

Additionally, while the creation / deletion of the shared directory on other peer servers for users will be facilitated using the dispatcher, the actual syncing of changes to the directory will be initially handled through rsync to the peer servers. Should a service be created by one of the users to replace the functionality rsync for this process, it can be put to a vote to use this service instead in favor of the HBOSP philosophy of the ~center project.

Database

While the LDAP database takes care of handling the user accounts, peer servers will also need to maintain an SQL database to be used holding the data utilized by the services used throughout the TCN. Also, while the system administrator can choose to use a different SQL database than the one setup by default using the sysadmin scripts, any changes needed to support this SQL database would initially need to be done by the sysadmin wanting to employ this change.

For the SQL database to be compliant with the ~center project, the database should have the following tables:

This is designed to provide the minimal amount of database tables for most services without any additional tables needed to be added in after-the-fact. The resources table can hold various key/value pairs such as usernames, names of services, etc with any additional information that may need to be attached to these values to be placed in the metadata table. Then any application data for a service can be stored in appdata with any additional information about this data stored under appmetadata. The tokens is to provide services access to the data they own while also offering a simple system to for services to share data amongst themselves without needing to duplicate entries.

Services

Regardless of the efforts that go into providing the ~center project with the architecture to decentralize or federate itself, the true value in the project is the services provided by the servers. Of which, there are two types of services that the server's users can anticipate: tilde services and ~center services.

Tilde Services

Due to the ~center project being a decentralized network of tilde servers at its core, the peer servers will need to provide services expected from a tilde server to become a full-fledged member of the TCN. While it would seem at first that these services would be automatically setup and standardized among the peers like the other components, this is actually not the case here. One of the unique joys of being the sysadmin for a tilde server is going through the rite of passage of choosing and setting up the services that will make up the tilde aspect of the server. So in this instance alone, it will be up to the sysadmin to fulfill this section on their own. However, for the sake of defining a standard, at the bare minimum the peer server should provide its users a shell account and access to an IRC server to communicate with the other users on the TCN to be considered an acceptable tilde server to become a peer server.

Tilde Chat

During the bootstrapping process of the ~center project, the IRC server currently used in the TCN is Tilde Chat since it is an existing decentralized IRC network for tilde servers (in particular, those who are members of the Tildeverse). However, this server can be changed later to a ~center operated decentralized IRC network if decided by the users through a vote.

Center Services

With one of the core philosophies of the ~center project being the development of HBOSPs by users of the TCN, one of the offerings that will bring value to the peer servers will be services created by its users. This can be anything and everything created by its users and will be able to utilize the ~center database if authorized by the sysadmin (primarily to prevent any malicious or ill-intentioned services from accessing the database). Once a service is accepted by a sysadmin, it will be shared to the other peer services so that users can access them from any peer server.

However, much like how sysadmins of peer servers can choose the kick / ban specific users from their server, they can also make the same decision for services if they find some issue to it running on their server. If this happens to take place, the dispatcher can also reject any transactions from the ~center database related to the rejected service by checking if the resource / resource_id of the transaction belongs to the service. However, realistically this will be done more efficiently by unsubscribing from the events related to the service in the dispatcher so they are never processed to begin with.

Proposals

While this specifies many factors of the ~center project, as time goes by there will likely be a desire to make amendments to the specification of the ~center project. For these circumstances, the Center Enhancement Proposals (CEP) system should be used to propose these changes. A CEP will be to the ~center project what PEPs are to Python, an RFC document isolated to Tilde Center. These documents will of course be federated across the TCN using the dispatcher once a draft is published and will be canonized once agreed upon by the leadership assigned to making these decisions by the currently used governance model.

Governance

While some projects do well with having a BDFL to handle all the decision-making, this does not scale well with a decentralized model. For this reason, the ~center project will need to establish a governance model to make decisions for itself moving forward as the project and TCN grows. The potential governance models can be proposed using the project's proposal system and the initial governance model chosen will be decided by a vote from the TCN's users using the Single Transferable Vote system of voting and can be changed later in the future using the same method. This voting system is used to ensure the system that makes the most users content is selected. The ability to change the governance model using the same method at any time is there to prevent the ~center project from being stuck with an undesired governance model and to permanently keep the power to change the governance model in the hands of the users. However, to prevent superfluous elections for new governance models from wasting the time of the community, this election can only be initiated by either having a petition signed by at least 25% of the TCN's user population (i.e. unique users), through a decision from whatever leadership is led by the current governance model to hold a new election, or if a majority (i.e. 50% or more) of the current governance model's leadership resign from their positions and are not replaced through the proper channels within 30 days (with the countdown beginning when the filled leadership positions reaches or goes below 50% and resets whenever the filled leadership positions returns above 50%).

During elections for new governance models for the ~center project, the voting day must be announced publicly in a manner that is easily accessible to the users in the TCN and be at least two weeks after the decision for a new election to be held is made. The candidates for new governance models must be submitted in the form of a CEP and must have been submitted before the day of the election. For an election to be valid, there must always be an option to vote for none of the proposed governance models and if the option for none of the governance models wins the election, another can be held in two weeks.

While the voting mechanism can change it must keep the user's vote anonymous, only allow unique users of the TCN to vote once, be stored using a blockchain or a tamper-less append-only log, and must be conducted over the dispatcher so that the results are publically available to all peer servers to verify the results.

In the event that the ~center project is holding an election for a new governance model, the first election held resulting in no governance model being passed, and over 30 days since the TCN decided to hold an election for a new governance model has passed the TCN will enter an emergency state. During an emergency state, any critical decisions that need to be decided can be made by either the ~center's current project leader (which will initially be the project's founder) but can be vetoed via a petition signed by 25% of unique TCN users within the next two weeks or the new leadership (whichever occurs first). Should the decision be vetoed via petition, the issue cannot be brought up again until the project is no longer in an emergency state.

Joining The Network

Given the unique privileges and influence that peer servers will have in the TCN, peer servers will need to invited into the network by at least one other sysadmin of an existing peer server. Until the first governance model is chosen, sysadmins can do so from their own judgment but this can be changed using a CEP. Should it be discovered that a sysadmin is intentionally adding malicious peer server and/or users to the TCN, their peer server will be ex-communicated from the TCN, they will be removed from any position of power they hold in the leadership, and their account will be banned from the TCN. Should any other sysadmin be found adding this sysadmin back into the TCN, their privileges in the TCN will be suspended until the current leadership decides how to address the situation.

To add a new peer server to the TCN, the sysadmin of the new server must have an existing account in the TCN and have an existing sysadmin agree to add the new server as one of their peers. To prepare the dispatcher of the existing peer server for the new server, the existing sysadmin must place a random SHA256 hash into a randomly named file in the $HOME directory of the new sysadmin's on the existing peer server. The new sysadmin will then encrypt the value stored in this file using their private GPG key followed by obtaining the UUID and public GPG key of the existing peer server from the sysadmin or public records (neither of these two values need to be hidden from the public). The sysadmin of the new server can then provide the dispatcher with this data to request the existing TCN to join the network.

Once a new server has been accepted by at least one peer server in the TCN it is officially a member of the ~center project, but it is recommended to have more than one peer server that it communicates with to strengthen its resilience to losing communications with the rest of the TCN.

Procedural Information

Security Considerations

The certificate authority used to sign the SSL certificate for the LDAP database, the signed certificate itself, credentials to the LDAP root user, credentials to the root user of the server, and the private GPG key should NEVER be made publicly available or be accessible by anyone aside from the sysadmin(s). Should any one of these be exposed in any way, they should be changed as soon as possible to retain the integrity of the server's and/or network's security.

Configuration Considerations

Outside of the configurations already mentioned prior in this document, there are no other required configurations to consider for the Tilde Center project.

For members of the tildeverse (aside from tilde.center and its peers) no configuration is needed to meet this RFC's request.