Running a cluster
Every node serves the whole UI and the whole API. There is no admin node and no node you have to be on to do a particular thing.
Status: shipped and running. This page is an outline of behaviour that is already stable.
How a request is served
- Reads come from the local replica, so they do not cross the network.
- Writes go through the Raft leader, wherever it happens to be.
- Node-local operations, anything that has to touch a specific host's Docker or ZFS, are proxied exactly one hop over mTLS, with the original user asserted so the audit trail names a person rather than a machine.
Joining a node
A node joins with a token generated on an existing node. The token is short-lived and single-use, and issuing it also issues the joining node's cluster-internal mTLS certificate. There is no shared secret to distribute and nothing to copy between hosts by hand.
Quorum
Losing a majority makes the cluster read-only, and it says so in a banner rather than failing one write at a time with an unhelpful error. Reads keep serving from the local replica throughout. Applications on a node that is merely unreachable keep running if the host itself is alive; they just cannot be managed until it comes back.
Migrating an application
Migration is the same code path as backup and restore, which means the restore path is exercised every time an application moves rather than on the day you need it.
- Preflight. Target pool, zone presence and free space are checked first.
- Pre-seed. The bulk of the data is sent while the application is still running.
- Cutover. Stop, incremental send, start.
The address moves with the application, and because the MAC is derived from the address, the rest of the network does not need to learn anything new. The previous dataset is retained for rollback.
Upgrading
Schema migrations are append-only and ship inside the binary, so upgrade every node in a cluster together when a release adds one. The changelog says which releases do.