feat: relay several servers, registered through the control API

ENGINE.md

@@ -16,7 +16,7 @@ Our ET server sits at home, reachable from the Internet only through the front s
 
 ## The fix
 
-`etrelay` runs on the front and owns the public UDP port 27960. It does two things on **the same socket**:
+`etrelay` runs on the front and owns the public UDP ports, one per ET server (27960 for the first). For each server it does two things on **the same socket**:
 
 - **Proxy**: forwards every client (players, server browsers, masters) to the ET server, one upstream socket per client address. Packets a client has no business sending (`rcon`...) are dropped on the way.
 - **Heartbeat**: sends the heartbeats itself, so the master challenges the **front** address. That challenge is just another client packet: it goes through the proxy, the ET server answers it, the answer leaves from the front address, and the master lists the front.
@@ -29,7 +29,7 @@ Masters are independent: a server is only listed on the masters it heartbeats, a
 
 ![After: everything goes through the front](diagrams/architecture.png)
 
-## Registration sequence
+## Master registration sequence
 
 ![Master registration (every 5 min)](diagrams/registration.png)
 
@@ -55,16 +55,42 @@ The last two lines are a server tracker that got the address from the master.
 
 The ET server sees every client as `<front-wg>:<port>`: one source IP (the front's tunnel address), one port per session, the port of the session's upstream socket. Its own IP bans can't tell players apart: bans are etrelay's (below). A player coming back after the session expired gets a new port.
 
+## Several servers: registering
+
+The relay holds several ET servers. The ET protocol has no host name to tell them apart: each one gets its own public port (`--ports`), with its own socket, sessions, heartbeats and bans. Masters list servers by address and port, so several on the front's address are fine.
+
+A server is relayed once its admin tool registered it, with `PUT /server` on the control API (`--control <addr>`, on the front's tunnel address, unreachable from the internet):
+
+![Registering a server](diagrams/pairing.png)
+
+- **The relay's token** only lets a tool register: it keeps strangers of the tunnel off the public ports.
+- **The key** is picked by the tool, and is what every later call carries: it names the server, so a tool only reaches its own. etrelay keeps its SHA-256 (`servers.json`), never the key.
+- **The proof**: before taking a key, etrelay checks the tool runs the ET server it registers. The tool makes the server show the key's fingerprint (the first 128 bits of its SHA-256) in the serverinfo cvar `etrelay_proof`, etrelay reads it there with a `getstatus`. Only who holds that server's console can do it, wherever the request comes from (a NAT between the two changes nothing), and the fingerprint, public for a moment, doesn't give the key away. An empty cvar is left out of serverinfo: the tool sets it to `""` afterwards (`unset` isn't in 2.60b).
+
+What can't happen:
+
+- **two servers on one port**: the second one gets a 409, the first one is untouched;
+- **another tool on a registered server**: it has neither the key, nor the server's console to prove another one.
+
+One limit: a server that is gone for good without `DELETE /server` keeps its port (the masters drop it, nothing answers there), until its entry is removed from `servers.json` and etrelay restarted.
+
+An ET server registered again is taken over by who proves to run it (a tool reinstalled, with a new key): the former key is refused from then on. On the same port the relay goes on (sessions, bans); on another port the former one is freed. A relay that lost its state answers 401 to every key: tools register again by themselves.
+
+The registered servers are relayed again after a restart. `DELETE /server` leaves the relay: the masters get a flatline, the port is freed, the bans dropped.
+
+`etrelay <UPSTREAM>` still relays one server without any of this, on `--listen`: it has no key, so no control API over it.
+
 ## Control API and bans
 
-With `--control <addr>`, an HTTP API listens on the front's tunnel address (unreachable from the internet), bearer token on top. It serves an admin tool on the tunnel side:
+Once registered, the admin tool asks about its server, with its key:
 
+- `GET /server`: the registration, and whether the ET server answered the last probe.
 - `GET /sessions`: the open sessions, real client address and port the ET server sees, so the tool can put a real address on each player.
 - `GET /bans`, `PUT /bans/{ip}`, `DELETE /bans/{ip}`: bans by source IP, with a reason and an optional duration.
 
 ![Control API: real address and ban of a player](diagrams/ban.png)
 
-A ban cuts the address's sessions at once, and the relay loop drops its packets before any session lookup: the ET server gets nothing more from the player and times them out, and they can't come back. Bans are saved to `<state-dir>/bans.json` on each change (written next to it, then renamed), expired ones forgotten.
+A ban cuts the address's sessions at once, and the relay loop drops its packets before any session lookup: the ET server gets nothing more from the player and times them out, and they can't come back. Each server has its own bans, saved to `<state-dir>/bans-<port>.json` on each change (written next to it, then renamed), expired ones forgotten.
 
 ## Filter
 
@@ -78,14 +104,16 @@ Only what clients and masters send a server gets through, checked on the first w
 
 ## Internals
 
-Single-threaded tokio runtime, four kinds of tasks. The sessions and the bans are shared by the relay loop and the control API behind a mutex, never held across an `await`:
+Single-threaded tokio runtime, four kinds of tasks. Each server has its relay loop, its heartbeat task and its state (sessions, bans), shared with the control API behind a mutex. The registry (the servers by public port, the hash of their key) is behind another one, which the packets never take. Neither is held across an `await`:
 
 ![Tasks and data flow](diagrams/tasks.png)
 
-- **relay loop**: reads the public socket, runs the filter and the ban check, finds the client's session and pushes the packet into its queue. No session, or an expired one: opens a new one (and purges expired entries), unless `MAX_SESSIONS` are open. A full queue drops the packet, like any congested UDP hop.
+- **relay loop** (one per server): reads the server's public socket, runs the filter and the ban check, finds the client's session and pushes the packet into its queue. No session, or an expired one: opens a new one (and purges expired entries), unless `MAX_SESSIONS` are open, all servers together. A full queue drops the packet, like any congested UDP hop.
 - **session task**: owns one ephemeral socket connected to the ET server. Forwards the queue to the ET server and the ET server answers back to the client **through the public socket** (so they come from `:27960`). Ends after `SESSION_TIMEOUT` without a packet either way, or at once when a ban drops its queue's sender.
-- **heartbeat task**: probes the ET server with `getinfo` on its own socket (up to 3 attempts, 3 s each), then heartbeats the masters through the public socket. Masters are resolved on every heartbeat (DNS may change).
-- **control API task** (with `--control`): axum on the tunnel address. Binding is retried every 10 s: the tunnel may come up after etrelay.
+- **heartbeat task** (one per server): probes the ET server with `getinfo` on its own socket (up to 3 attempts, 3 s each), then heartbeats the masters through the public socket. Masters are resolved on every heartbeat (DNS may change).
+- **control API task** (with `--control`): axum on the tunnel address. Binding is retried every 10 s: the tunnel may come up after etrelay. Registering starts a server's two tasks, leaving aborts them and drops its sessions.
+
+The code follows that split: `server` (one relayed server), `registry` (which servers, `servers.json`), `proof` (keys and their proof), `control` (the API), `et` (the probes: `getinfo`, `getstatus`), `filter`, `bans`.
 
 ### Heartbeat states
 
@@ -102,10 +130,11 @@ The master drops a server that stops heartbeating after a while anyway; the flat
 | `SESSION_TIMEOUT` | 2 min | Idle clients (server browsers) freed quickly; players send packets constantly |
 | `SESSION_QUEUE` | 64 packets | Burst absorption per client before dropping |
 | `MAX_PACKET` | 65 535 B | Any UDP datagram: `statusResponse` easily exceeds nginx's old 512 B buffer |
-| `MAX_SESSIONS` | 2048 | One socket each: spoofed sources must not use up the file descriptors (`LimitNOFILE=4096` in the unit) |
+| `MAX_SESSIONS` | 2048 | One socket each, all servers together: spoofed sources must not use up the file descriptors (`LimitNOFILE=4096` in the unit) |
 | `QUERY_BURST`, `QUERY_RATE` | 5, then 1/s | `getstatus`/`getinfo` per source IP: a browser refresh or a master's check passes, a reflection attack doesn't |
 | `MAX_QUERY_SOURCES`, `QUERY_FORGET` | 65 536 IPs, 60 s | The limiter's memory stays bounded under a flood of spoofed sources |
-| `MIN_TOKEN` | 32 characters | Shortest control API token accepted |
+| `MIN_TOKEN`, `MIN_KEY` | 32 characters | Shortest relay token and server key accepted |
+| `PROOF_BYTES` | 16 | Bytes of the key's SHA-256 shown as its fingerprint: serverinfo holds 1024 characters for every cvar |
 
 ### Protocol cheat sheet