etrelay/ENGINE.md

How etrelay works

Diagrams: PlantUML sources in diagrams/, PNGs regenerated with make diagrams.

The problem

A server shows up in the in-game browser only if a master server lists it. Registration is a 3-step handshake over UDP:

1. The game server sends heartbeat EnemyTerritory-1 to the master.
2. The master does not trust it: it queries the source IP:port of the heartbeat. etmaster.net sends getChallenge, then getInfo (classic Quake 3 masters send getinfo <challenge> directly).
3. The game server answers (challengeResponse, infoResponse). The master lists the source IP:port of those answers.

Our ET server sits at home, reachable from the Internet only through the front server and the WireGuard tunnel. Players went in through the front (DNAT, then nginx stream), but the ET server's own heartbeats went out through the home box: the master challenged the home IP, where nothing listens for it. Result: server reachable by IP, invisible in the browser.

The fix

etrelay runs on the front and owns the public UDP port 27960. It does two things on the same socket:

• Proxy: forwards every client (players, server browsers, masters) to the ET server, one upstream socket per client address. Packets a client has no business sending (rcon...) are dropped on the way.
• Heartbeat: sends the heartbeats itself, so the master challenges the front address. That challenge is just another client packet: it goes through the proxy, the ET server answers it, the answer leaves from the front address, and the master lists the front.

The ET server's own heartbeats are disabled (sv_master1..5 "").

The ET server is its own WireGuard peer of the front, with no proxy in between (the DNAT rule and the home nginx stream are gone). Each client reaches it from its own port of the front's tunnel address: the control API maps that port back to the real address, and bans addresses (see Control API and bans).

Masters are independent: a server is only listed on the masters it heartbeats, and each client browses its own master (ET 2.60b: etmaster.idsoftware.com, ET: Legacy: etmaster.net). The relay heartbeats both by default; diagrams show etmaster.net, the id Software master works the same way.

Registration sequence

What it looks like in production (--debug):

heartbeat EnemyTerritory-1 -> etmaster.net:27950 (198.51.100.10:27950)
session + 198.51.100.10:27950 (2 active)
198.51.100.10:27950 -> ET  getChallenge (27 B)
198.51.100.10:27950 <- ET  challengeResponse (32 B)
198.51.100.10:27950 -> ET  getInfo (22 B)
198.51.100.10:27950 <- ET  infoResponse (303 B)
session + 203.0.113.7:45488 (3 active)
203.0.113.7:45488 -> ET  getstatus (14 B)
203.0.113.7:45488 <- ET  statusResponse (993 B)

The last two lines are a server tracker that got the address from the master.

Player sequence

The ET server sees every client as <front-wg>:<port>: one source IP (the front's tunnel address), one port per session, the port of the session's upstream socket. Its own IP bans can't tell players apart: bans are etrelay's (below). A player coming back after the session expired gets a new port.

Control API and bans

With --control <addr>, an HTTP API listens on the front's tunnel address (unreachable from the internet), bearer token on top. It serves an admin tool on the tunnel side:

• GET /sessions: the open sessions, real client address and port the ET server sees, so the tool can put a real address on each player.
• GET /bans, PUT /bans/{ip}, DELETE /bans/{ip}: bans by source IP, with a reason and an optional duration.

A ban cuts the address's sessions at once, and the relay loop drops its packets before any session lookup: the ET server gets nothing more from the player and times them out, and they can't come back. Bans are saved to <state-dir>/bans.json on each change (written next to it, then renamed), expired ones forgotten.

Filter

Only what clients and masters send a server gets through, checked on the first word as the ET server tokenizes it:

• in-game packets (no \xff\xff\xff\xff prefix): always;
• getstatus, getinfo, getchallenge, connect, PunkBuster's PB_...: allowed;
• anything else, rcon above all, dropped: it travels in clear, and a stream of it blocks the local admin's own rcon (2.60b runs one rcon per 500 ms, whoever sends it).

getstatus and getinfo are limited per source IP (token bucket: a burst of 5, then 1/s): their answers are much bigger than the query, and from a spoofed source they would make the server a DDoS reflector. The limiter remembers at most 65 536 IPs, idle ones forgotten after 60 s.

Internals

Single-threaded tokio runtime, four kinds of tasks. The sessions and the bans are shared by the relay loop and the control API behind a mutex, never held across an await:

• relay loop: reads the public socket, runs the filter and the ban check, finds the client's session and pushes the packet into its queue. No session, or an expired one: opens a new one (and purges expired entries), unless MAX_SESSIONS are open. A full queue drops the packet, like any congested UDP hop.
• session task: owns one ephemeral socket connected to the ET server. Forwards the queue to the ET server and the ET server answers back to the client through the public socket (so they come from :27960). Ends after SESSION_TIMEOUT without a packet either way, or at once when a ban drops its queue's sender.
• heartbeat task: probes the ET server with getinfo on its own socket (up to 3 attempts, 3 s each), then heartbeats the masters through the public socket. Masters are resolved on every heartbeat (DNS may change).
• control API task (with --control): axum on the tunnel address. Binding is retried every 10 s: the tunnel may come up after etrelay.

Heartbeat states

The master drops a server that stops heartbeating after a while anyway; the flatline only makes it faster.

Constants

Constant	Value	Why
HEARTBEAT_INTERVAL	5 min	Same period as the ET server itself
PROBE_ATTEMPTS × PROBE_TIMEOUT	3 × 3 s	One lost UDP packet must not flatline the server
SESSION_TIMEOUT	2 min	Idle clients (server browsers) freed quickly; players send packets constantly
SESSION_QUEUE	64 packets	Burst absorption per client before dropping
MAX_PACKET	65 535 B	Any UDP datagram: statusResponse easily exceeds nginx's old 512 B buffer
MAX_SESSIONS	2048	One socket each: spoofed sources must not use up the file descriptors (LimitNOFILE=4096 in the unit)
QUERY_BURST, QUERY_RATE	5, then 1/s	getstatus/getinfo per source IP: a browser refresh or a master's check passes, a reflection attack doesn't
MAX_QUERY_SOURCES, QUERY_FORGET	65 536 IPs, 60 s	The limiter's memory stays bounded under a flood of spoofed sources
MIN_TOKEN	32 characters	Shortest control API token accepted

Protocol cheat sheet

Connectionless packets start with \xff\xff\xff\xff, then a command:

Command	From → to	Meaning
heartbeat EnemyTerritory-1	server → master	"I'm alive, come check"
heartbeat ETFlatline-1	server → master	"I'm going down"
getChallenge / challengeResponse	master ↔ server	Master check before getInfo (etmaster.net)
getinfo / getInfo	master/browser → server	Short info request (optional challenge)
infoResponse	server → master/browser	Hostname, map, player count, challenge
getstatus / statusResponse	browser ↔ server	Full cvars + player list
getchallenge / challengeResponse / connect	player ↔ server	Connection handshake
PB_...	PunkBuster ↔ server	PunkBuster's own packets
rcon <password> <command>	admin → server	Remote console: dropped by the relay

Packets with no prefix are in-game traffic: forwarded as-is, only counted in --debug logs.