snipe-worker

Volunteer worker for snipe, the osu! Switzerland #1 tracker. snipe writes lists of osu! API GETs (jobs); this program runs them with your own osu! OAuth application and your own IP, at most 60 requests per minute, and sends the raw bodies back. It decides nothing about the data: which maps, which players and what to do with the answers is the server's business.

Full tutorial (Windows installer, Linux + systemd, Docker, supporter login, troubleshooting): https://snipe.sulej.net/support

Country boards from Docker

The redirect after the osu! login cannot reach a listener inside the container, so paste it instead (-it gives the command a terminal; the env of the running container already holds the client id and secret):

docker exec -it snipe-worker snipe-worker login --paste --data-dir /data

Open the printed URL, authorize, and paste the URL your browser lands on (it starts with your redirect URI and shows an error page, that is expected). Then docker restart snipe-worker.

Run it

1. Create an OAuth application at https://osu.ppy.sh/home/account/edit#oauth (any name; the callback URL is only used if you log in for country boards, http://localhost:8091/callback works).
2. Register a worker on https://snipe.sulej.net/account and copy the token (shown once).
3. Run it, one of four ways:

Windows installer (snipe-worker-setup.exe from the releases): per-user install, Start menu entry, optional start at login, a tray icon, and the status page opens on first start so you can paste the token there.

Binary (Linux x64 static / Windows x64) from the same releases page:

./snipe-worker-linux-x64 --token <worker token> --client-id <id> --client-secret <secret>

Docker:

docker run -d --restart unless-stopped --name snipe-worker -v snipe-worker:/data \
  -e SNIPE_WORKER_TOKEN=<worker token> \
  -e OSU_CLIENT_ID=<id> -e OSU_CLIENT_SECRET=<secret> \
  git.sulej.net/skinhub/snipe-worker:beta

From source: cargo install --git https://git.sulej.net/skinhub/snipe-worker.

The binary keeps itself current from the releases page (stable builds follow stable, beta builds follow beta); the image does not, pull a new tag instead.

Status page

The worker serves a small page on http://127.0.0.1:27300/ (flag --ui-port, env SNIPE_WORKER_UI_PORT, 0 disables): version and registered name, what it is doing, jobs and requests this run, the last 100 log lines, a form for the token, osu! application, rate and job kinds (saved to config.json, applied without a restart), a "log in with osu!" button for country boards, an update button and quit. It is bound to the loopback interface and refuses any request whose Host/Origin is not exactly that address. When no token is configured yet, the page is opened in your browser at start. It is off with --once and in Docker (set SNIPE_WORKER_UI_BIND=0.0.0.0 and publish port 27300 to turn it on there; nothing on it is authenticated, so only do that on a private network).

Configuration

CLI beats env beats <data-dir>/config.json (same keys, e.g. {"token": "...", "client_id": "..."}). The token is re-read from the file on every claim, so pasting it takes effect without a restart.

Flag	Env	Default
--token	SNIPE_WORKER_TOKEN	required
--client-id	OSU_CLIENT_ID	required
--client-secret	OSU_CLIENT_SECRET	required
--api	SNIPE_API	https://snipe.sulej.net
--rpm	WORKER_RPM	60 (the osu! policy ceiling; higher is clamped)
--caps	WORKER_CAPS	everything you can run: board,pair,users,profile,user_pages,user_maps,replay
--no-replays	WORKER_NO_REPLAYS	off: a logged-in worker downloads replays (config.json "replays": false or the status page checkbox turns it off)
--data-dir	SNIPE_WORKER_DATA	snipe-worker-data next to the binary, /data in Docker
--once		run one claim batch, then exit
--no-update		never self-update
--ui-port	SNIPE_WORKER_UI_PORT	27300 (status page; 0 = off)

Without a token or application it idles with one log line every ten minutes instead of crash-looping, so a container can be started before it is configured.

Country boards (Swiss supporters)

osu!'s country leaderboards are only readable with a logged-in osu!supporter account of that country. If you are one:

snipe-worker login --client-id <id> --client-secret <secret> [--redirect-uri http://localhost:8091/callback]

(or the "Log in with osu!" button on the status page) opens the osu! authorization page (scope public identify), catches the redirect on a one-shot local listener (or lets you paste the URL), checks with /me that the account is Swiss and a supporter, and stores the token pair in <data-dir>/user-token.json (mode 0600). The pair is refreshed before it expires and the new one is written atomically before use. With it the worker also advertises board and snipe hands it country-board jobs; without it, it never does. The login also downloads replays (replay jobs, the lowest score of every map first, so the archive keeps them after osu! stops serving them): at most 10 a minute, osu!'s limit per login, on a lane of their own, unless replays are turned off.

What it does, exactly

• Asks snipe for its config (GET /api/worker/config: name, poll interval, batch size, rpm, allowed kinds, path allowlist) and for jobs (POST /api/worker/claim).
• For every job: runs the listed GETs against https://osu.ppy.sh/api/v2 in order, only paths under the server's allowlist (/beatmaps, /users, /rankings, /beatmapsets, /teams, /scores), each charged its cost against one 60/min limiter, honouring osu!'s Retry-After and X-RateLimit-Remaining, skipping a request when the server said so (skip_if: an empty board makes the stable board pointless, a short page ends a list).
• Heartbeats every 15 s while a job runs; a 409 means the job was handed to someone else and it is dropped. Posts the responses as soon as the job is finished. On SIGTERM/SIGINT it releases the job it holds and exits.
• Ships its own log lines to snipe (visible on your account page), batched, best effort, with your home directory scrubbed.

Trust model: the server writes the requests and validates every body it gets back (a score must belong to the pair, a board to the map and to Swiss players, a profile to the player, a replay file to the score it was asked for); a worker can at worst withhold data. The worker in turn only ever performs GETs under the allowlist with credentials that stay on your machine.

Development

cargo test (no network), cargo clippy --all-targets -- -D warnings, cargo fmt. Server side: snipe-api src/jobs.rs and src/routes/worker_plane.rs.