Version 5 · Updated 3 November 2026

Running branchscope

Services, configuration, data, security notes and troubleshooting.

Branchscope runs as a set of services; deploy/compose runs them all with Docker Compose for development and evaluation.

Services#

Service	Port (local)	Role
web	3300	Next.js web app, with the docs CMS (Payload) at `/cms`
control-plane	3001	NestJS API; Postgres for state; Keycloak for sign-in
git	8080 (HTTP), 2222 (SSH)	Go git service; its internal API (8081) stays inside the network
push-guard	(a Unix socket)	Rust; decides every push
orchestrator	9443	Go; where `bscope-runner` connects (gRPC; TLS outside development) for workflow jobs
indexer	(none)	builds graphs and reviews; each graph build is a hosted job (`JOB_RUNNER=hosted`)
sandbox-agent	(none)	a hosted runner for development: graph builds and `runs-on: hosted` jobs, each in a container of its own, through the Docker socket; `scripts/dev-setup` registers it
postgres	5432 (`POSTGRES_PORT`)	state; the CMS has its own `payload` database and role, created by `postgres-init` on every start
nats	(none)	push events and jobs (JetStream)
garage	(none)	S3-compatible storage for graphs, review documents, job logs, and runs' caches and artifacts
keycloak	8180	sign-in (OIDC)

Configuration#

deploy/compose/.env (from .env.example) holds:

the secrets: database password, Keycloak client secret, one token per service (GIT_SERVICE_TOKEN, INDEXER_TOKEN, CONTROL_PLANE_TOKEN, RUNNER_SERVICE_TOKEN);
the dev users' passwords;
the CMS's database password and signing secret (PAYLOAD_DB_PASSWORD, PAYLOAD_SECRET).
the encryption keys for external tokens and secrets (EXTERNAL_TOKEN_KEY, SECRETS_KEY; below), and the key that signs runs' identity tokens (OIDC_SIGNING_KEY).

Never commit it.

Public URLs, set where the services need them:

WEB_PUBLIC_URL;
API_PUBLIC_URL (what the CLI is told to call);
GIT_PUBLIC_URL;
GIT_SSH_PUBLIC_URL. Leave it empty to offer no SSH;
RUNNERS_URL (the web app): where runners connect, as the registration command shows it. RUNNERS_INSECURE=true adds --insecure, for a local orchestrator without TLS.

The orchestrator serves TLS with -tls-cert and -tls-key; the compose stack runs it with -insecure, on localhost only. Its -git-url is where runners clone from: the public git URL.

The API's limits (control plane):

RATE_LIMIT_PER_HOUR and RATE_LIMIT_PER_MINUTE: each token's budget (1,000 and 60);
RATE_LIMIT_ANONYMOUS_PER_HOUR: anonymous callers, per IP (60);
RATE_LIMIT_WEB_PER_HOUR and RATE_LIMIT_WEB_PER_MINUTE: the web app's sessions and visitors (10,000 and 600);
TRUSTED_PROXIES: whose X-Forwarded-For to believe, so visitors are counted one by one: the web app's and any load balancer's addresses, as Express's trust proxy list (loopback, uniquelocal, addresses, CIDR ranges). Nothing is trusted unless it's set.

The counters are kept in memory, in the one control plane. API_CHECK_RESPONSES=1 checks every response against the API's schemas; it's on in the compose stack, for development.

Encryption keys (control plane), each 32 bytes, base64. Make one with openssl rand -base64 32:

EXTERNAL_TOKEN_KEY: encrypts people's saved external tokens, which HTTPS mirrors sign in with. Without it, none can be saved; people can still paste a token when importing.
SECRETS_KEY: encrypts secrets for runs and mirrors' deploy keys. Use a key of its own, not EXTERNAL_TOKEN_KEY. Without it, no secrets can be set, runs get none, and mirrors can't use SSH.
Rotating: put the old key in EXTERNAL_TOKEN_PREVIOUS_KEYS or SECRETS_PREVIOUS_KEYS (comma-separated, for several) and the new one in its place. A value saved under an old key moves to the new one the next time it's used; one whose key is gone can't be read: save the token or secret again, or add the mirror again.

Without a key the feature is off; the rest of Branchscope works. A key that isn't 32 bytes is logged at start, and that feature is off too. Keep the keys out of the repository and back them up with the database: without them, what they encrypted is lost.

Identity tokens for runs (OIDC, control plane):

OIDC_SIGNING_KEY: an RSA private key of 2048 bits or more, in PEM or its base64, that signs runs' identity tokens (RS256). Make one with openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 | base64 | tr -d '\n'. Without it, identity tokens are off; a key that doesn't parse stops the control plane at start. Back it up with the other keys.
OIDC_PREVIOUS_KEYS: while rotating, the previous keys (comma-separated; private or public, PEM or base64), still published with the current one so the tokens they signed verify until they expire (10 minutes). Remove them after that.
OIDC_ISSUER: the issuer, WEB_PUBLIC_URL by default (https://<your domain>). The web app serves /.well-known/openid-configuration and /.well-known/jwks from the control plane; AWS, Google Cloud and Azure must reach them over HTTPS. Changing the issuer breaks every trust set up with the old one.

Signing what Branchscope commits (git service):

COMMIT_SIGNING_KEY: an unencrypted OpenSSH private key (ed25519 recommended), as its text or its base64, that signs the commits Branchscope makes: merge commits, squashes and rebased commits. Make one with ssh-keygen -t ed25519 -N "" -C branchscope -f signing && base64 < signing | tr -d '\n'. Without it, those commits aren't signed, and merging a change into a branch whose ruleset requires signed commits is refused, with why; a key that doesn't parse stops the git service at start. Its public key is logged at start; it verifies as Branchscope's own. Back it up with the other keys. Rotating: set the new key; commits signed with the old one keep their signatures, but no longer verify.

Private addresses, each for development and tests only. Never set them in production.

WEBHOOK_ALLOW_PRIVATE=true (control plane) lets webhooks, integrations and calls to Linear reach private addresses and plain http://, so the compose stack's tests can receive them.
MIRROR_ALLOW_PRIVATE=true, on the control plane and the git service, lets mirrors go to private addresses and plain http://, where the compose stack mirrors to its own git service.
SSO_ALLOW_PRIVATE=true (control plane) lets SSO providers be on plain http:// and private addresses (below).
IMPORT_ALLOW_PRIVATE=true (git service) lets imports come from private addresses, so tests can import from the compose stack's own git service.

Push mirrors:

MIRROR_POLL_MS (control plane): how often the mirror worker looks for syncs to do, 2000 by default; 0 turns it off. MIRROR_SETTLE_MS: how long it waits after a push before syncing, 3000 by default.

Single sign-on (SSO):

Keycloak's branchscope-web client needs its service account to hold realm-management's view-users, manage-users and manage-identity-providers, and a mapper that puts the identity_provider user session note into its tokens as identity_provider. keycloak/branchscope-realm.json has both; scripts/dev-setup adds them to a realm imported before, and in production deploy/production/keycloak-realm.sh does (below).
SECRETS_KEY (above) encrypts each organization's client secret. Without it, SSO can't be set up.
Each provider is a Keycloak identity provider, org-<organization>, hidden from Keycloak's sign-in page. Keycloak keeps its own copy of the client secret, in its database; point it at a Keycloak vault if that must be encrypted too.
SSO_ALLOW_PRIVATE=true (above) is for development, where another realm of the compose stack's Keycloak acts as a company's provider.
Domain verification asks the control plane's DNS resolver for TXT records.

Passkeys (Keycloak):

Keycloak makes and checks them, in the realm's WebAuthn Passwordless policy with passkeys on (webAuthnPolicyPasswordlessPasskeysEnabled). Its sign-in page then offers them beside the password, with no change to the sign-in flow.
The relying party ID (webAuthnPolicyPasswordlessRpId) must be the sign-in page's host name or a domain it's under, never an IP address. Production's realm uses BRANCHSCOPE_DOMAIN (Keycloak is at auth.<domain>, under it). Choose it once: changing it later makes every passkey stop working. The compose stack's Keycloak is on http://localhost:8180 for this reason, with the relying party localhost.
HTTPS: browsers only allow passkeys on https:// pages, or on localhost.
The service account also needs manage-users, to rename and remove passkeys, and view-events, to show when each was last used: the realm keeps LOGIN events for 30 days (eventsExpiration). Events hold the address each sign-in came from.
An existing realm doesn't take changes to its file: run deploy/production/keycloak-realm.sh on the server by hand, from /srv/branchscope/src, once after deploying the release that brings passkeys, and again after a Keycloak upgrade. scripts/deploy doesn't run it. It sets the passkey policy, the registration action (webauthn-register-passwordless), the session lifetimes (30 minutes idle, 10 hours at most, Keycloak's defaults written down), the events, the service account's roles and the identity_provider mapper, and asks site administrators without a passkey to add one at their next sign-in. Running it again changes nothing.

Linear (control plane):

SECRETS_KEY (above) encrypts each organization's Linear token or API key. Without it, Linear can't be connected.
LINEAR_API_URL: Linear's GraphQL API, https://api.linear.app/graphql unless set; its OAuth token and revocation endpoints are on the same host (/oauth/token, /oauth/revoke). Calls to it get webhooks' checks: public addresses only (unless WEBHOOK_ALLOW_PRIVATE=true), 10 seconds, no redirects. The compose stack points it at the end-to-end tests' fake Linear (http://host.docker.internal:4790/graphql); set it to Linear's own to try the real one. Leave it unset in production.
LINEAR_CLIENT_ID, LINEAR_CLIENT_SECRET and LINEAR_WEBHOOK_SECRET: the server's own Linear app, for Connect with Linear. Without the first two, organizations connect with API keys only. They're read from the environment only, never stored in the database. LINEAR_WEBHOOK_SECRET can hold two secrets, comma-separated, while you rotate it; without it, connections made with the app get no events back and read issues from Linear when a change is shown. The compose stack sets a dummy app that the end-to-end tests' fake Linear knows, with LINEAR_AUTHORIZE_URL pointing at the fake's consent page; leave LINEAR_AUTHORIZE_URL unset in production (https://linear.app/oauth/authorize).
RATE_LIMIT_LINEAR_PER_MINUTE: events Linear may send per minute, per workspace and per address, 600 by default. They don't count against the anonymous limit.
The webhook routes, /linear/webhook and /linear/webhook/{connection}, must be reachable from the internet without a session, with bodies up to 1 MB.

Registering the Linear app, once per server, by someone with a Linear account (in Linear, Settings → API → OAuth applications → New):

Name it Branchscope, with Branchscope's icon, so what it does in Linear shows under that name.
Callback URL: {WEB_PUBLIC_URL}/integrations/linear/callback, such as https://branchscope.example.com/integrations/linear/callback.
Turn on its webhook, with the URL {API_PUBLIC_URL}/linear/webhook, and the resource types Issue and Attachment. Its signing secret is LINEAR_WEBHOOK_SECRET.
Make it public, so other workspaces than yours can install it.
Put its client ID, client secret and webhook signing secret into the control plane's environment (deploy/compose/.env.example lists them, with dummy values), and restart it.

The audit log (control plane):

AUDIT_RETENTION_DAYS: how many days audit log entries are kept, for every repository, organization and person alike. Unset or 0 keeps them for good. Otherwise the control plane deletes older entries at start and once a day after that. A value that isn't a whole number of days stops the control plane at start.

Other settings#

Each has a default that suits most servers. Where it says so, 0 turns the timer off: the tests do, to run it by hand.

Variable	Service	Default	What it does
`DATABASE_URL`	control plane	(required)	its Postgres, as `postgres://user:password@host:5432/branchscope`
`NATS_URL`	control plane, indexer	(none)	NATS with JetStream (`nats://nats:4222`). Without it the control plane publishes and consumes nothing (no push events, no reviews) and the indexer takes no jobs. The git service takes `-nats-url` instead
`GRAPH_RETENTION_DAYS`	control plane	30	graphs of commits that haven't been a branch's tip for this many days are deleted, unless an open change's review or a current deployment uses them; they can be built again. `GRAPH_GC_MINUTES` (60) is how often it looks
`IMPORT_MAX_BYTES`	control plane	2147483648 (2 GB)	a source larger than this once fetched isn't imported; `0` for no limit
`IMPORT_CONCURRENCY`	control plane	2	imports running at once, at most one per person; the rest wait
`MCP_IDLE_SECONDS`	control plane	1800	an MCP session ends after this long without a request
`MCP_SESSIONS_PER_TOKEN`	control plane	10	open MCP sessions per authorization or installation; one more is refused with `429`
`USER_SYNC_SECONDS`	control plane	120	how often people disabled or deleted in Keycloak lose their sessions, tokens and SSH keys, and SSO links removed there are removed here; `0` turns it off
`WEBHOOK_POLL_MS`	control plane	1000	how often webhook and integration deliveries are sent and retried; `0` turns it off
`ACTIONS_TICK_SECONDS`	control plane	30	how often scheduled workflows start, and lost or timed-out jobs are found; `0` turns it off, and the daily cleanup of caches and artifacts with it
`POLICY_SYNC_SECONDS`	control plane	15	how often the push guard is given its policy again (it also is at start and after every change); `0` turns this and the probe off (the policy is still given at start), so leave it on
`POLICY_PROBE_SECONDS`	control plane	2	how often the control plane asks whether the guard has a policy, so a restarted guard gets one within seconds
`GRAPH_JOB_IMAGE`	control plane	`branchscope-indexer:dev`	the image graph builds run in as hosted jobs (the indexer's); `GRAPH_OUTPUT_URL` (`API_PUBLIC_URL` by default) is where they upload the graph
`JOB_IMAGE`, `JOBS_VOLUME`	indexer	(required with `JOB_RUNNER=docker`)	the image each graph build's container runs, and the volume it shares with the indexer
`JOB_NETWORK`	indexer	`branchscope-jobs`	the Docker network those containers join
`PUSH_GUARD_REQUIRE_POLICY`	push guard	off	`true` refuses every push until the control plane has given the guard its policy. Both stacks set it: keep it
`PUSH_GUARD_DENY_ALL`	push guard	off	`true` starts the guard with deny-all on, for tests; a site admin turns it off
`SHOWCASE_CHANGE`	web, control plane	(none)	a change in a public repository, as `owner/name#number`, whose review the product pages show. After a toolchain change its review runs again first, then those of open changes (200 at most). Unset, the pages leave it out

Caches and artifacts#

Runs' caches and artifacts are kept in the same S3 bucket as graphs (S3_BUCKET), under actions/. Runners never reach it: they send and fetch them through the control plane, with the job's token, at ACTIONS_API_URL (API_PUBLIC_URL by default) plus /internal/jobs/…. So runners must reach that address, as they reach the git service to clone. The compose stack's development hosted runner reaches the control plane inside the network instead (SANDBOX_AGENT_API_URL=http://control-plane:3001).

An upload goes to a temporary file on the control plane (TMPDIR) before it's stored, so leave room there for the largest one. The limits (control plane):

ACTIONS_CACHE_ENTRY_MAX_MB: one cache entry, 2048 by default;
ACTIONS_CACHE_REPO_MAX_MB: a repository's caches together, 10240 by default; past it, the entries used least recently are deleted;
ACTIONS_CACHE_DAYS: an entry nobody restored for this many days is deleted, 7 by default;
ACTIONS_ARTIFACT_MAX_MB: one artifact, 2048 by default;
ACTIONS_ARTIFACT_RETENTION_DAYS: how long artifacts are kept when their workflow doesn't say, 30 by default, at most 90.

Expired artifacts and unused caches are deleted a minute after the control plane starts, then once a day.

The docs and the CMS#

The docs at /docs live in the CMS (Payload), not in the repository. Site administrators edit them at /cms, signed in with their Branchscope account; there's no separate CMS login, and nobody else gets in.

Drafts: "Save draft" keeps changes out of /docs. "Publish changes" makes them live.
Versions: each publish raises the page's version by one and sets its "Updated" date. Both show on every page. The CMS keeps the last 100 versions of each page, and you can restore any of them.
New pages: pick a slug (the address, /docs/<slug>) and an order (lower comes first). Only published pages are listed.
A fresh install loads the guide from apps/web/cms/seed/ into an empty CMS, as version 1. After that, the seed is never read again.
Schema changes are Payload migrations in apps/web/cms/migrations, applied when the web app starts.

Data#

Named volumes hold:

repositories and the SSH host key (git-data);
the indexer's mirrors, and the development hosted runner's credential;
the push guard's sockets;
Postgres (including the docs) and Garage data (graphs, logs, caches and artifacts).

Graphs can be rebuilt from repositories; repositories and Postgres can't. Back those up. The docs are in Postgres too, in the payload database.

Production#

deploy/production runs Branchscope on one server (an Ubuntu 24.04 or later machine with 8 GB of memory is enough) with Docker Compose, behind Caddy. Every name gets a TLS certificate from Let's Encrypt, renewed automatically:

Name	What
`<domain>`	the web app, and the docs CMS at `/cms`
`api.<domain>`	the API, MCP and OAuth
`git.<domain>`	git over HTTPS; git over SSH is on port 2222 (`GIT_SSH_PORT`)
`auth.<domain>`	Keycloak
`runners.<domain>`	where `bscope-runner` connects (gRPC over TLS)

Point DNS A records for the domain and *.<domain> at the server. Only Caddy (80 and 443) and git over SSH are published; everything else stays inside the compose network. Caddy also sets HSTS and a few other security headers, and gives MCP and git requests the long timeouts they need.

Deploying#

scripts/deploy deploys main's committed, pushed HEAD, and nothing else: it refuses on any other branch, or when HEAD isn't what origin/main has, so work that's uncommitted or unpushed never ships. It copies that tree to /srv/branchscope/src on the server, builds the images there (no registry) and restarts what changed.

# The first time: prepare the server, make its .env with fresh secrets, deploy
scripts/deploy root@<server> --setup --domain branchscope.example.com --admin <username> --email <you@example.com>
# After that
scripts/deploy root@<server>

A deploy doesn't change Keycloak's realm, which is imported only once. When a release changes it, as passkeys do, run deploy/production/keycloak-realm.sh on the server by hand afterwards (see Configuration).

--setup needs all three: --domain, the site's domain; --admin, the first site admin's username; --email, their email, also used for Let's Encrypt and the contact page. It runs, once:

setup-server.sh, as root: updates, Docker, a firewall (SSH, 80, 443 and 2222 only), 4 GB of swap, SSH with keys only, unattended security updates, a weekly prune of Docker's build cache, a rule that keeps containers away from the cloud metadata address, and the nightly backup.
init-env.sh: writes /srv/branchscope/.env with fresh passwords, service tokens and keys (SECRETS_KEY, EXTERNAL_TOKEN_KEY, OIDC_SIGNING_KEY, COMMIT_SIGNING_KEY among them), readable by root only. It never overwrites an existing file. The first passwords go to /srv/branchscope/first-sign-in.txt, never to the terminal: read it with ssh, sign in, change both passwords, and delete the file.

backup.sh runs every night at 02:30, server time, and dumps every database, Branchscope's, the CMS's and Keycloak's, to /srv/branchscope/backups, keeping 14 days. Repositories and stored graphs are in /srv/branchscope/data and the Docker volumes: back the server up as well, and keep a copy of .env's keys apart from the database dumps. To restore a dump: gunzip -c <file> | docker compose exec -T postgres psql -U branchscope -d postgres.

The production .env#

Variable	What it does
`BRANCHSCOPE_DOMAIN`	the domain; every public URL is made from it
`ACME_EMAIL`	the email Let's Encrypt writes to about certificates
`CONTACT_EMAIL`	the address the contact page gives (the web app)
`BRANCHSCOPE_ADMIN_USERNAME`, `BRANCHSCOPE_ADMIN_EMAIL`, `BRANCHSCOPE_ADMIN_PASSWORD`	the first site admin, made in Keycloak when the realm is first imported; they're asked to change the password at first sign-in
`KEYCLOAK_ADMIN_USERNAME`, `KEYCLOAK_ADMIN_PASSWORD`	Keycloak's own administrator, for its admin console (`https://auth.<domain>/admin`), where accounts are made: nobody can register
`POSTGRES_PASSWORD`, `PAYLOAD_DB_PASSWORD`, `KEYCLOAK_DB_PASSWORD`	the databases' passwords; the CMS and Keycloak each have a database and role of their own
`GARAGE_RPC_SECRET`, `GARAGE_ADMIN_TOKEN`	Garage's own secrets (a single node)
`S3_ACCESS_KEY`, `S3_SECRET_KEY`	the key Garage is started with, and that the control plane and indexer use for the bucket `branchscope`
`DATA_DIR`	where Caddy, Postgres, NATS and Garage keep their data, `/srv/branchscope/data`

The compose file sets the rest from these: Keycloak runs in production mode on its database (KC_DB, KC_DB_URL, KC_DB_USERNAME, KC_DB_PASSWORD), at KC_HOSTNAME https://auth.<domain>, behind Caddy (KC_PROXY_HEADERS), and its first administrator is KC_BOOTSTRAP_ADMIN_USERNAME and KC_BOOTSTRAP_ADMIN_PASSWORD; S3_ENDPOINT is http://garage:3900; ACCOUNTS_URL, the site admins' Accounts link, opens Keycloak's admin console on the realm's users. The push guard starts with PUSH_GUARD_REQUIRE_POLICY=true, and the control plane checks Keycloak for disabled accounts every minute (USER_SYNC_SECONDS=60).

Graph builds run through the Docker socket (JOB_RUNNER=docker) until hosted runners take them over. The optional keys and settings above (Linear, OIDC_ISSUER, previous keys while rotating) go in the same .env.

Security notes#

Service tokens: each service authenticates to the others with its own token, limited to its own endpoints.
The Docker socket: in development, sandbox-agent holds it to start job containers. That's root on the host, acceptable for development only. Hosted machines in production run jobs in Firecracker microVMs instead, on machines of their own; with them, the indexer needs no socket (JOB_RUNNER=hosted), and until then it builds through the socket itself (JOB_RUNNER=docker).
Job containers: they can reach package registries, and of Branchscope's services only the control plane (to upload what they built).
The push path fails closed (see Push guard).

Troubleshooting#

A page is missing or 404 after an update. The web image didn't rebuild. Its build also type-checks the end-to-end tests, so a type error anywhere keeps the old container running. Run docker compose up -d --build web and read the build output.
Pushes are refused with "push guard unavailable". Start the guard (docker compose up -d push-guard). Within seconds the control plane gives it its policy.
Graph builds wait with "retrying after …". Storage or Docker was unavailable; builds retry on their own (30 seconds, 2, 10 and 30 minutes).
Docker stops responding. Check free disk space first. Build caches grow quickly; docker builder prune frees them.
Sign-in loops on localhost. Use http://127.0.0.1:3300: Keycloak is configured for that host name.