#dolt.sr.ht

A self-hosted Dolt database hosting service for a SourceHut instance — "DoltLab for SourceHut". It hosts Dolt databases the way git.sr.ht hosts git repos: dolt clone/push/pull over HTTPS plus an integrated web UI that shares the SourceHut nav, unified login, and Bootstrap theme.

Pure Go, one module (sourcecraft.dev/bigbes/sr-ht-dolt), no upstream SourceHut modification — integration is config-driven: a [dolt.sr.ht] section in the shared instance config.ini puts the service into every other service's nav.

Storage is bare NBS chunk-store directories (no .dolt/, no working set) at <repos>/~<user>/<name> — exactly what remotesrv serves and file:// remotes use. PostgreSQL holds metadata (a mirror of meta's users, plus repositories, ACLs, and dolt keys). Clone URL: dolt clone https://dolt.srht.bigb.es/~user/db.

Two auth flows are supported for dolt clone/push: username + meta personal access token (--user + DOLT_REMOTE_PASSWORD, HTTP Basic) and dolt's Ed25519 keypair flow (dolt creds / dolt login, Bearer EdDSA JWT — the git-SSH-key-like UX). Agents get a third door, the MCP surface at /mcp described at the end of this file.

#Status

Feature-complete for the v1 scope: the pure core/ domain (name/path validation and the access matrix), the db/ Postgres layer, the authn/ auth stack (unified-login cookie, meta PAT Basic auth, dolt-keypair Bearer JWT), the storage/ bare-store lifecycle and remotesapi DBCache, the remoteapi/ remotesrv assembly plus CredentialsService, the read-only browse/ UI, the web/ router/chrome, and the two binaries (doltsrht, doltsrht-migrate).

#Build prerequisites

• Go 1.26+

• No C toolchain, no ICU, no zstd headers. The default build is pure Go (CGO_ENABLED=0, statically linkable). github.com/dolthub/dolt/go normally needs CGO for two libraries; both are avoided:

  • ICU regex — the gms_pure_go build tag selects go-mysql-server's stdlib regexp fallback instead of go-icu-regex. Safe here because this service never runs the SQL engine (it serves bare NBS stores and browses read-only), so it never evaluates SQL REGEXP.
  • zstd — a replace github.com/dolthub/gozstd => ./third_party/gozstd-purego directive backs dolt's zstd dependency with a pure-Go shim over klauspost/compress/zstd (see that directory's README/tests, incl. libzstd interop). dolt itself is unmodified.

  make and make build pass -tags gms_pure_go and CGO_ENABLED=0 for you; a bare go build needs -tags gms_pure_go.

  • Optional cgo variant (upstream gozstd + ICU): make CGO_ENABLED=1 GO_TAGS=. It then needs a C toolchain and ICU4C headers — Debian/Ubuntu apt install libicu-dev; macOS keg-only brew install icu4c with CGO_CPPFLAGS="-I/opt/homebrew/opt/icu4c@78/include" / CGO_LDFLAGS="-L/opt/homebrew/opt/icu4c@78/lib".

• sassc + minify — only for building CSS (make css); not needed for the default build:

  brew install sassc                                   # or apt install sassc
  go install github.com/tdewolff/minify/v2/cmd/minify@latest
  

  CSS is compiled against the shared sourcehut SCSS: make css ASSETS=/path/to/sourcehut/scss/parent (ASSETS defaults to /usr/share/sourcehut; sassc is invoked with -I $(ASSETS)/scss).

#Pinned dependencies (and why)

• sourcecraft.dev/bigbes/sr-ht-core, replaced by the instance fork git.srht.bigb.es/~bigbes/core-go (pinned via a replace directive to the fork's master, currently c2c2f38 = upstream + instance patches). The fork is what production actually runs; token validation, config, and crypto must behave identically to the rest of the instance. Never go get -u it and never drop the replace; to bump, pin the fork's new commit pseudo-version in the replace line and re-run the test suite. Fetching the fork needs GOPRIVATE=git.srht.bigb.es (skips the public module proxy/sumdb).
• github.com/dolthub/dolt/go v0.40.5-0.20260626152440-45335d44ad79 — a pseudo-version pinned to the commit tagged v2.1.10 (45335d44), the dolt CLI version installed on the target host (/opt/homebrew/bin/dolt, v2.1.10). The dolt /go submodule's latest tag is the stale v0.40.4 (2021), which does not interop with a modern CLI; matching the CLI's commit guarantees a common NBS storage format (types.Format_DOLT / __DOLT__) and remotesapi proto. Verified end-to-end by the Phase-0 spike (see below). If the CLI is upgraded, re-pin dolt/go to the new CLI's commit and re-run the spike.
• gopkg.in/go-jose/go-jose.v2 v2.6.3 — the same JOSE major/version that dolt/go's creds package uses to sign the EdDSA keypair JWTs, so the Bearer verify path stays byte-compatible and no duplicate JOSE lib is pulled in.
• github.com/dolthub/gozstd, replaced by the local ./third_party/gozstd-purego shim — a pure-Go, drop-in reimplementation of the nine gozstd symbols dolt references (Compress/CompressDict/Decompress/DecompressDict/BuildDict, the CDict/DDict types and their constructors), backed by github.com/klauspost/compress/zstd (v1.18.0, already in the graph). This is what lets the default build be CGO_ENABLED=0. dolt uses gozstd only from its NBS archive subsystem; this service only ever hits the decompress side at runtime, and zstd frames/dictionaries are standard-format, so libzstd-authored archives decode correctly (proven by the shim's libzstd-interop tests). Keep the replace; to drop it, build the cgo variant (see Build prerequisites).
• grpc v1.79.3, logrus v1.8.3, lib/pq v1.10.9, chi/v5, and brant round out the transport, logging, Postgres driver, HTTP router, and migration tooling.

#The spike

storage/spike_test.go (build tag spike) is the Phase-0 de-risk gate. It inits a bare NBS store via doltdb.LoadDoltDB + WriteEmptyRepo, serves it with remotesrv.NewServer on an ephemeral localhost port (single-port http+gRPC multiplex, no auth), then drives the real dolt CLI through a full round-trip: clone → create table + insert → commit → push → fresh re-clone → verify the rows. It skips (does not fail) when the CLI is absent, and uses an isolated $HOME so your real dolt config is untouched.

go test -tags 'gms_pure_go spike' ./storage/ -run TestSpike -v

#Dev commands

go build ./...                 # build every package (needs the CGO env above)
go test ./...                  # full suite (db/ + remoteapi/ skip without Docker)
go test -tags spike ./storage/ -v         # the interop spike (needs dolt CLI)
go test -tags integration ./remoteapi/    # clone/push round-trips (needs Docker)
make                           # build both binaries into ./doltsrht[-migrate]
make css                       # build stylesheets (needs sassc + minify)

#Deployment

#1. Config

dolt.sr.ht reads the single shared instance config.ini (the same file every *.sr.ht service reads). Add a [dolt.sr.ht] section — the full block, copied from config.example.ini:

[dolt.sr.ht]
; External URL. Also the JWT audience checked in the `dolt creds` keypair flow,
; so it must match the host clients pass to `dolt login --auth-endpoint`.
origin=https://dolt.srht.bigb.es
; Postgres connection string for the metadata database.
connection-string=postgresql://doltsrht@localhost/dolt.sr.ht?sslmode=disable
; Root dir holding the bare NBS chunk-store dirs (<repos>/~<user>/<name>).
repos=/var/lib/dolt
; remotesapi listener (gRPC ChunkStoreService + HTTP chunk data plane, h2c).
remotesapi-listen=127.0.0.1:5306
; CredentialsService.WhoAmI listener (the `dolt login` keypair flow).
credsapi-listen=127.0.0.1:5308
; Directory of built static assets (main.min.<sha>.css, logo.svg).
static-dir=/usr/share/sourcehut/dolt.sr.ht/static
; Run brant migrations automatically on package upgrade.
migrate-on-upgrade=yes

connection-string and origin are required (doltsrht fails fast if either is missing); everything else has the default shown above.

The service also reuses these shared keys owned by other services — do not duplicate their values, they already live in the shared config.ini:

Deployment prerequisite: this host must be inside meta's internal-ipnet. Token validation (FetchMetaProfile + LookupTokenRevocation) and cookie resolution use internal auth against meta; a misconfig here surfaces as a "Temporary error" on the first login or push.

Adding the [dolt.sr.ht] origin= line to a shared config and restarting the other services is all that is needed for dolt.sr.ht to appear in their nav.

#2. Database + migrations

doltsrht-migrate is a single-service brant wrapper. It reads connection-string from [dolt.sr.ht] (override with --dsn) and loads migrations from ./migrations in a checkout or the installed /usr/share/sourcehut/migrations/dolt.sr.ht otherwise.

createdb dolt.sr.ht
doltsrht-migrate init        # apply schema.sql wholesale, stamp to head (fresh install)
doltsrht-migrate up          # apply pending migrations/*.sql (upgrades)
doltsrht-migrate current     # print the current schema version
doltsrht-migrate -a up       # honor migrate-on-upgrade; no-op when disabled

Use init once on a brand-new database; use up for every subsequent upgrade.

#3. nginx + DNS

Install contrib/dolt.sr.ht.conf into the nginx sites directory alongside the other *.sr.ht.conf files (TLS/http2 and the shared proxy snippets come from the included sourcehut.conf / port443.conf). It path-routes four back ends on one server_name dolt.srht.bigb.es:

• /dolt.services.remotesapi.v1alpha1.ChunkStoreService/ → grpc://127.0.0.1:5306 (clone/pull/push RPCs; sets X-Forwarded-Proto https so the server hands back https:// sealed chunk URLs).
• /dolt.services.remotesapi.v1alpha1.CredentialsService/ → grpc://127.0.0.1:5308 (the dolt login keypair WhoAmI).
• /single_symmetric_key_sealed_request/ → http://127.0.0.1:5306 (the AES-GCM sealed chunk data plane; client_max_body_size 0, buffering off).
• / → http://127.0.0.1:5307 (web UI + /static).

Add a dolt.srht.bigb.es DNS record pointing at the instance.

#4. systemd unit sketch

[Unit]
Description=dolt.sr.ht service
After=network.target postgresql.service

[Service]
User=dolt
ExecStart=/usr/local/bin/doltsrht
Restart=on-failure
# The bare NBS stores live here; the user must own it.
StateDirectory=dolt

[Install]
WantedBy=multi-user.target

doltsrht binds all three listeners (web -b localhost:5307, remotesapi 127.0.0.1:5306, credentials 127.0.0.1:5308) and shuts them down cleanly on SIGINT/SIGTERM. Point [dolt.sr.ht]repos at the StateDirectory (/var/lib/dolt).

#User quickstart

Once the service is up and you are logged into meta:

1. Create a database. Visit https://dolt.srht.bigb.es, click Create, pick a name and visibility (PUBLIC / UNLISTED / PRIVATE). It appears at ~<you>/<name>.

2. Clone/push with a meta personal access token (PAT):

   export DOLT_REMOTE_PASSWORD=<your meta PAT>
   dolt clone --user <you> https://dolt.srht.bigb.es/~<you>/<name>
   cd <name>
   dolt sql -q 'CREATE TABLE t (id INT PRIMARY KEY)'
   dolt add . && dolt commit -m 'init'
   dolt push origin main
   

   --user selects HTTP Basic auth; the password comes from DOLT_REMOTE_PASSWORD. A PAT with the dolt.sr.ht/repos:RW grant (or an empty-grant personal token) can push; :RO or no grant can only read.

3. Or clone/push with a dolt keypair (the git-SSH-key-like UX):

   dolt creds new          # generate an Ed25519 keypair
   dolt login --auth-endpoint dolt.srht.bigb.es:443 \
              --login-url https://dolt.srht.bigb.es/settings/keys
   

   dolt login opens the settings page with your public key in the URL fragment; associate it with your account, and dolt login confirms via the CredentialsService. Thereafter plain dolt clone/push (no --user, no env var) works. To make those the defaults so you can drop the flags:

   dolt config --global --add remotes.default_host dolt.srht.bigb.es
   dolt config --global --add creds.add_url https://dolt.srht.bigb.es/settings/keys
   

4. Rename a database. ~<you>/<name> → settings → Rename. The record and the stored data move together, and no redirect is left behind: the old address stops resolving, so an existing checkout needs its remote replaced (dolt remote has no set-url).

   dolt remote remove origin
   dolt remote add origin https://dolt.srht.bigb.es/~<you>/<new-name>
   

   A database provisioned as the companion of a git.sr.ht repository is re-created under its old name by the next push to that repository — the hook provisions by the git repo's name, which renaming here does not change.

Anonymous dolt clone works for PUBLIC and UNLISTED databases with no credentials at all; PRIVATE databases return "not found" to unauthorized callers (their existence is not leaked).

#The MCP surface

An agent reads a hosted database — and the beads tracker inside one — by calling tools instead of scraping HTML. The endpoint speaks the Model Context Protocol over streamable HTTP at /mcp on the web listener (https://dolt.srht.bigb.es/mcp); there is no second port and no switch that turns it off, because a surface that is off in production and on in a test is a surface nobody tests.

It is read-only, and structurally so: there is no query(sql) tool and no mutation of any kind. The tools list databases, branches, tables, rows, commits and diffs, and — for a database carrying the beads schema — issues, milestones, memories and the ready set. Writing stays with bd in a checkout.

The credential is Authorization: Bearer <token>, and nothing else: no cookie (an MCP client is not a browser) and no HTTP Basic (that is dolt clone's flow). Two tokens are accepted. A tokens.sr.ht working token must carry the dolt:read grant — one grant for the whole surface, because every tool on it is a read — and it works only on an instance that configures [tokens.sr.ht] origin; without that section there is no daemon to verify it against, so it is refused with 401 while everything else keeps working. A meta personal access token is accepted with the same dolt.sr.ht/repos:RO scoping the clone path applies. No credential at all is a normal caller: it reads what an anonymous visitor reads, and a PRIVATE database it may not see is "not found" rather than "forbidden", exactly as in the web UI.

#The GraphQL surface

/query on the same web listener (https://dolt.srht.bigb.es/query) serves a read-only GraphQL schema, so everything on the instance that already speaks SourceHut GraphQL can read this service too. It is POST only — a query in a URL is a cross-origin-readable address for data that is often private — and introspection is on, so a typed client can be generated against it.

curl -sS https://dolt.srht.bigb.es/query \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"query":"{ databases { results { name visibility owner { canonicalName } } } }"}'

The schema answers what a database is — databases, databasesByOwner and database(owner:name:), each with its branches, commit log and table list, and its ACL to the owner alone. It does not serve table rows or diffs: those live on /mcp, where a read that had to be clipped says so in its own answer. There are no mutations; creating, renaming and deleting a database stay behind the web UI.

The credential plane is /mcp's exactly — a meta personal access token scoped dolt.sr.ht/repos:RO, or a tokens.sr.ht working token carrying dolt:read — and no credential at all is a normal caller that reads what an anonymous visitor reads. A database the caller may not see resolves to null rather than to an authorization error, so its existence cannot be read out of the shape of the refusal.

Listings page with the instance-standard opaque cursor: databases(filter: {count: 10}) returns a cursor, and passing it back as databases(cursor: "…") continues the walk.

Beside it, /query/api-meta.json publishes the one grant this service defines (repos), which is what lets meta.sr.ht offer dolt.sr.ht/repos:RO on its personal-token page. To federate the schema into api.sr.ht, give the gateway's config a [dolt.sr.ht] api-origin= line pointing here and SIGHUP it; nothing in this service depends on the gateway existing.

hut graphql dolt needs a patched hut. Upstream v0.8.0 carries a hard-coded list of the ten sr.ht services and dereferences a nil entry for anything else, so it segfaults on every custom service. The fix is small and lives at git.srht.bigb.es/~bigbes/hut; with it, hut graphql dolt resolves this endpoint from the instance origin like any other service.