A self-hosted Dolt database hosting service for a
SourceHut instance — "DoltLab for SourceHut". It hosts Dolt databases the way
git.sr.ht hosts git repos: dolt clone/push/pull over HTTPS plus an
integrated web UI that shares the SourceHut nav, unified login, and Bootstrap
theme.
Pure Go, one module (sourcecraft.dev/bigbes/sr-ht-dolt), no upstream SourceHut
modification — integration is config-driven: a [dolt.sr.ht] section in the
shared instance config.ini puts the service into every other service's nav.
Storage is bare NBS chunk-store directories (no .dolt/, no working set) at
<repos>/~<user>/<name> — exactly what remotesrv serves and file://
remotes use. PostgreSQL holds metadata (a mirror of meta's users, plus
repositories, ACLs, and dolt keys). Clone URL:
dolt clone https://dolt.srht.bigb.es/~user/db.
Two auth flows are supported for dolt clone/push: username + meta personal
access token (--user + DOLT_REMOTE_PASSWORD, HTTP Basic) and dolt's Ed25519
keypair flow (dolt creds / dolt login, Bearer EdDSA JWT — the
git-SSH-key-like UX). Agents get a third door, the MCP surface at /mcp
described at the end of this file.
Feature-complete for the v1 scope: the pure core/ domain (name/path
validation and the access matrix), the db/ Postgres layer, the authn/ auth
stack (unified-login cookie, meta PAT Basic auth, dolt-keypair Bearer JWT), the
storage/ bare-store lifecycle and remotesapi DBCache, the remoteapi/
remotesrv assembly plus CredentialsService, the read-only browse/ UI, the
web/ router/chrome, and the two binaries (doltsrht, doltsrht-migrate).
Go 1.26+
No C toolchain, no ICU, no zstd headers. The default build is pure Go
(CGO_ENABLED=0, statically linkable). github.com/dolthub/dolt/go normally
needs CGO for two libraries; both are avoided:
gms_pure_go build tag selects go-mysql-server's stdlib
regexp fallback instead of go-icu-regex. Safe here because this service
never runs the SQL engine (it serves bare NBS stores and browses read-only),
so it never evaluates SQL REGEXP.replace github.com/dolthub/gozstd => ./third_party/gozstd-purego
directive backs dolt's zstd dependency with a pure-Go shim over
klauspost/compress/zstd (see that
directory's README/tests, incl. libzstd interop). dolt itself is unmodified.make and make build pass -tags gms_pure_go and CGO_ENABLED=0 for you; a
bare go build needs -tags gms_pure_go.
make CGO_ENABLED=1 GO_TAGS=.
It then needs a C toolchain and ICU4C headers — Debian/Ubuntu
apt install libicu-dev; macOS keg-only brew install icu4c with
CGO_CPPFLAGS="-I/opt/homebrew/opt/icu4c@78/include" /
CGO_LDFLAGS="-L/opt/homebrew/opt/icu4c@78/lib".sassc + minify — only for building CSS (make css); not needed for the
default build:
brew install sassc # or apt install sassc
go install github.com/tdewolff/minify/v2/cmd/minify@latest
CSS is compiled against the shared sourcehut SCSS: make css ASSETS=/path/to/sourcehut/scss/parent
(ASSETS defaults to /usr/share/sourcehut; sassc is invoked with
-I $(ASSETS)/scss).
sourcecraft.dev/bigbes/sr-ht-core, replaced by the instance fork
git.srht.bigb.es/~bigbes/core-go (pinned via a replace directive to the
fork's master, currently c2c2f38 = upstream + instance patches). The fork
is what production actually runs; token validation, config, and crypto must
behave identically to the rest of the instance. Never go get -u it and
never drop the replace; to bump, pin the fork's new commit pseudo-version
in the replace line and re-run the test suite. Fetching the fork needs
GOPRIVATE=git.srht.bigb.es (skips the public module proxy/sumdb).github.com/dolthub/dolt/go v0.40.5-0.20260626152440-45335d44ad79 — a
pseudo-version pinned to the commit tagged v2.1.10 (45335d44), the dolt
CLI version installed on the target host (/opt/homebrew/bin/dolt, v2.1.10).
The dolt /go submodule's latest tag is the stale v0.40.4 (2021), which
does not interop with a modern CLI; matching the CLI's commit guarantees a
common NBS storage format (types.Format_DOLT / __DOLT__) and remotesapi
proto. Verified end-to-end by the Phase-0 spike (see below). If the CLI is
upgraded, re-pin dolt/go to the new CLI's commit and re-run the spike.gopkg.in/go-jose/go-jose.v2 v2.6.3 — the same JOSE major/version that
dolt/go's creds package uses to sign the EdDSA keypair JWTs, so the Bearer
verify path stays byte-compatible and no duplicate JOSE lib is pulled in.github.com/dolthub/gozstd, replaced by the local ./third_party/gozstd-purego
shim — a pure-Go, drop-in reimplementation of the nine gozstd symbols dolt
references (Compress/CompressDict/Decompress/DecompressDict/BuildDict, the
CDict/DDict types and their constructors), backed by
github.com/klauspost/compress/zstd (v1.18.0, already in the graph). This is
what lets the default build be CGO_ENABLED=0. dolt uses gozstd only from its
NBS archive subsystem; this service only ever hits the decompress side at
runtime, and zstd frames/dictionaries are standard-format, so libzstd-authored
archives decode correctly (proven by the shim's libzstd-interop tests). Keep
the replace; to drop it, build the cgo variant (see Build prerequisites).storage/spike_test.go (build tag spike) is the Phase-0 de-risk gate. It
inits a bare NBS store via doltdb.LoadDoltDB + WriteEmptyRepo, serves it
with remotesrv.NewServer on an ephemeral localhost port (single-port
http+gRPC multiplex, no auth), then drives the real dolt CLI through a full
round-trip: clone → create table + insert → commit → push → fresh re-clone →
verify the rows. It skips (does not fail) when the CLI is absent, and uses an
isolated $HOME so your real dolt config is untouched.
go test -tags 'gms_pure_go spike' ./storage/ -run TestSpike -v
go build ./... # build every package (needs the CGO env above)
go test ./... # full suite (db/ + remoteapi/ skip without Docker)
go test -tags spike ./storage/ -v # the interop spike (needs dolt CLI)
go test -tags integration ./remoteapi/ # clone/push round-trips (needs Docker)
make # build both binaries into ./doltsrht[-migrate]
make css # build stylesheets (needs sassc + minify)
dolt.sr.ht reads the single shared instance config.ini (the same file every
*.sr.ht service reads). Add a [dolt.sr.ht] section — the full block, copied
from config.example.ini:
[dolt.sr.ht]
; External URL. Also the JWT audience checked in the `dolt creds` keypair flow,
; so it must match the host clients pass to `dolt login --auth-endpoint`.
origin=https://dolt.srht.bigb.es
; Postgres connection string for the metadata database.
connection-string=postgresql://doltsrht@localhost/dolt.sr.ht?sslmode=disable
; Root dir holding the bare NBS chunk-store dirs (<repos>/~<user>/<name>).
repos=/var/lib/dolt
; remotesapi listener (gRPC ChunkStoreService + HTTP chunk data plane, h2c).
remotesapi-listen=127.0.0.1:5306
; CredentialsService.WhoAmI listener (the `dolt login` keypair flow).
credsapi-listen=127.0.0.1:5308
; Directory of built static assets (main.min.<sha>.css, logo.svg).
static-dir=/usr/share/sourcehut/dolt.sr.ht/static
; Run brant migrations automatically on package upgrade.
migrate-on-upgrade=yes
connection-string and origin are required (doltsrht fails fast if
either is missing); everything else has the default shown above.
The service also reuses these shared keys owned by other services — do not
duplicate their values, they already live in the shared config.ini:
| Key | Used for |
|---|---|
[sr.ht]network-key |
internal service-to-service auth (crypto.InitCrypto) |
[sr.ht]owner-name / owner-email |
author of each database's initial empty commit |
[sr.ht]site-name |
shared nav brand |
[sr.ht]environment |
a non-production value adds a banner to every page |
[webhooks]private-key |
derives the offline HMAC key that validates meta PATs |
[meta.sr.ht]origin |
login redirects and profile fetches |
[meta.sr.ht::api]internal-ipnet |
subnets allowed to use meta's internal auth |
Deployment prerequisite: this host must be inside meta's
internal-ipnet. Token validation (FetchMetaProfile+LookupTokenRevocation) and cookie resolution use internal auth against meta; a misconfig here surfaces as a "Temporary error" on the first login or push.
Adding the [dolt.sr.ht] origin= line to a shared config and restarting the
other services is all that is needed for dolt.sr.ht to appear in their nav.
doltsrht-migrate is a single-service brant wrapper. It reads
connection-string from [dolt.sr.ht] (override with --dsn) and loads
migrations from ./migrations in a checkout or the installed
/usr/share/sourcehut/migrations/dolt.sr.ht otherwise.
createdb dolt.sr.ht
doltsrht-migrate init # apply schema.sql wholesale, stamp to head (fresh install)
doltsrht-migrate up # apply pending migrations/*.sql (upgrades)
doltsrht-migrate current # print the current schema version
doltsrht-migrate -a up # honor migrate-on-upgrade; no-op when disabled
Use init once on a brand-new database; use up for every subsequent upgrade.
Install contrib/dolt.sr.ht.conf into the nginx sites directory alongside the
other *.sr.ht.conf files (TLS/http2 and the shared proxy snippets come from
the included sourcehut.conf / port443.conf). It path-routes four back ends
on one server_name dolt.srht.bigb.es:
/dolt.services.remotesapi.v1alpha1.ChunkStoreService/ → grpc://127.0.0.1:5306
(clone/pull/push RPCs; sets X-Forwarded-Proto https so the server hands back
https:// sealed chunk URLs)./dolt.services.remotesapi.v1alpha1.CredentialsService/ → grpc://127.0.0.1:5308
(the dolt login keypair WhoAmI)./single_symmetric_key_sealed_request/ → http://127.0.0.1:5306 (the AES-GCM
sealed chunk data plane; client_max_body_size 0, buffering off)./ → http://127.0.0.1:5307 (web UI + /static).Add a dolt.srht.bigb.es DNS record pointing at the instance.
[Unit]
Description=dolt.sr.ht service
After=network.target postgresql.service
[Service]
User=dolt
ExecStart=/usr/local/bin/doltsrht
Restart=on-failure
# The bare NBS stores live here; the user must own it.
StateDirectory=dolt
[Install]
WantedBy=multi-user.target
doltsrht binds all three listeners (web -b localhost:5307, remotesapi
127.0.0.1:5306, credentials 127.0.0.1:5308) and shuts them down cleanly on
SIGINT/SIGTERM. Point [dolt.sr.ht]repos at the StateDirectory
(/var/lib/dolt).
Once the service is up and you are logged into meta:
Create a database. Visit https://dolt.srht.bigb.es, click Create,
pick a name and visibility (PUBLIC / UNLISTED / PRIVATE). It appears at
~<you>/<name>.
Clone/push with a meta personal access token (PAT):
export DOLT_REMOTE_PASSWORD=<your meta PAT>
dolt clone --user <you> https://dolt.srht.bigb.es/~<you>/<name>
cd <name>
dolt sql -q 'CREATE TABLE t (id INT PRIMARY KEY)'
dolt add . && dolt commit -m 'init'
dolt push origin main
--user selects HTTP Basic auth; the password comes from
DOLT_REMOTE_PASSWORD. A PAT with the dolt.sr.ht/repos:RW grant (or an
empty-grant personal token) can push; :RO or no grant can only read.
Or clone/push with a dolt keypair (the git-SSH-key-like UX):
dolt creds new # generate an Ed25519 keypair
dolt login --auth-endpoint dolt.srht.bigb.es:443 \
--login-url https://dolt.srht.bigb.es/settings/keys
dolt login opens the settings page with your public key in the URL
fragment; associate it with your account, and dolt login confirms via the
CredentialsService. Thereafter plain dolt clone/push (no --user, no env
var) works. To make those the defaults so you can drop the flags:
dolt config --global --add remotes.default_host dolt.srht.bigb.es
dolt config --global --add creds.add_url https://dolt.srht.bigb.es/settings/keys
Anonymous dolt clone works for PUBLIC and UNLISTED databases with no
credentials at all; PRIVATE databases return "not found" to unauthorized
callers (their existence is not leaked).
An agent reads a hosted database — and the beads tracker inside one — by calling
tools instead of scraping HTML. The endpoint speaks the Model Context Protocol
over streamable HTTP at /mcp on the web listener
(https://dolt.srht.bigb.es/mcp); there is no second port and no switch that
turns it off, because a surface that is off in production and on in a test is a
surface nobody tests.
It is read-only, and structurally so: there is no query(sql) tool and no
mutation of any kind. The tools list databases, branches, tables, rows, commits
and diffs, and — for a database carrying the beads schema — issues, milestones,
memories and the ready set. Writing stays with bd in a checkout.
The credential is Authorization: Bearer <token>, and nothing else: no cookie
(an MCP client is not a browser) and no HTTP Basic (that is dolt clone's
flow). Two tokens are accepted. A tokens.sr.ht working token must carry the
dolt:read grant — one grant for the whole surface, because every tool on it is
a read — and it works only on an instance that configures [tokens.sr.ht] origin; without that section there is no daemon to verify it against, so it is
refused with 401 while everything else keeps working. A meta personal access
token is accepted with the same dolt.sr.ht/repos:RO scoping the clone path
applies. No credential at all is a normal caller: it reads what an anonymous
visitor reads, and a PRIVATE database it may not see is "not found" rather than
"forbidden", exactly as in the web UI.