#dolt.sr.ht

A self-hosted Dolt database hosting service for a SourceHut instance — "DoltLab for SourceHut". It hosts Dolt databases the way git.sr.ht hosts git repos: dolt clone/push/pull over HTTPS plus an integrated web UI that shares the SourceHut nav, unified login, and Bootstrap theme.

Pure Go, one module (sourcecraft.dev/bigbes/sr-ht-dolt), no upstream SourceHut modification — integration is config-driven: a [dolt.sr.ht] section in the shared instance config.ini puts the service into every other service's nav.

Storage is bare NBS chunk-store directories (no .dolt/, no working set) at <repos>/~<user>/<name> — exactly what remotesrv serves and file:// remotes use. PostgreSQL holds metadata (a mirror of meta's users, plus repositories, ACLs, and dolt keys). Clone URL: dolt clone https://dolt.srht.bigb.es/~user/db.

Two auth flows are supported: username + meta personal access token (--user + DOLT_REMOTE_PASSWORD, HTTP Basic) and dolt's Ed25519 keypair flow (dolt creds / dolt login, Bearer EdDSA JWT — the git-SSH-key-like UX).

#Status

Feature-complete for the v1 scope: the pure core/ domain (name/path validation and the access matrix), the db/ Postgres layer, the authn/ auth stack (unified-login cookie, meta PAT Basic auth, dolt-keypair Bearer JWT), the storage/ bare-store lifecycle and remotesapi DBCache, the remoteapi/ remotesrv assembly plus CredentialsService, the read-only browse/ UI, the web/ router/chrome, and the two binaries (doltsrht, doltsrht-migrate).

#Build prerequisites

• Go 1.26+

• No C toolchain, no ICU, no zstd headers. The default build is pure Go (CGO_ENABLED=0, statically linkable). github.com/dolthub/dolt/go normally needs CGO for two libraries; both are avoided:

  • ICU regex — the gms_pure_go build tag selects go-mysql-server's stdlib regexp fallback instead of go-icu-regex. Safe here because this service never runs the SQL engine (it serves bare NBS stores and browses read-only), so it never evaluates SQL REGEXP.
  • zstd — a replace github.com/dolthub/gozstd => ./third_party/gozstd-purego directive backs dolt's zstd dependency with a pure-Go shim over klauspost/compress/zstd (see that directory's README/tests, incl. libzstd interop). dolt itself is unmodified.

  make and make build pass -tags gms_pure_go and CGO_ENABLED=0 for you; a bare go build needs -tags gms_pure_go.

  • Optional cgo variant (upstream gozstd + ICU): make CGO_ENABLED=1 GO_TAGS=. It then needs a C toolchain and ICU4C headers — Debian/Ubuntu apt install libicu-dev; macOS keg-only brew install icu4c with CGO_CPPFLAGS="-I/opt/homebrew/opt/icu4c@78/include" / CGO_LDFLAGS="-L/opt/homebrew/opt/icu4c@78/lib".

• sassc + minify — only for building CSS (make css); not needed for the default build:

  brew install sassc                                   # or apt install sassc
  go install github.com/tdewolff/minify/v2/cmd/minify@latest
  

  CSS is compiled against the shared sourcehut SCSS: make css ASSETS=/path/to/sourcehut/scss/parent (ASSETS defaults to /usr/share/sourcehut; sassc is invoked with -I $(ASSETS)/scss).

#Pinned dependencies (and why)

• sourcecraft.dev/bigbes/sr-ht-core, replaced by the instance fork git.srht.bigb.es/~bigbes/core-go (pinned via a replace directive to the fork's master, currently c2c2f38 = upstream + instance patches). The fork is what production actually runs; token validation, config, and crypto must behave identically to the rest of the instance. Never go get -u it and never drop the replace; to bump, pin the fork's new commit pseudo-version in the replace line and re-run the test suite. Fetching the fork needs GOPRIVATE=git.srht.bigb.es (skips the public module proxy/sumdb).
• github.com/dolthub/dolt/go v0.40.5-0.20260626152440-45335d44ad79 — a pseudo-version pinned to the commit tagged v2.1.10 (45335d44), the dolt CLI version installed on the target host (/opt/homebrew/bin/dolt, v2.1.10). The dolt /go submodule's latest tag is the stale v0.40.4 (2021), which does not interop with a modern CLI; matching the CLI's commit guarantees a common NBS storage format (types.Format_DOLT / __DOLT__) and remotesapi proto. Verified end-to-end by the Phase-0 spike (see below). If the CLI is upgraded, re-pin dolt/go to the new CLI's commit and re-run the spike.
• gopkg.in/go-jose/go-jose.v2 v2.6.3 — the same JOSE major/version that dolt/go's creds package uses to sign the EdDSA keypair JWTs, so the Bearer verify path stays byte-compatible and no duplicate JOSE lib is pulled in.
• github.com/dolthub/gozstd, replaced by the local ./third_party/gozstd-purego shim — a pure-Go, drop-in reimplementation of the nine gozstd symbols dolt references (Compress/CompressDict/Decompress/DecompressDict/BuildDict, the CDict/DDict types and their constructors), backed by github.com/klauspost/compress/zstd (v1.18.0, already in the graph). This is what lets the default build be CGO_ENABLED=0. dolt uses gozstd only from its NBS archive subsystem; this service only ever hits the decompress side at runtime, and zstd frames/dictionaries are standard-format, so libzstd-authored archives decode correctly (proven by the shim's libzstd-interop tests). Keep the replace; to drop it, build the cgo variant (see Build prerequisites).
• grpc v1.79.3, logrus v1.8.3, lib/pq v1.10.9, chi/v5, and brant round out the transport, logging, Postgres driver, HTTP router, and migration tooling.

#The spike

storage/spike_test.go (build tag spike) is the Phase-0 de-risk gate. It inits a bare NBS store via doltdb.LoadDoltDB + WriteEmptyRepo, serves it with remotesrv.NewServer on an ephemeral localhost port (single-port http+gRPC multiplex, no auth), then drives the real dolt CLI through a full round-trip: clone → create table + insert → commit → push → fresh re-clone → verify the rows. It skips (does not fail) when the CLI is absent, and uses an isolated $HOME so your real dolt config is untouched.

go test -tags 'gms_pure_go spike' ./storage/ -run TestSpike -v

#Dev commands

go build ./...                 # build every package (needs the CGO env above)
go test ./...                  # full suite (db/ + remoteapi/ skip without Docker)
go test -tags spike ./storage/ -v         # the interop spike (needs dolt CLI)
go test -tags integration ./remoteapi/    # clone/push round-trips (needs Docker)
make                           # build both binaries into ./doltsrht[-migrate]
make css                       # build stylesheets (needs sassc + minify)

#Deployment

#1. Config

dolt.sr.ht reads the single shared instance config.ini (the same file every *.sr.ht service reads). Add a [dolt.sr.ht] section — the full block, copied from config.example.ini:

[dolt.sr.ht]
; External URL. Also the JWT audience checked in the `dolt creds` keypair flow,
; so it must match the host clients pass to `dolt login --auth-endpoint`.
origin=https://dolt.srht.bigb.es
; Postgres connection string for the metadata database.
connection-string=postgresql://doltsrht@localhost/dolt.sr.ht?sslmode=disable
; Root dir holding the bare NBS chunk-store dirs (<repos>/~<user>/<name>).
repos=/var/lib/dolt
; remotesapi listener (gRPC ChunkStoreService + HTTP chunk data plane, h2c).
remotesapi-listen=127.0.0.1:5306
; CredentialsService.WhoAmI listener (the `dolt login` keypair flow).
credsapi-listen=127.0.0.1:5308
; Directory of built static assets (main.min.<sha>.css, logo.svg).
static-dir=/usr/share/sourcehut/dolt.sr.ht/static
; Run brant migrations automatically on package upgrade.
migrate-on-upgrade=yes

connection-string and origin are required (doltsrht fails fast if either is missing); everything else has the default shown above.

The service also reuses these shared keys owned by other services — do not duplicate their values, they already live in the shared config.ini:

Deployment prerequisite: this host must be inside meta's internal-ipnet. Token validation (FetchMetaProfile + LookupTokenRevocation) and cookie resolution use internal auth against meta; a misconfig here surfaces as a "Temporary error" on the first login or push.

Adding the [dolt.sr.ht] origin= line to a shared config and restarting the other services is all that is needed for dolt.sr.ht to appear in their nav.

#2. Database + migrations

doltsrht-migrate is a single-service brant wrapper. It reads connection-string from [dolt.sr.ht] (override with --dsn) and loads migrations from ./migrations in a checkout or the installed /usr/share/sourcehut/migrations/dolt.sr.ht otherwise.

createdb dolt.sr.ht
doltsrht-migrate init        # apply schema.sql wholesale, stamp to head (fresh install)
doltsrht-migrate up          # apply pending migrations/*.sql (upgrades)
doltsrht-migrate current     # print the current schema version
doltsrht-migrate -a up       # honor migrate-on-upgrade; no-op when disabled

Use init once on a brand-new database; use up for every subsequent upgrade.

#3. nginx + DNS

Install contrib/dolt.sr.ht.conf into the nginx sites directory alongside the other *.sr.ht.conf files (TLS/http2 and the shared proxy snippets come from the included sourcehut.conf / port443.conf). It path-routes four back ends on one server_name dolt.srht.bigb.es:

• /dolt.services.remotesapi.v1alpha1.ChunkStoreService/ → grpc://127.0.0.1:5306 (clone/pull/push RPCs; sets X-Forwarded-Proto https so the server hands back https:// sealed chunk URLs).
• /dolt.services.remotesapi.v1alpha1.CredentialsService/ → grpc://127.0.0.1:5308 (the dolt login keypair WhoAmI).
• /single_symmetric_key_sealed_request/ → http://127.0.0.1:5306 (the AES-GCM sealed chunk data plane; client_max_body_size 0, buffering off).
• / → http://127.0.0.1:5307 (web UI + /static).

Add a dolt.srht.bigb.es DNS record pointing at the instance.

#4. systemd unit sketch

[Unit]
Description=dolt.sr.ht service
After=network.target postgresql.service

[Service]
User=dolt
ExecStart=/usr/local/bin/doltsrht
Restart=on-failure
# The bare NBS stores live here; the user must own it.
StateDirectory=dolt

[Install]
WantedBy=multi-user.target

doltsrht binds all three listeners (web -b localhost:5307, remotesapi 127.0.0.1:5306, credentials 127.0.0.1:5308) and shuts them down cleanly on SIGINT/SIGTERM. Point [dolt.sr.ht]repos at the StateDirectory (/var/lib/dolt).

#User quickstart

Once the service is up and you are logged into meta:

1. Create a database. Visit https://dolt.srht.bigb.es, click Create, pick a name and visibility (PUBLIC / UNLISTED / PRIVATE). It appears at ~<you>/<name>.

2. Clone/push with a meta personal access token (PAT):

   export DOLT_REMOTE_PASSWORD=<your meta PAT>
   dolt clone --user <you> https://dolt.srht.bigb.es/~<you>/<name>
   cd <name>
   dolt sql -q 'CREATE TABLE t (id INT PRIMARY KEY)'
   dolt add . && dolt commit -m 'init'
   dolt push origin main
   

   --user selects HTTP Basic auth; the password comes from DOLT_REMOTE_PASSWORD. A PAT with the dolt.sr.ht/repos:RW grant (or an empty-grant personal token) can push; :RO or no grant can only read.

3. Or clone/push with a dolt keypair (the git-SSH-key-like UX):

   dolt creds new          # generate an Ed25519 keypair
   dolt login --auth-endpoint dolt.srht.bigb.es:443 \
              --login-url https://dolt.srht.bigb.es/settings/keys
   

   dolt login opens the settings page with your public key in the URL fragment; associate it with your account, and dolt login confirms via the CredentialsService. Thereafter plain dolt clone/push (no --user, no env var) works. To make those the defaults so you can drop the flags:

   dolt config --global --add remotes.default_host dolt.srht.bigb.es
   dolt config --global --add creds.add_url https://dolt.srht.bigb.es/settings/keys
   

Anonymous dolt clone works for PUBLIC and UNLISTED databases with no credentials at all; PRIVATE databases return "not found" to unauthorized callers (their existence is not leaked).