# dolt.sr.ht

A self-hosted [Dolt](https://www.dolthub.com/) database hosting service for a
SourceHut instance — "DoltLab for SourceHut". It hosts Dolt databases the way
git.sr.ht hosts git repos: `dolt clone`/`push`/`pull` over HTTPS plus an
integrated web UI that shares the SourceHut nav, unified login, and Bootstrap
theme.

Pure Go, one module (`go.bigb.es/sourcehut-dolt`), no upstream SourceHut
modification — integration is config-driven: a `[dolt.sr.ht]` section in the
shared instance `config.ini` puts the service into every other service's nav.

Storage is bare NBS chunk-store directories (no `.dolt/`, no working set) at
`<repos>/~<user>/<name>` — exactly what `remotesrv` serves and `file://`
remotes use. PostgreSQL holds metadata (a mirror of meta's users, plus
repositories, ACLs, and dolt keys). Clone URL:
`dolt clone https://dolt.srht.bigb.es/~user/db`.

Two auth flows are supported: username + meta personal access token
(`--user` + `DOLT_REMOTE_PASSWORD`, HTTP Basic) and dolt's Ed25519 keypair flow
(`dolt creds` / `dolt login`, Bearer EdDSA JWT — the git-SSH-key-like UX).

## Status

Phase 0 (foundation) only: the pure `core/` domain (name/path validation and
the access matrix), the SQL schema and brant migration, config/nginx/Makefile
scaffolding, and the de-risking storage spike. The service binaries, database
layer, auth, remotesapi, and web UI land in later phases.

## Build prerequisites

- **Go 1.26+**
- **A C toolchain** — `github.com/dolthub/dolt/go` uses CGO for
  [`gozstd`](https://github.com/valyala/gozstd) and
  [`go-icu-regex`](https://github.com/dolthub/go-icu-regex).
- **ICU4C development headers** — required by `go-icu-regex`.
  - Debian/Ubuntu: `apt install libicu-dev` (headers on the default path).
  - macOS (Homebrew): `brew install icu4c` installs a keg-only, versioned
    formula. Point CGO at it, e.g. for `icu4c@78`:

    ```sh
    export CGO_CPPFLAGS="-I/opt/homebrew/opt/icu4c@78/include"
    export CGO_LDFLAGS="-L/opt/homebrew/opt/icu4c@78/lib"
    ```

- **sassc + minify** — only for building CSS (`make css`); not needed for the
  default build:

  ```sh
  brew install sassc                                   # or apt install sassc
  go install github.com/tdewolff/minify/v2/cmd/minify@latest
  ```

  CSS is compiled against the shared sourcehut SCSS: `make css ASSETS=/path/to/sourcehut/scss/parent`
  (`ASSETS` defaults to `/usr/share/sourcehut`; `sassc` is invoked with
  `-I $(ASSETS)/scss`).

## Pinned dependencies (and why)

- **`git.sr.ht/~sircmpwn/core-go` v0.0.0-20260520082310-fdb3662452dc** — this
  exact pseudo-version matches the production instance's `core-go` submodule
  commit (`fdb3662`). **Never upgrade it** (no bare `go get -u`, no `@latest`);
  token validation, config, and crypto must behave identically to the rest of
  the instance.
- **`github.com/dolthub/dolt/go` v0.40.5-0.20260626152440-45335d44ad79** — a
  pseudo-version pinned to the commit tagged **v2.1.10** (`45335d44`), the dolt
  CLI version installed on the target host (`/opt/homebrew/bin/dolt`, v2.1.10).
  The dolt `/go` submodule's latest *tag* is the stale `v0.40.4` (2021), which
  does **not** interop with a modern CLI; matching the CLI's commit guarantees a
  common NBS storage format (`types.Format_DOLT` / `__DOLT__`) and remotesapi
  proto. Verified end-to-end by the Phase-0 spike (see below). If the CLI is
  upgraded, re-pin `dolt/go` to the new CLI's commit and re-run the spike.
- **`gopkg.in/go-jose/go-jose.v2` v2.6.3** — the same JOSE major/version that
  `dolt/go`'s `creds` package uses to sign the EdDSA keypair JWTs, so the Bearer
  verify path stays byte-compatible and no duplicate JOSE lib is pulled in.
- grpc v1.79.3, logrus v1.8.3, lib/pq v1.10.9, chi/v5, and brant round out the
  transport, logging, Postgres driver, HTTP router, and migration tooling.

## The spike

`storage/spike_test.go` (build tag `spike`) is the Phase-0 de-risk gate. It
inits a bare NBS store via `doltdb.LoadDoltDB` + `WriteEmptyRepo`, serves it
with `remotesrv.NewServer` on an ephemeral localhost port (single-port
http+gRPC multiplex, no auth), then drives the real `dolt` CLI through a full
round-trip: clone → create table + insert → commit → push → fresh re-clone →
verify the rows. It skips (does not fail) when the CLI is absent, and uses an
isolated `$HOME` so your real dolt config is untouched.

```sh
export CGO_CPPFLAGS="-I/opt/homebrew/opt/icu4c@78/include"
export CGO_LDFLAGS="-L/opt/homebrew/opt/icu4c@78/lib"
go test -tags spike ./storage/ -run TestSpike -v
```

## Dev commands

```sh
go build ./...                 # build every package (needs the CGO env above)
go test ./core/...             # pure-domain unit tests
go test -tags spike ./storage/ -v   # the interop spike
make                           # build the service binaries (no-ops until Phase 3)
make css                       # build stylesheets (needs sassc + minify)
```

## Deployment sketch

1. Add a `[dolt.sr.ht]` section to the shared instance `config.ini` (see
   `config.example.ini`) plus an `origin=` line so the service appears in every
   other service's nav. This host **must** be inside meta's `internal-ipnet`
   (token validation uses internal auth against meta).
2. Create the database and run migrations: `doltsrht-migrate` applies
   `migrations/*.sql` (brant format). `schema.sql` is the full init DDL.
3. Install the nginx site config (`contrib/dolt.sr.ht.conf`) and add a
   `dolt.srht.bigb.es` DNS record. nginx path-routes the remotesapi gRPC, the
   sealed-URL chunk data plane, the CredentialsService, and the web UI to
   separate listeners.
4. Start `doltsrht`.