5 files changed, 531 insertions(+), 85 deletions(-)

M .build.yml
M .gitignore
M APKBUILD
M Makefile
A docs/ci.md

@@ 1,3 1,11 @@
+# builds.sr.ht manifest for dolt.sr.ht. One linear pipeline: install the cache
+# helper, assemble the shared SCSS, restore caches, package with abuild, publish
+# the apk, save caches.
+#
+# The reasoning behind every task lives in docs/ci.md, not here: builds.sr.ht
+# stores the submitted manifest in a varchar(16384), so a manifest over 16 KiB
+# cannot be submitted at all — and the failure is a branch with no CI, not a red
+# build. Add paragraphs to docs/ci.md and a pointer here.
 image: alpine/edge
 packages:
   - abuild


@@ 24,24 32,23 @@ environment:
   S3_ENDPOINT: https://s3.bigb.es
   # Must track the srht deployment's SRHT_CORE_VER, or this service's theme
   # drifts from the rest of the instance. BOOTSTRAP_REV is the submodule commit
-  # core.sr.ht pins at that tag; bump the two together.
+  # core.sr.ht pins at that tag; bump the two together. docs/ci.md#environment.
   CORE_VER: "0.84.5"
   BOOTSTRAP_REV: 779ad9f174ea5ab7e755f6df0ec9e5912d67dd16
 submitter:
   git.sr.ht:
     allow-refs:
       - refs/heads/master
+      # Tags build too, now that the version task reads them: pushing v0.2.0 is
+      # what produces the 0.2.0 apk. See docs/ci.md#version.
+      - "refs/tags/v*"
 tasks:
-  - cacher: |
-      # S3-backed CI cache helper (go.bigb.es/cacher), dogfooded from its own
-      # published release — the same bootstrap the bencher/ci-cacher builds use.
-      # First task, so the scss assembly below can already use the cache.
-      mkdir -p ~/.local/bin
-      curl -sSL "https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64" \
-        -o ~/.local/bin/cacher
-      chmod +x ~/.local/bin/cacher
-      echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.buildenv
-      export PATH="$HOME/.local/bin:$PATH"
+  # S3-backed CI cache helper; installed first so scss can already use it.
+  # install.sh's PATH export goes to ~/.buildenv, which only the NEXT task
+  # sources — hence two tasks and not one. See docs/ci.md#cacher.
+  - cacher_install: |
+      curl -fsSL https://bigbes.pages.srht.bigb.es/ci-cacher/install.sh | sh
+  - cacher_init: |
       cacher init \
         --endpoint    https://s3.bigb.es \
         --region      garage \


@@ 50,14 57,12 @@ tasks:
         --key-file    ~/.s3-cache-key-id \
         --secret-file ~/.s3-cache-key-secret
   - scss: |
-      # No apk ships the shared sourcehut SCSS partials, so assemble them the
-      # way core.sr.ht's `make install-scss` would: its own scss/ plus the
-      # Bootstrap 4 submodule. `make css` runs sassc -I against this tree.
-      # The assembled tree is cached keyed by the two pins — on a hit this
-      # task touches neither git.sr.ht nor github.com, so their outages can't
-      # fail the build.
-      KEY_SCSS="scss/${CORE_VER}-${BOOTSTRAP_REV}.tar.zst"
-      if ! cacher dir download "$KEY_SCSS" ~/scss; then
+      # Assemble the shared sourcehut partials no apk ships, the way
+      # core.sr.ht's `make install-scss` would, cached by the two pins so an
+      # outage at git.sr.ht or github.com can't fail us. --exec runs on a miss
+      # and seeds the cache after; it sees exported vars only, hence the inline
+      # key and the single quotes. See docs/ci.md#scss.
+      cacher dir download "scss/${CORE_VER}-${BOOTSTRAP_REV}.tar.zst" ~/scss --exec '
         git clone --depth 1 --branch "$CORE_VER" \
           https://git.sr.ht/~sircmpwn/core.sr.ht /tmp/core
         mkdir -p ~/scss/bootstrap


@@ 67,52 72,75 @@ tasks:
         git -C /tmp/bootstrap fetch -q --depth 1 origin "$BOOTSTRAP_REV"
         git -C /tmp/bootstrap checkout -q FETCH_HEAD
         cp -r /tmp/bootstrap/scss ~/scss/bootstrap/scss
-        cacher dir upload "$KEY_SCSS" ~/scss
-      fi
+      '
       sudo mkdir -p /usr/share/sourcehut
       sudo cp -r ~/scss /usr/share/sourcehut/scss
   - keygen: |
-      # abuild insists on signing what it builds, but this key is deliberately
-      # throwaway: generated per build, dies with the VM, trusted by nothing.
-      # Clients verify against the index instead, which is rebuilt and signed on
-      # phoebe by the garage stack's apk-mirror service — it indexes this repo
-      # with --allow-untrusted precisely because of this.
-      #
-      # -i installs the public half into /etc/apk/keys. Without it abuild's own
-      # final "update the local repository index" step dies with UNTRUSTED
-      # signature, after having built the package perfectly well.
+      # Throwaway signing key, and -i is not optional: docs/ci.md#keygen.
       SUDO=sudo abuild-keygen -a -n -i -q
   - version: |
+      # ONE `git describe` decides the apk pkgver: a tag, else tag_git<n>, else
+      # the family's commit count. The raw describe output is not a legal pkgver
+      # and _git sorts AFTER the release: docs/ci.md#version.
+      #
+      # EXPORTED rather than sed-ed into the APKBUILD (which reads $PKGVER),
+      # because rewriting a tracked file would flip the VCS stamp Go records in
+      # every binary to dirty — do not "tidy" it back into a sed. The tree is
+      # printed because this is the last moment it is provably clean.
       cd "$REPO"
-      ver="0.0.$(git rev-list --count HEAD)"
-      sed -i "s/^pkgver=.*/pkgver=$ver/" APKBUILD
+      desc=$(git describe --tags --always --dirty)
+      base=${desc%-dirty}
+      case "$base" in
+        v*-g*) n=${base%-g*}; ver="${n%-*}"; ver="${ver#v}_git${n##*-}" ;;
+        v*)    ver="${base#v}" ;;
+        *)     ver="0.0.$(git rev-list --count HEAD)" ;;
+      esac
       echo "export PKGVER=$ver" >> ~/.buildenv
-      echo "building $ver"
+      echo "building $ver from $desc"
+      git status --porcelain
   - cache_restore: |
-      # Restore the Go module and build caches, both keyed by go.sum: the dolt
-      # dependency tree dominates compile time, and it only changes when go.sum
-      # does. A miss is just a cold build, never an error. abuild's `go build`
-      # runs as this same user, so the default ~/go/pkg/mod and ~/.cache/go-build
-      # locations are picked up without touching the APKBUILD.
+      # Go module and build caches keyed by go.sum: the dolt dependency tree
+      # dominates compile time and only changes when go.sum does. --optional
+      # makes a miss a cold build, not an error. abuild re-pins both, since it
+      # redirects the Go caches into its own $tmpdir (docs/ci.md#cache_restore).
       KEY_MOD=$(cacher key "gomod/{hash}.tar.zst" --hash-from "$REPO/go.sum")
       KEY_GOC=$(cacher key "gocache/{hash}.tar.zst" --hash-from "$REPO/go.sum")
       echo "export KEY_MOD=$KEY_MOD KEY_GOC=$KEY_GOC" >> ~/.buildenv
-      # abuild redirects the Go caches into its throwaway $tmpdir (and an
-      # upstream typo slaves GOMODCACHE to GOCACHE), so env exports here can't
-      # stick — the APKBUILD's build() re-pins both to these home locations.
-      cacher dir download "$KEY_MOD" ~/go/pkg/mod || true
-      cacher dir download "$KEY_GOC" ~/.cache/go-build || true
+      cacher dir download "$KEY_MOD" ~/go/pkg/mod        --optional
+      cacher dir download "$KEY_GOC" ~/.cache/go-build   --optional
+      # Repair block for the HALF-restored module cache — the normal failure
+      # here, not a freak one, and it reads like a code bug. Do not remove and
+      # do not soften to `|| true`:
+      # docs/ci.md#the-half-restored-module-cache.
+      cd "$REPO"
+      chmod -R u+w ~/go/pkg/mod 2>/dev/null || true
+      if ! go mod verify >/dev/null 2>&1; then
+        echo "restored module cache did not verify — discarding it"
+        rm -rf ~/go/pkg/mod
+      fi
+      # `go mod download`, NOT `go mod download all`: `all` resolves the whole
+      # module graph including dependencies' test deps and APPENDS their hashes
+      # to go.sum, which is a modified tracked file and therefore a "-dirty"
+      # apk. Measured. docs/ci.md#go-mod-download-and-the-word-all.
+      go mod download
+      go mod verify
+      # Neither line above may rewrite go.mod or go.sum — a dirty tree here is a
+      # "-dirty" apk, so check rather than trust. docs/ci.md#cache_restore.
+      git status --porcelain
   - build: |
       cd "$REPO"
       # -d: makedepends are already installed via `packages:` above.
       REPODEST=$HOME/packages abuild -d
       find "$HOME/packages" -name '*.apk'
-  - cache_save: |
-      # Seed the caches only when this go.sum has no entry yet — on a hit the
-      # tarballs are already up there and re-uploading identical bytes is waste.
-      cacher exists "$KEY_MOD" || cacher dir upload "$KEY_MOD" ~/go/pkg/mod
-      cacher exists "$KEY_GOC" || cacher dir upload "$KEY_GOC" ~/.cache/go-build
   - publish: |
+      # The gate is the honest answer to a build that was handed no secrets, not
+      # a fallback: with ~/.apk-ci.env absent every earlier task has still run
+      # and a signed apk is sitting in $HOME/packages. See docs/ci.md#publish.
+      if [ ! -r ~/.apk-ci.env ]; then
+        echo "no ~/.apk-ci.env: this build has no apk repo credentials"
+        echo "the package was built and signed, and is not published"
+        exit 0
+      fi
       set +x   # never echo the S3 credentials into the build log
       . ~/.apk-ci.env
       export RCLONE_CONFIG_GARAGE_TYPE=s3


@@ 123,10 151,15 @@ tasks:
       export RCLONE_CONFIG_GARAGE_ACCESS_KEY_ID="$APK_CI_S3_ACCESS_KEY"
       export RCLONE_CONFIG_GARAGE_SECRET_ACCESS_KEY="$APK_CI_S3_SECRET_KEY"
       set -x
-      # Upload only; never delete. Old versions stay so a pinned deployment can
-      # always be rebuilt — the same reason the upstream mirror is append-only.
+      # Upload only, never delete. See docs/ci.md#publish.
       find "$HOME/packages" -name '*.apk' -print | while read -r f; do
         rclone copyto "$f" "garage:$S3_BUCKET/$APK_REPO/$(basename "$f")"
         echo "uploaded $(basename "$f")"
       done
       echo "published; apk-mirror on phoebe re-indexes within 15 minutes"
+  - cache_save: |
+      # AFTER publish so an S3 hiccup cannot strand a good apk, and fatal on
+      # purpose. Without --force an upload skips a key already there, so no
+      # `cacher exists ||` guard is needed. See docs/ci.md#cache_save.
+      cacher dir upload "$KEY_MOD" ~/go/pkg/mod
+      cacher dir upload "$KEY_GOC" ~/.cache/go-build

@@ 8,6 8,28 @@
 /static/main.min.css
 /static/main.min.*.css
 
+# The three scratch directories abuild works in. With source="" and
+# builddir="$startdir" (see APKBUILD) it has no tarball to unpack elsewhere: it
+# unpacks into ./src, stages the package into ./pkg, and defaults tmpdir to
+# ./tmp — all three inside the very checkout it is packaging.
+#
+# Listing them is not tidiness. Go records vcs.modified in every binary built
+# inside a repository and reads that flag from `git status --porcelain`, which
+# counts UNTRACKED files — so an untracked directory left beside the build makes
+# the packaged binary stamp itself "<sha>-dirty" for the life of the apk. That
+# is the same failure the APKBUILD's pkgver comment is about, arriving by the
+# other door: there a tracked file was rewritten, here an untracked one appears.
+#
+# /tmp/ is the one that is easy to miss and the one that bites hardest. abuild
+# exports GOTMPDIR to it, so every `go build` puts its work directory inside
+# this checkout, and /usr/share/abuild/default.conf exports MAKEFLAGS=-j$(nproc)
+# — under which `make all-bin`'s three `go build`s run at the same time. Each
+# reads the tree once while loading packages, before it writes anything, so its
+# own work directory is still empty and unreported; the other two are not.
+/src/
+/pkg/
+/tmp/
+
 # Git worktrees, kept inside the repo so they never scatter as sibling dirs.
 # Both paths, because the two conventions in use across these repos are
 # .worktrees/<branch> and the older .claude/worktrees/<branch>.

@@ 4,10 4,21 @@
 # repo.bigb.es/alpine/v3.22/bigbes. The srht deployment installs it from there
 # instead of cloning and compiling this repo inside its Dockerfile.
 #
-# pkgver is rewritten by CI to 0.0.<commit count> before abuild runs — a
-# monotonic, unique-per-commit version that the deployment can pin.
+# pkgver is READ FROM THE ENVIRONMENT and must stay that way. CI used to `sed`
+# the literal below in place before calling abuild; that rewrites a tracked file
+# inside the checkout, and Go decides half of the VCS stamp it puts into every
+# binary — vcs.modified — from exactly that. Measured on go1.26.5: one `sed` of
+# APKBUILD makes every binary abuild compiles afterwards record itself as
+# "<sha>-dirty" for the life of the apk. The export cannot do that. .gitignore
+# covers the rest of what abuild writes into this directory.
+#
+# .build.yml's version task exports PKGVER out of a single `git describe`: a tag
+# (v0.2.0 -> 0.2.0), a tag plus commits since it (0.2.0_git7, which sorts AFTER
+# the release in Alpine's grammar), or 0.0.<commit count> when there is no tag —
+# which is where this repository stands today. A local `abuild` has no PKGVER
+# and builds 0.0.0, which is what a package built by hand honestly is.
 pkgname=dolt.sr.ht
-pkgver=0.0.0
+pkgver="${PKGVER:-0.0.0}"
 pkgrel=0
 pkgdesc="Dolt database hosting for a sourcehut instance"
 url="https://sourcecraft.dev/bigbes/sr-ht-dolt"


@@ 43,16 54,22 @@ build() {
 	# the default path because sassc/minify aren't always on a dev machine. So
 	# `css` has to be asked for by name; building `all` alone silently produces
 	# a package with no stylesheet.
-	CGO_ENABLED=0 make all-bin GO_TAGS="$GO_TAGS"
+	#
+	# -trimpath so the binaries do not carry this builder's directory layout,
+	# and -modcacherw for a reason beyond tidiness: without it the module cache
+	# is left read-only, and the CI cache tarball made from it cannot be
+	# unpacked on the next build (mkdir into 0555 dirs fails).
+	CGO_ENABLED=0 make all-bin GO_TAGS="$GO_TAGS" GOFLAGS="-trimpath -modcacherw"
 	# Needs the shared sourcehut scss partials pre-assembled at ASSETS/scss (no
 	# apk ships them); CI does that before calling abuild.
 	make css ASSETS=/usr/share/sourcehut
 
 	# install-share copies static/*.css under `2>/dev/null || true`, so a
 	# stylesheet that never got built would ship as an unstyled service without
-	# failing anything. Assert it exists instead. The hashed copy is the one the
-	# service finds by globbing its static dir at startup.
-	ls static/main.min.*.css >/dev/null
+	# failing anything. Assert instead. `make check-css` and not the `ls` that
+	# stood here: two hashed stylesheets are as wrong as none and quieter, since
+	# web/router.go resolves the glob and takes the first match.
+	make check-css
 }
 
 package() {


@@ 60,7 77,17 @@ package() {
 	# This Makefile's install rules do not honour DESTDIR (they write to
 	# $(BINDIR)/$(SHAREDIR) directly), so the staging dir is passed as PREFIX.
 	# MIGRATIONDIR and STATICDIR are both derived from SHAREDIR, so they follow.
-	make install PREFIX="$pkgdir/usr"
+	#
+	# `install-files` and not `install`, and no CGO_ENABLED or GOFLAGS beside it,
+	# because this function must not compile anything. abuild runs package() in a
+	# FRESH abuild process under fakeroot (abuild.in's rootpkg), which re-sources
+	# this file and never calls build(): the cache pins above are gone here and
+	# cannot be restored — abuild's own GOMODCACHE line overwrites them — so a
+	# `make install` staged a second compilation of all three binaries, made from
+	# a cold module cache and with none of the flags above, and it was that copy
+	# that went into the apk while build()'s checks were left behind with the
+	# copy nobody shipped.
+	make install-files PREFIX="$pkgdir/usr"
 }
 
 hook() {

@@ 23,7 23,17 @@ GO_TAGS?=gms_pure_go
 CGO_ENABLED?=0
 export CGO_ENABLED
 GO=go
-GOBUILD=$(GO) build $(if $(GO_TAGS),-tags "$(GO_TAGS)",)
+# The tag flag is a variable of its own because `go build` is not the only
+# command that needs it: `go vet` and `go test` type-check the same tree, and a
+# vet or a test run without gms_pure_go pulls go-icu-regex in and wants the ICU
+# headers the pure-Go build exists to avoid.
+GO_TAGSFLAG=$(if $(GO_TAGS),-tags "$(GO_TAGS)",)
+GOBUILD=$(GO) build $(GO_TAGSFLAG)
+
+# The one stylesheet the service looks for at startup: web/router.go resolves
+# `main.min.*.css` in its static dir by glob. Named here because check-css and
+# clean-share both have to mean exactly the same set of files as that glob.
+CSS=static/main.min.*.css
 
 BINARIES=\
 	doltsrht \


@@ 41,42 51,76 @@ all-share: static/main.min.css
 
 css: all-share
 
-# Build each binary if its cmd package exists yet. The cmd/ packages land in
-# Phase 3; until then these targets are no-ops rather than hard failures.
+# One line each, and no `if [ -d ./cmd/... ]` in front of it. The guard dated
+# from Phase 2, when cmd/ did not exist yet and a missing package had to be a
+# no-op rather than a hard failure; cmd/ has held all three since Phase 3. What
+# the guard does now is turn a deleted or renamed cmd/ directory into a GREEN
+# `make all-bin` that produced no binary at all — and, since abuild packages
+# whatever is on disk, into an apk quietly missing a program.
 doltsrht:
-	@if [ -d ./cmd/doltsrht ]; then \
-		echo "$(GOBUILD) -o $@ ./cmd/doltsrht"; \
-		$(GOBUILD) -o $@ ./cmd/doltsrht; \
-	else \
-		echo "skip $@: ./cmd/doltsrht not present yet"; \
-	fi
+	$(GOBUILD) -o $@ ./cmd/doltsrht
 
 doltsrht-migrate:
-	@if [ -d ./cmd/doltsrht-migrate ]; then \
-		echo "$(GOBUILD) -o $@ ./cmd/doltsrht-migrate"; \
-		$(GOBUILD) -o $@ ./cmd/doltsrht-migrate; \
-	else \
-		echo "skip $@: ./cmd/doltsrht-migrate not present yet"; \
-	fi
+	$(GOBUILD) -o $@ ./cmd/doltsrht-migrate
 
 dolt-git-hook:
-	@if [ -d ./cmd/dolt-git-hook ]; then \
-		echo "$(GOBUILD) -o $@ ./cmd/dolt-git-hook"; \
-		$(GOBUILD) -o $@ ./cmd/dolt-git-hook; \
-	else \
-		echo "skip $@: ./cmd/dolt-git-hook not present yet"; \
-	fi
+	$(GOBUILD) -o $@ ./cmd/dolt-git-hook
 
 # Compile every buildable package; used as the CI build gate.
 build:
 	$(GOBUILD) ./...
 
-install: install-bin install-share
-
-install-bin: all-bin
+# `go vet` and `go test` over the whole module, both with the pure-Go tag, so
+# that CI can name them without repeating CGO_ENABLED and GO_TAGS in the
+# manifest — a second copy of those two is a second copy to forget. CGO_ENABLED
+# is exported above, so both inherit it.
+#
+# Neither is gated on a DSN: db/'s suites skip themselves when DOLTSRHT_TEST_PG
+# is unset, which is what makes `make test` useful on a laptop. The refusal to
+# accept that in CI belongs to the manifest's test task, which knows it asked
+# for a Postgres.
+vet:
+	$(GO) vet $(GO_TAGSFLAG) ./...
+
+test:
+	$(GO) test $(GO_TAGSFLAG) ./...
+
+# `install` still means "build it, check it, then copy it", which is what a
+# person at a checkout wants. The build is a prerequisite and install-files is
+# invoked from the recipe rather than listed as a third prerequisite:
+# /usr/share/abuild/default.conf exports MAKEFLAGS=-j$(nproc), prerequisites of
+# one target run in parallel under -j, and `install: check-css all-bin
+# install-files` would let the copying start beside the build it is meant to
+# follow. A recipe line always runs after the prerequisites are done.
+install: check-css all-bin
+	@$(MAKE) install-files
+
+# The copying half of `install`, with nothing to build in front of it. This is
+# the target a packaging run calls once it has already built and checked, so
+# that what it stages is the very bytes it checked.
+#
+# That distinction is the whole point of the split. The binary targets above are
+# .PHONY (Go decides staleness itself), and abuild runs package() in a FRESH
+# abuild process under fakeroot which re-sources the APKBUILD and never calls
+# build() — so nothing build() exported reaches it, the CI cache pins included.
+# An `install` there recompiled all three binaries from a cold module cache and
+# packaged that second compilation, which is not the one anything had inspected.
+install-files: install-bin install-share
+
+# No `if [ -x $$bin ]` around the copy, deliberately, and this is the half of
+# the split that makes it safe: now that this target no longer builds, the one
+# way to reach it with a missing binary is to call it before a build, and
+# `install -Dm755 doltsrht` on a missing file is already a fatal error naming
+# the file. The old guard would instead have staged whichever binaries happened
+# to be there and returned 0.
+#
+# `|| exit 1` because a `for` loop reports only its LAST command: without it a
+# missing doltsrht followed by two binaries that copied fine is a green
+# install-bin, which is the same silence spelled differently.
+install-bin:
 	mkdir -p $(BINDIR)
 	for bin in $(BINARIES); do \
-		if [ -x $$bin ]; then install -Dm755 $$bin $(BINDIR)/; fi; \
+		install -Dm755 $$bin $(BINDIR)/ || exit 1; \
 	done
 
 install-share:


@@ 86,15 130,43 @@ install-share:
 	install -Dm644 migrations/*.sql $(MIGRATIONDIR)
 	if [ -d static ]; then install -Dm644 static/*.css static/*.svg $(STATICDIR) 2>/dev/null || true; fi
 
+# The packaging gate: no stylesheet, no package. It exists because the failure
+# it catches is invisible at build time — install-share copies static/*.css
+# under `2>/dev/null || true`, so a `make css` that produced nothing stages a
+# service with no stylesheet and fails nothing; the first sign of trouble is an
+# unstyled page in production.
+#
+# It counts the matches rather than asking whether there are any, because TWO
+# stylesheets are as wrong as none and quieter: web/router.go resolves $(CSS) by
+# glob and takes the first match, so a second file makes the served stylesheet
+# depend on readdir order. `css` does not remove the previous build's hashed
+# file, which is exactly why this must not assume it did.
+#
+# `set --` puts the matches in the positional parameters, so the count is $$#
+# and no `wc` output has to be parsed. Splitting on whitespace is fine: every
+# name it can see was produced by the recipe below, out of a hex digest.
+check-css:
+	@set -- $$(ls $(CSS) 2>/dev/null); \
+	if [ $$# -eq 0 ]; then \
+		echo "error: no $(CSS) — run 'make css' (sassc and minify required)"; \
+		exit 1; \
+	elif [ $$# -gt 1 ]; then \
+		echo "error: $$# files match $(CSS) — web/ takes the first, so the choice is arbitrary."; \
+		echo "       run 'make clean-share && make css' to get back to one:"; \
+		for f in "$$@"; do echo "         $$f"; done; \
+		exit 1; \
+	fi
+
 clean: clean-bin clean-share
 
 clean-bin:
 	rm -f $(BINARIES)
 
 clean-share:
-	rm -f static/main.min.css static/main.css static/main.min.*.css
+	rm -f static/main.min.css static/main.css $(CSS)
 
-.PHONY: all all-bin all-share css build install install-bin install-share
+.PHONY: all all-bin all-share css check-css build vet test
+.PHONY: install install-files install-bin install-share
 .PHONY: clean clean-bin clean-share $(BINARIES)
 
 static/main.css: scss/main.scss

@@ 0,0 1,292 @@
+# CI: what `.build.yml` does and why
+
+The manifest is deliberately short on prose. builds.sr.ht stores a submitted
+manifest in a `varchar(16384)`, so a manifest that grows past 16 KiB cannot be
+submitted at all — the failure is at submission time and reads like nothing in
+particular. Rationale therefore lives here, and the manifest carries pointers.
+
+The pipeline is one linear job on `alpine/edge`: install the cache helper,
+assemble the shared SCSS, decide a version, restore caches, package with
+`abuild`, publish the apk to `repo.bigb.es/alpine/v3.22/bigbes`, save caches.
+
+It is triggered by a push to the **sourcehut** side. A push to sourcecraft
+cannot reach builds.sr.ht; the gitsync mirror is what puts the commit on
+`git.srht.bigb.es`, and that push is what submits the job. So the mirror is on
+the critical path for the package repository, not merely an offsite copy.
+
+## packages
+
+`sassc` and `minify` build the stylesheet; `curl` installs `cacher`; `rclone`
+publishes the apk; `abuild`, `go` and `git` are the build itself.
+
+There is no `postgresql` here and no compiler toolchain beyond `go`: this
+package is built `CGO_ENABLED=0` with `-tags gms_pure_go` (see the APKBUILD and
+the Makefile), which is what keeps `libicu` and `gozstd` off the builder
+entirely. A default cgo build of this tree fails on missing ICU headers and
+always has.
+
+## secrets
+
+Three, all account-level and shared with the sibling services:
+
+| secret | lands at | used by |
+|---|---|---|
+| `apk-ci-s3` | `~/.apk-ci.env` | `publish` |
+| `7dde4219-…` | `~/.s3-cache-key-id` | `cacher_init` |
+| `0e5b3530-…` | `~/.s3-cache-key-secret` | `cacher_init` |
+
+They are file secrets. Listing them is what turns `publish` and the cache tasks
+on; a manual submission that asks for no secrets still runs the interesting part
+of the pipeline and stops at `cacher_init`, which is the right place to notice.
+
+`apk-ci-s3` is referenced by name and the other two by UUID, which is only
+because that is how they were written in the donor manifests; both forms work.
+
+## environment
+
+`CORE_VER` must track the deployment's `SRHT_CORE_VER`, and `BOOTSTRAP_REV` is
+the Bootstrap 4 submodule commit `core.sr.ht` pins at that tag. The two being
+out of step means this service renders against different partials than the rest
+of the instance, which shows up as a page that is subtly the wrong shape and as
+nothing at all in any log. Bump them together.
+
+## cacher
+
+`cacher` is an S3-backed cache helper, installed from pages.sr.ht.
+
+It is **two tasks and not one** because `install.sh` appends its `PATH` export
+to `~/.buildenv`, and `~/.buildenv` is sourced by the *next* task's preamble — a
+single task would install it and then not find it.
+
+It is `install.sh` and not a raw `curl` of the binary, for two reasons that both
+end in the same place. The installer verifies the download against the published
+`checksums.txt`; and the raw `curl` it replaced had no `-f`, so an S3 404 or a
+proxy error page was written to `~/.local/bin/cacher`, `chmod +x`-ed, and only
+noticed as a baffling "not found"/"exec format error" one task later.
+
+## scss
+
+`sassc` needs the shared sourcehut partials, and no apk ships them: they are
+`core.sr.ht`'s own `scss/` plus the Bootstrap 4 tree it pins as a submodule. The
+task clones both and drops them in `/usr/share/sourcehut/scss`, cached under a
+key made of the two pins, so an outage at git.sr.ht or GitHub cannot fail a
+build that changed nothing.
+
+The `--exec` block runs only on a cache miss, and `cacher` seeds the cache from
+the result afterwards. It sees exported variables only, which is why the cache
+key is spelled inline in the `cacher` invocation rather than computed inside the
+block.
+
+`--exec` and not the `if ! cacher dir download …; then … fi` this task used to
+be. The `if` form conflates two different failures into one branch: a genuine S3
+error and an ordinary cache miss both mean "clone it again", so an S3 outage was
+silently absorbed into a slower build and never reported. And it made the seed
+upload the last command of the branch, i.e. fatal — an S3 hiccup while *storing*
+a cache would fail a build whose SCSS had assembled perfectly. Under `--exec`
+the seeding is best-effort by construction and the download's own errors stay
+distinguishable from a miss.
+
+## keygen
+
+`abuild-keygen -a -n -i -q` makes a throwaway package signing key. `-i` is not
+optional: without it the key is generated but not installed into
+`/etc/apk/keys`, and `abuild` later fails to index its own output with an
+UNTRUSTED signature error, after having built the package perfectly well.
+
+The key is per-build and thrown away with the VM. That is fine because nothing
+verifies these signatures — the apk index on `repo.bigb.es` is signed by
+`apk-mirror` on phoebe, which indexes this repo with `--allow-untrusted`
+precisely because of this.
+
+## version
+
+One `git describe` decides the apk's `pkgver`:
+
+- a tag — `v0.2.0` becomes `0.2.0`;
+- a tag plus commits since it — `v0.2.0-7-gabc1234` becomes `0.2.0_git7`. Alpine's
+  version grammar does not accept the raw describe output, and `_git` sorts
+  **after** the release in Alpine's comparison, which is what makes an untagged
+  master build look newer than the tag it follows;
+- no tag at all — `0.0.<commit count>`, which is where this repository stands
+  today and is the scheme of the bench/spec/compare siblings.
+
+Tags are in `allow-refs` for exactly this reason: pushing `v0.2.0` is what
+produces the `0.2.0` apk.
+
+### Why it is exported and not `sed`-ed
+
+`PKGVER` reaches `abuild` through `~/.buildenv`, and `APKBUILD` reads
+`pkgver="${PKGVER:-0.0.0}"`. It used to `sed` the literal in the tracked
+`APKBUILD` instead. **Do not tidy it back.**
+
+Go records a VCS stamp in every binary it builds inside a repository, and it
+decides the `vcs.modified` half of that stamp from `git status --porcelain`.
+Rewriting a tracked file in the checkout sets it, and it stays set for the whole
+`abuild` run — so every binary in the apk records itself as `<sha>-dirty`.
+Measured on go1.26.5.
+
+The same flag is why `.gitignore` lists `/src/`, `/pkg/` and `/tmp/`: `git
+status --porcelain` counts **untracked** files too, and abuild — which works in
+this checkout, since `APKBUILD` sets `builddir="$startdir"` and has no tarball
+to unpack — creates all three inside it. `/tmp/` is the one that is easy to
+miss: abuild defaults `tmpdir` to `$startdir/tmp` and exports `GOTMPDIR` to it,
+and `/usr/share/abuild/default.conf` exports `MAKEFLAGS=-j$(nproc)`, under which
+`make all-bin`'s three `go build`s run at once and each sees the others' work
+directories.
+
+The task ends with `git status --porcelain` because this is the last moment the
+tree is provably clean, and the print costs nothing.
+
+## cache_restore
+
+The Go module cache and build cache are keyed by the hash of `go.sum`, so a
+build that changed no dependency reuses both. That matters more here than in the
+siblings: the `dolthub/dolt` dependency tree dominates compile time and only
+moves when `go.sum` does. `--optional` turns a miss into a cold build rather
+than a failure — it replaced a `|| true`, which also swallowed a genuine S3
+error and an unreadable credential.
+
+`abuild` redirects `GOCACHE` into its own `$tmpdir` (and an upstream typo slaves
+`GOMODCACHE` to `GOCACHE`'s value rather than its own), so exports here cannot
+stick. `APKBUILD`'s `build()` re-pins both to these home locations, which is
+what makes the tarballs saved below the same trees restored here.
+
+### The half-restored module cache
+
+The repair block is not defensive padding. A partially restored module cache is
+the *normal* failure mode of this arrangement — an interrupted upload, a
+truncated object, a key written while a build was still running — and what it
+produces is a compile error deep in a dependency, which reads exactly like a bug
+in the code under test.
+
+So: make the tree writable (the module cache is mode 555 and `rm -rf` cannot
+remove it otherwise), ask `go mod verify` whether what came back is intact, and
+throw the whole thing away if it is not. Then `go mod download` and verify for
+real, and let *that* failure be fatal.
+
+Do not soften the final `go mod verify` to `|| true`. A build that proceeds with
+a module cache it could not verify is a build whose result means nothing.
+
+### `go mod download` and the word `all`
+
+It is `go mod download`, **not** `go mod download all`, and the difference is
+not a matter of thoroughness.
+
+`all` is the package pattern meaning the whole module graph reachable from this
+one — dependencies' test dependencies included, which no build of this module
+ever compiles. Resolving it makes the toolchain **append** the missing hashes to
+`go.sum`, and `go.sum` is a tracked file. A tracked file modified in the
+checkout before `go build` runs is exactly the `-dirty` stamp the `version` task
+above exists to avoid: the task written to protect the version would have been
+the thing that broke it.
+
+Measured on this tree with go1.26.5: `go mod download all` took `go.sum` from
+671 lines to 1097 — 426 hashes added — and left the tree modified. Plain `go mod
+download` left `go.sum` byte-identical, and `go mod verify` answered "all
+modules verified" after it. This repository's `dolthub/dolt` graph is the
+largest in the family, so the gap here is the widest; it is not zero anywhere.
+
+The same failure was seen for real on a sibling: bench build #359 failed its
+version gate with `M go.sum` as the only thing in the way.
+
+## build
+
+`REPODEST=$HOME/packages abuild -d` builds and stages the apk.
+
+`-d` disables abuild's dependency check: the makedepends are already installed
+by the manifest's `packages:` list, and abuild has no way to know that.
+
+`builddir="$startdir"` in `APKBUILD` means abuild packages *this checkout in
+place* rather than unpacking a tarball. There is no tarball to unpack — the
+package is built from the commit under test, which has not been released
+anywhere yet.
+
+### build() and package()
+
+`build()` compiles with `GOFLAGS="-trimpath -modcacherw"` and then asserts with
+`make check-css`; `package()` calls `make install-files`, which **does not
+build**.
+
+That split is the point. abuild runs `package()` in a fresh abuild process under
+fakeroot which re-sources the `APKBUILD` and never calls `build()`, so nothing
+`build()` exported survives — the cache pins above included. A `make install`
+there recompiled all three binaries from a cold module cache, without
+`-trimpath`, and it was *that* copy that went into the apk, while the checks
+`build()` had run were left behind with the copy nobody shipped.
+
+`-modcacherw` matters beyond tidiness: without it the module cache is left
+read-only, and the cache tarball `cache_save` makes from it cannot be unpacked
+on the next build (`mkdir` into `0555` directories fails).
+
+`make check-css` replaced a bare `ls static/main.min.*.css`. `install-share`
+copies stylesheets under `2>/dev/null || true`, so a `make css` that produced
+nothing would stage an unstyled service and fail nothing; and `web/router.go`
+resolves `main.min.*.css` by glob and takes the first match, so **two** hashed
+stylesheets are as wrong as none and quieter. `check-css` counts.
+
+`PREFIX="$pkgdir/usr"` and not `DESTDIR`: this Makefile's install rules write to
+`$(BINDIR)`/`$(SHAREDIR)` directly and genuinely do not honour `DESTDIR`.
+
+## publish
+
+The `[ ! -r ~/.apk-ci.env ]` gate is the honest answer to a build that was handed
+no secrets, not a fallback. A manual submission without secrets still runs every
+earlier task and still leaves a built, signed apk in `$HOME/packages`; saying so
+and exiting 0 is more useful than a red build about a credential nobody meant to
+supply. On a push the secret is there and this publishes.
+
+`set +x` before sourcing `~/.apk-ci.env` and `set -x` after. Every task runs
+under `set -x`, so without this the S3 access key and secret are echoed into a
+build log that is world-readable.
+
+The upload is `rclone copyto` per file — copy only, never delete, never sync. A
+`sync` would mirror local absence onto the bucket, and the bucket holds every
+previously published version of every service on the instance. Old versions stay
+so a pinned deployment can always be rebuilt.
+
+`apk-mirror` on phoebe re-indexes within 15 minutes; nothing here waits for it.
+
+## cache_save
+
+After `publish`, deliberately: a cache upload that fails must not strand an apk
+that was built and signed successfully but never shipped.
+
+Fatal on purpose, though — an upload failure here means the next build pays for
+a cold cache, and that is worth knowing about rather than hiding behind
+`|| true`.
+
+There is no `cacher exists "$KEY" ||` guard in front of the uploads any more.
+`cacher dir upload` without `--force` already skips a key that is present, so
+the guard bought nothing and cost a second round trip whose failure mode — a
+transient error on the `exists` probe — was an unnecessary re-upload.
+
+## The `hook` subpackage
+
+`dolt.sr.ht-hook` carries `dolt-git-hook` alone. It is installed into the
+**git.sr.ht** container as its post-update-script, not into the dolt one, and
+pulling the full service package in there would drag `doltsrht` and its whole
+dependency closure along.
+
+Nothing in this wave changed it, and that is deliberate: the git.sr.ht container
+consumes it at a pinned version. But it is built from the same `APKBUILD` — the
+`hook()` function only `amove`s a file the main package already staged — so it
+is versioned in lockstep with `dolt.sr.ht` and inherits everything above.
+
+Two consequences worth stating out loud. Its `pkgver` now comes from the
+exported `PKGVER` like the parent's, so the first tagged build renames it from
+`dolt.sr.ht-hook-0.0.<n>` to `dolt.sr.ht-hook-<tag>` and a deployment pinning
+the old name must be bumped along with it. And `dolt-git-hook` is one of the
+three binaries `build()` now compiles with `-trimpath`, so the file the
+subpackage ships changes bytes on the next build even where nothing else did.
+
+## What is not here
+
+- **No coverage upload.** The sibling services POST their profile to
+  cover.sr.ht at the end of the pipeline. Adding it here needs a token secret
+  and a repository on cover.
+- **No artifacts.** There is nothing to download: the apk goes to S3, and its
+  name changes every commit, which `artifacts:` cannot express (it has no
+  globbing).
+- **No matrix.** One architecture, one image.
+- **No `gofmt` gate.** `web/beads.go` and four test files are gofmt-dirty on
+  master; a gate would be red on arrival. `go vet` runs instead.