# CI: what `.build.yml` does and why The manifest is deliberately short on prose. builds.sr.ht stores a submitted manifest in a `varchar(16384)`, so a manifest that grows past 16 KiB cannot be submitted at all — the failure is at submission time and reads like nothing in particular. Rationale therefore lives here, and the manifest carries pointers. The pipeline is one linear job on `alpine/edge`: install the cache helper, assemble the shared SCSS, decide a version, restore caches, start a Postgres, test, package with `abuild`, publish the apk to `repo.bigb.es/alpine/v3.22/bigbes`, save caches. It is triggered by a push to the **sourcehut** side. A push to sourcecraft cannot reach builds.sr.ht; the gitsync mirror is what puts the commit on `git.srht.bigb.es`, and that push is what submits the job. So the mirror is on the critical path for the package repository, not merely an offsite copy. ## packages `sassc` and `minify` build the stylesheet; `curl` installs `cacher`; `rclone` publishes the apk; `abuild`, `go` and `git` are the build itself. `postgresql` and `postgresql-client` are for the test suite, not for the package. The apk declares no runtime dependency on Postgres — the daemon talks to whatever `connection-string` names, which in production is another host. There is no compiler toolchain beyond `go`: this package is built `CGO_ENABLED=0` with `-tags gms_pure_go` (see the APKBUILD and the Makefile), which is what keeps `libicu` and `gozstd` off the builder entirely. A default cgo build of this tree fails on missing ICU headers and always has. ## secrets Three, all account-level and shared with the sibling services: | secret | lands at | used by | |---|---|---| | `apk-ci-s3` | `~/.apk-ci.env` | `publish` | | `7dde4219-…` | `~/.s3-cache-key-id` | `cacher_init` | | `0e5b3530-…` | `~/.s3-cache-key-secret` | `cacher_init` | They are file secrets. Listing them is what turns `publish` and the cache tasks on; a manual submission that asks for no secrets still runs the interesting part of the pipeline and stops at `cacher_init`, which is the right place to notice. `apk-ci-s3` is referenced by name and the other two by UUID, which is only because that is how they were written in the donor manifests; both forms work. ## environment `CORE_VER` must track the deployment's `SRHT_CORE_VER`, and `BOOTSTRAP_REV` is the Bootstrap 4 submodule commit `core.sr.ht` pins at that tag. The two being out of step means this service renders against different partials than the rest of the instance, which shows up as a page that is subtly the wrong shape and as nothing at all in any log. Bump them together. ## cacher `cacher` is an S3-backed cache helper, installed from pages.sr.ht. It is **two tasks and not one** because `install.sh` appends its `PATH` export to `~/.buildenv`, and `~/.buildenv` is sourced by the *next* task's preamble — a single task would install it and then not find it. It is `install.sh` and not a raw `curl` of the binary, for two reasons that both end in the same place. The installer verifies the download against the published `checksums.txt`; and the raw `curl` it replaced had no `-f`, so an S3 404 or a proxy error page was written to `~/.local/bin/cacher`, `chmod +x`-ed, and only noticed as a baffling "not found"/"exec format error" one task later. ## scss `sassc` needs the shared sourcehut partials, and no apk ships them: they are `core.sr.ht`'s own `scss/` plus the Bootstrap 4 tree it pins as a submodule. The task clones both and drops them in `/usr/share/sourcehut/scss`, cached under a key made of the two pins, so an outage at git.sr.ht or GitHub cannot fail a build that changed nothing. The `--exec` block runs only on a cache miss, and `cacher` seeds the cache from the result afterwards. It sees exported variables only, which is why the cache key is spelled inline in the `cacher` invocation rather than computed inside the block. `--exec` and not the `if ! cacher dir download …; then … fi` this task used to be. The `if` form conflates two different failures into one branch: a genuine S3 error and an ordinary cache miss both mean "clone it again", so an S3 outage was silently absorbed into a slower build and never reported. And it made the seed upload the last command of the branch, i.e. fatal — an S3 hiccup while *storing* a cache would fail a build whose SCSS had assembled perfectly. Under `--exec` the seeding is best-effort by construction and the download's own errors stay distinguishable from a miss. ## keygen `abuild-keygen -a -n -i -q` makes a throwaway package signing key. `-i` is not optional: without it the key is generated but not installed into `/etc/apk/keys`, and `abuild` later fails to index its own output with an UNTRUSTED signature error, after having built the package perfectly well. The key is per-build and thrown away with the VM. That is fine because nothing verifies these signatures — the apk index on `repo.bigb.es` is signed by `apk-mirror` on phoebe, which indexes this repo with `--allow-untrusted` precisely because of this. ## version One `git describe` decides the apk's `pkgver`: - a tag — `v0.2.0` becomes `0.2.0`; - a tag plus commits since it — `v0.2.0-7-gabc1234` becomes `0.2.0_git7`. Alpine's version grammar does not accept the raw describe output, and `_git` sorts **after** the release in Alpine's comparison, which is what makes an untagged master build look newer than the tag it follows; - no tag at all — `0.0.`, which is where this repository stands today and is the scheme of the bench/spec/compare siblings. Tags are in `allow-refs` for exactly this reason: pushing `v0.2.0` is what produces the `0.2.0` apk. ### Why it is exported and not `sed`-ed `PKGVER` reaches `abuild` through `~/.buildenv`, and `APKBUILD` reads `pkgver="${PKGVER:-0.0.0}"`. It used to `sed` the literal in the tracked `APKBUILD` instead. **Do not tidy it back.** Go records a VCS stamp in every binary it builds inside a repository, and it decides the `vcs.modified` half of that stamp from `git status --porcelain`. Rewriting a tracked file in the checkout sets it, and it stays set for the whole `abuild` run — so every binary in the apk records itself as `-dirty`. Measured on go1.26.5. The same flag is why `.gitignore` lists `/src/`, `/pkg/` and `/tmp/`: `git status --porcelain` counts **untracked** files too, and abuild — which works in this checkout, since `APKBUILD` sets `builddir="$startdir"` and has no tarball to unpack — creates all three inside it. `/tmp/` is the one that is easy to miss: abuild defaults `tmpdir` to `$startdir/tmp` and exports `GOTMPDIR` to it, and `/usr/share/abuild/default.conf` exports `MAKEFLAGS=-j$(nproc)`, under which `make all-bin`'s three `go build`s run at once and each sees the others' work directories. The task ends with `git status --porcelain` because this is the last moment the tree is provably clean, and the print costs nothing. ## cache_restore The Go module cache and build cache are keyed by the hash of `go.sum`, so a build that changed no dependency reuses both. That matters more here than in the siblings: the `dolthub/dolt` dependency tree dominates compile time and only moves when `go.sum` does. `--optional` turns a miss into a cold build rather than a failure — it replaced a `|| true`, which also swallowed a genuine S3 error and an unreadable credential. `abuild` redirects `GOCACHE` into its own `$tmpdir` (and an upstream typo slaves `GOMODCACHE` to `GOCACHE`'s value rather than its own), so exports here cannot stick. `APKBUILD`'s `build()` re-pins both to these home locations, which is what makes the tarballs saved below the same trees restored here. ### The half-restored module cache The repair block is not defensive padding. A partially restored module cache is the *normal* failure mode of this arrangement — an interrupted upload, a truncated object, a key written while a build was still running — and what it produces is a compile error deep in a dependency, which reads exactly like a bug in the code under test. So: make the tree writable (the module cache is mode 555 and `rm -rf` cannot remove it otherwise), ask `go mod verify` whether what came back is intact, and throw the whole thing away if it is not. Then `go mod download` and verify for real, and let *that* failure be fatal. Do not soften the final `go mod verify` to `|| true`. A build that proceeds with a module cache it could not verify is a build whose result means nothing. ### `go mod download` and the word `all` It is `go mod download`, **not** `go mod download all`, and the difference is not a matter of thoroughness. `all` is the package pattern meaning the whole module graph reachable from this one — dependencies' test dependencies included, which no build of this module ever compiles. Resolving it makes the toolchain **append** the missing hashes to `go.sum`, and `go.sum` is a tracked file. A tracked file modified in the checkout before `go build` runs is exactly the `-dirty` stamp the `version` task above exists to avoid: the task written to protect the version would have been the thing that broke it. Measured on this tree with go1.26.5: `go mod download all` took `go.sum` from 671 lines to 1097 — 426 hashes added — and left the tree modified. Plain `go mod download` left `go.sum` byte-identical, and `go mod verify` answered "all modules verified" after it. This repository's `dolthub/dolt` graph is the largest in the family, so the gap here is the widest; it is not zero anywhere. The same failure was seen for real on a sibling: bench build #359 failed its version gate with `M go.sum` as the only thing in the way. ## postgres A real Postgres in the VM, initialised from scratch each build. Until this task existed, **not one of this repository's 28 test files had ever run on the builder** — the manifest went straight from `cache_restore` to `abuild`, and `options="!check"` in the `APKBUILD` meant abuild did not run them either. Every apk this pipeline has published was built from code CI had compiled and never executed. What the database needs to be there for, measured on this tree: `db/` has 10 tests and 9 of them gate themselves on `DOLTSRHT_TEST_PG` (`db/db_test.go`'s `newTestStore`), so without a DSN the entire persistence layer — repository and key CRUD, the access-control queries, the schema — is skipped and the build is green regardless. The other packages do not need it and do not skip without it. The suites do not share state: each creates a scratch schema `doltsrht_test_`, applies `schema.sql` into it, routes its pool there with lib/pq's `options=-c search_path=…`, and drops it afterwards. So one database serves the whole suite, and no `CREATE DATABASE` privilege is needed past the one `createdb` above. `fsync=off`, `full_page_writes=off` and `synchronous_commit=off` are safe here and only here: the database lives for the length of one build and its durability guarantees protect nothing. `-k /run/postgresql` is why the task creates that directory — Alpine's package does not — and `createuser -s "$(id -un)"` is what lets the build user connect without a password, which is what keeps the DSN in `~/.buildenv` free of a credential. ### `remoteapi/integration_test.go` does not want Docker That file's `startPostgres` prefers `DOLTSRHT_TEST_PG` and falls back to `docker run postgres:16-alpine` only when the variable is empty, so with the DSN exported it never looks for a Docker daemon the builder does not have. It is moot in any case: the file is behind `//go:build integration`, so `go test ./...` does not even compile it. Running it would additionally need the `dolt` CLI, which it skips on. `storage/spike_test.go` is behind `//go:build spike` for the same kind of reason. Neither tag is set here, deliberately. ## test `make vet` and `make test`, not bare `go` commands: the Makefile is where `-tags gms_pure_go` and `CGO_ENABLED=0` are named, and a second copy of those two here is a second copy to forget. The tags are not optional — a `go vet` or `go test` without `gms_pure_go` pulls `go-icu-regex` in and wants ICU headers the builder has never had. The guard on an empty `DOLTSRHT_TEST_PG` exists because the failure it prevents is silent. If the `postgres` task did not export the DSN — or if someone reorders the two tasks — every database suite skips with a friendly message, `go test` exits 0, and the build is green over untested code. An explicit refusal is the difference between a broken pipeline and a lying one. It is also what makes `options="!check"` in the `APKBUILD` an honest claim rather than a licence: that option says "the suites ran in CI", and this guard is the only thing that keeps it true. There is no `gofmt` gate, unlike the tokens sibling. `web/beads.go` and four test files are gofmt-dirty on master, so a gate would be red on arrival; fixing them is a separate change and not one to smuggle into a CI wave. The task runs **before** `build`, so the tree it leaves behind is the tree `abuild` packages; the closing `git status --porcelain` is where anything the suites wrote into the checkout would show. ## build `REPODEST=$HOME/packages abuild -d` builds and stages the apk. `-d` disables abuild's dependency check: the makedepends are already installed by the manifest's `packages:` list, and abuild has no way to know that. `builddir="$startdir"` in `APKBUILD` means abuild packages *this checkout in place* rather than unpacking a tarball. There is no tarball to unpack — the package is built from the commit under test, which has not been released anywhere yet. ### build() and package() `build()` compiles with `GOFLAGS="-trimpath -modcacherw"` and then asserts with `make check-css`; `package()` calls `make install-files`, which **does not build**. That split is the point. abuild runs `package()` in a fresh abuild process under fakeroot which re-sources the `APKBUILD` and never calls `build()`, so nothing `build()` exported survives — the cache pins above included. A `make install` there recompiled all three binaries from a cold module cache, without `-trimpath`, and it was *that* copy that went into the apk, while the checks `build()` had run were left behind with the copy nobody shipped. `-modcacherw` matters beyond tidiness: without it the module cache is left read-only, and the cache tarball `cache_save` makes from it cannot be unpacked on the next build (`mkdir` into `0555` directories fails). `make check-css` replaced a bare `ls static/main.min.*.css`. `install-share` copies stylesheets under `2>/dev/null || true`, so a `make css` that produced nothing would stage an unstyled service and fail nothing; and `web/router.go` resolves `main.min.*.css` by glob and takes the first match, so **two** hashed stylesheets are as wrong as none and quieter. `check-css` counts. `PREFIX="$pkgdir/usr"` and not `DESTDIR`: this Makefile's install rules write to `$(BINDIR)`/`$(SHAREDIR)` directly and genuinely do not honour `DESTDIR`. ## publish The `[ ! -r ~/.apk-ci.env ]` gate is the honest answer to a build that was handed no secrets, not a fallback. A manual submission without secrets still runs every earlier task and still leaves a built, signed apk in `$HOME/packages`; saying so and exiting 0 is more useful than a red build about a credential nobody meant to supply. On a push the secret is there and this publishes. `set +x` before sourcing `~/.apk-ci.env` and `set -x` after. Every task runs under `set -x`, so without this the S3 access key and secret are echoed into a build log that is world-readable. The upload is `rclone copyto` per file — copy only, never delete, never sync. A `sync` would mirror local absence onto the bucket, and the bucket holds every previously published version of every service on the instance. Old versions stay so a pinned deployment can always be rebuilt. `apk-mirror` on phoebe re-indexes within 15 minutes; nothing here waits for it. ## cache_save After `publish`, deliberately: a cache upload that fails must not strand an apk that was built and signed successfully but never shipped. Fatal on purpose, though — an upload failure here means the next build pays for a cold cache, and that is worth knowing about rather than hiding behind `|| true`. There is no `cacher exists "$KEY" ||` guard in front of the uploads any more. `cacher dir upload` without `--force` already skips a key that is present, so the guard bought nothing and cost a second round trip whose failure mode — a transient error on the `exists` probe — was an unnecessary re-upload. ## The `hook` subpackage `dolt.sr.ht-hook` carries `dolt-git-hook` alone. It is installed into the **git.sr.ht** container as its post-update-script, not into the dolt one, and pulling the full service package in there would drag `doltsrht` and its whole dependency closure along. Nothing in this wave changed it, and that is deliberate: the git.sr.ht container consumes it at a pinned version. But it is built from the same `APKBUILD` — the `hook()` function only `amove`s a file the main package already staged — so it is versioned in lockstep with `dolt.sr.ht` and inherits everything above. Two consequences worth stating out loud. Its `pkgver` now comes from the exported `PKGVER` like the parent's, so the first tagged build renames it from `dolt.sr.ht-hook-0.0.` to `dolt.sr.ht-hook-` and a deployment pinning the old name must be bumped along with it. And `dolt-git-hook` is one of the three binaries `build()` now compiles with `-trimpath`, so the file the subpackage ships changes bytes on the next build even where nothing else did. ## What is not here - **No coverage upload.** The sibling services POST their profile to cover.sr.ht at the end of the pipeline. Adding it here needs a token secret and a repository on cover. - **No artifacts.** There is nothing to download: the apk goes to S3, and its name changes every commit, which `artifacts:` cannot express (it has no globbing). - **No matrix.** One architecture, one image. - **No `gofmt` gate.** See [test](#test). - **No `integration` or `spike` build tag.** See [`remoteapi/integration_test.go` does not want Docker](#remoteapiintegration_testgo-does-not-want-docker).