# CI: what `.build.yml` does and why The manifest is deliberately short on prose. builds.sr.ht stores a submitted manifest in a `varchar(16384)`, so a manifest that grows past 16 KiB cannot be submitted at all — the failure is at submission time and reads like nothing in particular. Rationale therefore lives here, and the manifest carries pointers. The pipeline is one linear job on `alpine/edge`: install the cache helper, assemble the shared SCSS, decide a version, restore caches, package with `abuild`, publish the apk to `repo.bigb.es/alpine/v3.22/bigbes`, save caches. It is triggered by a push to the **sourcehut** side. A push to sourcecraft cannot reach builds.sr.ht; the gitsync mirror is what puts the commit on `git.srht.bigb.es`, and that push is what submits the job. So the mirror is on the critical path for the package repository, not merely an offsite copy. ## packages `sassc` and `minify` build the stylesheet; `curl` installs `cacher`; `rclone` publishes the apk; `abuild`, `go` and `git` are the build itself. There is no `postgresql` here and no compiler toolchain beyond `go`: this package is built `CGO_ENABLED=0` with `-tags gms_pure_go` (see the APKBUILD and the Makefile), which is what keeps `libicu` and `gozstd` off the builder entirely. A default cgo build of this tree fails on missing ICU headers and always has. ## secrets Three, all account-level and shared with the sibling services: | secret | lands at | used by | |---|---|---| | `apk-ci-s3` | `~/.apk-ci.env` | `publish` | | `7dde4219-…` | `~/.s3-cache-key-id` | `cacher_init` | | `0e5b3530-…` | `~/.s3-cache-key-secret` | `cacher_init` | They are file secrets. Listing them is what turns `publish` and the cache tasks on; a manual submission that asks for no secrets still runs the interesting part of the pipeline and stops at `cacher_init`, which is the right place to notice. `apk-ci-s3` is referenced by name and the other two by UUID, which is only because that is how they were written in the donor manifests; both forms work. ## environment `CORE_VER` must track the deployment's `SRHT_CORE_VER`, and `BOOTSTRAP_REV` is the Bootstrap 4 submodule commit `core.sr.ht` pins at that tag. The two being out of step means this service renders against different partials than the rest of the instance, which shows up as a page that is subtly the wrong shape and as nothing at all in any log. Bump them together. ## cacher `cacher` is an S3-backed cache helper, installed from pages.sr.ht. It is **two tasks and not one** because `install.sh` appends its `PATH` export to `~/.buildenv`, and `~/.buildenv` is sourced by the *next* task's preamble — a single task would install it and then not find it. It is `install.sh` and not a raw `curl` of the binary, for two reasons that both end in the same place. The installer verifies the download against the published `checksums.txt`; and the raw `curl` it replaced had no `-f`, so an S3 404 or a proxy error page was written to `~/.local/bin/cacher`, `chmod +x`-ed, and only noticed as a baffling "not found"/"exec format error" one task later. ## scss `sassc` needs the shared sourcehut partials, and no apk ships them: they are `core.sr.ht`'s own `scss/` plus the Bootstrap 4 tree it pins as a submodule. The task clones both and drops them in `/usr/share/sourcehut/scss`, cached under a key made of the two pins, so an outage at git.sr.ht or GitHub cannot fail a build that changed nothing. The `--exec` block runs only on a cache miss, and `cacher` seeds the cache from the result afterwards. It sees exported variables only, which is why the cache key is spelled inline in the `cacher` invocation rather than computed inside the block. `--exec` and not the `if ! cacher dir download …; then … fi` this task used to be. The `if` form conflates two different failures into one branch: a genuine S3 error and an ordinary cache miss both mean "clone it again", so an S3 outage was silently absorbed into a slower build and never reported. And it made the seed upload the last command of the branch, i.e. fatal — an S3 hiccup while *storing* a cache would fail a build whose SCSS had assembled perfectly. Under `--exec` the seeding is best-effort by construction and the download's own errors stay distinguishable from a miss. ## keygen `abuild-keygen -a -n -i -q` makes a throwaway package signing key. `-i` is not optional: without it the key is generated but not installed into `/etc/apk/keys`, and `abuild` later fails to index its own output with an UNTRUSTED signature error, after having built the package perfectly well. The key is per-build and thrown away with the VM. That is fine because nothing verifies these signatures — the apk index on `repo.bigb.es` is signed by `apk-mirror` on phoebe, which indexes this repo with `--allow-untrusted` precisely because of this. ## version One `git describe` decides the apk's `pkgver`: - a tag — `v0.2.0` becomes `0.2.0`; - a tag plus commits since it — `v0.2.0-7-gabc1234` becomes `0.2.0_git7`. Alpine's version grammar does not accept the raw describe output, and `_git` sorts **after** the release in Alpine's comparison, which is what makes an untagged master build look newer than the tag it follows; - no tag at all — `0.0.`, which is where this repository stands today and is the scheme of the bench/spec/compare siblings. Tags are in `allow-refs` for exactly this reason: pushing `v0.2.0` is what produces the `0.2.0` apk. ### Why it is exported and not `sed`-ed `PKGVER` reaches `abuild` through `~/.buildenv`, and `APKBUILD` reads `pkgver="${PKGVER:-0.0.0}"`. It used to `sed` the literal in the tracked `APKBUILD` instead. **Do not tidy it back.** Go records a VCS stamp in every binary it builds inside a repository, and it decides the `vcs.modified` half of that stamp from `git status --porcelain`. Rewriting a tracked file in the checkout sets it, and it stays set for the whole `abuild` run — so every binary in the apk records itself as `-dirty`. Measured on go1.26.5. The same flag is why `.gitignore` lists `/src/`, `/pkg/` and `/tmp/`: `git status --porcelain` counts **untracked** files too, and abuild — which works in this checkout, since `APKBUILD` sets `builddir="$startdir"` and has no tarball to unpack — creates all three inside it. `/tmp/` is the one that is easy to miss: abuild defaults `tmpdir` to `$startdir/tmp` and exports `GOTMPDIR` to it, and `/usr/share/abuild/default.conf` exports `MAKEFLAGS=-j$(nproc)`, under which `make all-bin`'s three `go build`s run at once and each sees the others' work directories. The task ends with `git status --porcelain` because this is the last moment the tree is provably clean, and the print costs nothing. ## cache_restore The Go module cache and build cache are keyed by the hash of `go.sum`, so a build that changed no dependency reuses both. That matters more here than in the siblings: the `dolthub/dolt` dependency tree dominates compile time and only moves when `go.sum` does. `--optional` turns a miss into a cold build rather than a failure — it replaced a `|| true`, which also swallowed a genuine S3 error and an unreadable credential. `abuild` redirects `GOCACHE` into its own `$tmpdir` (and an upstream typo slaves `GOMODCACHE` to `GOCACHE`'s value rather than its own), so exports here cannot stick. `APKBUILD`'s `build()` re-pins both to these home locations, which is what makes the tarballs saved below the same trees restored here. ### The half-restored module cache The repair block is not defensive padding. A partially restored module cache is the *normal* failure mode of this arrangement — an interrupted upload, a truncated object, a key written while a build was still running — and what it produces is a compile error deep in a dependency, which reads exactly like a bug in the code under test. So: make the tree writable (the module cache is mode 555 and `rm -rf` cannot remove it otherwise), ask `go mod verify` whether what came back is intact, and throw the whole thing away if it is not. Then `go mod download` and verify for real, and let *that* failure be fatal. Do not soften the final `go mod verify` to `|| true`. A build that proceeds with a module cache it could not verify is a build whose result means nothing. ### `go mod download` and the word `all` It is `go mod download`, **not** `go mod download all`, and the difference is not a matter of thoroughness. `all` is the package pattern meaning the whole module graph reachable from this one — dependencies' test dependencies included, which no build of this module ever compiles. Resolving it makes the toolchain **append** the missing hashes to `go.sum`, and `go.sum` is a tracked file. A tracked file modified in the checkout before `go build` runs is exactly the `-dirty` stamp the `version` task above exists to avoid: the task written to protect the version would have been the thing that broke it. Measured on this tree with go1.26.5: `go mod download all` took `go.sum` from 671 lines to 1097 — 426 hashes added — and left the tree modified. Plain `go mod download` left `go.sum` byte-identical, and `go mod verify` answered "all modules verified" after it. This repository's `dolthub/dolt` graph is the largest in the family, so the gap here is the widest; it is not zero anywhere. The same failure was seen for real on a sibling: bench build #359 failed its version gate with `M go.sum` as the only thing in the way. ## build `REPODEST=$HOME/packages abuild -d` builds and stages the apk. `-d` disables abuild's dependency check: the makedepends are already installed by the manifest's `packages:` list, and abuild has no way to know that. `builddir="$startdir"` in `APKBUILD` means abuild packages *this checkout in place* rather than unpacking a tarball. There is no tarball to unpack — the package is built from the commit under test, which has not been released anywhere yet. ### build() and package() `build()` compiles with `GOFLAGS="-trimpath -modcacherw"` and then asserts with `make check-css`; `package()` calls `make install-files`, which **does not build**. That split is the point. abuild runs `package()` in a fresh abuild process under fakeroot which re-sources the `APKBUILD` and never calls `build()`, so nothing `build()` exported survives — the cache pins above included. A `make install` there recompiled all three binaries from a cold module cache, without `-trimpath`, and it was *that* copy that went into the apk, while the checks `build()` had run were left behind with the copy nobody shipped. `-modcacherw` matters beyond tidiness: without it the module cache is left read-only, and the cache tarball `cache_save` makes from it cannot be unpacked on the next build (`mkdir` into `0555` directories fails). `make check-css` replaced a bare `ls static/main.min.*.css`. `install-share` copies stylesheets under `2>/dev/null || true`, so a `make css` that produced nothing would stage an unstyled service and fail nothing; and `web/router.go` resolves `main.min.*.css` by glob and takes the first match, so **two** hashed stylesheets are as wrong as none and quieter. `check-css` counts. `PREFIX="$pkgdir/usr"` and not `DESTDIR`: this Makefile's install rules write to `$(BINDIR)`/`$(SHAREDIR)` directly and genuinely do not honour `DESTDIR`. ## publish The `[ ! -r ~/.apk-ci.env ]` gate is the honest answer to a build that was handed no secrets, not a fallback. A manual submission without secrets still runs every earlier task and still leaves a built, signed apk in `$HOME/packages`; saying so and exiting 0 is more useful than a red build about a credential nobody meant to supply. On a push the secret is there and this publishes. `set +x` before sourcing `~/.apk-ci.env` and `set -x` after. Every task runs under `set -x`, so without this the S3 access key and secret are echoed into a build log that is world-readable. The upload is `rclone copyto` per file — copy only, never delete, never sync. A `sync` would mirror local absence onto the bucket, and the bucket holds every previously published version of every service on the instance. Old versions stay so a pinned deployment can always be rebuilt. `apk-mirror` on phoebe re-indexes within 15 minutes; nothing here waits for it. ## cache_save After `publish`, deliberately: a cache upload that fails must not strand an apk that was built and signed successfully but never shipped. Fatal on purpose, though — an upload failure here means the next build pays for a cold cache, and that is worth knowing about rather than hiding behind `|| true`. There is no `cacher exists "$KEY" ||` guard in front of the uploads any more. `cacher dir upload` without `--force` already skips a key that is present, so the guard bought nothing and cost a second round trip whose failure mode — a transient error on the `exists` probe — was an unnecessary re-upload. ## The `hook` subpackage `dolt.sr.ht-hook` carries `dolt-git-hook` alone. It is installed into the **git.sr.ht** container as its post-update-script, not into the dolt one, and pulling the full service package in there would drag `doltsrht` and its whole dependency closure along. Nothing in this wave changed it, and that is deliberate: the git.sr.ht container consumes it at a pinned version. But it is built from the same `APKBUILD` — the `hook()` function only `amove`s a file the main package already staged — so it is versioned in lockstep with `dolt.sr.ht` and inherits everything above. Two consequences worth stating out loud. Its `pkgver` now comes from the exported `PKGVER` like the parent's, so the first tagged build renames it from `dolt.sr.ht-hook-0.0.` to `dolt.sr.ht-hook-` and a deployment pinning the old name must be bumped along with it. And `dolt-git-hook` is one of the three binaries `build()` now compiles with `-trimpath`, so the file the subpackage ships changes bytes on the next build even where nothing else did. ## What is not here - **No coverage upload.** The sibling services POST their profile to cover.sr.ht at the end of the pipeline. Adding it here needs a token secret and a repository on cover. - **No artifacts.** There is nothing to download: the apk goes to S3, and its name changes every commit, which `artifacts:` cannot express (it has no globbing). - **No matrix.** One architecture, one image. - **No `gofmt` gate.** `web/beads.go` and four test files are gofmt-dirty on master; a gate would be red on arrival. `go vet` runs instead.