~bigbes/sr-ht-compare

f18653603d66734b583b52d2fe0da7c59a7b1f32 — bigbes 9 days ago e0283ff 
ci: the family's build pipeline, and PKGVER by export

Brings this manifest to the shape the sibling services share.

cacher is installed by install.sh instead of a raw curl of the binary, in
two tasks: install.sh appends its PATH export to ~/.buildenv, which only
the next task sources. The old fetch had no -f, so an HTTP error body was
written to ~/.local/bin/cacher and chmod +x'd; the installer verifies
against checksums.txt.

The version task drops the sed of the tracked APKBUILD. Measured on
go1.26.5, rewriting a tracked file sets vcs.modified=true and Go stamps
+dirty into Main.Version for the whole abuild run, so PKGVER is exported
and APKBUILD reads pkgver="${PKGVER:-0.0.0}". One git describe decides
it: the tag, the tag plus commits, or the commit count. Tags join
allow-refs so pushing v0.9.0 produces the 0.9.0 apk. The export alone is
not enough — abuild works inside the checkout and puts src/, pkg/ and
GOTMPDIR=tmp/ there, so .gitignore covers all three.

cache_restore takes --optional rather than || true, which also swallowed
bad credentials and a dead bucket, and gains the half-restored module
cache repair block. cache_save moves after publish so an S3 hiccup cannot
strand a good apk, and loses its cacher exists guards: dir upload already
skips a present key. publish gains the no-credentials gate.

The stylesheet is now built in CI: sassc and minify, the scss task that
materializes the shared partials no apk ships, make css before the
compile and make check-css after it. The committed
web/static/main.min.79713f25.css stays tracked for now — build() prints
git status and the digest so the first CI run says whether the pipeline
reproduces those bytes, and dropping it from the index is a second step.

package() calls make install-files. abuild runs it in a fresh process
that never called build(), so make install would relink the binary from a
cold cache and ship a compilation nothing had checked.

Reasoning lives in docs/ci.md: the manifest is a varchar(16384) and a
branch over the cap gets no CI at all
patch browse .tar.gz .zip 
5 files changed, 621 insertions(+), 56 deletions(-)

M .build.yml
M .gitignore
M APKBUILD
M Makefile
A docs/ci.md

M .build.yml => .build.yml +107 -45
@@ 1,3 1,11 @@
# builds.sr.ht manifest for compare.sr.ht. One linear pipeline: install the
# cache helper, assemble the shared SCSS, restore caches, package with abuild,
# publish the apk, save the caches.
#
# The reasoning behind every task lives in docs/ci.md, not here: builds.sr.ht
# stores the submitted manifest in a varchar(16384), so a manifest over 16 KiB
# cannot be submitted at all — and the failure is a branch with no CI, not a red
# build. Add paragraphs to docs/ci.md and a pointer here.
image: alpine/edge
packages:
  - abuild


@@ 5,6 13,9 @@ packages:
  - go
  - git
  - rclone
  # For `make css`, not for the package — see docs/ci.md#packages.
  - sassc
  - minify
secrets:
  # File secret `apk-ci-s3`, installed at ~/.apk-ci.env, containing
  # APK_CI_S3_ACCESS_KEY / APK_CI_S3_SECRET_KEY for the Garage `repo` bucket.


@@ 20,37 31,23 @@ environment:
  APK_REPO: alpine/v3.22/bigbes/x86_64
  S3_BUCKET: repo
  S3_ENDPOINT: https://s3.bigb.es
  # CORE_VER must track the deployment's SRHT_CORE_VER; BOOTSTRAP_REV is the
  # submodule commit core.sr.ht pins at that tag. See docs/ci.md#environment.
  CORE_VER: "0.84.5"
  BOOTSTRAP_REV: 779ad9f174ea5ab7e755f6df0ec9e5912d67dd16
submitter:
  git.sr.ht:
    allow-refs:
      - refs/heads/master
      - "refs/tags/v*"
tasks:
  - keygen: |
      # abuild insists on signing what it builds, but this key is deliberately
      # throwaway: generated per build, dies with the VM, trusted by nothing.
      # Clients verify against the index instead, which is rebuilt and signed on
      # phoebe by the garage stack's apk-mirror service — it indexes this repo
      # with --allow-untrusted precisely because of this.
      #
      # -i installs the public half into /etc/apk/keys. Without it abuild's own
      # final "update the local repository index" step dies with UNTRUSTED
      # signature, after having built the package perfectly well.
      SUDO=sudo abuild-keygen -a -n -i -q
  - version: |
      cd "$REPO"
      ver="0.0.$(git rev-list --count HEAD)"
      sed -i "s/^pkgver=.*/pkgver=$ver/" APKBUILD
      echo "export PKGVER=$ver" >> ~/.buildenv
      echo "building $ver"
  - cacher: |
      # S3-backed CI cache helper (go.bigb.es/cacher), dogfooded from its own
      # published release — the same bootstrap the bencher/ci-cacher builds use.
      mkdir -p ~/.local/bin
      curl -sSL "https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64" \
        -o ~/.local/bin/cacher
      chmod +x ~/.local/bin/cacher
      echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.buildenv
      export PATH="$HOME/.local/bin:$PATH"
  # S3-backed CI cache helper (go.bigb.es/cacher), installed from its own
  # published release. Two tasks and not one: install.sh appends its PATH export
  # to ~/.buildenv, which only the NEXT task sources. The installer verifies
  # what it downloads against checksums.txt. See docs/ci.md#cacher.
  - cacher_install: |
      curl -fsSL https://bigbes.pages.srht.bigb.es/ci-cacher/install.sh | sh
  - cacher_init: |
      cacher init \
        --endpoint    https://s3.bigb.es \
        --region      garage \


@@ 58,30 55,90 @@ tasks:
        --prefix      sr-ht-compare/deps \
        --key-file    ~/.s3-cache-key-id \
        --secret-file ~/.s3-cache-key-secret
  - scss: |
      # Assemble the shared sourcehut partials no apk ships, the way
      # core.sr.ht's `make install-scss` would, cached by the two pins so an
      # outage at git.sr.ht or github.com can't fail us. --exec runs on a miss
      # and seeds the cache after; it sees exported vars only, hence the inline
      # key and the single quotes. See docs/ci.md#scss.
      cacher dir download "scss/${CORE_VER}-${BOOTSTRAP_REV}.tar.zst" ~/scss --exec '
        git clone --depth 1 --branch "$CORE_VER" \
          https://git.sr.ht/~sircmpwn/core.sr.ht /tmp/core
        mkdir -p ~/scss/bootstrap
        cp /tmp/core/scss/*.scss /tmp/core/scss/*.css ~/scss/
        git init -q /tmp/bootstrap
        git -C /tmp/bootstrap remote add origin https://github.com/twbs/bootstrap
        git -C /tmp/bootstrap fetch -q --depth 1 origin "$BOOTSTRAP_REV"
        git -C /tmp/bootstrap checkout -q FETCH_HEAD
        cp -r /tmp/bootstrap/scss ~/scss/bootstrap/scss
      '
      sudo mkdir -p /usr/share/sourcehut
      sudo cp -r ~/scss /usr/share/sourcehut/scss
  - keygen: |
      # Throwaway signing key, and -i is not optional: docs/ci.md#keygen.
      SUDO=sudo abuild-keygen -a -n -i -q
  - version: |
      # ONE `git describe` decides the pkgver: a tag, else <tag>_git<n>, else the
      # family's 0.0.<commit count>. EXPORTED rather than sed-ed into the tracked
      # APKBUILD (which reads $PKGVER), because rewriting a tracked file flips
      # the VCS stamp Go records into every binary built afterwards to dirty —
      # do not "tidy" it back into a sed. The tree is printed because this is the
      # last moment it is provably clean. See docs/ci.md#version.
      cd "$REPO"
      desc=$(git describe --tags --always --dirty)
      base=${desc%-dirty}
      case "$base" in
        v*-g*) n=${base%-g*}; ver="${n%-*}"; ver="${ver#v}_git${n##*-}" ;;
        v*)    ver="${base#v}" ;;
        *)     ver="0.0.$(git rev-list --count HEAD)" ;;
      esac
      echo "export PKGVER=$ver" >> ~/.buildenv
      echo "building $ver from $desc"
      git status --porcelain
  - cache_restore: |
      # Restore the Go module and build caches, both keyed by go.sum: the
      # dependency tree dominates compile time, and it only changes when go.sum
      # does. A miss is just a cold build, never an error.
      # Go module and build caches keyed by go.sum; --optional makes a genuine
      # miss a cold build rather than an error, which `|| true` could not tell
      # apart from bad credentials. abuild re-pins both, so the exports here are
      # only for the repair block below. See docs/ci.md#cache_restore.
      KEY_MOD=$(cacher key "gomod/{hash}.tar.zst" --hash-from "$REPO/go.sum")
      KEY_GOC=$(cacher key "gocache/{hash}.tar.zst" --hash-from "$REPO/go.sum")
      echo "export KEY_MOD=$KEY_MOD KEY_GOC=$KEY_GOC" >> ~/.buildenv
      # abuild redirects the Go caches into its throwaway $tmpdir (and an
      # upstream typo slaves GOMODCACHE to GOCACHE), so env exports here can't
      # stick — the APKBUILD's build() re-pins both to these home locations.
      cacher dir download "$KEY_MOD" ~/go/pkg/mod || true
      cacher dir download "$KEY_GOC" ~/.cache/go-build || true
      cacher dir download "$KEY_MOD" ~/go/pkg/mod        --optional
      cacher dir download "$KEY_GOC" ~/.cache/go-build   --optional
      # Repair block for the HALF-restored module cache — the normal failure
      # here, not a freak one, and it reads like a code bug. Do not remove and
      # do not soften to `|| true`:
      # docs/ci.md#the-half-restored-module-cache.
      cd "$REPO"
      chmod -R u+w ~/go/pkg/mod 2>/dev/null || true
      if ! go mod verify >/dev/null 2>&1; then
        echo "restored module cache did not verify — discarding it"
        rm -rf ~/go/pkg/mod
      fi
      go mod download all
      go mod verify
      # -mod=readonly is the default, so neither line above can rewrite go.mod
      # or go.sum — but an untracked or modified file here is a "+dirty" stamp
      # in the packaged binary, so say so out loud rather than trusting the
      # flag. docs/ci.md#cache_restore.
      git status --porcelain
  - build: |
      cd "$REPO"
      # -d: makedepends are already installed via `packages:` above, so skip
      # abuild's own dependency resolution (which would want to sudo apk add).
      # -d: makedepends come from `packages:` above, so skip abuild's own
      # dependency resolution. The APKBUILD runs `make css` before the compile
      # and `make check-css` after it, because web/ go:embed-s static/.
      # See docs/ci.md#build.
      REPODEST=$HOME/packages abuild -d
      find "$HOME/packages" -name '*.apk'
  - cache_save: |
      # Seed the caches only when this go.sum has no entry yet — on a hit the
      # tarballs are already up there and re-uploading identical bytes is waste.
      cacher exists "$KEY_MOD" || cacher dir upload "$KEY_MOD" ~/go/pkg/mod
      cacher exists "$KEY_GOC" || cacher dir upload "$KEY_GOC" ~/.cache/go-build
  - publish: |
      # The gate is the honest answer to a build handed no secrets, not a
      # fallback: without ~/.apk-ci.env every earlier task has still run and a
      # signed apk is sitting in $HOME/packages. See docs/ci.md#publish.
      if [ ! -r ~/.apk-ci.env ]; then
        echo "no ~/.apk-ci.env: this build has no apk repo credentials"
        echo "the package was built and signed, and is not published"
        exit 0
      fi
      set +x   # never echo the S3 credentials into the build log
      . ~/.apk-ci.env
      export RCLONE_CONFIG_GARAGE_TYPE=s3


@@ 92,12 149,17 @@ tasks:
      export RCLONE_CONFIG_GARAGE_ACCESS_KEY_ID="$APK_CI_S3_ACCESS_KEY"
      export RCLONE_CONFIG_GARAGE_SECRET_ACCESS_KEY="$APK_CI_S3_SECRET_KEY"
      set -x
      # Upload only; never delete. Old versions stay so a pinned deployment can
      # always be rebuilt — the same reason the upstream mirror is append-only.
      # abuild nests output under $REPODEST/<repo>/<arch>/, so flatten by
      # uploading each file to a fixed prefix rather than mirroring the tree.
      # Upload only, never delete; abuild nests output under
      # $REPODEST/<repo>/<arch>/, so flatten to a fixed prefix rather than
      # mirroring the tree. See docs/ci.md#publish.
      find "$HOME/packages" -name '*.apk' -print | while read -r f; do
        rclone copyto "$f" "garage:$S3_BUCKET/$APK_REPO/$(basename "$f")"
        echo "uploaded $(basename "$f")"
      done
      echo "published; apk-mirror on phoebe re-indexes within 15 minutes"
  - cache_save: |
      # AFTER publish so an S3 hiccup cannot strand a good apk, and fatal on
      # purpose. Without --force an upload skips a key already there, so no
      # `cacher exists ||` guard is needed. See docs/ci.md#cache_save.
      cacher dir upload "$KEY_MOD" ~/go/pkg/mod
      cacher dir upload "$KEY_GOC" ~/.cache/go-build


M .gitignore => .gitignore +27 -0
@@ 4,6 4,33 @@ node_modules/
# Compiled service binary
/comparesrht

# The three scratch directories abuild works in: with `source=""` and
# `builddir="$startdir"` (see APKBUILD) it unpacks into ./src, stages the
# package into ./pkg and keeps ./tmp for the toolchain caches it redirects —
# i.e. all three are inside the checkout it is packaging.
#
# Listing them is not tidiness. Go records vcs.modified in every binary built
# inside a repository and reads that flag from `git status --porcelain`, which
# counts UNTRACKED files — so one untracked directory left beside the build is
# what makes a released binary stamp itself "+dirty". /tmp/ is the one that is
# easy to miss: abuild defaults `tmpdir` to "$startdir/tmp" and exports GOTMPDIR
# to it, so every `go build` puts its work directory INSIDE this checkout.
/src/
/pkg/
/tmp/

# `make css` output: the intermediate and the hashed result. It is derived from
# scss/main.scss and a `base` partial that belongs to the deployment's core.sr.ht
# version, so it is build output — the `scss` task of .build.yml materializes the
# partial and the APKBUILD runs `make css` before the compile.
#
# NOTE: web/static/main.min.79713f25.css is still TRACKED in this revision, and a
# tracked file is unaffected by these patterns. Dropping it from the index is the
# second half of this change, and waits on the first CI run confirming that
# `make css` reproduces it — see docs/ci.md#css.
/web/static/main.css
/web/static/main.min.*.css

# Transient build artifacts
*.tmp



M APKBUILD => APKBUILD +54 -7
@@ 4,11 4,25 @@
# repo.bigb.es/alpine/v3.22/bigbes. The srht deployment installs it from there
# instead of cloning and compiling this repo inside its Dockerfile.
#
# pkgver is rewritten by CI to 0.0.<commit count> before abuild runs — a
# monotonic, unique-per-commit version that the deployment can pin. The literal
# below is only what a local `abuild` would use.
# pkgver is READ FROM THE ENVIRONMENT, not sed-ed into this file. The `version`
# task of .build.yml runs one `git describe` and exports PKGVER as whichever of
# three shapes that produces: the tag (`0.2.0`), the tag plus commits since it
# (`0.2.0_git7`, because Alpine's grammar rejects `v0.2.0-7-gabc1234` and `_git`
# sorts after the release), or `0.0.<commit count>` when there are no tags at
# all — which is this repository today. Do not restate one of the three here as
# if it were the rule; see docs/ci.md#version.
#
# Nothing rewrites a tracked file, and that is not a style choice: Go records
# vcs.modified in every binary it builds inside a repository, it reads that flag
# from `git status --porcelain`, and a CI task that patched this line would
# therefore stamp "+dirty" into every packaged binary for the life of the apk.
# Measured on go1.26.5; .gitignore covers the rest of what abuild writes into
# the checkout.
#
# A local `abuild` has no PKGVER and builds 0.0.0, which is what a package built
# by hand honestly is.
pkgname=compare.sr.ht
pkgver=0.0.0
pkgver="${PKGVER:-0.0.0}"
pkgrel=0
pkgdesc="Stateless diff/compare viewer for a sourcehut instance"
url="https://sourcecraft.dev/bigbes/sr-ht-compare"


@@ 32,17 46,50 @@ build() {
	# exports — so CI's cache_restore/cache_save tasks see them survive.
	export GOCACHE="$HOME/.cache/go-build"
	export GOMODCACHE="$HOME/go/pkg/mod"
	# web/static (bundle.js + hashed CSS) is committed and go:embed-ed, so
	# there is no `make css` / `make bundle` step here — see README.
	# CSS strictly before the binary: web/ go:embed-s static/, so a stylesheet
	# built afterwards would never make it into the binary. The shared scss
	# partials are assembled at ASSETS/scss by the `scss` task of .build.yml —
	# no apk ships them.
	#
	# The frontend bundle is NOT built here: web/static/bundle.<hash>.js is
	# committed, and building it would want npm and a network. `make bundle` is
	# an upgrade step, run by hand.
	make css ASSETS=/usr/share/sourcehut

	# web/static/main.min.<hash>.css is still COMMITTED in this revision, and
	# `make css` has just replaced it. These two lines are how the first CI run
	# reports whether the pipeline reproduces the committed bytes: a clean
	# `git status` here means the committed file can be dropped from the tree,
	# and anything else names the difference. See docs/ci.md#css.
	#
	# `|| true` because a diagnostic must never be the reason a build fails.
	git status --porcelain -- web/static || true
	sha256sum web/static/main.min.*.css || true

	# -modcacherw matters beyond convenience: without it the module cache is
	# extracted read-only, and the CI cache tarball made from it can't be
	# unpacked on the next build (mkdir into 0555 dirs fails).
	CGO_ENABLED=0 make build GOFLAGS="-trimpath -modcacherw"

	# `go build` succeeds perfectly well with an unstyled static/ — //go:embed
	# takes the directory, not the file — so a `make css` that produced nothing
	# would ship an unstyled service and fail nothing. This is the gate, and it
	# runs before anything is staged.
	make check-css
}

package() {
	cd "$builddir"
	# The Makefile honours DESTDIR; ASSETS must stay the real runtime path so
	# the binary's static-dir glob resolves after install.
	make install DESTDIR="$pkgdir" PREFIX=/usr ASSETS=/usr/share/sourcehut
	#
	# `install-files` and not `install`, because this function must not compile.
	# abuild runs package() in a FRESH abuild process under fakeroot, which
	# re-sources this file and never calls build() — so the GOCACHE/GOMODCACHE
	# pins above are gone here, $(BIN) is .PHONY and `install` depends on
	# `build`, and `make install` would therefore relink the binary from a cold
	# cache. The apk would then ship a second compilation that nothing in this
	# file has looked at. `install -Dm755` on a missing comparesrht is a fatal
	# error naming the file, so a package() reached without a build() says so.
	make install-files DESTDIR="$pkgdir" PREFIX=/usr ASSETS=/usr/share/sourcehut
}


M Makefile => Makefile +87 -4
@@ 16,6 16,12 @@ STATICDIR?=$(ASSETS)/$(SERVICE)/static

SASSC?=sassc
SASSC_INCLUDE=-I$(ASSETS)/scss
MINIFY?=minify

# The glob web/templates.go resolves at startup (cssGlob). Named once so that
# `css`, which removes the previous build's file, and `check-css`, which counts
# what is left, cannot drift apart.
CSS=web/static/main.min.*.css

all: build



@@ 33,15 39,67 @@ test:
# base.scss and bootstrap/scss there) and scss/main.scss to exist (phase 2b).
# Produces exactly ONE web/static/main.min.<sha256[:8]>.css (old ones and the
# intermediate main.css are removed).
#
# The three preflight checks are here because each of their failures is
# otherwise illegible. A missing minifier stops the pipeline after sassc has
# already written main.css, and the shell's "command not found" names a binary
# rather than a package; a missing $(ASSETS)/scss/base.scss is the family's
# standing rake — NO PACKAGE MANAGER SHIPS THAT PARTIAL, it is materialized by
# core.sr.ht's own `make install` locally and by the `scss` task of .build.yml
# in CI, and sassc's report of it is one line about an import.
css:
	@command -v $(SASSC) >/dev/null 2>&1 || { \
		echo "error: $(SASSC) not found — install sassc (apk add sassc, brew install sassc)"; \
		exit 1; }
	@command -v $(MINIFY) >/dev/null 2>&1 || { \
		echo "error: $(MINIFY) not found — install tdewolff/minify (apk add minify," \
			"go install github.com/tdewolff/minify/v2/cmd/minify@latest)"; \
		exit 1; }
	@[ -f $(ASSETS)/scss/base.scss ] || { \
		echo "error: no $(ASSETS)/scss/base.scss — the shared partial is not packaged;" \
			"materialize it with core.sr.ht's 'make install' (or the scss task of .build.yml)"; \
		exit 1; }
	mkdir -p web/static
	rm -f web/static/main.css web/static/main.min.*.css
	rm -f web/static/main.css $(CSS)
	$(SASSC) $(SASSC_INCLUDE) scss/main.scss web/static/main.css
	minify -o web/static/main.min.css web/static/main.css
	$(MINIFY) -o web/static/main.min.css web/static/main.css
	mv web/static/main.min.css \
		web/static/main.min.$$(sha256sum web/static/main.min.css | cut -c1-8).css
	rm -f web/static/main.css

# check-css is the packaging gate: no stylesheet, no release. It exists because
# the failure it catches is invisible at build time — `go build` succeeds
# perfectly well with an unstyled web/static (//go:embed takes the directory,
# not the file), and the first sign of trouble is an unstyled page in
# production.
#
# It answers about the DISK, and that is the limit of it: nothing here says the
# file it counted is the file a binary embedded.
#
# It COUNTS the matches rather than merely asking whether there are any, because
# two stylesheets are as wrong as none and quieter: web/templates.go resolves
# cssGlob and takes the first match, so a second file makes the served
# stylesheet depend on readdir order. `css` removes the previous build for that
# reason, which is exactly why this gate must not assume it did — a gate that
# promises "exactly one" and checks "at least one" is a promise the next person
# builds on.
#
# `set --` puts the matches in the positional parameters, so the count is $$#
# and no `wc` output has to be parsed. It is fine that this splits on
# whitespace: every name it can see was produced by the recipe above, out of a
# hex digest.
check-css:
	@set -- $$(ls $(CSS) 2>/dev/null); \
	if [ $$# -eq 0 ]; then \
		echo "error: no $(CSS) — run 'make css' before 'go build' (the CSS is embedded)"; \
		exit 1; \
	elif [ $$# -gt 1 ]; then \
		echo "error: $$# files match $(CSS) — web/ takes the first, so the choice is arbitrary."; \
		echo "       run 'make css' to get back to one:"; \
		for f in "$$@"; do echo "         $$f"; done; \
		exit 1; \
	fi

# Frontend diff/tree bundle: built once, output committed to web/static. The
# filename carries a content hash (like the CSS) so a deploy busts the browser
# cache — a stale bundle.js is otherwise served for up to max-age and can leave


@@ 64,9 122,34 @@ bundle:
run-dev: build
	./$(BIN) -b localhost:5090

install: build
install: check-css build
	@$(MAKE) install-files

# The copying half of `install`, with nothing to build in front of it — the
# target a packaging run calls once it has already built and checked the binary,
# so that the files it stages are the very bytes it checked and not a second
# compilation of the same sources.
#
# That distinction is the whole point and it is not theoretical. `build` above is
# .PHONY (Go decides staleness itself, a real file target would never rebuild
# after a source edit), so `install` recompiles; abuild runs package() in a FRESH
# abuild process under fakeroot, which re-sources the APKBUILD and never calls
# build(), so the GOCACHE/GOMODCACHE pins are gone and that recompilation comes
# from a cold cache. The packaged binary would be one nothing had inspected.
#
# It is invoked through a sub-make rather than listed as a third prerequisite of
# `install` on purpose: /usr/share/abuild/default.conf exports
# MAKEFLAGS=-j$(nproc), prerequisites of one target run in parallel under -j, and
# `install: check-css build install-files` would let the copying start beside the
# build it is supposed to follow. A recipe line always runs after the
# prerequisites are done.
#
# There is no check in front of the copies. `install -Dm755 comparesrht` on a
# missing file is already a fatal error naming the file, which is exactly the
# right report for the one way this target can be called too early.
install-files:
	install -Dm755 $(BIN) $(DESTDIR)$(BINDIR)/$(BIN)
	mkdir -p $(DESTDIR)$(STATICDIR)
	install -Dm644 -t $(DESTDIR)$(STATICDIR) web/static/*

.PHONY: all build test css bundle run-dev install
.PHONY: all build test css check-css bundle run-dev install install-files


A docs/ci.md => docs/ci.md +346 -0
@@ 0,0 1,346 @@
# CI: what `.build.yml` does and why

The manifest is deliberately short on prose. builds.sr.ht stores a submitted
manifest in a `varchar(16384)`, so a manifest that grows past 16 KiB cannot be
submitted at all — the failure is at submission time and reads like nothing in
particular: the branch simply has no CI. Rationale therefore lives here, and the
manifest carries pointers.

The pipeline is one linear job on `alpine/edge`: install the cache helper,
assemble the shared SCSS, make a throwaway signing key, decide the version,
restore the Go caches, package with `abuild`, publish the apk to
`repo.bigb.es/alpine/v3.22/bigbes`, save the caches.

It is triggered by a push to the **sourcehut** side. A push to sourcecraft
cannot reach builds.sr.ht; the gitsync mirror is what puts the commit on
`git.srht.bigb.es`, and that push is what submits the job. So the mirror is on
the critical path for the package repository, not merely an offsite copy.

## packages

`sassc` and `minify` are for `make css`, not for the package: the stylesheet is
compiled here and embedded into the binary, and the apk has no runtime
dependency on either.

`curl` is for the `cacher_install` bootstrap and `rclone` for `publish`. There is
no `nodejs`/`npm`: `web/static/bundle.<hash>.js` is committed, and `make bundle`
is an upgrade step run by hand rather than a build step.

There is no `postgresql`. This service is stateless — it holds no database at
all — and its suites are hermetic: `gitx/fixture_test.go` builds a bare
repository with the local `git`, and `web/web_test.go` is `httptest` plus
`ecoretest`. Nothing in the tree talks to a live git.sr.ht API.

## secrets

Three, all account-level and shared with the sibling services:

| secret | lands at | used by |
|---|---|---|
| `apk-ci-s3` | `~/.apk-ci.env` | `publish` |
| `7dde4219-…` | `~/.s3-cache-key-id` | `cacher_init` |
| `0e5b3530-…` | `~/.s3-cache-key-secret` | `cacher_init` |

They are file secrets. Listing them is what turns `publish` and the cache tasks
on; a manual submission that asks for no secrets still runs the interesting part
of the pipeline and stops at `cacher_init`, which is the right place to notice.

`apk-ci-s3` is referenced by name and the other two by UUID, which is only
because that is how they were written in the donor manifests; both forms work.

## environment

`CORE_VER` **must track the deployment's `SRHT_CORE_VER`**, and `BOOTSTRAP_REV`
is the Bootstrap submodule commit `core.sr.ht` pins at that tag. The two being
out of step means this service renders against different partials than the rest
of the instance, which shows up as a page that is subtly the wrong shape and as
nothing at all in any log. They are `0.84.5` / `779ad9f1` here because that is
what `srht/versions.env` pins in the deployment repo; bump all of them together.

`REPO` is the clone directory `sources:` produces, and every task that touches
the checkout `cd`s into it — task bodies do not inherit a working directory from
each other.

## cacher

`cacher` (`go.bigb.es/cacher`) is an S3-backed cache helper, installed from its
own release on pages.sr.ht.

It is **two tasks and not one** because `install.sh` appends its `PATH` export
to `~/.buildenv`, and `~/.buildenv` is sourced by the *next* task's preamble — a
single task would install the binary and then not find it on `PATH`.

The installer is fetched with `curl -fsSL` and verifies what it downloads
against `checksums.txt`. The predecessor of this task fetched the raw binary
with a bare `curl -sSL`; without `-f`, an HTTP error is not an error to `curl`,
so a 404 page was written to `~/.local/bin/cacher` and `chmod +x`-ed, and the
first failure was a syntax error out of a shell reading HTML.

## scss

`sassc` needs the shared sourcehut partials, and **no apk ships them**: they are
core.sr.ht's own `scss/` plus the Bootstrap 4 tree it pins as a submodule. The
task clones both and drops them in `/usr/share/sourcehut/scss`, which is the
`ASSETS` the Makefile and the APKBUILD name.

The whole thing is cached under a key made of the two pins
(`scss/$CORE_VER-$BOOTSTRAP_REV.tar.zst`), so an outage at git.sr.ht or GitHub
cannot fail a build that changed nothing. `--exec` runs only on a cache miss and
seeds the cache afterwards; the block sees exported variables only, which is why
the cache key is spelled inline in the `cacher` invocation rather than computed
inside the block, and why the block is in single quotes.

It runs before `keygen` and before anything Go, because the stylesheet has to
exist before the compiler does — see [css](#css).

## keygen

`abuild-keygen -a -n -i -q` makes a throwaway package signing key.

`-i` is not optional: without it the key is generated but not installed into
`/etc/apk/keys`, and `abuild`'s own final "update the local repository index"
step then dies with an UNTRUSTED signature after having built the package
perfectly well.

The key is per-build and dies with the VM. That is fine because nothing verifies
these signatures — the apk index on `repo.bigb.es` is rebuilt and signed on
phoebe by the garage stack's `apk-mirror` service, which indexes this repo with
`--allow-untrusted` precisely because of this.

## version

One `git describe` decides the apk's `pkgver`. Alpine's version grammar does not
accept `v1.2.3-4-gabc1234`, so a describe with commits since the tag is
rewritten to `1.2.3_git4`; `_git` sorts **after** the release in Alpine's
comparison, which is what makes an untagged master build look newer than the tag
it follows. With no tags at all — which is this repository today — it falls back
to `0.0.<commit count>`, monotonic and unique per commit, which a deployment can
pin.

Tags are in `allow-refs` for exactly this reason: pushing `v0.9.0` is what
produces the `0.9.0` apk.

`$ver` is **exported** through `~/.buildenv` as `PKGVER`, and the APKBUILD reads
`pkgver="${PKGVER:-0.0.0}"`. It is not `sed`-ed into the tracked APKBUILD, and
this is the one thing in this file that must not be "tidied" back:

> Go records `vcs.modified` in every binary it builds inside a repository, and
> it reads that flag from `git status --porcelain`. Measured on go1.26.5, a
> `sed -i` of a tracked file sets `vcs.modified=true` and Go stamps `+dirty`
> into `Main.Version` for the whole `abuild` run — for every binary in the
> package, permanently, for the life of the apk.

The export alone is not enough. Any **untracked** file present at `go build` time
does the same thing, and `abuild` works inside the checkout with
`builddir="$startdir"` and exports `GOTMPDIR=$startdir/tmp` — so `./src`,
`./pkg` and `./tmp` all appear inside the tree it is packaging. `.gitignore`
covers those three, and Go does not count ignored files.

The task ends with a bare `git status --porcelain` because this is the last
moment the tree is provably clean, and the output of a clean one is nothing.

## cache_restore

The Go module cache and build cache are keyed by the hash of `go.sum`, so a
build that changed no dependency reuses both.

`--optional` and not `|| true`. They are not the same promise: `--optional`
returns 0 only for a genuine cache miss, while `|| true` swallows every other
outcome too — bad credentials, a dead bucket, a typo in the prefix — and turns a
misconfigured cache into a pipeline that is merely slow, silently, forever.

The exports here are only for the repair block below and for `cache_save`:
`abuild` redirects both caches into its throwaway `$tmpdir` (and an upstream typo
in `abuild.in` slaves `GOMODCACHE` to `GOCACHE`), so environment exports from a
task cannot stick. The APKBUILD's `build()` re-pins both to the same home
locations, after abuild's own exports, which is what makes the tarballs saved at
the end be the ones the compile used.

### The half-restored module cache

The repair block is not defensive padding. A partially restored module cache is
the *normal* failure mode of this arrangement — an interrupted upload, a
truncated object, a key written while a build was still running — and what it
produces is a compile error deep in a dependency, which reads exactly like a bug
in the code under test.

So: make the tree writable (the module cache is mode 555 and `rm -rf` cannot
remove it otherwise), ask `go mod verify` whether what came back is intact, and
throw the whole thing away if it is not. Then `go mod download all` and verify
for real, and let *that* failure be fatal.

Do not soften the final `go mod verify` to `|| true`. A build that proceeds with
a module cache it could not verify is a build whose result means nothing.

The second `git status --porcelain` is there because `-mod=readonly` is the
default and neither command *can* rewrite `go.mod` or `go.sum` — but a dirty
tree at this point is a `+dirty` apk, so it is said out loud rather than trusted
to a flag.

## test

`gofmt -l .` piped to a file and then `test ! -s`, rather than a bare `gofmt -l`:
`gofmt -l` exits 0 whether or not it printed anything, so the only way to fail on
its output is to look at the output. The `tee` keeps the offending filenames in
the build log, where they are the entire diagnosis.

`go vet ./...` next, then `make test` — the Makefile, not a bare `go test ./...`,
so that the test command lives in one place.

No Postgres and no network. Every suite in this tree is hermetic:
`gitx/fixture_test.go` builds a bare repository with the local `git` binary, and
`web/web_test.go` is `httptest` plus `ecoretest`. The APKBUILD's `!check` is
about not running these a second time inside `abuild`, not about them needing
something the VM lacks.

## build

`REPODEST=$HOME/packages abuild -d` builds and stages the apk.

`-d` disables abuild's dependency check: the makedepends are already installed
by the manifest's `packages:` list, and abuild has no way to know that.

`builddir="$startdir"` in the APKBUILD means abuild packages *this checkout in
place* rather than unpacking a tarball. There is no tarball to unpack — the
package is built from the commit under test, which has not been released
anywhere yet.

`build()` runs `make css` before the compile and `make check-css` after it, and
`package()` runs `make install-files` rather than `make install`. Both are
explained under [css](#css) and [install-files](#install-files).

## publish

The `[ ! -r ~/.apk-ci.env ]` gate is the honest answer to a build that was handed
no secrets, not a fallback: on a manual submission without the secret, every
earlier task has still run and a signed apk is sitting in `$HOME/packages`, and
saying so and exiting 0 is more useful than a red build about a credential
nobody asked for. On a push the secret is there and this publishes.

`set +x` before sourcing `~/.apk-ci.env` and `set -x` after. Every task runs
under `set -x`, so without this the S3 access key and secret are echoed into a
build log that is world-readable.

The upload is `rclone copyto` per file — copy only, never delete, never sync. A
`sync` would mirror local absence onto the bucket, and the bucket holds every
previously published version of every service on the instance, which is what
lets a pinned deployment be rebuilt. abuild nests its output under
`$REPODEST/<repo>/<arch>/`, so each file is copied to a fixed prefix rather than
the tree being mirrored.

`apk-mirror` on phoebe re-indexes within 15 minutes; nothing here waits for it.

## cache_save

**After `publish`**, deliberately: a cache upload that fails must not strand an
apk that was built and tested successfully but never shipped.

Fatal on purpose, though — an upload failure here means the next build pays for
a cold cache, and that is worth knowing about rather than hiding behind
`|| true`.

There is no `cacher exists "$KEY" ||` guard in front of either upload. It was
dead code: without `--force`, `cacher dir upload` already skips a key that is
present, so the guard asked the same question twice and its only real effect was
to make a failure of the `exists` call look like a decision not to upload.

## css

The stylesheet is compiled from `scss/main.scss` against the shared partials and
is embedded into the binary by `//go:embed static` (`web/templates.go`), so it
has to exist **before** the compiler runs. Hence `make css` at the top of
`build()` and a fatal `make check-css` after the compile.

`check-css` counts the matches of `web/static/main.min.*.css` and fails on zero
*and* on two. Two is as wrong as none and quieter: `web/templates.go` resolves
`cssGlob` and takes the first match, so a second file makes the served
stylesheet depend on readdir order.

### The committed stylesheet, and the two steps

`web/static/main.min.79713f25.css` is **still tracked** in this revision. The end
state is the one the siblings have — the file is build output, ignored, and
produced by CI — but getting there is two steps and this is the first.

Step one, this change: CI builds the partials, `make css` runs in `build()`,
`check-css` gates the result, and the committed file stays in the index. The
`.gitignore` patterns are already in place and do nothing to it, because a
tracked file is unaffected by `.gitignore`.

`build()` therefore prints two lines right after `make css`:

```
git status --porcelain -- web/static || true
sha256sum web/static/main.min.*.css || true
```

Step two, after the first CI run, is decided by those two lines:

- **Empty `git status` output** — `make css` reproduced the committed bytes
  exactly, including the content hash in the filename. Nothing about the shipped
  service changes when the file leaves the index.
- **A `D` line for `main.min.79713f25.css` plus an untracked
  `main.min.<other>.css`** — the pipeline produces a different stylesheet than
  the one committed. That is the interesting case and it wants reading before it
  is dropped: a hash difference is either drift in `CORE_VER`/`BOOTSTRAP_REV`
  against whatever the committer had materialized locally, or a genuine change in
  `scss/main.scss` that was never recompiled. The `sha256sum` line names the new
  file so the comparison does not depend on scrolling the log.

Either way step two is `git rm --cached web/static/main.min.79713f25.css` and
nothing else — the ignore patterns are already there. Do it in the first case
straight away; in the second, reconcile the versions first, because dropping the
committed file makes CI the only producer and whatever it produces is what
ships.

Until step two lands, note that `make css` deletes a tracked file during
`build()`, so the tree is dirty from that point on in the mismatch case. It
costs nothing today — this service reads no VCS stamp of its own — but it is the
reason step two should not sit around.

## install-files

`package()` calls `make install-files`, not `make install`.

`abuild` runs `package()` in a **fresh abuild process** under fakeroot, which
re-sources the APKBUILD and never calls `build()`. So nothing `build()` exported
reaches it — the `GOCACHE`/`GOMODCACHE` pins in particular. `build` in the
Makefile is `.PHONY` (Go decides staleness itself; a real file target would never
rebuild after a source edit), and `install` depends on `build`, so `make install`
there recompiles the binary from a cold cache. The apk would then ship a second
compilation that nothing in the pipeline had looked at.

`install-files` is the copying half with no build in front of it: it stages the
very bytes `build()` produced and `check-css` was asked about. `install -Dm755`
on a missing `comparesrht` is a fatal error naming the file, which is the right
report for the one way that target can be called too early.

`install` itself keeps the sub-make (`@$(MAKE) install-files`) rather than
listing it as a third prerequisite: `/usr/share/abuild/default.conf` exports
`MAKEFLAGS=-j$(nproc)`, and under `-j` the prerequisites of one target run in
parallel — the copying would start beside the build it is supposed to follow.

## What is not here

- **No test task in the same commit as the version fix.** They are two commits
  on purpose: a red suite must not be able to hold up the packaging change.
- **No coverage upload.** The sibling services POST their profile to cover.sr.ht
  at the end of the pipeline. Adding it here needs a token secret and a
  repository on cover.
- **No artifacts.** There is nothing to download: the apk goes to S3, and its
  name changes every commit, which `artifacts:` cannot express (it has no
  globbing).
- **No matrix.** One architecture, one image.

## Known wart

`make install-files` still runs
`install -Dm644 -t $(DESTDIR)$(STATICDIR) web/static/*`, which stages
`bundle.<hash>.js` and `main.min.<hash>.css` into
`/usr/share/sourcehut/compare.sr.ht/static`. The binary already **embeds** both
(`//go:embed static` in `web/templates.go:25`) and serves them from there, so
those copies are dead weight in the apk and a second, divergable source of truth
for the same bytes. The bench sibling installs no static assets at all for this
reason. Left alone deliberately — it is not part of this change, and removing it
wants a check that nothing in the deployment's nginx tree serves that directory
directly.