~bigbes/sr-ht-ecore

54025f42346afbf561683c1d32c321ea875a421d — Eugene Blikh 2 days ago 92fea80 
ci: test, coverage and benchmarks on builds.sr.ht

This repository had no CI while seven services pinned it, so a break here
was discovered in whichever service next ran `go get -u`. The manifest is
the sibling services' pipeline with everything only a service needs taken
out: no scss, no keygen, no abuild, no publish, no publish_artifacts, and
no postgres — nothing in this module imports database/sql, no test opens a
database and no test skips on a missing DSN. What is left is the Go cache,
the gates, the two runs and the two uploads to cov.srht.bigb.es and
bench.srht.bigb.es.

The Makefile is what the manifest calls, so the Go command lines exist
once. Its bench target filters its own output, and that is not tidiness:
`go test` merges the test binary's stderr into stdout, bearer's
invalid-token benchmark makes sr-ht-core's auth.DecodeBearerToken log once
per iteration, and an unfiltered BENCH_COUNT=10 run was 901 MB and 9.5
million lines deep — around the 290 result lines it produces — when it was
stopped. benchfmt ignores what it cannot parse, so that file would have
uploaded "successfully". Filtered, the same run is 340 lines and 31 KB.
The cure belongs in the benchmark, which should silence the logger the way
middleware's discardLog already does; the filter is the pipeline's defence
either way.

check-fmt and not gofmt, because `gofmt -l` prints the offending files and
still exits 0; cover refuses an empty profile and the bench task greps for
one benchmark name per benchmark file, because both would otherwise upload
nothing and call it a success.
patch browse .tar.gz .zip 
2 files changed, 405 insertions(+), 0 deletions(-)

A .build.yml
A Makefile

A .build.yml => .build.yml +223 -0
@@ 0,0 1,223 @@
# builds.sr.ht manifest for sr-ht-ecore. One linear pipeline: restore the Go
# caches, refuse an unformatted or vet-dirty tree, test with a coverage profile,
# run the benchmarks, then upload both to the instance's own cov.sr.ht and
# bench.sr.ht.
#
# This repository is a LIBRARY — no cmd/, no daemon, no stylesheet, no apk — and
# seven services pin it by pseudo-version, which is exactly why it needs a
# pipeline of its own: a break here is discovered in whichever service next runs
# `go get -u`, at the moment that service is being released. Everything a
# sibling manifest carries for its package (scss, keygen, abuild, publish,
# publish_artifacts) has no meaning here and is absent rather than stubbed, and
# there is no postgres task because nothing in this module imports database/sql:
# no test opens a database, no test skips on a missing DSN.
#
# The reasoning lives in comments here rather than in a docs/ci.md, because this
# repository has no docs/ tree. Keep it under 16 KiB: builds.sr.ht stores the
# submitted manifest in a varchar(16384), and a manifest over that cannot be
# submitted at all — the failure is a branch with no CI, not a red build.
image: alpine/edge
packages:
  - go
  - git
  # Every task below drives the Makefile. The sibling manifests never list this
  # and their `make test` works, so the image evidently ships one — but abuild's
  # 14 dependencies do not include it (checked on pkgs.alpinelinux.org), so what
  # they rely on is the image's furniture rather than a package they asked for.
  # A build that needs make asks for make.
  - make
  # For the cacher's install.sh and for the two uploads, and for nothing else.
  - curl
secrets:
  # S3 credentials for the cacher CI cache (Garage `docker-cache` bucket).
  # Account-level secrets shared with the sibling services, so this repository
  # holds nothing of its own. Absent, cacher_install turns the cache off and the
  # build compiles cold instead of failing.
  - 7dde4219-0783-4581-a67d-c94749de3600   # ~/.s3-cache-key-id
  - 0e5b3530-6f19-4f30-9b73-9339dd382e46   # ~/.s3-cache-key-secret
  # ~/.srht-token holds a tokens.sr.ht WORKING TOKEN, and it is one secret shared
  # with the cov.sr.ht and bench.sr.ht pipelines rather than a per-repository
  # one: the credential is minted once, for a person, and carries the grants of
  # every service it is meant to reach. This build needs cov:upload and
  # bench:upload from it; a token missing one fails that upload and no other.
  # The grant was cover:upload before the service was renamed to cov.sr.ht, and
  # grants are compared literally, so a token minted before the rename uploads
  # no coverage here — re-mint it on tokens.sr.ht and replace this secret.
  - c7968415-1a6d-4ca0-a188-150fb7f57b65   # ~/.srht-token
sources:
  - https://git.srht.bigb.es/~bigbes/sr-ht-ecore
environment:
  REPO: sr-ht-ecore
  COVER_ORIGIN: https://cov.srht.bigb.es
  COVER_REPO: "~bigbes/sr-ht-ecore"
  BENCH_ORIGIN: https://bench.srht.bigb.es
  BENCH_REPO: "~bigbes/sr-ht-ecore"
# Literal paths relative to $HOME — which is why the two tasks below write there
# and not into the checkout. They are not a fallback for the uploads: a build
# submitted without secrets still leaves both files downloadable, and a POST
# that failed leaves the body that was meant to be sent.
artifacts:
  - cover.out
  - bench.txt
submitter:
  git.sr.ht:
    allow-refs:
      - refs/heads/master
      # A library is consumed by tag as well as by pseudo-version, so a pushed
      # tag has to be tested too. Without this line it starts no build at all.
      - "refs/tags/v*"
tasks:
  # S3-backed CI cache helper. install.sh's PATH export goes to ~/.buildenv,
  # which only the NEXT task sources — hence two tasks and not one.
  #
  # The guard is not in the donors' copies of this task, and it is here because
  # this repository is a library: a contributor's manually submitted build asks
  # for no secrets, and cacher init would then die on an unreadable key file
  # before a single test had run. Without the cache the build is slower and
  # exactly as truthful.
  - cacher_install: |
      if [ ! -r ~/.s3-cache-key-id ] || [ ! -r ~/.s3-cache-key-secret ]; then
        echo "no S3 cache credentials: this build compiles from cold"
        echo "export NO_CACHE=1" >> ~/.buildenv
        exit 0
      fi
      curl -sSL https://bigbes.pages.srht.bigb.es/ci-cacher/install.sh | sh
  - cacher_init: |
      [ -z "$NO_CACHE" ] || { echo "cache disabled: nothing to init"; exit 0; }
      cacher init \
        --endpoint    https://s3.bigb.es \
        --region      garage \
        --bucket      docker-cache \
        --prefix      sr-ht-ecore/deps \
        --key-file    ~/.s3-cache-key-id \
        --secret-file ~/.s3-cache-key-secret
  - cache_restore: |
      cd "$REPO"
      if [ -z "$NO_CACHE" ]; then
        # Module and build caches keyed by go.sum; --optional makes a miss a
        # cold build, not an error.
        KEY_MOD=$(cacher key "gomod/{hash}.tar.zst" --hash-from go.sum)
        KEY_GOC=$(cacher key "gocache/{hash}.tar.zst" --hash-from go.sum)
        echo "export KEY_MOD=$KEY_MOD KEY_GOC=$KEY_GOC" >> ~/.buildenv
        cacher dir download "$KEY_MOD" ~/go/pkg/mod      --optional
        cacher dir download "$KEY_GOC" ~/.cache/go-build --optional
        # Repair block for the HALF-restored module cache: it is the normal
        # failure of a restore, not a freak one, and it surfaces later as
        # compile errors inside a dependency that read like a code bug
        # (cover.sr.ht's build #284). Do not soften this to `|| true`.
        chmod -R u+w ~/go/pkg/mod 2>/dev/null || true
        if ! go mod verify >/dev/null 2>&1; then
          echo "restored module cache did not verify — discarding it"
          rm -rf ~/go/pkg/mod
        fi
      fi
      # `go mod download`, NEVER `go mod download all`. The `all` pattern
      # resolves the whole module graph including dependencies' test-only
      # requirements and APPENDS their hashes to the TRACKED go.sum — silently,
      # exit 0 — which leaves a modified file in the checkout. -mod=readonly
      # does NOT prevent it: readonly governs the module requirements, not
      # writes to go.sum. Without `all`, go.sum is untouched and what the cache
      # holds is what this module builds and tests anyway.
      go mod download
      go mod verify
      # And the proof that it did not: a print here would be a flag we trust,
      # so the clean tree is asserted instead. The `all` spelling fails this
      # line with a go.sum of a few hundred added hashes.
      test -z "$(git status --porcelain)" || { git status --porcelain; exit 1; }
  - lint: |
      cd "$REPO"
      # check-fmt and not fmt: `gofmt -l` prints the offending files and STILL
      # EXITS 0, so a task that ran plain gofmt could not fail, and `make fmt`
      # rewrites — a gate that edits the tree it is judging is not a gate.
      make check-fmt
      make vet
  - test: |
      cd "$REPO"
      # -covermode=atomic (real hit counts, which is what cov.sr.ht reads) and
      # $HOME, because that is where artifacts: looks. The Makefile owns the
      # command; this task owns the destination. `make cover` also refuses an
      # empty profile, which would otherwise upload a report covering nothing
      # and call it a success.
      make cover COVERPROFILE="$HOME/cover.out"
  - bench: |
      cd "$REPO"
      # -s so make does not echo the recipe into the body; a redirect and a cat
      # and NOT `| tee`, which would hand this task tee's exit status and let a
      # failing benchmark pass.
      #
      # What lands in the file is benchfmt and nothing else because the Makefile
      # filters it (see BENCH_FILTER there): a benchmark that provokes logging
      # in the code under test writes those lines into this same stream, and one
      # of the benchmarks in this tree currently produces hundreds of megabytes
      # of them. The command that ran is echoed to stderr, so it is in this
      # task's log and not in the body being uploaded.
      make -s bench > "$HOME/bench.txt"
      cat "$HOME/bench.txt"
      # `go test -bench` that matches nothing prints `ok` and exits 0, and a
      # file with no benchmark lines is still valid benchfmt — so a renamed or
      # deleted benchmark would upload an empty run and report success. One
      # name per benchmark file, so that losing any one file is caught:
      grep -q '^BenchmarkValidate'      "$HOME/bench.txt"   # bearer
      grep -q '^BenchmarkRequestLogger' "$HOME/bench.txt"   # chimw
      grep -q '^BenchmarkRequire'       "$HOME/bench.txt"   # csrf
      grep -q '^BenchmarkParse'         "$HOME/bench.txt"   # grants
      grep -q '^BenchmarkChain'         "$HOME/bench.txt"   # middleware
  - cache_save: |
      [ -z "$NO_CACHE" ] || { echo "cache disabled: nothing to save"; exit 0; }
      # After the runs that warm it, and fatal on purpose. Without --force an
      # upload skips a key that is already there, so no `cacher exists` guard is
      # needed.
      cacher dir upload "$KEY_MOD" ~/go/pkg/mod
      cacher dir upload "$KEY_GOC" ~/.cache/go-build
  - coverage: |
      cd "$REPO"
      # The gate is the honest answer to a build that was handed no secrets: the
      # profile is made, it is this build's cover.out artifact, and it can be
      # POSTed by hand. With the file present the upload is fatal on purpose.
      if [ ! -r ~/.srht-token ]; then
        echo "no ~/.srht-token: this build has no cov.sr.ht credentials"
        echo "the profile is still available as this build's cover.out artifact"
        exit 0
      fi
      # GIT_REF is absent on a manually submitted build and ref is optional for
      # the API; key is the idempotency key, so a resubmitted job replaces its
      # own report instead of adding a second one. Both prefixes are stripped
      # because this pipeline builds tags too, and a tag build would otherwise
      # report ref=refs/tags/v0.1.0.
      ref="${GIT_REF#refs/heads/}"
      ref="${ref#refs/tags/}"
      url="$COVER_ORIGIN/api/v1/repos/$COVER_REPO/reports"
      url="$url?commit=$(git rev-parse HEAD)&ref=$ref&key=$JOB_ID&job_url=$JOB_URL"
      echo "uploading cover.out to $url"
      # Tracing off to the end of the task: the Authorization header must not
      # reach the log. No Content-Type — the service sniffs the format, and a
      # wrong one is a 400. --fail-with-body prints the JSON error AND still
      # exits non-zero, which plain --fail does not.
      set +x
      curl -sS --fail-with-body -X POST \
        -H "Authorization: Bearer $(cat ~/.srht-token)" \
        --data-binary "@$HOME/cover.out" \
        "$url"
      echo
  - bench_upload: |
      cd "$REPO"
      if [ ! -r ~/.srht-token ]; then
        echo "no ~/.srht-token: this build has no bench.sr.ht credentials"
        echo "the benchmarks ran and are in the bench task's log"
        echo "the file is this build's bench.txt artifact, and can be POSTed by hand"
        exit 0
      fi
      # visibility acts only on the POST that creates $BENCH_REPO; on every
      # later run it is ignored.
      ref="${GIT_REF#refs/heads/}"
      ref="${ref#refs/tags/}"
      url="$BENCH_ORIGIN/api/v1/repos/$BENCH_REPO/runs"
      url="$url?commit=$(git rev-parse HEAD)&ref=$ref&key=$JOB_ID&job_url=$JOB_URL"
      url="$url&visibility=public"
      echo "uploading bench.txt to $url"
      set +x
      curl -sS --fail-with-body -X POST \
        -H "Authorization: Bearer $(cat ~/.srht-token)" \
        --data-binary "@$HOME/bench.txt" \
        "$url"
      echo


A Makefile => Makefile +182 -0
@@ 0,0 1,182 @@
# sr-ht-ecore — build scaffolding (bench.sr.ht / cov.sr.ht family style).
#
# This repository is a LIBRARY: no cmd/, no daemon, no stylesheet, no apk. There
# is nothing to install and nothing to package, so the targets a service's
# Makefile carries for those — css, keygen, install, check-version — have no
# meaning here and are absent rather than stubbed.
#
# What is left is what the seven services pinning this module actually need from
# it: that it compiles, that it is formatted and vets clean, that its tests pass,
# and that its coverage profile and its benchmarks come out in the shape
# .build.yml uploads to cov.srht.bigb.es and bench.srht.bigb.es. That is what the
# targets below are, and .build.yml calls them by name — the Go command lines
# live here, once, instead of being copied into the manifest.

GO?=go
GOFMT?=gofmt

# PKG is what to run and COUNT is `go test -count`, both variables so that
# `make test-race COUNT=10 PKG=./bearer/` needs no retyping of the flags that
# get forgotten when a command is retyped by hand.
PKG?=./...
COUNT?=1

# TEST_TIMEOUT is the per-package ceiling handed to `go test`, and -timeout is
# per test binary, so what it has to cover is the slowest package and not the
# suite. `make test` does not need it: 11 s wall for the whole tree on an M4
# Pro, measured with go1.25 at -count=1, no package over 10 s. `make bench`
# is where it earns its keep — measured on that same machine at BENCH_COUNT=10,
# 6 min 06 s wall over 29 benchmark cases, of which grants took 96.4 s, bearer
# 78.5 s and csrf 73.0 s. A sixth of the toolchain's ten-minute default is
# comfortable on a laptop and not obviously comfortable on a builds.sr.ht VM,
# which is a slower machine running the same fixed number of iterations.
# Twenty minutes is the number the sibling services carry, for the same reason:
# what it prevents — `panic: test timed out` over three hundred lines of
# goroutine dump — reads like a hang in the code rather than like a budget.
TEST_TIMEOUT?=20m

# BENCH_COUNT is `go test -count` for the `bench` target. Ten is bench.sr.ht's
# own number (its SPEC ch. 5): a confidence interval becomes finite at six
# repetitions and a comparison becomes significant at four, so anything under
# six uploads points the service can only mark "low n". It is a variable so a
# laptop can say `make bench BENCH_COUNT=1` when it only wants to know that the
# benchmarks still run.
BENCH_COUNT?=10

# Where `make cover` writes the profile. .build.yml overrides it to $HOME —
# that is where builds.sr.ht's artifacts: looks — and the default is the
# checkout, which is what a developer means by `make cover`. It lands untracked
# there; `make clean` is what removes it, and .gitignore does not list it.
COVERPROFILE?=cover.out

# The sibling services default to `build`, and each of them says in the same
# breath that it was `help` while there was nothing to build. Here there is
# nothing to build — `build` is a compile check, not a product — so the default
# stays where theirs started, and a bare `make` spends no time proving something
# the author did not ask about.
.DEFAULT_GOAL := help

help:
	@echo "targets:"
	@echo "  help       this list (the default target)"
	@echo "  build      compile every package: go build \$$(PKG)"
	@echo "  test       go test \$$(PKG) with an explicit -timeout"
	@echo "  test-race  the same under -race and repeatable:"
	@echo "               make test-race COUNT=10 PKG=./bearer/"
	@echo "  cover      test with -covermode=atomic into \$$(COVERPROFILE) and"
	@echo "             print the total; the profile .build.yml POSTs to cov.sr.ht"
	@echo "             (COVERPROFILE=$(COVERPROFILE))"
	@echo "  bench      go test -bench=. \$$(PKG) as benchfmt on stdout, the format"
	@echo "             .build.yml POSTs to bench.sr.ht (BENCH_COUNT=$(BENCH_COUNT));"
	@echo "             everything that is not benchfmt is filtered out — see"
	@echo "             BENCH_FILTER, and 'make bench BENCH_FILTER=.' for the raw run"
	@echo "  vet        go vet \$$(PKG)"
	@echo "  fmt        rewrite this module's Go files with gofmt"
	@echo "  check-fmt  the same as a refusal: fail if gofmt would rewrite anything"
	@echo "             (this is the one CI runs; fmt writes, gates must not)"
	@echo "  clean      remove \$$(COVERPROFILE)"

# There is no binary to link, so this is a compile check and nothing else. It is
# still worth a target: `go vet` compiles too, but a build failure reported by
# vet reads like a vet finding.
build:
	$(GO) build $(PKG)

test:
	$(GO) test -timeout $(TEST_TIMEOUT) $(PKG)

# The same suite under the race detector, repeatable. Half of what this library
# holds is middleware sitting on every request of every service — chimw's
# logger, middleware's chain, mcphttp's cache — so the run that matters for it
# is this one, not `test`.
test-race:
	$(GO) test -race -count=$(COUNT) -timeout $(TEST_TIMEOUT) $(PKG)

# -covermode=atomic and not the default `set`: `set` records whether a statement
# ran, atomic records how often, and cov.sr.ht reads hit counts. The profile is
# the artifact; the printed total is a convenience.
#
# The `test -s` in the middle is not decoration. A profile that came out empty
# is still a valid file, and uploading it would report success over a report
# that covers nothing — the same failure the .build.yml bench task greps
# against.
cover:
	$(GO) test -covermode=atomic -coverprofile=$(COVERPROFILE) -timeout $(TEST_TIMEOUT) $(PKG)
	@test -s $(COVERPROFILE) || { echo "$(COVERPROFILE) is empty" >&2; exit 1; }
	$(GO) tool cover -func=$(COVERPROFILE) | tail -1

# The benchmarks in the form bench.sr.ht ingests: benchfmt on stdout, which is
# what `go test -bench` writes. -run='^$$' keeps the tests out of it — their
# output is noise the parser skips and their runtime is time added to a run
# whose point is the benchmarks. -benchmem because B/op and allocs/op are half
# of what a middleware benchmark means and they cost nothing to collect.
#
# BENCH_FILTER is the part that is not boilerplate, and it is here because a
# benchmark's own stdout is not only the benchmark. `go test` merges the test
# binary's stderr into its stdout, so anything the code under test logs during a
# measured loop lands in the middle of the benchfmt document. Measured on this
# tree at BENCH_COUNT=10: bearer's invalid-token case makes sr-ht-core's
# auth.DecodeBearerToken call log.Printf once per iteration, and an unfiltered
# run was 901 MB and 9.5 million lines deep — around the 290 result lines a full
# run has — when it was stopped, a third of the way through. Filtered, the same
# run is 340 lines and 31 KB. benchfmt ignores lines it cannot parse, so the
# unfiltered file would upload "successfully": a POST of a gigabyte of noise,
# whatever bench.sr.ht's byte budget makes of it.
#
# So the target emits the benchfmt grammar (its configuration lines and its
# result lines) plus the words that say a run failed, and nothing else. The
# filter is a variable: `make bench BENCH_FILTER=.` is the same run with every
# line of it. It is a defence and not a cure — the cure is for a benchmark that
# provokes logging to silence the logger first, the way middleware's discardLog
# and chimw's io.Discard handler already do.
#
# The exit status travels in a file because the grep is a pipe: taking the
# pipeline's status would report grep's opinion of the output instead of whether
# the benchmarks ran, which is the failure mode that makes a red suite read
# green.
BENCH_FILTER?=^(goos|goarch|pkg|cpu): |^Benchmark|^(PASS|FAIL|ok|--- |panic: )

bench:
	@echo "+ $(GO) test -run='^$$' -bench=. -benchmem -count=$(BENCH_COUNT) -timeout $(TEST_TIMEOUT) $(PKG)" >&2
	@st=$$(mktemp); \
	{ $(GO) test -run='^$$' -bench=. -benchmem -count=$(BENCH_COUNT) \
		-timeout $(TEST_TIMEOUT) $(PKG) 2>&1; echo $$? > "$$st"; } \
		| grep -E '$(BENCH_FILTER)' || true; \
	rc=$$(cat "$$st"); rm -f "$$st"; \
	test -n "$$rc" || rc=1; \
	exit "$$rc"

vet:
	$(GO) vet $(PKG)

# GOFMT_DIRS is `go list`'s answer and not a literal `.`, and that is the whole
# point of the two targets below. The repository convention here is to create
# git worktrees under .worktrees/<branch>, so a plain `gofmt -l .` at the root
# would walk into another checkout of this same module and report — or, for
# `fmt`, rewrite — files that are not in this working tree at all. A worktree
# carries its own go.mod, so it is a different module to `go list`, and ./...
# never leaves this one.
GOFMT_DIRS=$$($(GO) list -f '{{.Dir}}' $(PKG))

fmt:
	$(GOFMT) -l -w $(GOFMT_DIRS)

# `gofmt -l` prints the offending files and STILL EXITS 0, so it passes any &&
# chain and any CI task that only looks at the status. This target is that check
# written so a failure stops the build, and it is the one .build.yml calls; fmt
# above rewrites and must never be what a gate runs.
check-fmt:
	@out=$$($(GOFMT) -l $(GOFMT_DIRS)); \
	if [ -n "$$out" ]; then \
		echo "gofmt would rewrite:" >&2; \
		echo "$$out" >&2; \
		exit 1; \
	fi; \
	echo "gofmt: clean"

# Only the profile: `bench` writes to stdout and creates no file of its own, so
# there is nothing else here to remove.
clean:
	rm -f $(COVERPROFILE)

.PHONY: help build test test-race cover bench vet fmt check-fmt clean