Add install.sh one-line bootstrap, published next to the binaries
Every manifest repeated the same four lines to get the binary: mkdir,
curl, chmod, and a PATH export into ~/.buildenv. install.sh does all of
it, picks the asset for the host platform, and verifies it against the
published checksums.txt — which the hand-rolled version never did.
On builds.sr.ht ~/.buildenv is sourced before each task, so the PATH
export lands from the next task onwards; keep install and `cacher init`
in separate tasks.
The checksums.txt glob stays cacher-*: install.sh must not appear in the
manifest it verifies against.
cmd: add --exec fallback and --optional to the download commands
--exec generalises the existing --url (file) and --pull (docker)
fallbacks to anything expressible as a shell command: on a cache miss
the script runs through sh -c and whatever it produced seeds the cache.
That collapses the restore-or-build if/fi block CI manifests repeat
around every cache into a single invocation.
The destination directory is created before the script runs, so the
script needs no mkdir -p of its own. The seed upload stays best-effort
(the content is already on disk), but a failing script is fatal and
propagates its own exit status rather than a generic 2. For docker the
tag is verified with docker image inspect afterwards — with a
best-effort upload, a script exiting 0 without building the image would
otherwise pass silently and fail a later task.
--optional turns a cache miss into exit 0 for download and dir
download, so set -e manifests drop the trailing || echo "cache miss".
README: refresh for v0.1.1+v0.1.2
* Lead-in mentions 'cacher docker download --pull' alongside the
existing 'cacher download --url' as the canonical one-liners.
* Docker section now shows the --pull cache-or-pull form first; the
exists/upload/download dance kept for locally-built images.
* New 'Full builds.sr.ht example' section — bootstrap-cacher,
init, install_go, cache_gomod, cache_postgres, test. Mirrors the
shape of this repo's .builds/test.yml.
* List subcommand: --recursive and --root flags documented.
* Install: name all four published platforms; link checksums.txt.
* Releases/changelog link to bigbes.pages.srht.bigb.es/ci-cacher.
* Drop the 'directory caching is brand new' line — shipped in v0.1.0.
Bump VERSION to 0.1.2; widen landing page to 55rem
test.yml: collapse cache_garage_image to single 'cacher docker download --pull'
Now that v0.1.1 (with --pull) is the published binary on pages, the
if/else with exists/download/upload can fold into the one-line form.
SilenceUsage in v0.1.1 also removes the cobra-dump-on-miss noise that
required the stderr swallow on the previous `exists` check.
Bump VERSION to 0.1.1 for tag
test.yml: use published cacher binary instead of building from source
Walking back e1b71b2. Building cacher from source dropped the
go-tarball caching (chicken-and-egg). Bootstrapping from the published
v0.1.0 binary instead lets all three caches (go tarball, gomod, garage
image) use cacher.
Cost: cache_garage_image stays on the if/else cacher docker
{exists,download,upload} pattern instead of the single `cacher docker
download --pull` introduced on master, because v0.1.0 doesn't have
--pull yet. Comment notes the switch as a follow-up after the next
release.
test.yml: build cacher from source, cache go.mod, use 'docker download --pull'
* install_cacher now goes via 'go build' from the cloned source
instead of curl-from-pages. The whole point of test.yml is to
exercise the binary this branch produces; bootstrapping from the
previously-published release defeats that. Loses go-tarball caching
as a side effect (chicken-and-egg: need go before cacher).
* cache_gomod task between cacher_init and test: tar ~/go/pkg/mod
keyed by sha256(go.sum). Skips proxy.golang.org on subsequent runs;
go.sum change invalidates automatically. Biggest cache win — the
docker SDK transitives pulled in by testcontainers are heavy.
* cache_garage_image collapsed to a single 'cacher docker download
--pull' call. Key prefix changed garage/ → docker/ (cache format
is now zstd-compressed via cacher docker, incompatible with the
raw-tar entry seeded by job 153).
* Dropped the redundant smoke 'build' task — install_cacher already
exercises 'go build' from the same sources.
cacher: add 'docker download --pull' and silence cobra usage on errors
* docker download --pull: on cache miss, falls back to docker pull
+ docker save + S3 seed, mirroring the --url fallback on file
download. Collapses the if/else cache-or-pull bash dance in CI
manifests to a single command.
* SilenceUsage on rootCmd: cache-miss / key-not-found exits aren't
bad-usage errors, so the cobra Usage block dump on every error was
noise. Short error line remains.
merge .builds/{unit,e2e}.yml into test.yml; cache go + garage image
Single job runs unit tests → smoke build → e2e (against a real Garage
container via testcontainers-go). Unit failure short-circuits before
paying the ~150 MB Garage image pull.
Caches via dogfooded cacher:
* Go tarball — `cacher download --url` fallback, same pattern as
publish.yml.
* Garage docker image — `cacher download` of the tar; on miss does
docker pull + save + upload to seed.
Also silences testcontainers' default logger in the garage testutil
(pull progress, container IDs, port mappings); test output now only
shows what the tests print themselves. `-v` dropped from the e2e
`go test` since per-test progress is no longer drowned out and not
worth printing on green runs.
`TESTCONTAINERS_RYUK_DISABLED=true` — VM is torn down at job end,
reaper container is just startup noise.
Verified end-to-end on job #153 (all 8 tasks ✔, cache MISS seeded).
publish.yml: restore artifacts block now that storage works
Job 148 reached the upload stage where 30347e5 didn't — admin has
configured the artifact storage backend, so the four cacher binaries
+ checksums.txt are useful again as a 90-day-TTL view on the job page
alongside the permanent pages.sr.ht copy.
publish.yml: dogfood cacher for deps, drop failing artifacts block
Two changes:
* Drop `artifacts:` block. The srht.bigb.es self-hosted instance
doesn't have artifact storage configured (the upload step fails
silently right after "Running task stage_artifacts"). Pages.sr.ht
already serves the binaries with stable URLs, so artifacts were
just redundant. Removes the stage_artifacts task too.
* Cache the Go and goreleaser tarballs through cacher itself.
Bootstrap: curl the previously-published cacher binary from pages;
cacher init against s3.bigb.es using the existing srht-registered
secrets (same UUIDs as tarantool-protobuf .builds); then cacher
download "goreleaser/<ver>.tar.gz" --url "https://github…" handles
cache-or-fetch + back-fill. First run after this commit pays the
full curl cost and seeds s3; every later run is an S3 hit.
hut isn't cached — no upstream binary release, building from
source is small + fast.
publish.yml: override hut pages origin to public URL
The srht.bigb.es OAuth worker writes the internal docker-compose
hostname (http://pages:5112) into ~/.config/hut/config, which is
unreachable from outside the cluster. sed the config to use the
public origin (https://pages.srht.bigb.es) before publish.
This is a srht admin-side gotcha; only `builds` and `meta` get
public origins in the worker config, the other services
(pages/git/hub/todo/paste) all keep internal hostnames.
publish.yml: goreleaser --skip=validate
When a CI manifest patch lands after tagging, the version tag won't
point at HEAD. --skip=validate lets goreleaser proceed with the
already-tagged version anyway. Safe because the binary built from
HEAD vs the tag is byte-identical (only publish.yml differs).
publish.yml: mkdir ~/.local/bin before extracting goreleaser
Caught by job 143 install_goreleaser step — install_go creates
~/.local but not ~/.local/bin, so the tar -C target didn't exist.
publish.yml: hut version (not --version)
Caught by job 141 install_hut step. `hut` doesn't expose a --version
flag; the subcommand is `hut version`. `go install` had succeeded
silently and the failure was just the version probe.
Bump VERSION to 0.1.0 for tag
Add goreleaser + CHANGELOG.md; cross-platform downloads on site
* .goreleaser.yml — builds linux/{amd64,arm64} and darwin/{amd64,arm64}
as raw binaries (formats: binary) with CGO_ENABLED=0, trimpath, version
ldflag injection. GitHub release disabled; we publish via pages.sr.ht.
* CHANGELOG.md — Keep-a-Changelog format. v0.1.0 entry summarizes the
full surface (init/doctor, single-file/docker/dir caching, --hash-from,
Garage compat, e2e suite, CI manifests).
* .builds/publish.yml — installs goreleaser binary release (pinned to
v2.7.0), runs `goreleaser release --clean --skip=publish`, flattens
the 4 dist/cacher_<os>_<arch>_v<n>/cacher paths into
pages/cacher-<os>-<arch>, regenerates checksums.txt to match the
published filenames, renders CHANGELOG.md to HTML via cmark with
heading demotion (h1→drop, h2→h3, h3→h4), substitutes per-platform
sha256s into docs/index.html, and ships everything to
bigbes.pages.srht.bigb.es/ci-cacher/ plus the build artifacts list.
* docs/index.html — Download section is now a 4-row table with per-
platform sha256s, plus a Changelog section that embeds the rendered
CHANGELOG.md inline.
Add docs/index.html landing page; publish.yml substitutes build info
Replaces the inline 4-line index.html written in publish.yml with a
proper text-mostly landing page under docs/index.html — sections for
what it is, install, the loop it replaces, commands, why, and a
'this build' block populated at publish time from VERSION, SHA-256,
and build timestamp via sed.
Single-column, ~44rem max width, system fonts, monospace for code,
dark-mode via prefers-color-scheme. No JS, no images, no remote
assets. ~130 lines of HTML+CSS.
Fix pages.sr.ht target: bigbes.pages.srht.bigb.es/ci-cacher
cacher.srht.bigb.es was a guess and isn't configured. Use the
user-level pages domain (bigbes.pages.srht.bigb.es) with hut's
`-s /ci-cacher` subpath flag so this manifest only touches files
under that prefix — leaves room for other projects under the same
domain without clobbering.