~bigbes/ci-cacher

bca3f572b28a6f4a945d92c2020b981af4cf58b4 — Eugene Blikh a day ago 9dc739e master
README: refresh for v0.1.1+v0.1.2

* Lead-in mentions 'cacher docker download --pull' alongside the
  existing 'cacher download --url' as the canonical one-liners.
* Docker section now shows the --pull cache-or-pull form first; the
  exists/upload/download dance kept for locally-built images.
* New 'Full builds.sr.ht example' section — bootstrap-cacher,
  init, install_go, cache_gomod, cache_postgres, test. Mirrors the
  shape of this repo's .builds/test.yml.
* List subcommand: --recursive and --root flags documented.
* Install: name all four published platforms; link checksums.txt.
* Releases/changelog link to bigbes.pages.srht.bigb.es/ci-cacher.
* Drop the 'directory caching is brand new' line — shipped in v0.1.0.
1 files changed, 84 insertions(+), 7 deletions(-)

M README.md
M README.md => README.md +84 -7
@@ 22,14 22,24 @@ fi
cacher download "$key" "$out" --url "$url"
```

Same collapse for docker images:

```sh
cacher docker download "$key" "$image:tag" --pull
```

Releases and changelog: [bigbes.pages.srht.bigb.es/ci-cacher](https://bigbes.pages.srht.bigb.es/ci-cacher/).

## Install

```sh
# Pre-built linux-amd64 binary (latest tag):
# Pre-built binary (latest tag). Substitute your platform:
#   cacher-linux-amd64, cacher-linux-arm64,
#   cacher-darwin-amd64, cacher-darwin-arm64
wget https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64 -O ~/.local/bin/cacher
chmod +x ~/.local/bin/cacher

# Pin to a specific build by sha256 (printed on the publish.yml job page):
# Pin to a known sha256 (checksums.txt lives next to the binaries):
wget https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64 -O ~/.local/bin/cacher
echo "<expected-sha256>  ~/.local/bin/cacher" | sha256sum -c



@@ 37,6 47,8 @@ echo "<expected-sha256>  ~/.local/bin/cacher" | sha256sum -c
go install go.bigb.es/cacher@latest
```

Verify with `cacher version`.

## Setup

`cacher init` writes `~/.config/cacher/config.toml` and runs a smoke test


@@ 79,14 91,27 @@ hits.
cacher upload my-key /path/to/artifact            # skip if present
cacher upload my-key /path/to/artifact --force    # overwrite
cacher exists my-key                              # exit 0 hit, 1 miss
cacher list   my-prefix                           # debug
cacher list   my-prefix                           # /-delimited (aws s3 ls)
cacher list   my-prefix --recursive               # flat
cacher list   --root                              # ignore configured prefix
cacher delete my-key                              # invalidate
```

### Docker images — streamed save/load

For an image pulled from a registry, the cache-or-pull pattern is a
single command:

```sh
# Cache HIT → docker load from S3. MISS → docker pull + seed S3 + tag stays local.
cacher docker download "docker/dxflrs-garage-v2.3.0.tar.zst" \
  dxflrs/garage:v2.3.0 --pull
```

For images you build locally, drive the cache by hand — same primitives,
key by Dockerfile content:

```sh
# Build, cache, and reuse a docker image keyed by its Dockerfile content:
KEY=$(cacher key "images/{hash}.tar.zst" --hash-from Dockerfile)

if ! cacher docker exists "$KEY"; then


@@ 145,6 170,54 @@ Substitution into the key template:
without doing anything else — handy when the same key feeds multiple
subsequent commands.

## Full builds.sr.ht example

Realistic CI flow: bootstrap the cacher binary itself from the previous
release, then dogfood it to cache the Go toolchain, module cache, and a
docker image before running tests. See this repo's
[`.builds/test.yml`](.builds/test.yml) for the live version.

```yaml
image: ubuntu/noble
packages: [curl, ca-certificates, docker.io]
secrets:
  - <s3-key-id-secret-uuid>
  - <s3-secret-key-secret-uuid>
environment:
  PATH: /home/build/.local/go/bin:/home/build/.local/bin:/usr/local/bin:/usr/bin:/bin
sources:
  - https://git.example.com/myproject
tasks:
  - install_cacher: |
      mkdir -p ~/.local/bin
      curl -sSL https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64 \
        -o ~/.local/bin/cacher
      chmod +x ~/.local/bin/cacher
  - cacher_init: |
      cacher init \
        --endpoint https://s3.example.com --region garage --bucket cache \
        --prefix myproject \
        --key-file ~/.s3-cache-key-id --secret-file ~/.s3-cache-key-secret
  - install_go: |
      cacher download "golang/go1.26.3.linux-amd64.tar.gz" /tmp/go.tar.gz \
        --url https://go.dev/dl/go1.26.3.linux-amd64.tar.gz
      mkdir -p ~/.local && tar -xzf /tmp/go.tar.gz -C ~/.local
  - cache_gomod: |
      KEY="gomod/$(sha256sum myproject/go.sum | cut -c1-16).tar.gz"
      if cacher download "$KEY" /tmp/gomod.tar.gz; then
        mkdir -p ~/go && tar -xzf /tmp/gomod.tar.gz -C ~/go
      else
        cd myproject && go mod download && cd ..
        tar -czf /tmp/gomod.tar.gz -C ~/go pkg/mod
        cacher upload "$KEY" /tmp/gomod.tar.gz
      fi
  - cache_postgres: |
      cacher docker download "docker/postgres-16.tar.zst" \
        postgres:16 --pull
  - test: |
      cd myproject && go test ./...
```

## Configuration

`~/.config/cacher/config.toml`:


@@ 212,9 285,13 @@ The shell version this replaced repeated five things in every CI task:
4. Branch HIT/MISS by hand.
5. For docker, pipe `docker save | zstd | aws s3 cp -` (and the inverse).

`cacher` does (1) by being a single 14 MB static binary that gets
fetched with one `wget`, and (2)-(5) as built-in commands. The directory
caching is brand new — the shell version only handled single files.
`cacher` does (1) by being a single ~14 MB static binary fetched with
one `wget`, and (2)–(5) as built-in commands. Directories aren't
handled by the shell version at all — `cacher dir` closes that gap.

`--pull` on `cacher docker download` collapses the docker HIT/MISS
branch into one call; the equivalent for files exists too via
`cacher download --url`.

## License