From a574cffb871d0b4324494880ee589fb41e76ba58 Mon Sep 17 00:00:00 2001
From: Eugene Blikh <bigbes@gmail.com>
Date: Fri, 7 Aug 2026 00:50:35 +0300
Subject: [PATCH] README, landing page, CHANGELOG: document --exec, --optional,
 install.sh

Rewrites the directory-caching and docker sections around --exec instead
of the shell branch, replaces the bootstrap task in the full builds.sr.ht
example with the install.sh one-liner, and records that the --exec
script's exit status is propagated verbatim in the exit-code table.
---
 CHANGELOG.md    |  24 +++++++++++
 README.md       | 108 ++++++++++++++++++++++++++++++++++--------------
 docs/index.html |  12 +++++-
 3 files changed, 112 insertions(+), 32 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2b8b5dd667fef19fb8914e4c468c60983812c5dd..b9fd65c5a54b52d376d8aaf2b7bdb699f720db28 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,30 @@ project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+### Added
+- `--exec <script>` on `download`, `dir download` and `docker download` —
+  on a cache miss the script runs through `sh -c` and its result seeds
+  the cache. Generalises the `--url` and `--pull` fallbacks to anything
+  expressible as a shell command, which collapses the
+  restore-or-build `if`/`fi` block CI manifests repeat per cache:
+
+  ```sh
+  cacher dir download "gomod/{hash}.tar.zst" ~/go/pkg/mod \
+    --hash-from go.sum --exec 'go mod download'
+  ```
+
+  The destination directory is created before the script runs; the seed
+  upload is best-effort; a failing script is fatal and propagates its own
+  exit status.
+- `--optional` on `download` and `dir download` — a cache miss exits 0
+  instead of 1/3, so `set -e` manifests no longer need a trailing
+  `|| echo "cache miss"`.
+- `install.sh`, published next to the binaries: detects the platform,
+  verifies against `checksums.txt`, installs into `~/.local/bin` and, on
+  builds.sr.ht, appends the `PATH` export to `~/.buildenv`. Reduces the
+  bootstrap task to
+  `curl -sSL https://bigbes.pages.srht.bigb.es/ci-cacher/install.sh | sh`.
+
 ## [0.1.2] — 2026-05-26
 
 ### Changed
diff --git a/README.md b/README.md
index 01f084532bcf01e0a5fcb29867c6834485ec2c5a..91e4f33b075615e34e93b6a790f2f04a57543512 100644
--- a/README.md
+++ b/README.md
@@ -28,10 +28,25 @@ Same collapse for docker images:
 cacher docker download "$key" "$image:tag" --pull
 ```
 
+…and for a directory you build yourself, where the fallback is your own
+build script:
+
+```sh
+cacher dir download "$key" ~/scss --exec 'git clone … && cp -r …'
+```
+
 Releases and changelog: [bigbes.pages.srht.bigb.es/ci-cacher](https://bigbes.pages.srht.bigb.es/ci-cacher/).
 
 ## Install
 
+```sh
+# One-liner: detects the platform, verifies against checksums.txt,
+# installs into ~/.local/bin (override with CACHER_BINDIR). On
+# builds.sr.ht it also appends the PATH export to ~/.buildenv, so the
+# whole bootstrap task is this single line.
+curl -sSL https://bigbes.pages.srht.bigb.es/ci-cacher/install.sh | sh
+```
+
 ```sh
 # Pre-built binary (latest tag). Substitute your platform:
 #   cacher-linux-amd64, cacher-linux-arm64,
@@ -87,6 +102,22 @@ Cache HIT: pulls from S3. Cache MISS: GETs the URL, verifies the optional
 sha256, writes to the destination, **and uploads to S3** so the next run
 hits.
 
+When the artifact isn't a URL away but a command away, `--exec` is the
+same shape — the script runs only on a miss and must leave the
+destination behind, which is then cached:
+
+```sh
+cacher download "bin/tool-$VER" ~/.local/bin/tool \
+  --exec 'go build -o ~/.local/bin/tool ./cmd/tool'
+```
+
+`--optional` turns a miss into exit 0 instead of an error, for caches
+whose absence just means a cold build:
+
+```sh
+cacher dir download "$KEY_GOC" ~/.cache/go-build --optional
+```
+
 ```sh
 cacher upload my-key /path/to/artifact            # skip if present
 cacher upload my-key /path/to/artifact --force    # overwrite
@@ -108,8 +139,16 @@ cacher docker download "docker/dxflrs-garage-v2.3.0.tar.zst" \
   dxflrs/garage:v2.3.0 --pull
 ```
 
-For images you build locally, drive the cache by hand — same primitives,
-key by Dockerfile content:
+For images you build locally, `--exec` replaces `--pull` — key by
+Dockerfile content, and the build only runs on a miss:
+
+```sh
+cacher docker download "images/{hash}.tar.zst" myimage:latest \
+  --hash-from Dockerfile \
+  --exec 'docker build -t myimage:latest .'
+```
+
+The primitives are still there if you want the branch by hand:
 
 ```sh
 KEY=$(cacher key "images/{hash}.tar.zst" --hash-from Dockerfile)
@@ -134,18 +173,30 @@ when nothing changed:
 
 ```sh
 # Go module cache, keyed by go.sum:
-KEY=$(cacher key "go-mod/{hash}.tar.zst" --hash-from go.sum)
-cacher dir download "$KEY" ~/go/pkg/mod 2>/dev/null || {
-  go mod download
-  cacher dir upload "$KEY" ~/go/pkg/mod
-}
+cacher dir download "go-mod/{hash}.tar.zst" ~/go/pkg/mod \
+  --hash-from go.sum --exec 'go mod download'
 
 # Lua rocks tree, keyed by rockspec:
-KEY=$(cacher key "rocks/{hash}.tar.zst" --hash-from project-scm-1.rockspec)
-cacher dir download "$KEY" .rocks 2>/dev/null || {
-  tt rocks install ...
-  cacher dir upload "$KEY" .rocks
-}
+cacher dir download "rocks/{hash}.tar.zst" .rocks \
+  --hash-from project-scm-1.rockspec --exec 'tt rocks install ...'
+```
+
+`--exec` runs only on a miss, with `sh -c`, stdout and stderr wired
+straight to the build log. The destination directory is created before
+the script starts, so the script can write into it without its own
+`mkdir -p`. On success the tree is packed and uploaded; if the upload
+fails the build still continues, because the content the script produced
+is already on disk. If the script itself fails, `cacher` exits with the
+script's own status and nothing is cached.
+
+When restore and save can't be one command — because the tree is only
+final several tasks later, as with a Go build cache — use the pair:
+
+```sh
+KEY=$(cacher key "gocache/{hash}.tar.zst" --hash-from go.sum)
+cacher dir download "$KEY" ~/.cache/go-build --optional  # miss is fine
+... build, test, package ...
+cacher dir upload "$KEY" ~/.cache/go-build               # no-op if present
 ```
 
 ### Key derivation
@@ -189,10 +240,7 @@ sources:
   - https://git.example.com/myproject
 tasks:
   - install_cacher: |
-      mkdir -p ~/.local/bin
-      curl -sSL https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64 \
-        -o ~/.local/bin/cacher
-      chmod +x ~/.local/bin/cacher
+      curl -sSL https://bigbes.pages.srht.bigb.es/ci-cacher/install.sh | sh
   - cacher_init: |
       cacher init \
         --endpoint https://s3.example.com --region garage --bucket cache \
@@ -203,14 +251,9 @@ tasks:
         --url https://go.dev/dl/go1.26.3.linux-amd64.tar.gz
       mkdir -p ~/.local && tar -xzf /tmp/go.tar.gz -C ~/.local
   - cache_gomod: |
-      KEY="gomod/$(sha256sum myproject/go.sum | cut -c1-16).tar.gz"
-      if cacher download "$KEY" /tmp/gomod.tar.gz; then
-        mkdir -p ~/go && tar -xzf /tmp/gomod.tar.gz -C ~/go
-      else
-        cd myproject && go mod download && cd ..
-        tar -czf /tmp/gomod.tar.gz -C ~/go pkg/mod
-        cacher upload "$KEY" /tmp/gomod.tar.gz
-      fi
+      cacher dir download "gomod/{hash}.tar.zst" ~/go/pkg/mod \
+        --hash-from myproject/go.sum \
+        --exec 'cd myproject && go mod download'
   - cache_postgres: |
       cacher docker download "docker/postgres-16.tar.zst" \
         postgres:16 --pull
@@ -258,12 +301,13 @@ default and works identically on MinIO and AWS S3:
 
 ## Exit codes
 
-| Code | Meaning                                                   |
-|------|-----------------------------------------------------------|
-| `0`  | Success                                                   |
-| `1`  | `exists` returned false / key missing                     |
-| `2`  | Operational error (credentials, network, permission, …)  |
-| `3`  | `download` cache miss with no `--url` fallback            |
+| Code | Meaning                                                    |
+|------|------------------------------------------------------------|
+| `0`  | Success (including a miss under `--optional`)              |
+| `1`  | `exists` returned false / key missing                      |
+| `2`  | Operational error (credentials, network, permission, …)   |
+| `3`  | `download` cache miss with no `--url`/`--exec` fallback     |
+| *n*  | The `--exec` script's own exit status, propagated verbatim |
 
 So shell can branch on `exists` cleanly:
 
@@ -291,7 +335,9 @@ handled by the shell version at all — `cacher dir` closes that gap.
 
 `--pull` on `cacher docker download` collapses the docker HIT/MISS
 branch into one call; the equivalent for files exists too via
-`cacher download --url`.
+`cacher download --url`, and `--exec` generalises both to any fallback
+you can express as a shell command — which is what removes the last
+`if`/`fi` block from a CI manifest.
 
 ## License
 
diff --git a/docs/index.html b/docs/index.html
index 68efb0cfbd96666f7e36d256a6ed034128552fb2..6327a3900f386c2dfc68949e846fef7860f7bb17 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -107,7 +107,15 @@
   </tbody>
 </table>
 <p>
-  All four hashes plus filenames are also available in a single
+  The <a href="install.sh">install.sh</a> bootstrap picks the right asset for
+  the host, verifies it against <a href="checksums.txt">checksums.txt</a>, and
+  installs into <code>~/.local/bin</code> — on builds.sr.ht it also appends the
+  <code>PATH</code> export to <code>~/.buildenv</code>, so a CI manifest needs
+  exactly one line:
+</p>
+<pre><code>curl -sSL https://bigbes.pages.srht.bigb.es/ci-cacher/install.sh | sh</code></pre>
+<p>
+  By hand, all four hashes plus filenames are also available in a single
   <a href="checksums.txt">checksums.txt</a> for piping into <code>sha256sum -c</code>:
 </p>
 <pre><code>wget https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64 \
@@ -135,6 +143,8 @@ cacher download "$key" "$out" --url "$url"</code></pre>
   <li><code>exists</code> / <code>list</code> / <code>delete</code> / <code>key</code> — management + shell helpers</li>
   <li><code>docker {exists,download,upload}</code> — streamed save/load via zstd</li>
   <li><code>dir {download,upload}</code> — tar+zstd directory caching, keyed by content hash</li>
+  <li><code>--exec</code> on every <code>download</code> — run a build script on a miss
+      and seed the cache with what it produced; <code>--optional</code> makes a miss exit 0</li>
 </ul>
 <p>
   Run <code>cacher --help</code> for the full surface. Read