~bigbes/ci-cacher

ref: e1b71b2fe9f2df0fd94378351a68d97a01879922 ci-cacher/docs/index.html -rw-r--r-- 6.7 KiB
e1b71b2f — Eugene Blikh test.yml: build cacher from source, cache go.mod, use 'docker download --pull' a day ago 
                                                                                
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>cacher — S3-backed CI cache helper</title>
<style>
  :root {
    --fg: #1a1a1a;
    --muted: #666;
    --link: #0a5a9c;
    --bg: #fdfdfd;
    --code-bg: #f3f3f0;
    --rule: #d8d8d4;
  }
  @media (prefers-color-scheme: dark) {
    :root {
      --fg: #e6e6e6;
      --muted: #9a9a9a;
      --link: #6bbfff;
      --bg: #161616;
      --code-bg: #222;
      --rule: #333;
    }
  }
  * { box-sizing: border-box; }
  html, body { background: var(--bg); color: var(--fg); }
  body {
    max-width: 44rem;
    margin: 2rem auto;
    padding: 0 1.25rem 4rem;
    font: 16px/1.55 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, system-ui, sans-serif;
  }
  h1, h2, h3, h4 { line-height: 1.25; }
  h1 { font-size: 1.7rem; margin-bottom: 0.15rem; }
  h2 { font-size: 1.15rem; margin-top: 2.2rem; padding-bottom: .2rem; border-bottom: 1px solid var(--rule); }
  h3 { font-size: 1rem; margin-top: 1.4rem; }
  h4 { font-size: .95rem; margin-top: 1rem; color: var(--muted); }
  table { border-collapse: collapse; width: 100%; margin: .7rem 0; font-size: .92em; }
  th, td { text-align: left; padding: .35rem .6rem; border-bottom: 1px solid var(--rule); }
  th { color: var(--muted); font-weight: 600; }
  td code { font-size: .85em; word-break: break-all; }
  .changelog ul { padding-left: 1.25rem; }
  .changelog li { margin: .25rem 0; }
  .changelog h2 { font-size: 1.05rem; margin-top: 1.4rem; }
  .changelog h3 { font-size: .95rem; margin-top: 1rem; color: var(--muted); border-bottom: none; }
  p, ul, ol { margin: .7rem 0; }
  a { color: var(--link); text-decoration: none; }
  a:hover { text-decoration: underline; }
  code, pre, kbd { font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, monospace; font-size: .92em; }
  code { background: var(--code-bg); padding: 1px 4px; border-radius: 3px; }
  pre { background: var(--code-bg); padding: .75rem 1rem; overflow-x: auto; border-radius: 4px; }
  pre code { background: none; padding: 0; }
  .tagline { color: var(--muted); margin-top: 0; }
  .meta { color: var(--muted); font-size: .9em; }
  .meta dt { float: left; clear: left; width: 6.5rem; }
  .meta dd { margin: 0 0 .15rem 6.5rem; word-break: break-all; }
  hr { border: 0; border-top: 1px solid var(--rule); margin: 2rem 0; }
  footer { color: var(--muted); font-size: .85em; margin-top: 3rem; }
</style>
</head>
<body>

<h1>cacher</h1>
<p class="tagline">S3-backed CI cache helper. A single static Go binary.</p>

<h2>What it is</h2>
<p>
  <code>cacher</code> is the build that the typical "check S3, fall back to
  upstream, upload for next time" shell loop should have been. It downloads,
  uploads, lists, and invalidates cached artifacts in any S3-compatible
  bucket — single files, docker images (via streamed
  <code>save | zstd | s3</code>), and whole directory trees keyed by lockfile
  hash. Built for <a href="https://builds.sr.ht">builds.sr.ht</a>; works
  anywhere you can run a binary and reach an S3 endpoint.
</p>

<h2>Download</h2>
<p>
  Pre-built binaries for {{VERSION}}. The shipping URL is stable; the
  SHA-256s below are specific to this build — paste them into your
  pin if you care.
</p>
<table>
  <thead><tr><th>Platform</th><th>Binary</th><th>SHA-256</th></tr></thead>
  <tbody>
    <tr>
      <td>linux/amd64</td>
      <td><a href="cacher-linux-amd64">cacher-linux-amd64</a></td>
      <td><code>{{SHA_LINUX_AMD64}}</code></td>
    </tr>
    <tr>
      <td>linux/arm64</td>
      <td><a href="cacher-linux-arm64">cacher-linux-arm64</a></td>
      <td><code>{{SHA_LINUX_ARM64}}</code></td>
    </tr>
    <tr>
      <td>darwin/amd64</td>
      <td><a href="cacher-darwin-amd64">cacher-darwin-amd64</a></td>
      <td><code>{{SHA_DARWIN_AMD64}}</code></td>
    </tr>
    <tr>
      <td>darwin/arm64</td>
      <td><a href="cacher-darwin-arm64">cacher-darwin-arm64</a></td>
      <td><code>{{SHA_DARWIN_ARM64}}</code></td>
    </tr>
  </tbody>
</table>
<p>
  All four hashes plus filenames are also available in a single
  <a href="checksums.txt">checksums.txt</a> for piping into <code>sha256sum -c</code>:
</p>
<pre><code>wget https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64 \
  -O ~/.local/bin/cacher
chmod +x ~/.local/bin/cacher
wget -qO- https://bigbes.pages.srht.bigb.es/ci-cacher/checksums.txt \
  | sha256sum -c --ignore-missing</code></pre>

<h2>The loop it replaces</h2>
<pre><code># before — install awscli, write ~/.aws/config, then in every task:
if aws s3api head-object --bucket "$B" --key "$K" &gt;/dev/null 2&gt;&amp;1; then
  aws s3 cp "s3://$B/$K" "$out"
else
  curl -sSL "$url" -o "$out"
  aws s3 cp "$out" "s3://$B/$K"
fi</code></pre>

<pre><code># after — one binary, one config, one command:
cacher download "$key" "$out" --url "$url"</code></pre>

<h2>Commands</h2>
<ul>
  <li><code>init</code> / <code>doctor</code> — persist config + smoke-test creds</li>
  <li><code>download</code> / <code>upload</code> — single file, with <code>--url</code> fallback and <code>--sha256</code> verify</li>
  <li><code>exists</code> / <code>list</code> / <code>delete</code> / <code>key</code> — management + shell helpers</li>
  <li><code>docker {exists,download,upload}</code> — streamed save/load via zstd</li>
  <li><code>dir {download,upload}</code> — tar+zstd directory caching, keyed by content hash</li>
</ul>
<p>
  Run <code>cacher --help</code> for the full surface. Read
  <a href="https://git.srht.bigb.es/~bigbes/ci-cacher/tree/master/README.md">the
  README</a> for usage patterns, Garage compatibility notes, and exit-code semantics.
</p>

<h2>Why</h2>
<p>
  The shell version this replaced repeated five things in every CI task:
  install AWS CLI v2 (≈50 MB per build), write a Garage-tuned
  <code>~/.aws/config</code>, compute cache keys from file content with
  <code>sha256sum&nbsp;|&nbsp;cut</code>, branch HIT/MISS by hand, and for
  docker images pipe <code>docker save | zstd | aws s3 cp -</code> (and the
  inverse). <code>cacher</code> collapses all of that into one fetched binary
  plus a config file. The directory caching is the genuinely new capability —
  the shell version only ever handled single files.
</p>

<h2>Changelog</h2>
<section class="changelog">
{{CHANGELOG}}
</section>

<h2>This build</h2>
<dl class="meta">
  <dt>Version</dt>      <dd>{{VERSION}}</dd>
  <dt>Built</dt>        <dd>{{BUILT}}</dd>
  <dt>Source</dt>       <dd><a href="https://git.srht.bigb.es/~bigbes/ci-cacher">git.srht.bigb.es/~bigbes/ci-cacher</a></dd>
  <dt>License</dt>      <dd>BSD-2-Clause</dd>
</dl>

<footer>
  Published via <a href="https://man.sr.ht/pages.sr.ht/">pages.sr.ht</a> by
  <a href="https://git.srht.bigb.es/~bigbes/ci-cacher/tree/master/.builds/publish.yml">.builds/publish.yml</a>.
</footer>

</body>
</html>