~bigbes/sourcehut-root

ref: 10a4d6690e13796b4c37be3c3f4ec93a364cd803 sourcehut-root/skills/sourcehut-ci/references/hut.md -rw-r--r-- 15.5 KiB
10a4d669 — Eugene Blikh Add sourcehut-ci skill backup and justfile install target 6 days ago

#hut — the SourceHut CLI in builds.sr.ht

hut (https://git.sr.ht/~xenrox/hut) is the official-in-practice SourceHut CLI: a single Go binary that talks GraphQL to every sr.ht service (builds, git, hg, hub, lists, meta, pages, paste, todo). It is not part of the ~sircmpwn/sourcehut umbrella — it lives under ~xenrox — but builds.sr.ht has first-class support for it, and most useful CI integrations (publishing pages, downloading artifacts, submitting follow-up builds, applying patchsets) go through hut.

This file documents what is and is not true about running hut from inside a builds.sr.ht job. For local/interactive hut usage, the upstream man page (hut.1.scd) is the better reference.

#How hut authenticates in CI — the real mechanism

Common misconception: "hut picks up $OAUTH2_TOKEN from the environment automatically." That's wrong. hut only reads its token from a config file ($XDG_CONFIG_HOME/hut/config, normally ~/.config/hut/config) — there is no env-var fallback in hut's code.

What actually happens: when a manifest has oauth: set and secrets are enabled for the build, the builds.sr.ht worker:

  1. Mints a short-lived OAuth 2.0 bearer token with exactly the requested grants, expiring at the build deadline.
  2. Writes /home/build/.config/hut/config with mode 0600, containing:
    • instance "<site-name>" (from the running sr.ht instance's sr.ht/site-name config — default if unset)
    • access-token "<the minted token>"
    • A <service> { origin "<service-origin>" } block for every <name>.sr.ht section that has api-origin (preferred) or origin in the instance's config — except man.sr.ht, which is excluded (the worker has a TODO to add it once hut supports it).
  3. Exports $OAUTH2_TOKEN in ~/.buildenv with the same token, for use with curl --oauth2-bearer or any other tool that doesn't read a hut-style config file.

So in a job, both work, but they reach the API differently:

hut builds list                                 # uses ~/.config/hut/config
curl --oauth2-bearer "$OAUTH2_TOKEN" \          # uses the env var
     https://meta.sr.ht/query -d '{...}'

The pre-written config is the load-bearing thing for hut. If a step removes it, chmods it unreadable, or runs as a different user, hut will fail with "missing access-token" even though $OAUTH2_TOKEN is set. Source: builds.sr.ht/worker/tasks.go SendHutConfig().

#Self-hosted instances work out of the box

Because the worker reads service origins from the same instance config the rest of the service uses, a self-hosted SourceHut (e.g. *.srht.bigb.es) gets a hut config pointing at its own services, not sr.ht. No extra plumbing required — oauth: pages.sr.ht/PAGES:RW in a manifest running on builds.srht.bigb.es produces a config that points pages → https://pages.srht.bigb.es.

If hut ever appears to be talking to sr.ht instead of the local instance, check that the running builds-worker actually has the per-service api-origin / origin keys set in /etc/sr.ht/config.ini — without them, the service block isn't emitted and hut falls back to service.<instance-name> which won't resolve.

#When hut config is NOT written

SendHutConfig early-returns when oauth2Token() returns nil. That happens whenever either:

  • The manifest has no oauth: directive, or
  • The build was submitted with secrets disabled (hut builds submit --no-secrets, the web "disable secrets" checkbox, or a manifest containing secrets: false in some pipelines).

In those cases neither ~/.config/hut/config nor $OAUTH2_TOKEN exists. hut will exit with error: missing access-token and curl --oauth2-bearer "$OAUTH2_TOKEN" will send Bearer with an empty token.

There is no override and no way to "borrow" a token from another build. If you need hut to work, you need oauth: and you need secrets enabled.

#Installing hut in a build

Image How
alpine/edge, alpine/latest packages: [hut] — in the main repo, current.
archlinux packages: [hut] — community/extra repo.
freebsd/latest packages: [hut] — ports.
nixos/unstable packages: [hut].
debian/*, ubuntu/* Not packaged. See below.
fedora/* Not packaged. Build from source or use a release tarball.

For Debian/Ubuntu/Fedora, the cleanest options are:

# Option 1: download a tagged release binary
tasks:
  - install-hut: |
      ver=v0.8.0
      curl -L "https://git.sr.ht/~xenrox/hut/refs/download/$ver/hut-${ver#v}-linux-amd64" \
        -o /tmp/hut
      chmod +x /tmp/hut
      sudo mv /tmp/hut /usr/local/bin/hut
      hut version
# Option 2: go install from source — slow but no version drift
packages: [golang, scdoc]
tasks:
  - install-hut: |
      git clone https://git.sr.ht/~xenrox/hut
      cd hut && make && sudo make install

Pin a version. The refs/download/<tag>/... URL is stable per tag; master is not. Latest tag at the time of writing is v0.8.0.

#CI-relevant commands and their OAuth scopes

The OAuth scope syntax is service/SCOPE[:MODE] (multiple space-separated). For a complete scope list per service, query meta.sr.ht/query interactively — the table below is what's actually useful in CI.

#pages — publish a static site

oauth: pages.sr.ht/PAGES:RW
hut pages publish -d <fqdn> [-s <subdir>] [-p https|gemini] [--site-config <path>] <tarball-or-dir>
hut pages unpublish -d <fqdn> [-p https|gemini]
hut pages list
hut pages acl list -d <fqdn>
hut pages acl update <user> [--id <site-id>] [--publish]

Input to publish can be a .tar.gz or a directory; passing - (or omitting the path on a non-terminal stdin) reads from stdin. No symlinks, no special files, mode 644 onlypages.sr.ht rejects them. The reliable invocation is:

tar -czf site.tar.gz -C <output-dir> .
hut pages publish -d "$site" site.tar.gz

-s <subdir> does a partial update — it touches only files under that path on the site, leaving the rest in place. Use this only when you actually want partial deployment; full-site publishes are atomic and rollback-safe.

#builds — submit, follow, fetch artifacts of other jobs

oauth: builds.sr.ht/JOBS:RW
hut builds submit [-f] [-n <note>] [-s] [-t <tags>] [-v public|unlisted|private] [manifest...]
hut builds resubmit <id> [-e] [-f] [-n <note>] [-s] [-v ...]
hut builds list [owner] [--count N] [-s <status>] [-t <tag-prefix>]
hut builds show [id] [-f] [--web]
hut builds artifacts <id>           # lists artifacts of a job
hut builds cancel <id...>
hut builds ssh <id>                 # local-only, not useful in CI
hut builds update <id> [-v ...] [-t <tags>]
hut builds secret list [--count N]
hut builds secret share <secret> -u <user>

builds submit without arguments auto-discovers .build.yml, .build.yaml, .builds/*.yml, .builds/*.yaml in the cwd. Pass -f (--follow) to stream logs and exit non-zero on a failed build — that's how to chain builds with hard dependencies. The minted CI token has JOBS:RW only if you asked for it; downgrade to JOBS:RO if you only need list/show/artifacts.

builds artifacts <id> returns JSON-ish output with download URLs that require the same token; pair with curl --oauth2-bearer "$OAUTH2_TOKEN" -L -o out.tar.gz <url> to actually fetch.

#git — repo metadata, artifacts attached to refs

oauth: git.sr.ht/REPOSITORIES:RW          # full
oauth: git.sr.ht/OBJECTS:RW               # artifact upload only
hut git artifact upload <file...> [--rev <tag>]
hut git artifact list
hut git artifact delete <id>
hut git acl list / update / delete
hut git create / delete / update / show / list
hut git clone / setup        # local-only

git artifact upload attaches files to a git tag (--rev; defaults to the latest tag in cwd). This is the CI-friendly way to ship release tarballs/binaries: tag → build → upload. The attached artifact is visible on the tag's page.

#lists — apply / update patchsets, test contributions

oauth: lists.sr.ht/PATCHES:RW
hut lists patchset apply <id>             # writes patches into cwd
hut lists patchset update <id> --status <NEEDS_REVISION|APPROVED|REJECTED|APPLIED|...>
hut lists patchset show <id>
hut lists patchset list <list>

Use case: a CI manifest triggered by a mailing-list webhook clones the repo, runs hut lists patchset apply <id>, then git am / git apply, then runs tests. On failure, hut lists patchset update <id> --status NEEDS_REVISION and email the thread.

#todo — create/close tickets from build outcomes

oauth: todo.sr.ht/TRACKERS:RW
hut todo ticket create -t <tracker> -s "<subject>" [-b <body>]
hut todo ticket update <ticket-ref> --status <STATUS> --resolution <RESOLUTION>
hut todo ticket comment <ticket-ref> -b <body>
hut todo ticket assign <ticket-ref> -u <user>
hut todo ticket list -t <tracker>

Useful for "open a ticket on flaky test" / "close the auto-generated ticket on green build" patterns.

#paste — short-lived dumps from the build

oauth: paste.sr.ht/PASTES:RW
hut paste create [-n <name>] [-v public|unlisted|private] <file...>
hut paste list
hut paste delete <id>

Quick way to ship a log excerpt out of a build for inspection without setting up artifacts.

#meta — query identity, ssh keys, pgp keys

oauth: meta.sr.ht/PROFILE:RO              # to read 'whoami'
hut meta show [user]
hut meta ssh-key list / create / delete
hut meta pgp-key list / create / delete
hut meta audit-log
hut meta api-key list / create / delete   # NEW since v0.7

Rarely needed from CI. Most useful for debugging "what user is this OAuth token actually authenticated as" — hut meta show returns the canonical name attached to the token.

#graphql — escape hatch for anything not covered

hut graphql <service> [-v key=value] [--stdin] [--file key=path] <<EOF
query { ... }
EOF

Reads a query from stdin (default if non-terminal) or $EDITOR and posts to <service>.sr.ht/query with the configured token. Use this when you need a mutation hut doesn't have a subcommand for (e.g. submit with manifest-as-string, group creation, custom webhook events). The token's grants still apply — hut graphql does not bypass scope checks.

#Useful CI patterns

#Self-publishing release with git artifact upload

image: alpine/edge
packages: [go, hut]
oauth: git.sr.ht/OBJECTS:RW
sources: [https://git.sr.ht/~user/mytool]
tasks:
  - build: |
      cd mytool
      tag=$(git describe --tags --abbrev=0)
      GOOS=linux  GOARCH=amd64 go build -o "mytool-$tag-linux-amd64" ./cmd/mytool
      GOOS=darwin GOARCH=arm64 go build -o "mytool-$tag-darwin-arm64" ./cmd/mytool
  - publish: |
      cd mytool
      hut git artifact upload --rev "$(git describe --tags --abbrev=0)" mytool-*-*

Pair with submitter.allow-refs: ["refs/tags/*"] so this only fires on tag pushes.

#Follow-up build after tests pass (no job: trigger needed)

image: alpine/edge
packages: [hut]
oauth: builds.sr.ht/JOBS:RW
sources: [https://git.sr.ht/~user/myproject]
tasks:
  - test: |
      cd myproject && make test
  - deploy: |
      cd myproject && hut builds submit -f .builds/deploy.yml

The -f flag makes hut wait for the deploy job and exit non-zero on failure, so the parent build accurately reflects the deploy result.

#Fetch an artifact produced by a previous build

oauth: builds.sr.ht/JOBS:RO
tasks:
  - fetch: |
      # Replace with the actual job id (e.g. from a webhook payload).
      hut builds artifacts 1234567 \
        | awk '/mybinary/ { print $NF }' \
        | xargs -I{} curl -L --oauth2-bearer "$OAUTH2_TOKEN" -o mybinary {}

hut builds artifacts currently prints a human-readable list rather than machine-readable JSON; if you need stable parsing, use hut graphql builds with the job(id: <id>) { artifacts { ... } } selector instead.

#Per-job tags for filtering

-t name/scope attaches slash-separated tags to a job. Useful when you submit many follow-up jobs and want hut builds list -t myproject/deploy to filter them.

#Common failure modes

error: missing access-token from hut The manifest has no oauth: directive, or secrets are disabled. Check the build's first few log lines — if you don't see "Sending hut config" / equivalent, no config was written. Fix: add oauth: <scope> (the smallest scope that works) and resubmit with secrets enabled. Personal-access-token via secrets: is a workable fallback but strictly worse — manage rotation yourself.

hut talks to sr.ht instead of the local instance (self-hosted) The local builds-worker's config is missing per-service api-origin / origin entries. SendHutConfig only emits a service block when one of those keys is present; without them, hut constructs the origin from the bare instance name, which won't resolve to your hosts. Fix: add [<service>.sr.ht] sections with origin (and ideally api-origin) in /etc/sr.ht/config.ini and restart the worker.

hut pages publish returns 401 Token has the wrong grant. PAGES:RW is required to publish; PAGES:RO only reads. There is no separate publish-vs-unpublish split.

hut pages publish returns 400 "invalid tarball" / "symlink rejected" / "mode not allowed" pages.sr.ht enforces: only regular files and directories; mode 0644 (files) / 0755 (dirs); no ./ prefix beyond what tar -C dir . produces; no extra top-level wrapper directory. A common offender: tar -czf out.tar.gz public/ — this nests everything under public/ on the site. Use tar -czf out.tar.gz -C public . instead.

hut builds submit -f exits 0 even though the child build failed -f follows the job and the exit code reflects the job's status. If the job was cancelled rather than failed, hut (older versions) exited 0; v0.7+ exits non-zero on CANCELLED too. Pin a recent version if you depend on this.

hut v0.x command exists in docs but hut --help says no such command The image's hut package is older than the docs you're reading. apk and archlinux images stay close to upstream; Debian/Ubuntu have no package at all so version drift only happens if you forget to bump your install-hut step. Always run hut version early to log what's actually installed.

Token leaks into the log via set -x ~/.buildenv exports OAUTH2_TOKEN, and set -x is on by default. Any command that mentions $OAUTH2_TOKEN echoes the expanded value. For commands that take the token from the config (hut) this is a non-issue; for curl --oauth2-bearer "$OAUTH2_TOKEN" it is. Wrap such commands:

set +x
curl --oauth2-bearer "$OAUTH2_TOKEN" https://meta.sr.ht/query -d "$query"
set -x

The token is short-lived (expires at the build deadline, typically ~4h), so a leaked one isn't a long-term credential — but it's still worth not leaking.

#Cross-references

  • references/pages.md — full pages.sr.ht workflow, tarball constraints, multi-domain publishing.
  • references/secrets-and-oauth.md — why prefer oauth: over secrets:, full scope reference, examples without hut.
  • references/manifest.md — the oauth: directive field reference.
  • references/integrations.mdgit.sr.ht push hooks, mailing-list patch testing, createGroup for cross-job triggers.
  • references/debugging.md — interpreting log output when hut (or anything else) goes wrong.