~bigbes/sourcehut-root

sourcehut-root/skills/sourcehut-ci/references/manifest.md -rw-r--r-- 21.2 KiB
72581ff9 — Eugene Blikh docs(sourcehut-ci): fix factual errors found in full audit 2 days ago

#Build Manifest Reference

Complete field reference for .build.yml. The canonical source is https://man.sr.ht/builds.sr.ht/manifest.md; this file is a working summary with usage notes.

#Top-level structure

image: alpine/edge       # required
arch: x86_64             # optional, defaults to x86_64
packages:                # optional
  - pkg1
repositories:            # optional
  reponame: "<url-or-line>"
sources:                 # optional
  - <url>[#ref]
environment:             # optional
  KEY: value
secrets:                 # optional
  - <uuid>
oauth: "<service>/<scope> ..."  # optional
shell: false             # optional, default false
artifacts:               # optional
  - path/to/file
tasks:                   # required, at least one
  - taskname: |
      script
triggers:                # optional
  - action: email|webhook       # only these two are implemented
    condition: success|failure|always
    ...
submitter:               # optional, controls integration-side submission
  git.sr.ht:
    enabled: true
    allow-refs: [...]

#image (required, string)

Names the base VM image. Format is <distro>/<release>, or just <distro> for single-recipe images. The canonical support matrix is at https://man.sr.ht/builds.sr.ht/compatibility.md; values listed there as of 2026:

  • alpine/edge (rolling, daily), alpine/latest (= 3.23), alpine/3.23, alpine/old (= 3.22), alpine/older (= 3.21), alpine/oldest (= 3.20)
  • archlinux
  • debian/stable (= trixie / 13), debian/oldstable (= bookworm / 12), debian/testing (= forky), debian/unstable (= sid)
  • ubuntu/lts (= noble / 24.04), ubuntu/oldlts (= jammy / 22.04), ubuntu/focal (20.04), ubuntu/oracular (24.10), ubuntu/plucky (25.04)
  • fedora/rawhide (= 44), fedora/latest (= 43), fedora/42
  • rockylinux/latest (= 8), rockylinux/9
  • freebsd/current (= 16.0-CURRENT), freebsd/latest (= 15.x), freebsd/14.x
  • openbsd/latest (= 7.8), openbsd/old (= 7.7)
  • netbsd/latest (= 10.x), netbsd/9.x
  • nixos/unstable, nixos/latest (= 25.05), nixos/24.11
  • guix

gentoo and 9front are not in upstream's published matrix or in the builds.sr.ht/images/ recipe directory. Submissions targeting them will fail with "no such image" on the upstream builds.sr.ht.org and almost all self-hosted instances. Don't suggest them by default.

Choosing: Default to alpine/edge (small, fast boot, recent packages). Pick a specific distro when packaging for that distro or when a package is only available there. archlinux is good for AUR / rolling. debian/stable for Debian packaging. NixOS images are best for reproducible builds (flakes are off by default — enable via NIX_CONFIG).

image-name/latest rolls forward over time; pin to a specific release if reproducibility matters.

#arch (optional, string)

CPU architecture. Default is whatever the image's recipe declares — x86_64 for Alpine/Arch/Fedora/NixOS/Guix/Rocky, amd64 for Debian/Ubuntu/FreeBSD/NetBSD/OpenBSD (BSDs and Debian-family use their upstream naming). The compatibility matrix at https://man.sr.ht/builds.sr.ht/compatibility.md shows the default arch per image with the table-primary row.

Other arches (aarch64/arm64, ppc64el, riscv64, …) are listed for each image but only the marked rows are actually built on upstream builds.sr.ht.org. On a self-hosted instance, only arches whose qcow2 has been built with genimg will work. If submission fails with "no such image", the combination isn't built.

#packages (optional, list of strings)

Packages to install before tasks run, using the image's native package manager:

  • Alpine: apk add — names match pkgs.alpinelinux.org
  • Arch: yay -Syu (AUR packages are transparently installed) — names match archlinux.org/packages and the AUR
  • Debian/Ubuntu: apt-get install — names match packages.debian.org / packages.ubuntu.com
  • Fedora / Rocky: dnf install
  • FreeBSD: pkg install — names match FreeBSD ports
  • NetBSD: pkgin
  • OpenBSD: pkg_add
  • NixOS: nix-env -iA — requires the full selection path (e.g. nixos.hello), not bare names
  • Guix: guix install

Cross-distro package names differ. There is no abstraction layer. Ninja is ninja on Alpine/Arch/Fedora/FreeBSD but ninja-build on Debian/Ubuntu. Look it up; don't guess.

#repositories (optional, dict)

Adds extra package repositories before installing packages. The value format is image-specific (buildsrht/manifest.py only validates that each value is a string); the worker hands name + value to the image's add-repo control script. Examples per the upstream compatibility matrix:

# Alpine — value is "repo-url key-url key-name" (space-separated).
# Prefix the *name* with @ if you want apk's @tag indirection.
repositories:
  sr.ht: >
    https://mirror.sr.ht/alpine/sr.ht/
    https://mirror.sr.ht/alpine/sr.ht/alpine%40sr.ht.rsa.pub
    alpine@sr.ht.rsa.pub
# Debian/Ubuntu — value is "url release component key-id" (space-separated, key-id optional).
repositories:
  sr.ht: https://mirror.sr.ht/debian/sr.ht/ bookworm main DEADBEEFCAFEF00D
# Arch — value is "url#key-id".
repositories:
  myrepo: https://mirror.example.org/repo/$arch#DEADBEEFCAFEF00D
# Fedora / Rocky — value is a single URL; the worker calls `dnf config-manager` against it.
repositories:
  example: https://example.org/example.repo
# NixOS — value is a channel URL; `nix-channel --add <value> <name>` then `--update <name>`.
repositories:
  nixpkgs: https://nixos.org/channels/nixpkgs-unstable

Custom repositories are not supported on FreeBSD, NetBSD, OpenBSD, or Guix — the recipes ignore the directive on those images.

Consult the compatibility page for the canonical syntax per image; sourcehut may add or refine formats over time.

#sources (optional, list of strings)

Repositories to clone into /home/build before tasks run. Each is cloned into a subdirectory named after the last URL component.

sources:
  - https://git.sr.ht/~user/projectA
  - https://github.com/user/projectB
  - https://git.example.org/projectC#some-tag
  - hg+https://hg.sr.ht/~user/mercurial-project
  - git+ssh://git@git.sr.ht/~user/private-repo
  - mydir::https://git.example.org/user/upstream     # clone into ./mydir
  - "::https://git.example.org/upstream"            # explicit "no rename"
  • Default SCM is git. Prefix the URL scheme with hg+ for Mercurial (hg+https://, hg+ssh://); the worker strips the prefix before invoking the underlying SCM.
  • Pin a ref by appending #<rev> — branch name, tag name, or commit SHA. For git, this becomes git checkout; for hg, hg update. Submodules are initialized (git submodule update --init --recursive) on git clones.
  • Rename the local clone directory with dir::url. By default the directory is the basename of the URL (with any .git stripped); mytool::https://example.com/foo clones into ./mytool instead of ./foo.
  • Public repos clone over HTTPS without credentials. Private repos: register an SSH-key secret and use the SSH form (git@host:user/repo or git+ssh://git@host/user/repo). The worker sets GIT_SSH_COMMAND to skip strict host-key checking, so you don't need ssh-keyscan for the clone (you do for arbitrary outbound SSH later in tasks).
  • 9front images use git9 instead of vanilla git, and the worker explicitly rejects SSH URLs there (no factotum support).
  • When git.sr.ht auto-submits the manifest, it rewrites the matching entry in sources: to the canonical clone URL plus #<commit-sha>. The build operates on the exact commit you pushed.

#environment (optional, dict)

Key/value pairs written to ~/.buildenv, which is sourced by every task's preamble.

environment:
  CGO_ENABLED: 0
  site: example.com
  deploy_user: deploy@example.com

Values are strings. To pass state between tasks, append to ~/.buildenv from inside a task:

tasks:
  - compute-version: |
      VERSION=$(git -C myrepo describe --tags --always)
      echo "VERSION=$VERSION" >> ~/.buildenv
  - use-version: |
      echo "Building version $VERSION"

Important: tasks are separate login sessions. Environment variables exported in one task do not persist to the next unless written to ~/.buildenv.

#secrets (optional, list of UUIDs or names)

Each entry is either a UUID or the human-readable name (3–512 chars) of a secret registered at https://builds.sr.ht/secrets (on builds.sr.ht, not meta.sr.ht). See secrets-and-oauth.md for the full discussion.

secrets:
  - 12345678-1234-1234-1234-123456789abc   # by UUID
  - github-deploy-key                      # by name

Secrets are installed into the build VM before tasks run. Whether they're available depends on submission context — patches from mailing lists, untrusted PRs, and some other integration paths disable secrets automatically.

#oauth (optional, string)

Space-separated list of sourcehut OAuth scope grants. builds.sr.ht generates a fresh OAuth 2.0 bearer token with exactly these grants for this build, exports it as $OAUTH2_TOKEN in ~/.buildenv, and also writes it to /home/build/.config/hut/config (with per-service origins from the running instance's config) so hut finds it without extra setup. Both die with the VM.

oauth: pages.sr.ht/PAGES:RW lists.sr.ht/PROFILE:RO

Scope syntax: <service>/<SCOPE>:<RO|RW>. Common grants:

  • pages.sr.ht/PAGES:RW — publish to pages.sr.ht
  • builds.sr.ht/JOBS:RW — submit further builds (used by triggers that chain jobs)
  • git.sr.ht/REPOSITORIES:RW — push to git repos
  • lists.sr.ht/PROFILE:RO — list user info
  • meta.sr.ht/PROFILE:RO — read account profile

hut reads ~/.config/hut/config (which the worker pre-writes) — no extra config needed. curl --oauth2-bearer "$OAUTH2_TOKEN" works for non-hut callers. Full mechanism + scope-to-command map in references/hut.md.

Use oauth: instead of secrets: whenever you're talking to a sourcehut service. It's safer (short-lived, narrowly-scoped, can't be exfiltrated for reuse) and simpler (no secret to register).

#shell (optional, bool)

If true, keeps the build VM alive after tasks finish, even on success, so you can SSH in. Default false. Use during iteration; remove before committing.

#artifacts (optional, list of strings)

Files to extract from the build VM after success. See artifacts.md.

artifacts:
  - mytool
  - myproject/dist/release.tar.gz
  • No globs, no shell expansion. Each path is literal.
  • Paths are relative to the build user's home directory (/home/build on Linux images; differs on BSDs — /root etc., as set by the image's Homedir), or absolute. The worker resolves relative paths against $HOME, not the project clone directory.
  • Basenames must be unique across the list (the manifest parser rejects duplicates) — dist/foo and bin/foo cannot both be listed; rename first.
  • Only single files; directories must be tarred first.
  • Hard cap of 8 artifacts per build; extras are silently dropped with a warning. Hard cap of 1 GiB per file; oversize files fail the upload step.
  • Only uploaded on successful builds.
  • Pruned after 90 days.

#tasks (required, list of one-key dicts)

Each task is a name: script pair. The order matters; tasks run sequentially in fresh login sessions.

tasks:
  - configure: |
      cd myproject
      ./configure
  - build: |
      cd myproject
      make
  - test: |
      cd myproject
      make check
  • Task names: lowercase alphanumeric, _, -. Max 128 chars.
  • Each task gets a section in the log.
  • A task fails the build if its script exits non-zero (the preamble has set -e).
  • Use complete-build inside a task to end the whole build successfully without running subsequent tasks.

The list-of-dicts form (each task is a dict with one key) is canonical. Older docs occasionally showed a list of strings; use the dict form. YAML's | block-scalar form is what most manifests use because tasks are usually multi-line.

#triggers (optional, list of dicts)

Post-build actions. Each trigger has an action and a condition. The current builds.sr.ht/worker only implements two actions: email and webhook. The Python manifest parser (buildsrht/manifest.py) accepts only those two TriggerAction enum values, and the Go worker's dispatch map (worker/triggers.go) has entries for only those two — unrecognized actions log Unknown trigger action and are skipped.

Older docs, blog posts, and out-of-date AI completions mention irc and job triggers. They do not exist in current builds.sr.ht; do not generate them.

triggers:
  - action: email
    condition: failure
    to: "Me <me@example.com>"
    cc: "Other <them@example.com>"
    in_reply_to: "<message-id@example.com>"   # optional; used by hub.sr.ht patchset emails
  - action: webhook
    condition: always
    url: https://example.com/sr-ht-hook
  • condition: success, failure, or always. Internally maps failed/timeout/cancelled → "failure" and success → "success".
  • email: requires to. Optional cc, in_reply_to. Subject is [<tags>] build <status>; body lists task results with ✓/✗ glyphs. The sender comes from instance config (builds.sr.ht::worker/trigger-from).
  • webhook: HTTP POSTs a signed JSON payload (status, task list, owner, runner) to url. Headers X-Payload-Nonce and X-Payload-Signature are set so the receiver can verify authenticity. The receiver has 10 seconds to respond; the body is read up to 2 KiB into the build log.

Substitutes for the non-existent actions:

  • irc → run an HTTP-to-IRC bridge on your own infrastructure, target it with action: webhook. Or write to a Matrix/Discord/IRC bridge service that accepts incoming webhooks (e.g. matterbridge, ZNC's web hook plugin).
  • job (chained deploy) → either inline the deploy as a later task in the same manifest (gated by submitter.allow-refs so it runs only on the right branch), or have your webhook receiver call hut builds submit / the submit GraphQL mutation.

#Job group triggers

When jobs are bundled into a job group via the GraphQL createGroup mutation, the API accepts TriggerInput records with type: EMAIL or type: WEBHOOK. Only EMAIL is wired up in the worker; WEBHOOK is a TODO (// ctx.processGroupWebhook, TODO in worker/triggers.go) and will be logged as "Unknown trigger action" if you set it today. hub.sr.ht's patchset auto-tester relies on this — it creates a per-patchset group with an EMAIL trigger that posts back to the mailing list thread.

#submitter (optional, dict)

Controls how integrations decide whether to submit this manifest on a given event. Only git.sr.ht is recognized by the git.sr.ht push hook today; the schema is:

submitter:
  git.sr.ht:
    enabled: true              # optional, default true
    allow-refs:                # optional, no filtering if absent
      - refs/heads/master
      - refs/heads/main
      - "refs/tags/*"

Semantics, as implemented in git.sr.ht's update-hook/submitter.go:

  • enabled is tri-state. If unset or true, the block is permissive. If false, the build is skipped immediately and allow-refs is not consulted. There is no reason to write enabled: true explicitly — it changes nothing.
  • allow-refs is a list of fnmatch(3)-style glob patterns matched against the full ref (e.g. refs/heads/master, refs/tags/v1.0). The match runs with no flags, so * does cross / (i.e. refs/heads/* matches feature branches with slashes in them too, unlike a pathname-style glob). An empty/absent list means "no filtering"; a non-empty list means "submit only if at least one pattern matches".
  • The git.sr.ht key matches the integration that sees the manifest. A hub.sr.ht: block (used for patchset auto-testing) is parsed by hub.sr.ht's own code, not by this hook — and the git.sr.ht hook silently drops unknown keys when it re-serializes the manifest to send to builds.sr.ht. So put a hub.sr.ht: submitter rule in a .build.yml only if you understand that it survives only the patchset path, not the git-push path.
  • The filter runs after the manifest is located in the tree but before the GraphQL submit mutation fires. Non-matching refs never create a job, so this is strictly cheaper than complete-build-based filtering inside a task (which would still cost a VM boot).
  • Only auto-submission honors submitter:. Manual submission via hut builds submit, the web submit form, or a direct GraphQL submit mutation ignores the block — the runner doesn't re-evaluate it.

This is the right way to limit which branches trigger CI. Filtering inside the script with complete-build works, but it wastes a worker slot to spin up a VM just to bail; submitter: prevents the job from being created at all.

#repositories for non-package mirrors

The repositories: directive is for package repos. To clone additional source repos, use sources:.

#Built-in environment variables

Always set by the worker before tasks run (see builds.sr.ht/worker/tasks.go SendEnv):

  • CI_NAME — literally sourcehut. Useful in scripts that fan out across CI systems.
  • JOB_ID — the integer job ID.
  • JOB_URL — full URL to the job page, e.g. https://builds.sr.ht/~user/job/12345.
  • OAUTH2_TOKEN — only set when the manifest has an oauth: directive and secrets are enabled for this submission. Bearer-token form, ready to pass to hut or curl --oauth2-bearer.

Injected by integrations into the manifest's environment: block before submission, so they're available like any normal env variable:

  • git.sr.ht push hook (update-hook/submitter.go):
    • BUILD_SUBMITTER=git.sr.ht
    • GIT_REF=<pushed-ref> (e.g. refs/heads/master, refs/tags/v1.0)
    • There is no BUILD_REASON set by git.sr.ht — only hub.sr.ht sets that.
  • hub.sr.ht patchset auto-tester (hubsrht/builds.py):
    • BUILD_SUBMITTER=hub.sr.ht
    • BUILD_REASON=patchset
    • PATCHSET_ID=<id>
    • PATCHSET_URL=<url>
    • These are set with setdefault, so the build manifest may override them.
  • Manual submission (web form, hut builds submit, GraphQL submit): neither BUILD_SUBMITTER nor BUILD_REASON is set — only the always-set worker variables.

The hook also rewrites sources: before submission. For each entry whose basename matches the pushed repo name, it replaces the URL with the canonical clone URL plus #<commit-sha>. If no entry matches the repo name, the clone URL is appended to sources:. The clone URL is the SSH form (git+ssh://git@<host>/~<user>/<repo>) for private repos and the HTTPS form for public repos. The commit-sha pin is what makes auto-submitted builds reproducible: the build operates on the exact commit you pushed, not on whatever the ref points to by the time the VM boots.

shell: true is stripped from the manifest with a Notice: removing 'shell: true' from build manifest log line. Use shell: true only for manual submissions (web form, hut builds submit) where you want to SSH in regardless of outcome.

Set when submitted via patchset (mailing list):

  • PATCHSET_ID, PATCHSET_URL

Use these for conditional logic in tasks (mind the [ -z "$VAR" ] guard for cases where the build was submitted outside the corresponding integration):

tasks:
  - check: |
      if [ -n "$GIT_REF" ] && [ "$GIT_REF" != "refs/heads/master" ]; then
          complete-build
      fi

#The complete-build command

A magic in-VM command. Calling complete-build from any task immediately ends the build successfully, skipping all subsequent tasks. Use it for early exit when conditions aren't met (wrong branch, no relevant changes, etc.).

It's technically undocumented but stable in practice — the sourcehut admins point users at it in support threads. Don't rely on it for security gating (it's a convenience, not an isolation primitive), and prefer submitter.allow-refs for ref filtering when the filter is known at submission time.

#Gotchas

  • packages: runs before tasks, so package failures fail the build before any of your scripts run.
  • sources: runs after packages:, so you can use git from packages if needed.
  • set -xe is on by default, including for the set +x you might add. Re-enable with set -x if you want logging back.
  • Multi-line YAML strings need |, not > (which folds newlines). Almost all task scripts want |.
  • cd doesn't persist across tasks — they're separate sessions. cd myproject at the start of each task that needs it is normal and expected.
  • The build user is build with /home/build as home and passwordless sudo. You can install additional packages mid-build with sudo, but prefer the manifest's packages: list.
  • Manifest size: no explicit limit in the validator; it's bounded only by what the GraphQL submit endpoint accepts. For very large multi-line scripts, put the script in the repo and call it from the task — much easier to read and review than a wall of YAML.
  • Job timeout is set per-instance in [builds.sr.ht::worker] timeout (the upstream config example uses 45m; builds.sr.ht.org runs longer; your self-hosted instance may differ). On timeout the job is marked timeout (treated as failure by triggers) and the VM is torn down. The post-failure SSH grace window is separate — 10 minutes if you don't log in, then "your remaining build time" once you do.