~bigbes/sourcehut-root

sourcehut-root/skills/sourcehut-ci/SKILL.md -rw-r--r-- 19.4 KiB
72581ff9 — Eugene Blikh docs(sourcehut-ci): fix factual errors found in full audit 2 days ago

#name: sourcehut-ci description: Author and debug builds.sr.ht CI manifests for SourceHut projects (including the user's self-hosted instance at *.srht.bigb.es), publishing static sites to pages.sr.ht, and producing downloadable build artifacts. Use this skill whenever the user mentions sourcehut, sr.ht, srht.bigb.es, builds.sr.ht, pages.sr.ht, .build.yml, .builds/, hut, a hut pages publish workflow, deploying a static site to srht.site or srht.bigb.es, sourcehut artifacts, OAuth grants in CI manifests, or post-build triggers — even when they don't explicitly say "CI". Also use it when the user is troubleshooting a failed sourcehut build, asks how to skip CI on push, wants to test patches on a mailing list, or wants matrix-style multi-image builds on sourcehut. SourceHut CI works very differently from GitHub Actions, GitLab CI, and Jenkins; this skill encodes those differences so the output works on the first submission rather than after several broken pushes. license: BSD-3-Clause metadata: audience: developers workflow: ci tags: [ci, sourcehut, sr.ht, builds.sr.ht, pages.sr.ht, yaml] version: 1.1.0 author: bigbes

#SourceHut CI (builds.sr.ht)

This skill covers the builds.sr.ht continuous integration service: how to write build manifests, deploy artifacts to pages.sr.ht, and produce downloadable build artifacts. It also covers the surrounding ecosystem (git.sr.ht push integration, secrets, hut, mailing list patch testing) to the extent needed to make CI work end-to-end.

The single most important mental model: SourceHut CI is stateless and explicit. Every build is described entirely by a YAML manifest submitted to the API. There is no web UI for configuring jobs, no environment-variable settings page that secretly modifies behavior, no implicit context beyond what's in the manifest. This is a feature, not a limitation — but it means habits from GitHub Actions / GitLab CI / Jenkins will mislead.

#When to consult which reference

This SKILL.md is enough for ~80% of tasks. For the rest, read the matching file in references/:

  • references/manifest.md — complete field reference for .build.yml (every key, every accepted value, every gotcha). Read when writing a non-trivial manifest from scratch, or when a user asks "does sr.ht support X" for a manifest feature.
  • references/pages.md — pages.sr.ht specifics: tarball constraints, custom domains, multi-domain publishing, hut pages publish flags, common static site generator recipes. Read whenever the task involves publishing a website.
  • references/artifacts.md — how the artifacts: directive works, its limitations, and patterns for working around them (notably no globbing). Read when the user wants downloadable build outputs.
  • references/secrets-and-oauth.md — how to register secrets, the difference between file and environment secrets, and how the in-manifest oauth: directive mints short-lived tokens. Read whenever secrets or OAuth grants are involved.
  • references/integrations.md — git.sr.ht auto-submission, push options, mailing list patch testing, multi-image fan-out, job groups, triggers. Read for "how does my repo trigger builds" questions.
  • references/debugging.md — interpreting log output, SSH into failed VMs, shell: true, complete-build, common errors. Read when the user reports a failed build.
  • references/images.md — quick selector for which image: value to use, package manager differences, architecture support. Read when the user asks "which image should I use" or "is X available".
  • references/hut.mdhut, the SourceHut CLI, as used inside builds. How the worker provisions ~/.config/hut/config, every CI-relevant subcommand mapped to the OAuth scope it needs, install recipes per image, common failure modes. Read for any task that runs hut <something> in CI, or when the user asks why hut can't authenticate.

When more than one reference is relevant, read them all before writing the manifest. They're short by design.

#What goes in .build.yml — the minimum

image: alpine/edge
tasks:
  - example: |
      echo "hello world"

That's a complete, valid manifest. image is the only required top-level key, and at least one task is required.

A more realistic manifest covers most projects:

image: alpine/edge
packages:
  - meson
  - ninja
sources:
  - https://git.sr.ht/~user/myproject
tasks:
  - configure: |
      cd myproject
      meson build
  - build: |
      cd myproject
      ninja -C build
  - test: |
      cd myproject
      ninja -C build test

Every task runs as a separate bash login session. The preamble injected before each task is:

#!/usr/bin/env bash
. ~/.buildenv
set -xe

This means: set -e is on (any non-zero exit terminates the task and fails the build), set -x is on (every command is echoed to the log with variables expanded — be careful with secrets), and ~/.buildenv is sourced (this is where environment: values and any state passed between tasks lives).

Task names must be lowercase alphanumeric, underscore, or dash, ≤128 characters. Tasks run in the order specified. Each task must be a single-key dict (name: |\n script); the older - "just a string" shorthand mentioned in some legacy docs is rejected by the current parser.

#Where the manifest lives

  • .build.yml at the repo root — git.sr.ht automatically submits this on every push.
  • .builds/*.yml — multiple manifests, up to 4 are submitted per push (chosen randomly if more exist, because the hook iterates a Go map). This is how sourcehut handles what other CIs call a build matrix.
  • Both .yaml and .yml extensions are auto-discovered: the default pattern is .build.yml,.builds/*.yml,.build.yaml,.builds/*.yaml. Override per-push with git push -o submit=....
  • The 4-build cap is per push, not per ref: if you push to several refs with builds, the hook stops submitting once it has produced 4 jobs in total across them.
  • Submitted ad-hoc via the web form at https://builds.sr.ht/submit, the hut builds submit CLI, or the GraphQL API — no repo required.

The simplest workflow when iterating: paste the manifest into the web submit form repeatedly until it works, then commit it as .build.yml. Do not commit broken manifests just to test them.

#Publishing to pages.sr.ht

pages.sr.ht hosts static sites. Every user gets username.srht.site for free, with automatic TLS. Custom domains work via a single CNAME record.

The CI pattern for publishing is reliable enough to memorize:

image: alpine/edge
packages:
  - hut
oauth: pages.sr.ht/PAGES:RW
environment:
  site: username.srht.site
sources:
  - https://git.sr.ht/~username/my-website
tasks:
  - package: |
      cd my-website
      tar -czvf ../site.tar.gz -C public .
  - publish: |
      hut pages publish -d "$site" site.tar.gz

Three things make this work:

  1. oauth: pages.sr.ht/PAGES:RW — builds.sr.ht mints a short-lived OAuth 2.0 token with exactly the PAGES:RW grant. The worker writes /home/build/.config/hut/config with the token as access-token (plus per-service origins from the running instance's config), so hut Just Works. The same token is also exported as $OAUTH2_TOKEN for tools like curl --oauth2-bearer — but hut itself reads only the config file, not the env var. No secret needed; the token is per-build and dies with the VM. Details: references/hut.md.
  2. The tarball must contain only directories and regular files, mode 644, no symlinks. Use tar -C <output-dir> . to put files at the top level of the tarball (this is what pages.sr.ht expects — no extra wrapping directory).
  3. hut pages publish -d <domain>-d takes the fully qualified domain. To publish to a custom domain you've configured, pass that domain; to publish to your subdomain, pass username.srht.site.

See references/pages.md for static site generator recipes (Hugo, Zola, Jekyll, mdBook), custom domain setup, and multi-domain publishing (e.g. apex + www must be published separately).

#Producing downloadable artifacts

Use the artifacts: directive when you want to make files downloadable from the job page:

image: alpine/edge
packages:
  - go
sources:
  - https://git.sr.ht/~user/mytool
tasks:
  - build: |
      cd mytool
      go build -o /home/build/mytool ./cmd/mytool
artifacts:
  - mytool

Critical constraints (these will bite you):

  • No globs. artifacts: ["*.tar.gz"] does not work. Each path is interpreted literally. If you need a wildcard, build the file under a stable name (rename in a task, or tar everything into a single archive).
  • Paths are interpreted literally — no ~, no $HOME, no shell expansion. Use absolute paths or paths relative to /home/build.
  • Only successful jobs upload artifacts. Failed builds discard whatever they produced.
  • Pruned after 90 days. Don't use artifacts as long-term storage; for that, push to pages.sr.ht, a git LFS endpoint, or an annotated git tag with attachments.
  • Single files, no directories. If you want to ship a directory tree, tar it first.

See references/artifacts.md for patterns: matrix-of-binaries, naming with version suffixes, combining with triggers for release automation.

#Secrets

Secrets live on your sourcehut account, identified by UUID (or by the human-readable name, 3–512 chars, you assigned at registration), and you opt them in per-build:

secrets:
  - 12345678-1234-1234-1234-123456789abc
  - github-deploy-key            # by name also works

Secrets come in three flavors registered at builds.sr.ht/secrets (on builds.sr.ht, not meta.sr.ht): SSH keys (mounted at ~/.ssh/id_* with appropriate permissions), PGP keys (imported into the build user's keyring), and files (placed at a path you specify with a mode you specify). Plain "environment variable" secrets aren't a separate type — use a file secret containing KEY=value and source it, or use the OAuth flow described above.

Watch out for set -x echoing secret values to the log. If a task references a secret-derived value, run set +x before the sensitive command and set -x after. See references/secrets-and-oauth.md for the full picture, including why oauth: is almost always better than secrets: for sr.ht-internal services.

#What sourcehut CI does not have

Don't suggest these — they don't exist, and trying to fake them with workarounds is usually worse than the obvious alternative:

  • No matrix syntax. Use multiple files in .builds/. Each file is a separate independent job.
  • No path-based triggers (e.g. "only run on changes to docs/"). Filter inside the script: if git diff --quiet HEAD HEAD^ -- docs/; then complete-build; fi.
  • No caching layer. Each build starts from a fresh VM. For caching: rsync to a server you control, or use a dedicated cache repo with a deploy key.
  • No "required for merge" gates in the GitHub PR sense. SourceHut's contribution model is patch-based via mailing lists; build results post back to the thread as replies.
  • No globs in artifacts:. Tar your outputs into a single file with a stable name. Hard cap of 8 artifacts per build, ≤1 GiB each; the worker rejects more.
  • No if: conditions on tasks. Use complete-build (ends the whole build successfully) or in-shell conditionals.
  • No first-class Docker support. Builds run inside KVM VMs. Docker can be installed inside those VMs, but the VM is the unit of isolation, not a container.
  • No environments / deployment protection in the GitHub Environments sense. Use a job group with a manual-start follow-up job if you need approval gates.
  • No irc or job triggers. Only email and webhook actions exist in the current worker. Chain deploys with a final task inside the same job, or have a webhook receiver submit a follow-up build.

When the user asks for one of these, name the substitute and write the manifest using it.

#Triggering builds from a push

When the manifest lives in the repo, git.sr.ht handles submission automatically on push. Useful push options:

  • git push -o skip-ci — don't submit any builds for this push. The push itself still goes through; only the build submission step is skipped server-side. Essential for self-pushing builds (auto-changelog, doc regeneration) to avoid infinite loops.
  • git push -o submit=".sourcehut/*.yml" — override the default manifest glob.
  • git push -o debug — print the push UUID, useful for support tickets.

The repo you pushed to is automatically rewritten in the manifest's sources: to point at the pushed ref, so the build operates on the version you just pushed, not the default branch. $GIT_REF is available in the build environment (e.g. refs/heads/master).

To restrict which refs trigger builds, use the in-manifest submitter: block instead of relying on complete-build or push options:

submitter:
  git.sr.ht:
    allow-refs:
      - refs/heads/master
      - "refs/tags/*"

This filters at submission time — non-matching refs never create a job. The block is interpreted by the git.sr.ht push hook, not by builds.sr.ht, so it only affects auto-submission. Manual submits via hut builds submit, the web form, or the GraphQL API ignore it entirely. See references/integrations.md.

The hook also rewrites the manifest before submission: it injects the cloning URL for the pushed commit into sources: (matching by repo basename, appending if no entry matches), sets BUILD_SUBMITTER=git.sr.ht and GIT_REF=<pushed ref> in environment:, and strips shell: true if present (with a log notice).

#Post-build triggers

Triggers fire after the build completes. Only two actions exist in the current builds.sr.ht code: email and webhook. Older docs and blog posts mention irc and job triggers — those do not exist, the worker logs Unknown trigger action and ignores them.

triggers:
  - action: email
    condition: failure
    to: "Me <me@example.com>"
  - action: webhook
    condition: always
    url: https://example.com/build-results

condition is one of success, failure, always. The webhook POSTs a signed JSON payload describing the job (see references/triggers.md for the shape).

For "deploy after tests pass", since there is no job: trigger:

  • From outside the build: have the webhook receiver submit a follow-up build via hut builds submit or the GraphQL submit mutation.
  • In-build: chain the deploy as a final task inside the same manifest, gated on the result of earlier tasks. Use submitter.allow-refs so the deploy only runs on branches where it's meant to.
  • Bundled across jobs: use the GraphQL createGroup mutation to bundle pending jobs into a group with a shared completion trigger; the trigger fires after every member finishes. Group-level triggers in the API accept EMAIL and WEBHOOK; only EMAIL is actually implemented in the worker today (webhook is a TODO).

See references/integrations.md for the webhook payload, mailing-list patch testing, and createGroup details.

#Debugging failed builds

When a build fails, the log prints an SSH command:

ssh -t builds@fra02.builds.sr.ht connect 1234567

You have ~4 hours to SSH in and poke around. The VM is exactly as the build left it. This is the single best feature for debugging — encourage users to use it rather than blindly iterating on the manifest.

For builds that pass but you want to inspect anyway, add shell: true to the manifest (keeps the VM alive even on success). Don't leave this in committed manifests — it ties up worker resources.

See references/debugging.md for: reading sr.ht log output, what set -xe reveals, how to interpret "no such image", debugging OAuth grant errors, why your tarball might be silently rejected by pages.sr.ht.

#Output format expectations

When the user asks to write a manifest:

  1. Lead with the manifest, not commentary. Put the YAML in a fenced code block as the first or second thing in the response.
  2. Make the manifest complete and submittable as-is. No ... placeholders unless the user has to fill in a value (username, UUID); when you do leave one, mark it <like-this> and call out below the manifest what they need to substitute.
  3. Use the simplest image: that works. Default to alpine/edge unless there's a specific reason (e.g. archlinux for AUR, debian/stable for Debian packaging). Don't reach for ubuntu/lts reflexively — Alpine is faster to boot and apk packages are usually what people want.
  4. Always include real OAuth grants and real environment values, not placeholder text. oauth: pages.sr.ht/PAGES:RW is correct; oauth: <scope here> is not — it forces the user to look up what the scope should be.
  5. Don't add features the user didn't ask for. A request for "deploy my Hugo site" doesn't need triggers, secrets, or artifacts. Keep manifests minimal and let the user add complexity.
  6. After the manifest, explain non-obvious choices in 1-3 sentences each. Why alpine/edge? Why oauth: instead of secrets:? Why this particular package name? Don't lecture; just unblock.
  7. For pages.sr.ht: always verify the tarball's structure. The -C <dir> . pattern is non-negotiable; if the user has the site at ./public/, the tar must be tar -czf site.tar.gz -C public . and not tar -czf site.tar.gz public/.

#Examples

Example 1 — User: "I have a Hugo site I want to deploy to my sourcehut subdomain."

image: alpine/edge
packages:
  - hugo
  - hut
oauth: pages.sr.ht/PAGES:RW
environment:
  site: username.srht.site
sources:
  - https://git.sr.ht/~username/my-site
tasks:
  - build: |
      cd my-site
      hugo --minify
      tar -czf ../site.tar.gz -C public .
  - publish: |
      hut pages publish -d "$site" site.tar.gz

Substitute username (twice) and the repo URL. Commit this as .build.yml in the repo root and push.

Example 2 — User: "I want CI to build my Go program and let me download the binary."

image: alpine/edge
packages:
  - go
sources:
  - https://git.sr.ht/~user/mytool
tasks:
  - build: |
      cd mytool
      go build -o /home/build/mytool ./cmd/mytool
  - test: |
      cd mytool
      go test ./...
artifacts:
  - mytool

The binary lives at /home/build/mytool after the build task; artifacts: takes the path relative to /home/build. The job page will show a download link for mytool. Artifacts are pruned after 90 days.

Example 3 — User: "I want to test on Alpine, Debian, and FreeBSD."

Create .builds/alpine.yml, .builds/debian.yml, .builds/freebsd.yml:

# .builds/alpine.yml
image: alpine/edge
packages:
  - meson
  - ninja
sources:
  - https://git.sr.ht/~user/myproject
tasks:
  - build: |
      cd myproject
      meson build && ninja -C build
  - test: |
      cd myproject
      ninja -C build test
# .builds/debian.yml
image: debian/stable
packages:
  - meson
  - ninja-build
sources:
  - https://git.sr.ht/~user/myproject
tasks:
  - build: |
      cd myproject
      meson build && ninja -C build
  - test: |
      cd myproject
      ninja -C build test
# .builds/freebsd.yml
image: freebsd/latest
packages:
  - meson
  - ninja
sources:
  - https://git.sr.ht/~user/myproject
tasks:
  - build: |
      cd myproject
      meson build && ninja -C build
  - test: |
      cd myproject
      ninja -C build test

Each is an independent job. Up to 4 of these submit per push. The package name differs (ninja on Alpine and FreeBSD, ninja-build on Debian) — this is normal and expected; sourcehut doesn't abstract over distros.