From 50df4c291341762cdb47437014ff157a91914add Mon Sep 17 00:00:00 2001 From: Eugene Blikh Date: Mon, 15 Jun 2026 22:55:28 +0300 Subject: [PATCH] Add sourcehut-custom-service skill Document how to build a custom service that integrates into a self-hosted SourceHut without modifying upstream source: config-driven nav/service-switcher, the shared sr.ht.unified-login.v1 cookie, theme reuse, GraphQL federation via api.sr.ht, and webhooks. Includes Python/Go/other-language recipes, a full configuration field reference, and guidance on reusing SourceHut's config key names and accessor helpers. --- .../skills/sourcehut-custom-service/SKILL.md | 262 ++++++++++++++++++ 1 file changed, 262 insertions(+) create mode 100644 .claude/skills/sourcehut-custom-service/SKILL.md diff --git a/.claude/skills/sourcehut-custom-service/SKILL.md b/.claude/skills/sourcehut-custom-service/SKILL.md new file mode 100644 index 0000000000000000000000000000000000000000..dae2c55d8f793e6da7182043c95d23b72fc9eaf5 --- /dev/null +++ b/.claude/skills/sourcehut-custom-service/SKILL.md @@ -0,0 +1,262 @@ +--- +name: sourcehut-custom-service +description: How to write your OWN service that integrates into a self-hosted SourceHut — appearing in the shared nav/service-switcher, reusing the unified login session, the shared theme, GraphQL federation, and webhooks, WITHOUT modifying upstream SourceHut source. Covers the integration mechanisms (all config-driven) and concrete recipes in Python, Go, and any other language. Use when the user asks about building a custom/third-party SourceHut service, a plugin, extending SourceHut, adding a service to their instance, or making something "show up like paste/todo/git" in the web UI. +--- + +# Writing a custom service that integrates into SourceHut + +## TL;DR + +SourceHut has **no plugin system** — nothing loads third-party code into a running service. But the things that make a service feel "integrated" (the nav service-switcher, the shared login session, the theme, the unified GraphQL endpoint, webhooks) are all **config-driven** and reusable from any language. Your service runs as its own separate process/daemon; you wire it in via `config.ini` + nginx, and optionally reuse `core.sr.ht` (Python) or `core-go` (Go) libraries. + +**This only works on a self-hosted instance you control** (config + nginx + DNS), e.g. the user's `*.srht.bigb.es`. You cannot add a service to hosted sr.ht. SourceHut services are tightly-coupled siblings sharing `core.sr.ht`/`core-go`, **not** a versioned plugin API — when you bump those on a refresh, your service can break and it's on you to track it. + +All file:line references below point into the read-only mirror at `~/data/home/sourcehut`. + +--- + +## The integration surfaces (all config-driven, language-agnostic) + +### 1. The nav / service-switcher +The list of services in the top nav is computed from config — **every section name ending in `.sr.ht`**: + +```python +# core.sr.ht/srht/app/flask.py:32 +_network = [ + s for s in config + if s.endswith(".sr.ht") and s not in ["paste.sr.ht", "pages.sr.ht"] +] +``` + +The nav template loops over `network` and links each via `get_origin()`, which just reads `[service] origin=` from config (`core.sr.ht/srht/templates/nav.html:17-32`, `core.sr.ht/srht/config.py:67`). + +→ **Add a `[myservice.sr.ht]` section with `origin=` to each service's `config.ini` and your service appears in the nav of every SourceHut web UI.** (paste/pages are hardcoded out of the switcher; your service won't be.) + +### 2. The unified login session (shared cookie — this is the important one) +Services do **not** each run a separate web-login OAuth dance. There is a single shared cookie, `sr.ht.unified-login.v1`, set on the **global domain** (e.g. `.srht.bigb.es`), httponly, containing the user's profile as Fernet-encrypted JSON: + +```python +# core.sr.ht/srht/app/flask.py:201-218 (read on every request) +cookie = request.cookies.get("sr.ht.unified-login.v1") +user_info = json.loads(fernet.decrypt(cookie.encode()).decode()) +user = self.oauth_service.lookup_user(user_info["name"]) +# ...g.current_user = user + +# core.sr.ht/srht/app/flask.py:267-285 (set after login) +response.set_cookie("sr.ht.unified-login.v1", + fernet.encrypt(user_info.encode()).decode(), + domain=global_domain, httponly=True, max_age=...) +``` + +The Fernet key is the shared **`[sr.ht] network-key`** from config (`core.sr.ht/srht/crypto.py:21`). meta.sr.ht sets this cookie when the user logs in; **every service — in any language — can read it, decrypt it with the same `network-key`, and know who the user is.** No per-service OAuth callback is needed just to render pages as the logged-in user. + +Login/logout are plain redirects to meta (`core.sr.ht/srht/app/flask.py:290-302`): +``` +{meta-origin}/login?return_to={your_url} +{meta-origin}/logout?return_to={your_url} +``` + +> Fernet = AES-128-CBC + HMAC-SHA256, base64url, with a version byte + timestamp + IV (the `cryptography` library's spec). Reimplementable in any language; Go has `github.com/fernet/fernet-go`. + +### 3. The shared theme +Just a compiled CSS asset. `core.sr.ht/scss/` is the Bootstrap-derived theme; each service's `scss/main.scss` imports it and the Makefile compiles to a hashed `main.min..css`. To match the look, **link/serve that compiled CSS** — you do not reimplement SCSS. + +### 4. The GraphQL federation gateway (`api.sr.ht`) +The aggregator merges per-service GraphQL schemas into one endpoint, also config-driven: + +```go +// api.sr.ht/main.go:30-35 +for name := range conf { + if strings.HasSuffix(name, ".sr.ht") { + services = append(services, thistle.NewService(name, getOrigin(conf, name))) + } +} +``` + +It fetches each service's `/query` schema and runs `thistle.BuildSchema(...)` (`api.sr.ht/main.go:140-168`). Add a config section pointing at your service's GraphQL `/query` endpoint and your types join the unified API — **no edit to api.sr.ht**, just config + a SIGHUP to reload (`main.go:74-87`). Internal service-to-service calls use HMAC auth (`core-go/auth/internal.go`, `api.sr.ht/auth.go` `InternalAuthTransport`). + +### 5. Webhooks +React to events from other services: GraphQL-native (`core-go/webhooks/`, `/api/webhooks/`) or legacy HTTP (`core.sr.ht/srht/webhook/`). Smaller services (`paste`, `man`, `pages`) may not emit them. + +--- + +## The two halves of a service + +| Half | What it is | Go support | Python support | +| --- | --- | --- | --- | +| **API / backend** | GraphQL API, DB, jobs, federation | **First-class** (`core-go` + gqlgen + thistle). The blessed path. | Yes (`core.sr.ht` GraphQL helpers) | +| **Web chrome** | nav switcher, login session, theme, rendered pages | **No shared code** — reimplement (small; see below) | **Free** via `srht.app.Flask` | + +The web-chrome code (shared Jinja `layout.html`/`nav.html` + `srht.app.Flask`) lives **only in `core.sr.ht` (Python)**. `core-go` is purely API-side (`auth`, `config`, `database`, `redis`, `server`, `webhooks`, `crypto`, `objects`) — no HTML templating, no session management. The Go-only services (`api.sr.ht`, `pages.sr.ht`, `sourcehut-ssh`) render **no** integrated web UI; `pages.sr.ht` has zero `.html` files. + +--- + +## Recipe A — Python web tier (the standard pattern, chrome for free) + +This is how every user-facing service is built. Subclass `srht.app.Flask` and you inherit nav + unified login + theme + GraphQL blueprint + error pages. paste's entire bootstrap is ~35 lines (`paste.sr.ht/pastesrht/app.py`): + +```python +from srht.app import Flask +from srht.config import cfg +from srht.database import DbSession + +db = DbSession(cfg("myservice.sr.ht", "connection-string")); db.init() + +class MyServiceApp(Flask): + def __init__(self): + super().__init__("myservice.sr.ht", __name__, user_class=User) + from myservicesrht.blueprints.public import public + from srht.graphql import gql_blueprint + self.register_blueprint(public) + self.register_blueprint(gql_blueprint) + +app = MyServiceApp() +``` + +Templates `{% extends "layout.html" %}`. Build against an installed `core.sr.ht` (`pip install -e ../core.sr.ht`). Best when you want the integrated shell with minimal work. + +## Recipe B — Pure Go (the user's preference) + +**API side:** straightforward and blessed — `core-go` gives you `auth` (validate meta OAuth bearer tokens: `core-go/auth/bearer.go`, `middleware.go`), `config`, `database`, `redis`, gqlgen scaffolding (`core-go/server/`), `webhooks`, `objects` (S3). Federate into `api.sr.ht` via thistle (config only). + +**Web side — reimplement the chrome, which is small:** +1. **Nav**: ~30 lines of `html/template` mirroring `nav.html`, reading the same config (`network` = config sections ending `.sr.ht`). +2. **Theme**: link/serve the already-compiled `main.min.css` from `core.sr.ht`'s build output. No reimplementation. +3. **Login/identity**: read the `sr.ht.unified-login.v1` cookie, Fernet-decrypt with `[sr.ht] network-key` (e.g. `github.com/fernet/fernet-go`), get `current_user`. For login, redirect to `{meta-origin}/login?return_to=...`; meta sets the shared cookie on the parent domain. For **write/API** actions on behalf of the user, also obtain an OAuth token (register an OAuthClient on meta, validate via `core-go/auth`). + +That's the whole gap between Go and "looks like paste": a small nav template + a cookie-reading login handler. Everything else is `core-go`. + +## Recipe C — Go API only, no web page + +If you just need your types in the unified GraphQL endpoint (no browser page in the shell): build a Go gqlgen API, register it in `api.sr.ht`'s config, done. Fully Go. + +## Recipe D — Any other language + +The integration contract is just HTTP + config + a known cookie format, so any stack works if it can: +- serve under a subdomain wired in nginx; +- read `[sr.ht] network-key` and Fernet-decrypt the `sr.ht.unified-login.v1` cookie for identity (or treat all traffic as anonymous + redirect to meta for login); +- (optional) speak the gqlgen/thistle federation conventions to join `api.sr.ht`; +- link the shared `main.min.css` and replicate the ~30-line nav from config. + +There is no language lock-in — `core.sr.ht`/`core-go` are conveniences, not requirements. + +--- + +## Wiring checklist (self-hosted) + +1. **Build** your service (Recipe A/B/C/D). +2. **Config** — add to the `config.ini` of **every** service (each independently builds its own nav): + ```ini + [myservice.sr.ht] + origin=https://myservice.srht.bigb.es + #api-origin=https://myservice.srht.bigb.es # if federating + connection-string=postgresql://.../myservice + ``` + Your service also needs `[sr.ht] network-key` (the shared Fernet key) and `[meta.sr.ht] origin=` to read the login cookie and link login/register. +3. **nginx** — add a `server`/subdomain route to reach your app (`sr.ht-nginx/` style). +4. **api.sr.ht** (optional federation) — the matching config section + SIGHUP; types merge in. +5. **DNS** — `myservice.srht.bigb.es` under the same global domain as the rest, so the shared cookie (`domain=.srht.bigb.es`) is visible to your service. + +--- + +## Configuration field reference + +SourceHut config is INI (`config.ini`, parsed by `core-go/config/config.go` and `core.sr.ht/srht/config.py`). A custom service draws from four kinds of section. **Generate keys with `sr.ht-keygen {service,network,webhook}`** — never hand-roll them. + +### `[sr.ht]` — global, must match the rest of the instance +| Field | Required | Who reads it | Notes | +| --- | --- | --- | --- | +| `network-key` | **Yes** | `core-go/crypto/crypto.go:38`, `core.sr.ht/srht/crypto.py:21` | **The single most important field.** Shared Fernet key used to decrypt the `sr.ht.unified-login.v1` cookie and sign internal messages. **Must be byte-identical across every service** or you can't read who's logged in. `sr.ht-keygen network`. | +| `service-key` | Recommended | `core.sr.ht` session layer | Encrypts this service's own session cookies. May differ per service; identical is fine if you share one config. `sr.ht-keygen service`. | +| `redis-host` | If you use Redis | `core-go/server/server.go:302` | Cache / pubsub / webhook queue. Shared between nodes of a service. | +| `internal-ipnet` | Recommended | `core-go/config/config.go:71` | CIDRs trusted as internal (service-to-service) callers. Defaults to loopback+private ranges; set explicitly to match your cluster. | +| `owner-name`, `owner-email` | **Yes (Go)** | `core-go/config/config.go:89` `GetOwner` — **panics if missing** | Required by any `core-go` service at startup. | +| `site-name` | Recommended | nav/templates (`site_name`) | Brand text shown in the nav across all services. | +| `environment` | Recommended | `layout.html:25` | Anything other than `production` shows a colored banner; admins always see it. | +| `site-info`, `site-blurb`, `source-url` | Optional | templates | Cosmetic / footer links. | + +### `[meta.sr.ht]` — the identity provider, always needed +| Field | Required | Notes | +| --- | --- | --- | +| `origin` | **Yes** | Used to build `login`/`logout`/register URLs (`flask.py:290-302`) and to validate tokens. Without it your service can't send users to log in. | +| `oauth-client-id`, `oauth-client-secret` | If you make API calls *as a client* | Register an OAuth client on meta and put its credentials here (pattern: `git.sr.ht/config.example.ini:169-171` declares the *integrated* service's id under its own section). Only needed for write/API actions on the user's behalf — **not** needed merely to read the login cookie. | + +### `[myservice.sr.ht]` — your own section (the name MUST end in `.sr.ht`) +The `.sr.ht` suffix is what puts you in the nav `network` list (`flask.py:32`) and the federation loop (`api.sr.ht/main.go:31`). Pick a service "prefix" (the part before `.sr.ht`) that's unique on the instance. +| Field | Required | Who reads it | Notes | +| --- | --- | --- | --- | +| `origin` | **Yes** | nav, `get_origin` (`config.py:67`, `core-go/config/config.go:107`) | External `scheme://host` your web UI is served at. Drives every cross-service link to you. | +| `connection-string` | If you have a DB | `core-go/server/server.go:288`, `core.sr.ht` `DbSession` | Postgres DSN. | +| `internal-origin` | Optional | `GetOrigin` (internal pref.) | LAN/cluster URL preferred for service-to-service traffic; falls back to `origin`. | +| `api-origin` | If you expose an API | `GetAPI` (`core-go/config/config.go:125`) | External API URL. `api.sr.ht` federates `api-origin + "/query"`. | +| `api-internal-origin` | Optional | `GetAPI` (internal pref.) | Internal API URL for the federation gateway / other services. | +| `migrate-on-upgrade` | Optional | packaging | `yes` to auto-run migrations on package upgrade. | +| `webhooks` | If you emit webhooks | webhook worker | Redis URL/db for the webhook queue (e.g. `redis://localhost:6379/1`). | +| `debug-host`, `debug-port` | Dev only | debug server | Bind address for `run.py` / local dev. Pick a port not used by another service. | +| `s3-bucket`, `s3-prefix` | If you store objects | `core-go/objects` | Leave bucket empty to disable object storage. | + +### `[webhooks]` — only if your service signs outgoing webhook payloads +| Field | Required | Who reads it | Notes | +| --- | --- | --- | --- | +| `private-key` | If emitting webhooks | `core-go/crypto/crypto.go:27` | base64 Ed25519 signing key, **shared across services**. `sr.ht-keygen webhook`. Distribute the public half to consumers. | +| `queue-size` | Optional | `core-go/webhooks/queue.go:47` | Defaults to `config.DefaultQueueSize`. | + +### `[mail]` — only if your service sends email +`smtp-host`, `smtp-port`, `smtp-user`, `smtp-password`, `smtp-from`, `smtp-encryption` (`starttls`/`tls`/`insecure`), `smtp-auth` (`plain`/`none`), `error-to`, `error-from`, and `pgp-privkey`/`pgp-pubkey`/`pgp-key-id` for signing. Skip the whole section if you don't email. + +### Reuse the names and accessors — don't invent your own + +Two reasons to mirror upstream's config vocabulary instead of designing fresh keys: (1) you can read config with the **shared accessor helpers**, which already encode SourceHut's resolution rules; (2) anyone who knows SourceHut configs (or runs one shared `config.ini`) can configure your service without surprises. + +**Read config through the shared helpers, not raw INI parsing** — they give you internal/external origin fallback, `api-origin` defaulting, S3 resolution, and the internal-IP trust check for free: + +| Concept | Go (`core-go/config`) | Python (`srht.config`) | +| --- | --- | --- | +| Load file | `LoadConfig()` (`config.go:31`) | `load_config()` (`config.py:23`) | +| String / int / bool | `GetString` / `GetInt` / `GetBool` (`:156`/`:167`/`:181`) | `cfg` / `cfgi` / `cfgb` (`:35`/`:44`/`:50`) | +| A service's web URL | `GetOrigin(conf, svc, external)` (`:107`) | `get_origin(svc, external=)` (`:67`) | +| A service's API URL | `GetAPI(conf, svc, external)` (`:125`) | `get_api(svc, external=)` (`:80`) | +| Owner name/email | `GetOwner` (`:89`) | — | +| Internal-caller check | `IsInternalIP(ip)` (`:198`) | — | +| Global domain | — | `get_global_domain(site)` (`:107`) | +| S3 upstream | — | `get_s3_upstream()` (`:119`) | + +**Use the canonical key names for your `[myservice.sr.ht]` section** — every other service uses these exact spellings, so reuse them rather than coining synonyms: + +| Use this (canonical) | Not these (invented) | Why | +| --- | --- | --- | +| `origin` | `url`, `base-url`, `web-url` | `GetOrigin`/`get_origin` look up exactly `origin` (+ `internal-origin`). | +| `internal-origin` | `lan-url`, `private-origin` | The internal-preferred fallback partner of `origin`. | +| `api-origin` / `api-internal-origin` | `graphql-url`, `api-url` | `GetAPI`/`get_api` probe these names in a fixed order. | +| `connection-string` | `db`, `dsn`, `database-url`, `pg` | `core-go/server/server.go:288` and `DbSession` read `connection-string`. | +| `redis-host` | `redis`, `redis-url` | Read from `[sr.ht]`, not your section (`server.go:302`). | +| `migrate-on-upgrade` | `auto-migrate` | Packaging convention. | +| `webhooks` | `webhook-redis`, `hooks-db` | Redis URL for the webhook queue. | +| `debug-host` / `debug-port` | `host` / `port`, `bind` | Dev-server bind convention. | +| `s3-bucket` / `s3-prefix` | `bucket` / `object-store` | `core-go/objects` + `get_s3_upstream` expect these. | + +**Reuse the global keys in place — don't re-declare them.** `network-key`, `service-key`, `redis-host`, `internal-ipnet`, `owner-*` live in `[sr.ht]` and are read there by the helpers. Read them via `cfg("sr.ht", "network-key")` / `config.Get("sr.ht", ...)`; do **not** add a private copy under your own section. + +**Reference other services by their canonical section name.** To talk to meta/git/etc., call `get_origin("meta.sr.ht")` / `GetAPI(conf, "git.sr.ht", false)` — you integrate by *reading* their well-known sections, never by hardcoding URLs. This is also why your section name must be the real `myservice.sr.ht` string everywhere: other tools (and `api.sr.ht`) find you by that exact key. + +### Two rules that bite people +- **Shared-vs-per-service.** `network-key` and `[webhooks] private-key` **must be identical everywhere**; `service-key` and `redis-host` may differ per service. Getting `network-key` wrong = "logged out" on your service even though the user is logged in. +- **Every service reads its OWN copy of config to build the nav.** Adding `[myservice.sr.ht]` to *your* config alone is not enough — each existing service must also have an `[myservice.sr.ht] origin=` entry, or you won't appear in *their* nav. The simplest operationally-correct setup is one shared `config.ini` distributed to all services (which is why upstream notes you *may* use one `service-key` for all). + +## Caveats (state these every time) + +- **Self-hosted only.** Requires control of config + nginx + DNS. Not possible on hosted sr.ht. +- **No stable plugin API.** You couple to `core.sr.ht`/`core-go` internals; refreshes can break you. +- **Cookie ⇒ identity, not authorization.** The unified-login cookie tells you *who* is browsing; for API writes you still need a real OAuth token scoped via meta.sr.ht. +- **`dispatch.sr.ht` is gone** — the old "third-party integrations" service was removed upstream. (The `dispatch` package in `sourcehut-ssh` is unrelated SSH shell dispatch.) + +## Key files to re-read when working on this + +- `core.sr.ht/srht/app/flask.py` — `_network` (nav list, :32), unified-login cookie read (:201-218) + write (:267-285), `login_url`/`logout_url` (:290-302) +- `core.sr.ht/srht/templates/nav.html`, `layout.html` — the shared chrome +- `core.sr.ht/srht/config.py:67` `get_origin` / `:80` `get_api`; `core.sr.ht/srht/crypto.py:21` Fernet key +- `core-go/config/config.go` — `:71` `internal-ipnet`, `:89` `GetOwner` (panics), `:107` `GetOrigin`, `:125` `GetAPI`; `core-go/crypto/crypto.go:27` webhook key / `:38` `network-key`; `core-go/server/server.go:288` `connection-string` / `:302` `redis-host` +- `*/config.example.ini` — real field names per service (`paste.sr.ht` minimal, `git.sr.ht` rich: S3, dispatch, optional `[builds.sr.ht]` integration block at `:168-171`) +- `paste.sr.ht/pastesrht/app.py` — minimal Python service bootstrap +- `api.sr.ht/main.go:26-168` — federation gateway (config-driven, thistle) +- `core-go/auth/` (`bearer.go`, `middleware.go`, `internal.go`) — Go token validation + internal HMAC