~bigbes/sourcehut-root

sourcehut-root/skills/sourcehut-custom-service/SKILL.md -rw-r--r-- 21.2 KiB
d9aac8a2 — Eugene Blikh Make sourcehut-custom-service a global skill installed via Justfile 26 days ago

#name: sourcehut-custom-service description: How to write your OWN service that integrates into a self-hosted SourceHut — appearing in the shared nav/service-switcher, reusing the unified login session, the shared theme, GraphQL federation, and webhooks, WITHOUT modifying upstream SourceHut source. Covers the integration mechanisms (all config-driven) and concrete recipes in Python, Go, and any other language. Use when the user asks about building a custom/third-party SourceHut service, a plugin, extending SourceHut, adding a service to their instance, or making something "show up like paste/todo/git" in the web UI.

#Writing a custom service that integrates into SourceHut

#TL;DR

SourceHut has no plugin system — nothing loads third-party code into a running service. But the things that make a service feel "integrated" (the nav service-switcher, the shared login session, the theme, the unified GraphQL endpoint, webhooks) are all config-driven and reusable from any language. Your service runs as its own separate process/daemon; you wire it in via config.ini + nginx, and optionally reuse core.sr.ht (Python) or core-go (Go) libraries.

This only works on a self-hosted instance you control (config + nginx + DNS), e.g. the user's *.srht.bigb.es. You cannot add a service to hosted sr.ht. SourceHut services are tightly-coupled siblings sharing core.sr.ht/core-go, not a versioned plugin API — when you bump those on a refresh, your service can break and it's on you to track it.

#Reference notation

Citations are written repo::<path>::<symbol>: open <path> relative to the mirror root and search for <symbol> (a function, class, template anchor, or config section). Symbol names survive refreshes; line numbers don't, so they are deliberately omitted. The mirror root is the single absolute path in this skill, substituted at install time:

__SRHT_MIRROR__

(In this workspace that is the repo holding the SourceHut documentation mirror. The just install target rewrites the token to the real path.)


#The integration surfaces (all config-driven, language-agnostic)

#1. The nav / service-switcher

The list of services in the top nav is computed from config — every section name ending in .sr.ht:

# repo::core.sr.ht/srht/app/flask.py::_network
_network = [
    s for s in config
    if s.endswith(".sr.ht") and s not in ["paste.sr.ht", "pages.sr.ht"]
]

The nav template loops over network and links each via get_origin(), which just reads [service] origin= from config (repo::core.sr.ht/srht/templates/nav.html::for _site in network, repo::core.sr.ht/srht/config.py::get_origin).

Add a [myservice.sr.ht] section with origin= to each service's config.ini and your service appears in the nav of every SourceHut web UI. (paste/pages are hardcoded out of the switcher; your service won't be.)

Services do not each run a separate web-login OAuth dance. There is a single shared cookie, sr.ht.unified-login.v1, set on the global domain (e.g. .srht.bigb.es), httponly, containing the user's profile as Fernet-encrypted JSON:

# repo::core.sr.ht/srht/app/flask.py::get_session_cookie   (read on every request)
cookie = request.cookies.get("sr.ht.unified-login.v1")
user_info = json.loads(fernet.decrypt(cookie.encode()).decode())
user = self.oauth_service.lookup_user(user_info["name"])
# ...g.current_user = user

# repo::core.sr.ht/srht/app/flask.py::make_response         (set after login)
response.set_cookie("sr.ht.unified-login.v1",
    fernet.encrypt(user_info.encode()).decode(),
    domain=global_domain, httponly=True, max_age=...)

The Fernet key is the shared [sr.ht] network-key from config (repo::core.sr.ht/srht/crypto.py::fernet). meta.sr.ht sets this cookie when the user logs in; every service — in any language — can read it, decrypt it with the same network-key, and know who the user is. No per-service OAuth callback is needed just to render pages as the logged-in user.

Login/logout are plain redirects to meta (repo::core.sr.ht/srht/app/flask.py::login_url, ::logout_url):

{meta-origin}/login?return_to={your_url}
{meta-origin}/logout?return_to={your_url}

Fernet = AES-128-CBC + HMAC-SHA256, base64url, with a version byte + timestamp + IV (the cryptography library's spec). Reimplementable in any language; Go has github.com/fernet/fernet-go (the same library core-go uses).

#3. The shared theme

Just a compiled CSS asset. repo::core.sr.ht/scss/ is the Bootstrap-derived theme; each service's scss/main.scss imports it and the Makefile compiles to a hashed main.min.<sha>.css. To match the look, link/serve that compiled CSS — you do not reimplement SCSS.

#4. The GraphQL federation gateway (api.sr.ht)

The aggregator merges per-service GraphQL schemas into one endpoint, also config-driven:

// repo::api.sr.ht/main.go::main
for name := range conf {
    if strings.HasSuffix(name, ".sr.ht") {
        services = append(services, thistle.NewService(name, getOrigin(conf, name)))
    }
}

It fetches each service's /query schema and runs thistle.BuildSchema(...) (repo::api.sr.ht/main.go::updateSchema). Add a config section pointing at your service's GraphQL /query endpoint and your types join the unified API — no edit to api.sr.ht, just config + a SIGHUP to reload (handled in repo::api.sr.ht/main.go::main). Internal service-to-service calls use HMAC auth (repo::api.sr.ht/auth.go::InternalAuthTransport on the caller side, repo::core-go/auth/middleware.go::internalAuth on the receiver side).

#5. Webhooks

React to events from other services: GraphQL-native (repo::core-go/webhooks/queue.go::NewQueue, plus per-service api/webhooks/) or legacy HTTP (repo::core.sr.ht/srht/webhook/). Smaller services (paste, man, pages) may not emit them.


#The two halves of a service

Half What it is Go support Python support
API / backend GraphQL API, DB, jobs, federation First-class (core-go + gqlgen + thistle). The blessed path. Yes (core.sr.ht GraphQL helpers)
Web chrome nav switcher, login session, theme, rendered pages No shared code — reimplement (small; see below) Free via srht.app.Flask

The web-chrome code (shared Jinja layout.html/nav.html + repo::core.sr.ht/srht/app/flask.py::Flask) lives only in core.sr.ht (Python). core-go is purely API-side (auth, config, database, redis, server, webhooks, crypto, objects) — no HTML templating. (It does have API-side cookie auth in repo::core-go/auth/middleware.go::cookieAuth, but no rendered web UI / Jinja chrome.) The Go-only services (api.sr.ht, pages.sr.ht, sourcehut-ssh) render no integrated web UI; pages.sr.ht has zero .html files.


#Recipe A — Python web tier (the standard pattern, chrome for free)

This is how every user-facing service is built. Subclass srht.app.Flask and you inherit nav + unified login + theme + GraphQL blueprint + error pages. paste's entire bootstrap is ~35 lines (repo::paste.sr.ht/pastesrht/app.py::PasteApp):

from srht.app import Flask
from srht.config import cfg
from srht.database import DbSession

db = DbSession(cfg("myservice.sr.ht", "connection-string")); db.init()

class MyServiceApp(Flask):
    def __init__(self):
        super().__init__("myservice.sr.ht", __name__, user_class=User)
        from myservicesrht.blueprints.public import public
        from srht.graphql import gql_blueprint
        self.register_blueprint(public)
        self.register_blueprint(gql_blueprint)

app = MyServiceApp()

Templates {% extends "layout.html" %}. Build against an installed core.sr.ht (pip install -e ../core.sr.ht). Best when you want the integrated shell with minimal work.

#Recipe B — Pure Go (the user's preference)

API side: straightforward and blessed — core-go gives you auth (validate meta OAuth bearer tokens: repo::core-go/auth/bearer.go::DecodeBearerToken, repo::core-go/auth/middleware.go::Middleware), config, database, redis, gqlgen scaffolding (repo::core-go/server/server.go::WithDefaultMiddleware), webhooks, objects (S3). Federate into api.sr.ht via thistle (config only).

Web side — reimplement the chrome, which is small:

  1. Nav: ~30 lines of html/template mirroring nav.html, reading the same config (network = config sections ending .sr.ht).
  2. Theme: link/serve the already-compiled main.min.css from core.sr.ht's build output. No reimplementation.
  3. Login/identity: read the sr.ht.unified-login.v1 cookie, Fernet-decrypt with [sr.ht] network-key (github.com/fernet/fernet-go), get the user. For login, redirect to {meta-origin}/login?return_to=...; meta sets the shared cookie on the parent domain. For write/API actions on behalf of the user, also obtain an OAuth token (register an OAuthClient on meta, validate via repo::core-go/auth/middleware.go::Middleware).

That's the whole gap between Go and "looks like paste": a small nav template + a cookie-reading login handler. Everything else is core-go.

#Recipe C — Go API only, no web page

If you just need your types in the unified GraphQL endpoint (no browser page in the shell): build a Go gqlgen API, register it in api.sr.ht's config, done. Fully Go.

#Recipe D — Any other language

The integration contract is just HTTP + config + a known cookie format, so any stack works if it can:

  • serve under a subdomain wired in nginx;
  • read [sr.ht] network-key and Fernet-decrypt the sr.ht.unified-login.v1 cookie for identity (or treat all traffic as anonymous + redirect to meta for login);
  • (optional) speak the gqlgen/thistle federation conventions to join api.sr.ht;
  • link the shared main.min.css and replicate the ~30-line nav from config.

There is no language lock-in — core.sr.ht/core-go are conveniences, not requirements.


#Wiring checklist (self-hosted)

  1. Build your service (Recipe A/B/C/D).
  2. Config — add to the config.ini of every service (each independently builds its own nav):
    [myservice.sr.ht]
    origin=https://myservice.srht.bigb.es
    #api-origin=https://myservice.srht.bigb.es   # if federating
    connection-string=postgresql://.../myservice
    
    Your service also needs [sr.ht] network-key (the shared Fernet key) and [meta.sr.ht] origin= to read the login cookie and link login/register.
  3. nginx — add a server/subdomain route to reach your app (repo::sr.ht-nginx/ style).
  4. api.sr.ht (optional federation) — the matching config section + SIGHUP; types merge in.
  5. DNSmyservice.srht.bigb.es under the same global domain as the rest, so the shared cookie (domain=.srht.bigb.es) is visible to your service.

#Configuration field reference

SourceHut config is INI (config.ini, parsed by repo::core-go/config/config.go::LoadConfig and repo::core.sr.ht/srht/config.py::load_config). A custom service draws from four kinds of section. Generate keys with sr.ht-keygen {service,network,webhook} — never hand-roll them.

#[sr.ht] — global, must match the rest of the instance

Field Required Who reads it Notes
network-key Yes repo::core-go/crypto/crypto.go::InitCrypto, repo::core.sr.ht/srht/crypto.py::fernet The single most important field. Shared Fernet key used to decrypt the sr.ht.unified-login.v1 cookie and sign internal messages. Must be byte-identical across every service or you can't read who's logged in. sr.ht-keygen network.
service-key Recommended core.sr.ht session layer Encrypts this service's own session cookies. May differ per service; identical is fine if you share one config. sr.ht-keygen service.
redis-host If you use Redis repo::core-go/server/server.go::WithDefaultMiddleware Cache / pubsub / webhook queue. Shared between nodes of a service.
internal-ipnet Recommended repo::core-go/config/config.go::LoadConfig CIDRs trusted as internal (service-to-service) callers. Defaults to loopback+private ranges; set explicitly to match your cluster.
owner-name, owner-email Yes (Go) repo::core-go/config/config.go::GetOwnerpanics if missing Required by any core-go service at startup.
site-name Recommended nav/templates (site_name) Brand text shown in the nav across all services.
environment Recommended repo::core.sr.ht/srht/templates/layout.html::ENVIRONMENT Anything other than production shows a colored banner; admins always see it.
site-info, site-blurb, source-url Optional templates Cosmetic / footer links.

#[meta.sr.ht] — the identity provider, always needed

Field Required Notes
origin Yes Used to build login/logout/register URLs (repo::core.sr.ht/srht/app/flask.py::login_url) and to validate tokens. Without it your service can't send users to log in.
oauth-client-id, oauth-client-secret If you make API calls as a client Register an OAuth client on meta and put its credentials here (pattern: repo::git.sr.ht/config.example.ini::[builds.sr.ht] declares the integrated service's id under its own section). Only needed for write/API actions on the user's behalf — not needed merely to read the login cookie.

#[myservice.sr.ht] — your own section (the name MUST end in .sr.ht)

The .sr.ht suffix is what puts you in the nav network list (repo::core.sr.ht/srht/app/flask.py::_network) and the federation loop (repo::api.sr.ht/main.go::main). Pick a service "prefix" (the part before .sr.ht) that's unique on the instance.

Field Required Who reads it Notes
origin Yes nav, repo::core.sr.ht/srht/config.py::get_origin, repo::core-go/config/config.go::GetOrigin External scheme://host your web UI is served at. Drives every cross-service link to you.
connection-string If you have a DB repo::core-go/server/server.go::WithDefaultMiddleware, core.sr.ht DbSession Postgres DSN.
internal-origin Optional GetOrigin (internal pref.) LAN/cluster URL preferred for service-to-service traffic; falls back to origin.
api-origin If you expose an API repo::core-go/config/config.go::GetAPI External API URL. api.sr.ht federates api-origin + "/query".
api-internal-origin Optional GetAPI (internal pref.) Internal API URL for the federation gateway / other services.
migrate-on-upgrade Optional packaging yes to auto-run migrations on package upgrade.
webhooks If you emit webhooks webhook worker Redis URL/db for the webhook queue (e.g. redis://localhost:6379/1).
debug-host, debug-port Dev only debug server Bind address for run.py / local dev. Pick a port not used by another service.
s3-bucket, s3-prefix If you store objects core-go/objects Leave bucket empty to disable object storage.

#[webhooks] — only if your service signs outgoing webhook payloads

Field Required Who reads it Notes
private-key If emitting webhooks repo::core-go/crypto/crypto.go::InitCrypto base64 Ed25519 signing key, shared across services. sr.ht-keygen webhook. Distribute the public half to consumers.
queue-size Optional repo::core-go/webhooks/queue.go::NewQueue Defaults to config.DefaultQueueSize.

#[mail] — only if your service sends email

smtp-host, smtp-port, smtp-user, smtp-password, smtp-from, smtp-encryption (starttls/tls/insecure), smtp-auth (plain/none), error-to, error-from, and pgp-privkey/pgp-pubkey/pgp-key-id for signing. Skip the whole section if you don't email.

#Reuse the names and accessors — don't invent your own

Two reasons to mirror upstream's config vocabulary instead of designing fresh keys: (1) you can read config with the shared accessor helpers, which already encode SourceHut's resolution rules; (2) anyone who knows SourceHut configs (or runs one shared config.ini) can configure your service without surprises.

Read config through the shared helpers, not raw INI parsing — they give you internal/external origin fallback, api-origin defaulting, S3 resolution, and the internal-IP trust check for free:

Concept Go — repo::core-go/config/config.go::<sym> Python — repo::core.sr.ht/srht/config.py::<sym>
Load file LoadConfig load_config
String / int / bool GetString / GetInt / GetBool cfg / cfgi / cfgb
A service's web URL GetOrigin get_origin
A service's API URL GetAPI get_api
Owner name/email GetOwner
Internal-caller check IsInternalIP
Global domain get_global_domain
S3 upstream get_s3_upstream

Use the canonical key names for your [myservice.sr.ht] section — every other service uses these exact spellings, so reuse them rather than coining synonyms:

Use this (canonical) Not these (invented) Why
origin url, base-url, web-url GetOrigin/get_origin look up exactly origin (+ internal-origin).
internal-origin lan-url, private-origin The internal-preferred fallback partner of origin.
api-origin / api-internal-origin graphql-url, api-url GetAPI/get_api probe these names in a fixed order.
connection-string db, dsn, database-url, pg repo::core-go/server/server.go::WithDefaultMiddleware and DbSession read connection-string.
redis-host redis, redis-url Read from [sr.ht], not your section (repo::core-go/server/server.go::WithDefaultMiddleware).
migrate-on-upgrade auto-migrate Packaging convention.
webhooks webhook-redis, hooks-db Redis URL for the webhook queue.
debug-host / debug-port host / port, bind Dev-server bind convention.
s3-bucket / s3-prefix bucket / object-store core-go/objects + get_s3_upstream expect these.

Reuse the global keys in place — don't re-declare them. network-key, service-key, redis-host, internal-ipnet, owner-* live in [sr.ht] and are read there by the helpers. Read them via cfg("sr.ht", "network-key") / config.Get("sr.ht", ...); do not add a private copy under your own section.

Reference other services by their canonical section name. To talk to meta/git/etc., call get_origin("meta.sr.ht") / GetAPI(conf, "git.sr.ht", false) — you integrate by reading their well-known sections, never by hardcoding URLs. This is also why your section name must be the real myservice.sr.ht string everywhere: other tools (and api.sr.ht) find you by that exact key.

#Two rules that bite people

  • Shared-vs-per-service. network-key and [webhooks] private-key must be identical everywhere; service-key and redis-host may differ per service. Getting network-key wrong = "logged out" on your service even though the user is logged in.
  • Every service reads its OWN copy of config to build the nav. Adding [myservice.sr.ht] to your config alone is not enough — each existing service must also have an [myservice.sr.ht] origin= entry, or you won't appear in their nav. The simplest operationally-correct setup is one shared config.ini distributed to all services (which is why upstream notes you may use one service-key for all).

#Caveats (state these every time)

  • Self-hosted only. Requires control of config + nginx + DNS. Not possible on hosted sr.ht.
  • No stable plugin API. You couple to core.sr.ht/core-go internals; refreshes can break you.
  • Cookie ⇒ identity, not authorization. The unified-login cookie tells you who is browsing; for API writes you still need a real OAuth token scoped via meta.sr.ht.
  • dispatch.sr.ht is gone — the old "third-party integrations" service was removed upstream. (The dispatch package in sourcehut-ssh is unrelated SSH shell dispatch.)

#Key files to re-read when working on this

  • repo::core.sr.ht/srht/app/flask.py::_network (nav list), ::get_session_cookie (unified-login read) + ::make_response (write), ::login_url / ::logout_url, ::Flask (the base class)
  • repo::core.sr.ht/srht/templates/nav.html, repo::core.sr.ht/srht/templates/layout.html — the shared chrome
  • repo::core.sr.ht/srht/config.py::get_origin / ::get_api; repo::core.sr.ht/srht/crypto.py::fernet — Fernet key
  • repo::core-go/config/config.go::LoadConfig (incl. internal-ipnet), ::GetOwner (panics), ::GetOrigin, ::GetAPI; repo::core-go/crypto/crypto.go::InitCrypto (webhook key + network-key); repo::core-go/server/server.go::WithDefaultMiddleware (connection-string + redis-host); repo::core-go/auth/middleware.go::Middleware / ::cookieAuth / ::internalAuth
  • */config.example.ini — real field names per service (paste.sr.ht minimal, git.sr.ht rich: S3, dispatch, optional repo::git.sr.ht/config.example.ini::[builds.sr.ht] integration block)
  • repo::paste.sr.ht/pastesrht/app.py::PasteApp — minimal Python service bootstrap
  • repo::api.sr.ht/main.go::main + ::updateSchema — federation gateway (config-driven, thistle)