~bigbes/sr-ht-ecore

ref: b36a927213562090e100c8b43ab27ef9094b1de9 sr-ht-ecore/README.md -rw-r--r-- 12.0 KiB
b36a9272 — Eugene Blikh beads: ignore the JSONL exports a day ago

#sr-ht-ecore

Extended core for the custom services of a self-hosted SourceHut instance (diff, spec, dolt, cov, bench, ...). Everything these services share that is ours — not upstream's — lives here, so the sr-ht-core fork can stay a clean mirror of upstream core-go, and so the services stop carrying drifting copies of the same code.

#Packages

  • chrome — the shared page chrome: service-switcher nav built from the shared config.ini (chrome.BuildNav), per-request chrome.Page with login/logout/profile URLs against meta.sr.ht's unified login, embedded srht-nav / srht-env-banner / srht-head-links template partials (circle brand + red service label + switcher + login box, and the stylesheet and favicon links), the srht-repo-list and srht-repo-table listings over one dot, and the generic template helpers (dict, shortsha, reltime, abstime). A service that ships no icon still gets one: FaviconHref defaults to the brand's ring as a data: URI, which — unlike a path into a static tree the binary does not have — cannot 404 once per page load.
  • grants — the grant vocabulary of tokens.sr.ht (SPEC ch. 3): <service>:<action> members split on ASCII whitespace, * for every action of every service, the reserved id:<n> member a registered token carries, and the subset rule an exchange narrows by. Shared because the daemon that mints and every service that validates have to read one grammar — two parsers that disagree about what counts as a permission is a hole on the security path, not a cosmetic divergence.
  • bearer — the shared working-token validator (SPEC ch. 6), one copy for every service that accepts a tokens.sr.ht token: verify the signature, decide whether the token is ours, check the grant, and — only for a registered token — ask tokens.sr.ht whether it is still live, behind a 60s cache. Every step that can refuse locally runs before the one that cannot, so a short token never touches the network at all.
  • pages — the page-template machinery: discover a service's pages from its embedded tree, refuse at startup a page that defines no content, render into a buffer before touching the response, and carry the shared error page. Both invariants are the point. A content-less page would serve the chrome around a hole with a 200, which is why content is invoked with {{template}} and must never become a {{block}} — a block quietly defines an empty default for every page at once. And a template that fails halfway has already written a partial page unless it rendered into a buffer, at which point the error page cannot be rendered through the call that just failed. Render answers the response itself and returns the error only for the log, because handing it to a service's fail would either overwrite a committed response or recurse through the page that just broke.
  • assets — hashed static assets: find the current main.min.<sha>.css in an fs.FS (an embed.FS or an os.DirFS, since the services use both), and serve the static tree with the cache policy the hash implies — immutable for a content-addressed name, an hour for everything else — without publishing a directory listing. An absent asset resolves to "" rather than an error: a service that will not boot without a build artefact cannot be run from a checkout. The empty href must then be guarded in the template, because <link href=""> re-requests the page it is on.
  • csrf — the same-origin guard. These services have no CSRF token: the session cookie is meta.sr.ht's unified-login cookie, set on the parent domain, and no individual service can set its SameSite. So the defence is an Origin/Referer check against the service's own origin, and it belongs here for the same reason grants does — one rule, five copies, applied inconsistently. A request that carries neither header is refused: one that will not say where it came from cannot be shown to have come from us.
  • middleware — the small HTTP middleware the services were copying between each other verbatim: private, no-store for anything behind the login cookie (no-cache still permits a stored copy, revalidated against the next viewer's cookie), panic recovery through a service-supplied error renderer, and nginx's 499 for a client that hung up — which a service wants distinguished from a real failure, or a disconnected browser inflates the 5xx rate. A panic that arrives after the response has started aborts the connection instead of appending an error page to a truncated one.
  • login — the one decoder of meta.sr.ht's unified-login cookie. Six services read the same fernet-sealed cookie and each made the same five decisions alone: decrypt without an expiry (a service-side TTL logs a viewer out of that one service on a schedule no sibling shares), take the name, strip the ~, treat every failure as anonymity rather than as an error, and validate the name before it reaches a path, a log line or a query. Optional never refuses and Required(deny) does, because folding the gate into the decoder is exactly how the six copies drifted. The validator cannot be switched off: the two services that validate nothing today are the two that would go on passing nothing.
  • internalauth — both ends of the Authorization: Internal handshake between services, which core-go implements unexported. Minting and verifying live in one package on purpose: dolt had the receiver in one repository and the caller in another, so a change to the payload broke provisioning at runtime instead of at compile time.
  • instconf — canonical reading of the shared config's origins. Five services and this library disagreed about what "the origin" is: TrimRight against TrimSuffix, a malformed one becoming "" here and "localhost" there — on a path that includes a DNS-rebinding guard, where an empty host means the guard is off. OriginHost (no port) and OriginAuthority (with it) are two concepts, not a drift.
  • logging — the instance's slog policy without the handler: level from -d/$LOG_LEVEL/config, colour from NO_COLOR and the terminal, and the credential mask set. The mask set is the shared part — what is being redacted is the instance's login cookie and its bearer tokens — while tint-versus-JSON is the service's own call, so the policy also ships as a stdlib ReplaceAttr. middleware.RecoverPanics reports through slog.Default(), which is what gave ecore a stake in every service installing a compatible handler.
  • chimw — the chi-shaped helpers, kept out of middleware so that package can stay dependency-free: GetHead, the 404/405 installer pointing at the service's own error page, and a request logger that emits a slog record instead of chi's unstructured line on stdout.
  • ecoretest — the test bootstrap: a synthetic instance config.ini with the sections the nav rules need (canonical services, hub, the excluded paste/pages, the custom ones, and one section with no origin that must not appear in a switcher) and the fernet/webhook key seeding every service's TestMain was doing by hand. Fixed keys, not generated ones — they secure nothing inside a test process, and constancy is what lets two packages of one service both initialise without the second rotating what the first sealed with.

#Usage: chrome

svc := chrome.NewService(conf, "diff.sr.ht")
svc.StyleHref = cssHref // after discovering the hashed stylesheet
svc.Assets = map[string]string{"bundle.js": bundleHref} // any further artefacts

t := chrome.MustAttach(template.New("layout").Funcs(chrome.Funcs()))
// ... parse the service's own templates into t ...

page := svc.Page(r, "Page title", username) // username "" = anonymous

In the layout:

{{template "srht-env-banner" .}}
<nav class="container navbar navbar-light navbar-expand-sm">
  {{template "srht-nav" .}}
</nav>

The template dot must expose the chrome.Page fields — either a Page itself, or a service view struct that embeds one (promoted fields resolve in templates).

#Usage: bearer

v, err := bearer.New(bearer.Options{
    Origin:   conf.Get("tokens.sr.ht", "origin"), // https://tokens.srht.bigb.es
    ClientID: "bench.sr.ht",                      // the CALLING service
    NodeID:   hostname,
})

tok, err := v.Validate(r.Context(), presented, "bench:upload")
switch {
case err == nil:
    // tok.Username, tok.Grants, tok.TokenID
case errors.Is(err, bearer.ErrNotOurs):
    // service policy: accept as a meta.sr.ht PAT, or refuse
case errors.Is(err, bearer.ErrForbidden):
    http.Error(w, "insufficient grants", http.StatusForbidden)
case errors.Is(err, bearer.ErrUnavailable):
    http.Error(w, "token service unavailable", http.StatusServiceUnavailable)
default: // ErrInvalid, ErrRevoked
    http.Error(w, "invalid token", http.StatusUnauthorized)
}

crypto.InitCrypto must have run first — the signing key and the network key both live in that package's globals.

Two of those arms are the ones to get right. ErrNotOurs is deliberately not decided by the validator: SPEC ch. 6 step 2 leaves it to each service whether a foreign bearer token is a meta PAT to accept (dolt) or something to refuse (bench, cov). And ErrUnavailable is 503, never 401 — reading an unreachable daemon as "revoked" would refuse live tokens across the instance for the length of a tokens.sr.ht restart.

#Usage: the web tier

The five web-side packages are independent — adopt them one at a time — but a service's New ends up reading roughly like this:

//go:embed static templates
var webFS embed.FS

cssHref, err := assets.Resolve(webFS, "static/main.min.*.css", assets.DefaultPrefix)
// err is a malformed glob, not a missing stylesheet: absent resolves to "".

svc := chrome.NewService(conf, "diff.sr.ht")
svc.StyleHref = cssHref

set, err := pages.Load(webFS, pages.Options{Funcs: myHelpers})
// Fatal at startup: a page that defines no "content" is a 200 around a hole.

r.Use(middleware.PrivateCache)
r.Use(middleware.RecoverPanics(func(w http.ResponseWriter, r *http.Request, _ any) {
    renderError(w, r, http.StatusInternalServerError)
}))
r.Use(csrf.Require(svc.SelfOrigin(), nil)) // nil deny = plain 403 with csrf.Message
r.Mount(assets.DefaultPrefix, assets.Handler(webFS, assets.DefaultPrefix, notFound))

and a handler renders through the set, with the view struct embedding the chrome:

vd := viewData{Page: svc.Page(r, "Title", username), Data: payload}
if err := set.Render(w, http.StatusOK, "index", vd); err != nil {
    log.Printf("render index: %v", err) // the response is already answered
}

In tests, ecoretest replaces the hand-built config and the TestMain that seeded crypto:

func TestMain(m *testing.M) { ecoretest.InitCrypto(); os.Exit(m.Run()) }

conf := ecoretest.Config("diff.sr.ht")
noHub := ecoretest.Config("diff.sr.ht", ecoretest.Delete("hub.sr.ht"))

#Policy: chrome

The chrome bakes in the instance-wide decisions instead of parameterizing them: the switcher renders only for authenticated viewers; paste, pages and hub never appear in it; the brand is always circle + site name + red service label, where the name links to hub (or to the service root on an instance without one) and the label links to the service root; the profile link prefers hub's ~username page when hub is configured. Service-specific nav entries go through Service.ExtraNav; per-page width through Page.ContainerClass.

The brand is two links rather than upstream's one because upstream has to choose between them: with hub configured it points the whole brand at hub and drops the service label, without hub it keeps the label and points at the service root. Both halves are load-bearing — hub is excluded from the switcher, so the brand is the only route to it, and chrome that does not name its own service is worse chrome.

Note that embedding names the field Page: a view struct that wants Page for its own payload (a pagination counter, usually) must rename that field. The collision is a compile error, not a silent shadow.