~bigbes/sr-ht-compare

ref: 0b9a8233d3735b5c609c92af5f0ac16d95c5e261 sr-ht-compare/README.md -rw-r--r-- 11.9 KiB
0b9a8233 — bigbes deps: sr-ht-ecore whose middleware reports panics through slog 9 days ago

#compare.sr.ht

A git compare/diff viewer for a self-hosted SourceHut instance. It compares two refs (branches, tags, or commits) of any git.sr.ht repository — base...head or base..head — and renders the diff with @pierre/diffs (Shiki-based syntax-highlighted diffs) and a @pierre/trees file tree, plus a single-commit view. Runs at https://compare.srht.bigb.es.

#Architecture

compare.sr.ht is a stateless Go daemon: it owns no database and no object storage, so it needs neither Postgres nor Redis. It integrates into SourceHut purely through configuration — no upstream sources are modified (the sourcehut-custom-service integration model). Repository data is read directly from the bare repos on disk ({[git.sr.ht] repos}/~{owner}/{name}) using go-git in-process; no git binary is executed at runtime. Every request is authorized against the git.sr.ht internal GraphQL API using core-go's internal-auth client, acting as the cookie's logged-in user — or anonymously, which git.sr.ht's own loader scopes to public/unlisted repositories, so private repos are never leaked (an unauthorized or missing repo is always a 404, never a 403). Identity comes from decrypting the shared sr.ht.unified-login.v1 cookie with the instance network key; there is no login flow of our own. Pages are server-rendered Go templates wrapped in the SourceHut chrome (nav, service-switcher, login block, environment banner), which comes from the shared sr-ht-ecore chrome package rather than from a copy of its own. The diff/tree UI is one vendored pierre-libs esbuild bundle driven by a JSON blob embedded in the page, so the browser does all diff rendering from a single request with no second authorization round-trip. A short-TTL in-memory cache in front of the authorizer spares git.sr.ht a GraphQL call on every page load.

#Layout

  • core/ — pure domain: owner/repo/ref validation, the compare-spec grammar, sentinel errors. No external dependencies.
  • gitx/ — bare-repo access over go-git: refs, ref-to-ref diffs, single-commit diffs, and commit logs, all bounded by context timeouts and output-size caps.
  • authz/ — cookie→identity and the git.sr.ht GraphQL authorizer with a short TTL cache.
  • web/ — chi router, handlers, Go templates, embedded static assets. Most of the machinery around them is not here but in sr-ht-ecore, wired up in web/server.go: chrome for the page frame, pages for template discovery and the shared error page, assets for the hashed artefacts and their cache policy, csrf and middleware for the router's guards. What is left is this service's own: the routes, the handlers, and the templates they render.
  • frontend/ + scss/ — build-time TypeScript diff bundle and the SCSS entry.
  • cmd/comparesrht/ — the daemon entry point and startup validation.
  • contrib/ — nginx server block, systemd unit, and a dev GraphQL stub.

The core-go dependency is pinned to a private fork (git.srht.bigb.es/~bigbes/core-go) via a replace directive in go.mod; the fork carries one production S3 patch on top of upstream.

#Build requirements

  • Go 1.26+ (module sourcecraft.dev/bigbes/sr-ht-compare).
  • sassc and minify — CSS pipeline (make css), building against the shared sourcehut scss partials installed at /usr/share/sourcehut/scss (from core.sr.ht's make install). dart-sass also works for local builds; the production css target uses sassc -I /usr/share/sourcehut/scss.
  • Node.js (build time only) — the frontend diff bundle (make bundle, esbuild). The vendored, content-hashed web/static/bundle.<hash>.js is committed (the hash busts the browser cache on deploy, like the stylesheet), so Node is not needed at runtime.
make build   # compile ./comparesrht
make test    # go test ./...
make css     # build the hashed stylesheet (sassc + minify)
make bundle  # rebuild the vendored frontend bundle (esbuild)

Static assets are //go:embed-ed into the binary (web/templates.go embeds static). make css / make bundle only update the files under web/static/ — the running daemon serves the copy baked into ./comparesrht, so re-run make build after either or the change won't appear. Rebuilding the CSS or bundle without a following go build is a common way to chase a stale asset.

#Configuration

See config.example.ini. On a real instance, append the [compare.sr.ht] section to the shared /etc/sr.ht/config.ini; the service reads the instance's existing shared keys ([sr.ht], [webhooks], [meta.sr.ht], [git.sr.ht]) in place. core-go's LoadConfig() searches, in order: ./config.ini, ../config.ini, /etc/sr.ht/config.ini, /etc/sr.ht/*.ini.

At startup the daemon validates that all required keys are present — the [sr.ht] network-key, [webhooks] private-key, [git.sr.ht] repos, a git.sr.ht API origin candidate (api-internal-origin / internal-origin / api-origin / origin), [meta.sr.ht] origin, and [compare.sr.ht] origin — and exits with a single clear message listing anything missing.

#Development

Run the service locally against a directory of bare repositories and the dev-stub GraphQL API (no real instance required).

  1. Write a local config.ini in the repo root (git-ignored). Minimal template — the two crypto keys can be any dev values (see below):

    [sr.ht]
    network-key=<a fresh Fernet key>
    site-name=sourcehut
    environment=development
    
    [webhooks]
    private-key=<base64 of 32 random bytes>
    
    [compare.sr.ht]
    origin=http://localhost:5090
    
    [meta.sr.ht]
    origin=http://localhost:5100
    
    [git.sr.ht]
    origin=http://localhost:5101
    api-origin=http://localhost:5101   # dev-stub serves POST /query here
    repos=/path/to/local/bare/repos    # holds ~owner/name bare repos
    

    Generate the keys with a throwaway program (both use core-go's fernet dependency):

    // go run ./gen-keys.go
    package main
    
    import (
        "crypto/rand"
        "encoding/base64"
        "fmt"
    
        "github.com/fernet/fernet-go"
    )
    
    func main() {
        var k fernet.Key
        _ = k.Generate()
        seed := make([]byte, 32)
        _, _ = rand.Read(seed)
        fmt.Println("network-key =", k.Encode())
        fmt.Println("private-key =", base64.StdEncoding.EncodeToString(seed))
    }
    
  2. Prepare a bare repo for the stub to resolve. The dev-stub reports every repository as PUBLIC, so gitx just needs the on-disk repo to exist:

    mkdir -p /path/to/local/bare/repos/~alice
    git clone --bare /some/existing/repo /path/to/local/bare/repos/~alice/demo
    
  3. Start the dev-stub (fake git.sr.ht GraphQL API):

    go run ./contrib/dev-stub -addr 127.0.0.1:5101
    
  4. Run the daemon:

    make run-dev            # builds, then ./comparesrht -b localhost:5090
    

    Visit http://localhost:5090/~alice/demo, then a compare such as http://localhost:5090/~alice/demo/compare/main...some-branch, or a commit page at /~alice/demo/commit/<sha>. Append .patch to any compare/commit URL for the raw unified diff.

There is no dedicated helper. Identity is just the sr.ht.unified-login.v1 cookie: a Fernet token (sealed with [sr.ht] network-key) whose JSON payload's only meaningful field is name. Mint one with a five-line program that reuses the same config and crypto core-go uses at runtime:

// go run ./mint-cookie.go   (run from the dir holding config.ini)
package main

import (
    "encoding/json"
    "fmt"

    "sourcecraft.dev/bigbes/sr-ht-core/config"
    "sourcecraft.dev/bigbes/sr-ht-core/crypto"
)

func main() {
    conf := config.LoadConfig()
    crypto.InitCrypto(conf)
    payload, _ := json.Marshal(map[string]string{"name": "bigbes"})
    fmt.Println(string(crypto.Encrypt(payload)))
}

Set the printed value as the sr.ht.unified-login.v1 cookie for localhost:5090 (browser devtools → Application → Cookies), reload, and the nav shows "Logged in as bigbes" with your repository list (from the stub's me.repositories).

#Deployment

compare.sr.ht deploys like any other SourceHut web service. On the instance:

  1. Install the binary and static assets:

    make install PREFIX=/usr/local
    # → /usr/local/bin/comparesrht
    # → /usr/share/sourcehut/compare.sr.ht/static/{bundle.<hash>.js,main.min.<hash>.css,logo.svg}
    
  2. Config. Append the [compare.sr.ht] section (origin= and static-dir=) to the shared /etc/sr.ht/config.ini. To make compare.sr.ht appear in the shared nav/service-switcher of the other services, that same [compare.sr.ht] origin= line must be visible to their configs too (on a single shared config.ini this is automatic) — then restart those services (git.sr.ht, meta.sr.ht, …) so they pick up the new switcher entry.

  3. DNS. Point compare.srht.bigb.es at the instance. It must be a subdomain of the shared cookie domain (*.srht.bigb.es) so the unified-login cookie is sent to us.

  4. internal-ipnet. This host must fall inside the git.sr.ht API's [sr.ht] internal-ipnet CIDR, or the internal-auth GraphQL calls (including anonymous AUTH_ANON_INTERNAL) are rejected. If compare.sr.ht runs on the same box as git.sr.ht, the default loopback/private ranges already cover it.

  5. nginx. Install contrib/compare.sr.ht.conf alongside the other *.sr.ht.conf files (adjust server_name), provide the TLS cert include it references (compare-ssl.conf), and reload nginx. It is a plain proxy_pass http://127.0.0.1:5090; static assets are served by the app from its embedded FS with immutable cache headers.

  6. systemd. Install contrib/compare-srht.service, then systemctl enable --now compare-srht. The unit runs as User=git so it has read access to the bare repositories under /var/lib/git (ReadOnlyPaths=/var/lib/git, ProtectSystem=strict — the service only ever reads). It stops with KillSignal=SIGINT because core-go's server.Run performs its warm shutdown on SIGINT, not the systemd default SIGTERM.

  7. Verify. GET https://compare.srht.bigb.es/healthzok. Logged in, the landing page lists your repos and a PUBLIC repo's main...branch compare renders with syntax highlighting, the tree sidebar, and the split/unified toggle; a PRIVATE repo is a 404 to anonymous viewers and 200 to its owner.

#Design notes

  • go-git, not git shell-out. git.sr.ht itself shells out to the git binary, and the original plan did too. On the user's decision we read repositories in-process with go-git instead: no subprocess management, no PATH/argument-injection surface, and cleaner output-size bounding. The diff output was fidelity-verified against real git (see gitx/fidelity_test.go) so the patches @pierre/diffs parses carry standard diff --git headers and match git's rename/binary handling.
  • 10 MB vendored bundle. web/static/bundle.<hash>.js is a ~10 MB committed esbuild output. It is large because @pierre/diffs bundles Shiki with its grammars/themes for client-side syntax highlighting. We accept the size: it is built once, served with a long immutable cache lifetime, gzips/brotlis down substantially on the wire, and keeps the runtime a single static Go binary with zero Node dependency. Trimming Shiki's language set is the obvious future lever if the transfer size becomes a problem.
  • LICENSE pending. No LICENSE file has been chosen yet. SourceHut's own services are typically AGPL/GPL; pick and add one before any public distribution.