From 72f589404c6017c55decb59aafd8337996d45139 Mon Sep 17 00:00:00 2001 From: bigbes Date: Wed, 22 Jul 2026 04:19:36 +0300 Subject: [PATCH] docs: federate the read side into api.sr.ht from Phase 2 Reverses the earlier no-federation call. The deciding arguments are one endpoint plus one meta PAT for agents already querying git/todo/builds, hut ergonomics, and core-go/webhooks being GraphQL-native so Phase 5 pulls gqlgen in regardless. The dolt precedent does not generalize: its API is a chunk-store protocol, whereas documents and proposals are an ordinary CRUD graph. Mutations stay on REST + MCP. If-Match optimistic concurrency is an HTTP idiom with well-defined 409 semantics, and a federated type is a consumed contract, so the unsettled proposal types stay out of the gateway. Also records that federation is not cross-service search: thistle merges schemas and routes fields, so the meta-project still needs our own index. --- docs/DESIGN.md | 102 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 76 insertions(+), 26 deletions(-) diff --git a/docs/DESIGN.md b/docs/DESIGN.md index ab547839608397d071eeb30311b1095a8b0562dc..372bb1c016e58bccfc316d20f129c0a80695b573 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -42,6 +42,7 @@ modification**. Runs at `https://spec.srht.bigb.es`. | `../warren` | **Absorbed.** Its `vault`/`render`/`index`/`search`/`mcpsrv` packages become the read plane of this module. warren remains a standalone tool for local vaults. | | v1 scope | **Full loop, thin.** Read plane + upload-as-proposal (REST + MCP) + minimal review page (prose diff, approve/reject) + agent tokens. Inline comments and the web editor come later. | | Aggregation | **Projects** (below) — a named set of spaces with one unified index. The "meta-project" is the degenerate case, not special machinery. | +| GraphQL federation | **Read side in Phase 2**, joining `api.sr.ht`'s unified schema; mutations stay on REST + MCP until the review model settles. Reversed an earlier "not in v1" call — see the section below for why. | ## Architecture summary @@ -289,7 +290,7 @@ consume this: `spec_search`, `spec_read`, `spec_propose`, `spec_comment`, | `../warren` | `vault/`, `render/`, `index/`, `search/`, `mcpsrv/`, `linkcheck/` | The entire read plane and bot retrieval surface, already written. Absorbed into this module. | | `../sourcehut-compare` | chrome/templates, cookie→identity, GraphQL authorizer + TTL cache, SCSS pipeline, `contrib/` nginx+systemd shape | Closest sibling; copy the integration scaffolding wholesale. | | `../sourcehut-dolt` | bare-store lifecycle under `/~user/`, brant migration wrapper, config validation | Same storage-root and migration patterns. | -| `sr-ht-core` (fork) | config, crypto, auth, database, server | Pinned to `git.srht.bigb.es/~bigbes/core-go` via `replace`, as in both siblings. Never `go get -u`. | +| `sr-ht-core` (fork) | config, crypto, auth, database, server, **gqlgen scaffolding + `webhooks`** | Pinned to `git.srht.bigb.es/~bigbes/core-go` via `replace`, as in both siblings. Never `go get -u`. The gqlgen path is the blessed one and is what Phase 2's `/query` is built on. | Genuinely net-new: proposals, prose diff, review UI, agent tokens, frontmatter lifecycle, projects/aggregation. @@ -313,6 +314,9 @@ and what other services look us up by. ```ini [spec.sr.ht] origin=https://spec.srht.bigb.es +; Federated into api.sr.ht from Phase 2 (read side). api.sr.ht fetches +; /query; omit this key and it falls back to /query. +api-origin=https://spec.srht.bigb.es connection-string=postgresql://specsrht@localhost/spec.sr.ht?sslmode=disable ; Bare repos, service-owned: /~/ repos=/var/lib/spec @@ -323,10 +327,11 @@ static-dir=/usr/share/sourcehut/spec.sr.ht/static migrate-on-upgrade=yes ``` -Canonical key names only — `origin`, `connection-string`, `migrate-on-upgrade` -are read by the shared accessors (`config.GetOrigin`, `server.WithDefaultMiddleware`). -`repos` mirrors dolt.sr.ht/git.sr.ht; `static-dir` and `bind-address` are the -local house convention already used by both siblings. `cache` is ours. +Canonical key names only — `origin`, `api-origin`, `connection-string`, +`migrate-on-upgrade` are read by the shared accessors (`config.GetOrigin`, +`config.GetAPI`, `server.WithDefaultMiddleware`). `repos` mirrors +dolt.sr.ht/git.sr.ht; `static-dir` and `bind-address` are the local house +convention already used by both siblings. `cache` is ours. ### Shared keys we read in place (never duplicate) @@ -356,24 +361,64 @@ local house convention already used by both siblings. `cache` is ours. 5. **Migrations** — `specsrht-migrate`, a brant wrapper, copied from `doltsrht-migrate`. -### GraphQL federation: deliberately not in v1 - -Bots use REST + MCP; a gqlgen schema buys nothing for the loop this service -exists to serve. dolt.sr.ht made the same call. - -**Verified that opting out is safe.** `api.sr.ht` federates *every* config -section ending in `.sr.ht` — it does not consult an allow-list, so our section -does get added as a federated service pointed at `origin + "/query"`, which we -will not serve. `updateSchema` (`api.sr.ht/main.go`) handles this explicitly: -services that are offline or have an invalid schema are logged and **skipped**, -and `BuildSchema` runs over only the healthy ones. So the effect is one -`Unable to update service "spec.sr.ht"` log line, not a broken gateway. - -It is also not a hot loop: schema refresh is **SIGHUP-driven, not on a ticker** -(the goroutine selects on `signalChan`), so the failing fetch happens at startup -and on explicit reload only. dolt.sr.ht and compare.sr.ht already run with this -exact property today. Should we ever want federation, adding `api-origin=` and -serving `/query` is the whole change. +### GraphQL federation: read side in Phase 2, writes deferred + +**Federate the read side, and do it early** — `space`, `document`, `project`, +`search`, and read-only proposal listing join `api.sr.ht`'s unified schema in +Phase 2. **Mutations stay on REST + MCP** until the review model has settled. + +First, a precision that kills the most tempting argument *for* federating: +**federation is not cross-service search.** thistle merges schemas and routes +each field to its owning service. There is no join engine and no unified index. +Federation does **not** deliver the meta-project — that remains spec.sr.ht's own +bleve index over member spaces plus read-only mounts, exactly as specified above. +Do not let the gateway create the illusion that aggregation comes for free. + +The reasons that actually justify it: + +- **One endpoint, one token.** An agent working this instance already speaks + GraphQL to git/todo/builds with a meta PAT. Making specs the one service + needing a separate base URL and a separate auth path is friction paid by every + agent ever written against it. +- **`hut`.** The house CLI is generated against SourceHut GraphQL APIs. + Federating is the difference between a `hut`-shaped workflow being plausible + and being impossible. +- **Phase 5 pulls gqlgen in regardless.** `core-go/webhooks` is GraphQL-native. + Skipping GraphQL is a deferral, not a saving — and a costlier one once a schema + has to be retrofitted around an established REST surface. +- **The dolt precedent does not generalize.** dolt.sr.ht skipped federation + because its API *is* a chunk-store wire protocol with no sane graph to expose. + Documents, spaces, proposals and comments are an ordinary CRUD graph — exactly + what gqlgen and thistle are good at. + +**Why writes stay on REST for now**, and this is a technical reason rather than +scope discipline: the write plane's concurrency story is `If-Match: `, +an HTTP idiom with well-defined 409 semantics that agents get right by default. +Modeling base-rev as a mutation argument is perfectly doable, but it is a +contract worth designing once, after the proposal state machine has stopped +moving. Federation compounds this — **once a type is in the gateway it is a +consumed contract**, so churning the proposal/review types there is expensive. +Read types (space, document, project, search) are stable from the start; the +review types are not, which is precisely the line drawn above. + +MCP and GraphQL are not competitors here. The MCP tools call the **same resolver +layer**, not a parallel implementation. + +**Opting out, if we ever do, is verified safe.** `api.sr.ht` federates *every* +config section ending in `.sr.ht` with no allow-list, pointing at +`api-origin` (else `origin`) `+ "/query"`. `updateSchema` (`api.sr.ht/main.go`) +logs and **skips** services that are offline or serve an invalid schema, and +`BuildSchema` runs over the healthy ones only — so a non-GraphQL service costs +one `Unable to update service` log line, not a broken gateway. It is also not a +hot loop: schema refresh is **SIGHUP-driven, not on a ticker** (the goroutine +selects on `signalChan`), so the fetch happens at startup and explicit reload +only. dolt.sr.ht and compare.sr.ht run with exactly this property today. + +**Open:** whether `api.sr.ht` is deployed on this instance at all is unconfirmed — +`sourcehut/sr.ht-nginx/` is the upstream mirror, not our instance config, and no +`api.srht.bigb.es` reference exists in the tree. If the gateway is not running, +Phase 2 still builds and serves `/query` (it is useful directly, and `hut` can +target a service origin), but the federation payoff waits on standing it up. ## Phases @@ -393,16 +438,21 @@ checkouts, Postgres schema + migrations. **Phase 2 — read plane.** warren absorbed, chrome from compare.sr.ht, unified login, ACLs, `?rev=` pinning, read-only mounts, project index with query-time ACL -filtering. +filtering. Plus the **read-side GraphQL schema on `/query`** (space, document, +project, search, proposal listing) and `api-origin` in config, federating into +`api.sr.ht`. **Phase 3 — write plane.** Proposals, `If-Match` concurrency, the merge model, -agent tokens + provenance trailers, REST + MCP. +agent tokens + provenance trailers, REST + MCP. MCP tools and GraphQL resolvers +share one service layer — no parallel implementations. **Phase 4 — review UI.** Inbox, prose diff, approve / request-changes, status lifecycle transitions. **Phase 5 — later.** Inline comments (anchoring above), web editor, webhooks and -notifications, vector search over the project index. +notifications (`core-go/webhooks`, GraphQL-native — the schema from Phase 2 is +the foundation), GraphQL **mutations** once the proposal state machine has +stopped moving, vector search over the project index. ## Open items