~bigbes/sr-ht-spec

72f589404c6017c55decb59aafd8337996d45139 — bigbes 27 days ago c945f38
docs: federate the read side into api.sr.ht from Phase 2

Reverses the earlier no-federation call. The deciding arguments are one
endpoint plus one meta PAT for agents already querying git/todo/builds,
hut ergonomics, and core-go/webhooks being GraphQL-native so Phase 5
pulls gqlgen in regardless. The dolt precedent does not generalize: its
API is a chunk-store protocol, whereas documents and proposals are an
ordinary CRUD graph.

Mutations stay on REST + MCP. If-Match optimistic concurrency is an HTTP
idiom with well-defined 409 semantics, and a federated type is a consumed
contract, so the unsettled proposal types stay out of the gateway.

Also records that federation is not cross-service search: thistle merges
schemas and routes fields, so the meta-project still needs our own index.
1 files changed, 76 insertions(+), 26 deletions(-)

M docs/DESIGN.md
M docs/DESIGN.md => docs/DESIGN.md +76 -26
@@ 42,6 42,7 @@ modification**. Runs at `https://spec.srht.bigb.es`.
| `../warren` | **Absorbed.** Its `vault`/`render`/`index`/`search`/`mcpsrv` packages become the read plane of this module. warren remains a standalone tool for local vaults. |
| v1 scope | **Full loop, thin.** Read plane + upload-as-proposal (REST + MCP) + minimal review page (prose diff, approve/reject) + agent tokens. Inline comments and the web editor come later. |
| Aggregation | **Projects** (below) — a named set of spaces with one unified index. The "meta-project" is the degenerate case, not special machinery. |
| GraphQL federation | **Read side in Phase 2**, joining `api.sr.ht`'s unified schema; mutations stay on REST + MCP until the review model settles. Reversed an earlier "not in v1" call — see the section below for why. |

## Architecture summary



@@ 289,7 290,7 @@ consume this: `spec_search`, `spec_read`, `spec_propose`, `spec_comment`,
| `../warren` | `vault/`, `render/`, `index/`, `search/`, `mcpsrv/`, `linkcheck/` | The entire read plane and bot retrieval surface, already written. Absorbed into this module. |
| `../sourcehut-compare` | chrome/templates, cookie→identity, GraphQL authorizer + TTL cache, SCSS pipeline, `contrib/` nginx+systemd shape | Closest sibling; copy the integration scaffolding wholesale. |
| `../sourcehut-dolt` | bare-store lifecycle under `<repos>/~user/<name>`, brant migration wrapper, config validation | Same storage-root and migration patterns. |
| `sr-ht-core` (fork) | config, crypto, auth, database, server | Pinned to `git.srht.bigb.es/~bigbes/core-go` via `replace`, as in both siblings. Never `go get -u`. |
| `sr-ht-core` (fork) | config, crypto, auth, database, server, **gqlgen scaffolding + `webhooks`** | Pinned to `git.srht.bigb.es/~bigbes/core-go` via `replace`, as in both siblings. Never `go get -u`. The gqlgen path is the blessed one and is what Phase 2's `/query` is built on. |

Genuinely net-new: proposals, prose diff, review UI, agent tokens, frontmatter
lifecycle, projects/aggregation.


@@ 313,6 314,9 @@ and what other services look us up by.
```ini
[spec.sr.ht]
origin=https://spec.srht.bigb.es
; Federated into api.sr.ht from Phase 2 (read side). api.sr.ht fetches
; <api-origin>/query; omit this key and it falls back to <origin>/query.
api-origin=https://spec.srht.bigb.es
connection-string=postgresql://specsrht@localhost/spec.sr.ht?sslmode=disable
; Bare repos, service-owned: <repos>/~<user>/<space>
repos=/var/lib/spec


@@ 323,10 327,11 @@ static-dir=/usr/share/sourcehut/spec.sr.ht/static
migrate-on-upgrade=yes
```

Canonical key names only — `origin`, `connection-string`, `migrate-on-upgrade`
are read by the shared accessors (`config.GetOrigin`, `server.WithDefaultMiddleware`).
`repos` mirrors dolt.sr.ht/git.sr.ht; `static-dir` and `bind-address` are the
local house convention already used by both siblings. `cache` is ours.
Canonical key names only — `origin`, `api-origin`, `connection-string`,
`migrate-on-upgrade` are read by the shared accessors (`config.GetOrigin`,
`config.GetAPI`, `server.WithDefaultMiddleware`). `repos` mirrors
dolt.sr.ht/git.sr.ht; `static-dir` and `bind-address` are the local house
convention already used by both siblings. `cache` is ours.

### Shared keys we read in place (never duplicate)



@@ 356,24 361,64 @@ local house convention already used by both siblings. `cache` is ours.
5. **Migrations** — `specsrht-migrate`, a brant wrapper, copied from
   `doltsrht-migrate`.

### GraphQL federation: deliberately not in v1

Bots use REST + MCP; a gqlgen schema buys nothing for the loop this service
exists to serve. dolt.sr.ht made the same call.

**Verified that opting out is safe.** `api.sr.ht` federates *every* config
section ending in `.sr.ht` — it does not consult an allow-list, so our section
does get added as a federated service pointed at `origin + "/query"`, which we
will not serve. `updateSchema` (`api.sr.ht/main.go`) handles this explicitly:
services that are offline or have an invalid schema are logged and **skipped**,
and `BuildSchema` runs over only the healthy ones. So the effect is one
`Unable to update service "spec.sr.ht"` log line, not a broken gateway.

It is also not a hot loop: schema refresh is **SIGHUP-driven, not on a ticker**
(the goroutine selects on `signalChan`), so the failing fetch happens at startup
and on explicit reload only. dolt.sr.ht and compare.sr.ht already run with this
exact property today. Should we ever want federation, adding `api-origin=` and
serving `/query` is the whole change.
### GraphQL federation: read side in Phase 2, writes deferred

**Federate the read side, and do it early** — `space`, `document`, `project`,
`search`, and read-only proposal listing join `api.sr.ht`'s unified schema in
Phase 2. **Mutations stay on REST + MCP** until the review model has settled.

First, a precision that kills the most tempting argument *for* federating:
**federation is not cross-service search.** thistle merges schemas and routes
each field to its owning service. There is no join engine and no unified index.
Federation does **not** deliver the meta-project — that remains spec.sr.ht's own
bleve index over member spaces plus read-only mounts, exactly as specified above.
Do not let the gateway create the illusion that aggregation comes for free.

The reasons that actually justify it:

- **One endpoint, one token.** An agent working this instance already speaks
  GraphQL to git/todo/builds with a meta PAT. Making specs the one service
  needing a separate base URL and a separate auth path is friction paid by every
  agent ever written against it.
- **`hut`.** The house CLI is generated against SourceHut GraphQL APIs.
  Federating is the difference between a `hut`-shaped workflow being plausible
  and being impossible.
- **Phase 5 pulls gqlgen in regardless.** `core-go/webhooks` is GraphQL-native.
  Skipping GraphQL is a deferral, not a saving — and a costlier one once a schema
  has to be retrofitted around an established REST surface.
- **The dolt precedent does not generalize.** dolt.sr.ht skipped federation
  because its API *is* a chunk-store wire protocol with no sane graph to expose.
  Documents, spaces, proposals and comments are an ordinary CRUD graph — exactly
  what gqlgen and thistle are good at.

**Why writes stay on REST for now**, and this is a technical reason rather than
scope discipline: the write plane's concurrency story is `If-Match: <base-rev>`,
an HTTP idiom with well-defined 409 semantics that agents get right by default.
Modeling base-rev as a mutation argument is perfectly doable, but it is a
contract worth designing once, after the proposal state machine has stopped
moving. Federation compounds this — **once a type is in the gateway it is a
consumed contract**, so churning the proposal/review types there is expensive.
Read types (space, document, project, search) are stable from the start; the
review types are not, which is precisely the line drawn above.

MCP and GraphQL are not competitors here. The MCP tools call the **same resolver
layer**, not a parallel implementation.

**Opting out, if we ever do, is verified safe.** `api.sr.ht` federates *every*
config section ending in `.sr.ht` with no allow-list, pointing at
`api-origin` (else `origin`) `+ "/query"`. `updateSchema` (`api.sr.ht/main.go`)
logs and **skips** services that are offline or serve an invalid schema, and
`BuildSchema` runs over the healthy ones only — so a non-GraphQL service costs
one `Unable to update service` log line, not a broken gateway. It is also not a
hot loop: schema refresh is **SIGHUP-driven, not on a ticker** (the goroutine
selects on `signalChan`), so the fetch happens at startup and explicit reload
only. dolt.sr.ht and compare.sr.ht run with exactly this property today.

**Open:** whether `api.sr.ht` is deployed on this instance at all is unconfirmed —
`sourcehut/sr.ht-nginx/` is the upstream mirror, not our instance config, and no
`api.srht.bigb.es` reference exists in the tree. If the gateway is not running,
Phase 2 still builds and serves `/query` (it is useful directly, and `hut` can
target a service origin), but the federation payoff waits on standing it up.

## Phases



@@ 393,16 438,21 @@ checkouts, Postgres schema + migrations.

**Phase 2 — read plane.** warren absorbed, chrome from compare.sr.ht, unified
login, ACLs, `?rev=` pinning, read-only mounts, project index with query-time ACL
filtering.
filtering. Plus the **read-side GraphQL schema on `/query`** (space, document,
project, search, proposal listing) and `api-origin` in config, federating into
`api.sr.ht`.

**Phase 3 — write plane.** Proposals, `If-Match` concurrency, the merge model,
agent tokens + provenance trailers, REST + MCP.
agent tokens + provenance trailers, REST + MCP. MCP tools and GraphQL resolvers
share one service layer — no parallel implementations.

**Phase 4 — review UI.** Inbox, prose diff, approve / request-changes, status
lifecycle transitions.

**Phase 5 — later.** Inline comments (anchoring above), web editor, webhooks and
notifications, vector search over the project index.
notifications (`core-go/webhooks`, GraphQL-native — the schema from Phase 2 is
the foundation), GraphQL **mutations** once the proposal state machine has
stopped moving, vector search over the project index.

## Open items