@@ 724,58 724,251 @@ barely matters: Phase 2 serves `/query` at our own origin either way, and `hut`
targets that origin directly. Federation is a config line to set if the gateway
happens to exist.
-## Phases
+## Repo layout
-**Phase 0 — de-risk gate.** Two spikes, both must pass before anything else is
-built:
+Mirrors the siblings: a pure `core/` with no external dependencies, one package
+per subsystem, one `service/` layer that everything user-facing calls.
-1. go-git write path in-process: create branch → commit blob → build merged tree
- → two-parent commit, on a real bare repo, no shell-out.
-2. Prose word-diff rendered against two real revisions of an actual existing spec.
- **If the diff does not read well, the design needs rethinking before any
- further code.** This is the compare.sr.ht/dolt.sr.ht spike discipline applied
- to the riskiest assumption here.
+```
+core/ pure domain. Space/doc/rev/ID validation, .spec.yml policy,
+ proposal state machine, sentinel errors. No external deps.
+gitx/ bare-repo lifecycle, tree walk, blob read, proposal branches,
+ the tree-splice merge, refs-rule enforcement. go-git only.
+db/ Postgres: proposals, ID registry, agent token, index stamps.
+doc/ absorbed warren vault/ + render/: frontmatter, Archive,
+ goldmark rendering, wikilink resolution. Fed by gitx trees.
+search/ absorbed warren index/ + search/: one global bleve index,
+ query-time project filtering.
+prosediff/ word-level prose diff over rendered block structure. Net-new,
+ riskiest, and the Phase 0 gate.
+authn/ unified-login cookie -> identity, agent token validation,
+ provenance trailer construction.
+service/ orchestration: read, propose, merge, reconcile, digest. The
+ single layer REST, MCP and GraphQL all call.
+api/ REST handlers (write plane + JSON reads).
+mcpsrv/ absorbed warren mcpsrv/, wired to service/.
+graph/ gqlgen read schema + resolvers (Phase 2).
+web/ chi router, SourceHut chrome, templates, review UI, static.
+hooks/ update + post-receive shims and the daemon-side RPC endpoints.
+cmd/specsrht/ daemon entry point, startup validation.
+cmd/specsrht-migrate/ brant wrapper, copied from doltsrht-migrate.
+contrib/ nginx block, systemd unit, forced-command wrapper.
+migrations/ brant .sql migrations. scss/ stylesheet entry.
+```
+
+Dependency direction is strictly downward: `core/` depends on nothing,
+`service/` depends on everything below it, and `api`/`mcpsrv`/`graph`/`web`
+depend only on `service/`. Nothing above `service/` may touch `gitx/` or `db/`
+directly — that is what keeps the three agent-facing surfaces behaviourally
+identical.
+
+## Postgres schema
+
+Git holds the documents; Postgres holds only what git cannot answer cheaply.
+Every table here is reconstructable from refs by the reconciler, which is what
+makes the non-transactional writes tolerable.
+
+```sql
+-- Spaces exist as repos; this table is for listing and index bookkeeping.
+CREATE TABLE space (
+ id SERIAL PRIMARY KEY,
+ owner TEXT NOT NULL, -- "bigbes", no ~ prefix
+ name TEXT NOT NULL,
+ created TIMESTAMPTZ NOT NULL DEFAULT now(),
+ UNIQUE (owner, name)
+);
+
+-- Global, not per-project: a later import cannot collide.
+CREATE TABLE document_id (
+ doc_id TEXT PRIMARY KEY, -- "SPEC-0007"
+ space_id INTEGER NOT NULL REFERENCES space(id) ON DELETE CASCADE,
+ path TEXT NOT NULL, -- current path on the approved branch
+ updated_rev TEXT NOT NULL
+);
+
+CREATE TABLE proposal (
+ id SERIAL PRIMARY KEY,
+ space_id INTEGER NOT NULL REFERENCES space(id) ON DELETE CASCADE,
+ title TEXT NOT NULL,
+ rationale TEXT,
+ base_rev TEXT NOT NULL, -- the If-Match value; does not move
+ branch TEXT NOT NULL, -- "proposals/42"
+ state TEXT NOT NULL, -- open | merged | rejected
+ approval TEXT, -- human | policy, set on merge
+ merged_rev TEXT,
+ agent TEXT NOT NULL, -- "claude-code/spec-writer"
+ agent_session TEXT NOT NULL,
+ created TIMESTAMPTZ NOT NULL DEFAULT now(),
+ resolved TIMESTAMPTZ
+);
+CREATE INDEX ON proposal (state, created DESC);
+
+CREATE TABLE agent_token (
+ id SERIAL PRIMARY KEY,
+ name TEXT NOT NULL,
+ token_hash BYTEA NOT NULL UNIQUE,
+ created TIMESTAMPTZ NOT NULL DEFAULT now(),
+ revoked TIMESTAMPTZ
+);
+
+-- Index staleness: compared against the space's approved head.
+CREATE TABLE index_stamp (
+ space_id INTEGER PRIMARY KEY REFERENCES space(id) ON DELETE CASCADE,
+ rev TEXT NOT NULL,
+ indexed_at TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+-- "What landed since you last looked", for the policy-merged digest.
+CREATE TABLE digest_mark (
+ owner TEXT PRIMARY KEY,
+ seen_at TIMESTAMPTZ NOT NULL
+);
+```
+
+Projects are deliberately absent: a project is a saved filter, so it is a name
+plus a space-id list, and adding it before there are several spaces to filter
+would be speculative. It arrives with the meta-project in Phase 2.
+
+`comment` is likewise absent — inline comments are post-v1, and the anchoring
+model (`doc_id`, heading path, block index, block hash) should be settled by
+building the review UI before it is committed to a schema.
+
+## Implementation plan
+
+Milestones are feature-shaped; within each, work is dispatched in **waves**
+following the parallel-implementer convention: the root package is implemented
+and committed first, then siblings that import it and write disjoint directories
+go out in parallel. Every external dependency is added in the foundation commit
+(with a throwaway smoke-import build to populate `go.sum`) so `go.mod` stays out
+of every parallel wave's file set; `go mod tidy` runs once, at the very end. Each
+parallel implementer builds and tests **only its own package** — never `./...`,
+since siblings may not compile yet.
**Ordering constraint.** Because the confirmed scope is a *fresh silo* with no
read-only mounts, the store is **empty until somebody fills it** — a read plane
-shipped first would have nothing to render. The human push path therefore moves
-up into Phase 1: it is how content first exists, and it needs almost no service
-code.
-
-**Phase 1 — core + storage + your push path.** Pure domain (space/doc/rev/ID
-validation, frontmatter parse + schema validation), bare-repo lifecycle,
-the git-object read layer (tree walk → `vault.FromPages`), Postgres schema +
-migrations, the reconciler, and the **SSH push path**: an `update` hook that
-validates and enforces the refs rule, plus `post-receive` that notifies. Both
-hooks RPC into the daemon. No indexing yet — that arrives with the read plane in
-Phase 2. End state: you can `git push` a space, bad pushes are rejected, and the
-service can read what landed.
-
-**Phase 2 — read plane.** warren absorbed, chrome from compare.sr.ht, unified
-login, `?rev=` pinning, one global bleve index with query-time project filtering,
-render cache, search, and **MCP read tools**. Plus the **read-side GraphQL schema on `/query`** and
-`api-origin` in config, federating into `api.sr.ht`. End state: agents can read
-and search everything you have pushed — useful on its own, before any review
-machinery exists.
-
-**Phase 3 — write plane.** Proposals, `If-Match` concurrency, the merge model,
-the agent token + provenance trailers, REST + MCP write tools (every write
-response carrying the proposal URL). MCP tools
-and GraphQL resolvers share one service layer — no parallel implementations.
-
-**Phase 4 — review plane.** Proposal pages at stable URLs (returned by every
-write, so agents can hand you a link), inbox, prose diff, approve / reject, status
-lifecycle, and the **digest of policy-merged content** (the firehose half is
-unreviewed by design, so it must at least be *visible* or it rots silently).
-
-**Phase 5 — later.** Inline comments (anchoring above), webhooks and
-notifications (`core-go/webhooks`, GraphQL-native — the Phase 2 schema is the
-foundation), GraphQL **mutations** once the proposal state machine has stopped
-moving, vector search, per-space token scoping and read-only mounts if the
-boundary ever moves.
-
-**Dropped outright:** the web editor (you edit via clone and push), and with it
-the concurrent-web-edit-vs-push conflict problem it would have created.
+shipped first would have nothing to render. The human push path therefore lands
+in Phase 1: it is how content first exists, and it needs little service code.
+
+### Phase 0 — de-risk gate (throwaway code, no packages)
+
+Two spikes. Both must pass before Phase 1 starts.
+
+1. **go-git write path.** On a real bare repo: create a branch, commit a blob,
+ splice a tree, write a two-parent commit, and read it back — all in-process.
+ Confirms the merge model is expressible with plumbing alone.
+2. **Prose word-diff.** Render a word-level diff between two real revisions of an
+ existing spec of yours. **If it does not read well, stop and rethink** — the
+ browser-review decision and most of Phase 4 rest on it. This is the single
+ riskiest assumption in the document.
+
+Spike 2 is deliberately the one with no fallback plan, so it runs first.
+
+### Phase 1 — foundation, storage, and your push path
+
+**Foundation commit (serial).** `go.mod` with every external dependency
+(go-git, bleve, goldmark, chi, lib/pq, yaml.v3, the MCP SDK, brant, plus the
+`core-go` fork `replace`), then `core/` complete with table-driven tests: name
+and path validation, ID grammar, `.spec.yml` parse and policy evaluation,
+frontmatter schema validation, the proposal state machine.
+
+**Wave A (parallel — `gitx/`, `db/`, `authn/`).** Disjoint directories, all
+importing only the committed `core/`.
+- `gitx/` — bare-repo lifecycle, tree walk, blob read, branch create, tree-splice
+ merge, refs-rule predicate. Tests build fixtures programmatically.
+- `db/` — the schema above plus queries; `schema.sql` and the first brant
+ migration.
+- `authn/` — cookie decrypt to identity, agent-token validation, provenance
+ trailer construction. Tests forge cookies with a synthesized ini (random fernet
+ key + ed25519 seed), as dolt.sr.ht's `authn/` does.
+
+**Wave B (serial).** `service/` read + reconcile paths, `hooks/` (the `update`
+and `post-receive` shims and their daemon RPC endpoints), `cmd/specsrht` skeleton,
+`cmd/specsrht-migrate`.
+
+*End state:* `git push` a space; malformed pushes are rejected with a useful
+message; the daemon can read what landed; the reconciler repairs a killed daemon.
+
+### Phase 2 — read plane
+
+**Wave A (parallel — `doc/`, `search/`).** `doc/` is warren's `vault` + `render`
+with `Scan` replaced by a git-tree walk feeding the existing `FromPages` seam.
+`search/` is warren's `index` + `search` against one global bleve index, with
+query-time project filtering. Both are ports with focused tests, not new design.
+
+**Wave B (parallel — `web/`, `graph/`, `mcpsrv/` read tools).** All depend only
+on `service/`. `web/` brings the chrome, cookie login and SCSS pipeline copied
+from compare.sr.ht.
+
+*End state:* agents read and search everything you have pushed, over MCP, REST
+and `/query`. **Useful on its own, before any review machinery exists** — which
+is the point of this ordering.
+
+### Phase 3 — write plane
+
+`service/` write paths (propose, `If-Match` resolution, merge, auto-merge policy,
+digest bookkeeping), then `api/` and `mcpsrv/` write tools in parallel. Every
+write response carries the proposal URL.
+
+*End state:* an agent proposes and hands you a link; approving from an API call
+merges; a stale base returns 409.
+
+### Phase 4 — review plane
+
+`prosediff/` productionised from the Phase 0 spike, then the review UI: proposal
+pages at stable URLs, approve/reject, the inbox, and the policy-merged digest.
+
+*End state:* the full loop — bot produces, you curate in a browser, bots consume.
+
+### Phase 5 — later
+
+Inline comments (settle anchoring against the built UI first), webhooks and
+notifications, GraphQL mutations once the proposal types stop moving, vector
+search, per-space token scoping, read-only mounts.
+
+**Dropped outright:** the web editor, and with it the concurrent-web-edit-versus-
+push conflict it would have created.
+
+## Verification (post-deploy, end-to-end)
+
+Run against the real instance, in order. Each step fails loudly rather than
+degrading, which is the point of listing them.
+
+1. `GET https://spec.srht.bigb.es/healthz` → `ok`; the service appears in the nav
+ of git.sr.ht and meta.sr.ht (after restarting them).
+2. `git push` a space with a valid document → accepted; it renders in the browser
+ and appears in search within one merge cycle.
+3. `git push` a document with a duplicate `id:` → **rejected** by the `update`
+ hook with a message naming the collision; retry with
+ `--push-option=skip-validation` → accepted.
+4. `git push --force` to the approved branch → rejected.
+5. An agent token proposing to `proposals/*` → accepted; the same token
+ attempting the approved branch → rejected.
+6. Propose via MCP → response carries a URL; opening it shows the prose diff;
+ approve → merges, and the document's approved text changes.
+7. Propose against a stale base → 409 with the current head.
+8. Propose into an `auto_merge` path → lands immediately, and shows in the digest
+ marked `approval: policy`, not `human`.
+9. `?rev=<sha>` of a superseded revision → renders that revision, not the head.
+10. `kill -9` the daemon mid-merge, restart → the reconciler repairs the proposal
+ row and the index stamp; no manual intervention.
+11. Anonymous request → no content leaks; logged-in as bigbes → full access.
+
+## Open risks
+
+- **Prose diff quality (highest).** Everything downstream of the browser-review
+ decision assumes it reads well. No fallback is designed; Phase 0 exists to find
+ out early rather than late.
+- **go-git and native `receive-pack` on the same refs.** Mitigated by a per-space
+ mutex and CAS retry, but the interoperation of the two locking implementations
+ is assumed rather than proven. Worth a deliberate concurrent-push test.
+- **Empty-store bootstrapping.** With no mounts, the service is worthless until
+ content exists. Phase 1's push path is the mitigation; if it slips, the whole
+ thing feels stillborn.
+- **warren absorption drift.** Once `vault`/`render`/`search` are copied in, they
+ fork from upstream warren. Accepted deliberately — but it means bug fixes do
+ not flow back, and that should be a conscious choice each time.
+- **Comment anchoring, deferred not solved.** The tuple is specified; whether it
+ survives real reflowed prose is untested, and Phase 5 will discover it.
## Open items
@@ 828,8 1021,8 @@ into the sections above:
### Still open
-- **Prose diff quality** — not a question to answer in prose but the Phase 0 gate
- to run. Everything else assumes it comes out well.
+- **Prose diff quality** — tracked under "Open risks" above; not a question to
+ answer in prose but the Phase 0 gate to run.
- **Mixed ru/en per-document language routing** (above): still needs a decision
once there is a real corpus to test against.
- **Attachments and binaries** (above): unresolved, and cheap to defer until a