@@ 1,336 @@
+# spec.sr.ht — reviewable document storage for humans and agents
+
+Status: **design / proposal** — nothing implemented yet.
+Author: bigbes (with Claude). Last updated: 2026-07-22.
+
+## Context
+
+bigbes wants a third custom service on the self-hosted SourceHut instance
+(`*.srht.bigb.es`), after `dolt.sr.ht` (`../sourcehut-dolt`) and `compare.sr.ht`
+(`../sourcehut-compare`): **separate storage for specs and documents that is
+reviewable by a human, modifiable by a human, and uploaded to and read by bots.**
+
+Those three verbs are not three features. They are one loop:
+
+> **bot produces → human curates → bots consume.**
+
+Everything below follows from taking that loop seriously. Three properties of the
+loop are what a plain git repo, a wiki, or upstream `man.sr.ht` each fail to
+provide:
+
+1. **The unit of work is a proposal, not a commit.** Agents produce more text
+ than a human can read. A store where agents write freely and nobody reviews is
+ a wiki that rots within a week. The thing you interact with daily is a *review
+ queue*, not a file tree.
+2. **Bots need a stable read contract.** "Give me the approved text of SPEC-0007",
+ not "HEAD of main, which another agent is halfway through rewriting". Draft and
+ canonical must be different addresses.
+3. **Provenance is review context.** Which agent, which session, on whose behalf,
+ against which base revision. Reviewing agent output without that is reviewing
+ anonymous text.
+
+Same integration model as the two siblings: one Go module, config-driven
+`[spec.sr.ht]` section, unified-login cookie, **no upstream SourceHut
+modification**. Runs at `https://spec.srht.bigb.es`.
+
+### Decisions taken (user-confirmed, 2026-07-22)
+
+| Decision | Choice |
+|---|---|
+| Review gate | **Proposal-first.** Bots always write to a proposal branch; a human approves before anything reaches the approved revision. Per-space policy may whitelist auto-merge namespaces. |
+| Storage substrate | **Own bare git repos, service-owned** (`<repos>/~user/<space>`), like dolt.sr.ht owns its NBS stores. |
+| `../warren` | **Absorbed.** Its `vault`/`render`/`index`/`search`/`mcpsrv` packages become the read plane of this module. warren remains a standalone tool for local vaults. |
+| v1 scope | **Full loop, thin.** Read plane + upload-as-proposal (REST + MCP) + minimal review page (prose diff, approve/reject) + agent tokens. Inline comments and the web editor come later. |
+| Aggregation | **Projects** (below) — a named set of spaces with one unified index. The "meta-project" is the degenerate case, not special machinery. |
+
+## Architecture summary
+
+One Go module **`sourcecraft.dev/bigbes/sr-ht-spec`**, one binary `specsrht`
+(plus `specsrht-migrate`, a brant wrapper, exactly as dolt.sr.ht does it).
+
+```
+ agents ──MCP / REST──┐
+ ├──► specsrht ──► bare git repos (authoritative)
+ humans ──web UI──────┘ │ <repos>/~user/<space>
+ │
+ ├──► materialized checkouts (read/index cache)
+ │ <cache>/~user/<space>/
+ ├──► bleve indexes, one per project
+ └──► Postgres (proposals, comments, ACLs,
+ agent tokens, ID registry)
+```
+
+**Git is authoritative for document bodies. Postgres never stores a body.**
+That keeps `git clone` a complete export, keeps `git log` a complete provenance
+record offline, and keeps the blast radius of a Postgres restore small.
+
+### Two-tier storage, and why
+
+- **Bare repo** — the push target and the source of truth. One repo per *space*.
+- **Materialized checkout** of the approved branch — a plain directory of
+ markdown, rebuilt on every merge. This exists so warren's indexer and renderer
+ can work on files (which is what they already do), and so reads never pay
+ object-database costs. It is a **cache**: deletable, rebuildable from the bare
+ repo at any time.
+
+## Domain model
+
+### Space
+
+One bare git repo = one space, addressed `~<user>/<space>`. The unit of
+ownership, ACL, and review policy. Contains markdown documents with YAML
+frontmatter, plus attachments.
+
+A versioned `.spec.yml` at the repo root carries the space's own policy — which
+means policy changes are themselves reviewable:
+
+```yaml
+review:
+ required: 1 # approvals needed to merge
+ approvers: [~bigbes]
+ auto_merge: [notes/**] # paths that skip the gate
+schema: # frontmatter contract, enforced at propose time
+ required: [id, title, status, owners]
+ status: [draft, review, approved, superseded]
+```
+
+### Document
+
+Markdown + YAML frontmatter, matching the `second-brain` / warren conventions
+already in use (wikilinks, `type`/`summary`/`tags`).
+
+```yaml
+id: SPEC-0007 # stable; NEVER changes, including on rename
+title: Proposal storage model
+status: draft | review | approved | superseded
+supersedes: SPEC-0003
+owners: [~bigbes]
+tags: [storage, review]
+```
+
+`id` is the load-bearing field. Paths move; IDs do not. Cross-space links
+resolve by ID, comment anchors reference IDs, and bots pin to IDs. The service
+enforces ID uniqueness per project via a registry table.
+
+**Frontmatter is validated at propose time.** A bot that omits `status:` gets a
+422, not a silent merge. Schema validation at the door is the cheapest available
+defense against agent slop, and it costs almost nothing to implement.
+
+### Proposal
+
+A branch `proposals/<id>` plus a Postgres row. Bundles N document edits with a
+title and rationale. State machine:
+
+```
+draft ──► open ──► approved ──► merged
+ │└──► changes-requested ──┘
+ └──► rejected
+```
+
+### Project
+
+A named set of spaces plus read-only external mounts, with **one unified index,
+one search endpoint, one MCP view, and one wikilink namespace**. Modeled on
+hub.sr.ht's project-groups-repos idea.
+
+```yaml
+project: ~bigbes/+everything
+spaces:
+ - ~bigbes/tarantool-rfcs
+ - ~bigbes/home-ops
+mounts: # read-only, indexed, searchable, not proposable
+ - {type: dir, path: /Users/blikh/data/home/second-brain}
+ - {type: gitsrht, repo: ~bigbes/warren, subdir: docs/}
+```
+
+The requested **meta-project — "merge all my doc work into one searchable
+thing" — is just a project whose membership is everything.** There is no
+separate aggregate entity, no copying, and no sync job. Three consequences that
+must be designed in from the start rather than retrofitted:
+
+1. **ACL filtering happens at query time**, against a single shared index. Never
+ build per-user indexes — that path ends in N indexes and a stale-permission
+ bug. Each indexed document carries its space ID; the query layer intersects
+ with the viewer's readable spaces. An agent token scoped to a project resolves
+ to the intersection of the spaces it may read.
+2. **Cross-space links resolve by ID.** `[[SPEC-0007]]` works project-wide;
+ relative paths work within a space only. The ID registry rejects a merge that
+ would collide two IDs inside one project.
+3. **Read-only mounts are first class.** `second-brain`, `docs/` directories of
+ existing git.sr.ht repos, man.sr.ht wikis — all searchable without migrating
+ anything anywhere. This is likely the fastest route to the service being
+ useful on day one, since it needs no content to be moved into it.
+
+Incremental reindex on merge: a merge to a space's approved branch reindexes only
+the changed documents, in every project index that includes that space.
+
+## The three planes
+
+### 1. Read plane (anonymous-capable, cached)
+
+`GET /~user/space/path.md` renders. Content negotiation gives `.md` raw,
+`.json` metadata+body, `?rev=<sha>` pinned to an immutable revision.
+
+**Reads default to the approved revision**, with a visible "draft is 3 changes
+ahead" affordance. This is the plane bots consume; it must be boring and
+pinnable. Serving drafts by default would poison every downstream agent context
+with unreviewed text — which is the exact failure this whole service exists to
+prevent.
+
+Implementation is warren, absorbed: `vault/` (scan + frontmatter), `render/`
+(goldmark + wikilinks), `index/` + `search/` (bleve keyword + optional vector),
+`linkcheck/`.
+
+### 2. Write plane (agents)
+
+```http
+PUT /api/v1/spaces/~bigbes/rfcs/docs/specs/0007-storage.md
+If-Match: <base-rev-sha>
+X-Proposal: <id> # omit to open a new one
+```
+
+The body is the whole document. **Whole-document upload, not patches** — that is
+how agents actually work, and it makes the merge model trivial (see below).
+`If-Match` gives optimistic concurrency: two agents editing the same document
+cannot silently clobber each other, and the loser gets a 409 telling it to refetch
+and re-propose. That is exactly the right UX for a bot, which can re-derive its
+edit cheaply.
+
+The write always lands on `proposals/<id>`, never on the approved branch.
+
+### 3. Review plane (humans)
+
+**The inbox is the product.** The landing page is "N proposals waiting on you",
+not a file browser. A proposal page shows the prose diff, per-document, with
+approve / request-changes / reject, and (post-v1) inline comments and a web
+editor that turns a human edit into a commit on the proposal branch.
+
+## Merge model: no text merge, ever
+
+**Verified constraint:** `go-git` v5.19.1 implements only `FastForwardMerge`
+(`repository.go:1800`; anything else returns `ErrUnsupportedMergeStrategy`).
+There is no three-way merge available in-process, and shelling out to `git` is
+against the house pattern established by compare.sr.ht.
+
+This constraint is a gift, because the whole-document grain makes a text merge
+unnecessary. Merging proposal `P` (based on `B`, touching file set `F`) into
+approved head `H`:
+
+```
+for f in F:
+ if blob(f)@H != blob(f)@B: # someone else changed this doc since B
+ return 409 stale # refetch and re-propose
+newTree = tree(H) with each f replaced by P's blob
+commit newTree with parents [H, P.head]
+```
+
+Pure plumbing — `object.Tree` manipulation plus a commit with two parents, all of
+which go-git supports directly. No merge algorithm, no conflict markers, no
+conflict-resolution UI, ever. A conflict is always "your base moved, re-propose",
+which is trivial for an agent and comprehensible for a human. The real merge
+commit keeps the proposal visible in `git log`.
+
+## The two hard parts
+
+Naming these now so they are not discovered late.
+
+### Prose diff, not line diff
+
+Markdown reflows. A one-word edit renders as a whole-paragraph replace under a
+line-oriented differ, which makes reviewing agent output miserable — and
+reviewing agent output is the entire product. What is needed is **word-level
+intra-paragraph diffing over the rendered block structure**, not
+`diff --git` output piped into a viewer.
+
+This is the single UI decision that determines whether the service is pleasant or
+useless, which is why it is the Phase 0 gate below. Note that compare.sr.ht's
+`@pierre/diffs` bundle is a *code* differ and is the wrong tool here; the prose
+differ is likely net-new (segment into blocks → align blocks → word-diff within
+matched blocks).
+
+### Comment anchoring (post-v1, but design now)
+
+`../sourcehut-compare/docs/inline-comments.md` already hit this with
+`(file, side, line)` against moving refs. In prose it is worse, because line
+numbers are meaningless across a reflow.
+
+Anchor to `(doc id, heading path, block index, block content hash)`. Resolve by
+content hash first, fall back to heading-path + block index, and when both fail
+mark the comment **outdated** rather than silently relocating it. Anchoring to
+`doc id` rather than path is what makes comments survive renames.
+
+## Agent identity and provenance
+
+Per-space (or per-project) tokens with a role — `reader` / `proposer` /
+`writer` — and a **required agent identity string**. Every commit records it in
+a way that survives clone:
+
+```
+Author: claude-code/spec-writer (for bigbes) <agent@srht.bigb.es>
+Committer: bigbes <bigbes@gmail.com>
+
+ Add storage model section
+
+ X-Agent-Session: 8fb9c9a4-b078-4af1-89eb-d97c522f9921
+ X-Agent-Base: <sha>
+```
+
+Git trailers rather than a Postgres-only audit table, so provenance is visible in
+plain `git log` on any clone and cannot drift from the content it describes.
+
+MCP is a **first-class surface, not a wrapper** — it is how agents will actually
+consume this: `spec_search`, `spec_read`, `spec_propose`, `spec_comment`,
+`spec_status`. warren's `mcpsrv/` is the starting point.
+
+## Reuse inventory
+
+| From | What | Notes |
+|---|---|---|
+| `../warren` | `vault/`, `render/`, `index/`, `search/`, `mcpsrv/`, `linkcheck/` | The entire read plane and bot retrieval surface, already written. Absorbed into this module. |
+| `../sourcehut-compare` | chrome/templates, cookie→identity, GraphQL authorizer + TTL cache, SCSS pipeline, `contrib/` nginx+systemd shape | Closest sibling; copy the integration scaffolding wholesale. |
+| `../sourcehut-dolt` | bare-store lifecycle under `<repos>/~user/<name>`, brant migration wrapper, config validation | Same storage-root and migration patterns. |
+| `sr-ht-core` (fork) | config, crypto, auth, database, server | Pinned to `git.srht.bigb.es/~bigbes/core-go` via `replace`, as in both siblings. Never `go get -u`. |
+
+Genuinely net-new: proposals, prose diff, review UI, agent tokens, frontmatter
+lifecycle, projects/aggregation.
+
+## Phases
+
+**Phase 0 — de-risk gate.** Two spikes, both must pass before anything else is
+built:
+
+1. go-git write path in-process: create branch → commit blob → build merged tree
+ → two-parent commit, on a real bare repo, no shell-out.
+2. Prose word-diff rendered against two real revisions of an actual existing spec.
+ **If the diff does not read well, the design needs rethinking before any
+ further code.** This is the compare.sr.ht/dolt.sr.ht spike discipline applied
+ to the riskiest assumption here.
+
+**Phase 1 — core + storage.** Pure domain (space/doc/rev/ID validation,
+frontmatter parse + schema validation), bare-repo lifecycle, materialized
+checkouts, Postgres schema + migrations.
+
+**Phase 2 — read plane.** warren absorbed, chrome from compare.sr.ht, unified
+login, ACLs, `?rev=` pinning, read-only mounts, project index with query-time ACL
+filtering.
+
+**Phase 3 — write plane.** Proposals, `If-Match` concurrency, the merge model,
+agent tokens + provenance trailers, REST + MCP.
+
+**Phase 4 — review UI.** Inbox, prose diff, approve / request-changes, status
+lifecycle transitions.
+
+**Phase 5 — later.** Inline comments (anchoring above), web editor, webhooks and
+notifications, vector search over the project index.
+
+## Open items
+
+- **Naming.** `spec.sr.ht` / `spec.srht.bigb.es` follows the
+ named-by-function pattern of the two siblings. `docs.sr.ht` collides
+ conceptually with upstream `man.sr.ht`.
+- **Project URL namespace.** `~user/+project` distinguishes projects from spaces
+ (`~user/space`) in one character; alternatives are `/projects/~user/name` or
+ reusing hub.sr.ht's own namespace.
+- **Port.** compare.sr.ht is on 5090, dolt.sr.ht on 5306–5308. 5091 is free.
+- **LICENSE.** Unchosen, same as compare.sr.ht. SourceHut's own services are
+ AGPL/GPL.