#spec.sr.ht — reviewable document storage for humans and agents

Status: design / proposal — nothing implemented yet. Author: bigbes (with Claude). Last updated: 2026-07-22.

#Context

bigbes wants a third custom service on the self-hosted SourceHut instance (*.srht.bigb.es), after dolt.sr.ht (../sourcehut-dolt) and compare.sr.ht (../sourcehut-compare): separate storage for specs and documents that is reviewable by a human, modifiable by a human, and uploaded to and read by bots.

Those three verbs are not three features. They are one loop:

bot produces → human curates → bots consume.

Everything below follows from taking that loop seriously. Three properties of the loop are what a plain git repo, a wiki, or upstream man.sr.ht each fail to provide:

1. The unit of work is a proposal, not a commit. Agents produce more text than a human can read. A store where agents write freely and nobody reviews is a wiki that rots within a week. The thing you interact with daily is a review queue, not a file tree.
2. Bots need a stable read contract. "Give me the approved text of SPEC-0007", not "HEAD of main, which another agent is halfway through rewriting". Draft and canonical must be different addresses.
3. Provenance is review context. Which agent, which session, on whose behalf, against which base revision. Reviewing agent output without that is reviewing anonymous text.

Same integration model as the two siblings: one Go module, config-driven [spec.sr.ht] section, unified-login cookie, no upstream SourceHut modification. Runs at https://spec.srht.bigb.es.

#Decisions taken (user-confirmed, 2026-07-22)

#Architecture summary

One Go module sourcecraft.dev/bigbes/sr-ht-spec, one binary specsrht (plus specsrht-migrate, a brant wrapper, exactly as dolt.sr.ht does it).

   agents ──MCP / REST──┐
                        ├──►  specsrht  ──►  bare git repos   (authoritative)
   humans ──web UI──────┘        │           <repos>/~user/<space>
                                 │
                                 ├──►  materialized checkouts (read/index cache)
                                 │           <cache>/~user/<space>/
                                 ├──►  bleve indexes, one per project
                                 └──►  Postgres (proposals, comments, ACLs,
                                                 agent tokens, ID registry)

Git is authoritative for document bodies. Postgres never stores a body. That keeps git clone a complete export, keeps git log a complete provenance record offline, and keeps the blast radius of a Postgres restore small.

#Two-tier storage, and why

• Bare repo — the push target and the source of truth. One repo per space.
• Materialized checkout of the approved branch — a plain directory of markdown, rebuilt on every merge. This exists so warren's indexer and renderer can work on files (which is what they already do), and so reads never pay object-database costs. It is a cache: deletable, rebuildable from the bare repo at any time.

#Domain model

#Space

One bare git repo = one space, addressed ~<user>/<space>. The unit of ownership, ACL, and review policy. Contains markdown documents with YAML frontmatter, plus attachments.

A versioned .spec.yml at the repo root carries the space's own policy — which means policy changes are themselves reviewable:

review:
  required: 1                 # approvals needed to merge
  approvers: [~bigbes]
  auto_merge: [notes/**]      # paths that skip the gate
schema:                       # frontmatter contract, enforced at propose time
  required: [id, title, status, owners]
  status: [draft, review, approved, superseded]

#Document

Markdown + YAML frontmatter, matching the second-brain / warren conventions already in use (wikilinks, type/summary/tags).

id: SPEC-0007                 # stable; NEVER changes, including on rename
title: Proposal storage model
status: draft | review | approved | superseded
supersedes: SPEC-0003
owners: [~bigbes]
tags: [storage, review]

id is the load-bearing field. Paths move; IDs do not. Cross-space links resolve by ID, comment anchors reference IDs, and bots pin to IDs. The service enforces ID uniqueness per project via a registry table.

Frontmatter is validated at propose time. A bot that omits status: gets a 422, not a silent merge. Schema validation at the door is the cheapest available defense against agent slop, and it costs almost nothing to implement.

#Proposal

A branch proposals/<id> plus a Postgres row. Bundles N document edits with a title and rationale. State machine:

draft ──► open ──► approved ──► merged
             │└──► changes-requested ──┘
             └──► rejected

#Project

A named set of spaces plus read-only external mounts, with one unified index, one search endpoint, one MCP view, and one wikilink namespace. Modeled on hub.sr.ht's project-groups-repos idea.

project: ~bigbes/+everything
spaces:
  - ~bigbes/tarantool-rfcs
  - ~bigbes/home-ops
mounts:                       # read-only, indexed, searchable, not proposable
  - {type: dir,  path: /Users/blikh/data/home/second-brain}
  - {type: gitsrht, repo: ~bigbes/warren, subdir: docs/}

The requested meta-project — "merge all my doc work into one searchable thing" — is just a project whose membership is everything. There is no separate aggregate entity, no copying, and no sync job. Three consequences that must be designed in from the start rather than retrofitted:

1. ACL filtering happens at query time, against a single shared index. Never build per-user indexes — that path ends in N indexes and a stale-permission bug. Each indexed document carries its space ID; the query layer intersects with the viewer's readable spaces. An agent token scoped to a project resolves to the intersection of the spaces it may read.
2. Cross-space links resolve by ID. [[SPEC-0007]] works project-wide; relative paths work within a space only. The ID registry rejects a merge that would collide two IDs inside one project.
3. Read-only mounts are first class. second-brain, docs/ directories of existing git.sr.ht repos, man.sr.ht wikis — all searchable without migrating anything anywhere. This is likely the fastest route to the service being useful on day one, since it needs no content to be moved into it.

Incremental reindex on merge: a merge to a space's approved branch reindexes only the changed documents, in every project index that includes that space.

#The three planes

#1. Read plane (anonymous-capable, cached)

GET /~user/space/path.md renders. Content negotiation gives .md raw, .json metadata+body, ?rev=<sha> pinned to an immutable revision.

Reads default to the approved revision, with a visible "draft is 3 changes ahead" affordance. This is the plane bots consume; it must be boring and pinnable. Serving drafts by default would poison every downstream agent context with unreviewed text — which is the exact failure this whole service exists to prevent.

Implementation is warren, absorbed: vault/ (scan + frontmatter), render/ (goldmark + wikilinks), index/ + search/ (bleve keyword + optional vector), linkcheck/.

#2. Write plane (agents)

PUT /api/v1/spaces/~bigbes/rfcs/docs/specs/0007-storage.md
If-Match: <base-rev-sha>
X-Proposal: <id>            # omit to open a new one

The body is the whole document. Whole-document upload, not patches — that is how agents actually work, and it makes the merge model trivial (see below). If-Match gives optimistic concurrency: two agents editing the same document cannot silently clobber each other, and the loser gets a 409 telling it to refetch and re-propose. That is exactly the right UX for a bot, which can re-derive its edit cheaply.

The write always lands on proposals/<id>, never on the approved branch.

#3. Review plane (humans)

The inbox is the product. The landing page is "N proposals waiting on you", not a file browser. A proposal page shows the prose diff, per-document, with approve / request-changes / reject, and (post-v1) inline comments and a web editor that turns a human edit into a commit on the proposal branch.

#Merge model: no text merge, ever

Verified constraint: go-git v5.19.1 implements only FastForwardMerge (repository.go:1800; anything else returns ErrUnsupportedMergeStrategy). There is no three-way merge available in-process, and shelling out to git is against the house pattern established by compare.sr.ht.

This constraint is a gift, because the whole-document grain makes a text merge unnecessary. Merging proposal P (based on B, touching file set F) into approved head H:

for f in F:
    if blob(f)@H != blob(f)@B:      # someone else changed this doc since B
        return 409 stale            # refetch and re-propose
newTree = tree(H) with each f replaced by P's blob
commit newTree with parents [H, P.head]

Pure plumbing — object.Tree manipulation plus a commit with two parents, all of which go-git supports directly. No merge algorithm, no conflict markers, no conflict-resolution UI, ever. A conflict is always "your base moved, re-propose", which is trivial for an agent and comprehensible for a human. The real merge commit keeps the proposal visible in git log.

#The two hard parts

Naming these now so they are not discovered late.

#Prose diff, not line diff

Markdown reflows. A one-word edit renders as a whole-paragraph replace under a line-oriented differ, which makes reviewing agent output miserable — and reviewing agent output is the entire product. What is needed is word-level intra-paragraph diffing over the rendered block structure, not diff --git output piped into a viewer.

This is the single UI decision that determines whether the service is pleasant or useless, which is why it is the Phase 0 gate below. Note that compare.sr.ht's @pierre/diffs bundle is a code differ and is the wrong tool here; the prose differ is likely net-new (segment into blocks → align blocks → word-diff within matched blocks).

#Comment anchoring (post-v1, but design now)

../sourcehut-compare/docs/inline-comments.md already hit this with (file, side, line) against moving refs. In prose it is worse, because line numbers are meaningless across a reflow.

Anchor to (doc id, heading path, block index, block content hash). Resolve by content hash first, fall back to heading-path + block index, and when both fail mark the comment outdated rather than silently relocating it. Anchoring to doc id rather than path is what makes comments survive renames.

#Agent identity and provenance

Per-space (or per-project) tokens with a role — reader / proposer / writer — and a required agent identity string. Every commit records it in a way that survives clone:

Author:  claude-code/spec-writer (for bigbes) <agent@srht.bigb.es>
Committer: bigbes <bigbes@gmail.com>

    Add storage model section

    X-Agent-Session: 8fb9c9a4-b078-4af1-89eb-d97c522f9921
    X-Agent-Base: <sha>

Git trailers rather than a Postgres-only audit table, so provenance is visible in plain git log on any clone and cannot drift from the content it describes.

MCP is a first-class surface, not a wrapper — it is how agents will actually consume this: spec_search, spec_read, spec_propose, spec_comment, spec_status. warren's mcpsrv/ is the starting point.

#Reuse inventory

Genuinely net-new: proposals, prose diff, review UI, agent tokens, frontmatter lifecycle, projects/aggregation.

#Phases

Phase 0 — de-risk gate. Two spikes, both must pass before anything else is built:

1. go-git write path in-process: create branch → commit blob → build merged tree → two-parent commit, on a real bare repo, no shell-out.
2. Prose word-diff rendered against two real revisions of an actual existing spec. If the diff does not read well, the design needs rethinking before any further code. This is the compare.sr.ht/dolt.sr.ht spike discipline applied to the riskiest assumption here.

Phase 1 — core + storage. Pure domain (space/doc/rev/ID validation, frontmatter parse + schema validation), bare-repo lifecycle, materialized checkouts, Postgres schema + migrations.

Phase 2 — read plane. warren absorbed, chrome from compare.sr.ht, unified login, ACLs, ?rev= pinning, read-only mounts, project index with query-time ACL filtering.

Phase 3 — write plane. Proposals, If-Match concurrency, the merge model, agent tokens + provenance trailers, REST + MCP.

Phase 4 — review UI. Inbox, prose diff, approve / request-changes, status lifecycle transitions.

Phase 5 — later. Inline comments (anchoring above), web editor, webhooks and notifications, vector search over the project index.

#Open items

• Naming. spec.sr.ht / spec.srht.bigb.es follows the named-by-function pattern of the two siblings. docs.sr.ht collides conceptually with upstream man.sr.ht.
• Project URL namespace. ~user/+project distinguishes projects from spaces (~user/space) in one character; alternatives are /projects/~user/name or reusing hub.sr.ht's own namespace.
• Port. compare.sr.ht is on 5090, dolt.sr.ht on 5306–5308. 5091 is free.
• LICENSE. Unchosen, same as compare.sr.ht. SourceHut's own services are AGPL/GPL.