~bigbes/sr-ht-spec

e1b3c64e — Eugene Blikh 27 days ago
feat: service — wiring, space lifecycle, read paths, push validation, reconciler

Phase 1 of the implementation plan: the orchestration layer's read, validate
and repair halves. The write plane (propose/merge) is Phase 3 and is absent.

- Config/New assemble a Service from the shared config.ini, reporting every
  missing key in one message; TokenStore adapts db/ to authn.TokenStore,
  mapping db.ErrNotFound onto authn.ErrUnknownToken so an unknown credential
  is a 401 rather than a 503.
- CreateSpace writes the repository first and the row second, removing the
  repository again if the insert fails; a crash between the two leaves content
  that is merely unlisted rather than a phantom space.
- ReadDocument/ListDocuments/Policy/ResolveRev resolve the approved head or a
  pinned revision through one code path.
- ValidatePush checks the refs rule first and unconditionally, then frontmatter
  and document-id uniqueness, which --push-option=skip-validation waives. The
  rejection is a structured, terminal-shaped message naming the document.
- PlanRepairs is the repair table as a pure, table-tested function; Reconcile
  gathers the facts and applies them, listing stale-index spaces for Phase 2.

Two departures from the design's repair table, both to stop the reconciler
destroying live state, documented at their definitions: an open row with no
branch is left alone inside a grace window (every propose passes through that
state), and a branch still sitting on its recorded base is never treated as
merged (its tip is trivially an ancestor of the approved head).
2b11f1ae — bigbes 27 days ago
docs: the design is no longer unimplemented

The header still claimed nothing was built. Six packages and one binary
are committed and green, and the document has been corrected against them
six times.
5d3d9441 — bigbes 27 days ago
docs: pin URL grammar, document addressing, and the attachment gap

The read plane specified both that /~user/space/path.md renders and that
.md returns raw source, which cannot both hold for one URL. Extensions are
format selectors, so a document's own address carries none.

Adds the addressing rule for documents whose id is absent or duplicated,
which skip-validation makes reachable on the approved branch. A duplicated
id resolves to neither document: picking one silently would aim links and
search results at a document nobody chose, with no visible sign.

Records that cross-space id resolution cannot live in the read layer,
since an archive is one space at one revision; the global hop needs the
registry and therefore belongs in service/.

Promotes attachments from a hypothetical to a measured gap: the git layer
walks and accepts .md only, so embeds have nothing to resolve against.
305bb09f — Eugene Blikh 27 days ago
feat: doc — warren's vault+render absorbed onto the git-object read path

warren scanned a directory (filepath.WalkDir + os.ReadFile). There is no
checkout here, so Scan walks a git tree through gitx instead and feeds the
FromPages seam: the approved head, a pinned sha and a proposal branch are one
code path with a different revision, and nothing downstream of Archive knows
where its pages came from.

Frontmatter is core's, not warren's. Front embeds core.Frontmatter and adds
only what core deliberately does not model — parent, aliases, planned, and the
ordered key list used for display and search text. Two parsers that disagree
about a document header is a bug that surfaces in the ID registry months later.

Documents key on their frontmatter id, falling back to their path: paths move
and ids do not, and a duplicated id resolves to neither document rather than
letting one win silently, matching what the merge already does with the
approved branch. A header core rejects degrades to "no frontmatter" instead of
failing, because --push-option=skip-validation means such a document can exist
and refusing to render it would turn a typo into an outage.
64488fc9 — bigbes 27 days ago
fix: anchor installed data paths to ASSETS, and install schema.sql

Two install-time bugs that only show up on a packaged deploy, never in a
checkout.

MIGRATIONDIR derived from PREFIX, so a default build installed migrations
to /usr/local/share/sourcehut/migrations while specsrht-migrate looked
under [sr.ht]assets, default /usr/share/sourcehut. The two agreed only at
PREFIX=/usr. Every installed data path now anchors to ASSETS, which is the
runtime lookup root.

schema.sql was never installed, so specsrht-migrate init could not find
the schema it applies wholesale on a fresh database. sourcehut-dolt has
the same latent path mismatch.

Static assets are guarded so install works before web/ exists.
4562b448 — Eugene Blikh 27 days ago
feat: specsrht-migrate — brant wrapper for the spec.sr.ht schema

Single-service wrapper around git.sr.ht/~bitfehler/brant, a close sibling of
doltsrht-migrate: the brant subcommands (up, down, current, list, stamp,
validate, ping) plus an extra `init` that applies schema.sql wholesale and
stamps to head for a fresh install.

The DSN comes from [spec.sr.ht]connection-string unless --dsn overrides it;
a missing or empty value is a fatal error rather than a default connection.
Migrations load from ./migrations in a checkout, else from
<[sr.ht]assets>/migrations/spec.sr.ht. -a honours [spec.sr.ht]migrate-on-upgrade
and exits early when it is off. lib/pq's "postgres" driver replaces brant's
pgx default, which this module does not link.

Tests cover flag parsing, DSN precedence and the migrations-directory
resolution order without a database; the init/up round trips against a scratch
schema skip unless SPECSRHT_TEST_PG is set.
1f4a8eda — bigbes 27 days ago
docs: record the Phase 0 verdict and its Phase 4 requirement

The gate passed on real data: rewrapping this document from 80 to 58
columns without changing a word gives 1563 changed lines under a line
differ and zero under the prose differ.

The verdict carries a requirement rather than just a pass. Thirteen
percent of real prose modifications shred into interleaved fragments
because those paragraphs genuinely were rewritten sentence by sentence,
so it cannot be fixed in the differ. Every such block scores similarity
at or below 0.73, so the review UI must switch to a two-column old/new
view below ~0.75; building only the inline renderer would make one review
in eight unreadable.

Also records that the differ already emits the comment-anchor tuple, with
heading path excluded from the hash so renaming a section does not orphan
every comment beneath it.
5eab0915 — bigbes 27 days ago
chore: promote fernet-go and go-ini to direct dependencies

authn/ imports both directly to decrypt the unified-login cookie and read
the instance ini, so the toolchain reclassifies them. No version changed
and no module was added.

Deliberately not running go mod tidy yet: bleve, chi, brant and the MCP
SDK have no importer until phases 2 and 3, and tidy would drop them from
go.mod, reintroducing it into every later parallel wave's file set.
3510f9c3 — Eugene Blikh 27 days ago
feat: prosediff — word-level prose diff over markdown block structure

The Phase 0 de-risk gate. Segments a document into blocks with goldmark
(headings, paragraphs, list items, code fences, table rows, block quotes,
frontmatter), aligns the two block sequences with Myers over content
hashes, and diffs word-by-word inside modified prose blocks and
line-by-line inside modified code fences.

Whitespace and line wrapping alone produce no diff in prose, and always
do in code — that split is the whole point. Moves are detected by
verbatim anchor and grown over their neighbours, so a relocated section
does not explode into add+remove; a move that also bridges one edited
block is recognised, a section rewritten while moving is not, and that
limit is pinned by a test rather than papered over.

SPIKE.md reports the verdict against twelve real revisions of
docs/DESIGN.md: rewrapping the whole 1139-line document produces 1563
changed lines for git and zero changes here; 77% of real prose
modifications read as small edits; 13% shred and want a two-column
fallback in the web layer, which BlockChange.Similarity already gates.
Verdict: the approach works, build the review UI on it.
db7d6c0f — bigbes 27 days ago
docs: correct the merge model against implementation

Resolves a contradiction: the Space section called policy changes
reviewable, but the merge is keyed by document ID and .spec.yml has none,
so a proposal could never express one. Policy joins deletion and rename as
human-push-only, which costs nothing with a single reviewer.

Corrects the claimed cost of ID-keyed staleness. It needs an ID-to-path
index over both whole trees, not one parse of the changed set; free at our
volume, two full-tree reads at ten thousand documents.

Adds the three staleness cases the two-line rule omitted, each of which is
silent corruption if unhandled, and records that an already-merged
proposal needs an ancestry check because the staleness rule reports a
confusing 409 instead.

Records that malformed documents on the approved branch are tolerated
rather than fatal, since skip-validation guarantees they can exist and
failing the index build would turn one typo into a space-wide outage.
29f6e881 — Eugene Blikh 27 days ago
feat: gitx — bare space repos, the git-object read path, and the id-keyed merge

One storage tier: there is no checkout on disk, so every read resolves a tree
and reads blobs. The approved head, a pinned ?rev= and a proposal branch are
the same code path with a different revision.

Lifecycle: Create makes a bare repo at <root>/~<owner>/<space> with HEAD on the
approved branch and one empty initial commit, so nothing downstream has to
special-case an unborn branch. Open reads the approved branch back from HEAD.

Merge: tree-splice, exactly as designed. Staleness is keyed by document ID, not
path — each changed document is resolved to its path on the approved head
through its frontmatter id before its blob is compared, so a rename between the
base and the head neither invents a conflict nor resurrects the moved path. On
staleness a *StaleError carries the current head for the caller's 409.
Otherwise the approved tree is rewritten with each document's blob at its path
in the head and committed with two explicit parents [approvedHead,
proposalHead]. Repository.Merge is never called; go-git v5 supports only
FastForwardMerge and this is pure object.Tree plumbing.

Refs rule: CheckRefUpdate is a pure predicate over (principal, ref, ancestry) —
agents may only touch proposals/*, the approved branch takes fast-forwards from
the human only and is never force-updated or deleted. hooks/ calls it without a
repository, which is why ancestry is an input rather than a lookup.

Concurrency: a process-wide per-space mutex keyed by repository directory guards
every write, and every ref move is a compare-and-swap that rebuilds and retries
when it loses to a concurrent native receive-pack push.

Bounded throughout: context deadlines on every operation, a per-blob cap and
per-walk byte and entry budgets. Nothing is truncated — a partially read
document would be indexed and served as though whole, so it fails instead.

Tests build every fixture programmatically through this package's own plumbing,
never the git binary, and cover the rename between base and head, each
staleness case, the two-parent merge shape, the refs rule for both principals
including force-update rejection, oversized blobs, and the CAS retry.
d148a661 — bigbes 27 days ago
docs: fix an unimplementable crash repair

The repair table claimed the reconciler recreates a proposal row from an
orphan ref. It cannot: the id is a Postgres serial, and title, rationale,
base_rev, agent and agent_session exist nowhere in a ref, since trailers
live on commits and title lives nowhere in git.

Splits the truth rule instead. Refs stay authoritative for merged-ness;
the row is authoritative for existence and metadata. Row-first ordering is
already forced by deriving the branch name from the serial, and it is also
the safe order, so the unrecoverable case is designed out rather than
repaired. Orphan refs and contentless rows are deleted, which is safe
because an agent still holds the content and can re-propose.

Also records the schema-enforced invariants, notably that merged implies a
non-null approval so policy merges cannot be laundered as human ones.
dc1e5d0c — Eugene Blikh 27 days ago
feat: Postgres schema, initial migration and the db/ query layer

The six tables of the design's Postgres schema — space, document_id,
proposal, agent_token, index_stamp, digest_mark — plus a typed query layer
over them. project and comment stay deliberately absent.

Two invariants are enforced in SQL rather than in Go, because a
check-then-write in Go has a window and this does not:

  - Global document ID uniqueness is the doc_id PRIMARY KEY, and the merge
    path's multi-row upsert guards its ON CONFLICT branch on the owning
    space, so a batch claiming another space's ID leaves that row untouched
    and surfaces as a CollisionError naming where the ID really lives.
  - Proposal transitions run as UPDATE ... WHERE state = 'open'; the current
    state is read back only to name it in the error. CHECK constraints make
    an unknown state, a merged row without an approval kind or merged_rev,
    and empty provenance unwritable by anything, including SQL that does not
    go through this package.

MergeProposal wraps the transition and the registry re-pointing in one
transaction: a merged proposal whose documents are still registered at their
pre-merge paths would break link resolution and the next staleness check.

Agent tokens store only a SHA-256 hash; the lookup boundary re-verifies with
subtle.ConstantTimeCompare and rejects revoked tokens distinctly.

Tests skip when SPECSRHT_TEST_PG is unset, so the pure-logic half — batch
duplicate detection, token comparison, argument guards, and the agreement
between schema.sql and migrations/0001_initial.sql — still runs with no
database available.
7c827866 — bigbes 27 days ago
docs: align provenance section with the one-token decision

The section still described per-space tokens with reader/proposer/writer
roles, which the decisions table and the authorization section had already
retracted in favour of one token plus mandatory provenance.

Also records what implementation had to settle: the agent mailbox derives
from our own origin rather than the bare cookie domain, X-Agent-Base is
mandatory and must be an object name, identity travels as X-Agent and
X-Agent-Session with the base coming only from If-Match, a non-owner human
resolves to anonymous, and cookie expiry is not enforceable so network-key
rotation is the only logout-everywhere lever.
fe913656 — bigbes 27 days ago
feat(authn): unified-login identity, agent token validation, provenance trailers

Cookie identity is fail-open to anonymous: a missing, tampered or
foreign-key cookie yields no user rather than an error, so anonymous
browsing never breaks on a bad cookie.

Agent tokens are the opposite. A bad bearer is a permanent 401, a store
outage a transient 503, and revoked rows are returned so operators can
distinguish a killed token from one that never existed.

TokenStore is declared here rather than imported from db/, so the two
packages can be built in parallel and service/ wires them together.
ba61af74 — bigbes 27 days ago
docs: correct the core-go dependency pin

The doc repeated dolt.sr.ht's README claim that core-go is wired via a
replace directive to the instance fork. Neither sibling's go.mod contains
one; both require sourcecraft.dev/bigbes/sr-ht-core directly at
dd418a200152, which is the module path the fork declares from that commit
onward. A replace against the older c2c2f38 is rejected outright, since
its go.mod still declared git.sr.ht/~sircmpwn/core-go.
33a3deaf — Eugene Blikh 27 days ago
feat: foundation — go.mod with every dependency, and the core/ domain

Phase 1 foundation commit. Two things, so that later parallel waves write
disjoint directories and never touch go.mod:

  - go.mod / go.sum carrying every external dependency the whole module will
    need (go-git, bleve, goldmark, chi, lib/pq, yaml.v3, the MCP SDK, brant,
    auxilia, testify, and the sr-ht-core fork). Populated by building a
    throwaway blank-import file, which is then deleted; `go mod tidy` runs
    once, at the very end of the build-out.
  - core/, the pure domain: owner/space names, safe relative paths, the
    globally-unique document ID grammar, frontmatter parsing and schema
    validation, `.spec.yml` policy with auto_merge glob matching, and the
    proposal state machine. Standard library plus yaml.v3, nothing else.

Two design invariants are enforced here rather than documented and hoped for:
"approved" is not a status (it is a property of the branch a document is
reachable from), and the proposal machine has exactly open/merged/rejected.
A per-space `.spec.yml` cannot reintroduce either.

Note on the sr-ht-core pin: the design calls for a `replace` onto
git.srht.bigb.es/~bigbes/core-go at c2c2f38, but that commit's go.mod still
declares `module git.sr.ht/~sircmpwn/core-go`, so Go rejects the replacement.
Both siblings pin the later dd418a20 under the canonical path with no
replace; this does the same.
a7336236 — bigbes 27 days ago
docs: add repo layout, schema, implementation plan and verification

Extends the design with the sections the sibling services carry: package
tree with a strict downward dependency rule, the Postgres schema, a
wave-based implementation plan, an end-to-end verification checklist and
the open risks.

Work is dispatched per the parallel-implementer convention: foundation
commit carries every external dependency and the core package, then
siblings importing it write disjoint directories in parallel. Phase 0
runs the prose-diff spike first because it is the assumption with no
fallback.

Projects and comments are deliberately absent from the schema: a project
is a saved filter that is speculative before there are spaces to filter,
and comment anchoring should be settled against a built review UI rather
than committed to a schema first.
12e043e2 — bigbes 27 days ago
docs: settle the seven questions raised by review

- Read contract keeps pinned ?rev= and the approved/draft split. The
  review UI needs blob->render at arbitrary revs regardless, so pinning is
  nearly free and makes X-Agent-Base auditable.
- Materialized checkout dropped. One read path over git objects for
  approved head, pinned revs and proposal branches alike, which removes
  atomic swap, rev stamps, a cache directory and two crash-repair rows.
  Cost is bounded: vault.Scan touches the filesystem twice and
  vault.FromPages already accepts pre-loaded pages.
- Volume is tens of documents a day, so warren's batch index rebuild is
  absorbed unchanged and incremental indexing is explicitly not built.
- One agent token plus mandatory provenance. The refs rule is the boundary
  that bounds damage; per-space scoping defers to a column and a filter.
- Human pushes are validated too, with a skip-validation push option. The
  risk is a typo corrupting the global ID registry, not malice.
- External corpora left unspecified; global IDs are the only forward
  compatibility needed.
- Name confirmed as spec.sr.ht.
f0dcfb4d — bigbes 27 days ago
docs: confirm browser review, and make the link the entry point

Review happens in a browser, so the prose differ stays in v1 and remains
the Phase 0 gate.

The link, not the inbox, is the normal way in: you are usually already
talking to an agent when it proposes, so every write response must carry
the proposal URL for the agent to surface. Proposal URLs are stable past
merge or rejection. The inbox demotes to a backstop for unattended work,
sharing a page with the policy-merged digest, which is invisible for the
same reason.
Next