refactor: move the reconciler's two deletes down to the layers that own them
The reconciler reached past its layer twice, because the primitives it
needed did not exist: a raw DELETE FROM proposal — the only SQL written
outside db/ — and a go-git RemoveReference under gitx's write lock. Both
move down, with no change in behaviour.
db.Store.DeleteOpenProposal keeps the guard in the statement, as
resolveProposal does, and distinguishes the two ways it can bite: a row
that is gone is ErrNotFound, a row that has been resolved is the new
ErrProposalNotOpen, which tells the reconciler "nothing to repair" apart
from "the repair no longer applies".
gitx.Repo.DeleteProposalBranch refuses anything outside proposals/* —
the only thing between a caller bug and a deleted approved branch — takes
the per-space write lock like every other write, and treats an
already-absent branch as success: the repair is a postcondition, and the
ref may legitimately vanish between the listing that found it and the
delete.
docs: guard the reconciler against eating live proposals
Implementing the corrected repair table showed it destroys data as
written, in two ways.
Row-first ordering is forced by the schema, so every proposal passes
through the exact state the table says to delete. An ungraced reconciler
on a timer would delete an agent's work at random and never once in a
test. A row younger than the grace window is in flight, not abandoned.
A proposal branch is cut at the approved head, so its tip is trivially an
ancestor of that head until the agent's first commit. The plain ancestry
rule marks a contentless proposal merged, and open to merged is terminal.
Comparing the tip against the recorded base distinguishes never-written
from actually-merged.
Also records the within-pass ordering that makes the grace window
sufficient, that marking merged must invent approval and merged_rev and
why policy is the safe lie, the two states left unrepaired, and that
spaces are created repository-first while proposals are created row-first.
feat: service — wiring, space lifecycle, read paths, push validation, reconciler
Phase 1 of the implementation plan: the orchestration layer's read, validate
and repair halves. The write plane (propose/merge) is Phase 3 and is absent.
- Config/New assemble a Service from the shared config.ini, reporting every
missing key in one message; TokenStore adapts db/ to authn.TokenStore,
mapping db.ErrNotFound onto authn.ErrUnknownToken so an unknown credential
is a 401 rather than a 503.
- CreateSpace writes the repository first and the row second, removing the
repository again if the insert fails; a crash between the two leaves content
that is merely unlisted rather than a phantom space.
- ReadDocument/ListDocuments/Policy/ResolveRev resolve the approved head or a
pinned revision through one code path.
- ValidatePush checks the refs rule first and unconditionally, then frontmatter
and document-id uniqueness, which --push-option=skip-validation waives. The
rejection is a structured, terminal-shaped message naming the document.
- PlanRepairs is the repair table as a pure, table-tested function; Reconcile
gathers the facts and applies them, listing stale-index spaces for Phase 2.
Two departures from the design's repair table, both to stop the reconciler
destroying live state, documented at their definitions: an open row with no
branch is left alone inside a grace window (every propose passes through that
state), and a branch still sitting on its recorded base is never treated as
merged (its tip is trivially an ancestor of the approved head).
docs: the design is no longer unimplemented
The header still claimed nothing was built. Six packages and one binary
are committed and green, and the document has been corrected against them
six times.
docs: pin URL grammar, document addressing, and the attachment gap
The read plane specified both that /~user/space/path.md renders and that
.md returns raw source, which cannot both hold for one URL. Extensions are
format selectors, so a document's own address carries none.
Adds the addressing rule for documents whose id is absent or duplicated,
which skip-validation makes reachable on the approved branch. A duplicated
id resolves to neither document: picking one silently would aim links and
search results at a document nobody chose, with no visible sign.
Records that cross-space id resolution cannot live in the read layer,
since an archive is one space at one revision; the global hop needs the
registry and therefore belongs in service/.
Promotes attachments from a hypothetical to a measured gap: the git layer
walks and accepts .md only, so embeds have nothing to resolve against.
feat: doc — warren's vault+render absorbed onto the git-object read path
warren scanned a directory (filepath.WalkDir + os.ReadFile). There is no
checkout here, so Scan walks a git tree through gitx instead and feeds the
FromPages seam: the approved head, a pinned sha and a proposal branch are one
code path with a different revision, and nothing downstream of Archive knows
where its pages came from.
Frontmatter is core's, not warren's. Front embeds core.Frontmatter and adds
only what core deliberately does not model — parent, aliases, planned, and the
ordered key list used for display and search text. Two parsers that disagree
about a document header is a bug that surfaces in the ID registry months later.
Documents key on their frontmatter id, falling back to their path: paths move
and ids do not, and a duplicated id resolves to neither document rather than
letting one win silently, matching what the merge already does with the
approved branch. A header core rejects degrades to "no frontmatter" instead of
failing, because --push-option=skip-validation means such a document can exist
and refusing to render it would turn a typo into an outage.
fix: anchor installed data paths to ASSETS, and install schema.sql
Two install-time bugs that only show up on a packaged deploy, never in a
checkout.
MIGRATIONDIR derived from PREFIX, so a default build installed migrations
to /usr/local/share/sourcehut/migrations while specsrht-migrate looked
under [sr.ht]assets, default /usr/share/sourcehut. The two agreed only at
PREFIX=/usr. Every installed data path now anchors to ASSETS, which is the
runtime lookup root.
schema.sql was never installed, so specsrht-migrate init could not find
the schema it applies wholesale on a fresh database. sourcehut-dolt has
the same latent path mismatch.
Static assets are guarded so install works before web/ exists.
feat: specsrht-migrate — brant wrapper for the spec.sr.ht schema
Single-service wrapper around git.sr.ht/~bitfehler/brant, a close sibling of
doltsrht-migrate: the brant subcommands (up, down, current, list, stamp,
validate, ping) plus an extra `init` that applies schema.sql wholesale and
stamps to head for a fresh install.
The DSN comes from [spec.sr.ht]connection-string unless --dsn overrides it;
a missing or empty value is a fatal error rather than a default connection.
Migrations load from ./migrations in a checkout, else from
<[sr.ht]assets>/migrations/spec.sr.ht. -a honours [spec.sr.ht]migrate-on-upgrade
and exits early when it is off. lib/pq's "postgres" driver replaces brant's
pgx default, which this module does not link.
Tests cover flag parsing, DSN precedence and the migrations-directory
resolution order without a database; the init/up round trips against a scratch
schema skip unless SPECSRHT_TEST_PG is set.
docs: record the Phase 0 verdict and its Phase 4 requirement
The gate passed on real data: rewrapping this document from 80 to 58
columns without changing a word gives 1563 changed lines under a line
differ and zero under the prose differ.
The verdict carries a requirement rather than just a pass. Thirteen
percent of real prose modifications shred into interleaved fragments
because those paragraphs genuinely were rewritten sentence by sentence,
so it cannot be fixed in the differ. Every such block scores similarity
at or below 0.73, so the review UI must switch to a two-column old/new
view below ~0.75; building only the inline renderer would make one review
in eight unreadable.
Also records that the differ already emits the comment-anchor tuple, with
heading path excluded from the hash so renaming a section does not orphan
every comment beneath it.
chore: promote fernet-go and go-ini to direct dependencies
authn/ imports both directly to decrypt the unified-login cookie and read
the instance ini, so the toolchain reclassifies them. No version changed
and no module was added.
Deliberately not running go mod tidy yet: bleve, chi, brant and the MCP
SDK have no importer until phases 2 and 3, and tidy would drop them from
go.mod, reintroducing it into every later parallel wave's file set.
feat: prosediff — word-level prose diff over markdown block structure
The Phase 0 de-risk gate. Segments a document into blocks with goldmark
(headings, paragraphs, list items, code fences, table rows, block quotes,
frontmatter), aligns the two block sequences with Myers over content
hashes, and diffs word-by-word inside modified prose blocks and
line-by-line inside modified code fences.
Whitespace and line wrapping alone produce no diff in prose, and always
do in code — that split is the whole point. Moves are detected by
verbatim anchor and grown over their neighbours, so a relocated section
does not explode into add+remove; a move that also bridges one edited
block is recognised, a section rewritten while moving is not, and that
limit is pinned by a test rather than papered over.
SPIKE.md reports the verdict against twelve real revisions of
docs/DESIGN.md: rewrapping the whole 1139-line document produces 1563
changed lines for git and zero changes here; 77% of real prose
modifications read as small edits; 13% shred and want a two-column
fallback in the web layer, which BlockChange.Similarity already gates.
Verdict: the approach works, build the review UI on it.
docs: correct the merge model against implementation
Resolves a contradiction: the Space section called policy changes
reviewable, but the merge is keyed by document ID and .spec.yml has none,
so a proposal could never express one. Policy joins deletion and rename as
human-push-only, which costs nothing with a single reviewer.
Corrects the claimed cost of ID-keyed staleness. It needs an ID-to-path
index over both whole trees, not one parse of the changed set; free at our
volume, two full-tree reads at ten thousand documents.
Adds the three staleness cases the two-line rule omitted, each of which is
silent corruption if unhandled, and records that an already-merged
proposal needs an ancestry check because the staleness rule reports a
confusing 409 instead.
Records that malformed documents on the approved branch are tolerated
rather than fatal, since skip-validation guarantees they can exist and
failing the index build would turn one typo into a space-wide outage.
feat: gitx — bare space repos, the git-object read path, and the id-keyed merge
One storage tier: there is no checkout on disk, so every read resolves a tree
and reads blobs. The approved head, a pinned ?rev= and a proposal branch are
the same code path with a different revision.
Lifecycle: Create makes a bare repo at <root>/~<owner>/<space> with HEAD on the
approved branch and one empty initial commit, so nothing downstream has to
special-case an unborn branch. Open reads the approved branch back from HEAD.
Merge: tree-splice, exactly as designed. Staleness is keyed by document ID, not
path — each changed document is resolved to its path on the approved head
through its frontmatter id before its blob is compared, so a rename between the
base and the head neither invents a conflict nor resurrects the moved path. On
staleness a *StaleError carries the current head for the caller's 409.
Otherwise the approved tree is rewritten with each document's blob at its path
in the head and committed with two explicit parents [approvedHead,
proposalHead]. Repository.Merge is never called; go-git v5 supports only
FastForwardMerge and this is pure object.Tree plumbing.
Refs rule: CheckRefUpdate is a pure predicate over (principal, ref, ancestry) —
agents may only touch proposals/*, the approved branch takes fast-forwards from
the human only and is never force-updated or deleted. hooks/ calls it without a
repository, which is why ancestry is an input rather than a lookup.
Concurrency: a process-wide per-space mutex keyed by repository directory guards
every write, and every ref move is a compare-and-swap that rebuilds and retries
when it loses to a concurrent native receive-pack push.
Bounded throughout: context deadlines on every operation, a per-blob cap and
per-walk byte and entry budgets. Nothing is truncated — a partially read
document would be indexed and served as though whole, so it fails instead.
Tests build every fixture programmatically through this package's own plumbing,
never the git binary, and cover the rename between base and head, each
staleness case, the two-parent merge shape, the refs rule for both principals
including force-update rejection, oversized blobs, and the CAS retry.
docs: fix an unimplementable crash repair
The repair table claimed the reconciler recreates a proposal row from an
orphan ref. It cannot: the id is a Postgres serial, and title, rationale,
base_rev, agent and agent_session exist nowhere in a ref, since trailers
live on commits and title lives nowhere in git.
Splits the truth rule instead. Refs stay authoritative for merged-ness;
the row is authoritative for existence and metadata. Row-first ordering is
already forced by deriving the branch name from the serial, and it is also
the safe order, so the unrecoverable case is designed out rather than
repaired. Orphan refs and contentless rows are deleted, which is safe
because an agent still holds the content and can re-propose.
Also records the schema-enforced invariants, notably that merged implies a
non-null approval so policy merges cannot be laundered as human ones.
feat: Postgres schema, initial migration and the db/ query layer
The six tables of the design's Postgres schema — space, document_id,
proposal, agent_token, index_stamp, digest_mark — plus a typed query layer
over them. project and comment stay deliberately absent.
Two invariants are enforced in SQL rather than in Go, because a
check-then-write in Go has a window and this does not:
- Global document ID uniqueness is the doc_id PRIMARY KEY, and the merge
path's multi-row upsert guards its ON CONFLICT branch on the owning
space, so a batch claiming another space's ID leaves that row untouched
and surfaces as a CollisionError naming where the ID really lives.
- Proposal transitions run as UPDATE ... WHERE state = 'open'; the current
state is read back only to name it in the error. CHECK constraints make
an unknown state, a merged row without an approval kind or merged_rev,
and empty provenance unwritable by anything, including SQL that does not
go through this package.
MergeProposal wraps the transition and the registry re-pointing in one
transaction: a merged proposal whose documents are still registered at their
pre-merge paths would break link resolution and the next staleness check.
Agent tokens store only a SHA-256 hash; the lookup boundary re-verifies with
subtle.ConstantTimeCompare and rejects revoked tokens distinctly.
Tests skip when SPECSRHT_TEST_PG is unset, so the pure-logic half — batch
duplicate detection, token comparison, argument guards, and the agreement
between schema.sql and migrations/0001_initial.sql — still runs with no
database available.
docs: align provenance section with the one-token decision
The section still described per-space tokens with reader/proposer/writer
roles, which the decisions table and the authorization section had already
retracted in favour of one token plus mandatory provenance.
Also records what implementation had to settle: the agent mailbox derives
from our own origin rather than the bare cookie domain, X-Agent-Base is
mandatory and must be an object name, identity travels as X-Agent and
X-Agent-Session with the base coming only from If-Match, a non-owner human
resolves to anonymous, and cookie expiry is not enforceable so network-key
rotation is the only logout-everywhere lever.
feat(authn): unified-login identity, agent token validation, provenance trailers
Cookie identity is fail-open to anonymous: a missing, tampered or
foreign-key cookie yields no user rather than an error, so anonymous
browsing never breaks on a bad cookie.
Agent tokens are the opposite. A bad bearer is a permanent 401, a store
outage a transient 503, and revoked rows are returned so operators can
distinguish a killed token from one that never existed.
TokenStore is declared here rather than imported from db/, so the two
packages can be built in parallel and service/ wires them together.
docs: correct the core-go dependency pin
The doc repeated dolt.sr.ht's README claim that core-go is wired via a
replace directive to the instance fork. Neither sibling's go.mod contains
one; both require sourcecraft.dev/bigbes/sr-ht-core directly at
dd418a200152, which is the module path the fork declares from that commit
onward. A replace against the older c2c2f38 is rejected outright, since
its go.mod still declared git.sr.ht/~sircmpwn/core-go.
feat: foundation — go.mod with every dependency, and the core/ domain
Phase 1 foundation commit. Two things, so that later parallel waves write
disjoint directories and never touch go.mod:
- go.mod / go.sum carrying every external dependency the whole module will
need (go-git, bleve, goldmark, chi, lib/pq, yaml.v3, the MCP SDK, brant,
auxilia, testify, and the sr-ht-core fork). Populated by building a
throwaway blank-import file, which is then deleted; `go mod tidy` runs
once, at the very end of the build-out.
- core/, the pure domain: owner/space names, safe relative paths, the
globally-unique document ID grammar, frontmatter parsing and schema
validation, `.spec.yml` policy with auto_merge glob matching, and the
proposal state machine. Standard library plus yaml.v3, nothing else.
Two design invariants are enforced here rather than documented and hoped for:
"approved" is not a status (it is a property of the branch a document is
reachable from), and the proposal machine has exactly open/merged/rejected.
A per-space `.spec.yml` cannot reintroduce either.
Note on the sr-ht-core pin: the design calls for a `replace` onto
git.srht.bigb.es/~bigbes/core-go at c2c2f38, but that commit's go.mod still
declares `module git.sr.ht/~sircmpwn/core-go`, so Go rejects the replacement.
Both siblings pin the later dd418a20 under the canonical path with no
replace; this does the same.
docs: add repo layout, schema, implementation plan and verification
Extends the design with the sections the sibling services carry: package
tree with a strict downward dependency rule, the Postgres schema, a
wave-based implementation plan, an end-to-end verification checklist and
the open risks.
Work is dispatched per the parallel-implementer convention: foundation
commit carries every external dependency and the core package, then
siblings importing it write disjoint directories in parallel. Phase 0
runs the prose-diff spike first because it is the assumption with no
fallback.
Projects and comments are deliberately absent from the schema: a project
is a saved filter that is speculative before there are spaces to filter,
and comment anchoring should be settled against a built review UI rather
than committed to a schema first.