~bigbes/sr-ht-spec

ref: 0a32fd7a58ef967cab397053f9f3b59abd719375 sr-ht-spec/docs/DESIGN.md -rw-r--r-- 80.9 KiB
a4d8cc52 — Eugene Blikh 10 days ago
authn: remove the local agent-token plane

The agent_token table and everything that minted, verified, listed or revoked
it. Agent credentials are tokens.sr.ht working tokens now: signed, expiring,
owned by a meta.sr.ht account and carrying grants, verified locally by
sr-ht-ecore's bearer package. One door, and nothing behind it — a credential the
instance plane refuses is refused, where it used to be offered to a second store
that might say yes.

DEPLOY GATE: do not deploy this until every agent configured with the shared
secret holds a tokens.sr.ht token with spec:propose (spec:read to read).
Migration 0005 drops the table, migrate-on-upgrade runs it on deploy, and
deploying early locks out every agent at once — including the SSH push path.

hooks/ no longer reads agent_token directly. The push path goes through the same
authn.Resolver the HTTP surfaces use and demands spec:propose, because a push by
an agent is a proposal by another transport. The refs rule and the provenance
requirement are untouched on both counts: a universal grant is still only an
agent to the receive path, and X-Agent / X-Agent-Session are still mandatory on
every agent write.

bearer.ErrNotOurs is now a permanent refusal. A meta.sr.ht PAT used to fall
through to the local store and miss there; with no store to fall through to it
earns a 401 rather than the 503 an unclassified error would.

[tokens.sr.ht] origin becomes a required config key: with no issuer there is no
credential to check, so the daemon fails startup instead of serving reads and
refusing every agent write one request at a time.

/tokens redirects to the daemon that issues (tokens SPEC ch. 7).
0879325b — bigbes 27 days ago
docs: the empty-space-list polarity is retracted, not just documented

Search queries now take the filter type with three explicit states, so an
empty set selects nothing everywhere and the corpus-wide case is spelled
EverythingFilter. The document previously said an empty search space list
means every space, which the fix deliberately reverses.

Also records that introspection is gated along with the rest of the read
plane, and that nothing bounds GraphQL query complexity because the
service authenticates on its own router rather than core-go's.
cac9fe1f — bigbes 27 days ago
docs: three receive hooks, and the read plane's rev guard

The two-hook split was unimplementable. Push options reach pre-receive and
post-receive only, so the skip-validation escape hatch was unreachable
where the design put it; and pre-receive cannot read the pushed objects at
all, because they sit in the quarantine until after it runs. update is
therefore the earliest hook that can validate, which the design relied on
without saying why. Work splits across three hooks correlated by
repository and receive-pack pid.

Records that per-ref rejection is not atomic: a push of main plus a
proposal branch, with main rejected, still lands the proposal.

Settles the read plane's audience, which was anonymous-capable in one
section and leak-free in the verification checklist. It is fail-closed.

Documents the rev guard: reads take the approved head or a full object
name, never a ref, because rev=proposals/42 would otherwise serve
unreviewed text as approved through the read plane itself.
c833182e — bigbes 27 days ago
docs: add the project schema and record the filter-polarity trap

Projects are no longer deferred, so the schema section carries them.

Records why the meta-project is an implicit filter rather than a row: a
stored +everything needs a sync job on every space creation, whose only
failure mode is silently omitting a space, and a corpus-wide view must not
be renameable or deletable.

Flags the polarity hazard between the two layers. An empty project selects
nothing while an empty search space list means every space, so passing a
project's members straight into a query turns an empty project into the
whole corpus. The durable fix is for queries to take the filter type
rather than a bare slice, which is deferred while the read surfaces are
mid-build against the current signature.
54fac8dd — bigbes 27 days ago
docs: resolve bilingual search with per-line routing

Per-document language routing, the option this document suggested, was
implemented and measured as insufficient: a dominantly Russian spec
quoting English verbatim matches attachments but not attachment. A single
analyzer does not break the other language, it silently stops stemming it,
which is why the flaw survives casual testing.

Per-line routing into ru and en title/body fields, queried across all
four, finds both halves. Dual-writing was rejected despite also fixing
stemming, because it biases ranking toward mixed documents by summing two
field scores. The threshold is 35 percent Cyrillic rather than 50, since
Russian technical prose here routinely runs a third Latin.

Also records that filters must be exact-match — warren filtered on an
analyzed field, which at our scale means a filter for one space can return
another's documents, and a project is defined as a space filter — and
corrects post-receive's unit of freshness to a space at a revision.
0780b4cc — bigbes 27 days ago
docs: guard the reconciler against eating live proposals

Implementing the corrected repair table showed it destroys data as
written, in two ways.

Row-first ordering is forced by the schema, so every proposal passes
through the exact state the table says to delete. An ungraced reconciler
on a timer would delete an agent's work at random and never once in a
test. A row younger than the grace window is in flight, not abandoned.

A proposal branch is cut at the approved head, so its tip is trivially an
ancestor of that head until the agent's first commit. The plain ancestry
rule marks a contentless proposal merged, and open to merged is terminal.
Comparing the tip against the recorded base distinguishes never-written
from actually-merged.

Also records the within-pass ordering that makes the grace window
sufficient, that marking merged must invent approval and merged_rev and
why policy is the safe lie, the two states left unrepaired, and that
spaces are created repository-first while proposals are created row-first.
2b11f1ae — bigbes 27 days ago
docs: the design is no longer unimplemented

The header still claimed nothing was built. Six packages and one binary
are committed and green, and the document has been corrected against them
six times.
5d3d9441 — bigbes 27 days ago
docs: pin URL grammar, document addressing, and the attachment gap

The read plane specified both that /~user/space/path.md renders and that
.md returns raw source, which cannot both hold for one URL. Extensions are
format selectors, so a document's own address carries none.

Adds the addressing rule for documents whose id is absent or duplicated,
which skip-validation makes reachable on the approved branch. A duplicated
id resolves to neither document: picking one silently would aim links and
search results at a document nobody chose, with no visible sign.

Records that cross-space id resolution cannot live in the read layer,
since an archive is one space at one revision; the global hop needs the
registry and therefore belongs in service/.

Promotes attachments from a hypothetical to a measured gap: the git layer
walks and accepts .md only, so embeds have nothing to resolve against.
1f4a8eda — bigbes 27 days ago
docs: record the Phase 0 verdict and its Phase 4 requirement

The gate passed on real data: rewrapping this document from 80 to 58
columns without changing a word gives 1563 changed lines under a line
differ and zero under the prose differ.

The verdict carries a requirement rather than just a pass. Thirteen
percent of real prose modifications shred into interleaved fragments
because those paragraphs genuinely were rewritten sentence by sentence,
so it cannot be fixed in the differ. Every such block scores similarity
at or below 0.73, so the review UI must switch to a two-column old/new
view below ~0.75; building only the inline renderer would make one review
in eight unreadable.

Also records that the differ already emits the comment-anchor tuple, with
heading path excluded from the hash so renaming a section does not orphan
every comment beneath it.
db7d6c0f — bigbes 27 days ago
docs: correct the merge model against implementation

Resolves a contradiction: the Space section called policy changes
reviewable, but the merge is keyed by document ID and .spec.yml has none,
so a proposal could never express one. Policy joins deletion and rename as
human-push-only, which costs nothing with a single reviewer.

Corrects the claimed cost of ID-keyed staleness. It needs an ID-to-path
index over both whole trees, not one parse of the changed set; free at our
volume, two full-tree reads at ten thousand documents.

Adds the three staleness cases the two-line rule omitted, each of which is
silent corruption if unhandled, and records that an already-merged
proposal needs an ancestry check because the staleness rule reports a
confusing 409 instead.

Records that malformed documents on the approved branch are tolerated
rather than fatal, since skip-validation guarantees they can exist and
failing the index build would turn one typo into a space-wide outage.
d148a661 — bigbes 27 days ago
docs: fix an unimplementable crash repair

The repair table claimed the reconciler recreates a proposal row from an
orphan ref. It cannot: the id is a Postgres serial, and title, rationale,
base_rev, agent and agent_session exist nowhere in a ref, since trailers
live on commits and title lives nowhere in git.

Splits the truth rule instead. Refs stay authoritative for merged-ness;
the row is authoritative for existence and metadata. Row-first ordering is
already forced by deriving the branch name from the serial, and it is also
the safe order, so the unrecoverable case is designed out rather than
repaired. Orphan refs and contentless rows are deleted, which is safe
because an agent still holds the content and can re-propose.

Also records the schema-enforced invariants, notably that merged implies a
non-null approval so policy merges cannot be laundered as human ones.
7c827866 — bigbes 27 days ago
docs: align provenance section with the one-token decision

The section still described per-space tokens with reader/proposer/writer
roles, which the decisions table and the authorization section had already
retracted in favour of one token plus mandatory provenance.

Also records what implementation had to settle: the agent mailbox derives
from our own origin rather than the bare cookie domain, X-Agent-Base is
mandatory and must be an object name, identity travels as X-Agent and
X-Agent-Session with the base coming only from If-Match, a non-owner human
resolves to anonymous, and cookie expiry is not enforceable so network-key
rotation is the only logout-everywhere lever.
ba61af74 — bigbes 27 days ago
docs: correct the core-go dependency pin

The doc repeated dolt.sr.ht's README claim that core-go is wired via a
replace directive to the instance fork. Neither sibling's go.mod contains
one; both require sourcecraft.dev/bigbes/sr-ht-core directly at
dd418a200152, which is the module path the fork declares from that commit
onward. A replace against the older c2c2f38 is rejected outright, since
its go.mod still declared git.sr.ht/~sircmpwn/core-go.
a7336236 — bigbes 27 days ago
docs: add repo layout, schema, implementation plan and verification

Extends the design with the sections the sibling services carry: package
tree with a strict downward dependency rule, the Postgres schema, a
wave-based implementation plan, an end-to-end verification checklist and
the open risks.

Work is dispatched per the parallel-implementer convention: foundation
commit carries every external dependency and the core package, then
siblings importing it write disjoint directories in parallel. Phase 0
runs the prose-diff spike first because it is the assumption with no
fallback.

Projects and comments are deliberately absent from the schema: a project
is a saved filter that is speculative before there are spaces to filter,
and comment anchoring should be settled against a built review UI rather
than committed to a schema first.
12e043e2 — bigbes 27 days ago
docs: settle the seven questions raised by review

- Read contract keeps pinned ?rev= and the approved/draft split. The
  review UI needs blob->render at arbitrary revs regardless, so pinning is
  nearly free and makes X-Agent-Base auditable.
- Materialized checkout dropped. One read path over git objects for
  approved head, pinned revs and proposal branches alike, which removes
  atomic swap, rev stamps, a cache directory and two crash-repair rows.
  Cost is bounded: vault.Scan touches the filesystem twice and
  vault.FromPages already accepts pre-loaded pages.
- Volume is tens of documents a day, so warren's batch index rebuild is
  absorbed unchanged and incremental indexing is explicitly not built.
- One agent token plus mandatory provenance. The refs rule is the boundary
  that bounds damage; per-space scoping defers to a column and a filter.
- Human pushes are validated too, with a skip-validation push option. The
  risk is a typo corrupting the global ID registry, not malice.
- External corpora left unspecified; global IDs are the only forward
  compatibility needed.
- Name confirmed as spec.sr.ht.
f0dcfb4d — bigbes 27 days ago
docs: confirm browser review, and make the link the entry point

Review happens in a browser, so the prose differ stays in v1 and remains
the Phase 0 gate.

The link, not the inbox, is the normal way in: you are usually already
talking to an agent when it proposes, so every write response must carry
the proposal URL for the agent to surface. Proposal URLs are stable past
merge or rejection. The inbox demotes to a backstop for unattended work,
sharing a page with the policy-merged digest, which is invisible for the
same reason.
1e7e4b1e — bigbes 27 days ago
docs: correct errors found by independent review

Two claims were plainly wrong and load-bearing:

- post-receive cannot reject a push; its exit status is ignored once refs
  have moved. Validation and the refs rule move to the update hook. The
  hooks are also not 'zero service code': bleve is single-writer and the
  daemon holds the index, so both hooks RPC into the daemon, and push
  fails closed when it is unreachable.
- hut builds endpoints from per-service origins, never from api.sr.ht, so
  federation buys it nothing and 'one endpoint, one token' is hollow. The
  decision is now a read schema at our own /query; federation is a free
  config line rather than a motivation.

Also resolves an internal contradiction: the doc argued for a single
shared index and then specified one bleve index per project. Now one
global index with projects as query-time filters, and globally unique
document IDs, which removes the per-project collision hole.

Further corrections: merge staleness keyed by document ID rather than
path, deletion and rename made human-push-only, approval expressed as a
property of the branch rather than frontmatter, the proposal state machine
collapsed to open/merged/rejected for a single reviewer, If-Match pinned
to the approved head, and a new consistency section covering crash repair,
the reconciler, per-space mutexes and unix ownership.

Records warren's index as a batch full rebuild rather than incremental,
and the eight questions from review that still need answers.
fc499243 — bigbes 27 days ago
docs: fold in scope, audience, cadence and edit-path decisions

Four confirmed answers reshape the design rather than just the backlog:

- Agent-authored specs only. Read-only mounts leave v1; the meta-project
  now unifies owned spaces. The store is a fresh silo, so it is empty
  until filled, which reorders the phases.
- Single-user. Visibility levels, approver lists, approval counts and
  request-changes cycles drop. Authorization relocates rather than
  disappearing: it is about scoping agents, not separating humans.
- Bimodal cadence. Records approval as human or policy so auto-merged
  notes are not laundered as reviewed, and adds a digest so the firehose
  half stays visible.
- Human edits via clone and push. Adds the git remote as a v1 requirement
  and drops the web editor. Humans push to the approved branch, agents may
  only write proposals/*; SSH plus a post-receive hook needs no service
  code and validates every write path.

Also records mixed ru/en search analyzers as an unresolved question that
determines whether search is usable at all.
72f58940 — bigbes 27 days ago
docs: federate the read side into api.sr.ht from Phase 2

Reverses the earlier no-federation call. The deciding arguments are one
endpoint plus one meta PAT for agents already querying git/todo/builds,
hut ergonomics, and core-go/webhooks being GraphQL-native so Phase 5
pulls gqlgen in regardless. The dolt precedent does not generalize: its
API is a chunk-store protocol, whereas documents and proposals are an
ordinary CRUD graph.

Mutations stay on REST + MCP. If-Match optimistic concurrency is an HTTP
idiom with well-defined 409 semantics, and a federated type is a consumed
contract, so the unsettled proposal types stay out of the gateway.

Also records that federation is not cross-service search: thistle merges
schemas and routes fields, so the meta-project still needs our own index.
c945f382 — bigbes 27 days ago
docs: add SourceHut integration section to the spec.sr.ht design

Records the config-driven wiring (Recipe B: pure Go, chrome copied from
compare.sr.ht), the canonical [spec.sr.ht] keys, and the shared keys read
in place.

Documents why skipping GraphQL federation is safe: api.sr.ht federates
every .sr.ht config section with no allow-list, but updateSchema skips
services that fail to serve /query, and the refresh is SIGHUP-driven
rather than a ticker.
Next