~bigbes/sr-ht-spec

4ad85257 — bigbes 26 days ago
feat(cmd): reindex a space when a push lands

The push notifier was a Phase 1 stub that logged 'reindex pending' because
bleve did not exist yet. It does now, so a pushed document was readable
but never searchable, and the reconciler reported permanent staleness.

Pushes touching only proposal branches skip the rebuild: the index holds
the approved revision, and making unreviewed text searchable is the same
leak as serving it from the read plane.

The index stamp is written last and only on success. Written earlier it
would assert the index reflects a revision it does not — the exact
staleness the reconciler exists to catch, and it would catch nothing.

The hook server is now built after the surfaces, since the notifier
reindexes through the same single-writer index they read from.
2928bf9d — bigbes 26 days ago
feat(cmd): specsrht space create/list

Spaces had no entry point at all: the read plane only reads, and the
proposal API operates on documents inside a space that already exists, so
a freshly deployed instance could not hold anything.

Creation installs the receive hooks itself rather than relying on the
daemon's startup refresh. A space created while the daemon runs would
otherwise accept unvalidated pushes until the next restart — the exact
fail-open the receive path exists to prevent.
96471669 — bigbes 27 days ago
feat(cmd): mount the read plane, MCP and GraphQL surfaces

The three Phase 2 surfaces were built but never served: each was written
under an instruction not to touch cmd/, so every one reported its mounting
call and none of them wired it. The daemon answered /healthz and 404'd
everything else.

They share one search.Index, because bleve is single-writer and a second
Open on the same directory is wrong rather than merely wasteful.

Route order is load-bearing: /mcp and /query register before the web UI
mounts at /, which would otherwise swallow them as document paths — the
router has no reason to think 'mcp' is not a space name.

The MCP handler takes the configured origin as its Host allowlist, which
is why Traefik must pass the Host header through.
0879325b — bigbes 27 days ago
docs: the empty-space-list polarity is retracted, not just documented

Search queries now take the filter type with three explicit states, so an
empty set selects nothing everywhere and the corpus-wide case is spelled
EverythingFilter. The document previously said an empty search space list
means every space, which the fix deliberately reverses.

Also records that introspection is gated along with the rest of the read
plane, and that nothing bounds GraphQL query complexity because the
service authenticates on its own router rather than core-go's.
6f4efc2b — bigbes 27 days ago
feat(graph): the read-only GraphQL schema at /query

Eight query fields over the service layer, no Mutation and no
Subscription — the design defers mutations until the proposal state
machine settles, and TestSchemaHasNoMutations stands guard on that.

Access is fail-closed and gated before parse, matching web's ACL exactly,
so introspection is treated as content too. A federating api.sr.ht must
therefore present a token or skip us, which costs one log line.

A malformed rev is reported as a GraphQL error rather than folded into
null. service/ deliberately hides malformed-versus-absent from probing,
but the caller here is already authenticated as the owner or its own
agent, and a bare null for rev=proposals/42 is indistinguishable from an
absent document — it reads as a silently dropped argument.

Proposal listing declares its port but is unwired: service/ exposes no
proposal read yet, and returning an empty list would tell a reviewer their
queue is clear when it is merely unread.
e0938e94 — Eugene Blikh 27 days ago
refactor: make the filter-polarity trap inexpressible

service.SpaceFilter meant "empty membership selects nothing" — a newly
created project has no members — while search.Query.Spaces was a bare
[]core.SpaceRef whose empty case meant every space. Passing a project's
members into a query therefore turned an empty project into the whole
corpus: a silent scope inversion, invisible when it happens, and passing
every test written with a non-empty project.

The filter moves to core.SpaceFilter with unexported fields, and
search.Query takes it whole. There is no slice to hand over any more, so
the inversion cannot be written. Its three states are distinct: every
space (EverythingFilter), exactly these — possibly none — (SpacesFilter),
and the zero value, which is neither. Search refuses the zero value
rather than defaulting it, because both plausible defaults are wrong for
one of the two callers that can produce one, and returns no hits for a
filter that selects no space without asking the index.

service.SpaceFilter is now an alias for the core type, so ResolveProject
and its callers keep their names. Tests that built a Query without a
scope now say core.EverythingFilter(), which is what they always meant.
5a10600a — Eugene Blikh 27 days ago
feat: service.Archive — one accessor, one tree walk, one link graph

web/ and mcpsrv/ both needed a *doc.Archive and each invented its own way
to get one. web/ handed sp.Repo to doc.Scan — reaching past service/ into
gitx, which the layering rule forbids — and then listed the documents a
second time for their bodies, two tree walks per page view. mcpsrv/
converted []service.Document back into []gitx.Document, rebuilding hashes
service/ had already stringified. Two workarounds around one missing
accessor is how three agent-facing surfaces stop being identical.

Service.Archive resolves the revision, walks the tree once, and returns
the archive with the bodies. ArchiveFrom is the same construction over a
Document set a caller already holds — the one place a hex object name is
converted back to a hash, malformed ids refused rather than zeroed.

doc.Page.Links and Page.WordCount were documented as "filled in by a
render pass" and nothing filled them, so Archive.Backlinks always
returned empty; web/ worked around that by re-rendering every document of
the space on each page view. doc.Archive.LinkPass now owns the pass and
the accessor runs it, so the link graph exists wherever the archive does.

web/ and mcpsrv/ no longer import gitx or go-git, in tests either.
doc.DirOf replaces the three copies of the same fromDir helper.
29370c57 — Eugene Blikh 27 days ago
refactor: one derivation of a proposal's branch name, in core

db.ProposalBranch and gitx.ProposalBranch each spelled out "proposals/" +
id with different signatures and different error behaviour: db formatted
anything including 0, gitx refused a non-positive id. Two derivations of
one name is a proposal whose row and whose ref can disagree, which is
painful to trace and cheap to prevent.

core now owns ProposalPrefix and ProposalBranch(int64) (string, error).
gitx.ProposalPrefix and db.BranchPrefix are that constant, gitx wraps
core's error in ErrBadRev so its callers keep their failure class, and
db.ProposalBranch delegates — inheriting core's refusal of a non-positive
id, which is why its signature grew an error.
e5927f58 — bigbes 27 days ago
chore: promote go-arg to a direct dependency

specsrht-migrate embeds brant's cli.Args, which is go-arg based, so the
toolchain reclassifies it. No version changed and no module was added.
cac9fe1f — bigbes 27 days ago
docs: three receive hooks, and the read plane's rev guard

The two-hook split was unimplementable. Push options reach pre-receive and
post-receive only, so the skip-validation escape hatch was unreachable
where the design put it; and pre-receive cannot read the pushed objects at
all, because they sit in the quarantine until after it runs. update is
therefore the earliest hook that can validate, which the design relied on
without saying why. Work splits across three hooks correlated by
repository and receive-pack pid.

Records that per-ref rejection is not atomic: a push of main plus a
proposal branch, with main rejected, still lands the proposal.

Settles the read plane's audience, which was anonymous-capable in one
section and leak-free in the verification checklist. It is fail-closed.

Documents the rev guard: reads take the approved head or a full object
name, never a ref, because rev=proposals/42 would otherwise serve
unreviewed text as approved through the read plane itself.
eaa2b65f — bigbes 27 days ago
fix(service): separate the read-plane rev guard from the review path

The previous commit's guard broke two service tests, which were right and
the guard was too broad: rendering and diffing a proposal branch genuinely
needs to read one, so a blanket ban on ref names is not the shape.

ReadDocument keeps the strict object-name rule. ReadDocumentAtRef reaches
an arbitrary ref, deliberately as a separate awkwardly-named method rather
than a flag, so serving unreviewed content is something a caller asks for
by name and a reviewer can grep for.

A rejected rev now wraps ErrNotFound as well, preserving the existing
deliberate property that a crafted revision cannot distinguish malformed
from absent by probing.
8edca94e — bigbes 27 days ago
feat(mcpsrv): MCP read tools, with Host validation replacing the SDK guard

Adds the Phase 2 read tools (spec_search, spec_read, spec_list) over the
service layer.

Two security fixes came out of building them.

The read plane could serve proposal content. gitx resolves ref names, and
service.resolveRev passed any string through, so rev=proposals/42 made the
READ plane hand back unreviewed text — which would then flow into agent
context as though approved, the single failure this service exists to
prevent. ValidateReadRev now admits only the approved-head sentinel or a
full 40-character object name, at the layer all three surfaces share.
Abbreviations are refused too: one that is unique today can become
ambiguous later, so a pinned revision would silently stop meaning one
thing.

The MCP SDK's DNS-rebinding guard rejects a loopback listener whose Host
is not loopback, which is exactly nginx forwarding to 127.0.0.1 — it would
403 only in production, passing every local test. The SDK offers no
allowlist, so the guard is disabled and replaced by a stricter check: Host
must equal the configured origin, or a loopback name for development. A
rebinding attack carries the attacker's name in Host and fails it. An
unusable origin logs loudly rather than quietly unguarding the endpoint.
c7e4f9f9 — Eugene Blikh 27 days ago
feat: hooks — the receive path, its daemon RPC, and the specsrht daemon

The hooks are thin shims that RPC into the running daemon over a unix
socket under the repos root, so validation lives in one place and the
push path cannot drift from the API. They fail closed: an unreachable
daemon rejects the push.

Three hooks are installed, not two, because of two properties of git
verified against 2.55 rather than inferred:

  - GIT_PUSH_OPTION_* reaches pre-receive and post-receive only; the
    update hook observably runs without them, so it cannot see
    --push-option=skip-validation on its own.
  - during pre-receive the pushed objects are still in receive-pack's
    quarantine and unreadable by any other process, so the daemon
    cannot validate there. git migrates them out before the first
    update hook, which makes update the earliest hook that works.

pre-receive therefore forwards the push options and the ref list; the
daemon records them keyed by (repository, receive-pack pid) and by the
exact ref update; update does the rejecting; post-receive notifies.
An update with no recorded pre-receive phase is refused rather than
assumed unskippable.

Each hook is a symlink to the specsrht binary, which dispatches on the
name git invoked it as. Install also sets receive.advertisePushOptions,
without which the escape hatch fails client-side. The daemon refreshes
every space's hooks at startup and treats a failure as fatal.

cmd/specsrht validates every config key at once before opening
anything, serves /healthz on -b (default localhost:5091), runs the
reconciler, and warm-shuts-down on SIGINT and SIGTERM alike by
bridging SIGTERM into the SIGINT core-go's server.Run waits for.

Tested with a real git push against a real bare repo with the hooks
installed: a valid push lands, bad frontmatter is rejected with a
readable message, skip-validation waives it, a force-push to the
approved branch is refused with or without skip-validation, a mistyped
push option is refused rather than ignored, and a push with no daemon
is rejected.
2a7569f4 — Eugene Blikh 27 days ago
feat: web — the Phase 2 read plane UI and its SCSS entry

SourceHut chrome (nav service-switcher, login block, environment banner,
error page, embedded hashed static assets) ported from compare.sr.ht, plus
the read-plane pages: landing, space document tree, rendered document with
frontmatter/backlinks, and search.

The URL grammar is the design's pinned one: a document's address carries no
extension, ".md" is raw source, ".json" is metadata plus body, and ?rev=
pins any of the three to an immutable revision. Links rendered inside a
pinned page keep the pin, so following one does not silently land on the
approved head.

One human and no visibility levels, so the read ACL is one line: the owner
and its agents read, everyone else is redirected to meta's login (a browser)
or refused with 401 (a client asking for .md/.json).
c833182e — bigbes 27 days ago
docs: add the project schema and record the filter-polarity trap

Projects are no longer deferred, so the schema section carries them.

Records why the meta-project is an implicit filter rather than a row: a
stored +everything needs a sync job on every space creation, whose only
failure mode is silently omitting a space, and a corpus-wide view must not
be renameable or deletable.

Flags the polarity hazard between the two layers. An empty project selects
nothing while an empty search space list means every space, so passing a
project's members straight into a query turns an empty project into the
whole corpus. The durable fix is for queries to take the filter type
rather than a bare slice, which is deferred while the read surfaces are
mid-build against the current signature.
c23aec57 — bigbes 27 days ago
feat: projects — a saved filter over one global index, not a container

A project is a named space set; querying one filters the single global
index. The meta-project is an implicit filter at a reserved address rather
than a row: a stored +everything would need a sync job on every space
creation, and its one failure mode is silently omitting a space.

SpaceFilter distinguishes All from an empty member list, because a freshly
created project has no members and must mean selects-nothing. Collapsing
the two would make every new project silently match the whole corpus.
54fac8dd — bigbes 27 days ago
docs: resolve bilingual search with per-line routing

Per-document language routing, the option this document suggested, was
implemented and measured as insufficient: a dominantly Russian spec
quoting English verbatim matches attachments but not attachment. A single
analyzer does not break the other language, it silently stops stemming it,
which is why the flaw survives casual testing.

Per-line routing into ru and en title/body fields, queried across all
four, finds both halves. Dual-writing was rejected despite also fixing
stemming, because it biases ranking toward mixed documents by summing two
field scores. The threshold is 35 percent Cyrillic rather than 50, since
Russian technical prose here routinely runs a third Latin.

Also records that filters must be exact-match — warren filtered on an
analyzed field, which at our scale means a filter for one space can return
another's documents, and a project is defined as a space filter — and
corrects post-receive's unit of freshness to a space at a revision.
36b9e183 — Eugene Blikh 27 days ago
feat: search — one global bleve index with per-line ru/en routing

Absorbs warren's index/ and search/ packages, keyword half only, with the
three structural changes the design calls for:

- One global index, not one per project. Every document carries its space;
  a project is a term filter over that field (Query.Spaces), so a merge
  touches one index and the meta-project is a filter that excludes nothing.
- Rebuilds, not incremental updates. RebuildSpace replaces one space at a
  revision, RebuildAll replaces the corpus by building beside the live index
  and swapping. Both report duration in Stats.
- Keyword only. warren's vector store and RRF fusion are not ported, not
  even as dead code; Search returns ranked hits a later ranker can fuse.

Resolves the design's open mixed Russian/English question. Per-document
routing is not sufficient: the ru analyzer passes English through unstemmed
and vice versa, so a Russian spec quoting English requirements loses
singular/plural matching on whichever half is the minority. Text is routed
per line into ru- and en-analyzed field pairs and queried across both.

Not ported: chunking (an embedding concern, and the vector path is Phase 5),
the graph/pages/meta JSON artifacts, and the vault-commit shell-out.
b217c7a7 — Eugene Blikh 27 days ago
refactor: move the reconciler's two deletes down to the layers that own them

The reconciler reached past its layer twice, because the primitives it
needed did not exist: a raw DELETE FROM proposal — the only SQL written
outside db/ — and a go-git RemoveReference under gitx's write lock. Both
move down, with no change in behaviour.

db.Store.DeleteOpenProposal keeps the guard in the statement, as
resolveProposal does, and distinguishes the two ways it can bite: a row
that is gone is ErrNotFound, a row that has been resolved is the new
ErrProposalNotOpen, which tells the reconciler "nothing to repair" apart
from "the repair no longer applies".

gitx.Repo.DeleteProposalBranch refuses anything outside proposals/* —
the only thing between a caller bug and a deleted approved branch — takes
the per-space write lock like every other write, and treats an
already-absent branch as success: the repair is a postcondition, and the
ref may legitimately vanish between the listing that found it and the
delete.
0780b4cc — bigbes 27 days ago
docs: guard the reconciler against eating live proposals

Implementing the corrected repair table showed it destroys data as
written, in two ways.

Row-first ordering is forced by the schema, so every proposal passes
through the exact state the table says to delete. An ungraced reconciler
on a timer would delete an agent's work at random and never once in a
test. A row younger than the grace window is in flight, not abandoned.

A proposal branch is cut at the approved head, so its tip is trivially an
ancestor of that head until the agent's first commit. The plain ancestry
rule marks a contentless proposal merged, and open to merged is terminal.
Comparing the tip against the recorded base distinguishes never-written
from actually-merged.

Also records the within-pass ordering that makes the grace window
sufficient, that marking merged must invent approval and merged_rev and
why policy is the safe lie, the two states left unrepaired, and that
spaces are created repository-first while proposals are created row-first.
Next