~bigbes/sr-ht-dolt

ref: f91a2e80c6848098c00ef56d75f205e8eb88f744 sr-ht-dolt/docs/DESIGN.views.md -rw-r--r-- 18.7 KiB
f91a2e80 — Eugene Blikh docs: read_rows answers strings or null 5 days ago

#dolt.sr.ht — the beads views, round two

Six changes to the read-only beads surface: a one-column layout beside the parade board, a view for the memories bd remember writes, a freshness line so a stale page says so, a cross-database "what is ready" page, cross-database issue links, and copy-ready bd commands on the detail pane.

docs/DESIGN.md remains the service's architecture document; docs/DESIGN.mcp.md is the agent surface and shares this document's ch. 4 aggregator and its beads/ extraction (that document's §8, a prerequisite here too).

#0. What stays true

  • Read-only. No INSERT, no dolt_commit, no working set. The pure-Go build (gms_pure_go, no CGO/ICU/zstd) is affordable only because this service never starts the SQL engine, and nothing below changes that. Ch. 6 is the deliberate answer to "I can see it but cannot act": hand the human the command.
  • The host's clothes. The board deliberately mimics todo.sr.ht — flat, square, hairline borders, monospace ids, no radius, no shadow, no gradient, colours from the CSS variables that mirror core.sr.ht's Bootstrap palette in both the light default and prefers-color-scheme: dark. Every new surface here is drawn in that same idiom. This is a design decision and not an absence of one: the page is one tab away from todo.sr.ht's own ticket list, and a second visual language inside the shared chrome would read as a different application bolted on.
  • One reading of the schema. After the beads/ extraction, the fingerprint, the lane rule, the ready rule and the status categories exist once and are called by the web views, by /mcp and by ch. 4's aggregator.

#1. The stream layout (?layout=stream)

#1.1 What and why

The parade board is four lanes side by side, each ~15rem wide with the row of lanes scrolling horizontally. That is right when comparing lanes and wrong when reading one: on a laptop the fourth lane is off-screen, on a phone the board is a horizontal scroll of vertical scrolls, and a lane of 35 closed issues pushes everything else out of reach.

Add a one-column stream: the same cards, top to bottom, under section headers in parade order — Rolling → Lined Up → Stalled → Past Stand.

It is a layout of the Beads view, not a fifth tab. ?layout=stream alongside the existing filters, a Board | Stream toggle in the filter bar. Filters, the ready toggle and the search box apply unchanged, links to a card are unchanged (?issue=<id> wins over any layout, as it does today), and the tab bar does not grow a fourth entry for what is one page in two shapes.

#1.2 Data

beadsView.Build gains one branch. BeadsData carries Layout string ("board" — the default and what an unknown value falls back to — or "stream"), and in stream mode fills Sections []BeadsSection, which is BeadsLane plus the two fields a section header needs:

type BeadsSection struct {
    BeadsLane           // Name, Slug, Accent, Issues
    Collapsed  bool     // rendered inside <details> without open
    Note       string   // "" or a one-line hint, e.g. "closed, newest first"
}

Lanes and sections are the same bucketing over the same filtered set: one issue lands in exactly one of them, and the counts in the marquee are the counts in the section headers.

#1.3 Inner sorting

The board sorts every lane the same way (priority, then created_at, then id). A column that is read top to bottom can afford one sort per section, because each lane answers a different question:

Section Order Why
Rolling started_at desc, then priority, then id in-progress work: what was picked up most recently is what is actually being worked on
Lined Up ready first, then priority, then created_at asc, then id this is the "what can I take" section; the ⚡ ready set leads it
Stalled blocked-by count asc, then priority, then id one blocker away is nearer to moving than five
Past Stand closed_at desc, then id a log: the most recently finished on top

started_at and closed_at are already read on the detail pane (BeadIssue); the card model gains them as sort keys and does not display them. A row missing the timestamp sorts last within its section rather than first — an unset value is not a very old one.

Past Stand is collapsed (<details> with a summary carrying the count, no open). It is the largest section, it is the least actionable, and a <details> element needs no JavaScript. The other three are always open.

#1.4 Markup

Reuse .bead-row verbatim — the card is the same card. New: .beads-stream as a single column (max-width: 52rem) and .stream-head, which is .lane-head with position: sticky; top: 0 so the section name stays visible while its issues scroll past. The 2px accent cap and the swatch come along, so a section is recognisable as the lane it is.

┌ Search ────────────┐ ┌types▾┐ ┌prio▾┐ ┌who▾┐ ┌label▾┐ □⚡ready  [Filter]
                                              view: Board · (Stream)

┃ ROLLING                                                              10
  artifacts-46c.1   Бакеты и ключ в garage-стеке
                    P1  task  @Eugene Blikh  milestone:v0.5.0  🚧2  blocks 1
  artifacts-46c.4   artifacts and srht stacks both resolve 'garage' …
                    P1  bug   @Eugene Blikh  milestone:v0.5.0  🚧1
┃ LINED UP                                                             20
  ⚡ artifacts-734  Пакетные репозитории
                    P1  epic  milestone:v0.2.0  pkg  blocks 18
┃ STALLED                                                              13
  artifacts-46c.2   Стек srht: БД, конфиг, образ, Traefik, DNS
                    P1  task  milestone:v0.5.0  rollout  🚧2  blocks 1
▸ PAST STAND (35)

The Board | Stream toggle rebuilds the current query with layout replaced, so a filtered board switches to the same filtered stream. That needs one template func, withQuery (see ch. 7).

#2. The Memory view

Bead sr-ht-dolt-b08 already carries the shape of this view; it is repeated here only where this document adds to it, and the bead's design section stands for the rest.

bd remember writes into the beads config table as ordinary key/value rows — key kv.memory.<slug>, value the memory text (verified against the beads-global and sourcehut-artifacts companions). For a tracker, that is half the content, and today it is visible only in the generic table browser, one line per memory, mixed in with compact_tier2_days and issue_prefix.

  • New view web/memory.go: Name() == "memory", Label() == "Memory", Template() == "memory.html", registered from init() like the others, tab appearing after Milestones (registration order is tab order, and views.go preserves it).
  • Applies: the beads fingerprint plus a config table carrying key and value. Applies sees table shapes and never rows, so a tracker with no memories still gets the tab and renders an empty state — the same contract Milestones has.
  • Build: config rows whose key has the kv.memory. prefix, slug = key without it, sorted by slug (or by age, ch. 2.2). ?q= filters by substring over slug + text; ?key=<slug> renders one memory.
  • Values are stored as typed, so they carry both real newlines and literal \n escapes that agents put in shell strings. Normalise both into paragraphs rather than dumping one blob.

#2.1 The revision each memory was last written at — and why

A memory has no timestamp: the config row is (key, value) and nothing else. So a two-month-old note about a service that has since been rewritten looks exactly like one written this morning, which is how a stale memory keeps being believed. The date is not absent, though — it is in the history. Dolt keeps every commit, and bd remember commits with the message bd: remember (auto-commit) by <author>.

Answer the question from the history rather than from the row:

MemoryRevision{Commit, Date, Author}  — when this key's value last changed

The walk. From the head of ref, newest to oldest, at most memoryWalkMax = 500 commits:

  1. Take the content hash of the config table at the commit — root.GetTable then (*doltdb.Table).HashOf(), which is O(1) and reads no rows.
  2. If it equals the hash at the newer neighbour, config did not change in the newer commit: skip, read nothing.
  3. Otherwise read config at this commit (a tiny table — a dozen rows in every tracker checked) and compare each tracked key against its value at the newer neighbour. A key whose value differs was written by the newer commit; record that commit's hash, date and author, and stop tracking that key.

Cost is one table-hash lookup per commit plus one row read per commit that touched config at all. The most active tracker on this instance has 225 commits three weeks in (measured today via dolt_log), so 500 is roughly two months of headroom at that rate; a key not resolved inside the walk renders "older than the last 500 commits" rather than a date the walk cannot support. The commit message is not the signal — it is a claim by whoever wrote it, and the table hash is the fact.

This needs one new browse primitive:

// TableHash returns the content hash of a table at ref, and ok=false when the
// table does not exist there.
func (db *DB) TableHash(ctx context.Context, refStr, table string) (string, bool, error)

added to browse/tables.go and to the BrowseSession interfaces in web/deps.go and mcpsrv/ports.go. It is computed only by the Memory view and by list_memories — the board never pays for it.

#2.2 Rendering staleness

Each memory shows written <relative> · <short hash> · <author>, the hash linking to the existing commit page. Sort is ?sort=slug (default) or ?sort=age (oldest first — the review queue). A memory older than 60 days carries a muted stale? marker; the number is a default in one constant, not a per-request knob, and the marker is a question rather than a verdict — some memories are meant to be permanent.

Memory · 9 entries                          master · last commit 4 minutes ago
┌ Search ─────────────────┐  sort: (slug) · age                    [Filter]

  handoff-2026-08-12                    written 3 hours ago · a1b2c3d · bigbes
  State after the 2026-08-12 round. Supersedes the two 2026-08-10 handoff
  memories, which asserted …

  metered-link-calibration      stale?  written 71 days ago · 9f8e7d6 · bigbes
  Owner is frequently on metered mobile internet (ZeroTier to the lab, mobile
  uplink). Calibrate downloads …

#3. Freshness in the header

Nothing on the board says how fresh it is. bd pushes with a 30-second debounce and a pull is manual, so a board rendered from a store that stopped receiving pushes yesterday is indistinguishable from a current one — and the whole page reads as fact.

Add a shared partial beadsHead rendering <branch> · last commit <relative> · <short hash> in the header of the Beads, Milestones and Memory views, with the hash linking to the commit page. Data is one Log(ctx, ref, "", 1) call, whose CommitInfo.Date is already carried; the envelope in handleView gains a Head *browse.CommitInfo field so every view gets it without each one asking.

A relative time needs a template func (ago, ch. 7). Absolute time in the title attribute, so the exact stamp is one hover away.

#4. The cross-database ready page

Seventeen databases on this instance carry the beads fingerprint (sixteen per-project trackers and the global one). "What is ready to work" is answerable in each of them and nowhere across them, which is the question the split into a global tracker plus project trackers was supposed to make askable.

Route: GET /ready, its own page (not a View — a View is a rendering of one repository). Linked from the dashboard.

What it does: for every database the caller may browse, open a browse session, check the fingerprint, and if it holds, collect the ready set (open, unblocked, not template/ephemeral — the rule in beads/, not a second copy). Group by database, order groups by ready count desc then name, and inside a group by priority then id. Filters: ?q=, ?assignee=, ?priority=, ?db=<owner>/<name> (repeatable).

Cost, and how it is bounded. N stores opened per request is exactly what the per-request browse discipline does not scale to. Three bounds:

  1. A head-hash gate. Opening a session and calling Branches is cheap; reading and projecting issues + dependencies is not. Cache per database keyed by (repoID, head hash): when the head has not moved, the cached projection stands, no rows are read.
  2. A TTL. Entries expire after 60s regardless, so a store rewritten under the same head (it cannot be, but the cache should not depend on that) heals on its own.
  3. A ceiling. At most readyMaxDatabases = 64 databases per request, and a page that hit the ceiling says so. A silent cap reads as "that is everything".

The cache is a small sync.Mutex-guarded map in the handler's state, sized by entry count, not a new subsystem — and it holds a projection (the ready cards), never a browse.DB handle: an open store is a file handle and a memory mapping, and this is precisely the read pattern the per-request open exists to avoid hoarding.

The aggregator is shared with /mcp. ready_work (DESIGN.mcp.md §9.2) with no database named is this same function; the page and the tool differ in rendering only. Visibility is core.Allowed/OpBrowse per database, applied before a store is opened.

A global-tracker issue that says "blocked by artifacts-nex.2" is naming a row in another database, and the reader has to know which one and go there by hand.

Recognise <prefix>-<suffix> in rendered text — descriptions, design, acceptance criteria, notes, comment bodies, event summaries — and link the ones whose prefix belongs to a database on this instance:

  • The prefix index. Every beads database stores its own prefix in config under issue_prefix (verified: global, artifacts, …). Build an index prefix → repository, warmed lazily and refreshed on the same head-hash / TTL basis as ch. 4's cache, over the databases the caller may browse. A prefix belonging to a database the caller cannot see is not linked, and the page must not reveal that it exists.
  • The pattern. <prefix>-<suffix> where prefix is a known one and suffix is [0-9a-z]+(\.[0-9a-z]+)* — the shape bd generates, including the 46c.2 subtask form. An id in the current database keeps linking to the current view, as it does today.
  • The rendering. Long text is currently dumped into <pre class="field-body"> as plain text. Linkification must escape first, then wrap the matches, building the result as a sequence of escaped segments and generated anchors and only then marking it template.HTML. Marking user-stored text as HTML and running a regexp over it is how a stored payload becomes a rendered one. A unit test with <script> inside an issue description belongs to this change.

#6. Copy-ready bd commands

The page shows work and cannot change it, and that is the architecture, not a gap to fill: a write path means the SQL engine, a working set, a commit and a push — the dependency the build is deliberately free of.

What it can do is hand over the command. On the issue detail pane, a small block under the header:

bd update sr-ht-dolt-b08 --claim
bd close  sr-ht-dolt-b08
  • The block is user-select: all per line, so one click selects a whole command and nothing else. No JavaScript, no clipboard API — the one script in this service is a progressive-enhancement prefill on the keys page, and a copy button that needs JS to work at all would be the first piece of UI here that does.
  • bd is run in a checkout, so the command names the issue and nothing about paths: this service does not know where the tracker is checked out, and guessing ~/data/home/<name> would be inventing a fact about the reader's machine. The database name is right there in the breadcrumb if the reader needs to pick a directory.
  • Commands offered follow status: an open issue gets --claim and close, an in-progress one close and --status=open, a closed one reopen. Nothing is offered that bd would refuse.

#7. Shared bits

Three additions serve several chapters and land with the first that needs them:

Addition Where Used by
ago(t time.Time) string — relative time, absolute in the title web/templates.go funcs ch. 2, 3
withQuery(url.Values, k, v) string — the current query with one key replaced web/templates.go funcs ch. 1 (toggle), 2 (sort)
TableHash(ctx, ref, table) browse/tables.go + both BrowseSession seams ch. 2

#8. Phases

  1. beads/ extraction (DESIGN.mcp.md §8) — pure move, tests move with it. Blocks ch. 4 and ch. 5; ch. 1–3 and 6 can precede it but are cheaper after.
  2. Ch. 1 stream layout + withQuery.
  3. Ch. 3 freshness header + ago (small, and it makes ch. 2's rendering free).
  4. Ch. 2 Memory view + TableHash + the revision walk (bead sr-ht-dolt-b08).
  5. Ch. 6 copy-ready commands (independent of everything above).
  6. Ch. 4 /ready + the aggregator and its cache — after phase 1.
  7. Ch. 5 cross-database links — after phase 6, whose prefix index it shares.

Phases 2, 3 and 5 touch disjoint files and can run in parallel once phase 1 is committed; 4 depends on 3 only for ago.

#9. Tests

  • Stream layout: bucketing identical to the board for the same filters (one test asserting section counts equal lane counts), each section's order, an unknown layout value falling back to the board, the toggle preserving every filter.
  • Memory view: prefix filtering, \n-escape normalisation, ?q=, ?key=, the empty state on a tracker with no memories, and the revision walk against a fixture history where one key is written, changed, and left alone for several commits — including the "not found within the walk" arm.
  • Freshness: an empty database (no commits) renders the header without one.
  • /ready: visibility per database (a PRIVATE tracker the caller may not browse is absent, not 403), the head-hash cache serving a second request without a second read, the ceiling reporting itself.
  • Cross-database links: an id in a visible database links, one in an invisible database renders as plain text, <script> in a description stays escaped, an id-shaped string with an unknown prefix is left alone.
  • Copy block: the offered commands follow the issue's status.