From 1f4a8edab72379e44a1e25c116e35a2c877d6045 Mon Sep 17 00:00:00 2001 From: bigbes Date: Wed, 22 Jul 2026 11:31:22 +0300 Subject: [PATCH] docs: record the Phase 0 verdict and its Phase 4 requirement The gate passed on real data: rewrapping this document from 80 to 58 columns without changing a word gives 1563 changed lines under a line differ and zero under the prose differ. The verdict carries a requirement rather than just a pass. Thirteen percent of real prose modifications shred into interleaved fragments because those paragraphs genuinely were rewritten sentence by sentence, so it cannot be fixed in the differ. Every such block scores similarity at or below 0.73, so the review UI must switch to a two-column old/new view below ~0.75; building only the inline renderer would make one review in eight unreadable. Also records that the differ already emits the comment-anchor tuple, with heading path excluded from the hash so renaming a section does not orphan every comment beneath it. --- docs/DESIGN.md | 57 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 50 insertions(+), 7 deletions(-) diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 41f2d1883314321bfd53db0397a25f220d384eda..21039d0250446f81887d0f258057385c290d75f3 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -568,10 +568,45 @@ intra-paragraph diffing over the rendered block structure**, not `diff --git` output piped into a viewer. This is the single UI decision that determines whether the service is pleasant or -useless, which is why it is the Phase 0 gate below. Note that compare.sr.ht's -`@pierre/diffs` bundle is a *code* differ and is the wrong tool here; the prose -differ is likely net-new (segment into blocks → align blocks → word-diff within -matched blocks). +useless, which is why it was the Phase 0 gate. compare.sr.ht's `@pierre/diffs` +bundle is a *code* differ and is the wrong tool here; the prose differ is +net-new (segment into blocks → align blocks → word-diff within matched blocks). + +#### Phase 0 verdict: PASSED — build the review UI on this + +Measured against 11 real adjacent-commit pairs of this very document, plus a +control. **The headline number: rewrapping the real 1139-line `docs/DESIGN.md` +from ~80 to 58 columns, without changing a single word, produces 1563 changed +lines under a line differ and exactly 0 changes under the prose differ.** That is +the whole premise of the service, confirmed rather than assumed. + +**But the verdict carries a Phase 4 requirement.** Of 69 real prose +modifications: 39% are one clean substitution, 38% a few separate edits, 10% +busy but followable, and **13% shred** into interleaved fragments. The shredding +is genuine — those paragraphs really were rewritten sentence by sentence — and is +therefore **not fixable in the differ**. It is a presentation problem: + +> Every badly-shredded block scores `Similarity ≤ 0.73`. The review UI must +> render **inline word diffs above ~0.75 and a two-column old/new view below +> it**. `Similarity` is exported on every change for exactly this switch. + +Building only the inline renderer would make one review in eight unreadable, and +that is the one where the agent rewrote the most. + +**Known limitations, each pinned by a test rather than discovered later:** + +- **Moves need a verbatim anchor.** A verbatim section move reports as moves, and + a move with one edited block inside is bridged — but a section rewritten *while* + moving falls through as add + remove. Deliberate: matching moves by similarity + alone invents relationships, and a wrong "moved from line 1065" costs a reviewer + more than an honest add + remove. +- **Table alignment rows are invisible.** goldmark folds `|---|---|` into the + table node, so `|---|` → `|:--|` reports no change at all. A real gap; small, + but silent. +- Word diffs inside a table row can straddle a cell boundary; cell-aware diffing + is a cheap later addition. +- Blocks of two words or fewer fall below the short-block similarity floor and + become add + remove rather than a word diff. ### Comment anchoring (post-v1, but design now) @@ -584,6 +619,12 @@ content hash first, fall back to heading-path + block index, and when both fail mark the comment **outdated** rather than silently relocating it. Anchoring to `doc id` rather than path is what makes comments survive renames. +**The Phase 0 differ already emits this tuple**, so the anchoring model is +validated earlier than expected: every block carries `Hash`, `HeadingPath` and +`Ordinal`. One detail worth keeping — **`HeadingPath` is deliberately not part of +the hash**, so renaming a section does not dirty every block beneath it and +orphan every comment in it at once. + ## Agent identity and provenance ### Authorization is about agents, not people @@ -1064,9 +1105,11 @@ degrading, which is the point of listing them. ## Open risks -- **Prose diff quality (highest).** Everything downstream of the browser-review - decision assumes it reads well. No fallback is designed; Phase 0 exists to find - out early rather than late. +- ~~**Prose diff quality (highest).**~~ **Retired — Phase 0 passed.** Rewrapping + this document from 80 to 58 columns yields 1563 line-differ changes and 0 + prose-differ changes. The residual risk moved to the review UI: 13% of real + modifications shred, and the two-column fallback above is now a Phase 4 + requirement rather than a nicety. - **go-git and native `receive-pack` on the same refs.** Mitigated by a per-space mutex and CAS retry, but the interoperation of the two locking implementations is assumed rather than proven. Worth a deliberate concurrent-push test.