M CLAUDE.md => CLAUDE.md +3 -1
@@ 65,11 65,13 @@ These are subtle traps the round-trip has been bitten by; touching the related c
- **Nested-list indent depends on the *parent* list type** — CommonMark places content of `1. ` at column 4 and `- ` at column 3, so a nested item must be indented by ≥ the parent marker's width. `xmlConverter.listIndents` is a stack of per-level widths (3 for `<ol>`, 2 for `<ul>` / task-list); `itemIndent()` sums all *ancestor* contributions. Using a fixed `2*(depth-1)` collapses nested ordered lists into siblings. *Tests:* `TestRoundTrip_NestedOrderedList`, `TestRoundTrip_NestedMixedList`, `TestRoundTrip_DoubleNestedUnorderedList`.
- **Tight lists are one stray `\n` away from loose** — a blank line anywhere in a list turns it loose, which makes goldmark wrap every `<li>` in `<p>`. `ensureSingleNewline()` writes `\n` only when the buffer doesn't already end in one, preventing the double newline that appeared when a `<li>`'s body ended with a nested list (which itself terminated in `\n`). Always use it instead of `c.buf.WriteString("\n")` after walking list-item content.
- **Literal angle-bracket text turns into phantom elements** — text stored as `<table>` (a function signature like `box.backup.start(<table>)`) decodes to `<table>` in the text node; emitted raw into Markdown, goldmark parses it as an HTML table and the phantom element swallows the rest of the document. `xml2md.escapeMarkdownAngle` backslash-escapes a *tag-like* `<` (followed by a letter, `/`, `!`, `?`) in **both** text paths — the `walk` TextNode branch *and* `walkChildrenInline` (used for `<li>` bodies, a separate branch that's easy to miss). The matching unescape is in `confluence.Renderer.renderText`: it strips only `\<` (goldmark keeps a backslash-escaped run as one Text node with the backslash intact, and this renderer emits the raw segment) — **never** a blanket `util.UnescapePunctuations`, which would also eat the deliberate `\|` table-pipe escaping and re-break `base.xml`. A non-tag `<` (`a < b`, `<=`) is left alone. *Test:* `TestRoundTrip_LiteralAngleBracketText`.
-- **Emphasis can't be flanked by whitespace** — `<strong> x </strong>` (space just inside the tag) renders as literal `** x **` because CommonMark requires the run to be flanked by non-whitespace. `xml2md.writeEmphasis` (used for strong/em/del) captures the children into a scratch buffer and moves leading/trailing whitespace *outside* the markers (` **x** `). The stored and round-tripped forms then differ only in space placement, which `normalizeForVerify` reconciles (`reEmphLeadWS`/`reEmphTrailWS`). *Test:* `TestRoundTrip_EmphasisInteriorSpaces`.
+- **Emphasis can't be flanked by whitespace** — `<strong> x </strong>` (space just inside the tag) renders as literal `** x **` because CommonMark requires the run to be flanked by non-whitespace. `xml2md.writeEmphasis` (used for strong/em/del) captures the children into a scratch buffer and moves leading/trailing whitespace *outside* the markers (` **x** `). Its trim cutset **must include U+00A0** — Confluence stores stray bold applied to a lone nbsp (`<strong> </strong>`), and an nbsp left as content emits `** **`, which round-trips as literal `** **`; `normalizeForVerify` drops the empty emphasis (`reEmptyStrong`/`reEmptyEm`/`reEmptyDel`) and reconciles the surviving space placement (`reEmphLeadWS`/`reEmphTrailWS`). *Tests:* `TestRoundTrip_EmphasisInteriorSpaces`, `TestRoundTrip_EmphasisNbspOnly`.
+- **A macro marker comment can be inline, not just block** — a TOC macro that lived inside a heading (`<h1><ac:structured-macro ac:name="toc"/></h1>`) round-trips as `# <!-- ac:toc … -->`, where goldmark parses the comment as *inline* RawHTML. `confluence.Renderer.renderRawHTML` must therefore call `convertComment` (the same reconstruction the block path `renderHTMLBlock` uses) before falling back to `convertRawSpan`, or the macro leaks back out as a literal comment. *Test:* `TestRoundTrip_TocMacroInHeading`.
These live in the formatter / verify layer rather than the converter, but bite the same way:
- **Some XML carries values Markdown has no slot for** — `<ac:task-id>` (Confluence renumbers them every save), `<ul style="list-style-type: square;">` bullet styling, and a stray trailing nbsp before `</ac:task-body>` cannot survive a Markdown round-trip and aren't meaningful content. `normalizeForVerify` canonicalizes them on **both** sides (`reTaskID` zeroes the id, `reListStyleAttr` drops `<ul>/<ol>` attributes, `reTrailWSTaskBody` strips the trailing space — note the whitespace class must include `\x{00a0}`, the actual byte stored). *Tests:* `TestNormalizeForVerify_TaskID`, `TestNormalizeForVerify_ListStyleAttr`, `TestNormalizeForVerify_TaskBodyTrailingWS`, `TestNormalizeForVerify_EmphasisWhitespace`.
+- **Structured-macro attribute order isn't canonical** — Confluence emits a TOC macro's attributes as `macro-id,name,schema-version` on one page and `name,schema-version,macro-id` on another, and the verify formatter preserves order, so no fixed reconstruction order matches everything. `normalizeForVerify.sortMacroAttrs` sorts the attributes of every `<ac:structured-macro …>` opener on both sides. Likewise `<s>` ↔ `<del>` (both strikethrough — xml2md emits `~~…~~`, md2xml renders `<del>`) is canonicalized via `reStrikeOpen`/`reStrikeClose`. *Tests:* `TestNormalizeForVerify_MacroAttrOrder`, `TestNormalizeForVerify_StrikeTag`, `TestNormalizeForVerify_EmptyEmphasis`.
- **A "visible char" class in regexes must exclude `<`, not just whitespace** — `verify.normalizeForVerify`'s `reBareTextAfterHeading` wraps bare text between a heading and a following block in `<p>…</p>`. Its "≥1 visible char" token must be `[^\s<]`, never `\S`: `\S` matches `<`, so the capture swallows the next tag's opening bracket (e.g. `<ac:structured-macro>`) and the inserted `</p>` lands between a code macro's open tag and its body — destroying round-trip for a no-language fenced code block after a heading. *Tests:* `TestNormalizeForVerify_BareTextAfterHeading`, `TestVerifyRoundTrip_CodeBlockAfterHeading`.
- **`fmt` line-wrapping must keep `<code>…</code>` whole** — breaking a line inside an inline code span injects the continuation indent into the span's text, which `xml2md` collapses into a spurious space *inside* the span (`` ` app.manifest.lock` ``). `format.splitAtoms` emits an entire `<code>…</code>` as one unbreakable atom. Significant whitespace between adjacent inline elements (`</strong> <code>`) is carried on each atom's `spaceBefore` flag — never via `strings.Fields`, which drops trailing space. *Tests:* `TestPrettyXML_WrapKeepsCodeSpanWhole`, `TestPrettyXML_WrapPreservesSpaceBeforeCodeAfterWord`, `TestPrettyXML_SpaceBetweenInlineElements`.
M cmd/mdcx/verify.go => cmd/mdcx/verify.go +50 -0
@@ 5,6 5,7 @@ import (
"io"
"os"
"regexp"
+ "sort"
"strings"
"github.com/spf13/cobra"
@@ 149,8 150,48 @@ var (
// Markdown emphasis cannot be flanked by whitespace) to one shape.
reEmphLeadWS = regexp.MustCompile(`<(strong|em|del)>([ \t]+)`)
reEmphTrailWS = regexp.MustCompile(`([ \t]+)</(strong|em|del)>`)
+ // reStrikeTag canonicalizes `<s>`/`</s>` to `<del>`/`</del>`. Both are
+ // strikethrough; xml2md emits `~~…~~`, which md2xml renders as <del>, so the
+ // stored `<s>` and the round-tripped `<del>` must compare equal.
+ reStrikeOpen = regexp.MustCompile(`<s>`)
+ reStrikeClose = regexp.MustCompile(`</s>`)
+ // reEmptyEmphasis drops an emphasis tag whose entire body is whitespace
+ // (including nbsp) — Confluence stores stray bold/italic applied to a lone
+ // space, which carries no content and round-trips as bare whitespace.
+ reEmptyStrong = regexp.MustCompile(`<strong>([ \t\x{00a0}]*)</strong>`)
+ reEmptyEm = regexp.MustCompile(`<em>([ \t\x{00a0}]*)</em>`)
+ reEmptyDel = regexp.MustCompile(`<del>([ \t\x{00a0}]*)</del>`)
+ // reStructMacroOpen matches a `<ac:structured-macro …>` opening tag (self-
+ // closing or not). Confluence does not store its attributes in a fixed order
+ // — different pages emit a TOC macro as `macro-id,name,schema-version` or
+ // `name,schema-version,macro-id` — so the round-trip can't reproduce the
+ // original order. sortMacroAttrs canonicalizes it on both sides instead.
+ reStructMacroOpen = regexp.MustCompile(`<ac:structured-macro\s+[^>]*?\s*/?>`)
+ reTagAttr = regexp.MustCompile(`[\w:-]+="[^"]*"`)
)
+// sortMacroAttrs rewrites every <ac:structured-macro …> opener with its
+// attributes sorted, so attribute order never affects the verify comparison.
+func sortMacroAttrs(xml string) string {
+ return reStructMacroOpen.ReplaceAllStringFunc(xml, func(tag string) string {
+ selfClose := strings.HasSuffix(tag, "/>")
+ attrs := reTagAttr.FindAllString(tag, -1)
+ sort.Strings(attrs)
+ var b strings.Builder
+ b.WriteString("<ac:structured-macro")
+ for _, a := range attrs {
+ b.WriteString(" ")
+ b.WriteString(a)
+ }
+ if selfClose {
+ b.WriteString("/>")
+ } else {
+ b.WriteString(">")
+ }
+ return b.String()
+ })
+}
+
// normalizeForVerify strips XML patterns that cannot survive a round-trip
// through Markdown, so verify compares only what the converter can preserve.
// This is applied to BOTH the original input and the round-trip output so that
@@ 190,6 231,15 @@ func normalizeForVerify(xml string) string {
xml = reListStyleAttr.ReplaceAllString(xml, "<$1>")
// Strip trailing whitespace before a task-body close.
xml = reTrailWSTaskBody.ReplaceAllString(xml, "$1")
+ // Canonicalize <ac:structured-macro> attribute order.
+ xml = sortMacroAttrs(xml)
+ // Canonicalize <s> strikethrough to <del>.
+ xml = reStrikeOpen.ReplaceAllString(xml, "<del>")
+ xml = reStrikeClose.ReplaceAllString(xml, "</del>")
+ // Drop emphasis tags whose body is only whitespace.
+ xml = reEmptyStrong.ReplaceAllString(xml, "$1")
+ xml = reEmptyEm.ReplaceAllString(xml, "$1")
+ xml = reEmptyDel.ReplaceAllString(xml, "$1")
// Move whitespace from just inside an emphasis tag to just outside it.
xml = reEmphLeadWS.ReplaceAllString(xml, "$2<$1>")
xml = reEmphTrailWS.ReplaceAllString(xml, "</$2>$1")
M cmd/mdcx/verify_test.go => cmd/mdcx/verify_test.go +19 -0
@@ 208,6 208,25 @@ func TestNormalizeForVerify_EmphasisWhitespace(t *testing.T) {
assert.Equal(t, `a <strong>x</strong> b`, normalizeForVerify(stored))
}
+func TestNormalizeForVerify_MacroAttrOrder(t *testing.T) {
+ // Confluence stores TOC-macro attributes in different orders on different
+ // pages; sorting them on both sides makes order irrelevant.
+ a := `<ac:structured-macro ac:macro-id="X" ac:name="toc" ac:schema-version="1" />`
+ b := `<ac:structured-macro ac:name="toc" ac:schema-version="1" ac:macro-id="X" />`
+ assert.Equal(t, normalizeForVerify(a), normalizeForVerify(b))
+}
+
+func TestNormalizeForVerify_StrikeTag(t *testing.T) {
+ // <s> and <del> are both strikethrough; canonicalize to <del>.
+ assert.Equal(t, `<del>x</del>`, normalizeForVerify(`<s>x</s>`))
+}
+
+func TestNormalizeForVerify_EmptyEmphasis(t *testing.T) {
+ // Whitespace-only emphasis (incl. nbsp) is dropped to its bare content.
+ assert.Equal(t, "a b", normalizeForVerify("a<strong> </strong>b"))
+ assert.Equal(t, "a\u00a0b", normalizeForVerify("a<strong>\u00a0</strong>b"))
+}
+
// --- computeDiffOps ---
func TestComputeDiffOps_IdenticalInputs(t *testing.T) {
M confluence/renderer.go => confluence/renderer.go +24 -14
@@ 300,6 300,19 @@ func (r *Renderer) renderHTMLBlock(w util.BufWriter, source []byte, node ast.Nod
return ast.WalkSkipChildren, nil
}
+// tocMacro builds a TOC structured-macro, emitting attributes in Confluence's
+// stored order (macro-id, name, schema-version) so a round-tripped macro matches
+// the original byte-for-byte under the verify formatter, which preserves order.
+func tocMacro(macroID string) string {
+ var b strings.Builder
+ b.WriteString("<ac:structured-macro")
+ if macroID != "" {
+ b.WriteString(` ac:macro-id="` + macroID + `"`)
+ }
+ b.WriteString(` ac:name="toc" ac:schema-version="1"/>`)
+ return b.String()
+}
+
// convertComment converts preserved HTML comments back to Confluence XML.
func (r *Renderer) convertComment(raw string) (string, bool) {
trimmed := strings.TrimSpace(raw)
@@ 325,23 338,11 @@ func (r *Renderer) convertComment(raw string) (string, bool) {
// TOC macro (in-p variant: wrap in <p> for original <p><toc/></p> shapes)
case strings.HasPrefix(trimmed, "<!-- ac:toc-in-p"):
- macroID := extractCommentAttr(trimmed, "macro-id")
- macro := `<ac:structured-macro ac:name="toc" ac:schema-version="1"`
- if macroID != "" {
- macro += ` ac:macro-id="` + macroID + `"`
- }
- macro += "/>"
- return "<p>" + macro + "</p>", true
+ return "<p>" + tocMacro(extractCommentAttr(trimmed, "macro-id")) + "</p>", true
// TOC macro
case strings.HasPrefix(trimmed, "<!-- ac:toc"):
- macroID := extractCommentAttr(trimmed, "macro-id")
- macro := `<ac:structured-macro ac:name="toc" ac:schema-version="1"`
- if macroID != "" {
- macro += ` ac:macro-id="` + macroID + `"`
- }
- macro += "/>"
- return macro, true
+ return tocMacro(extractCommentAttr(trimmed, "macro-id")), true
// Table attributes — store for next table
case strings.HasPrefix(trimmed, "<!-- table-attrs:"):
@@ 535,6 536,15 @@ func (r *Renderer) renderRawHTML(w util.BufWriter, source []byte, node ast.Node,
for i := 0; i < n.Segments.Len(); i++ {
seg := n.Segments.At(i)
raw := string(seg.Value(source))
+ // A preserved macro marker comment can appear inline, not just at block
+ // level — e.g. a TOC macro that lived inside a heading round-trips as
+ // `# <!-- ac:toc ... -->`, where goldmark parses the comment as inline
+ // RawHTML. Reconstruct the macro here too; otherwise it leaks back out as
+ // a literal comment instead of `<ac:structured-macro ac:name="toc"/>`.
+ if converted, ok := r.convertComment(raw); ok {
+ w.WriteString(converted)
+ continue
+ }
w.WriteString(r.convertRawSpan(raw))
}
return ast.WalkContinue, nil
M converter/md2xml_test.go => converter/md2xml_test.go +28 -0
@@ 690,6 690,34 @@ func TestRoundTrip_EmphasisInteriorSpaces(t *testing.T) {
assert.NotContains(t, xmlOut, "**", "asterisks must not leak as literal text")
}
+// TestRoundTrip_TocMacroInHeading guards against the TOC macro that lives inside
+// a heading round-tripping lossily. xml2md emits it as `# <!-- ac:toc … -->`;
+// goldmark parses that comment as *inline* RawHTML, so md2xml must reconstruct
+// the structured-macro from the inline path (renderRawHTML), not only the block
+// path — otherwise it leaks back out as a literal comment instead of a macro.
+func TestRoundTrip_TocMacroInHeading(t *testing.T) {
+ xmlIn := `<h1><ac:structured-macro ac:macro-id="f9579fa5-2198-4556-92de-04db447accec" ac:name="toc" ac:schema-version="1" /></h1>`
+
+ md, err := ConfluenceToMarkdown(xmlIn)
+ require.NoError(t, err)
+
+ xmlOut, err := MarkdownToConfluence([]byte(md))
+ require.NoError(t, err)
+ assert.Contains(t, xmlOut, `ac:name="toc"`, "TOC macro must be reconstructed, not left as a comment")
+ assert.Contains(t, xmlOut, `ac:macro-id="f9579fa5-2198-4556-92de-04db447accec"`)
+ assert.NotContains(t, xmlOut, "<!-- ac:toc", "comment marker must not leak into the XML")
+}
+
+// TestRoundTrip_EmphasisNbspOnly guards against bold/italic applied to a lone
+// non-breaking space (`<strong> </strong>`, Confluence copy-paste cruft)
+// round-tripping as literal `** **`: nbsp must count as whitespace, so the
+// emphasis markers are dropped entirely.
+func TestRoundTrip_EmphasisNbspOnly(t *testing.T) {
+ md, err := ConfluenceToMarkdown("<p>a<strong>\u00a0</strong>b</p>")
+ require.NoError(t, err)
+ assert.NotContains(t, md, "**", "nbsp-only bold must not emit asterisks")
+}
+
// TestRoundTrip_NestedOrderedList guards against the regression where a
// nested <ol> inside an outer <ol><li> was flattened into a single flat list
// because the markdown was emitted with a 2-space indent (insufficient for
M converter/xml2md.go => converter/xml2md.go +6 -2
@@ 128,9 128,13 @@ func (c *xmlConverter) writeEmphasis(marker string, n *html.Node, depth int) {
inner := c.buf.String()
c.buf = saved
- body := strings.TrimLeft(inner, " \t\r\n")
+ // The cutset includes U+00A0 (nbsp): Confluence stores stray bold applied to
+ // a lone nbsp (`<strong> </strong>`), and treating it as content would
+ // emit `** **`, which is not valid emphasis and leaks as literal `** **`.
+ const emphWS = " \t\r\n\u00a0"
+ body := strings.TrimLeft(inner, emphWS)
lead := inner[:len(inner)-len(body)]
- core := strings.TrimRight(body, " \t\r\n")
+ core := strings.TrimRight(body, emphWS)
trail := body[len(core):]
if core == "" {