~bigbes/confluence-md-utilities

ref: 3e8e97f02c9e39714adc195299d8499a5b5f5169 confluence-md-utilities/CLAUDE.md -rw-r--r-- 11.2 KiB
3e8e97f0 — Eugene Blikh fix: round-trip fidelity for inline TOC macros, nbsp emphasis, and strikethrough a month ago

#CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

#Build & Test

just build          # build binary to ./mdcx
just install        # go install to $GOPATH/bin
just test           # go test ./...
just test-v         # verbose tests
go test ./converter/... -run TestRoundTrip  # run specific tests

Every bug fix gets a regression test. Whenever you fix a bug, add a test that reproduces the exact case and fails on the original code (verify it fails before the fix, passes after). This codebase is a web of subtle round-trip traps — see "Round-trip gotchas" below; each was re-broken at least once before a test pinned it.

#Architecture

mdcx converts Markdown ↔ Confluence storage-format XML with round-trip fidelity. The binary lives at cmd/mdcx/.

#Package graph

cmd/mdcx/      CLI (cobra commands, package main)
  ├─ converter  Core bidirectional conversion
  ├─ api        Confluence REST client (pull/push)
  ├─ template   Marker-based embed/extract
  └─ format     XML pretty-printer + ANSI colorizer

converter/
  ├─ md2xml.go  goldmark parser → confluence.Renderer → XML
  └─ xml2md.go  golang.org/x/net/html walker → Markdown

confluence/
  ├─ renderer.go  goldmark NodeRenderer (registered at priority 100)
  └─ elements.go  Macro builders: CodeMacro, InfoPanel, etc.

#Key design decisions

Bidirectional round-trip preservation — Confluence elements with no Markdown equivalent (inline comments, user mentions, attachment images, layout sections) survive round-trips via HTML <span data-*> attributes and <!-- comment --> markers. The md→xml renderer reconstructs proper ac:*/ri:* tags from these.

CDATA preprocessingxml2md.go replaces <![CDATA[...]]> with <cdatacontent> fake elements before parsing, because golang.org/x/net/html doesn't handle CDATA. The reverse direction uses confluence.escapeCDATA() to split ]]> sequences.

Renderer stateconfluence.Renderer tracks taskIDCounter, inTaskBody, inlineCommentDepth, pageLinkDepth, and pending attributes from HTML comments. This state is per-conversion and must remain consistent across the AST walk.

Template markers<!-- MD_CONTENT_START --> / <!-- MD_CONTENT_END --> delimit the editable region in Confluence pages. The push --template flow: fetch page → replace between markers → update with version increment.

Comment sidecar — Confluence comment threads live outside the page body. pull --with-comments calls /rest/api/content/{id}/child/comment (paginated, with children.comment expanded two levels deep) and writes them to <output>.comments.json. Inline comments link back to body markers via the marker_ref UUID, which equals the data-inline-comment="..." attribute the renderer emits. Comment round-trip (push) is not implemented.

#Semantic mapping (non-obvious)

  • Markdown blockquotes → Confluence info panel macro (not <blockquote>)
  • Task lists detected by checkbox presence, rendered as <ac:task-list>
  • Code blocks → <ac:structured-macro ac:name="code"> with CDATA body
  • XML namespaces (ac:, ri:) are string-constructed, not from an XML library

#Round-trip gotchas (with regression tests)

These are subtle traps the round-trip has been bitten by; touching the related code without updating the tests in converter/md2xml_test.go will likely re-break a real document. Each item names the test that pins it.

  • GFM tables and | inside <code> — a literal | in a cell is a column separator, even inside backticks. xml2md.writeTableRow runs every cell through escapeTablePipes which writes \|; the GFM parser strips the backslash at table-parse time before inline parsing, so the pipe survives inside the resulting code span. Tests: TestRoundTrip_TablePipeInCode, TestEscapeTablePipes_NoDoubleEscape.
  • Adjacent <code> merge must not span whitespace<code>a</code><code>b</code> (no gap) merges into one code span on the way to MD; <code>a</code> <code>b</code> (whitespace text node between) must stay as two. isPrevSiblingCode / isNextSiblingCode therefore look at the immediate neighbour only and refuse to skip whitespace text nodes. Test: TestConfluenceToMarkdown_AdjacentCodeMerge (case "whitespace-separated …").
  • Page links have no markdown form<ac:link><ri:page ri:space-key="…" ri:content-title="…"/></ac:link> is preserved as a <span data-page-link="1" …> placeholder. Goldmark cannot parse <ac:link> as raw inline HTML because of the : in the tag name, so a span wrapper is required. The reverse path lives in confluence.Renderer.convertRawSpan; closing </span> consults pageLinkDepth to emit </ac:link> at the right point. Test: TestRoundTrip_PageLink.
  • Nested-list indent depends on the parent list type — CommonMark places content of 1. at column 4 and - at column 3, so a nested item must be indented by ≥ the parent marker's width. xmlConverter.listIndents is a stack of per-level widths (3 for <ol>, 2 for <ul> / task-list); itemIndent() sums all ancestor contributions. Using a fixed 2*(depth-1) collapses nested ordered lists into siblings. Tests: TestRoundTrip_NestedOrderedList, TestRoundTrip_NestedMixedList, TestRoundTrip_DoubleNestedUnorderedList.
  • Tight lists are one stray \n away from loose — a blank line anywhere in a list turns it loose, which makes goldmark wrap every <li> in <p>. ensureSingleNewline() writes \n only when the buffer doesn't already end in one, preventing the double newline that appeared when a <li>'s body ended with a nested list (which itself terminated in \n). Always use it instead of c.buf.WriteString("\n") after walking list-item content.
  • Literal angle-bracket text turns into phantom elements — text stored as &lt;table&gt; (a function signature like box.backup.start(<table>)) decodes to <table> in the text node; emitted raw into Markdown, goldmark parses it as an HTML table and the phantom element swallows the rest of the document. xml2md.escapeMarkdownAngle backslash-escapes a tag-like < (followed by a letter, /, !, ?) in both text paths — the walk TextNode branch and walkChildrenInline (used for <li> bodies, a separate branch that's easy to miss). The matching unescape is in confluence.Renderer.renderText: it strips only \< (goldmark keeps a backslash-escaped run as one Text node with the backslash intact, and this renderer emits the raw segment) — never a blanket util.UnescapePunctuations, which would also eat the deliberate \| table-pipe escaping and re-break base.xml. A non-tag < (a < b, <=) is left alone. Test: TestRoundTrip_LiteralAngleBracketText.
  • Emphasis can't be flanked by whitespace<strong> x </strong> (space just inside the tag) renders as literal ** x ** because CommonMark requires the run to be flanked by non-whitespace. xml2md.writeEmphasis (used for strong/em/del) captures the children into a scratch buffer and moves leading/trailing whitespace outside the markers (**x**). Its trim cutset must include U+00A0 — Confluence stores stray bold applied to a lone nbsp (<strong> </strong>), and an nbsp left as content emits ** **, which round-trips as literal ** **; normalizeForVerify drops the empty emphasis (reEmptyStrong/reEmptyEm/reEmptyDel) and reconciles the surviving space placement (reEmphLeadWS/reEmphTrailWS). Tests: TestRoundTrip_EmphasisInteriorSpaces, TestRoundTrip_EmphasisNbspOnly.
  • A macro marker comment can be inline, not just block — a TOC macro that lived inside a heading (<h1><ac:structured-macro ac:name="toc"/></h1>) round-trips as # <!-- ac:toc … -->, where goldmark parses the comment as inline RawHTML. confluence.Renderer.renderRawHTML must therefore call convertComment (the same reconstruction the block path renderHTMLBlock uses) before falling back to convertRawSpan, or the macro leaks back out as a literal comment. Test: TestRoundTrip_TocMacroInHeading.

These live in the formatter / verify layer rather than the converter, but bite the same way:

  • Some XML carries values Markdown has no slot for<ac:task-id> (Confluence renumbers them every save), <ul style="list-style-type: square;"> bullet styling, and a stray trailing nbsp before </ac:task-body> cannot survive a Markdown round-trip and aren't meaningful content. normalizeForVerify canonicalizes them on both sides (reTaskID zeroes the id, reListStyleAttr drops <ul>/<ol> attributes, reTrailWSTaskBody strips the trailing space — note the whitespace class must include \x{00a0}, the actual byte stored). Tests: TestNormalizeForVerify_TaskID, TestNormalizeForVerify_ListStyleAttr, TestNormalizeForVerify_TaskBodyTrailingWS, TestNormalizeForVerify_EmphasisWhitespace.

  • Structured-macro attribute order isn't canonical — Confluence emits a TOC macro's attributes as macro-id,name,schema-version on one page and name,schema-version,macro-id on another, and the verify formatter preserves order, so no fixed reconstruction order matches everything. normalizeForVerify.sortMacroAttrs sorts the attributes of every <ac:structured-macro …> opener on both sides. Likewise <s><del> (both strikethrough — xml2md emits ~~…~~, md2xml renders <del>) is canonicalized via reStrikeOpen/reStrikeClose. Tests: TestNormalizeForVerify_MacroAttrOrder, TestNormalizeForVerify_StrikeTag, TestNormalizeForVerify_EmptyEmphasis.

  • A "visible char" class in regexes must exclude <, not just whitespaceverify.normalizeForVerify's reBareTextAfterHeading wraps bare text between a heading and a following block in <p>…</p>. Its "≥1 visible char" token must be [^\s<], never \S: \S matches <, so the capture swallows the next tag's opening bracket (e.g. <ac:structured-macro>) and the inserted </p> lands between a code macro's open tag and its body — destroying round-trip for a no-language fenced code block after a heading. Tests: TestNormalizeForVerify_BareTextAfterHeading, TestVerifyRoundTrip_CodeBlockAfterHeading.

  • fmt line-wrapping must keep <code>…</code> whole — breaking a line inside an inline code span injects the continuation indent into the span's text, which xml2md collapses into a spurious space inside the span (` app.manifest.lock`). format.splitAtoms emits an entire <code>…</code> as one unbreakable atom. Significant whitespace between adjacent inline elements (</strong> <code>) is carried on each atom's spaceBefore flag — never via strings.Fields, which drops trailing space. Tests: TestPrettyXML_WrapKeepsCodeSpanWhole, TestPrettyXML_WrapPreservesSpaceBeforeCodeAfterWord, TestPrettyXML_SpaceBetweenInlineElements.

#Verifying real documents

mdcx verify <file.xml> formats the input, round-trips it through xml2md → md2xml, formats the output, and diffs them. The justfile in ~/data/1 runs this against five real Confluence pages — exercise it after any change to converter or renderer code, since the unit tests can miss multi-feature interactions (e.g. a task list inside a <div class="content-wrapper"> inside a table cell).