~bigbes/confluence-md-utilities

ref: 3c0272ac2232ee3a0edd0c259b71e9fd3384f211 confluence-md-utilities/CLAUDE.md -rw-r--r-- 9.8 KiB
3c0272ac — Eugene Blikh fix: round-trip fidelity for heading comments, literal tags, and emphasis a month ago

#CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

#Build & Test

just build          # build binary to ./mdcx
just install        # go install to $GOPATH/bin
just test           # go test ./...
just test-v         # verbose tests
go test ./converter/... -run TestRoundTrip  # run specific tests

Every bug fix gets a regression test. Whenever you fix a bug, add a test that reproduces the exact case and fails on the original code (verify it fails before the fix, passes after). This codebase is a web of subtle round-trip traps — see "Round-trip gotchas" below; each was re-broken at least once before a test pinned it.

#Architecture

mdcx converts Markdown ↔ Confluence storage-format XML with round-trip fidelity. The binary lives at cmd/mdcx/.

#Package graph

cmd/mdcx/      CLI (cobra commands, package main)
  ├─ converter  Core bidirectional conversion
  ├─ api        Confluence REST client (pull/push)
  ├─ template   Marker-based embed/extract
  └─ format     XML pretty-printer + ANSI colorizer

converter/
  ├─ md2xml.go  goldmark parser → confluence.Renderer → XML
  └─ xml2md.go  golang.org/x/net/html walker → Markdown

confluence/
  ├─ renderer.go  goldmark NodeRenderer (registered at priority 100)
  └─ elements.go  Macro builders: CodeMacro, InfoPanel, etc.

#Key design decisions

Bidirectional round-trip preservation — Confluence elements with no Markdown equivalent (inline comments, user mentions, attachment images, layout sections) survive round-trips via HTML <span data-*> attributes and <!-- comment --> markers. The md→xml renderer reconstructs proper ac:*/ri:* tags from these.

CDATA preprocessingxml2md.go replaces <![CDATA[...]]> with <cdatacontent> fake elements before parsing, because golang.org/x/net/html doesn't handle CDATA. The reverse direction uses confluence.escapeCDATA() to split ]]> sequences.

Renderer stateconfluence.Renderer tracks taskIDCounter, inTaskBody, inlineCommentDepth, pageLinkDepth, and pending attributes from HTML comments. This state is per-conversion and must remain consistent across the AST walk.

Template markers<!-- MD_CONTENT_START --> / <!-- MD_CONTENT_END --> delimit the editable region in Confluence pages. The push --template flow: fetch page → replace between markers → update with version increment.

Comment sidecar — Confluence comment threads live outside the page body. pull --with-comments calls /rest/api/content/{id}/child/comment (paginated, with children.comment expanded two levels deep) and writes them to <output>.comments.json. Inline comments link back to body markers via the marker_ref UUID, which equals the data-inline-comment="..." attribute the renderer emits. Comment round-trip (push) is not implemented.

#Semantic mapping (non-obvious)

  • Markdown blockquotes → Confluence info panel macro (not <blockquote>)
  • Task lists detected by checkbox presence, rendered as <ac:task-list>
  • Code blocks → <ac:structured-macro ac:name="code"> with CDATA body
  • XML namespaces (ac:, ri:) are string-constructed, not from an XML library

#Round-trip gotchas (with regression tests)

These are subtle traps the round-trip has been bitten by; touching the related code without updating the tests in converter/md2xml_test.go will likely re-break a real document. Each item names the test that pins it.

  • GFM tables and | inside <code> — a literal | in a cell is a column separator, even inside backticks. xml2md.writeTableRow runs every cell through escapeTablePipes which writes \|; the GFM parser strips the backslash at table-parse time before inline parsing, so the pipe survives inside the resulting code span. Tests: TestRoundTrip_TablePipeInCode, TestEscapeTablePipes_NoDoubleEscape.
  • Adjacent <code> merge must not span whitespace<code>a</code><code>b</code> (no gap) merges into one code span on the way to MD; <code>a</code> <code>b</code> (whitespace text node between) must stay as two. isPrevSiblingCode / isNextSiblingCode therefore look at the immediate neighbour only and refuse to skip whitespace text nodes. Test: TestConfluenceToMarkdown_AdjacentCodeMerge (case "whitespace-separated …").
  • Page links have no markdown form<ac:link><ri:page ri:space-key="…" ri:content-title="…"/></ac:link> is preserved as a <span data-page-link="1" …> placeholder. Goldmark cannot parse <ac:link> as raw inline HTML because of the : in the tag name, so a span wrapper is required. The reverse path lives in confluence.Renderer.convertRawSpan; closing </span> consults pageLinkDepth to emit </ac:link> at the right point. Test: TestRoundTrip_PageLink.
  • Nested-list indent depends on the parent list type — CommonMark places content of 1. at column 4 and - at column 3, so a nested item must be indented by ≥ the parent marker's width. xmlConverter.listIndents is a stack of per-level widths (3 for <ol>, 2 for <ul> / task-list); itemIndent() sums all ancestor contributions. Using a fixed 2*(depth-1) collapses nested ordered lists into siblings. Tests: TestRoundTrip_NestedOrderedList, TestRoundTrip_NestedMixedList, TestRoundTrip_DoubleNestedUnorderedList.
  • Tight lists are one stray \n away from loose — a blank line anywhere in a list turns it loose, which makes goldmark wrap every <li> in <p>. ensureSingleNewline() writes \n only when the buffer doesn't already end in one, preventing the double newline that appeared when a <li>'s body ended with a nested list (which itself terminated in \n). Always use it instead of c.buf.WriteString("\n") after walking list-item content.
  • Literal angle-bracket text turns into phantom elements — text stored as &lt;table&gt; (a function signature like box.backup.start(<table>)) decodes to <table> in the text node; emitted raw into Markdown, goldmark parses it as an HTML table and the phantom element swallows the rest of the document. xml2md.escapeMarkdownAngle backslash-escapes a tag-like < (followed by a letter, /, !, ?) in both text paths — the walk TextNode branch and walkChildrenInline (used for <li> bodies, a separate branch that's easy to miss). The matching unescape is in confluence.Renderer.renderText: it strips only \< (goldmark keeps a backslash-escaped run as one Text node with the backslash intact, and this renderer emits the raw segment) — never a blanket util.UnescapePunctuations, which would also eat the deliberate \| table-pipe escaping and re-break base.xml. A non-tag < (a < b, <=) is left alone. Test: TestRoundTrip_LiteralAngleBracketText.
  • Emphasis can't be flanked by whitespace<strong> x </strong> (space just inside the tag) renders as literal ** x ** because CommonMark requires the run to be flanked by non-whitespace. xml2md.writeEmphasis (used for strong/em/del) captures the children into a scratch buffer and moves leading/trailing whitespace outside the markers (**x**). The stored and round-tripped forms then differ only in space placement, which normalizeForVerify reconciles (reEmphLeadWS/reEmphTrailWS). Test: TestRoundTrip_EmphasisInteriorSpaces.

These live in the formatter / verify layer rather than the converter, but bite the same way:

  • Some XML carries values Markdown has no slot for<ac:task-id> (Confluence renumbers them every save), <ul style="list-style-type: square;"> bullet styling, and a stray trailing nbsp before </ac:task-body> cannot survive a Markdown round-trip and aren't meaningful content. normalizeForVerify canonicalizes them on both sides (reTaskID zeroes the id, reListStyleAttr drops <ul>/<ol> attributes, reTrailWSTaskBody strips the trailing space — note the whitespace class must include \x{00a0}, the actual byte stored). Tests: TestNormalizeForVerify_TaskID, TestNormalizeForVerify_ListStyleAttr, TestNormalizeForVerify_TaskBodyTrailingWS, TestNormalizeForVerify_EmphasisWhitespace.

  • A "visible char" class in regexes must exclude <, not just whitespaceverify.normalizeForVerify's reBareTextAfterHeading wraps bare text between a heading and a following block in <p>…</p>. Its "≥1 visible char" token must be [^\s<], never \S: \S matches <, so the capture swallows the next tag's opening bracket (e.g. <ac:structured-macro>) and the inserted </p> lands between a code macro's open tag and its body — destroying round-trip for a no-language fenced code block after a heading. Tests: TestNormalizeForVerify_BareTextAfterHeading, TestVerifyRoundTrip_CodeBlockAfterHeading.

  • fmt line-wrapping must keep <code>…</code> whole — breaking a line inside an inline code span injects the continuation indent into the span's text, which xml2md collapses into a spurious space inside the span (` app.manifest.lock`). format.splitAtoms emits an entire <code>…</code> as one unbreakable atom. Significant whitespace between adjacent inline elements (</strong> <code>) is carried on each atom's spaceBefore flag — never via strings.Fields, which drops trailing space. Tests: TestPrettyXML_WrapKeepsCodeSpanWhole, TestPrettyXML_WrapPreservesSpaceBeforeCodeAfterWord, TestPrettyXML_SpaceBetweenInlineElements.

#Verifying real documents

mdcx verify <file.xml> formats the input, round-trips it through xml2md → md2xml, formats the output, and diffs them. The justfile in ~/data/1 runs this against five real Confluence pages — exercise it after any change to converter or renderer code, since the unit tests can miss multi-feature interactions (e.g. a task list inside a <div class="content-wrapper"> inside a table cell).