bench: starwing cross-runtime comparison harness

Adds bench/starwing_bench.lua, a sibling to bench/bench.lua that
runs the same Person and BenchPayload payloads through Starwing's
lua-protobuf (installed as a rock at .rocks/lib/tarantool/pb.so)
instead of our pure-Lua pb runtime. Same payload shapes, same
iteration counts, same measurement loop — so the two outputs
line up column-for-column.

Loading approach: strip runtime/ off package.path so the `pb`
module name resolves to starwing's C module. We use
package.loadlib to grab `luaopen_pb` from the .so and inject
into package.loaded before require('pb') has a chance to hit the
Lua-path loader; starwing's pb.option('int64_as_number') matches
the regime our bench/bench.lua exercises (bench payloads stay in
the Lua number range, avoiding per-call cdata allocation noise).

bench/starwing/{hello.pb,proto2_basic.pb} are the FileDescriptor
binaries starwing's pb.load() consumes; produced by mainline
protoc --descriptor_set_out from the same .proto sources we
generate Lua from.

Mirrors bench/go/'s role for Go (`bench: Go cross-runtime
comparison harness`, 07eef11) but for the in-Tarantool C-module
alternative — answers "how do we compare to the C-module
incumbent on the same host" rather than "how do we compare to
mainline C++/Go on a different host."

bench/jit_trace: harden mcode arena + jit.off the listener

Two infra fixes for the trace-stability gate, both reproduced on
Tarantool 3.8.0-entrypoint / LuaJIT 2.1.0-beta3 / macOS arm64.

1. Default JIT mcode arena (sizemcode=32K, maxmcode=512K) is too
   small for our hot-path codegen footprint. Roughly 1 in 10 runs
   the gate reports every check as 'stops=0' with no diagnostic —
   indistinguishable from a real JIT topology regression. jit.v
   shows 'failed to allocate mcode memory at hello_pb.lua:890'
   (the packed lucky_numbers varint loop). Calling
   jit.opt.start('sizemcode=64','maxmcode=4096') at the top of
   the gate gives the arena enough headroom to hold both encoder
   and decoder bodies for the proto3 + proto2 fixtures.

2. The trace listener callback itself can become hot enough to
   be JIT-compiled, which then races with the function being
   recorded (recording-while-recording). Reproduced with a fat
   cb that appends event tuples to a table: starts go up but
   stops drop to ~0. Calling jit.off(cb) on the listener
   prevents this. The current cb body happens to dodge this
   because its branches keep call sites polymorphic — but it's
   fragile; any future extension (per-event timing, pc context)
   would re-trigger the bug. Adds the jit.off as defense.

Verified 20/20 consecutive runs now report 37/37 passing (was
intermittently 0/37 before). Also drops a now-stale "(PLAN.M6)"
reference from the header comment.

beads-tarantool-protobuf-3o2

docs: retire PLAN.md in favor of Beads issue tracking

Replaces the slow-changing monolithic PLAN.md roadmap with the
Beads tracker, which is now the durable source of truth for
project state. Roadmap-and-task-list functions sit in `bd ready`
and `bd show <id>`; cross-codebase invariants stay in CLAUDE.md.

References to PLAN.md in README.md, docs/index.md, and one test
file comment now point at the bd CLI instead. A trailing stale
reference in bench/jit_trace.lua's header comment is cleaned up
in a follow-up alongside its substantive change.

The PLAN.md file's vision/architecture material is already
captured across README.md, CLAUDE.md, and docs/, so nothing
informational is lost.

Closes the goal of bd-8k2.

bench: C-acceleration spike — measure four Lua↔C boundaries

Adds bench/c_accel/ with hello.Person codecs at four boundary
strategies and a harness comparing them across 10B–100KB:

  S1 pure Lua (full mode) — existing baseline
  S2 prim.c + prim_ffi.lua — per-primitive FFI calls
  S3 generic_codec.c — one C call per message, descriptor-walking
  S4 person_codec.c — hand-written, no dispatch, upper bound

Plus ffi_probe.lua, a microbenchmark decomposing FFI call cost.

Headline numbers (×L = speedup vs pure-Lua full mode):

  ENCODE       S2     S3     S4
  10B        0.99   3.29   3.79
  100B       0.93   3.12   3.46
  1KB        0.82   5.33   4.94
  10KB       0.60   3.53   2.86
  100KB      0.63   3.49   2.85

  DECODE       S2     S3     S4
  10B        0.29   2.59   2.54
  100B       0.33   2.94   2.95
  1KB        0.39   7.22   7.62
  10KB       0.38   9.64  10.23
  100KB      0.37  10.87  11.33

S3 lands within ±15% of S4 at every size and beats it on encode
at 1KB+: the descriptor-walking loop is uniformly branch-predictable;
hand-written has more divergent per-field paths.

S2 loses to pure Lua at every size ≥1KB on encode and at every size
on decode. FFI cost decomposition (ffi_probe.lua):

  bare FFI call into ffi.load lib        33 ns
  pointer-returning FFI                  73 ns
  + v_out[0] read + tonumber            118 ns
  ffi.C.memcmp (for comparison)          60 ns
  ffi.cast(const uint8_t*, str)         156 ns
  ffi.string(p, 32)                      23 ns
  pure-Lua varint decode                 75 ns
  read_varint in tight traced loop       59 ns (floor)

Per-call FFI dispatch into a ffi.load'd lib is ~60–75 ns — the
same order of magnitude as pure-Lua varint decode. The "win" from
going to C only materializes when you cross the boundary ONCE per
message, not per primitive.

C encode plateaus at ~2.5–2.9 GB/s from 1KB upward; bottleneck
moves to Lua table reads and output string allocation. C decode
hits 2.5 GB/s at 100KB, while pure-Lua decode hits a per-byte
cliff (158 k msg/s @ 1KB → 2.3 k @ 100KB).

Architecture implication for pf6: the generic C runtime (ra6) gets
the full perf envelope; codegen-emitted per-message C (c0i) earns
≤15% headroom and goes the wrong way at scale. Per-primitive FFI
is a non-starter.

Required impl pattern in any future ra6: cache per-field stack
indices for repeated/packed arrays for the duration of
decode_message — the naive lazy-getfield variant was 2× slower
than hand-written at 100KB (see bench/c_accel/README.md).

beads-tarantool-protobuf-04c

bench: record u39 (table.new pre-sized result) post-mortem

Attempted emitting table_new(0, N) for decode result tables. Tests
passed, JIT passed, but small-payload decode (10B, 100B) regressed
15-18% because the table.new upvalue-call cost exceeds any rehash
savings at that scale, and the small case never reaches the first
rehash anyway. Large-payload decode flat. Confirms drm's claim that
alloc shape is not the bottleneck on the bench corpus.

Code change reverted; only the writeup lands.

beads-tarantool-protobuf-u39

codegen: local counter per repeated field on decode

Replaces `list[#list + 1] = val` with `_n_<fname> = _n_<fname> + 1;
list[_n_<fname>] = val` in every generated M.X_decode repeated-field
append site. One counter local per repeated non-map field, declared at
function entry. Counters survive across loop iterations so out-of-order
wire entries for the same field continue past the existing length
without re-scanning.

Profile attributed 6.2% of hello.Person 1KB decode to the `#list + 1`
re-traversal (26-email Person paid 26 list scans per decode).

Bench (Person full decode, msgs/s, median-of-3 vs post-4kj):
1KB +5.0%, 10KB +5.0%, 100KB +7.1%. Encode flat. Tests 745/745.
JIT 37/37, 0 bridges.

beads-tarantool-protobuf-cch

codegen: inline 1-byte tag fast path at decode call sites

Hoists wire.decode_tag's 1-byte fast path into every generated
M.X_decode while-loop, falling back to the helper for multi-byte
tags (field ids > 15). The 1-byte case covers every protobuf field
with id 1..15 and is the dominant decode dispatch in real payloads.

Header now localizes string.byte, bit.band, and bit.rshift so the
inlined ops compile to straight local calls.

Bench (Person full decode, msgs/s, median-of-3 vs proper post-h8v
3-run baseline): 10B +14.7%, 100B +8.9%, 1KB +7.5%, 10KB +7.0%,
100KB +8.5%. Full encode is flat to small (-0.1% to -2.6%) at large
sizes, plausibly from header-upvalue layout. JIT trace gate: 37/37,
all bridges still 0. Tests: 745/745.

Also documented in bench/PERF_LOG.md, including the methodology
note that h8v's earlier numbers used single-run baselines and are
therefore ~3-5% optimistic; medians-of-3 are the standard now.

beads-tarantool-protobuf-4kj

bench: record gcy (inline nested-message decode) post-mortem

Implemented and benchmarked the inline-nested-decode plan from
tarantool-protobuf-gcy. Test suite and JIT gate both pass, but
median-of-3 bench shows 3-8% regressions on Person 1KB/10KB/100KB
encode AND decode. Profile's "100% interpreter bail at Address_decode
call" turned out to be a vl trace-attribution artifact; LuaJIT was
already handling the call well.

Lesson recorded in bench/PERF_LOG.md so the next person who reads the
profile entry knows the obvious-looking inline transformation does
not deliver here. Code change reverted; only the writeup lands.

beads-tarantool-protobuf-gcy

codegen: inline 1-byte varint length prefix at every LEN emit site

Eliminates the wire.encode_varint(#body) call + dispatch for every
length-delimited field in mode=full codegen. Profile flagged the
out[n] = wire.encode_varint(#_b) line as ~33% of hello.Person 1KB
encode time, with another ~17% in encode_varint dispatch — together
~50% of encode time. Lifting the 1-byte fast path (the dominant case
for proto strings and small message bodies) to the call site removes
the function frame entirely for lengths < 128.

Applied at every LEN emit site: singular/repeated message body,
singular/repeated string|bytes, packed scalar bundle, packed enum
bundle, map entry. Map value pieces (emitMapPiece message branch)
left as-is — they sit inside a single slot assignment that would
require a deeper restructure, and maps are not on the current hot
benchmark.

Results (hello.Person full encode, msgs/s): 10B +6.9%, 100B +7.9%,
1KB +25.7%, 10KB +48.1%, 100KB +31.9%. Decode flat (unchanged path).
Runtime mode flat (descriptor dispatch still calls encode_varint).
JIT trace gate: 37/37. Test suite: 745/745.

Bench history saved to bench/PERF_LOG.md with full numbers and the
workflow this iteration follows.

beads-tarantool-protobuf-h8v

bench: Go cross-runtime comparison harness

Adds bench/go/ — single-threaded Go benchmarks against the same proto
schemas and payload sizes as bench/bench.lua, run against
google.golang.org/protobuf v1.36 (reflective apiv2) and
planetscale/vtprotobuf v0.6 (codegen marshalers). Wired through
`just gen-go` + `just bench-go`. *.pb.go is gitignored repo-wide so
the generated outputs are regenerated locally — not committed.

bench/COMPARISON.md documents the full table per fixture / size / op:
MB/s for Lua full + Lua runtime + apiv2 + vtproto, alloc bytes/op and
allocs/op side-by-side, and ratios. proto2 BenchPayload vtproto cases
are intentionally skipped — MarshalVT drops proto2 extensions and
would understate bytes vs apiv2 / Lua.

bench,codec: pin proto2 paths, kill pairs() on extension hot path

JIT trace gate gains 12 proto2-specific checks (required encode/decode,
groups singular + repeated, extension encode/decode). The extension-encode
check fired NYI on bytecode 72 (ISNEXT) — pairs() over the new
extensions_by_full_name hash is the same trace-abort that gates map
encode. Fix: register_extension now also appends to extensions_list,
an array view. codec.encode_message, text.emit_message and
json.encode_message all switch to ipairs over the list. Hash tables
stay around for O(1) lookups in decode (extensions_by_id) and bracket-
name resolution (extensions_by_full_name).

Inline (full-mode) codegen learns to walk extensions too: before this
commit the inline encoder skipped t._extensions entirely (only the
runtime codec walked it), so a generated _encode silently dropped any
extension data set on the message. Add the array-walk after the field
loop and a decode_extension dispatch in the unknown-tag branch — the
inline path now matches runtime byte-for-byte.

bench/bench.lua parameterizes over a FIXTURES list so the baseline can
cover both hello.Person and a new proto2_basic.BenchPayload fixture
(required + group + extension + repeated). baseline.json restructured
under "schemas": [{schema, results}, …]; compare walks both. Fresh
numbers committed.

740/740 luatest cases pass; JIT gate 37/37; conformance 2806 binary+JSON
and 434 text-format, both 0 failures.

docs: 100% proto2 + proto3 conformance

README status table: replace the four \"deferred\" rows with
explicit checkmarks for groups, extensions, closed enums; bump
both conformance lines to 2806 and 434 successes. Note MessageSet
as the only known gap (protoreflect rejects upstream, our vendored
proto2 schema has those four nested messages stripped).

docs/codegen.md proto2-support section gets three subsections —
Groups, Extensions, Closed enums — each describing the runtime
surface a user touches (decode_group, _extensions, the closed flag).
The descriptor-table example includes a group field and the new
extensions_by_id / extensions_by_full_name indices.

PLAN.md M9 closes with the conformance numbers; M5 header updated
to reflect that proto2 is no longer deferred.

codec,text,json: proto2 extensions end-to-end

Plugin iterates file.Extensions and per-message Extensions, emitting
\`pb.register_extension(extendee_desc, {…})\` calls. Each extension
becomes a field descriptor attached to the extendee under two
indices: extensions_by_id (wire-decode lookup) and
extensions_by_full_name (encode + textual emission ordering).

Codec encode_message walks data._extensions after the regular field
loop and calls encode_field through the extension's descriptor.
decode_message routes an unknown tag through decode_extension when
extensions_by_id matches; the helper mirrors the in-line dispatch
on field kind (scalar/enum/message/group, singular/repeated) and
stores into result._extensions[full_name].

Text decoder recognizes bracket syntax \`[pkg.ext_name]\` outside an
Any-target and looks the name up in extensions_by_full_name. Names
that don't resolve (e.g. \`[pkg.GroupField]\` for the CapitalCase
group type instead of the lowercase field name) raise a parse error
per spec. Text encoder emits each set extension as
\`[full.name]: value\` or \`[full.name] { … }\`.

JSON encoder emits set extensions under the bracketed full-name key
(\`"[pkg.ext_name]": value\`). JSON decoder recognizes the same
shape and stores into _extensions; unknown bracket-names stay
silently dropped to match the spec's tolerant behavior for unknown
JSON keys.

Extensions on google.protobuf.* descriptors (file/message/field
option extends) are skipped — those are meta-only and our WKT
module doesn't surface their descriptors.

Conformance baseline (strict --enforce_recommended, protobuf v34.1):
  Binary + JSON suite: 2806 ✓ / 0 failures   (was 1493 ✓ / 1313 skipped)
  Text-format suite:    434 ✓ / 0 failures   (was  416 ✓ /   18 skipped)

100% pass.

text,codec: proto2 group text syntax + closed-enum semantics

Text decoder now resolves a group field reference by the group's
capitalized submessage name (e.g. \`Data\`, \`MultiWordGroupField\`)
and by the ASCII-lowercase fold (\`data\`, \`multiwordgroupfield\`),
in addition to the existing lowercase field-name lookup. The \`:\`
between field and \`{\` is optional for groups (matches the
message/map rule).

Proto2 enums are closed: parse_enum_value now rejects an integer
literal that doesn't map to any declared value. The plugin emits
\`closed = true\` on every proto2 enum descriptor (driven by
protoreflect's IsClosed()); proto3 enums stay open to preserve
forward-compatibility on the wire.

Conformance text-format suite: 16 unexpected failures → 3, all in
the remaining extension-bracketed-group cases.

conformance: dispatch proto2 TestAllTypesProto2

Register the proto2 test message in MESSAGE_REGISTRY and the
Any/JSON registry. The harness now picks up 1312 previously-skipped
binary + JSON cases; whole suite jumps to 2805 ✓ / 0 failures (was
1493 ✓ with proto2 fully skipped).

Text-format suite still has 16 unexpected failures, all bound up
in proto2 group-field text syntax (CamelCase submessage label, no
mandatory `:` before `{`, extension-bracketed group form). Tackled
in the next commit.

The conformance unit test that pinned the proto2-skipped behavior
now targets an editions test type instead so the skip-path still
has a hard assertion.

codec: proto2 groups (SGROUP/EGROUP wire format)

Plugin: detect protoreflect.GroupKind, emit kind='group' in the
field descriptor and a SGROUP opening tag in inline codegen. The
group body bypasses the LEN-prefix path entirely — encoder emits
start-tag, body bytes, end-tag in three slots; decoder calls
pb.codec.decode_group(desc, buf, pos, field_id) which reads inner
tags until the matching EGROUP id.

Wire layer already understood SGROUP/EGROUP for skip_field; the new
decode_group reuses the per-tag dispatch from decode_message but
stops on EGROUP instead of end-of-buffer. Repeated groups bracket
each element with its own SGROUP/EGROUP pair.

Runtime parser now desugars `optional|required|repeated group Name
= id { body }` into (a) a nested message named Name and (b) a
synthetic field of kind=group whose lowercased name is `name` — so
source-parsed proto2 schemas behave the same as build-time codegen.

Text format renders groups under the submessage's capitalized name
(`SingleGroup { ... }`) rather than the lowercase field name,
matching mainline protoc's convention.

Adds a vendored, MessageSet-stripped test_messages_proto2.proto in
test/conformance/proto/ (header comment documents the patch).
10 new luatest cases (group_*) cover singular + repeated + text +
descriptor kind, across both codegen modes. 739 tests pass.

docs: proto2 is supported for the core surface

README's status table swaps the single \"proto2 deferred\" row for
four entries that name what works (required / optional / custom
defaults / packed semantics) and what doesn't (extend, extensions,
group, MessageSet) so users can decide before they generate.

docs/codegen.md replaces the proto2 deferral section with a
\"Proto2 support\" page: the three new descriptor attributes
(required, default_value, optional everywhere), the existing
IsPacked() trick that already produces the right unpacked default,
and a concrete list of the four features that still refuse to
generate (with the MessageSet upstream-rejection caveat).

PLAN.md gains M9 — Proto2 baseline. Wire format, codecs, parser,
dynamic, tests are checked off; extend / group / MessageSet stay
open because the conformance schema needs all three and they're
each their own slice.

json,text: presence-tracked fields skip proto3 default elision

Both codecs gated default-value elision on `f.optional or f.oneof` —
the proto3 implicit-presence shape. That elides proto2 `required`
fields set to zero and any other presence-tracked descriptor that
doesn't carry the proto3 explicit-optional flag.

Extend the bypass to `f.required` so a Cardinality{r=0} survives
the round-trip through pb.json.encode / pb.text.encode. 4 new
luatest cases pin the rule.

runtime: parser + dynamic accept proto2 sources

Parser now captures `required=true` and `default_value=…` from the
proto2 keywords (instead of dropping `required` silently and ignoring
field options). Dynamic descriptor builder reads `parsed.syntax`,
flips the repeated-scalar packing default for proto2, and calls
`codec.compile_writers/compile_readers` so the per-field
required-writer specialization actually fires — without that the
generic encode_field path silently elides missing required fields.

64-bit-int defaults are coerced into the appropriate cdata type
inside `coerce_default` so the codec's value-comparison rules
match what generated code emits.

Adds 7 dynamic-mode luatest cases including a static-vs-dynamic
byte-parity check. 725 tests pass.

codegen: proto2 baseline — required, optional, custom defaults

Lifts the proto3-only syntax gate in the plugin and threads three
new field-descriptor attributes through codegen and the codec:

  * required=true   — fields declared with the proto2 `required` keyword.
                      Inline codegen and the runtime codec both error when
                      a required field is missing on encode (vs the silent
                      elide that proto3 implicit-presence fields get).
  * optional=true   — already wired for proto3 explicit `optional`; in
                      proto2 every singular field carries it via the
                      existing HasOptionalKeyword() check, giving presence
                      semantics without a separate emission path.
  * default_value=… — proto2 [default = X] from the field descriptor,
                      rendered as a Lua literal (cdata for 64-bit ints,
                      symbolic name for enums) so consumers can surface
                      it; the codec itself does not auto-materialize
                      defaults on decode, matching how proto3 absent
                      fields stay nil.

Packed-by-default already flips correctly because we ask
protoreflect's `IsPacked()`, which is syntax-aware.

Adds test/proto/proto2_basic.proto with 33 luatest cases covering
required validation, optional presence, custom defaults, the proto2
unpacked-by-default repeated rule, nested-required messages, and
full-vs-runtime mode parity. `just gen-proto2-tests` regenerates the
fixture into examples/expected/{full,runtime}/.

Out of scope: extensions, extend, group; conformance harness still
skips TestAllTypesProto2.