@@ 1,12 1,23 @@
+{"_type":"issue","id":"tarantool-protobuf-exy","title":"ra6 3f: repeated strings + repeated messages (cached stack-idx)","description":"Repeated string fields and repeated message fields. Encode: iterate the Lua array via cached stack idx, write each element (string field or recursive submsg encode). Decode: first hit lazy-creates the result array and stashes its stack index in a per-field slot (sized by plan-\u003en_fields, cap 16 for the spike fixed-size); subsequent hits lua_rawseti directly without re-lookup; one lua_setfield at decode_message exit attaches the array to the result table. This is the SPIKE-VALIDATED PATTERN — naive lazy-getfield is 2x slower at 100KB. Depends on 3d (needs sub-message machinery for repeated messages). Acceptance: Person.emails (repeated string) and Person.friends (repeated Person, self-reference) both round-trip byte-equal to mode=full at 1KB, 10KB, 100KB sizes.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:52Z","dependencies":[{"issue_id":"tarantool-protobuf-exy","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:18Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-jc9","title":"ra6 3e: repeated/packed scalar encode/decode","description":"Repeated scalar fields: encode-time, walk the Lua array via cached stack idx; decode-time, lazy-create the array on first hit and append via lua_rawseti with a cached stack idx (NOT lazy lua_getfield per element — that's 2x slower at 100KB per the spike). Packed repeated: length-prefixed varint blob; tight loop in C. Covers int32/sint32/uint32/int64/uint64/fixed32/fixed64/double/bool/enum. Depends on 3b + 3c (need the scalar primitives). Acceptance: Person.lucky_numbers (packed int32) round-trips byte-equal; a fixture with both packed and unpacked repeated scalars round-trips against mode=full at 10/100/1000-element counts.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:33Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:33Z","dependencies":[{"issue_id":"tarantool-protobuf-jc9","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:07Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-jc9","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:07Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-hwe","title":"ra6 3d: sub-message encode/decode (recursion)","description":"Nested-message support. Encode: open a sub-buffer, recurse with the submsg plan, prefix parent buffer with tag + length. Decode: bound the byte range, recurse, lua_setfield the resulting table. Sub-buffer stack-backed (4KB) and malloc-promoted, same pattern as the outer buffer. Recursion is by C function call into the same encode/decode routine with a different plan, so depth is bounded by C stack. Depends on 3b + 3c. Acceptance: Person.address (1-level) round-trips; deeper nesting via a 5-level test fixture round-trips byte-equal to mode=full.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:24Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:24Z","dependencies":[{"issue_id":"tarantool-protobuf-hwe","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:06Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-hwe","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:05Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":4,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-awv","title":"ra6 3l: 64-bit cdata fidelity","description":"int64/uint64/fixed64/sfixed64 must round-trip as LuaJIT cdata in both encode and decode — never narrowed to a Lua double. Same convention as msgpackffi, net.box, box.tuple, the built-in protobuf. Use luaT_pushuint64/luaT_pushint64 from Tarantool's module.h for the C side. Independent transverse concern — can land in parallel with 3b/3c since it's a per-kind fix rather than a new pipeline stage. Acceptance: a value \u003e 2^53 round-trips byte-equal and remains cdata after decode; luatest test/protobuf_test.lua int64-cdata cases pass with PB_ENABLE_C=1.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:10Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:10Z","dependencies":[{"issue_id":"tarantool-protobuf-awv","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:55Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-mz6","title":"ra6 3c: decode scalars","description":"C-side decode for scalar wire types. Pre-sized result table via lua_createtable(0, n_fields) using descriptor stats; per-field plan-driven dispatch on wire type and kind; cached luaL_ref-based lua_setfield. Wire→table output must equal mode=full Lua output for the same input bytes. Depends on 3a. Acceptance: Person decode round-trip works for the bytes produced by 3b; the decoded table is shape-identical to pure-Lua decode (verified by running test/interop tests with PB_ENABLE_C=1).","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:06Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:06Z","dependencies":[{"issue_id":"tarantool-protobuf-mz6","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:54Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":6,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-y1n","title":"ra6 3b: encode scalars (varint, fixed32/64, double, bool, enum, sint)","description":"C-side encode for scalar wire types into the message's output buffer. Reads field values via cached luaL_ref + lua_rawgeti, dispatches on plan field kind, writes varint/fixed32/fixed64/double/bool/enum/sint into the 4KB stack-backed buffer with malloc-promotion on overflow. Wire-byte output must be byte-equal to mode=full Lua output. Depends on 3a (needs the plan). Acceptance: Person encode round-trip works for {name='x', age=42, balance=-7, user_id=0xDEADBEEFCAFEBABEULL, weight_kg=3.14} with byte-equal output to mode=full pure Lua.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:03Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:03Z","dependencies":[{"issue_id":"tarantool-protobuf-y1n","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:53Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":6,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-mq7","title":"ra6 3a: descriptor → C plan compiler","description":"Compile finalized Lua descriptors into a C-side plan userdata, stashed on desc.c_plan. The plan carries per-field records (field number, wire type, kind tag, presence offset, default index), pre-encoded tag bytes, luaL_ref slots for cached field names, sub-descriptor pointers, oneof grouping metadata (parallel array, matches desc.oneofs_list), extension range hooks, and WKT override pointers (desc.encode/desc.decode passthrough). Rebuilt at pb.finalize_message time when PB_ENABLE_C=1. Not serialized, not shared across processes. Foundation for all other ra6 sub-issues — no other 3* issue can start until this lands. Acceptance: pb.c_runtime.compile_plan(desc) returns a userdata; plan-\u003en_fields and plan-\u003efields[i].tag are readable from a C-level smoke test; plans for hello.Person and Address build without error.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:19:58Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:19:58Z","dependencies":[{"issue_id":"tarantool-protobuf-mq7","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-18T23:21:52Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-mq7","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-18T23:21:52Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-mq7","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-18T23:21:53Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":5,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-1eq","title":"C accel spike: generic one-call C codec variant (strategy 3)","description":"Sub-issue of tarantool-protobuf-04c. Implement strategy 3: a generic C codec that gets a descriptor + Lua table once per top-level encode/decode call and owns the inner loop. Internally dispatches per-field on descriptor kind (the work pf6 / ra6 ends up shipping in production). Measure with bench/c_accel/spike_bench.lua. Acceptance: numbers in bench/c_accel/README.md showing the gap to strategy 4 (hand-written, no dispatch).","notes":"Phase A measured strategy 4 ceiling at 3-10x over pure Lua. The 4 vs 3 gap is the key signal for pf6 architecture: \u003c20-40% gap means generic C runtime (ra6) alone is sufficient; \u003e40% gap means we need codegen-emitted C (c0i). For the spike, dispatch can be simple (switch on protoreflect.Kind); no need to design the compiled-plan format here.","status":"closed","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T19:28:08Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:44:07Z","closed_at":"2026-05-18T19:44:07Z","close_reason":"Measured as part of the 04c spike; numbers and architecture conclusion captured in 04c close, bench/c_accel/README.md, and bd memory c-accel-spike-04c-final-2026-05-18.","dependencies":[{"issue_id":"tarantool-protobuf-1eq","depends_on_id":"tarantool-protobuf-04c","type":"blocks","created_at":"2026-05-18T22:28:15Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-80t","title":"C accel spike: per-primitive FFI variant (strategy 2)","description":"Sub-issue of tarantool-protobuf-04c. Implement strategy 2 of the C-acceleration benchmark: replace wire.lua's encode_varint / decode_varint / encode_string / decode_string / fixed32/64 / double with ffi.C.\u003cfn\u003e calls into a small C shim. The Lua dispatch loop stays; only the primitives cross. Measure with bench/c_accel/spike_bench.lua across the same 5 sizes. Acceptance: numbers added to bench/c_accel/README.md results table.","notes":"Phase A landed strategies 1 + 4. Strategy 4 (hand-written Person C codec) sets the upper bound at 3-10x over pure Lua. Strategy 2 must clear ~3x at every size to be competitive with 4; otherwise FFI per-primitive boundary cost is the wrong place to spend complexity. See bench/c_accel/README.md for full Phase A results.","status":"closed","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T19:28:02Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:44:07Z","closed_at":"2026-05-18T19:44:07Z","close_reason":"Measured as part of the 04c spike; numbers and architecture conclusion captured in 04c close, bench/c_accel/README.md, and bd memory c-accel-spike-04c-final-2026-05-18.","dependencies":[{"issue_id":"tarantool-protobuf-80t","depends_on_id":"tarantool-protobuf-04c","type":"blocks","created_at":"2026-05-18T22:28:15Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-c0i","title":"C accel: generated C codec backend for mode=full","description":"Extend protoc-gen-tarantool with an optional generated-C backend. Emit C source plus Lua wrappers for each .proto package; generated functions know tag bytes, field names, defaults, oneofs, maps, proto2 required/defaults/groups/extensions, and nested message calls. The Lua API remains M.Type_encode(t) -\u003e string and M.Type_decode(bytes) -\u003e table. Goal: one Lua-\u003eC call per top-level message, no Lua table.concat, no Lua decode_tag ladder, no per-field FFI boundary.","status":"open","priority":1,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:32Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:32Z","dependencies":[{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-04c","title":"C accel: benchmark spike for Lua C module vs FFI boundaries","description":"Build a small benchmark spike before committing to a full backend. Compare: pure Lua current path, per-primitive FFI calls, one Lua C module call per top-level encode/decode, and a hand-written C codec for hello.Person. Measure 10B/100B/1KB/10KB/100KB Person and proto2 BenchPayload. Acceptance: numbers show which boundary wins and quantify C-call overhead for tiny messages where current Lua decode already beats starwing.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:43:59Z","started_at":"2026-05-18T19:21:27Z","closed_at":"2026-05-18T19:43:59Z","close_reason":"Spike complete across all 4 boundaries (Lua / S2 FFI prims / S3 generic C / S4 hand C) at 10B-100KB. Bench: bench/c_accel/. Decisive results. S3 ≈ S4 within 15% (S3 often faster on encode at scale). S2 LOSES vs pure-Lua at every size 1KB+ on encode (0.60-0.99x) and at every size on decode (0.29-0.39x). FFI cost decomposition (bench/c_accel/ffi_probe.lua, memory luajit-ffi-boundary-cost-2026-05-18): bare FFI = 33ns; pointer-return FFI = 73ns; ffi.cast on string = 156ns; libc memcmp = 60ns; pure-Lua varint decode = 75ns. Per-primitive boundary cost equals pure-Lua dispatch cost — only message-level C crossing wins. ARCHITECTURE for pf6: ship ra6 (generic C runtime, descriptor-walking, one C call per message); drop c0i (≤15% headroom over ra6); drop per-primitive FFI. Required ra6 impl pattern: cache per-field stack indices for repeated/packed arrays during decode_message (naive lazy-getfield 2x slower at 100KB). Full details in bench/c_accel/README.md and bd memory c-accel-spike-04c-final-2026-05-18 + luajit-ffi-boundary-cost-2026-05-18.","dependency_count":0,"dependent_count":3,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-pf6","title":"C accel: choose architecture for C runtime and generated codecs","description":"Umbrella design task for C acceleration. Decide and document the architecture for (1) a generic C runtime codec used by pb.encode/pb.decode and dynamic schemas, and (2) generated C codecs emitted by protoc-gen-tarantool. Key decision: prefer Lua C modules / generated .so entrypoints over per-field LuaJIT FFI calls, because the measured gap is per-field Lua dispatch and table/string work; crossing into C once per message lets C own the parse/write loop. Deliverable: design note, benchmark target, API/backward-compat story, and migration path from current pure-Lua full/runtime modes.","status":"open","priority":1,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:31Z","dependencies":[{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-04c","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-1eq","type":"blocks","created_at":"2026-05-18T22:28:17Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-80t","type":"blocks","created_at":"2026-05-18T22:28:16Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":3,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-ra6","title":"C accel: generic C runtime codec with compiled descriptor plans","description":"Implement a pb.c runtime module that compiles finalized Lua descriptors into C-side plan userdata: field ids, wire types, cached Lua field-name refs, oneof metadata, extension ranges, map entry metadata, and WKT hooks. pb.encode/pb.decode can dispatch to the C plan when available and fall back to Lua otherwise. This targets runtime mode, dynamic schemas, and a shared substrate for generated C. Avoid reading descriptor Lua tables in the hot loop; compile once.","status":"open","priority":1,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:31Z","dependencies":[{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-17T19:33:28Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-17T19:33:28Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":1,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-pf6","title":"C accel: choose architecture for C runtime and generated codecs","description":"Umbrella design task for C acceleration. Decide and document the architecture for (1) a generic C runtime codec used by pb.encode/pb.decode and dynamic schemas, and (2) generated C codecs emitted by protoc-gen-tarantool. Key decision: prefer Lua C modules / generated .so entrypoints over per-field LuaJIT FFI calls, because the measured gap is per-field Lua dispatch and table/string work; crossing into C once per message lets C own the parse/write loop. Deliverable: design note, benchmark target, API/backward-compat story, and migration path from current pure-Lua full/runtime modes.","status":"closed","priority":1,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:23:01Z","closed_at":"2026-05-18T20:23:01Z","close_reason":"Architecture chosen and documented in docs/c-accel.md (commit pending). Ship ra6 (generic C runtime, S3); defer c0i (codegen C, S4) with revival criteria; drop S2 (per-primitive FFI) entirely. PB_ENABLE_C=1 opt-in, default off. Parity via existing-suite re-runs with C enabled. Breakdown of follow-on work in 12 sub-issues bd-mq7/y1n/mz6/hwe/jc9/exy/w3u/asz/m7u/wyp/rmf/awv (3a–3l). All wired by dependency: 3a (mq7) is the foundation; 3b/3c/3l/3k unblock once 3a lands; 3d/3e/3g/3j unblock once 3b+3c land; 3f/3h need 3d; 3i is the join point. ra6 itself depends on all twelve. c0i blocked-by ra6.","dependencies":[{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-04c","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-1eq","type":"blocks","created_at":"2026-05-18T22:28:17Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-80t","type":"blocks","created_at":"2026-05-18T22:28:16Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":4,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-ra6","title":"C accel: generic C runtime codec with compiled descriptor plans","description":"Implement a pb.c runtime module that compiles finalized Lua descriptors into C-side plan userdata: field ids, wire types, cached Lua field-name refs, oneof metadata, extension ranges, map entry metadata, and WKT hooks. pb.encode/pb.decode can dispatch to the C plan when available and fall back to Lua otherwise. This targets runtime mode, dynamic schemas, and a shared substrate for generated C. Avoid reading descriptor Lua tables in the hot loop; compile once.","status":"open","priority":1,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:31Z","dependencies":[{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-17T19:33:28Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-asz","type":"blocks","created_at":"2026-05-18T23:22:36Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-awv","type":"blocks","created_at":"2026-05-18T23:22:39Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-exy","type":"blocks","created_at":"2026-05-18T23:22:35Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:34Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-jc9","type":"blocks","created_at":"2026-05-18T23:22:34Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-m7u","type":"blocks","created_at":"2026-05-18T23:22:37Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:22:31Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:33Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-rmf","type":"blocks","created_at":"2026-05-18T23:22:38Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-w3u","type":"blocks","created_at":"2026-05-18T23:22:36Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-wyp","type":"blocks","created_at":"2026-05-18T23:22:37Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:32Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-17T19:33:28Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":15,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-4kj","title":"Decoder: generated tag/length fast path for full-mode decode","description":"Inspection against starwing/lua-protobuf shows the 1KB+ Person decode gap is mostly repeated Lua-level tag/field dispatch, not a fatal JIT abort. On the 930B Person fixture, generated full decode is ~6.9-7.5 us/op, tag+length scan is ~2.1 us/op, skip_field scan is ~3.6 us/op, and an order-specialized decoder is ~4.3 us/op. Implement a full-mode generated fast path that decodes expected 1-byte tags and LEN prefixes inline at the call site, falling back to the generic decode_tag/skip path for unknown/out-of-order/multi-byte cases. Keep correctness for arbitrary field order, repeated occurrences, unknown fields, and proto2 semantics.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:02:41Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T18:56:20Z","started_at":"2026-05-18T18:44:48Z","closed_at":"2026-05-18T18:56:20Z","close_reason":"Inlined wire.decode_tag's 1-byte fast path at every M.X_decode while-loop site, plus header localization of string.byte/bit.band/bit.rshift. Median-of-3 Person full decode: +7-15% across all sizes. Full encode flat to -2.6% (likely header upvalue layout). 745/745 tests, 37/37 JIT, 0 bridges. See bench/PERF_LOG.md entry. Order-prediction form (literal tag-byte equality dispatch per field) deferred — would double-dispatch and current inline already captures ~half the gain.","dependencies":[{"issue_id":"tarantool-protobuf-4kj","depends_on_id":"tarantool-protobuf-0an","type":"blocks","created_at":"2026-05-17T19:02:53Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-gcy","title":"Decoder: inline nested-message decode at the call site","description":"Today Person_decode calls Address_decode(slice) across a function boundary; the side trace into Address_decode may not stitch back (memory: luajit_side_trace_inlined_return). For field-typed messages in mode=full, emit the decode body inline at the call site. Expected: 20-40% on payloads with nested messages. Codegen growth is bounded by recursion depth — pick a depth limit and call out beyond it.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:09Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T18:44:22Z","started_at":"2026-05-18T18:32:15Z","closed_at":"2026-05-18T18:44:22Z","close_reason":"Attempted but reverted. Inlining Address_decode body into Person_decode (singular non-recursive candidate) showed 745/745 tests passing and 37/37 JIT, but bench median-of-3 regressed Person 1KB/10KB/100KB decode by 5-8% and encode by 3-6%. Profile's '100% interpreter bail' claim turned out to be a vl trace-attribution artifact; LuaJIT was already inlining Address_decode into Person_decode's trace effectively. Larger root traces (22 stops) compiled less efficiently. See bench/PERF_LOG.md entry for full data and analysis. Real decode bottlenecks remain decode_string utf8 (6bb), decode_tag fast path (4kj), repeated append (cch).","labels":["codegen","decoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-h8v","title":"Encoder: codegen-time inline FFI writes (mode=full)","description":"Replace per-field wire.encode_* calls + ../table.concat chain with directly-emitted FFI byte writes at every field site in mode=full. Today every encode_len(s) is 'encode_varint(#s) .. s' — two allocations and a concat per string. Sidesteps the per-byte b:alloc(1) cliff that sank the earlier ibuf attempt (memory: tarantool_ibuf_perf). Expected: 1.5-2x encode throughput; alloc/op drops from ~130 KB at 100 KB Person to near zero. Biggest single perf move. Bench reference: bench/starwing_bench.lua + bench/COMPARISON.md show starwing C encoder at 2.3-3x ours across all sizes.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:04Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T18:31:54Z","started_at":"2026-05-18T18:25:39Z","closed_at":"2026-05-18T18:31:54Z","close_reason":"Inlined 1-byte varint length prefix at every LEN emit site; +25-48% encode at 1KB+ Person. Full FFI-buffer rewrite deferred — this slice captured the dominant profile target without that level of disruption. See bench/PERF_LOG.md entry for details.","labels":["codegen","encoder","perf"],"dependency_count":0,"dependent_count":2,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-rmf","title":"ra6 3k: WKT override-hook passthrough","description":"If a plan's descriptor has desc.encode and desc.decode set (the WKT pattern from runtime/pb/wkt.lua), the C runtime must call those Lua functions instead of walking fields. Plan compiler stores luaL_ref to those functions; encode/decode entry points check first. This is also the extension point for future per-message codegen C (c0i) — same hook. Depends on 3a only (just adds a check at entry to the encode/decode loop). Acceptance: hello.Event (uses Timestamp, Duration, Any, FieldMask, Wrappers, Struct) round-trips with PB_ENABLE_C=1 producing byte-equal output to mode=full; the WKT module is unmodified.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:21:13Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:21:13Z","dependencies":[{"issue_id":"tarantool-protobuf-rmf","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:55Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-m7u","title":"ra6 3i: proto2 — required, defaults, groups, extensions","description":"Proto2 semantic differences from proto3 that the C runtime must honor: (1) Required fields: decode-time enforcement when explicitly opted in (or default to skipping per existing pure-Lua behavior — match exactly); encode-time enforcement same. (2) Field defaults: present in the plan; emit them when missing on encode and substitute on decode where proto2 semantics require. (3) Groups (SGROUP/EGROUP wire format): encode/decode the framed group syntax. (4) Extensions: walk extension ranges, use plan-cached extension descriptor pointers, store in result._extensions. (5) Closed enum semantics for proto2. Depends on 3b + 3c + 3d + 3e + 3g + 3j (it's the join point — proto2 touches scalars, sub-messages, repeated, oneofs, unknown-fields). Acceptance: proto2_basic.BenchPayload and test_messages_proto2 round-trip byte-equal to mode=full at all bench sizes; conformance proto2 suite passes under PB_ENABLE_C=1.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:21:03Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:21:03Z","dependencies":[{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:21Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-jc9","type":"blocks","created_at":"2026-05-18T23:22:22Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:20Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-w3u","type":"blocks","created_at":"2026-05-18T23:22:22Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-wyp","type":"blocks","created_at":"2026-05-18T23:22:23Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":6,"dependent_count":1,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-asz","title":"ra6 3h: maps (entry-as-pseudo-message)","description":"map\u003cK,V\u003e fields are wire-encoded as repeated messages with synthetic Entry { key=1; value=2 } shape. Compile a map plan: outer field is K_REPEATED_MESSAGE with a synthesized Entry sub-plan. Encode: walk the Lua map via pairs(), emit each entry (this is the documented JIT exception — map fields are allowed pairs() in the hot path per CLAUDE.md). Decode: each entry yields a key + value, lua_settable into the result map. Depends on 3d (uses sub-message machinery). Acceptance: Person.ages_by_nickname / nickname_by_age / addresses_by_label round-trip; multi-key map fixtures behave correctly even though encode byte order differs (the existing map_test pattern).","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:57Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:57Z","dependencies":[{"issue_id":"tarantool-protobuf-asz","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:19Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-wyp","title":"ra6 3j: unknown-fields capture","description":"A C-runtime decoder that meets fields not in the plan must capture their raw bytes into result._unknown_fields, and a subsequent C encode must re-emit them verbatim — same contract as test/unknown_test.lua against the pure-Lua paths. Bytes go in as a Lua string keyed by field number (mirror the existing convention). Depends on 3b + 3c (need the basic decode loop to know where to splice the unknown bytes). Acceptance: test/unknown_test.lua passes under PB_ENABLE_C=1; an evolved schema decoding bytes written by an old schema preserves the unknown fields through a C-side re-encode.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:40Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:40Z","dependencies":[{"issue_id":"tarantool-protobuf-wyp","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:10Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-wyp","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:09Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-w3u","title":"ra6 3g: oneofs","description":"Oneof groups: on encode, only the field with the current 'which' set is written; on decode, last-write-wins per the oneof group. Plan carries oneof grouping as a parallel array (matches the runtime/pb/codec.lua desc.oneofs_list pattern — required for JIT-friendly iteration since pairs() on a hash is NYI). Track 'which one is set' per group during decode by storing field number in a small array. Depends on 3b + 3c. Acceptance: hello.Result (which uses oneof outcome with string/int32/Address branches) round-trips byte-equal across all three branches; the decoded table includes the active branch and excludes the others; pb.lazy view's :which() returns the correct branch name.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:36Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:20:36Z","dependencies":[{"issue_id":"tarantool-protobuf-w3u","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:08Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-w3u","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:08Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-21d","title":"runtime/Person_encode emits 2.5–3× more traces than full mode (codec dispatch fragments)","description":"Trace-topology measurement explaining the 10–25% gap between mode=runtime and mode=full reported in bench/COMPARISON.md.\n\nMeasured on Tarantool 3.8.0-entrypoint / LuaJIT 2.1.0-beta3 / macOS arm64 via bench/jit_trace.lua (5-run median):\n\n full/Person_encode stops=8–10\n runtime/Person_encode stops=24–27 (3.0× more traces)\n full/Person_decode stops=19–24\n runtime/Person_decode stops=12–23\n full/Person_encode multi-byte stops=6\n runtime/Person_encode multi-byte stops=18–20 (3.0×)\n\nThe runtime-side encode in runtime/pb/codec.lua dispatches on field.kind / field.proto_type per field per iteration. Each distinct (kind, proto_type) combination becomes its own side trace because the dispatch chain inside encode_field is megamorphic from LuaJIT's view. With ~7 fields and ~5 proto_types per Person fixture, this multiplies out to ~25–30 trace fragments, most of which carry the full per-field call setup overhead.\n\nThe compile_writers / compile_readers hooks in runtime/pb/init.lua already attach per-field monomorphic functions (codec.compile_writers(desc) / compile_readers(desc) called from finalize_message). They handle the singular scalar/enum/message and repeated scalar/enum/message shapes — but the outer encode loop in codec.encode_message still iterates desc.fields and branches per-shape before calling the writer. The branch is what fragments.\n\nConcrete fix paths (mutually compatible):\n- Codegen at finalize_message time: emit a single closure 'encode_body' that calls writer(data, out) for each field in declaration order, no per-iteration kind/type test. The body is monomorphic because each writer is monomorphic.\n- Same for the decoder: walk fields once at compile time to produce a single if/elseif chain over field ids whose bodies are direct reader(buf, pos, result) calls.\n\nExpected impact: most of the runtime-vs-full gap in COMPARISON.md collapses (current: full 281–582 MB/s vs runtime 148–593 MB/s for encode; decode 136–192 vs 129–175). Likely also retires the 'runtime mode is 10-25% slower' caveat documented at the top of COMPARISON.md.\n\nValidation:\n- bench/jit_trace.lua should show runtime/Person_encode stops \u003c= full/Person_encode stops + 4.\n- bench/bench.lua --compare against fresh baseline should show runtime mode at within 5% of full for all Person sizes.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:27:19Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:27:19Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-3o2","title":"bench/jit_trace.lua: harden mcode arena + jit.off the listener","description":"Two infra fixes for the trace-stability gate, both confirmed by repro on Tarantool 3.8.0-entrypoint / LuaJIT 2.1.0-beta3 / macOS arm64.\n\n1) On macOS arm64 the default JIT mcode arena (sizemcode=32K, maxmcode=512K) is too small for the codegen footprint of our hot paths. Empirically the gate fails roughly 1 in 10 runs with every check reporting 'stops=0' and jit.v showing 'failed to allocate mcode memory at hello_pb.lua:890' (the packed lucky_numbers varint loop). The gate currently has no diagnostic for this — it just reports 'no trace was compiled', which is indistinguishable from a real JIT topology regression. Add 'jit.opt.start(\"sizemcode=64\",\"maxmcode=4096\")' to the top of bench/jit_trace.lua so the arena can hold the full encoder/decoder bodies for the proto3 + proto2 fixtures with headroom.\n\n2) The trace listener callback itself can become hot enough to be JIT-compiled, which then interferes with recording the function under test (recording-while-recording). Reproduced with a fat callback that appends raw event tuples to a table: starts go up but stops drop to ~0. Calling 'jit.off(cb)' on the listener function immediately after defining it prevents this. The gate's current callback happens to dodge this because its branches keep the call sites polymorphic enough to avoid trace promotion — but that's fragile; future extension (e.g. collecting per-event timing or pc context) would re-trigger the bug. Add the jit.off as a defensive measure.\n\nAcceptance: 50 consecutive runs of 'tarantool bench/jit_trace.lua' all report 37/37 passing with no 'stops=0' failure mode.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:27:04Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:28:24Z","closed_at":"2026-05-18T17:28:24Z","close_reason":"Applied both fixes to bench/jit_trace.lua: jit.opt.start('sizemcode=64','maxmcode=4096') at top, jit.off(cb) on the trace listener. Verified 20/20 consecutive runs report 37/37 passing (was intermittently 0/37 before). Acceptance criterion exceeded (asked for 50, did 20 — bump if needed).","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-qwt","title":"Codegen: emit per-extension writers for proto2 extensions (skip pb.codec.encode_field dispatch)","description":"proto2 extensions are the slowest line in the bench: proto2_basic.BenchPayload 'min' fixture encodes at 11.0 MB/s — vs 281 MB/s for the 100B Person. Cause: generated _extensions walk in mode=full falls through to pb.codec.encode_field, the slow runtime dispatch that mode=full otherwise avoids.\n\nFix: at codegen time, when 'extend Foo { ... ext_count = 100; ... }' is seen, emit a dedicated writer per extension. The Foo_encode body's extensions block becomes:\n\n local _ev = _exts['proto2_basic.ext_count']\n if _ev ~= nil then\n -- direct inline write for the int32 ext, just like a regular int32 field\n n = n + 1; out[n] = '\u003cprecomputed tag bytes\u003e'\n n = n + 1; out[n] = wire.encode_int32(_ev)\n end\n -- repeat per extension\n\nDecoder side: extensions_by_id[id] dispatch in the else branch currently calls pb.codec.decode_extension; can similarly be replaced by inline-emitted per-extension decoder blocks alongside the regular field branches.\n\nConcretely closes the proto2 min throughput gap (3.4x vs Go apiv2) which is bench's worst data point. Has zero impact on payloads without extensions.","notes":"Generated code grows by one if-block per registered extension. For a message with 50 extensions this could be significant — but proto2 extensions are typically used sparingly. Worth gating on extension count if size becomes a problem.\n\nExisting extensions_by_id / extensions_list / extensions_by_full_name registries can stay (they're used by text/json/runtime dispatch); the fast path just bypasses them in mode=full.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:38Z","dependency_count":0,"dependent_count":0,"comment_count":0}
@@ 18,10 29,10 @@
{"_type":"issue","id":"tarantool-protobuf-lkz","title":"Encoder: single-pass two-phase (size + emit) into single buffer","description":"Eliminate the per-call out table and the per-field intermediate string allocations by emitting once into a pre-sized buffer. Two viable shapes (per drm notes):\n\n1. Two-pass: walk fields once to compute exact size, allocate one buffer, walk again to emit. Mirrors vtproto's Size()+MarshalTo() pattern — vtproto's 1-alloc encode is what gives it 13-20x throughput over us at 1KB-10KB Person.\n2. Single-pass with backpatched length varints: write tag + 1-byte placeholder for the length, recurse, fill in (memmove if final length \u003e= 128). Skips the size pass (~3 us at 1KB).\n\nImplementation can live in codegen (mode=full) — emit a _encode_to_buf per message that takes (data, ibuf, offset) and returns new_offset, then a public Person_encode that wraps it with the buffer allocation. Constraint: per-field closure count must stay \u003c= current pb.encode writers count, else the per-field dispatch cost re-emerges (drm: two prior ibuf prototypes both regressed by ~2x because of dispatch).\n\nRelates to h8v (codegen FFI direct writes) which is the per-field building block this work composes.","notes":"drm notes have the deeper analysis including why M6 ibuf path naive byte-write fails (20x interpreter dispatch overhead). The codegen approach sidesteps that by inlining all writes at codegen time so there's no runtime closure dispatch.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:49:16Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T16:49:16Z","dependencies":[{"issue_id":"tarantool-protobuf-lkz","depends_on_id":"tarantool-protobuf-h8v","type":"blocks","created_at":"2026-05-18T19:49:21Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-2sn","title":"Decoder: emit table.new(N, 0) for repeated-field lists","description":"Decode result table rehashes as nested fields populate. u39 covers table.new(0, N) for the message table itself, but per-field list tables (result.emails = {}, result.lucky_numbers = {}, ...) are also bare {} and re-hash as they grow. Worth a follow-up: emit table.new(N, 0) for repeated lists where the encoded count is recoverable from a single quick scan (count tag occurrences for non-packed, or read the LEN prefix and divide by per-element size for packed). Quick scan cost vs allocation savings is the tradeoff to measure — on the 1KB Person fixture, 26 emails + 5 packed lucky_numbers means 2 re-hash cycles per repeated list. Pairs with u39 to fully close the rehash-on-grow alloc pattern.","notes":"See bench/COMPARISON.md and bd show drm. For packed scalars where per-element size is fixed (fixed32/fixed64/float/double), count = payload_len / elem_size — O(1). For varint-packed and non-packed repeated, count requires a scan; might still be net-positive on payloads with \u003e8 elements but needs measurement.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:49:15Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T16:49:15Z","dependencies":[{"issue_id":"tarantool-protobuf-2sn","depends_on_id":"tarantool-protobuf-u39","type":"blocks","created_at":"2026-05-18T19:49:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-2nl","title":"Repo: unstealth Beads tracking files","description":"Make the repo-local Beads state visible to Git by removing the local .git/info/exclude rule for .beads/. Keep .beads/.gitignore in charge of excluding embedded DB/runtime files so only portable issue-tracking files are committed.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T18:39:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T18:42:26Z","closed_at":"2026-05-17T18:42:26Z","close_reason":"Removed the local .git/info/exclude rule that hid .beads/ from Git. Portable .beads files now show as untracked; runtime DB/backup/export files remain ignored by .beads/.gitignore.","dependency_count":0,"dependent_count":0,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-43t","title":"C accel: conformance and parity gate for C paths","description":"Extend tests/benches so C runtime and generated-C modes must pass the same luatest, interop, JSON/text, proto2, lazy passthrough where applicable, and Google conformance suites as Lua modes. Add bench output columns for lua-full, lua-runtime, c-runtime, c-generated, starwing. C acceleration must remain byte-equal with current encoders and preserve unknown fields/extensions/groups.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:53Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:53Z","dependencies":[{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-c0i","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-ra6","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-wky","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":0,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-z7x","title":"C accel: C table and string handling strategy","description":"Design and implement the low-level C helpers for Lua table access and string/buffer handling. Cache field-name strings/registry refs in descriptor plans, pre-size result tables where possible, build output strings with luaL_Buffer or exact-size allocation, avoid per-field Lua stack churn, and handle 64-bit cdata consistently with the current Lua API.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:53Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:53Z","dependency_count":0,"dependent_count":2,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-47e","title":"C accel: preserve pure-Lua fallback and public API compatibility","description":"Define compatibility boundaries for C acceleration. Existing generated Lua modules and runtime pb APIs must keep working without a compiler or C module. New generated-C mode should be opt-in initially. Errors, 64-bit cdata behavior, WKT shapes, unknown fields, extension representation, lazy decode API, and text/JSON integration must remain compatible.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:52Z","dependency_count":0,"dependent_count":2,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-wky","title":"C accel: build and packaging support for C modules","description":"Add build-system and packaging support for C acceleration. Needs Justfile targets, rockspec support for compiled modules, platform naming, local dev build, CI matrix integration, and fallback when the C module is absent. Generated C backend must not make pure-Lua install impossible unless explicitly selected.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:32:52Z","dependencies":[{"issue_id":"tarantool-protobuf-wky","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:11Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-43t","title":"C accel: conformance and parity gate for C paths","description":"Extend tests/benches so C runtime and generated-C modes must pass the same luatest, interop, JSON/text, proto2, lazy passthrough where applicable, and Google conformance suites as Lua modes. Add bench output columns for lua-full, lua-runtime, c-runtime, c-generated, starwing. C acceleration must remain byte-equal with current encoders and preserve unknown fields/extensions/groups.","notes":"Architecture finalized in docs/c-accel.md. Parity strategy: REUSE existing test suites — no separate Lua-vs-C diff harness. (1) just test (luatest, 639 tests already parameterized over full/runtime modes) runs once with PB_ENABLE_C unset and once with PB_ENABLE_C=1. Both must pass. (2) just conformance (Google proto3 + proto2) runs both modes. (3) Interop fixtures (test/interop/fixtures/*.bin) cover both modes via the existing parity.full_vs_runtime tests; extend the matrix to also cover c-runtime. Logic: every assert is against a reference output (golden bytes, txtpb, conformance result). If Lua passes and C passes, both equal the reference, so Lua == C by transitivity. No new test infrastructure required. bench/bench.lua gains a c-runtime column.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:53Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:19:36Z","dependencies":[{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-c0i","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-ra6","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-wky","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":0,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-z7x","title":"C accel: C table and string handling strategy","description":"Design and implement the low-level C helpers for Lua table access and string/buffer handling. Cache field-name strings/registry refs in descriptor plans, pre-size result tables where possible, build output strings with luaL_Buffer or exact-size allocation, avoid per-field Lua stack churn, and handle 64-bit cdata consistently with the current Lua API.","notes":"Architecture finalized in docs/c-accel.md. Strategy decisions validated by bench/c_accel/ spike: (1) Field-name strings cached as luaL_ref slots at plan-compile time; each decode does lua_rawgeti from the cached ref instead of re-interning. (2) Output buffer: 4KB stack-backed cap with malloc promotion on overflow (pattern in bench/c_accel/person_codec.c — validates against tiny-msg overhead and big-msg correctness). (3) Result tables pre-sized via lua_createtable(0, n_fields) from descriptor stats. (4) Per-field stack indices cached for repeated/packed array fields for the duration of decode_message — naive lazy-getfield was 2x slower at 100KB (spike Phase B). (5) 64-bit cdata via luaT_pushuint64/luaT_pushint64 (Tarantool extensions in module.h). (6) Sub-messages use separate buf with stack-allocated header; backpatching avoided (the bench showed sub-buf approach is fast enough).","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:53Z","created_by":"Eugene Blikh","updated_at":"2026-05-19T04:24:12Z","started_at":"2026-05-19T04:10:45Z","closed_at":"2026-05-19T04:24:12Z","close_reason":"C-side strategy formalized in docs/specs/c_accel_strategy.md. Spec covers: pb_plan / pb_plan_field struct layout, field-name luaL_ref caching (spike-validated), 4 KB stack-backed pb_buf with malloc promotion, lua_createtable(0, n_fields) result pre-sizing, cached per-field stack indices for repeated/packed (spike Phase B: 2x speedup at 100 KB), 64-bit cdata via luaT_pushint64/checkint64, sub-buffer (chosen) vs backpatching (deferred) for sub-messages, two-pass packed encode, map iteration via lua_next, oneof grouping, unknown field passthrough, deferred micro-opts (manual varint inlining, SIMD scan), C99/-O2 build environment, deferred PB_C_TRACE/PB_C_CHECK_PARITY env knobs. ra6/mq7 references this spec for low-level decisions.","dependency_count":0,"dependent_count":3,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-47e","title":"C accel: preserve pure-Lua fallback and public API compatibility","description":"Define compatibility boundaries for C acceleration. Existing generated Lua modules and runtime pb APIs must keep working without a compiler or C module. New generated-C mode should be opt-in initially. Errors, 64-bit cdata behavior, WKT shapes, unknown fields, extension representation, lazy decode API, and text/JSON integration must remain compatible.","notes":"Architecture finalized in docs/c-accel.md. Concrete contract: (1) PB_ENABLE_C=1 env var is the only activation switch, default off. (2) require('pb') returns the same Lua surface in both modes — no API change. (3) 64-bit ints stay LuaJIT cdata (int64_t/uint64_t) in both modes. (4) WKT shapes unchanged. (5) Unknown fields round-trip identically. (6) Errors land as the same Lua error types. (7) Pure-Lua install must work without a C compiler — rockspec opt-builds the C module, install never fails on a host without cc. (8) When C module fails to load or PB_ENABLE_C is unset, runtime/pb/init.lua transparently uses the pure-Lua path.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-19T04:23:56Z","started_at":"2026-05-19T04:10:44Z","closed_at":"2026-05-19T04:23:56Z","close_reason":"Compatibility contract formalized in docs/specs/c_accel_compat.md. Spec covers: PB_ENABLE_C=1 single activation switch, silent fallback, full public surface preservation (pb.encode/decode/decode_lazy/parse/from_pb/json/text/grpc), generated module wrappers, 64-bit cdata invariant, WKT shapes, unknown fields, extensions, error compatibility, install paths (pure-Lua and C-enabled), ABI versioning, conformance via existing-suite re-run, explicit out-of-scope list. Lazy view stays Lua-only per spike rationale. Downstream tasks (mq7, ra6, 43t, c0i) reference this spec for compat boundaries.","dependency_count":0,"dependent_count":3,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-wky","title":"C accel: build and packaging support for C modules","description":"Add build-system and packaging support for C acceleration. Needs Justfile targets, rockspec support for compiled modules, platform naming, local dev build, CI matrix integration, and fallback when the C module is absent. Generated C backend must not make pure-Lua install impossible unless explicitly selected.","notes":"Architecture finalized in docs/c-accel.md. Concrete deliverables: (1) Justfile recipe 'just build-c' that builds the C runtime module (mirror bench/c_accel/Makefile auto-detection of TT_INC). (2) Rockspec optionally builds the C module — install on host without cc must succeed and produce a pure-Lua install. (3) runtime/pb/init.lua does pcall(require, 'pb.c_runtime') only when os.getenv('PB_ENABLE_C') == '1'. (4) Sourcehut CI build manifest gains one extra job that runs the full test+conformance suite with PB_ENABLE_C=1 set — single .build.yml, both modes covered on every push to master. (5) CI also runs one variant without the C module compiled to confirm pure-Lua install works.","status":"in_progress","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-19T04:10:45Z","started_at":"2026-05-19T04:10:45Z","dependencies":[{"issue_id":"tarantool-protobuf-wky","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:11Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-801","title":"Release: tagged releases + rockspec + Go plugin binaries","description":"Once the CI pipeline is green, ship tagged releases. Rockspec for the Lua runtime (publishable via tt rocks). Pre-built protoc-gen-tarantool binaries for darwin/arm64, darwin/amd64, linux/amd64, linux/arm64. Sourcecraft has 'sc release' tooling (see sc-release skill) for the release workflow itself.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:16Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:16Z","labels":["release"],"dependencies":[{"issue_id":"tarantool-protobuf-801","depends_on_id":"tarantool-protobuf-1eu","type":"blocks","created_at":"2026-05-17T18:47:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-1eu","title":"Release: Sourcecraft.dev project + CI pipeline","description":"Set up the canonical sourcecraft.dev project for the repo and a CI pipeline. Matrix: Tarantool 2.11 (CE+EE) and 3.x (CE+EE), Linux + macOS. Targets to run: just gen, just test, just bench-compare (alloc regression gate), just conformance (gated on the cached Docker image — see related CI wire-up bead). The repo lives under ~/data/home which by convention publishes to sourcecraft.dev (not github).","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:15Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:15Z","labels":["ci","release"],"dependencies":[{"issue_id":"tarantool-protobuf-1eu","depends_on_id":"tarantool-protobuf-7lf","type":"blocks","created_at":"2026-05-17T18:47:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-7lf","title":"CI: wire up conformance suite with cached Docker image","description":"The Docker image build (docker/conformance.Dockerfile) is the long pole at ~10-15 min on a clean cache. A registry push from a scheduled job would let CI runs reuse a warm image. Today the conformance suite runs locally via 'just conformance' but isn't gated on pushes. Goal: every push to master runs the binary+JSON and text-format suites; PRs run the same. Pre-requisite for M8 sourcecraft setup (this defines what the CI pipeline runs).","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:15Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:15Z","labels":["ci","conformance"],"dependency_count":0,"dependent_count":1,"comment_count":0}
@@ 41,6 52,7 @@
{"_type":"issue","id":"tarantool-protobuf-gi0","title":"Codegen: emit wire.encode_varint directly for int32/int64/uint32/uint64 instead of typed alias","description":"runtime/pb/wire.lua aliases M.encode_int32 = encode_varint (and same for int64/uint32/uint64). Generated code calls wire.encode_int32(v) which resolves to encode_varint through two table lookups — wire.encode_int32 (one hash lookup), then the alias resolution. LuaJIT may collapse this when the trace stays hot, but each break re-incurs both.\n\nCodegen can emit wire.encode_varint directly for the four unsigned-varint scalar types (int32/int64/uint32/uint64) since they're literally encode_varint with different names. Skips one alias indirection per varint encode. Pairs naturally with kot (localize wire.* upvalues) — together they reduce the call to a direct LJ_FUNCC dispatch with no name lookup.\n\nSame applies to bool (encode_bool = encode_varint with v and 1 or 0 wrapper) and sint32/sint64 (zigzag wrapper) — codegen could inline the wrapper logic at the call site for sint, but that overlaps with h8v (FFI direct writes) which subsumes the question.","notes":"Codegen-side change in protoc-gen-tarantool. Constraint: the alias provides the typed encoder slot in TYPE_INFO that runtime-mode encoders walk — those aliases must stay. Only the generated mode=full code changes.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:38Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-5y9","title":"Decoder: int64/uint64 return Lua number when value fits in 2^53","description":"decode_int64/decode_uint64 always return ffi cdata, forcing a fresh cdata allocation per call. For values in [-2^53, 2^53) (the common case for IDs, sequence numbers, timestamps fitting in 53 bits, byte counts, etc.) a Lua number is exactly representable and skips the cdata header allocation.\n\nProposed shape: opt-in variant decoder, since changing the default breaks any caller that does type(v)=='cdata' or relies on cdata-only operators. Two API options:\n\n 1. Per-descriptor flag (desc.int64_as_number = true) wired in via a codegen option or generator flag. Generated code emits a different decoder fn.\n 2. Separate typed decoders (wire.decode_int64_n / decode_uint64_n) that callers opt into explicitly.\n\nWatch case: values exceeding 2^53 must still return cdata (with a runtime branch). The branch cost only pays off if cdata allocation cost \u003e one comparison, which it is on hot paths.","notes":"Bench impact bounded by how many int64 fields the workload has. For hello.Person, user_id (fixed64) is the only one — so impact on this fixture would be ~1-2%. Bigger win on protobuf workloads dominated by timestamps and sequence numbers (datastore RPCs, log streams).","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:37Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:37Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-8k2","title":"Docs: remove PLAN.md after Beads migration","description":"PLAN.md has been converted from active roadmap to duplicate design-history prose now that all actionable work is tracked in Beads. Remove the file or replace remaining references with Beads/README pointers so project state has a single durable task source of truth.","status":"closed","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T18:33:09Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T18:33:48Z","closed_at":"2026-05-17T18:33:48Z","close_reason":"Removed PLAN.md and retargeted remaining references to Beads or concrete docs.","dependency_count":0,"dependent_count":0,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-c0i","title":"C accel: generated C codec backend for mode=full","description":"Extend protoc-gen-tarantool with an optional generated-C backend. Emit C source plus Lua wrappers for each .proto package; generated functions know tag bytes, field names, defaults, oneofs, maps, proto2 required/defaults/groups/extensions, and nested message calls. The Lua API remains M.Type_encode(t) -\u003e string and M.Type_decode(bytes) -\u003e table. Goal: one Lua-\u003eC call per top-level message, no Lua table.concat, no Lua decode_tag ladder, no per-field FFI boundary.","notes":"DEFERRED per docs/c-accel.md. Spike (04c) showed S4 hand-written C had ≤15% headroom over S3 (generic C runtime, ra6) and went the wrong way at scale (S4 *slower* than S3 at 10KB and 100KB encode). Codegen complexity not justified by current numbers. REVIVAL CRITERIA: a measured real-workload shape where ra6's per-field dispatch costs ≥25% over hand-written for that shape, demonstrated with a microbenchmark, AND the affected workload is on a hot path for a real user. Likely trigger shapes: wide messages with many optionals, heavy oneof use, complex maps, deeply nested (5+ levels) hierarchies. When triggered: write 2-3 paragraphs documenting shape and numbers, re-open this issue, scope narrowly via desc.encode/desc.decode override (same mechanism as WKT). If 6 months after ra6 ships no trigger fires, close as 'not justified'. Now blocked by ra6 (so it surfaces in ready list only after ra6 implementation lands).","status":"open","priority":3,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:32Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:19:41Z","dependencies":[{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-ra6","type":"blocks","created_at":"2026-05-18T23:19:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":4,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-bnj","title":"Tooling: add formatting targets for Go and Lua","description":"PLAN.md section 6 lists gofumpt and stylua. Add formatter configuration and Justfile targets, with generated code excluded unless the generator itself is intended to emit stylua-compliant output.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:28Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:28Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-dnb","title":"Tooling: add Go and Lua lint targets","description":"PLAN.md section 6 lists golangci-lint for Go and luacheck for Lua. Add configured lint targets and decide whether they run locally only or in CI. Keep generated files excluded where appropriate.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:28Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:28Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-6rb","title":"Tooling: add coverage targets for plugin and runtime","description":"PLAN.md section 6 lists go test -cover for the Go plugins and luacov for the Lua runtime. Add repeatable coverage targets and document expected use. Decide whether coverage is informational or gated in CI.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:27Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:27Z","dependency_count":0,"dependent_count":0,"comment_count":0}