{"_type":"issue","id":"tarantool-protobuf-rc8","title":"C decoder: strict-decode parity with pure-Lua codec (wire-type 6/7, UTF-8, field-num, merge)","description":"After bd-43t wiring (init.lua dispatch + codegen lazy-compile prologue), the C codec is actually exercised by `PB_ENABLE_C=1 just test` and `just conformance-c`. 77 conformance failures and 13 luatest failures cluster into 7 categories that the pure-Lua decoder already rejects:\n\n1. UnknownWireType6 / UnknownWireType7 — C decoder accepts illegal wire types instead of erroring (Required.Proto[23].ProtobufInput.UnknownWireType{6,7}_Field{1,2,3}_Version{0..3}, 32 cases).\n2. IllegalZeroFieldNum_Case_{0,1,3} — field number 0 must be rejected.\n3. BadTag_FieldNumberSlightlyTooHigh / FieldNumberTooHigh — field numbers \u003e 2^29-1 must be rejected.\n4. BadTag_OverlongVarint — tag varints over the spec-mandated length must be rejected.\n5. RejectInvalidUtf8.String.{Singular,Repeated,Oneof,MapKey,MapValue} — invalid UTF-8 in string/map-key fields must be rejected.\n6. UnmatchedStartGroup / UnmatchedStartGroupNested (proto2) — group balancing.\n7. ValidDataOneof.MESSAGE.Merge / RepeatedScalarMessageMerge — oneof message-merge semantics and repeated-scalar merge accumulate vs replace.\n\nReference impl: runtime/pb/codec.lua + wire.lua (decode_tag rejects wt 6/7 since commit 7442ea2; utf8 validation via utf8.len; merge_message recurses; etc.). C decoder lives in runtime/pb/c/c_runtime.c.\n\nStatus: pb.encode/pb.decode lazy-dispatch to pb.c_runtime.encode/decode whenever PB_ENABLE_C=1 and a c_plan exists. Wrappers in mode=full prologue + runtime mode via pb.encode central dispatch. Dispatch is proven (in-container probe + 1026/1039 luatest pass after wiring vs 748 with C inert / skip).\n\nRepro: PB_ENABLE_C=1 just test → 13 fails in conformance.core.* group. just conformance-c → 77 unexpected failures in binary suite.\n\nBlocks: tarantool-protobuf-43t conformance gate.","notes":"Fixed in c_runtime.c. Strict-decode parity items implemented:\n- decode_body: reject wire types 6/7, field number 0, field number \u003e 2^29-1, overlong tag varint\n- dec_push_kind PB_KIND_STRING: RFC 3629 UTF-8 validator (port of utf8.len)\n- decode_body: track egroup_seen, error on loop exit if stop_group_id != 0\n- decode_body singular message dispatch + decode_extension_into singular: merge into prev table via merge_subresult_into, recurses for nested message, concat for repeated, last-wins per key for map, skip for WKT custom-decode\n- merge_subresult_into ports codec.lua's merge_message C-side\n\nRegression coverage (test/conformance_test.lua conformance.core group):\n- pre-existing tests covered field-number/wire-type/overlong/UTF-8 singular-repeated-oneof/merge — now exercise C path via PB_ENABLE_C=1\n- added test_invalid_utf8_map_key_rejected, test_invalid_utf8_map_value_rejected\n- added test_unmatched_start_group_rejected, test_unmatched_start_group_nested_rejected (proto2 via PROTO2_NAME)\n\nResults:\n- PB_ENABLE_C=1 just test: 1043/1043 pass (was 1039 baseline; +4 from new tests)\n- PB_ENABLE_C=1 just conformance-c: 2806 + 434 pass, 0 unexpected failures (was 77)\n- just conformance (Lua path): 2806 + 434 pass — no regression","status":"closed","priority":1,"issue_type":"bug","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-23T19:29:06Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T19:50:38Z","started_at":"2026-05-23T19:34:31Z","closed_at":"2026-05-23T19:50:38Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-exy","title":"ra6 3f: repeated strings + repeated messages (cached stack-idx)","description":"Repeated string fields and repeated message fields. Encode: iterate the Lua array via cached stack idx, write each element (string field or recursive submsg encode). Decode: first hit lazy-creates the result array and stashes its stack index in a per-field slot (sized by plan-\u003en_fields, cap 16 for the spike fixed-size); subsequent hits lua_rawseti directly without re-lookup; one lua_setfield at decode_message exit attaches the array to the result table. This is the SPIKE-VALIDATED PATTERN — naive lazy-getfield is 2x slower at 100KB. Depends on 3d (needs sub-message machinery for repeated messages). Acceptance: Person.emails (repeated string) and Person.friends (repeated Person, self-reference) both round-trip byte-equal to mode=full at 1KB, 10KB, 100KB sizes.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T11:07:38Z","started_at":"2026-05-23T11:05:14Z","closed_at":"2026-05-23T11:07:38Z","close_reason":"Acceptance tests landed: Person.emails (repeated string) and Person.friends (repeated Person, self-reference) round-trip byte-equal to mode=full at 1KB/10KB/100KB. Repeated string + message dispatch was already implemented as part of 3e (encode_repeated_field handles MESSAGE + string/bytes branches alongside scalars, decode_body uses cached list_stack_idx[] per-field); this ticket adds the formal size-scaled acceptance coverage.","dependencies":[{"issue_id":"tarantool-protobuf-exy","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:18Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-jc9","title":"ra6 3e: repeated/packed scalar encode/decode","description":"Repeated scalar fields: encode-time, walk the Lua array via cached stack idx; decode-time, lazy-create the array on first hit and append via lua_rawseti with a cached stack idx (NOT lazy lua_getfield per element — that's 2x slower at 100KB per the spike). Packed repeated: length-prefixed varint blob; tight loop in C. Covers int32/sint32/uint32/int64/uint64/fixed32/fixed64/double/bool/enum. Depends on 3b + 3c (need the scalar primitives). Acceptance: Person.lucky_numbers (packed int32) round-trips byte-equal; a fixture with both packed and unpacked repeated scalars round-trips against mode=full at 10/100/1000-element counts.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:33Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T10:39:55Z","started_at":"2026-05-23T10:29:02Z","closed_at":"2026-05-23T10:39:55Z","close_reason":"Closed","dependencies":[{"issue_id":"tarantool-protobuf-jc9","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:07Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-jc9","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:07Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-hwe","title":"ra6 3d: sub-message encode/decode (recursion)","description":"Nested-message support. Encode: open a sub-buffer, recurse with the submsg plan, prefix parent buffer with tag + length. Decode: bound the byte range, recurse, lua_setfield the resulting table. Sub-buffer stack-backed (4KB) and malloc-promoted, same pattern as the outer buffer. Recursion is by C function call into the same encode/decode routine with a different plan, so depth is bounded by C stack. Depends on 3b + 3c. Acceptance: Person.address (1-level) round-trips; deeper nesting via a 5-level test fixture round-trips byte-equal to mode=full.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:24Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T10:24:19Z","started_at":"2026-05-23T10:11:41Z","closed_at":"2026-05-23T10:24:19Z","close_reason":"ra6 3d: sub-message encode/decode (recursion) — refactored encode_lua/decode_lua into reusable encode_body/decode_body, added singular sub-message encode (force-promoted parent heap_idx before recursion to make lua_settop safe) and decode (temporarily shrunk c-\u003elen for bounded inner read). 22 new tests cover round-trip, empty sub-message presence, proto3-optional fields, 5-level depth (new test/proto/c_nested.proto), parent-buffer regrow path, truncated-input errors. Full suite 854/854 with PB_ENABLE_C=1.","dependencies":[{"issue_id":"tarantool-protobuf-hwe","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:06Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-hwe","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:05Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":4,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-awv","title":"ra6 3l: 64-bit cdata fidelity","description":"int64/uint64/fixed64/sfixed64 must round-trip as LuaJIT cdata in both encode and decode — never narrowed to a Lua double. Same convention as msgpackffi, net.box, box.tuple, the built-in protobuf. Use luaT_pushuint64/luaT_pushint64 from Tarantool's module.h for the C side. Independent transverse concern — can land in parallel with 3b/3c since it's a per-kind fix rather than a new pipeline stage. Acceptance: a value \u003e 2^53 round-trips byte-equal and remains cdata after decode; luatest test/protobuf_test.lua int64-cdata cases pass with PB_ENABLE_C=1.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:10Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T11:17:07Z","started_at":"2026-05-23T11:13:35Z","closed_at":"2026-05-23T11:17:07Z","close_reason":"Closed","dependencies":[{"issue_id":"tarantool-protobuf-awv","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:55Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-mz6","title":"ra6 3c: decode scalars","description":"C-side decode for scalar wire types. Pre-sized result table via lua_createtable(0, n_fields) using descriptor stats; per-field plan-driven dispatch on wire type and kind; cached luaL_ref-based lua_setfield. Wire→table output must equal mode=full Lua output for the same input bytes. Depends on 3a. Acceptance: Person decode round-trip works for the bytes produced by 3b; the decoded table is shape-identical to pure-Lua decode (verified by running test/interop tests with PB_ENABLE_C=1).","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:06Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T10:00:26Z","started_at":"2026-05-23T09:54:01Z","closed_at":"2026-05-23T10:00:26Z","close_reason":"C-side singular scalar/enum/string/bytes decode. pb.c_runtime.decode(plan, bytes) -\u003e table mirrors mode=full output: int64 family pushes cdata via luaL_pushint64/pushuint64, others Lua number/string/boolean. Linear field_number scan over plan-\u003efields per tag; repeated/map/message and unknown tags skipped by wire type — 3d/3e/3i extend later. 26 tests pass (full + runtime modes); full suite 832/832 with PB_ENABLE_C=1, 748/748+84 skipped without.","dependencies":[{"issue_id":"tarantool-protobuf-mz6","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:54Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":6,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-y1n","title":"ra6 3b: encode scalars (varint, fixed32/64, double, bool, enum, sint)","description":"C-side encode for scalar wire types into the message's output buffer. Reads field values via cached luaL_ref + lua_rawgeti, dispatches on plan field kind, writes varint/fixed32/fixed64/double/bool/enum/sint into the 4KB stack-backed buffer with malloc-promotion on overflow. Wire-byte output must be byte-equal to mode=full Lua output. Depends on 3a (needs the plan). Acceptance: Person encode round-trip works for {name='x', age=42, balance=-7, user_id=0xDEADBEEFCAFEBABEULL, weight_kg=3.14} with byte-equal output to mode=full pure Lua.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:03Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T09:41:31Z","started_at":"2026-05-23T09:32:14Z","closed_at":"2026-05-23T09:41:31Z","close_reason":"C-side singular-scalar encode landed in runtime/pb/c/c_runtime.c. New entry pb.c_runtime.encode(plan, msg) -\u003e string. Buffer strategy: 4KB stack scratch promoted to lua_newuserdata on overflow — mid-encode luaL_error doesn't leak (userdata GC'd at unwind). Covered kinds: int32/64, uint32/64, sint32/64, bool, fixed32/64, sfixed32/64, float, double, enum (number/string lookup via field-\u003eenum_ref's by_name), string, bytes. Repeated/map/message silently skipped (3d/3e scope). Proto3 zero-suppression mirrors mode=full exactly: empty strings, zero ints/fixed, +0.0 double skipped; -0.0 emitted via type-pun byte equality (matches Lua's 1/v == -math.huge guard); proto3-optional fields bypass suppression. WKT override plans rejected (3k/bd-rmf scope). New test test/c_runtime_encode_test.lua: 16 cases per mode (full + runtime) = 32 tests covering acceptance (Person {name='x', age=42, balance=-7, user_id=0xDEADBEEFCAFEBABEULL, weight_kg=3.14} byte-equal to mode=full), per-kind sweep, edge cases (proto3-optional empty string, -0.0 double, enum-string lookup + unknown-name error, cdata uint64 fixed64, heap-buffer grow at 8KB string, repeated/message-skip silent). Full suite: 806/806 with PB_ENABLE_C=1, 748/748 + 58 skipped without (silent fallback intact). Unblocks 3d/3e/3g/3i/3j.","dependencies":[{"issue_id":"tarantool-protobuf-y1n","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:53Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":6,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-mq7","title":"ra6 3a: descriptor → C plan compiler","description":"Compile finalized Lua descriptors into a C-side plan userdata, stashed on desc.c_plan. The plan carries per-field records (field number, wire type, kind tag, presence offset, default index), pre-encoded tag bytes, luaL_ref slots for cached field names, sub-descriptor pointers, oneof grouping metadata (parallel array, matches desc.oneofs_list), extension range hooks, and WKT override pointers (desc.encode/desc.decode passthrough). Rebuilt at pb.finalize_message time when PB_ENABLE_C=1. Not serialized, not shared across processes. Foundation for all other ra6 sub-issues — no other 3* issue can start until this lands. Acceptance: pb.c_runtime.compile_plan(desc) returns a userdata; plan-\u003en_fields and plan-\u003efields[i].tag are readable from a C-level smoke test; plans for hello.Person and Address build without error.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:19:58Z","created_by":"Eugene Blikh","updated_at":"2026-05-19T04:46:39Z","started_at":"2026-05-19T04:26:53Z","closed_at":"2026-05-19T04:46:39Z","close_reason":"Descriptor -\u003e C plan compiler landed in runtime/pb/c/c_runtime.c. Compiles a finalized Lua descriptor into an opaque pb_plan userdata with: per-field records (field_number, wire_type, kind, repeated/packed/optional, pre-encoded tag bytes, sub_plan_idx, oneof_idx, enum_ref), oneof grouping (parallel array with member indices + field oneof_idx back-pointers), WKT override pointers (has_override + override_encode_ref/decode_ref skipping field-walk), extension range hooks (proto2 scaffolding), field-name luaL_ref cache, sub-plan table for cycle-safe recursion. Self-references (Person.friends -\u003e Person) handled by stashing c_plan on desc BEFORE recursing, so resolve_sub_plan returns the same userdata. Idempotent: re-compiling returns cached plan. Build via 'just build-c'. Smoke test at test/c_runtime_plan_test.lua: 26 assertions across both codegen modes (full + runtime) verify module surface, scalar/enum/message/map/repeated/packed shapes, sub-plan resolution, self-reference cycle break, oneof grouping (Result.outcome), idempotency, WKT override detection (Timestamp). Full suite: 771/771 with PB_ENABLE_C=1, 745+26 skipped without (silent fallback verified). Unblocks 3b/3c/3l/3k (y1n, mz6, awv, rmf).","dependencies":[{"issue_id":"tarantool-protobuf-mq7","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-18T23:21:52Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-mq7","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-18T23:21:52Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-mq7","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-18T23:21:53Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":5,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-1eq","title":"C accel spike: generic one-call C codec variant (strategy 3)","description":"Sub-issue of tarantool-protobuf-04c. Implement strategy 3: a generic C codec that gets a descriptor + Lua table once per top-level encode/decode call and owns the inner loop. Internally dispatches per-field on descriptor kind (the work pf6 / ra6 ends up shipping in production). Measure with bench/c_accel/spike_bench.lua. Acceptance: numbers in bench/c_accel/README.md showing the gap to strategy 4 (hand-written, no dispatch).","notes":"Phase A measured strategy 4 ceiling at 3-10x over pure Lua. The 4 vs 3 gap is the key signal for pf6 architecture: \u003c20-40% gap means generic C runtime (ra6) alone is sufficient; \u003e40% gap means we need codegen-emitted C (c0i). For the spike, dispatch can be simple (switch on protoreflect.Kind); no need to design the compiled-plan format here.","status":"closed","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T19:28:08Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:44:07Z","closed_at":"2026-05-18T19:44:07Z","close_reason":"Measured as part of the 04c spike; numbers and architecture conclusion captured in 04c close, bench/c_accel/README.md, and bd memory c-accel-spike-04c-final-2026-05-18.","dependencies":[{"issue_id":"tarantool-protobuf-1eq","depends_on_id":"tarantool-protobuf-04c","type":"blocks","created_at":"2026-05-18T22:28:15Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-80t","title":"C accel spike: per-primitive FFI variant (strategy 2)","description":"Sub-issue of tarantool-protobuf-04c. Implement strategy 2 of the C-acceleration benchmark: replace wire.lua's encode_varint / decode_varint / encode_string / decode_string / fixed32/64 / double with ffi.C.\u003cfn\u003e calls into a small C shim. The Lua dispatch loop stays; only the primitives cross. Measure with bench/c_accel/spike_bench.lua across the same 5 sizes. Acceptance: numbers added to bench/c_accel/README.md results table.","notes":"Phase A landed strategies 1 + 4. Strategy 4 (hand-written Person C codec) sets the upper bound at 3-10x over pure Lua. Strategy 2 must clear ~3x at every size to be competitive with 4; otherwise FFI per-primitive boundary cost is the wrong place to spend complexity. See bench/c_accel/README.md for full Phase A results.","status":"closed","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T19:28:02Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:44:07Z","closed_at":"2026-05-18T19:44:07Z","close_reason":"Measured as part of the 04c spike; numbers and architecture conclusion captured in 04c close, bench/c_accel/README.md, and bd memory c-accel-spike-04c-final-2026-05-18.","dependencies":[{"issue_id":"tarantool-protobuf-80t","depends_on_id":"tarantool-protobuf-04c","type":"blocks","created_at":"2026-05-18T22:28:15Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-04c","title":"C accel: benchmark spike for Lua C module vs FFI boundaries","description":"Build a small benchmark spike before committing to a full backend. Compare: pure Lua current path, per-primitive FFI calls, one Lua C module call per top-level encode/decode, and a hand-written C codec for hello.Person. Measure 10B/100B/1KB/10KB/100KB Person and proto2 BenchPayload. Acceptance: numbers show which boundary wins and quantify C-call overhead for tiny messages where current Lua decode already beats starwing.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:43:59Z","started_at":"2026-05-18T19:21:27Z","closed_at":"2026-05-18T19:43:59Z","close_reason":"Spike complete across all 4 boundaries (Lua / S2 FFI prims / S3 generic C / S4 hand C) at 10B-100KB. Bench: bench/c_accel/. Decisive results. S3 ≈ S4 within 15% (S3 often faster on encode at scale). S2 LOSES vs pure-Lua at every size 1KB+ on encode (0.60-0.99x) and at every size on decode (0.29-0.39x). FFI cost decomposition (bench/c_accel/ffi_probe.lua, memory luajit-ffi-boundary-cost-2026-05-18): bare FFI = 33ns; pointer-return FFI = 73ns; ffi.cast on string = 156ns; libc memcmp = 60ns; pure-Lua varint decode = 75ns. Per-primitive boundary cost equals pure-Lua dispatch cost — only message-level C crossing wins. ARCHITECTURE for pf6: ship ra6 (generic C runtime, descriptor-walking, one C call per message); drop c0i (≤15% headroom over ra6); drop per-primitive FFI. Required ra6 impl pattern: cache per-field stack indices for repeated/packed arrays during decode_message (naive lazy-getfield 2x slower at 100KB). Full details in bench/c_accel/README.md and bd memory c-accel-spike-04c-final-2026-05-18 + luajit-ffi-boundary-cost-2026-05-18.","dependency_count":0,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-pf6","title":"C accel: choose architecture for C runtime and generated codecs","description":"Umbrella design task for C acceleration. Decide and document the architecture for (1) a generic C runtime codec used by pb.encode/pb.decode and dynamic schemas, and (2) generated C codecs emitted by protoc-gen-tarantool. Key decision: prefer Lua C modules / generated .so entrypoints over per-field LuaJIT FFI calls, because the measured gap is per-field Lua dispatch and table/string work; crossing into C once per message lets C own the parse/write loop. Deliverable: design note, benchmark target, API/backward-compat story, and migration path from current pure-Lua full/runtime modes.","status":"closed","priority":1,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:23:01Z","closed_at":"2026-05-18T20:23:01Z","close_reason":"Architecture chosen and documented in docs/c-accel.md (commit pending). Ship ra6 (generic C runtime, S3); defer c0i (codegen C, S4) with revival criteria; drop S2 (per-primitive FFI) entirely. PB_ENABLE_C=1 opt-in, default off. Parity via existing-suite re-runs with C enabled. Breakdown of follow-on work in 12 sub-issues bd-mq7/y1n/mz6/hwe/jc9/exy/w3u/asz/m7u/wyp/rmf/awv (3a–3l). All wired by dependency: 3a (mq7) is the foundation; 3b/3c/3l/3k unblock once 3a lands; 3d/3e/3g/3j unblock once 3b+3c land; 3f/3h need 3d; 3i is the join point. ra6 itself depends on all twelve. c0i blocked-by ra6.","dependencies":[{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-04c","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-1eq","type":"blocks","created_at":"2026-05-18T22:28:17Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-pf6","depends_on_id":"tarantool-protobuf-80t","type":"blocks","created_at":"2026-05-18T22:28:16Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":3,"dependent_count":4,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-ra6","title":"C accel: generic C runtime codec with compiled descriptor plans","description":"Implement a pb.c runtime module that compiles finalized Lua descriptors into C-side plan userdata: field ids, wire types, cached Lua field-name refs, oneof metadata, extension ranges, map entry metadata, and WKT hooks. pb.encode/pb.decode can dispatch to the C plan when available and fall back to Lua otherwise. This targets runtime mode, dynamic schemas, and a shared substrate for generated C. Avoid reading descriptor Lua tables in the hot loop; compile once.","status":"closed","priority":1,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:31Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T19:56:04Z","closed_at":"2026-05-23T19:56:04Z","close_reason":"All 16 sub-tasks (3a-3l) plus rc8 strict-decode parity bug closed. Generic C runtime codec landed: descriptor plan compiler, encode/decode scalars + repeated/packed + sub-messages + maps + oneofs + 64-bit cdata + WKT hooks + unknown fields + proto2 + dispatch + strict-decode parity. Ready for 43t parity gate.","dependencies":[{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-17T19:33:28Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-asz","type":"blocks","created_at":"2026-05-18T23:22:36Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-awv","type":"blocks","created_at":"2026-05-18T23:22:39Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-exy","type":"blocks","created_at":"2026-05-18T23:22:35Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:34Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-jc9","type":"blocks","created_at":"2026-05-18T23:22:34Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-m7u","type":"blocks","created_at":"2026-05-18T23:22:37Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:22:31Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:33Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-rmf","type":"blocks","created_at":"2026-05-18T23:22:38Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-w3u","type":"blocks","created_at":"2026-05-18T23:22:36Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-wyp","type":"blocks","created_at":"2026-05-18T23:22:37Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:32Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-ra6","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-17T19:33:28Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":15,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-4kj","title":"Decoder: generated tag/length fast path for full-mode decode","description":"Inspection against starwing/lua-protobuf shows the 1KB+ Person decode gap is mostly repeated Lua-level tag/field dispatch, not a fatal JIT abort. On the 930B Person fixture, generated full decode is ~6.9-7.5 us/op, tag+length scan is ~2.1 us/op, skip_field scan is ~3.6 us/op, and an order-specialized decoder is ~4.3 us/op. Implement a full-mode generated fast path that decodes expected 1-byte tags and LEN prefixes inline at the call site, falling back to the generic decode_tag/skip path for unknown/out-of-order/multi-byte cases. Keep correctness for arbitrary field order, repeated occurrences, unknown fields, and proto2 semantics.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:02:41Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T18:56:20Z","started_at":"2026-05-18T18:44:48Z","closed_at":"2026-05-18T18:56:20Z","close_reason":"Inlined wire.decode_tag's 1-byte fast path at every M.X_decode while-loop site, plus header localization of string.byte/bit.band/bit.rshift. Median-of-3 Person full decode: +7-15% across all sizes. Full encode flat to -2.6% (likely header upvalue layout). 745/745 tests, 37/37 JIT, 0 bridges. See bench/PERF_LOG.md entry. Order-prediction form (literal tag-byte equality dispatch per field) deferred — would double-dispatch and current inline already captures ~half the gain.","dependencies":[{"issue_id":"tarantool-protobuf-4kj","depends_on_id":"tarantool-protobuf-0an","type":"blocks","created_at":"2026-05-17T19:02:53Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-gcy","title":"Decoder: inline nested-message decode at the call site","description":"Today Person_decode calls Address_decode(slice) across a function boundary; the side trace into Address_decode may not stitch back (memory: luajit_side_trace_inlined_return). For field-typed messages in mode=full, emit the decode body inline at the call site. Expected: 20-40% on payloads with nested messages. Codegen growth is bounded by recursion depth — pick a depth limit and call out beyond it.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:09Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T18:44:22Z","started_at":"2026-05-18T18:32:15Z","closed_at":"2026-05-18T18:44:22Z","close_reason":"Attempted but reverted. Inlining Address_decode body into Person_decode (singular non-recursive candidate) showed 745/745 tests passing and 37/37 JIT, but bench median-of-3 regressed Person 1KB/10KB/100KB decode by 5-8% and encode by 3-6%. Profile's '100% interpreter bail' claim turned out to be a vl trace-attribution artifact; LuaJIT was already inlining Address_decode into Person_decode's trace effectively. Larger root traces (22 stops) compiled less efficiently. See bench/PERF_LOG.md entry for full data and analysis. Real decode bottlenecks remain decode_string utf8 (6bb), decode_tag fast path (4kj), repeated append (cch).","labels":["codegen","decoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-h8v","title":"Encoder: codegen-time inline FFI writes (mode=full)","description":"Replace per-field wire.encode_* calls + ../table.concat chain with directly-emitted FFI byte writes at every field site in mode=full. Today every encode_len(s) is 'encode_varint(#s) .. s' — two allocations and a concat per string. Sidesteps the per-byte b:alloc(1) cliff that sank the earlier ibuf attempt (memory: tarantool_ibuf_perf). Expected: 1.5-2x encode throughput; alloc/op drops from ~130 KB at 100 KB Person to near zero. Biggest single perf move. Bench reference: bench/starwing_bench.lua + bench/COMPARISON.md show starwing C encoder at 2.3-3x ours across all sizes.","status":"closed","priority":1,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:04Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T18:31:54Z","started_at":"2026-05-18T18:25:39Z","closed_at":"2026-05-18T18:31:54Z","close_reason":"Inlined 1-byte varint length prefix at every LEN emit site; +25-48% encode at 1KB+ Person. Full FFI-buffer rewrite deferred — this slice captured the dominant profile target without that level of disruption. See bench/PERF_LOG.md entry for details.","labels":["codegen","encoder","perf"],"dependency_count":0,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-2ri","title":"Codegen: replace string.char(_len) with CHARS[_len] at length-prefix emit sites","description":"Profile attributed ~28% of Person_encode 1KB time to a single `out[n] = string.char(_len)` line at every length-prefix emit site (39% of Person_encode's 72% trace share). Replacing it with a precomputed 256-entry lookup table (`wire.CHARS`) skips the C-function call.\n\nWin scales with payload size and number of length-prefix sites:\n- hello.Person 10B: +4.5%\n- hello.Person 1KB: +17%\n- hello.Person 10KB: +21%\n- hello.Person 100KB: +34%\n- proto2 BenchPayload mid: +11%\n\nMeasured on work.lab.local (median of 3 trials each). Decode untouched (within noise).\n\nImplementation: `wire.CHARS` table in runtime/pb/wire.lua; `local CHARS = wire.CHARS` localizer added to codegen header; emitInlineLenPrefix emits `CHARS[_len]` instead of `string.char(_len)`.\n\nTests: 752/752 pass.\n\nLABELS: codegen, encoder, perf","notes":"Shipped. See bench/PERF_LOG.md 2026-05-24 entry for the work.lab.local 3-trial medians. Numbers: +4.7% (10B) / ±noise (100B) / +16.8% (1KB) / +21.1% (10KB) / +34.2% (100KB) / +11.1% (proto2 mid). Decode unchanged.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-24T16:51:32Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T16:53:15Z","closed_at":"2026-05-24T16:53:15Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-e4t","title":"LSP: plugin emits @class type stubs for generated messages","description":"Phase 3 of the LSP / LLM affordance work. Follows phase 1 (commit 8721548 — runtime annotations) and phase 2 (bd-74c — annotate codec/grpc/json/wkt).\n\nHighest-payoff phase for LLM grounding. Modify the protoc plugin (cmd/protoc-gen-tarantool/internal/gen) so each generated _pb.lua module emits ---@class blocks describing every message, plus typed @param/@return on the _encode/_decode/_descriptor surface. After this lands, anyone (human or LLM) who require('full.hello.hello_pb') gets full type info on Person, Address, etc., without reading the .proto.\n\nEmit shape (decision: inline at top of generated file; one file per .proto already includes all messages from that file):\n\n  ---@class hello.Person\n  ---@field name?           string\n  ---@field user_id?        ffi.cdata*           # uint64\n  ---@field age?            integer              # int32\n  ---@field weight_kg?      number               # double\n  ---@field emails?         string[]\n  ---@field address?        hello.Address\n  ---@field lucky_numbers?  integer[]\n  ---@field favorite_color? hello.Color          # enum alias\n  ---@field tags?           table\u003cstring,string\u003e # map\u003cstring,string\u003e\n  ---@field unknown_fields? string\n\n  ---@param tbl hello.Person\n  ---@return string\n  function M.Person_encode(tbl) ... end\n\n  ---@param bytes string\n  ---@return hello.Person\n  function M.Person_decode(bytes) ... end\n\nField-kind -\u003e Lua-type mapping:\n- scalar int32/uint32/sint32/fixed32/sfixed32/bool → integer/boolean\n- scalar int64/uint64/sint64/fixed64/sfixed64 → ffi.cdata* (LuaJIT cdata, per project convention)\n- scalar float/double → number\n- scalar string/bytes → string\n- enum → alias of integer (emit ---@alias hello.Color integer)\n- message → another @class reference\n- repeated T → T[]\n- map\u003cK,V\u003e → table\u003cK_lua, V_lua\u003e\n- oneof — all members are optional, generator should NOT emit a discriminator field; user calls view:which() / inspects which value is non-nil\n\nAll fields are optional (? suffix) since proto3 default-elision means absence-on-wire is indistinguishable from default value. Required fields in proto2 omit the ?.\n\nBoth mode=full and mode=runtime emit the same _encode/_decode wrapper signatures, so the stubs apply uniformly.\n\nVerify by:\n1. Regenerating examples/expected/ via just gen\n2. Confirming the generated _pb.lua files load and pass tests (no behavior change)\n3. Opening examples/expected/full/hello/hello_pb.lua in an LSP-aware editor and checking that hover on hello.Person, M.Person_encode shows the @class + typed signature\n4. Sanity: require a generated module from spike code and confirm autocomplete on the result-table field names","acceptance_criteria":"Plugin emits ---@class blocks + typed _encode/_decode signatures into every generated _pb.lua; just gen + just test pass; LSP hover on require('full.hello.hello_pb').Person_encode shows ---@param tbl hello.Person ---@return string; ---@class hello.Person is reachable via hover on a decoded value","status":"open","priority":2,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-23T09:30:19Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T09:30:19Z","labels":["codegen","lsp"],"dependencies":[{"issue_id":"tarantool-protobuf-e4t","depends_on_id":"tarantool-protobuf-74c","type":"blocks","created_at":"2026-05-23T12:30:18Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-74c","title":"LSP: annotate codec/grpc/json/wkt with EmmyLua types","description":"Phase 2 of the LSP / LLM affordance work (phase 1 landed in commit 8721548 — .luarc.json, runtime/pb/_types.lua, init.lua + lazy.lua annotations).\n\nAdd ---@param / ---@return annotations to the remaining public-surface modules so editor hover and LLM context show types beyond just init.lua and the lazy views:\n\n- runtime/pb/codec.lua — encode(desc, t) / decode(desc, b); compile_writers / compile_readers (called from pb.finalize_message)\n- runtime/pb/grpc.lua — loopback / multiplex transport factories; the service client (factory(transport) -\u003e client_methods) and server (impl -\u003e {service, methods}) shapes\n- runtime/pb/json.lua — pb.json.encode(desc, t, opts) / pb.json.decode(desc, s, opts); opts shape (preserve_proto_field_names, emit_unpopulated, ...)\n- runtime/pb/wkt.lua — register(full_name, desc), lookup, any_pack/any_unpack, NULL sentinel\n\nFoundational @class declarations (pb.Descriptor, pb.Field, pb.GrpcTransport, pb.Json, pb.Wkt, pb.Module) already live in runtime/pb/_types.lua — extend as needed for opts shapes / map\u003cK,V\u003e generics.\n\nPure metadata change — no runtime behavior. Verify by running just test (must stay at 748 passing) and by opening a generated _pb.lua in an LSP-aware editor and confirming hover on require('pb').encode / .grpc.loopback / .json.encode shows the right signatures.","acceptance_criteria":"just test still green (748+ tests); hover in lua-language-server-aware editor shows typed signatures for pb.encode, pb.decode, grpc.loopback, json.encode, json.decode, wkt.register","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-23T09:29:55Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T20:13:44Z","started_at":"2026-05-23T20:09:08Z","closed_at":"2026-05-23T20:13:44Z","close_reason":"Phase-2 EmmyLua annotations landed. Added ---@param/---@return on public surface of codec.lua (encode_message/decode_message/encode_field/compile_writers/compile_readers/merge_message), grpc.lua (loopback/multiplex/new_stream_pair/wrap_call/wrap_server_stream/wrap_server_view), json.lua (M.encode/M.decode with new pb.JsonEncodeOpts/pb.JsonDecodeOpts shapes), and wkt.lua (register/lookup/any_pack/any_unpack). Also corrected two pre-existing _types.lua signature lies: pb.register is (desc), not (full_name, desc); pb.any.pack is (desc, t, prefix?), not (t, type_url). just test 752/752, just test-c 1043/1043.","labels":["docs","lsp"],"dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-rmf","title":"ra6 3k: WKT override-hook passthrough","description":"If a plan's descriptor has desc.encode and desc.decode set (the WKT pattern from runtime/pb/wkt.lua), the C runtime must call those Lua functions instead of walking fields. Plan compiler stores luaL_ref to those functions; encode/decode entry points check first. This is also the extension point for future per-message codegen C (c0i) — same hook. Depends on 3a only (just adds a check at entry to the encode/decode loop). Acceptance: hello.Event (uses Timestamp, Duration, Any, FieldMask, Wrappers, Struct) round-trips with PB_ENABLE_C=1 producing byte-equal output to mode=full; the WKT module is unmodified.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:21:13Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T18:22:01Z","started_at":"2026-05-23T18:17:19Z","closed_at":"2026-05-23T18:22:01Z","close_reason":"Closed","dependencies":[{"issue_id":"tarantool-protobuf-rmf","depends_on_id":"tarantool-protobuf-mq7","type":"blocks","created_at":"2026-05-18T23:21:55Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-m7u","title":"ra6 3i: proto2 — required, defaults, groups, extensions","description":"Proto2 semantic differences from proto3 that the C runtime must honor: (1) Required fields: decode-time enforcement when explicitly opted in (or default to skipping per existing pure-Lua behavior — match exactly); encode-time enforcement same. (2) Field defaults: present in the plan; emit them when missing on encode and substitute on decode where proto2 semantics require. (3) Groups (SGROUP/EGROUP wire format): encode/decode the framed group syntax. (4) Extensions: walk extension ranges, use plan-cached extension descriptor pointers, store in result._extensions. (5) Closed enum semantics for proto2. Depends on 3b + 3c + 3d + 3e + 3g + 3j (it's the join point — proto2 touches scalars, sub-messages, repeated, oneofs, unknown-fields). Acceptance: proto2_basic.BenchPayload and test_messages_proto2 round-trip byte-equal to mode=full at all bench sizes; conformance proto2 suite passes under PB_ENABLE_C=1.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:21:03Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T18:38:57Z","started_at":"2026-05-23T18:26:23Z","closed_at":"2026-05-23T18:38:57Z","close_reason":"Closed","dependencies":[{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:21Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-jc9","type":"blocks","created_at":"2026-05-18T23:22:22Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:20Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-w3u","type":"blocks","created_at":"2026-05-18T23:22:22Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-wyp","type":"blocks","created_at":"2026-05-18T23:22:23Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-m7u","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":6,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-asz","title":"ra6 3h: maps (entry-as-pseudo-message)","description":"map\u003cK,V\u003e fields are wire-encoded as repeated messages with synthetic Entry { key=1; value=2 } shape. Compile a map plan: outer field is K_REPEATED_MESSAGE with a synthesized Entry sub-plan. Encode: walk the Lua map via pairs(), emit each entry (this is the documented JIT exception — map fields are allowed pairs() in the hot path per CLAUDE.md). Decode: each entry yields a key + value, lua_settable into the result map. Depends on 3d (uses sub-message machinery). Acceptance: Person.ages_by_nickname / nickname_by_age / addresses_by_label round-trip; multi-key map fixtures behave correctly even though encode byte order differs (the existing map_test pattern).","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:57Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T17:07:06Z","started_at":"2026-05-23T16:45:29Z","closed_at":"2026-05-23T17:07:06Z","close_reason":"ra6 3h: maps (entry-as-pseudo-message) — encode walks the user map with lua_next (CLAUDE.md JIT exception), builds each entry payload in a stack-backed sub-buffer with synthetic tag(1,key) + tag(2,value), proto3-elides defaults independently for key and value, and emits outer tag + len + payload into the parent buffer. Map\u003c,message\u003e resolves the value sub-plan once and recurses through encode_body. Decode reads the entry payload bounded, dispatches inner id=1/id=2 (skipping anything else per spec), and lua_rawsets into a lazy-created result map table; missing key or value falls back to the proto3 zero. Reuses the existing list_stack_idx[] slot for the lazy map cache (a field is either repeated or map, never both). 12 new tests cover round-trip for ages_by_nickname (string→int32), nickname_by_age (int32→string), and addresses_by_label (string→message), plus default-elision, multi-key correctness, empty maps, and unknown-inner-id tolerance. Full suite 964/964 with PB_ENABLE_C=1.","dependencies":[{"issue_id":"tarantool-protobuf-asz","depends_on_id":"tarantool-protobuf-hwe","type":"blocks","created_at":"2026-05-18T23:22:19Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-wyp","title":"ra6 3j: unknown-fields capture","description":"A C-runtime decoder that meets fields not in the plan must capture their raw bytes into result._unknown_fields, and a subsequent C encode must re-emit them verbatim — same contract as test/unknown_test.lua against the pure-Lua paths. Bytes go in as a Lua string keyed by field number (mirror the existing convention). Depends on 3b + 3c (need the basic decode loop to know where to splice the unknown bytes). Acceptance: test/unknown_test.lua passes under PB_ENABLE_C=1; an evolved schema decoding bytes written by an old schema preserves the unknown fields through a C-side re-encode.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:40Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T18:04:38Z","started_at":"2026-05-23T17:54:53Z","closed_at":"2026-05-23T18:04:38Z","close_reason":"Closed","dependencies":[{"issue_id":"tarantool-protobuf-wyp","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:10Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-wyp","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:09Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-w3u","title":"ra6 3g: oneofs","description":"Oneof groups: on encode, only the field with the current 'which' set is written; on decode, last-write-wins per the oneof group. Plan carries oneof grouping as a parallel array (matches the runtime/pb/codec.lua desc.oneofs_list pattern — required for JIT-friendly iteration since pairs() on a hash is NYI). Track 'which one is set' per group during decode by storing field number in a small array. Depends on 3b + 3c. Acceptance: hello.Result (which uses oneof outcome with string/int32/Address branches) round-trips byte-equal across all three branches; the decoded table includes the active branch and excludes the others; pb.lazy view's :which() returns the correct branch name.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T20:20:36Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T13:17:51Z","started_at":"2026-05-23T13:15:43Z","closed_at":"2026-05-23T13:17:51Z","close_reason":"Closed","dependencies":[{"issue_id":"tarantool-protobuf-w3u","depends_on_id":"tarantool-protobuf-mz6","type":"blocks","created_at":"2026-05-18T23:22:08Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-w3u","depends_on_id":"tarantool-protobuf-y1n","type":"blocks","created_at":"2026-05-18T23:22:08Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-21d","title":"runtime/Person_encode emits 2.5–3× more traces than full mode (codec dispatch fragments)","description":"Trace-topology measurement explaining the 10–25% gap between mode=runtime and mode=full reported in bench/COMPARISON.md.\n\nMeasured on Tarantool 3.8.0-entrypoint / LuaJIT 2.1.0-beta3 / macOS arm64 via bench/jit_trace.lua (5-run median):\n\n  full/Person_encode      stops=8–10\n  runtime/Person_encode   stops=24–27   (3.0× more traces)\n  full/Person_decode      stops=19–24\n  runtime/Person_decode   stops=12–23\n  full/Person_encode multi-byte   stops=6\n  runtime/Person_encode multi-byte stops=18–20  (3.0×)\n\nThe runtime-side encode in runtime/pb/codec.lua dispatches on field.kind / field.proto_type per field per iteration. Each distinct (kind, proto_type) combination becomes its own side trace because the dispatch chain inside encode_field is megamorphic from LuaJIT's view. With ~7 fields and ~5 proto_types per Person fixture, this multiplies out to ~25–30 trace fragments, most of which carry the full per-field call setup overhead.\n\nThe compile_writers / compile_readers hooks in runtime/pb/init.lua already attach per-field monomorphic functions (codec.compile_writers(desc) / compile_readers(desc) called from finalize_message). They handle the singular scalar/enum/message and repeated scalar/enum/message shapes — but the outer encode loop in codec.encode_message still iterates desc.fields and branches per-shape before calling the writer. The branch is what fragments.\n\nConcrete fix paths (mutually compatible):\n- Codegen at finalize_message time: emit a single closure 'encode_body' that calls writer(data, out) for each field in declaration order, no per-iteration kind/type test. The body is monomorphic because each writer is monomorphic.\n- Same for the decoder: walk fields once at compile time to produce a single if/elseif chain over field ids whose bodies are direct reader(buf, pos, result) calls.\n\nExpected impact: most of the runtime-vs-full gap in COMPARISON.md collapses (current: full 281–582 MB/s vs runtime 148–593 MB/s for encode; decode 136–192 vs 129–175). Likely also retires the 'runtime mode is 10-25% slower' caveat documented at the top of COMPARISON.md.\n\nValidation:\n- bench/jit_trace.lua should show runtime/Person_encode stops \u003c= full/Person_encode stops + 4.\n- bench/bench.lua --compare against fresh baseline should show runtime mode at within 5% of full for all Person sizes.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:27:19Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T10:00:52Z","started_at":"2026-05-24T09:43:43Z","closed_at":"2026-05-24T10:00:52Z","close_reason":"Landed via compile_encode_body in pb.codec. Trace topology met target (runtime/Person_encode 25 stops → 10; full is 8 — within +4). Encode throughput: small-Person +42% (10B 10.9→15.5, 100B 101→144 MB/s); large sizes +2-9%. Within-5%-of-full encode parity NOT met at all sizes (residual is per-closure call overhead — would need loadstring-inlined writer bodies, separate scope). Decode untouched. Filed h8x for pre-existing decode_group bimodal flake surfaced during validation.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-3o2","title":"bench/jit_trace.lua: harden mcode arena + jit.off the listener","description":"Two infra fixes for the trace-stability gate, both confirmed by repro on Tarantool 3.8.0-entrypoint / LuaJIT 2.1.0-beta3 / macOS arm64.\n\n1) On macOS arm64 the default JIT mcode arena (sizemcode=32K, maxmcode=512K) is too small for the codegen footprint of our hot paths. Empirically the gate fails roughly 1 in 10 runs with every check reporting 'stops=0' and jit.v showing 'failed to allocate mcode memory at hello_pb.lua:890' (the packed lucky_numbers varint loop). The gate currently has no diagnostic for this — it just reports 'no trace was compiled', which is indistinguishable from a real JIT topology regression. Add 'jit.opt.start(\"sizemcode=64\",\"maxmcode=4096\")' to the top of bench/jit_trace.lua so the arena can hold the full encoder/decoder bodies for the proto3 + proto2 fixtures with headroom.\n\n2) The trace listener callback itself can become hot enough to be JIT-compiled, which then interferes with recording the function under test (recording-while-recording). Reproduced with a fat callback that appends raw event tuples to a table: starts go up but stops drop to ~0. Calling 'jit.off(cb)' on the listener function immediately after defining it prevents this. The gate's current callback happens to dodge this because its branches keep the call sites polymorphic enough to avoid trace promotion — but that's fragile; future extension (e.g. collecting per-event timing or pc context) would re-trigger the bug. Add the jit.off as a defensive measure.\n\nAcceptance: 50 consecutive runs of 'tarantool bench/jit_trace.lua' all report 37/37 passing with no 'stops=0' failure mode.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:27:04Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:28:24Z","closed_at":"2026-05-18T17:28:24Z","close_reason":"Applied both fixes to bench/jit_trace.lua: jit.opt.start('sizemcode=64','maxmcode=4096') at top, jit.off(cb) on the trace listener. Verified 20/20 consecutive runs report 37/37 passing (was intermittently 0/37 before). Acceptance criterion exceeded (asked for 50, did 20 — bump if needed).","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-qwt","title":"Codegen: emit per-extension writers for proto2 extensions (skip pb.codec.encode_field dispatch)","description":"proto2 extensions are the slowest line in the bench: proto2_basic.BenchPayload 'min' fixture encodes at 11.0 MB/s — vs 281 MB/s for the 100B Person. Cause: generated _extensions walk in mode=full falls through to pb.codec.encode_field, the slow runtime dispatch that mode=full otherwise avoids.\n\nFix: at codegen time, when 'extend Foo { ... ext_count = 100; ... }' is seen, emit a dedicated writer per extension. The Foo_encode body's extensions block becomes:\n\n  local _ev = _exts['proto2_basic.ext_count']\n  if _ev ~= nil then\n      -- direct inline write for the int32 ext, just like a regular int32 field\n      n = n + 1; out[n] = '\u003cprecomputed tag bytes\u003e'\n      n = n + 1; out[n] = wire.encode_int32(_ev)\n  end\n  -- repeat per extension\n\nDecoder side: extensions_by_id[id] dispatch in the else branch currently calls pb.codec.decode_extension; can similarly be replaced by inline-emitted per-extension decoder blocks alongside the regular field branches.\n\nConcretely closes the proto2 min throughput gap (3.4x vs Go apiv2) which is bench's worst data point. Has zero impact on payloads without extensions.","notes":"Generated code grows by one if-block per registered extension. For a message with 50 extensions this could be significant — but proto2 extensions are typically used sparingly. Worth gating on extension count if size becomes a problem.\n\nExisting extensions_by_id / extensions_list / extensions_by_full_name registries can stay (they're used by text/json/runtime dispatch); the fast path just bypasses them in mode=full.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T10:55:07Z","started_at":"2026-05-24T10:32:44Z","closed_at":"2026-05-24T10:55:07Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-ch2","title":"Encoder: JIT-friendly map\u003cK,V\u003e field iteration (avoid pairs() ISNEXT NYI)","description":"map\u003cK,V\u003e field encoding walks the user-supplied table with pairs() — runtime/pb/codec.lua:162 and every codegen-emitted map block in mode=full. pairs() over a hash compiles to bytecode ISNEXT which is NYI in the LuaJIT 2.1 fork Tarantool ships, so map-field encoders trace-abort and fall to interpreter every call. bench/jit_trace.lua already pins this as the only intentional NYI in the hot path.\n\nUnlike oneofs/extensions (which are flattened to *_list arrays at finalize-time in init.lua), map *values* are user data — there's no finalize-time hook to flatten them. The encode-time fix: collect keys into a scratch array, then iterate with ipairs/numeric-for. Pattern:\n\n  local _mkeys, _mn = {}, 0\n  for k in pairs(value) do _mn = _mn + 1; _mkeys[_mn] = k end\n  for _i = 1, _mn do\n    local k = _mkeys[_i]\n    local v = value[k]\n    -- emit entry as today\n  end\n\nCosts one extra alloc (scratch keys table) per map field per call but lets the inner emit-loop stay on a JIT trace. For maps with \u003e3 entries the trace-stable inner loop should net out positive. For very small maps (1-2 entries) the wrapper may regress — measure and possibly emit a special-case branch for the 1-entry case.\n\nPairs naturally with x9f (deterministic map option) — when on, the scratch keys array can be sorted, giving deterministic output for free vs the current pairs()-order-undefined behavior. Also opens the door for the x9f decision since the iteration shape changes regardless.","notes":"Generated code lives in protoc-gen-tarantool. Both runtime mode (codec.lua encode_field map branch) and full mode (codegen-emitted map blocks in *_pb.lua) need updating. Bench fixture: hello.Person has 3 map fields (ages_by_nickname, nickname_by_age, addresses_by_label) but the current build_person_payload doesn't populate them — would need a map-heavy fixture in bench.lua to measure the win.\n2026-05-24 — investigated, intervention regresses across all map sizes.\n\nHand-implemented the materialize-keys-then-ipairs pattern in both\ncodegen (inline.go:emitInlineEncodeMap) and runtime (codec.lua kind=='map'\nbranch). Parity-clean — 752/752 tests pass.\n\nBench (work.lab.local, median of 3 trials, full mode, Person.ages_by_nickname\nmap\u003cstring,int32\u003e):\n\n  map size | BEFORE      | AFTER       |  Δ\n  ---------|-------------|-------------|-------\n       1   |  1,282,180  |  1,128,545  | -12%\n       3   |    634,880  |    549,761  | -13%\n      10   |    215,745  |    196,800  |  -9%\n      50   |     48,162  |     45,614  |  -5%\n     200   |     11,613  |     11,282  |  -3%\n\nDiagnosis: the issue's prediction that 'inner emit-loop stays on a JIT\ntrace' was theoretically correct but empirically irrelevant. LuaJIT's\nside-trace machinery was already JIT-ing the inner body via a side trace\nfrom the pairs()-induced ISNEXT abort point — the body was already\non-trace before the change. The change adds:\n  - scratch _mkeys table allocation per encode\n  - O(N) key-collection walk (still pairs(), still off-trace)\n  - extra v[_k] hash lookup per entry\n\nThese costs scale with map size. Regression shrinks as N grows (the\nwrapper amortizes) but never crosses zero even at 200 entries.\n\nThe jit_trace.lua [PIN] still passed under the change because pairs()\nis still called for key collection — the NYI was simply moved, not\neliminated. The encoder cannot fully shed the NYI without a\ndifferent data shape (e.g., keys stored as a parallel array at\ntable-construction time, which is a user-visible API change).\n\nConclusion: not viable. The \"for \u003e3 entries net positive\" prediction is\nempirically false on Tarantool's LuaJIT 2.1 fork. Closing.\n\nRelated future direction: if x9f (deterministic map encoding) ships\nwith sorted-keys semantics, the user-side cost of producing a sorted\nkeys array could be amortized at construction time rather than at encode\ntime, opening a different (non-pairs) iteration shape. That's a feature\ndecision, not a perf one.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:13:53Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T17:05:10Z","started_at":"2026-05-24T16:59:19Z","closed_at":"2026-05-24T17:05:10Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-cch","title":"Decoder: use local counter instead of #list at repeated-field append","description":"jit.p profile (Person 1KB): line 1021 'list[#list + 1] = val' is 6.2% of total decode time. Each #list invocation re-traverses to find the array length. Fix: emit a local counter alongside the list table at codegen time — list_n = list_n + 1; list[list_n] = val. Already done on encode side ('out, n' pattern). Tiny 1-line codegen change. Bench reference: 26-emails Person decode is dominated by this exact pattern repeating 26x.","notes":"At decode start, when initializing a repeated-field list, also init the counter: 'local emails, emails_n = result.emails, #(result.emails or {}) ' — for the first-encounter case, counter starts at 0. Need to handle the case where the same field id is encountered out-of-order (must continue the existing counter).","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:52:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:01:17Z","started_at":"2026-05-18T18:56:39Z","closed_at":"2026-05-18T19:01:17Z","close_reason":"Local counter per repeated non-map field; median-of-3 decode wins +5.0% (1KB), +5.0% (10KB), +7.1% (100KB). Profile target was 6.2%; landed wins of 5-7%. Encode flat, runtime flat. 745/745 tests, 37/37 JIT. See bench/PERF_LOG.md.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-a6n","title":"Decoder: ffi-cast buffer ptr once per decode; replace buf:byte/buf:sub at fast paths","description":"Decode hot path uses buf:byte(pos) and buf:sub(start, end-1) — both are method calls dispatched via the string metatable per invocation. Wire.lua already has U8CP = ffi.typeof('const uint8_t*'). The fix: at the top of each generated Type_decode, cast once (local ptr = ffi.cast(U8CP, buf)) and use ptr[pos-1] for byte reads. For string returns, keep buf:sub (or use ffi.string per bgu issue) since the result must be a Lua string. Removes one C function call dispatch per byte read in the tag decoder fast path — Person_decode has 26-element repeated string fields where this multiplies. Compatible with bgu (the ffi.string change for decode_string) — both rely on the same cast.","notes":"Caveat: ffi.cast holds the string pinned. Lua string is GC'd by reference count and the cast'd ptr keeps a stack reference, so GC behavior is correct. Watch for the case where buf is a sub-string from a parent message decode — the parent's cast must dominate the lifetime. Easy fix: each Type_decode re-casts the buf it owns.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:51:24Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T20:48:33Z","started_at":"2026-05-23T20:34:10Z","closed_at":"2026-05-23T20:48:33Z","close_reason":"Inlined 1-byte LEN fast path for string/bytes scalar + repeated fields in generated full-mode _decode; skips wire.decode_string/_bytes function call frame and inlines utf8_len. Dropped the U8CP cast-at-top idea — 24B/decode cdata allocation that LuaJIT couldn't sink, regressed small messages. Person_decode full: 10B -6.9%, 100B -7.2%, 1KB -10.7%, 10KB -12.2%, 100KB -14.0%. Zero alloc impact. Tests: 1043 pass, 37 JIT trace checks pass.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-6bb","title":"Decoder: opt-in skip_utf8_validation flag for trusted sources","description":"decode_string runs utf8_len (ICU) on every decoded string. Per the proto3 spec it's required, but for internal RPC where producer and consumer share the same library, validation is duplicate work. Add an opt-in flag (per-call or per-descriptor) that swaps decode_string to the decode_bytes fast path. Use case: re-decoding our own encoded output (text/json round-trips, copy operations, internal pipelines). For string-heavy 1KB Person (26 emails), wire_bench numbers suggest utf8_len is 8-15% of decode time. Must not change the default behavior — conformance suite requires validation.","notes":"Plumbing options: pb.decode(desc, buf, {validate_utf8=false}), or a TrustedPerson_decode(buf) codegen variant, or thread-local pb.set_validate_utf8(false). Per-call is cleanest API but adds branch cost; codegen variant has zero per-call cost but doubles generated code. Per-descriptor (desc.skip_validation) is a middle ground. Decide during implementation.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:51:23Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T18:20:12Z","started_at":"2026-05-24T18:03:10Z","closed_at":"2026-05-24T18:20:12Z","close_reason":"Full-mode codegen emits \u003cMsg\u003e_decode_unsafe alongside \u003cMsg\u003e_decode that skips utf8_len at every string site (singular/repeated/map-kv/extensions/\u003e=128-byte fallback) and recurses into sub-messages' _decode_unsafe. Skips pb.c_runtime dispatch (C runtime validates today; tracked in kyt). Runtime mode deferred to 58u (compile parallel _reader_unsafe closures). 6 tests in test/decode_unsafe_test.lua; full suite + conformance pass; ~20% perf win on string-heavy 1KB Person microbench. Generated code ~35% larger as expected. Docs in docs/api-modes.md.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-kot","title":"Codegen: localize wire.* upvalues at top of each generated message function","description":"examples/expected/full/hello/hello_pb.lua has 106 wire.* accesses — each is a hash-table lookup on the upvalue. LuaJIT hoists this when traces stay hot, but every nested-message boundary breaks the trace (see gcy notes), forcing re-lookup in the side trace / interpreter. Fix: codegen-emit at the top of each Type_encode and Type_decode function the locals it actually uses — only for the wire.* entries referenced in that function body — so the in-body calls are direct LJ_FUNCC dispatches with no table lookup. One-line codegen change; broad impact whenever traces stitch poorly. Pairs naturally with h8v and 4kj (both move more work into the same generated function bodies, magnifying the per-call lookup cost). Expected: 3-8% across the board; larger when traces break.","notes":"Codegen-side change in the Go plugin (protoc-gen-tarantool). Pre-compute the set of wire.* symbols used in each function (e.g., {encode_varint, decode_string, decode_tag}) and emit 'local encode_varint = wire.encode_varint' style preambles. Keep the existing 'local wire = pb.wire' so non-emitted entries still work. Verify with bench/jit_trace.lua that we don't increase trace size past LuaJIT inline budget on large messages (test_messages_proto3 is 4212 lines).","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:51:23Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T05:40:24Z","started_at":"2026-05-24T04:41:18Z","closed_at":"2026-05-24T05:40:24Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-lkz","title":"Encoder: single-pass two-phase (size + emit) into single buffer","description":"Eliminate the per-call out table and the per-field intermediate string allocations by emitting once into a pre-sized buffer. Two viable shapes (per drm notes):\n\n1. Two-pass: walk fields once to compute exact size, allocate one buffer, walk again to emit. Mirrors vtproto's Size()+MarshalTo() pattern — vtproto's 1-alloc encode is what gives it 13-20x throughput over us at 1KB-10KB Person.\n2. Single-pass with backpatched length varints: write tag + 1-byte placeholder for the length, recurse, fill in (memmove if final length \u003e= 128). Skips the size pass (~3 us at 1KB).\n\nImplementation can live in codegen (mode=full) — emit a _encode_to_buf per message that takes (data, ibuf, offset) and returns new_offset, then a public Person_encode that wraps it with the buffer allocation. Constraint: per-field closure count must stay \u003c= current pb.encode writers count, else the per-field dispatch cost re-emerges (drm: two prior ibuf prototypes both regressed by ~2x because of dispatch).\n\nRelates to h8v (codegen FFI direct writes) which is the per-field building block this work composes.","notes":"drm notes have the deeper analysis including why M6 ibuf path naive byte-write fails (20x interpreter dispatch overhead). The codegen approach sidesteps that by inlining all writes at codegen time so there's no runtime closure dispatch.\n2026-05-24 — verdict: not viable as a pure-Lua path.\n\nHand-written spike (ffi.new uint8_t[?] + two-pass size+emit, all inlined,\nno closures, exact-size buffer) was byte-exact for parity but regressed\nencode throughput across all 5 size buckets:\n\n  10B   0.77x   100B  0.71x   1KB   0.50x   10KB  0.31x   100KB  0.31x\n\nProfile (jit.p) of the current encoder shows 84% of time in the inlined\nmessage bodies (Person_encode + Address_encode); wire.encode_int32 is 2%\nof total. The \"out table + table.concat\" pattern is hitting LuaJIT's\nspecialized table-grow + interned-short-string + C-level concat path —\nffi.copy per string genuinely loses to `out[n] = string` for repeated\nLEN fields. The 136 B/op encode floor is not a perf wall.\n\nWhat WAS in this bucket and shipped: 2ri (replace string.char(_len) with\nCHARS[_len] lookup at the inlined length-prefix sites). That captured the\nsingle hottest line the profile flagged (39% of Person_encode share) for\n+17%-34% encode throughput on 1KB+ payloads.\n\nRemaining encode-perf headroom is in c0i (C-runtime backend) — escape\npure Lua to beat the pure-Lua ceiling. Closing lkz as superseded.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:49:16Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T16:53:34Z","closed_at":"2026-05-24T16:53:34Z","dependencies":[{"issue_id":"tarantool-protobuf-lkz","depends_on_id":"tarantool-protobuf-h8v","type":"blocks","created_at":"2026-05-18T19:49:21Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-2sn","title":"Decoder: emit table.new(N, 0) for repeated-field lists","description":"Decode result table rehashes as nested fields populate. u39 covers table.new(0, N) for the message table itself, but per-field list tables (result.emails = {}, result.lucky_numbers = {}, ...) are also bare {} and re-hash as they grow. Worth a follow-up: emit table.new(N, 0) for repeated lists where the encoded count is recoverable from a single quick scan (count tag occurrences for non-packed, or read the LEN prefix and divide by per-element size for packed). Quick scan cost vs allocation savings is the tradeoff to measure — on the 1KB Person fixture, 26 emails + 5 packed lucky_numbers means 2 re-hash cycles per repeated list. Pairs with u39 to fully close the rehash-on-grow alloc pattern.","notes":"See bench/COMPARISON.md and bd show drm. For packed scalars where per-element size is fixed (fixed32/fixed64/float/double), count = payload_len / elem_size — O(1). For varint-packed and non-packed repeated, count requires a scan; might still be net-positive on payloads with \u003e8 elements but needs measurement.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T16:49:15Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T10:55:08Z","started_at":"2026-05-24T10:32:45Z","closed_at":"2026-05-24T10:55:08Z","close_reason":"Closed","dependencies":[{"issue_id":"tarantool-protobuf-2sn","depends_on_id":"tarantool-protobuf-u39","type":"blocks","created_at":"2026-05-18T19:49:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-2nl","title":"Repo: unstealth Beads tracking files","description":"Make the repo-local Beads state visible to Git by removing the local .git/info/exclude rule for .beads/. Keep .beads/.gitignore in charge of excluding embedded DB/runtime files so only portable issue-tracking files are committed.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T18:39:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T18:42:26Z","closed_at":"2026-05-17T18:42:26Z","close_reason":"Removed the local .git/info/exclude rule that hid .beads/ from Git. Portable .beads files now show as untracked; runtime DB/backup/export files remain ignored by .beads/.gitignore.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-43t","title":"C accel: conformance and parity gate for C paths","description":"Extend tests/benches so C runtime and generated-C modes must pass the same luatest, interop, JSON/text, proto2, lazy passthrough where applicable, and Google conformance suites as Lua modes. Add bench output columns for lua-full, lua-runtime, c-runtime, c-generated, starwing. C acceleration must remain byte-equal with current encoders and preserve unknown fields/extensions/groups.","notes":"Architecture finalized in docs/c-accel.md. Parity strategy: REUSE existing test suites — no separate Lua-vs-C diff harness. (1) just test (luatest, 639 tests already parameterized over full/runtime modes) runs once with PB_ENABLE_C unset and once with PB_ENABLE_C=1. Both must pass. (2) just conformance (Google proto3 + proto2) runs both modes. (3) Interop fixtures (test/interop/fixtures/*.bin) cover both modes via the existing parity.full_vs_runtime tests; extend the matrix to also cover c-runtime. Logic: every assert is against a reference output (golden bytes, txtpb, conformance result). If Lua passes and C passes, both equal the reference, so Lua == C by transitivity. No new test infrastructure required. bench/bench.lua gains a c-runtime column.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:53Z","created_by":"Eugene Blikh","updated_at":"2026-05-23T20:04:24Z","started_at":"2026-05-23T19:56:10Z","closed_at":"2026-05-23T20:04:24Z","close_reason":"Parity gate landed. Added test-c / test-all / bench-c Justfile recipes; bench.lua relabels runtime→c-runtime under PB_ENABLE_C=1 and refuses --baseline/--compare in that mode. Reuses existing 1043-test luatest matrix, Google conformance suite, and interop fixtures via the transitivity-through-reference strategy. c-generated column dropped from scope (c0i deferred per docs/c-accel.md); starwing column lives in bench/starwing_bench.lua separately.","dependencies":[{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-ra6","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-43t","depends_on_id":"tarantool-protobuf-wky","type":"blocks","created_at":"2026-05-17T19:33:44Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-z7x","title":"C accel: C table and string handling strategy","description":"Design and implement the low-level C helpers for Lua table access and string/buffer handling. Cache field-name strings/registry refs in descriptor plans, pre-size result tables where possible, build output strings with luaL_Buffer or exact-size allocation, avoid per-field Lua stack churn, and handle 64-bit cdata consistently with the current Lua API.","notes":"Architecture finalized in docs/c-accel.md. Strategy decisions validated by bench/c_accel/ spike: (1) Field-name strings cached as luaL_ref slots at plan-compile time; each decode does lua_rawgeti from the cached ref instead of re-interning. (2) Output buffer: 4KB stack-backed cap with malloc promotion on overflow (pattern in bench/c_accel/person_codec.c — validates against tiny-msg overhead and big-msg correctness). (3) Result tables pre-sized via lua_createtable(0, n_fields) from descriptor stats. (4) Per-field stack indices cached for repeated/packed array fields for the duration of decode_message — naive lazy-getfield was 2x slower at 100KB (spike Phase B). (5) 64-bit cdata via luaT_pushuint64/luaT_pushint64 (Tarantool extensions in module.h). (6) Sub-messages use separate buf with stack-allocated header; backpatching avoided (the bench showed sub-buf approach is fast enough).","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:53Z","created_by":"Eugene Blikh","updated_at":"2026-05-19T04:24:12Z","started_at":"2026-05-19T04:10:45Z","closed_at":"2026-05-19T04:24:12Z","close_reason":"C-side strategy formalized in docs/specs/c_accel_strategy.md. Spec covers: pb_plan / pb_plan_field struct layout, field-name luaL_ref caching (spike-validated), 4 KB stack-backed pb_buf with malloc promotion, lua_createtable(0, n_fields) result pre-sizing, cached per-field stack indices for repeated/packed (spike Phase B: 2x speedup at 100 KB), 64-bit cdata via luaT_pushint64/checkint64, sub-buffer (chosen) vs backpatching (deferred) for sub-messages, two-pass packed encode, map iteration via lua_next, oneof grouping, unknown field passthrough, deferred micro-opts (manual varint inlining, SIMD scan), C99/-O2 build environment, deferred PB_C_TRACE/PB_C_CHECK_PARITY env knobs. ra6/mq7 references this spec for low-level decisions.","dependency_count":0,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-47e","title":"C accel: preserve pure-Lua fallback and public API compatibility","description":"Define compatibility boundaries for C acceleration. Existing generated Lua modules and runtime pb APIs must keep working without a compiler or C module. New generated-C mode should be opt-in initially. Errors, 64-bit cdata behavior, WKT shapes, unknown fields, extension representation, lazy decode API, and text/JSON integration must remain compatible.","notes":"Architecture finalized in docs/c-accel.md. Concrete contract: (1) PB_ENABLE_C=1 env var is the only activation switch, default off. (2) require('pb') returns the same Lua surface in both modes — no API change. (3) 64-bit ints stay LuaJIT cdata (int64_t/uint64_t) in both modes. (4) WKT shapes unchanged. (5) Unknown fields round-trip identically. (6) Errors land as the same Lua error types. (7) Pure-Lua install must work without a C compiler — rockspec opt-builds the C module, install never fails on a host without cc. (8) When C module fails to load or PB_ENABLE_C is unset, runtime/pb/init.lua transparently uses the pure-Lua path.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-19T04:23:56Z","started_at":"2026-05-19T04:10:44Z","closed_at":"2026-05-19T04:23:56Z","close_reason":"Compatibility contract formalized in docs/specs/c_accel_compat.md. Spec covers: PB_ENABLE_C=1 single activation switch, silent fallback, full public surface preservation (pb.encode/decode/decode_lazy/parse/from_pb/json/text/grpc), generated module wrappers, 64-bit cdata invariant, WKT shapes, unknown fields, extensions, error compatibility, install paths (pure-Lua and C-enabled), ABI versioning, conformance via existing-suite re-run, explicit out-of-scope list. Lazy view stays Lua-only per spike rationale. Downstream tasks (mq7, ra6, 43t, c0i) reference this spec for compat boundaries.","dependency_count":0,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-wky","title":"C accel: build and packaging support for C modules","description":"Add build-system and packaging support for C acceleration. Needs Justfile targets, rockspec support for compiled modules, platform naming, local dev build, CI matrix integration, and fallback when the C module is absent. Generated C backend must not make pure-Lua install impossible unless explicitly selected.","notes":"Architecture finalized in docs/c-accel.md. Concrete deliverables: (1) Justfile recipe 'just build-c' that builds the C runtime module (mirror bench/c_accel/Makefile auto-detection of TT_INC). (2) Rockspec optionally builds the C module — install on host without cc must succeed and produce a pure-Lua install. (3) runtime/pb/init.lua does pcall(require, 'pb.c_runtime') only when os.getenv('PB_ENABLE_C') == '1'. (4) Sourcehut CI build manifest gains one extra job that runs the full test+conformance suite with PB_ENABLE_C=1 set — single .build.yml, both modes covered on every push to master. (5) CI also runs one variant without the C module compiled to confirm pure-Lua install works.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:52Z","created_by":"Eugene Blikh","updated_at":"2026-05-19T04:24:13Z","started_at":"2026-05-19T04:10:45Z","closed_at":"2026-05-19T04:24:13Z","close_reason":"Build and packaging scaffolding landed: docs/specs/c_accel_build_packaging.md spec, runtime/pb/init.lua PB_ENABLE_C=1 opt-in pcall hook (exposed as pb.c_runtime field), Justfile build-c/clean-c recipes (stub erroring cleanly until bd-ra6 lands runtime/pb/c/), Justfile lua_cpath constant + LUA_CPATH on test/test-one, .gitignore for compiled .so/.dylib, sourcehut .builds/{pure-lua,c-enabled}.yml manifests (ubuntu/noble, parallel jobs via separate files since sourcehut has no matrix), .sourcehut/conformance.yml for manual conformance triggers. Rockspec changes deferred to first C-source landing (described in spec). All 745 tests pass with PB_ENABLE_C unset and PB_ENABLE_C=1 (silent fallback verified).","dependencies":[{"issue_id":"tarantool-protobuf-wky","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:11Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-801","title":"Release: tagged releases + rockspec + Go plugin binaries","description":"Once the CI pipeline is green, ship tagged releases. Rockspec for the Lua runtime (publishable via tt rocks). Pre-built protoc-gen-tarantool binaries for darwin/arm64, darwin/amd64, linux/amd64, linux/arm64. Sourcecraft has 'sc release' tooling (see sc-release skill) for the release workflow itself.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:16Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:16Z","labels":["release"],"dependencies":[{"issue_id":"tarantool-protobuf-801","depends_on_id":"tarantool-protobuf-1eu","type":"blocks","created_at":"2026-05-17T18:47:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-1eu","title":"Release: Sourcecraft.dev project + CI pipeline","description":"Set up the canonical sourcecraft.dev project for the repo and a CI pipeline. Matrix: Tarantool 2.11 (CE+EE) and 3.x (CE+EE), Linux + macOS. Targets to run: just gen, just test, just bench-compare (alloc regression gate), just conformance (gated on the cached Docker image — see related CI wire-up bead). The repo lives under ~/data/home which by convention publishes to sourcecraft.dev (not github).","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:15Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:15Z","labels":["ci","release"],"dependencies":[{"issue_id":"tarantool-protobuf-1eu","depends_on_id":"tarantool-protobuf-7lf","type":"blocks","created_at":"2026-05-17T18:47:20Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-7lf","title":"CI: wire up conformance suite with cached Docker image","description":"The Docker image build (docker/conformance.Dockerfile) is the long pole at ~10-15 min on a clean cache. A registry push from a scheduled job would let CI runs reuse a warm image. Today the conformance suite runs locally via 'just conformance' but isn't gated on pushes. Goal: every push to master runs the binary+JSON and text-format suites; PRs run the same. Pre-requisite for M8 sourcecraft setup (this defines what the CI pipeline runs).","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:15Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:15Z","labels":["ci","conformance"],"dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-0an","title":"Profile the 1 KB decode cliff in hello.Person","description":"bench/bench.lua shows hello.Person decode dropping 260 MB/s @ 100 B to 128 MB/s @ 1 KB — per-MB throughput halves at the payload-shape transition (scalars-only -\u003e emails[] + nested address + packed lucky_numbers). starwing's C decoder scales smoothly across the same transition. Smoking gun: a trace abort or side-trace stitching failure at the array/nested branch. Capture jit.dump=tbim over the transition, identify the abort, fix it. Likely small change once located. Prereq for several decoder fixes — pin down what we're actually hitting first.","notes":"2026-05-17 inspection vs starwing/lua-protobuf 0.5.3 on Tarantool 3.8.0: reproduced starwing hello.Person encode at 75/688/888/1391/1431 MB/s and decode at 11.5/101.6/378/698.7/702.8 MB/s for 10B/100B/1KB/10KB/100KB. Current full mode is encode 30/257/287/541/643 MB/s and decode 30.7/247.5/128.8/186.4/184.3 MB/s. So we beat starwing on tiny decode, but lose 2.9-3.8x once Person switches to repeated strings + nested Address + packed ints. jit_trace passes including multi-byte varint paths; no fatal abort or bridge smoking gun. Focused probes on 930B Person: generated full decode ~6.9-7.5 us/op; scan tags+skip_field ~3.6 us/op; scan tags/lengths only ~2.1 us/op; hand order-specialized decoder with same string extraction/Address decode/lucky parse ~4.3 us/op. Patching decode_string to skip UTF-8 improves only ~16% (7.1 -\u003e 6.1 us/op), so UTF-8 is material but not the main cliff. Likely cause: per-field Lua dispatch/function-call/table growth overhead repeated across ~30 LEN/VARINT fields; C starwing parses from pointer/buffer and lands near our tag-only scan cost. Existing follow-ups bgu (ffi.string string decode), gcy (inline nested decode), and 0u1 (typed decoder byte fast paths) are relevant, but the largest remaining gap likely needs generated ordered/specialized decode loops or ptr+offset readers that avoid decode_tag/decode_len helper dispatch and substring slicing.","status":"closed","priority":2,"issue_type":"bug","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:12Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:02:54Z","started_at":"2026-05-17T15:56:33Z","closed_at":"2026-05-17T16:02:54Z","close_reason":"Profiled against starwing and local focused probes. No fatal JIT abort found; cliff attributed mainly to repeated Lua tag/field dispatch plus LEN/string slicing/validation. Follow-up fast-path work captured in tarantool-protobuf-4kj and existing decoder tasks.","labels":["decoder","investigation","perf"],"dependency_count":0,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-b12","title":"Encoder: parallel-array iteration for proto2 extensions","description":"Mirror the desc.oneofs_list finalize-time pattern: build desc.extensions_list once in pb.finalize_message, iterate with 'for i=1,#' in encode instead of walking the descriptor's hash table on every call. The 4-7x proto2 gap (vs 2-4x proto3 in bench/starwing_bench.lua) is partly explained by hash-walked extensions on the hot path. Expected: brings proto2 BenchPayload encode closer to proto3 Person encode at the same shape.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:12Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T19:00:34Z","started_at":"2026-05-24T18:59:15Z","closed_at":"2026-05-24T19:00:34Z","close_reason":"Already fixed in commit 2656c97 (2026-05-17, same day issue filed). register_extension appends to extensions_list (parallel array); codec.encode_message, text.emit_message, json.encode_message all iterate via 'for i=1,#elist'. Inline-mode codegen also walks the list via 'for _i = N+1, #_elist' for runtime-registered extensions past the statically-known set. No pairs() over extensions remains anywhere on the hot path. The issue was never closed in bd.","labels":["encoder","perf","proto2"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-bgu","title":"Decoder: ffi.string(ptr+off, len) for string/bytes decode","description":"Replace buf:sub(npos, npos+len-1) in decode_string/decode_bytes with ffi.string against a cached ffi.cast('const uint8_t*', buf). Removes method-call dispatch and lets the JIT fuse the read with surrounding code. Hits hardest on repeated-string fields (26 emails strings in 1KB Person -\u003e 26 sub calls today). Watch for LuaJIT string-intern collision behavior to stay identical to sub().","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:10Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T04:20:11Z","closed_at":"2026-05-24T04:20:11Z","close_reason":"Spike (bench/c_accel/bgu_probe.lua, removed) measured ffi.string vs buf:sub for the 1-byte LEN string fast path on Tarantool 3.8.0/LuaJIT 2.1.0-beta3 / Apple M-series. Results (ns/call):\n\n  size  buf:sub  per-call-cast  amortized/26  amortized/4\n  10B   89.7     280.5 (3.1x)   224.9 (2.5x)  246.2 (2.7x)\n  32B   107.6    299.4 (2.8x)   243.8 (2.3x)  260.9 (2.4x)\n  80B   154.3    362.4 (2.3x)   300.1 (1.9x)  314.9 (2.0x)\n\nV1 (per-call ffi.cast(U8CP, buf) inside decode_string): 2.3-3.1x slower.\nV2 (cast hoisted to caller, threaded as ptr arg, amortized over 26 emails — the Person 1KB peak shape): still 1.9-2.5x slower. Cast-once-amortize-many fails because 'ptr + np - 1' mints a fresh cdata wrapper per call that the JIT cannot sink across the decode_string frame — same root cause as a6n's failed top-of-_decode cast. The amortization helps a little (V2 \u003c V1) but does not close the gap.\n\nConclusion: bgu's premise (ffi.string lets the JIT fuse the string read with surrounding code) does not hold on the current LuaJIT fork — the cdata allocation cost dominates any reduction in dispatch. Closing as superseded by a6n-decode-inline-len-fast-path-2026-05-23. Revival criteria: a measurable cdata-sink win on a future LuaJIT (likely needs upstream allocation-sink improvements for pointer-arith cdata), OR an API redesign where the decoder works on a cdata buffer end-to-end (not a Lua string).","labels":["decoder","perf","wire"],"dependencies":[{"issue_id":"tarantool-protobuf-bgu","depends_on_id":"tarantool-protobuf-0an","type":"blocks","created_at":"2026-05-17T18:47:21Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-u39","title":"Decoder: emit table.new(0, N) for result tables","description":"Codegen knows the field count of every message at emission time. Emit table.new(0, N) instead of {} so the result table is sized correctly from the start. Cheap, broad win — especially closes the 1KB cliff where hello.Person rehashes as nested address + repeated fields populate. LuaJIT-only (table.new is in require('table.new')).","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:08Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T19:06:00Z","started_at":"2026-05-18T19:01:45Z","closed_at":"2026-05-18T19:06:00Z","close_reason":"Attempted and reverted. table_new(0, N) for decode result tables passed tests + JIT but regressed small-payload decode (10B -15.4%, 100B -17.7%) because the call cost exceeds rehash savings at small scale, and the small case doesn't reach the first rehash. Large-payload decode flat. Confirms drm bug: alloc is not the throughput bottleneck. See bench/PERF_LOG.md.","labels":["codegen","decoder","perf"],"dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-aah","title":"Encoder: codegen-emitted packed-scalar tight loops","description":"Replace per-element wire.encode_int32 calls with one inlined varint-emitting loop per packed field. Applies to packed int32, int64, sint32, sint64, bool, enum. The 1KB Person has 5 packed lucky_numbers + 26 emails — current per-element function-call boundary costs add up. Expected: 30%+ on packed-heavy payloads.","notes":"Shipped. work.lab.local 3-trial medians: packed_int32 +90/+113/+106%, packed_sint32 +66/+236/+156%, packed_uint32 +73/+105/+102%, packed_bool +44/+51/+39% (at 10/100/1000 elements). packed_int64 +22% (cdata path; fast path skipped, gain from table_new pre-sizing). Headline Person 1KB +3.6%, proto2 mid +10.5%.","status":"closed","priority":2,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:07Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T17:20:54Z","started_at":"2026-05-24T17:08:14Z","closed_at":"2026-05-24T17:20:54Z","labels":["codegen","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-4ql","title":"Encoder: optional caller-owned ffi.cdata ibuf API","description":"Separate API surface, not the default. ibuf-based encoder that writes into a caller-owned ffi.cdata buffer instead of returning a fresh Lua string. Targets hot RPC paths where the caller already owns a reusable buffer (e.g. net.box send path). Independent of the codegen rewrite — different API contract. Earlier attempt parked in stash@{0}; revisit after the codegen-time emission lands so we can compare apples-to-apples.","status":"closed","priority":2,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:06Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T04:30:27Z","closed_at":"2026-05-24T04:30:27Z","close_reason":"Spike (bench/c_accel/ibuf_probe.lua, removed) implemented a hand-coded Person_encode_ibuf mirroring what protoc-gen-tarantool would emit: stable pre-allocated cdata scratch buffer, direct p[i] byte writes, single-pass with backpatched length for nested Address + packed lucky_numbers, ffi.copy(p+i, lua_str, n) for strings. Same-byte correctness verified across 10B/100B/1KB/10KB.\n\nBench (ns/op, Tarantool 3.8.0 / Apple M-series), Person_encode vs ibuf scratch-only:\n\n  size    Person_encode  ibuf scratch  ibuf+ffi.string  speedup\n  10B     462.8          239.1         273.7            1.69x (win)\n  100B    470.1          243.1         286.3            1.64x (win)\n  1KB     3693.8         4872.6        5141.6           0.72x (loss)\n  10KB    17146          42807         43023            0.40x (loss)\n  100KB   159081         422510        432701           0.37x (loss)\n\nCrossover ~26 emails. Root cause: each email pays an ffi.copy(p+i, lua_str, n) boundary (~50 ns/call). At 26 emails = ~1.3 us pure boundary; at 2800 emails (100KB) = ~140 us pure boundary. Meanwhile Person_encode appends Lua-string refs to an out table (no FFI boundary) and pays ONE bulk table.concat memcpy at the end regardless of count. Per-field boundary work beats per-message bulk work only when field count is very small.\n\nFor 4ql's stated use case (net.box send path), typical Tarantool RPC payloads are \u003e=1 KB — exactly the regression zone (1.4-2.7x slower). Win window (\u003c100B) is too narrow to justify a separate API surface, especially since pb.encode is already 462-470ns at that size — saving 200ns on a sub-microsecond operation is not a meaningful net.box gain.\n\nThree abandoned attempts now (per-byte b:alloc cliff, v2 two-pass bulk reserve, this single-pass backpatch). All hit the same per-field FFI boundary tax. Closing as 'not viable on current LuaJIT'. REVIVAL CRITERIA: a future LuaJIT that can sink FFI calls into traces (so per-field ffi.copy stops paying the boundary cost), OR a use case where caller passes pre-cdata-cast strings (no per-email lua-string -\u003e cdata copy), OR a hand-written C encoder behind the c_runtime FFI surface (different path entirely — see c0i).","labels":["api","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-86g","title":"Encoder: two-pass with exact size precomputation","description":"Walk fields once to sum byte sizes, allocate the final string at exact size, walk again to write. Eliminates buffer-grow realloc; one lua_pushlstring. Same pattern as vtprotobuf's Size() + MarshalToVT. Stacks with the codegen-time inline writes (depends-on). Expected: additional 30-50% on large-message encode beyond the inline-FFI baseline.","notes":"2026-05-24 — verdict: not viable as a pure-Lua path. Same evidence as lkz close.\n\nHand-spike of the two-pass exact-size+emit shape (ffi.new uint8_t[?],\ninlined per-message sizer + writer, recursive) regressed encode 0.31x-0.77x\nacross all 5 size buckets vs the current `out` table + `table.concat` shape.\nProfile (jit.p) showed 84% of time in the inlined message bodies, not in\nbuffering — the perceived buffering inefficiency simply isn't there.\n\nThe single profile-hottest line (39% of Person_encode share) was replaced\nin 2ri (CHARS[_len] lookup) for +17%-34% encode throughput on 1KB+ payloads.\n\nBeating the pure-Lua ceiling requires escaping it — c0i (C-runtime backend)\nis the remaining path. Closing 86g as superseded.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:05Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T16:53:44Z","closed_at":"2026-05-24T16:53:44Z","labels":["codegen","encoder","perf"],"dependencies":[{"issue_id":"tarantool-protobuf-86g","depends_on_id":"tarantool-protobuf-h8v","type":"blocks","created_at":"2026-05-17T18:47:19Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-drm","title":"Inspect: ~120 B/op fixed encode/decode overhead in Lua hot path","description":"Cross-runtime bench (bench/COMPARISON.md) shows a fixed per-call\nallocation floor on both encode and decode in `mode=full`:\n\n- encode: 136 B/op even at 10 B payload (output is 10 bytes; ~126 B overhead)\n- decode: 112 B/op even at 10 B payload (top-level table = 2 hash-tables)\n\nFor comparison, Go vtproto runs encode at 16 B/op and decode at 8 B/op\non the same 10 B fixture.\n\nInspect:\n  1. What's the source of the 100+ B encode overhead? Suspect candidates:\n     - per-call buffer.ibuf workspace allocation in wire.lua\n     - Lua string concat building the result\n     - GCstr header on the output string itself (~24-32 B)\n  2. Decode floor: 112 B/op. Likely 2x table headers + small alloc for\n     the top-level message table.\n  3. For large payloads (100 KB encode = 131605 B/op vs output 96674 B)\n     the overhead is ~28 KB. Is that one big realloc tail, or many small?\n\nOutcome: a writeup pointing to specific lines and a recommendation\non whether the M6 ibuf path would actually help.","notes":"Findings from bench/alloc_probe.lua:\n\nENCODE 10B floor = 136 B/op is the `out` Lua table:\n- `local out, n = {}, 0` alone: 64 B (Lua GCtab base)\n- After 5 array entries: 136 B (matches the encode floor exactly)\n- table.concat: 0 B in the bench because output bytes intern\n- Varints: 0 B per call because 1-byte string.char outputs intern\n  globally (small string dedup)\n- 2-byte varints DO allocate: ~33 B per fresh value (encode_varint_slow\n  path returns a fresh string from string.char + bit.bor)\n\nDECODE 10B floor = 112 B/op is the top-level result table\n{name=..., age=...} — same shape across iters means it'd allocate the\nsame in a real workload.\n\nLARGE PAYLOAD (1KB Person):\n- same input (output interned): 1368 B/op\n- varying age (output unique): 2368 B/op\n- Delta = ~1000 B is the result string for 930 bytes of output (the\n  GCstr header + 930 content; the extra ~70 B is presumably padding /\n  alignment / the Address's nested concat).\n- So real cost per call has TWO components: (a) the small fixed tables\n  for the encoder workspace, (b) the output bytes themselves.\n\nKEY INSIGHT: bench numbers UNDERSTATE real allocation. The bench\niterates the SAME input → output string interns → bench reads only\nthe table cost. Real workloads where every message is unique pay\noutput-size + table-cost.\n\nOPTIMIZATION RANKING:\n1. Highest ROI: encoder workspace table. ~136 B per top-level encode\n   + ~136 B per nested message encode (Address adds its own). For a\n   1KB Person we have ~5 nested encoders → ~700 B of tables. Replacing\n   the array-of-string-pieces with a single growable `buffer.ibuf` cuts\n   this to ~0. This is the M6 ibuf path.\n2. Medium: encode_varint_slow returns a fresh string on each call for\n   non-fast-path values. Tag bytes are precomputed as literals; only\n   payload varints hit this. Inlining the slow path into wire.lua's\n   hot caller (or returning into a passed-in buffer) drops this.\n3. Lowest: result string. Unavoidable for the encoder's API contract\n   (returns a string). Only the lazy path avoids it.\n\nDECISION POINT: M6 ibuf path (already prototyped, deferred per\nmemory/decode_perf_deferred.md) is the right intervention.\nConservatively halves encode B/op on small messages, larger savings\non nested-heavy payloads.\nCORRECTION to earlier note.\n\nRecommendation to \"use M6 ibuf path to cut the 136 B/op floor\" was\nwrong. Per memory/tarantool_ibuf_perf.md (verified against current\nrepo state — feature NOT in HEAD):\n\n- Naive per-byte b:alloc(1): ~20× slower realistic, ~36× synthetic\n- Two-pass bulk-reserve: byte-equal correct, ~2× slower at every\n  payload size. Sitting in git stash@{0}.\n\nBoth prototypes lose on wall time because:\n  (a) Bench harness runs with jit.off, closure dispatch interpreted\n  (b) Realistic encoder cost is dominated by per-field closures\n      (sizer + writer + emit_tag + pwrite_*), NOT byte writes\n  (c) Two-pass adds an extra walk on top\n\nAlso: the 136 B/op encode floor is NOT the throughput bottleneck.\nCross-runtime gap (5× apiv2, 13-20× vtproto) is JIT/dispatch\noverhead per field, not allocator pressure. Cutting 136 B doesn't\nclose that gap.\n\nVIABLE PATHS (neither tried):\n1. Single-pass with backpatched length varints. Walk once. For nested\n   msgs: write tag + 1-byte placeholder, recurse, fill in (or memmove\n   if final length ≥ 128). Eliminates the size pass (~3 µs at 1 KB).\n2. Codegen-time `_encode_ibuf` per message in protoc-gen-tarantool\n   (mode=full). Straight-line ibuf writes, no descriptor walk at\n   runtime. Mirrors what pb.encode already does for the table path.\n   Probably the only approach that actually closes the throughput gap.\n\nEither has to keep per-field closure count ≤ pb.encode's writers,\notherwise we re-introduce the dispatch cost both stashed prototypes\nfell on.\n2026-05-24 re-verification (post-21d/qwt):\n\nAllocation floor unchanged. Fresh `tarantool bench/alloc_probe.lua`:\n  Person_encode 10B (same input)         136.0 B/op\n  Person_encode 100B (same input)        136.0 B/op\n  Person_decode 10B (same input)         112.0 B/op\n\nDrill-down isolates the source line-by-line — `local out, n = {}, 0` (64 B base) + 5 array entries (8 B each at LuaJIT tab growth = +72 B) = 136 B exactly. Lines: examples/expected/full/hello/hello_pb.lua:1044 (the `out` table) and 1049/1056/1061+ (the `n = n + 1; out[n] = ...` writes that populate it).\n\n21d closed dispatch fragmentation (small-encode +42% throughput per memory/21d-encode-dispatch-codegen-2026-05-24) and the alloc floor did not move — consistent with the CORRECTION note: the 5x cross-runtime throughput gap is not allocator pressure, and naive byte-buffer rewrites already lost in two stashed prototypes.\n\nInvestigation complete. Actionable interventions live in:\n- lkz (single-pass with backpatched length varints into one buffer)\n- 86g (two-pass with exact-size precompute; vtproto's Size+MarshalTo)\n\nBoth depend on closed h8v (codegen FFI direct writes). Closing drm.","status":"closed","priority":2,"issue_type":"bug","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:13:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T15:24:55Z","closed_at":"2026-05-24T15:24:55Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-58u","title":"Runtime-mode decode_unsafe support (compile parallel _reader_unsafe closures)","description":"6bb landed _decode_unsafe in full mode only. Runtime mode (pb.decode(desc, buf) and the runtime-mode codegen wrapper) currently has no unsafe path because compile_readers builds f._reader closures that capture handler.decode by value — a runtime swap of scalar.string.decode wouldn't reach them. Implementation sketch: add M.compile_readers_unsafe(desc) that builds f._reader_unsafe by passing an alternate scalar table where scalar.string = scalar.bytes. Call it from pb.finalize_message alongside compile_readers. Add M.decode_unsafe(desc, buf) as a near-clone of decode_message that uses f._reader_unsafe and routes the map-fallback scalar dispatch through the unsafe scalar table. Then emit M.\u003cName\u003e_decode_unsafe = function(b) return pb.decode_unsafe(M.\u003cName\u003e_descriptor, b) end in gen.go's runtime-mode emitMessageWrappers for API symmetry with full mode.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T19:11:20Z","started_at":"2026-05-24T18:59:16Z","closed_at":"2026-05-24T19:11:20Z","close_reason":"Runtime mode now exposes pb.decode_unsafe and M.\u003cName\u003e_decode_unsafe. codec.lua adds scalar_unsafe table, parameterizes build_reader/build_repeated_reader/decode_one to accept (scalar_tbl, decode_msg_fn, decode_group_fn), and adds compile_readers_unsafe + decode_message_unsafe + decode_group_unsafe + decode_extension_unsafe as literal clones with the three substitutions documented in codec.lua. Tests parameterized over both modes (14 cases), perf microbench shows ~8% gain in runtime mode (~20% in full mode). Conformance + JIT trace gates still pass. kyt remains open for unifying with C accel.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-kyt","title":"C runtime: skip_utf8_validation plan flag (unblock _decode_unsafe + C accel)","description":"6bb's full-mode _decode_unsafe skips the pb.c_runtime dispatch because runtime/pb/c/c_runtime.c calls is_valid_utf8 unconditionally on every string field. Result: when PB_ENABLE_C=1 the safe _decode wins on C but _decode_unsafe runs the inline Lua path and may be slower than C. To unify: add a skip_utf8_validation flag to the decode plan (or expose pb.c_runtime.decode_unsafe(plan, buf)), gate the is_valid_utf8 call on it in c_runtime.c, and wire _decode_unsafe to take the C path when c_runtime is available.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T19:48:24Z","started_at":"2026-05-24T19:41:09Z","closed_at":"2026-05-24T19:48:24Z","close_reason":"C runtime: decode_unsafe(plan, buf) entrypoint added; full-mode codegen _decode_unsafe dispatches via pb.c_runtime.decode_unsafe; pb.decode_unsafe in init.lua routes through C when c_runtime loaded. dec_ctx.skip_utf8 gates is_valid_utf8. Suites: test 766/766, test-c 1057/1057. Perf on string-heavy Person (418B, 16 emails + 16 nicknames): C-unsafe 455 MB/s vs C-safe 364 MB/s (+25%) vs Lua-unsafe 142 MB/s (3.2x).","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-h8x","title":"decode_group: 'inner loop in root trace' abort makes trace topology bimodal","description":"decode_group (runtime/pb/codec.lua:1089) has an 'inner loop in root trace' abort condition — the per-tag while-loop is hot enough to be a trace root itself but is reached from another root trace that tries to extend through it.\n\nObserved via bench/jit_trace.lua probe in isolation on full/WithGroup_decode (group):\n  Mode A (typical, ~9/10 runs): starts=38, stops=5, aborts=33  (recompile loop)\n  Mode B (rare, ~1/10 runs):    starts=102, stops=100, aborts=2  (side-trace cascade)\n\nPre-existing on master (probed before/after the compile_encode_body fix landed for tarantool-protobuf-21d). Neither mode breaks the gate (both have stops\u003e0, no FATAL aborts), but the bimodal behavior is unstable and confused 21d's measurements.\n\nFix direction: same idea as 21d — emit a generated per-descriptor decode body (or at minimum, refactor decode_group so the inner while-loop is a separate function the JIT can compile as its own root trace). Mirrors compile_encode_body from the 21d fix.","status":"closed","priority":3,"issue_type":"bug","owner":"bigbes@gmail.com","created_at":"2026-05-24T09:53:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T14:18:34Z","closed_at":"2026-05-24T14:18:34Z","close_reason":"wont_fix","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-3qu","title":"bench/*.lua: apply mcode arena hardening + re-snapshot baseline.json if numbers shift","description":"Sibling to 3o2 (closed). The trace gate had an intermittent 'fails silently with no JIT' mode on macOS arm64 caused by the default mcode arena being too small for our codegen footprint. The fix in 3o2 added jit.opt.start('sizemcode=64','maxmcode=4096') to bench/jit_trace.lua and 20/20 runs are now stable.\n\nThe other bench scripts have the same risk and none have the fix:\n  bench/bench.lua\n  bench/lazy_bench.lua\n  bench/profile.lua\n  bench/shapes_bench.lua\n  bench/starwing_bench.lua\n  bench/wire_bench.lua\n  bench/alloc_probe.lua\n\nThese scripts have larger codegen footprints than the trace gate (they require more modules, run for longer, and accumulate more traces), so they're MORE likely to hit the same intermittent JIT-fails-silently mode than the gate was. When that happens the script reports throughput that includes interpreter-only iterations — underreporting the real numbers without any diagnostic.\n\nConcrete steps:\n1) Add the jit.opt.start line to each script (same comment block as 3o2, or factor into a tiny bench/_setup.lua included from each).\n2) Re-run bench/bench.lua --baseline to refresh bench/baseline.json.\n3) Re-run bench/bench.lua --print and update the MB/s tables in bench/COMPARISON.md if any number moved \u003e5%.\n4) bench/COMPARISON.md notes 'Numbers will drift run-to-run by 5–10%' — verify that variance band shrinks after the fix.\n\nalloc_probe.lua doesn't need it (allocator counters don't depend on JIT), but adding the line costs nothing and keeps the bench/ scripts uniform.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:29:35Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:29:35Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-auj","title":"Quantify and address Person_decode multi-byte varint side-trace bridge (intermittent ~1/5 runs)","description":"Trace-topology finding from bench/jit_trace.lua. The 'full/Person_decode multi-byte varint' check intermittently reports one bridge — a side trace whose linktype=interpreter, costing interp dispatch per multi-byte tag/length-prefix on the hot decode path.\n\nCaptured output (1–2 runs out of 10):\n\n  [ OK ] full/Person_decode multi-byte varint  (stops=10, bridges=1)\n         info: bridge tr9  side-of tr5  hello_pb.lua:997  pc=55\n\ntr5 is the Person_decode while loop (entry at hello_pb.lua:997). pc=55 falls inside the wire.decode_tag inlined fast path: the guard 'if b \u003c 0x80' fails on a 2+ byte tag, exits to side trace tr9, which contains the multi-byte continuation loop but can't self-link back to the parent — drops to the interpreter to walk the rest of the dispatcher and re-enter on next iteration.\n\nExisting context:\n- wire.lua duplicates the 1-byte varint fast path at every hot decode call site precisely because LuaJIT side traces can't stitch returns from an inlined helper frame. That works for the 1-byte case. The 2+ byte case still calls decode_varint() (the multi-byte fallback) which has its own internal while loop.\n- gcy (inline nested decode) and kot (localize wire.* upvalues) are the structurally related items already filed; they don't cover this specific bridge though.\n\nWhy P3 (not P2):\n- Intermittent (~1/5 runs in the gate). The trace topology is mostly stable.\n- The multi-byte tag path is \u003c 5% of typical RPC payloads (field IDs 1..15 = 1-byte tag, lengths \u003c 128 = 1-byte length). Larger impact would require \u003e127-byte fields or field IDs \u003e= 16.\n- The 'multi-byte varint' fixture in bench/jit_trace.lua (200-byte name + lucky_numbers including 200000, 500000) was added specifically to expose this — and it does, intermittently. The intermittency is the JIT settling on different trace shapes across runs.\n\nConcrete approaches to investigate:\n1) Inline a 2-byte varint fast path inside decode_tag (and decode_string LEN prefix, etc.) — 'if b \u003c 0x80 then ... elseif b2 \u003c 0x80 then ...' — keeping the 3+ byte case in the fallback. Covers field IDs up to 4095 and length prefixes up to 16383, which is almost all real payloads.\n2) Profile-driven: run bench/jit_trace.lua 100x with a fixed seed, collect bridges by location, and decide whether the intermittency rate justifies (1) at all.\n\nAcceptance: 50 consecutive runs of bench/jit_trace.lua report bridges=0 for full/Person_decode multi-byte varint, OR a measured throughput improvement on the bench at the multi-byte-varint fixture.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:27:36Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T20:08:54Z","started_at":"2026-05-24T19:51:38Z","closed_at":"2026-05-24T20:08:54Z","close_reason":"Codegen: extended inline tag fast path (1-byte → 1+2-byte) and inline LEN fast path (1-byte → 1+2-byte) in emitInlineDecode / emitInlineStringBytesScalar / emitInlineStringBytesRepeated. Covers field IDs 1..4095 and string/bytes lengths 0..16383 without leaving the parent trace. Acceptance: criterion #2 met — multi-byte-varint Person decode 31.6→34.7 MB/s (+10%) on the big_person fixture (200B name + multi-byte packed lucky_numbers), small Person unchanged at 14 MB/s. Bridge rate now 4/100 (down from 10-20% at issue filing). Residual bridges have migrated from the tag site to the packed-varint inner loop (wire.decode_int32 in a packed payload); fully eliminating them would require per-scalar-type inline 2-byte paths in the packed-loop emitter — diminishing-returns territory, skipped. Suites: test 766/766, test-c 1057/1057.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-3e6","title":"Encoder: fiber-local recycled output buffer (Tarantool-specific)","description":"The 136 B/op encode floor is the 'local out, n = {}, 0' workspace table allocated per top-level encode call. Tarantool ships a per-fiber ibuf via require('buffer'); could grab one from a pool, reset it on each top-level encode, write into it, then ffi.string(ibuf.rpos, ibuf:size()) at the end.\n\nTradeoffs:\n  - Saves the workspace alloc (~136 B per top-level call + ~136 B per nested message).\n  - Adds fiber-local state — non-Tarantool LuaJIT runs would need a different path or none.\n  - Subsumed by lkz (single-pass two-phase) which goes directly to a single buffer without intermediate strings. Worth filing as a smaller alternative path in case lkz proves too invasive to land.\n  - drm notes record two prior ibuf prototypes (stashed) that regressed by ~2x because per-byte b:alloc(1) was 20x interpreter-dispatch-bound. This proposal avoids that pitfall only because it composes with h8v (codegen FFI direct writes that bypass alloc(1)).\n\nConcrete path: only useful in combination with h8v. Without h8v this would just shift the alloc from 'out table' to 'output string' without removing any per-string varint allocation in the field bodies — net neutral on alloc and worse on speed.","notes":"Sketch: shared scratch buffer per encoding, capacity exposed as ffi cdata pointer + length. Codegen emits 'local _buf = pb.codec.acquire_buf(); local _off = 0; ...; return pb.codec.finalize_buf(_buf, _off)'. acquire_buf returns a pre-allocated buffer of growing capacity; finalize_buf returns a string and recycles. Care needed for recursive encode calls (nested message encoding into the same buffer).","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:15:08Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:15:08Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-0gg","title":"Decoder: investigate field-id dispatch as binary search or jump table","description":"Generated Type_decode bodies use linear 'if id == 1 elseif id == 2 ...' chains. For Person (~15 fields) the chain is short and the comparisons cheap. For messages with 30+ fields the linear walk to a high-id tag costs N comparisons per occurrence.\n\nOpen question: does LuaJIT's IR already lower this to a switch/jump? If yes, no work needed. If no, three options:\n\n  1. Codegen-emit a balanced if-elseif tree (binary search) when field count exceeds threshold.\n  2. For messages where field IDs are dense and small, emit a numeric branch table: 'local _f = _dispatch[id]; if _f then return _f(buf, pos, result) end'. Closure-per-field has setup cost but avoids the dispatch cost on every tag.\n  3. Sort by frequency (impossible to know at codegen time without profiling input). Skip this option.\n\nPre-work: jit.dump on Person_decode to check whether LuaJIT collapses the if-elseif. If it does, close as won't-fix; if it doesn't, decide between (1) and (2).\n\nLowest priority of the perf items because: (a) we don't know it's a problem yet, (b) Person profile shows decode_string + decode_tag + utf8 dominate, not the if-elseif walk.","notes":"Recent profile (Person 1KB): line 1007 (decode_tag call) is 14% of generated body time, and line 1021 (list append) is 24%. Linear if-elseif walk doesn't show up as a hotspot for Person, but Person has only 15 fields. Needs a bigger fixture to manifest.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:15:07Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:15:07Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-1bo","title":"Codegen: chunk oversized generated decode bodies into per-range helpers","description":"test_messages_proto3_pb.lua is 4212 lines — the conformance fixture has so many fields that a single Type_decode body exceeds LuaJIT's inline budget and trace size limits. Symptom (to verify): bench/jit_trace.lua trace aborts on the big conformance message, despite no NYI bytecodes in the body.\n\nFix: when a message has \u003eN fields (threshold to find empirically; LuaJIT defaults LJ_TRACE_MAX_BC=8000), codegen splits the if-elseif chain into per-range helpers: _decode_fields_1_to_15, _decode_fields_16_to_31, etc. The top-level Type_decode dispatches to the right chunk by id range:\n\n  if id \u003c= 15 then\n    pos = _decode_fields_1_to_15(buf, pos, id, wt, result)\n  elseif id \u003c= 31 then\n    pos = _decode_fields_16_to_31(buf, pos, id, wt, result)\n  ...\n  end\n\nEach chunk is small enough to JIT-compile cleanly. Adds one function call per tag, but only on messages large enough to need it — small messages stay inline.\n\nThe range size and split threshold need measurement. Start with: split when message has \u003e32 fields, into chunks of 16. Measure throughput on conformance message before/after to validate the trade.","notes":"Pre-work: confirm the abort is real by running bench/jit_trace.lua against a conformance-like fixture and checking the FATAL traceerr codes. If LuaJIT compiles the 4212-line body fine, this is a non-issue.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:39Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:39Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-gi0","title":"Codegen: emit wire.encode_varint directly for int32/int64/uint32/uint64 instead of typed alias","description":"runtime/pb/wire.lua aliases M.encode_int32 = encode_varint (and same for int64/uint32/uint64). Generated code calls wire.encode_int32(v) which resolves to encode_varint through two table lookups — wire.encode_int32 (one hash lookup), then the alias resolution. LuaJIT may collapse this when the trace stays hot, but each break re-incurs both.\n\nCodegen can emit wire.encode_varint directly for the four unsigned-varint scalar types (int32/int64/uint32/uint64) since they're literally encode_varint with different names. Skips one alias indirection per varint encode. Pairs naturally with kot (localize wire.* upvalues) — together they reduce the call to a direct LJ_FUNCC dispatch with no name lookup.\n\nSame applies to bool (encode_bool = encode_varint with v and 1 or 0 wrapper) and sint32/sint64 (zigzag wrapper) — codegen could inline the wrapper logic at the call site for sint, but that overlaps with h8v (FFI direct writes) which subsumes the question.","notes":"Codegen-side change in protoc-gen-tarantool. Constraint: the alias provides the typed encoder slot in TYPE_INFO that runtime-mode encoders walk — those aliases must stay. Only the generated mode=full code changes.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:38Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-5y9","title":"Decoder: int64/uint64 return Lua number when value fits in 2^53","description":"decode_int64/decode_uint64 always return ffi cdata, forcing a fresh cdata allocation per call. For values in [-2^53, 2^53) (the common case for IDs, sequence numbers, timestamps fitting in 53 bits, byte counts, etc.) a Lua number is exactly representable and skips the cdata header allocation.\n\nProposed shape: opt-in variant decoder, since changing the default breaks any caller that does type(v)=='cdata' or relies on cdata-only operators. Two API options:\n\n  1. Per-descriptor flag (desc.int64_as_number = true) wired in via a codegen option or generator flag. Generated code emits a different decoder fn.\n  2. Separate typed decoders (wire.decode_int64_n / decode_uint64_n) that callers opt into explicitly.\n\nWatch case: values exceeding 2^53 must still return cdata (with a runtime branch). The branch cost only pays off if cdata allocation cost \u003e one comparison, which it is on hot paths.","notes":"Bench impact bounded by how many int64 fields the workload has. For hello.Person, user_id (fixed64) is the only one — so impact on this fixture would be ~1-2%. Bigger win on protobuf workloads dominated by timestamps and sequence numbers (datastore RPCs, log streams).","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:14:37Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:14:37Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-8k2","title":"Docs: remove PLAN.md after Beads migration","description":"PLAN.md has been converted from active roadmap to duplicate design-history prose now that all actionable work is tracked in Beads. Remove the file or replace remaining references with Beads/README pointers so project state has a single durable task source of truth.","status":"closed","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T18:33:09Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T18:33:48Z","closed_at":"2026-05-17T18:33:48Z","close_reason":"Removed PLAN.md and retargeted remaining references to Beads or concrete docs.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-c0i","title":"C accel: generated C codec backend for mode=full","description":"Extend protoc-gen-tarantool with an optional generated-C backend. Emit C source plus Lua wrappers for each .proto package; generated functions know tag bytes, field names, defaults, oneofs, maps, proto2 required/defaults/groups/extensions, and nested message calls. The Lua API remains M.Type_encode(t) -\u003e string and M.Type_decode(bytes) -\u003e table. Goal: one Lua-\u003eC call per top-level message, no Lua table.concat, no Lua decode_tag ladder, no per-field FFI boundary.","notes":"DEFERRED per docs/c-accel.md. Spike (04c) showed S4 hand-written C had ≤15% headroom over S3 (generic C runtime, ra6) and went the wrong way at scale (S4 *slower* than S3 at 10KB and 100KB encode). Codegen complexity not justified by current numbers. REVIVAL CRITERIA: a measured real-workload shape where ra6's per-field dispatch costs ≥25% over hand-written for that shape, demonstrated with a microbenchmark, AND the affected workload is on a hot path for a real user. Likely trigger shapes: wide messages with many optionals, heavy oneof use, complex maps, deeply nested (5+ levels) hierarchies. When triggered: write 2-3 paragraphs documenting shape and numbers, re-open this issue, scope narrowly via desc.encode/desc.decode override (same mechanism as WKT). If 6 months after ra6 ships no trigger fires, close as 'not justified'. Now blocked by ra6 (so it surfaces in ready list only after ra6 implementation lands).","status":"open","priority":3,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:32:32Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T20:19:41Z","dependencies":[{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-47e","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-pf6","type":"blocks","created_at":"2026-05-17T19:33:12Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-ra6","type":"blocks","created_at":"2026-05-18T23:19:44Z","created_by":"Eugene Blikh","metadata":"{}"},{"issue_id":"tarantool-protobuf-c0i","depends_on_id":"tarantool-protobuf-z7x","type":"blocks","created_at":"2026-05-17T19:33:29Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":4,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-bnj","title":"Tooling: add formatting targets for Go and Lua","description":"PLAN.md section 6 lists gofumpt and stylua. Add formatter configuration and Justfile targets, with generated code excluded unless the generator itself is intended to emit stylua-compliant output.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:28Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:28Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-dnb","title":"Tooling: add Go and Lua lint targets","description":"PLAN.md section 6 lists golangci-lint for Go and luacheck for Lua. Add configured lint targets and decide whether they run locally only or in CI. Keep generated files excluded where appropriate.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:28Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:28Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-6rb","title":"Tooling: add coverage targets for plugin and runtime","description":"PLAN.md section 6 lists go test -cover for the Go plugins and luacov for the Lua runtime. Add repeatable coverage targets and document expected use. Decide whether coverage is informational or gated in CI.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:27Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:27Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-3xh","title":"Docs: example apps (pet-clinic CRUD + queue replication)","description":"Two new runnable examples to live alongside the existing examples/grpc + examples/dynamic + examples/http set. (1) pet-clinic CRUD over gRPC — Greeter-shaped service exposing CRUD on a small entity (Pet, Owner). Covers unary + server-streaming list. (2) Replication of state via protobuf-encoded events on a queue — events serialized via pb.encode, dispatched via tarantool/queue, consumed and decoded by a follower. Each example must be hermetic per CLAUDE.md (memtx/wal/log in /tmp). Each gets a matching how-to in docs/howto/.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:18Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:18Z","labels":["docs","examples"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-b4j","title":"Docs: migration guide from built-in protobuf to pb","description":"Tarantool ships an encode-only require('protobuf') module. Document the migration path for users currently on the built-in. Cover: API surface differences (no decode in builtin; pb has encode+decode+JSON+text+gRPC+WKT), 64-bit cdata convention parity (both use cdata, no surprise), field-name conventions (both snake_case), the rename rationale (we couldn't override 'protobuf' without breaking existing users). Concrete before/after code samples.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:17Z","labels":["docs","release"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-o8n","title":"gRPC: reference network transports (http_server, netbox, http_client_unary)","description":"Ship reference transports as separate Tarantool modules (deferred from M4). Designs in docs/specs/grpc_transports.md. Three transports: (1) pb.grpc.transport.http_server — Connect-JSON over HTTP/1.1 via tarantool/http; default external transport; works with browsers and curl without an HTTP/2 proxy. (2) pb.grpc.transport.netbox — gRPC tunneled over net.box calls; first-class in-cluster path. (3) pb.grpc.transport.http_client_unary — outbound unary only via http_client. HTTP/2 termination explicitly out of scope; the transport contract is HTTP/2-shaped so the same generated code works behind Envoy. Conformance anchor: connectrpc/conformance harness (memory: connect_conformance_suite).","status":"open","priority":3,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:14Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:14Z","labels":["grpc","transport"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-2me","title":"Decide lazy-default for repeated string|bytes fields","description":"Open design question. For routing/proxy workloads that decode but don't iterate every element, repeated LEN fields could stay as unread ArrayViews until first access. Eager callers pay one extra table alloc; lazy callers skip N per-element string allocations. At 100KB Person, 26+ emails = 26+ string allocations deferrable. Decide: (a) flip default for repeated LEN, (b) gate behind a codegen mode/option, (c) leave eager as default and rely on _decode_lazy. Surface compatibility implications (callers iterate with ipairs today and expect a real array) before deciding.","status":"open","priority":3,"issue_type":"decision","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:13Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:13Z","labels":["decoder","design","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-0u1","title":"Decoder: decode_len-style byte-then-varint symmetry in typed decoders","description":"The single-byte peek shortcut in decode_len (wire.lua:410-419) isn't replicated in decode_int32 / decode_uint32 / decode_int64 / decode_uint64 / decode_bool / decode_enum, which always call decode_varint(). Mirror the fast path in each typed decoder so the dominant 1-byte varint case skips a function call. Expected: 5-10% on scalar-heavy decodes. Memory: wire_2byte_varint_cliff (same shape, encode side, already fixed).","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:11Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:11Z","labels":["decoder","perf","wire"],"dependencies":[{"issue_id":"tarantool-protobuf-0u1","depends_on_id":"tarantool-protobuf-0an","type":"blocks","created_at":"2026-05-17T18:47:21Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-a7l","title":"Encoder: static type-elision on scalars in mode=full","description":"wire.to_uint64(v) does a type(v) dispatch in three branches (number/cdata/boolean); in mode=full codegen the field's static type is known. Emit UINT64(v) / INT64(v) directly at the call site and let LuaJIT type-guard the trace. Expected: 10-15% on scalar-heavy messages.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:07Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:07Z","labels":["codegen","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-cma","title":"Encoder: inline zigzag_encode32 into encode_sint32 (one function call vs two)","description":"encode_sint32(v) = encode_varint(zigzag_encode32(v)) — two function calls. Inline as:\n\n  local function encode_sint32(v)\n      local n = bit.bxor(bit.lshift(v, 1), bit.arshift(v, 31))\n      -- existing 1-byte fast path\n      if n \u003e= 0 and n \u003c 0x80 then return string.char(n) end\n      return encode_varint_slow(n)\n  end\n\nSame for encode_sint64 (with the int64 zigzag formula).\n\nTiny win per sint encode — only matters for sint-heavy payloads. Filed for completeness; h8v makes this moot for mode=full (FFI direct writes inline the whole thing anyway). Real applicability: runtime mode encoders.","notes":"If h8v lands first, this becomes runtime-mode-only and may not be worth the duplication. Defer the call until h8v ships.","status":"open","priority":4,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:15:08Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:15:08Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-yvc","title":"Feature: gRPC server reflection service","description":"PLAN.md section 8 lists gRPC server reflection as a non-goal until after M5. M5 is complete, so keep this as deferred future work. Implement standard reflection once real transport/server use cases require it.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:29:18Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:29:18Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-0j5","title":"Feature: Tarantool 3.x config integration for schema registration","description":"PLAN.md section 7 mentions a pb.types.\u003cname\u003e declarative config role as a possible Tarantool 3.x integration. Define the use case first, then design how protobuf schemas/descriptors are declared, loaded, validated, and made available to application roles.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:29:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:29:17Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-41f","title":"Docs: Go plugin API documentation generation","description":"PLAN.md section 6 lists pkgsite for Go docs while Lua docs remain manually maintained Markdown. Decide what Go packages need public documentation and add a repeatable docs target or release note flow for pkgsite/sourcecraft publication.","status":"open","priority":4,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:29:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:29:17Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-5pu","title":"Feature: gRPC-Web / Connect conformance support","description":"PLAN.md section 8 lists gRPC-Web as not on the v1 path and notes it would be covered by the same Connect conformance harness if added. Track future decision and implementation scope separately from the current Connect-JSON/http_server transport work.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:29:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:29:17Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-bz6","title":"Decision: module path mapping for package-less protos","description":"PLAN.md section 7 notes the current behavior when lua_package and proto package are absent: use the bare filename. Decide whether to keep that behavior or error out to avoid unstable module paths. Update codegen docs and tests accordingly.","status":"open","priority":4,"issue_type":"decision","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:50Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-sty","title":"Feature: msgpack-flavored encoder for protobuf schemas","description":"PLAN.md section 7 references docs/specs/msgpack_encoding.md. Decide and implement, if still desired, a msgpack-flavored encoder driven by protobuf schemas. Open design points: map keyed by field number vs field name, ARRAY layout for box.space feeders, and MP_TUPLE extension opt-in.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:50Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-usk","title":"Decision: public C-FFI varint accelerator","description":"PLAN.md section 7 asks whether to expose a public C-FFI varint accelerator. Deferred until benchmarks prove pure-Lua varint code is the bottleneck. Capture benchmark threshold, API surface, and fallback behavior before implementation.","status":"open","priority":4,"issue_type":"decision","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:50Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-x9f","title":"Decision: deterministic map encoding option","description":"PLAN.md section 7 asks whether map encoder should be deterministic by sorting keys. Spec does not require deterministic map order, but some users may need byte-stable output. Decide API shape, likely pb.encode_deterministic or an encode option, and document perf/canonicalization tradeoffs.","status":"open","priority":4,"issue_type":"decision","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:50Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-oz5","title":"Testing: add malformed-input and random-valid fuzz harnesses","description":"PLAN.md section 5 lists fuzz harnesses as not yet automated. Add durable fuzz coverage for malformed wire inputs plus random-valid generated messages. Goal: catch decoder crashes, infinite loops, overlong varints, invalid LEN boundaries, unknown-field preservation bugs, and encode/decode round-trip mismatches beyond fixed fixtures.","status":"open","priority":4,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T16:28:27Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T16:28:27Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-xnn","title":"Protobuf editions support (deferred)","description":"We advertise FEATURE_PROTO3_OPTIONAL only, not FEATURE_SUPPORTS_EDITIONS. protoc errors out on edition='2023' files with our plugin today. See PLAN.md section 4.6 for the full design — editions replaces proto2-vs-proto3 with per-field FeatureSet annotations (field_presence, repeated_field_encoding, enum_type, utf8_validation, json_format). Practical win is modest: the only knob proto3 doesn't expose today is per-field packed opt-out. Revisit when (a) someone needs per-field packed control, (b) conformance suite gates on editions, or (c) protoc deprecates proto3 syntax hard enough that real users hit the error.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:19Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:19Z","labels":["deferred","editions"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-02o","title":"Proto2: MessageSet wire format (deferred)","description":"protobuf-go's protoreflect refuses to load a FileDescriptor declaring 'option message_set_wire_format = true;', calling it a removed proto1 feature. We strip the four MessageSet-flavored nested messages from our vendored test/conformance/proto/test_messages_proto2.proto so the rest of the schema compiles. Real support would require forking protoreflect; deferred until a real Tarantool consumer asks. Track here so we don't lose the context.","status":"open","priority":4,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:18Z","created_by":"Eugene Blikh","updated_at":"2026-05-17T15:47:18Z","labels":["deferred","proto2"],"dependency_count":0,"dependent_count":0,"comment_count":0}