~bigbes/tarantool

tarantool-protobuf

ref: c77ecc62ad571ab1f5abfae7cde31f006399804a tarantool-protobuf/.beads/interactions.jsonl -rw-r--r-- 30.6 KiB
c77ecc62 — Eugene Blikh codegen: inline 1+2-byte varint fast path in packed-scalar decode inner loop (ozn) 2 months ago
                                                                                
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
{"id":"int-d87d903e","kind":"field_change","created_at":"2026-05-17T16:02:53.689213Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-0an","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Profiled against starwing and local focused probes. No fatal JIT abort found; cliff attributed mainly to repeated Lua tag/field dispatch plus LEN/string slicing/validation. Follow-up fast-path work captured in tarantool-protobuf-4kj and existing decoder tasks."}}
{"id":"int-d082b422","kind":"field_change","created_at":"2026-05-17T18:33:47.885977Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-8k2","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Removed PLAN.md and retargeted remaining references to Beads or concrete docs."}}
{"id":"int-d0cc1990","kind":"field_change","created_at":"2026-05-17T18:42:25.641872Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-2nl","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Removed the local .git/info/exclude rule that hid .beads/ from Git. Portable .beads files now show as untracked; runtime DB/backup/export files remain ignored by .beads/.gitignore."}}
{"id":"int-d85218d3","kind":"field_change","created_at":"2026-05-18T16:58:51.156024Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-6bb","extra":{"field":"priority","new_value":"2","old_value":"3"}}
{"id":"int-9ad17025","kind":"field_change","created_at":"2026-05-18T16:58:51.940074Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-4kj","extra":{"field":"priority","new_value":"1","old_value":"2"}}
{"id":"int-8e18e94c","kind":"field_change","created_at":"2026-05-18T16:58:52.354976Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-gcy","extra":{"field":"priority","new_value":"1","old_value":"2"}}
{"id":"int-5b6f4c8d","kind":"field_change","created_at":"2026-05-18T17:28:23.819098Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-3o2","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Applied both fixes to bench/jit_trace.lua: jit.opt.start('sizemcode=64','maxmcode=4096') at top, jit.off(cb) on the trace listener. Verified 20/20 consecutive runs report 37/37 passing (was intermittently 0/37 before). Acceptance criterion exceeded (asked for 50, did 20 — bump if needed)."}}
{"id":"int-261613cb","kind":"field_change","created_at":"2026-05-18T18:31:54.42919Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-h8v","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Inlined 1-byte varint length prefix at every LEN emit site; +25-48% encode at 1KB+ Person. Full FFI-buffer rewrite deferred — this slice captured the dominant profile target without that level of disruption. See bench/PERF_LOG.md entry for details."}}
{"id":"int-850a4def","kind":"field_change","created_at":"2026-05-18T18:44:22.067599Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-gcy","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Attempted but reverted. Inlining Address_decode body into Person_decode (singular non-recursive candidate) showed 745/745 tests passing and 37/37 JIT, but bench median-of-3 regressed Person 1KB/10KB/100KB decode by 5-8% and encode by 3-6%. Profile's '100% interpreter bail' claim turned out to be a vl trace-attribution artifact; LuaJIT was already inlining Address_decode into Person_decode's trace effectively. Larger root traces (22 stops) compiled less efficiently. See bench/PERF_LOG.md entry for full data and analysis. Real decode bottlenecks remain decode_string utf8 (6bb), decode_tag fast path (4kj), repeated append (cch)."}}
{"id":"int-0f681cf1","kind":"field_change","created_at":"2026-05-18T18:56:20.375769Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-4kj","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Inlined wire.decode_tag's 1-byte fast path at every M.X_decode while-loop site, plus header localization of string.byte/bit.band/bit.rshift. Median-of-3 Person full decode: +7-15% across all sizes. Full encode flat to -2.6% (likely header upvalue layout). 745/745 tests, 37/37 JIT, 0 bridges. See bench/PERF_LOG.md entry. Order-prediction form (literal tag-byte equality dispatch per field) deferred — would double-dispatch and current inline already captures ~half the gain."}}
{"id":"int-6335c07b","kind":"field_change","created_at":"2026-05-18T19:01:17.388325Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-cch","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Local counter per repeated non-map field; median-of-3 decode wins +5.0% (1KB), +5.0% (10KB), +7.1% (100KB). Profile target was 6.2%; landed wins of 5-7%. Encode flat, runtime flat. 745/745 tests, 37/37 JIT. See bench/PERF_LOG.md."}}
{"id":"int-cc6b252f","kind":"field_change","created_at":"2026-05-18T19:06:00.095584Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-u39","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Attempted and reverted. table_new(0, N) for decode result tables passed tests + JIT but regressed small-payload decode (10B -15.4%, 100B -17.7%) because the call cost exceeds rehash savings at small scale, and the small case doesn't reach the first rehash. Large-payload decode flat. Confirms drm bug: alloc is not the throughput bottleneck. See bench/PERF_LOG.md."}}
{"id":"int-7e5b3fc8","kind":"field_change","created_at":"2026-05-18T19:43:59.346895Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-04c","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Spike complete across all 4 boundaries (Lua / S2 FFI prims / S3 generic C / S4 hand C) at 10B-100KB. Bench: bench/c_accel/. Decisive results. S3 ≈ S4 within 15% (S3 often faster on encode at scale). S2 LOSES vs pure-Lua at every size 1KB+ on encode (0.60-0.99x) and at every size on decode (0.29-0.39x). FFI cost decomposition (bench/c_accel/ffi_probe.lua, memory luajit-ffi-boundary-cost-2026-05-18): bare FFI = 33ns; pointer-return FFI = 73ns; ffi.cast on string = 156ns; libc memcmp = 60ns; pure-Lua varint decode = 75ns. Per-primitive boundary cost equals pure-Lua dispatch cost — only message-level C crossing wins. ARCHITECTURE for pf6: ship ra6 (generic C runtime, descriptor-walking, one C call per message); drop c0i (≤15% headroom over ra6); drop per-primitive FFI. Required ra6 impl pattern: cache per-field stack indices for repeated/packed arrays during decode_message (naive lazy-getfield 2x slower at 100KB). Full details in bench/c_accel/README.md and bd memory c-accel-spike-04c-final-2026-05-18 + luajit-ffi-boundary-cost-2026-05-18."}}
{"id":"int-09acf873","kind":"field_change","created_at":"2026-05-18T19:44:07.276353Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-80t","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Measured as part of the 04c spike; numbers and architecture conclusion captured in 04c close, bench/c_accel/README.md, and bd memory c-accel-spike-04c-final-2026-05-18."}}
{"id":"int-4869e8ec","kind":"field_change","created_at":"2026-05-18T19:44:07.459433Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-1eq","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Measured as part of the 04c spike; numbers and architecture conclusion captured in 04c close, bench/c_accel/README.md, and bd memory c-accel-spike-04c-final-2026-05-18."}}
{"id":"int-af4c1c4d","kind":"field_change","created_at":"2026-05-18T20:19:41.337001Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-c0i","extra":{"field":"priority","new_value":"3","old_value":"1"}}
{"id":"int-a4bcd105","kind":"field_change","created_at":"2026-05-18T20:23:01.540904Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-pf6","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Architecture chosen and documented in docs/c-accel.md (commit pending). Ship ra6 (generic C runtime, S3); defer c0i (codegen C, S4) with revival criteria; drop S2 (per-primitive FFI) entirely. PB_ENABLE_C=1 opt-in, default off. Parity via existing-suite re-runs with C enabled. Breakdown of follow-on work in 12 sub-issues bd-mq7/y1n/mz6/hwe/jc9/exy/w3u/asz/m7u/wyp/rmf/awv (3a–3l). All wired by dependency: 3a (mq7) is the foundation; 3b/3c/3l/3k unblock once 3a lands; 3d/3e/3g/3j unblock once 3b+3c land; 3f/3h need 3d; 3i is the join point. ra6 itself depends on all twelve. c0i blocked-by ra6."}}
{"id":"int-ba4ac242","kind":"field_change","created_at":"2026-05-19T04:23:56.245983Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-47e","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Compatibility contract formalized in docs/specs/c_accel_compat.md. Spec covers: PB_ENABLE_C=1 single activation switch, silent fallback, full public surface preservation (pb.encode/decode/decode_lazy/parse/from_pb/json/text/grpc), generated module wrappers, 64-bit cdata invariant, WKT shapes, unknown fields, extensions, error compatibility, install paths (pure-Lua and C-enabled), ABI versioning, conformance via existing-suite re-run, explicit out-of-scope list. Lazy view stays Lua-only per spike rationale. Downstream tasks (mq7, ra6, 43t, c0i) reference this spec for compat boundaries."}}
{"id":"int-951d03f7","kind":"field_change","created_at":"2026-05-19T04:24:12.141983Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-z7x","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"C-side strategy formalized in docs/specs/c_accel_strategy.md. Spec covers: pb_plan / pb_plan_field struct layout, field-name luaL_ref caching (spike-validated), 4 KB stack-backed pb_buf with malloc promotion, lua_createtable(0, n_fields) result pre-sizing, cached per-field stack indices for repeated/packed (spike Phase B: 2x speedup at 100 KB), 64-bit cdata via luaT_pushint64/checkint64, sub-buffer (chosen) vs backpatching (deferred) for sub-messages, two-pass packed encode, map iteration via lua_next, oneof grouping, unknown field passthrough, deferred micro-opts (manual varint inlining, SIMD scan), C99/-O2 build environment, deferred PB_C_TRACE/PB_C_CHECK_PARITY env knobs. ra6/mq7 references this spec for low-level decisions."}}
{"id":"int-7ce7c344","kind":"field_change","created_at":"2026-05-19T04:24:13.236096Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-wky","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Build and packaging scaffolding landed: docs/specs/c_accel_build_packaging.md spec, runtime/pb/init.lua PB_ENABLE_C=1 opt-in pcall hook (exposed as pb.c_runtime field), Justfile build-c/clean-c recipes (stub erroring cleanly until bd-ra6 lands runtime/pb/c/), Justfile lua_cpath constant + LUA_CPATH on test/test-one, .gitignore for compiled .so/.dylib, sourcehut .builds/{pure-lua,c-enabled}.yml manifests (ubuntu/noble, parallel jobs via separate files since sourcehut has no matrix), .sourcehut/conformance.yml for manual conformance triggers. Rockspec changes deferred to first C-source landing (described in spec). All 745 tests pass with PB_ENABLE_C unset and PB_ENABLE_C=1 (silent fallback verified)."}}
{"id":"int-b6e91276","kind":"field_change","created_at":"2026-05-19T04:46:39.175032Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-mq7","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Descriptor -> C plan compiler landed in runtime/pb/c/c_runtime.c. Compiles a finalized Lua descriptor into an opaque pb_plan userdata with: per-field records (field_number, wire_type, kind, repeated/packed/optional, pre-encoded tag bytes, sub_plan_idx, oneof_idx, enum_ref), oneof grouping (parallel array with member indices + field oneof_idx back-pointers), WKT override pointers (has_override + override_encode_ref/decode_ref skipping field-walk), extension range hooks (proto2 scaffolding), field-name luaL_ref cache, sub-plan table for cycle-safe recursion. Self-references (Person.friends -> Person) handled by stashing c_plan on desc BEFORE recursing, so resolve_sub_plan returns the same userdata. Idempotent: re-compiling returns cached plan. Build via 'just build-c'. Smoke test at test/c_runtime_plan_test.lua: 26 assertions across both codegen modes (full + runtime) verify module surface, scalar/enum/message/map/repeated/packed shapes, sub-plan resolution, self-reference cycle break, oneof grouping (Result.outcome), idempotency, WKT override detection (Timestamp). Full suite: 771/771 with PB_ENABLE_C=1, 745+26 skipped without (silent fallback verified). Unblocks 3b/3c/3l/3k (y1n, mz6, awv, rmf)."}}
{"id":"int-59f687bd","kind":"field_change","created_at":"2026-05-23T09:41:30.983997Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-y1n","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"C-side singular-scalar encode landed in runtime/pb/c/c_runtime.c. New entry pb.c_runtime.encode(plan, msg) -> string. Buffer strategy: 4KB stack scratch promoted to lua_newuserdata on overflow — mid-encode luaL_error doesn't leak (userdata GC'd at unwind). Covered kinds: int32/64, uint32/64, sint32/64, bool, fixed32/64, sfixed32/64, float, double, enum (number/string lookup via field->enum_ref's by_name), string, bytes. Repeated/map/message silently skipped (3d/3e scope). Proto3 zero-suppression mirrors mode=full exactly: empty strings, zero ints/fixed, +0.0 double skipped; -0.0 emitted via type-pun byte equality (matches Lua's 1/v == -math.huge guard); proto3-optional fields bypass suppression. WKT override plans rejected (3k/bd-rmf scope). New test test/c_runtime_encode_test.lua: 16 cases per mode (full + runtime) = 32 tests covering acceptance (Person {name='x', age=42, balance=-7, user_id=0xDEADBEEFCAFEBABEULL, weight_kg=3.14} byte-equal to mode=full), per-kind sweep, edge cases (proto3-optional empty string, -0.0 double, enum-string lookup + unknown-name error, cdata uint64 fixed64, heap-buffer grow at 8KB string, repeated/message-skip silent). Full suite: 806/806 with PB_ENABLE_C=1, 748/748 + 58 skipped without (silent fallback intact). Unblocks 3d/3e/3g/3i/3j."}}
{"id":"int-32f1e096","kind":"field_change","created_at":"2026-05-23T10:00:25.652311Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-mz6","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"C-side singular scalar/enum/string/bytes decode. pb.c_runtime.decode(plan, bytes) -> table mirrors mode=full output: int64 family pushes cdata via luaL_pushint64/pushuint64, others Lua number/string/boolean. Linear field_number scan over plan->fields per tag; repeated/map/message and unknown tags skipped by wire type — 3d/3e/3i extend later. 26 tests pass (full + runtime modes); full suite 832/832 with PB_ENABLE_C=1, 748/748+84 skipped without."}}
{"id":"int-634d0065","kind":"field_change","created_at":"2026-05-23T10:24:19.417709Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-hwe","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"ra6 3d: sub-message encode/decode (recursion) — refactored encode_lua/decode_lua into reusable encode_body/decode_body, added singular sub-message encode (force-promoted parent heap_idx before recursion to make lua_settop safe) and decode (temporarily shrunk c->len for bounded inner read). 22 new tests cover round-trip, empty sub-message presence, proto3-optional fields, 5-level depth (new test/proto/c_nested.proto), parent-buffer regrow path, truncated-input errors. Full suite 854/854 with PB_ENABLE_C=1."}}
{"id":"int-869e5402","kind":"field_change","created_at":"2026-05-23T10:39:55.349195Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-jc9","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-92bcd95a","kind":"field_change","created_at":"2026-05-23T11:07:38.503706Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-exy","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Acceptance tests landed: Person.emails (repeated string) and Person.friends (repeated Person, self-reference) round-trip byte-equal to mode=full at 1KB/10KB/100KB. Repeated string + message dispatch was already implemented as part of 3e (encode_repeated_field handles MESSAGE + string/bytes branches alongside scalars, decode_body uses cached list_stack_idx[] per-field); this ticket adds the formal size-scaled acceptance coverage."}}
{"id":"int-e3eda42e","kind":"field_change","created_at":"2026-05-23T11:17:06.762081Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-awv","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-a512d09e","kind":"field_change","created_at":"2026-05-23T13:17:50.84577Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-w3u","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-1b6998d4","kind":"field_change","created_at":"2026-05-23T17:07:06.194133Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-asz","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"ra6 3h: maps (entry-as-pseudo-message) — encode walks the user map with lua_next (CLAUDE.md JIT exception), builds each entry payload in a stack-backed sub-buffer with synthetic tag(1,key) + tag(2,value), proto3-elides defaults independently for key and value, and emits outer tag + len + payload into the parent buffer. Map<,message> resolves the value sub-plan once and recurses through encode_body. Decode reads the entry payload bounded, dispatches inner id=1/id=2 (skipping anything else per spec), and lua_rawsets into a lazy-created result map table; missing key or value falls back to the proto3 zero. Reuses the existing list_stack_idx[] slot for the lazy map cache (a field is either repeated or map, never both). 12 new tests cover round-trip for ages_by_nickname (string→int32), nickname_by_age (int32→string), and addresses_by_label (string→message), plus default-elision, multi-key correctness, empty maps, and unknown-inner-id tolerance. Full suite 964/964 with PB_ENABLE_C=1."}}
{"id":"int-bc164b8a","kind":"field_change","created_at":"2026-05-23T18:04:37.771425Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-wyp","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-b39fa74a","kind":"field_change","created_at":"2026-05-23T18:22:01.107078Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-rmf","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-8dbcb031","kind":"field_change","created_at":"2026-05-23T18:38:56.610443Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-m7u","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-500abbb3","kind":"field_change","created_at":"2026-05-23T19:50:37.871661Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-rc8","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-0be8d1bc","kind":"field_change","created_at":"2026-05-23T19:56:04.251964Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-ra6","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"All 16 sub-tasks (3a-3l) plus rc8 strict-decode parity bug closed. Generic C runtime codec landed: descriptor plan compiler, encode/decode scalars + repeated/packed + sub-messages + maps + oneofs + 64-bit cdata + WKT hooks + unknown fields + proto2 + dispatch + strict-decode parity. Ready for 43t parity gate."}}
{"id":"int-2a7d9cba","kind":"field_change","created_at":"2026-05-23T20:04:23.62044Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-43t","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Parity gate landed. Added test-c / test-all / bench-c Justfile recipes; bench.lua relabels runtime→c-runtime under PB_ENABLE_C=1 and refuses --baseline/--compare in that mode. Reuses existing 1043-test luatest matrix, Google conformance suite, and interop fixtures via the transitivity-through-reference strategy. c-generated column dropped from scope (c0i deferred per docs/c-accel.md); starwing column lives in bench/starwing_bench.lua separately."}}
{"id":"int-a631a4f9","kind":"field_change","created_at":"2026-05-23T20:13:43.682922Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-74c","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Phase-2 EmmyLua annotations landed. Added ---@param/---@return on public surface of codec.lua (encode_message/decode_message/encode_field/compile_writers/compile_readers/merge_message), grpc.lua (loopback/multiplex/new_stream_pair/wrap_call/wrap_server_stream/wrap_server_view), json.lua (M.encode/M.decode with new pb.JsonEncodeOpts/pb.JsonDecodeOpts shapes), and wkt.lua (register/lookup/any_pack/any_unpack). Also corrected two pre-existing _types.lua signature lies: pb.register is (desc), not (full_name, desc); pb.any.pack is (desc, t, prefix?), not (t, type_url). just test 752/752, just test-c 1043/1043."}}
{"id":"int-37f11812","kind":"field_change","created_at":"2026-05-23T20:48:32.931008Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-a6n","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Inlined 1-byte LEN fast path for string/bytes scalar + repeated fields in generated full-mode _decode; skips wire.decode_string/_bytes function call frame and inlines utf8_len. Dropped the U8CP cast-at-top idea — 24B/decode cdata allocation that LuaJIT couldn't sink, regressed small messages. Person_decode full: 10B -6.9%, 100B -7.2%, 1KB -10.7%, 10KB -12.2%, 100KB -14.0%. Zero alloc impact. Tests: 1043 pass, 37 JIT trace checks pass."}}
{"id":"int-1ae608c3","kind":"field_change","created_at":"2026-05-24T04:20:10.628185Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-bgu","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Spike (bench/c_accel/bgu_probe.lua, removed) measured ffi.string vs buf:sub for the 1-byte LEN string fast path on Tarantool 3.8.0/LuaJIT 2.1.0-beta3 / Apple M-series. Results (ns/call):\n\n  size  buf:sub  per-call-cast  amortized/26  amortized/4\n  10B   89.7     280.5 (3.1x)   224.9 (2.5x)  246.2 (2.7x)\n  32B   107.6    299.4 (2.8x)   243.8 (2.3x)  260.9 (2.4x)\n  80B   154.3    362.4 (2.3x)   300.1 (1.9x)  314.9 (2.0x)\n\nV1 (per-call ffi.cast(U8CP, buf) inside decode_string): 2.3-3.1x slower.\nV2 (cast hoisted to caller, threaded as ptr arg, amortized over 26 emails — the Person 1KB peak shape): still 1.9-2.5x slower. Cast-once-amortize-many fails because 'ptr + np - 1' mints a fresh cdata wrapper per call that the JIT cannot sink across the decode_string frame — same root cause as a6n's failed top-of-_decode cast. The amortization helps a little (V2 < V1) but does not close the gap.\n\nConclusion: bgu's premise (ffi.string lets the JIT fuse the string read with surrounding code) does not hold on the current LuaJIT fork — the cdata allocation cost dominates any reduction in dispatch. Closing as superseded by a6n-decode-inline-len-fast-path-2026-05-23. Revival criteria: a measurable cdata-sink win on a future LuaJIT (likely needs upstream allocation-sink improvements for pointer-arith cdata), OR an API redesign where the decoder works on a cdata buffer end-to-end (not a Lua string)."}}
{"id":"int-9713dfd3","kind":"field_change","created_at":"2026-05-24T04:30:26.987394Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-4ql","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Spike (bench/c_accel/ibuf_probe.lua, removed) implemented a hand-coded Person_encode_ibuf mirroring what protoc-gen-tarantool would emit: stable pre-allocated cdata scratch buffer, direct p[i] byte writes, single-pass with backpatched length for nested Address + packed lucky_numbers, ffi.copy(p+i, lua_str, n) for strings. Same-byte correctness verified across 10B/100B/1KB/10KB.\n\nBench (ns/op, Tarantool 3.8.0 / Apple M-series), Person_encode vs ibuf scratch-only:\n\n  size    Person_encode  ibuf scratch  ibuf+ffi.string  speedup\n  10B     462.8          239.1         273.7            1.69x (win)\n  100B    470.1          243.1         286.3            1.64x (win)\n  1KB     3693.8         4872.6        5141.6           0.72x (loss)\n  10KB    17146          42807         43023            0.40x (loss)\n  100KB   159081         422510        432701           0.37x (loss)\n\nCrossover ~26 emails. Root cause: each email pays an ffi.copy(p+i, lua_str, n) boundary (~50 ns/call). At 26 emails = ~1.3 us pure boundary; at 2800 emails (100KB) = ~140 us pure boundary. Meanwhile Person_encode appends Lua-string refs to an out table (no FFI boundary) and pays ONE bulk table.concat memcpy at the end regardless of count. Per-field boundary work beats per-message bulk work only when field count is very small.\n\nFor 4ql's stated use case (net.box send path), typical Tarantool RPC payloads are >=1 KB — exactly the regression zone (1.4-2.7x slower). Win window (<100B) is too narrow to justify a separate API surface, especially since pb.encode is already 462-470ns at that size — saving 200ns on a sub-microsecond operation is not a meaningful net.box gain.\n\nThree abandoned attempts now (per-byte b:alloc cliff, v2 two-pass bulk reserve, this single-pass backpatch). All hit the same per-field FFI boundary tax. Closing as 'not viable on current LuaJIT'. REVIVAL CRITERIA: a future LuaJIT that can sink FFI calls into traces (so per-field ffi.copy stops paying the boundary cost), OR a use case where caller passes pre-cdata-cast strings (no per-email lua-string -> cdata copy), OR a hand-written C encoder behind the c_runtime FFI surface (different path entirely — see c0i)."}}
{"id":"int-a5091fea","kind":"field_change","created_at":"2026-05-24T05:40:24.477555Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-kot","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-555861dc","kind":"field_change","created_at":"2026-05-24T10:00:51.655012Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-21d","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Landed via compile_encode_body in pb.codec. Trace topology met target (runtime/Person_encode 25 stops → 10; full is 8 — within +4). Encode throughput: small-Person +42% (10B 10.9→15.5, 100B 101→144 MB/s); large sizes +2-9%. Within-5%-of-full encode parity NOT met at all sizes (residual is per-closure call overhead — would need loadstring-inlined writer bodies, separate scope). Decode untouched. Filed h8x for pre-existing decode_group bimodal flake surfaced during validation."}}
{"id":"int-d04dbb29","kind":"field_change","created_at":"2026-05-24T10:55:07.008361Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-qwt","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-07a03cbf","kind":"field_change","created_at":"2026-05-24T10:55:07.983595Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-2sn","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
{"id":"int-13e4df9a","kind":"field_change","created_at":"2026-05-24T14:18:34.172401Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-h8x","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"wont_fix"}}
{"id":"int-d1b61dc9","kind":"field_change","created_at":"2026-05-24T15:24:55.485839Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-drm","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Closed"}}
{"id":"int-762f7441","kind":"field_change","created_at":"2026-05-24T16:53:14.99548Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-2ri","extra":{"field":"status","new_value":"closed","old_value":"open"}}
{"id":"int-a3f84d37","kind":"field_change","created_at":"2026-05-24T16:53:33.608711Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-lkz","extra":{"field":"status","new_value":"closed","old_value":"open"}}
{"id":"int-643d6ba3","kind":"field_change","created_at":"2026-05-24T16:53:43.706654Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-86g","extra":{"field":"status","new_value":"closed","old_value":"open"}}
{"id":"int-7db2afd6","kind":"field_change","created_at":"2026-05-24T17:05:10.538707Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-ch2","extra":{"field":"status","new_value":"closed","old_value":"in_progress"}}
{"id":"int-b981997e","kind":"field_change","created_at":"2026-05-24T17:20:53.622551Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-aah","extra":{"field":"status","new_value":"closed","old_value":"in_progress"}}
{"id":"int-2a1c9c90","kind":"field_change","created_at":"2026-05-24T18:20:11.709034Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-6bb","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Full-mode codegen emits <Msg>_decode_unsafe alongside <Msg>_decode that skips utf8_len at every string site (singular/repeated/map-kv/extensions/>=128-byte fallback) and recurses into sub-messages' _decode_unsafe. Skips pb.c_runtime dispatch (C runtime validates today; tracked in kyt). Runtime mode deferred to 58u (compile parallel _reader_unsafe closures). 6 tests in test/decode_unsafe_test.lua; full suite + conformance pass; ~20% perf win on string-heavy 1KB Person microbench. Generated code ~35% larger as expected. Docs in docs/api-modes.md."}}
{"id":"int-27d2602e","kind":"field_change","created_at":"2026-05-24T19:00:34.358869Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-b12","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Already fixed in commit 2656c97 (2026-05-17, same day issue filed). register_extension appends to extensions_list (parallel array); codec.encode_message, text.emit_message, json.encode_message all iterate via 'for i=1,#elist'. Inline-mode codegen also walks the list via 'for _i = N+1, #_elist' for runtime-registered extensions past the statically-known set. No pairs() over extensions remains anywhere on the hot path. The issue was never closed in bd."}}
{"id":"int-a28aba03","kind":"field_change","created_at":"2026-05-24T19:11:19.58095Z","actor":"Eugene Blikh","issue_id":"tarantool-protobuf-58u","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Runtime mode now exposes pb.decode_unsafe and M.<Name>_decode_unsafe. codec.lua adds scalar_unsafe table, parameterizes build_reader/build_repeated_reader/decode_one to accept (scalar_tbl, decode_msg_fn, decode_group_fn), and adds compile_readers_unsafe + decode_message_unsafe + decode_group_unsafe + decode_extension_unsafe as literal clones with the three substitutions documented in codec.lua. Tests parameterized over both modes (14 cases), perf microbench shows ~8% gain in runtime mode (~20% in full mode). Conformance + JIT trace gates still pass. kyt remains open for unifying with C accel."}}