M .beads/issues.jsonl => .beads/issues.jsonl +2 -2
@@ 48,8 48,8 @@
{"_type":"issue","id":"tarantool-protobuf-4ql","title":"Encoder: optional caller-owned ffi.cdata ibuf API","description":"Separate API surface, not the default. ibuf-based encoder that writes into a caller-owned ffi.cdata buffer instead of returning a fresh Lua string. Targets hot RPC paths where the caller already owns a reusable buffer (e.g. net.box send path). Independent of the codegen rewrite — different API contract. Earlier attempt parked in stash@{0}; revisit after the codegen-time emission lands so we can compare apples-to-apples.","status":"closed","priority":2,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:06Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T04:30:27Z","closed_at":"2026-05-24T04:30:27Z","close_reason":"Spike (bench/c_accel/ibuf_probe.lua, removed) implemented a hand-coded Person_encode_ibuf mirroring what protoc-gen-tarantool would emit: stable pre-allocated cdata scratch buffer, direct p[i] byte writes, single-pass with backpatched length for nested Address + packed lucky_numbers, ffi.copy(p+i, lua_str, n) for strings. Same-byte correctness verified across 10B/100B/1KB/10KB.\n\nBench (ns/op, Tarantool 3.8.0 / Apple M-series), Person_encode vs ibuf scratch-only:\n\n size Person_encode ibuf scratch ibuf+ffi.string speedup\n 10B 462.8 239.1 273.7 1.69x (win)\n 100B 470.1 243.1 286.3 1.64x (win)\n 1KB 3693.8 4872.6 5141.6 0.72x (loss)\n 10KB 17146 42807 43023 0.40x (loss)\n 100KB 159081 422510 432701 0.37x (loss)\n\nCrossover ~26 emails. Root cause: each email pays an ffi.copy(p+i, lua_str, n) boundary (~50 ns/call). At 26 emails = ~1.3 us pure boundary; at 2800 emails (100KB) = ~140 us pure boundary. Meanwhile Person_encode appends Lua-string refs to an out table (no FFI boundary) and pays ONE bulk table.concat memcpy at the end regardless of count. Per-field boundary work beats per-message bulk work only when field count is very small.\n\nFor 4ql's stated use case (net.box send path), typical Tarantool RPC payloads are \u003e=1 KB — exactly the regression zone (1.4-2.7x slower). Win window (\u003c100B) is too narrow to justify a separate API surface, especially since pb.encode is already 462-470ns at that size — saving 200ns on a sub-microsecond operation is not a meaningful net.box gain.\n\nThree abandoned attempts now (per-byte b:alloc cliff, v2 two-pass bulk reserve, this single-pass backpatch). All hit the same per-field FFI boundary tax. Closing as 'not viable on current LuaJIT'. REVIVAL CRITERIA: a future LuaJIT that can sink FFI calls into traces (so per-field ffi.copy stops paying the boundary cost), OR a use case where caller passes pre-cdata-cast strings (no per-email lua-string -\u003e cdata copy), OR a hand-written C encoder behind the c_runtime FFI surface (different path entirely — see c0i).","labels":["api","encoder","perf"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-86g","title":"Encoder: two-pass with exact size precomputation","description":"Walk fields once to sum byte sizes, allocate the final string at exact size, walk again to write. Eliminates buffer-grow realloc; one lua_pushlstring. Same pattern as vtprotobuf's Size() + MarshalToVT. Stacks with the codegen-time inline writes (depends-on). Expected: additional 30-50% on large-message encode beyond the inline-FFI baseline.","notes":"2026-05-24 — verdict: not viable as a pure-Lua path. Same evidence as lkz close.\n\nHand-spike of the two-pass exact-size+emit shape (ffi.new uint8_t[?],\ninlined per-message sizer + writer, recursive) regressed encode 0.31x-0.77x\nacross all 5 size buckets vs the current `out` table + `table.concat` shape.\nProfile (jit.p) showed 84% of time in the inlined message bodies, not in\nbuffering — the perceived buffering inefficiency simply isn't there.\n\nThe single profile-hottest line (39% of Person_encode share) was replaced\nin 2ri (CHARS[_len] lookup) for +17%-34% encode throughput on 1KB+ payloads.\n\nBeating the pure-Lua ceiling requires escaping it — c0i (C-runtime backend)\nis the remaining path. Closing 86g as superseded.","status":"closed","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:47:05Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T16:53:44Z","closed_at":"2026-05-24T16:53:44Z","labels":["codegen","encoder","perf"],"dependencies":[{"issue_id":"tarantool-protobuf-86g","depends_on_id":"tarantool-protobuf-h8v","type":"blocks","created_at":"2026-05-17T18:47:19Z","created_by":"Eugene Blikh","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-drm","title":"Inspect: ~120 B/op fixed encode/decode overhead in Lua hot path","description":"Cross-runtime bench (bench/COMPARISON.md) shows a fixed per-call\nallocation floor on both encode and decode in `mode=full`:\n\n- encode: 136 B/op even at 10 B payload (output is 10 bytes; ~126 B overhead)\n- decode: 112 B/op even at 10 B payload (top-level table = 2 hash-tables)\n\nFor comparison, Go vtproto runs encode at 16 B/op and decode at 8 B/op\non the same 10 B fixture.\n\nInspect:\n 1. What's the source of the 100+ B encode overhead? Suspect candidates:\n - per-call buffer.ibuf workspace allocation in wire.lua\n - Lua string concat building the result\n - GCstr header on the output string itself (~24-32 B)\n 2. Decode floor: 112 B/op. Likely 2x table headers + small alloc for\n the top-level message table.\n 3. For large payloads (100 KB encode = 131605 B/op vs output 96674 B)\n the overhead is ~28 KB. Is that one big realloc tail, or many small?\n\nOutcome: a writeup pointing to specific lines and a recommendation\non whether the M6 ibuf path would actually help.","notes":"Findings from bench/alloc_probe.lua:\n\nENCODE 10B floor = 136 B/op is the `out` Lua table:\n- `local out, n = {}, 0` alone: 64 B (Lua GCtab base)\n- After 5 array entries: 136 B (matches the encode floor exactly)\n- table.concat: 0 B in the bench because output bytes intern\n- Varints: 0 B per call because 1-byte string.char outputs intern\n globally (small string dedup)\n- 2-byte varints DO allocate: ~33 B per fresh value (encode_varint_slow\n path returns a fresh string from string.char + bit.bor)\n\nDECODE 10B floor = 112 B/op is the top-level result table\n{name=..., age=...} — same shape across iters means it'd allocate the\nsame in a real workload.\n\nLARGE PAYLOAD (1KB Person):\n- same input (output interned): 1368 B/op\n- varying age (output unique): 2368 B/op\n- Delta = ~1000 B is the result string for 930 bytes of output (the\n GCstr header + 930 content; the extra ~70 B is presumably padding /\n alignment / the Address's nested concat).\n- So real cost per call has TWO components: (a) the small fixed tables\n for the encoder workspace, (b) the output bytes themselves.\n\nKEY INSIGHT: bench numbers UNDERSTATE real allocation. The bench\niterates the SAME input → output string interns → bench reads only\nthe table cost. Real workloads where every message is unique pay\noutput-size + table-cost.\n\nOPTIMIZATION RANKING:\n1. Highest ROI: encoder workspace table. ~136 B per top-level encode\n + ~136 B per nested message encode (Address adds its own). For a\n 1KB Person we have ~5 nested encoders → ~700 B of tables. Replacing\n the array-of-string-pieces with a single growable `buffer.ibuf` cuts\n this to ~0. This is the M6 ibuf path.\n2. Medium: encode_varint_slow returns a fresh string on each call for\n non-fast-path values. Tag bytes are precomputed as literals; only\n payload varints hit this. Inlining the slow path into wire.lua's\n hot caller (or returning into a passed-in buffer) drops this.\n3. Lowest: result string. Unavoidable for the encoder's API contract\n (returns a string). Only the lazy path avoids it.\n\nDECISION POINT: M6 ibuf path (already prototyped, deferred per\nmemory/decode_perf_deferred.md) is the right intervention.\nConservatively halves encode B/op on small messages, larger savings\non nested-heavy payloads.\nCORRECTION to earlier note.\n\nRecommendation to \"use M6 ibuf path to cut the 136 B/op floor\" was\nwrong. Per memory/tarantool_ibuf_perf.md (verified against current\nrepo state — feature NOT in HEAD):\n\n- Naive per-byte b:alloc(1): ~20× slower realistic, ~36× synthetic\n- Two-pass bulk-reserve: byte-equal correct, ~2× slower at every\n payload size. Sitting in git stash@{0}.\n\nBoth prototypes lose on wall time because:\n (a) Bench harness runs with jit.off, closure dispatch interpreted\n (b) Realistic encoder cost is dominated by per-field closures\n (sizer + writer + emit_tag + pwrite_*), NOT byte writes\n (c) Two-pass adds an extra walk on top\n\nAlso: the 136 B/op encode floor is NOT the throughput bottleneck.\nCross-runtime gap (5× apiv2, 13-20× vtproto) is JIT/dispatch\noverhead per field, not allocator pressure. Cutting 136 B doesn't\nclose that gap.\n\nVIABLE PATHS (neither tried):\n1. Single-pass with backpatched length varints. Walk once. For nested\n msgs: write tag + 1-byte placeholder, recurse, fill in (or memmove\n if final length ≥ 128). Eliminates the size pass (~3 µs at 1 KB).\n2. Codegen-time `_encode_ibuf` per message in protoc-gen-tarantool\n (mode=full). Straight-line ibuf writes, no descriptor walk at\n runtime. Mirrors what pb.encode already does for the table path.\n Probably the only approach that actually closes the throughput gap.\n\nEither has to keep per-field closure count ≤ pb.encode's writers,\notherwise we re-introduce the dispatch cost both stashed prototypes\nfell on.\n2026-05-24 re-verification (post-21d/qwt):\n\nAllocation floor unchanged. Fresh `tarantool bench/alloc_probe.lua`:\n Person_encode 10B (same input) 136.0 B/op\n Person_encode 100B (same input) 136.0 B/op\n Person_decode 10B (same input) 112.0 B/op\n\nDrill-down isolates the source line-by-line — `local out, n = {}, 0` (64 B base) + 5 array entries (8 B each at LuaJIT tab growth = +72 B) = 136 B exactly. Lines: examples/expected/full/hello/hello_pb.lua:1044 (the `out` table) and 1049/1056/1061+ (the `n = n + 1; out[n] = ...` writes that populate it).\n\n21d closed dispatch fragmentation (small-encode +42% throughput per memory/21d-encode-dispatch-codegen-2026-05-24) and the alloc floor did not move — consistent with the CORRECTION note: the 5x cross-runtime throughput gap is not allocator pressure, and naive byte-buffer rewrites already lost in two stashed prototypes.\n\nInvestigation complete. Actionable interventions live in:\n- lkz (single-pass with backpatched length varints into one buffer)\n- 86g (two-pass with exact-size precompute; vtproto's Size+MarshalTo)\n\nBoth depend on closed h8v (codegen FFI direct writes). Closing drm.","status":"closed","priority":2,"issue_type":"bug","owner":"bigbes@gmail.com","created_at":"2026-05-17T15:13:38Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T15:24:55Z","closed_at":"2026-05-24T15:24:55Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-58u","title":"Runtime-mode decode_unsafe support (compile parallel _reader_unsafe closures)","description":"6bb landed _decode_unsafe in full mode only. Runtime mode (pb.decode(desc, buf) and the runtime-mode codegen wrapper) currently has no unsafe path because compile_readers builds f._reader closures that capture handler.decode by value — a runtime swap of scalar.string.decode wouldn't reach them. Implementation sketch: add M.compile_readers_unsafe(desc) that builds f._reader_unsafe by passing an alternate scalar table where scalar.string = scalar.bytes. Call it from pb.finalize_message alongside compile_readers. Add M.decode_unsafe(desc, buf) as a near-clone of decode_message that uses f._reader_unsafe and routes the map-fallback scalar dispatch through the unsafe scalar table. Then emit M.\u003cName\u003e_decode_unsafe = function(b) return pb.decode_unsafe(M.\u003cName\u003e_descriptor, b) end in gen.go's runtime-mode emitMessageWrappers for API symmetry with full mode.","status":"in_progress","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T18:59:16Z","started_at":"2026-05-24T18:59:16Z","dependency_count":0,"dependent_count":0,"comment_count":0}
-{"_type":"issue","id":"tarantool-protobuf-kyt","title":"C runtime: skip_utf8_validation plan flag (unblock _decode_unsafe + C accel)","description":"6bb's full-mode _decode_unsafe skips the pb.c_runtime dispatch because runtime/pb/c/c_runtime.c calls is_valid_utf8 unconditionally on every string field. Result: when PB_ENABLE_C=1 the safe _decode wins on C but _decode_unsafe runs the inline Lua path and may be slower than C. To unify: add a skip_utf8_validation flag to the decode plan (or expose pb.c_runtime.decode_unsafe(plan, buf)), gate the is_valid_utf8 call on it in c_runtime.c, and wire _decode_unsafe to take the C path when c_runtime is available.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T18:20:01Z","dependency_count":0,"dependent_count":0,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-58u","title":"Runtime-mode decode_unsafe support (compile parallel _reader_unsafe closures)","description":"6bb landed _decode_unsafe in full mode only. Runtime mode (pb.decode(desc, buf) and the runtime-mode codegen wrapper) currently has no unsafe path because compile_readers builds f._reader closures that capture handler.decode by value — a runtime swap of scalar.string.decode wouldn't reach them. Implementation sketch: add M.compile_readers_unsafe(desc) that builds f._reader_unsafe by passing an alternate scalar table where scalar.string = scalar.bytes. Call it from pb.finalize_message alongside compile_readers. Add M.decode_unsafe(desc, buf) as a near-clone of decode_message that uses f._reader_unsafe and routes the map-fallback scalar dispatch through the unsafe scalar table. Then emit M.\u003cName\u003e_decode_unsafe = function(b) return pb.decode_unsafe(M.\u003cName\u003e_descriptor, b) end in gen.go's runtime-mode emitMessageWrappers for API symmetry with full mode.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T19:11:20Z","started_at":"2026-05-24T18:59:16Z","closed_at":"2026-05-24T19:11:20Z","close_reason":"Runtime mode now exposes pb.decode_unsafe and M.\u003cName\u003e_decode_unsafe. codec.lua adds scalar_unsafe table, parameterizes build_reader/build_repeated_reader/decode_one to accept (scalar_tbl, decode_msg_fn, decode_group_fn), and adds compile_readers_unsafe + decode_message_unsafe + decode_group_unsafe + decode_extension_unsafe as literal clones with the three substitutions documented in codec.lua. Tests parameterized over both modes (14 cases), perf microbench shows ~8% gain in runtime mode (~20% in full mode). Conformance + JIT trace gates still pass. kyt remains open for unifying with C accel.","dependency_count":0,"dependent_count":0,"comment_count":0}
+{"_type":"issue","id":"tarantool-protobuf-kyt","title":"C runtime: skip_utf8_validation plan flag (unblock _decode_unsafe + C accel)","description":"6bb's full-mode _decode_unsafe skips the pb.c_runtime dispatch because runtime/pb/c/c_runtime.c calls is_valid_utf8 unconditionally on every string field. Result: when PB_ENABLE_C=1 the safe _decode wins on C but _decode_unsafe runs the inline Lua path and may be slower than C. To unify: add a skip_utf8_validation flag to the decode plan (or expose pb.c_runtime.decode_unsafe(plan, buf)), gate the is_valid_utf8 call on it in c_runtime.c, and wire _decode_unsafe to take the C path when c_runtime is available.","status":"closed","priority":3,"issue_type":"task","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-24T18:20:01Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T19:48:24Z","started_at":"2026-05-24T19:41:09Z","closed_at":"2026-05-24T19:48:24Z","close_reason":"C runtime: decode_unsafe(plan, buf) entrypoint added; full-mode codegen _decode_unsafe dispatches via pb.c_runtime.decode_unsafe; pb.decode_unsafe in init.lua routes through C when c_runtime loaded. dec_ctx.skip_utf8 gates is_valid_utf8. Suites: test 766/766, test-c 1057/1057. Perf on string-heavy Person (418B, 16 emails + 16 nicknames): C-unsafe 455 MB/s vs C-safe 364 MB/s (+25%) vs Lua-unsafe 142 MB/s (3.2x).","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-h8x","title":"decode_group: 'inner loop in root trace' abort makes trace topology bimodal","description":"decode_group (runtime/pb/codec.lua:1089) has an 'inner loop in root trace' abort condition — the per-tag while-loop is hot enough to be a trace root itself but is reached from another root trace that tries to extend through it.\n\nObserved via bench/jit_trace.lua probe in isolation on full/WithGroup_decode (group):\n Mode A (typical, ~9/10 runs): starts=38, stops=5, aborts=33 (recompile loop)\n Mode B (rare, ~1/10 runs): starts=102, stops=100, aborts=2 (side-trace cascade)\n\nPre-existing on master (probed before/after the compile_encode_body fix landed for tarantool-protobuf-21d). Neither mode breaks the gate (both have stops\u003e0, no FATAL aborts), but the bimodal behavior is unstable and confused 21d's measurements.\n\nFix direction: same idea as 21d — emit a generated per-descriptor decode body (or at minimum, refactor decode_group so the inner while-loop is a separate function the JIT can compile as its own root trace). Mirrors compile_encode_body from the 21d fix.","status":"closed","priority":3,"issue_type":"bug","owner":"bigbes@gmail.com","created_at":"2026-05-24T09:53:50Z","created_by":"Eugene Blikh","updated_at":"2026-05-24T14:18:34Z","closed_at":"2026-05-24T14:18:34Z","close_reason":"wont_fix","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-3qu","title":"bench/*.lua: apply mcode arena hardening + re-snapshot baseline.json if numbers shift","description":"Sibling to 3o2 (closed). The trace gate had an intermittent 'fails silently with no JIT' mode on macOS arm64 caused by the default mcode arena being too small for our codegen footprint. The fix in 3o2 added jit.opt.start('sizemcode=64','maxmcode=4096') to bench/jit_trace.lua and 20/20 runs are now stable.\n\nThe other bench scripts have the same risk and none have the fix:\n bench/bench.lua\n bench/lazy_bench.lua\n bench/profile.lua\n bench/shapes_bench.lua\n bench/starwing_bench.lua\n bench/wire_bench.lua\n bench/alloc_probe.lua\n\nThese scripts have larger codegen footprints than the trace gate (they require more modules, run for longer, and accumulate more traces), so they're MORE likely to hit the same intermittent JIT-fails-silently mode than the gate was. When that happens the script reports throughput that includes interpreter-only iterations — underreporting the real numbers without any diagnostic.\n\nConcrete steps:\n1) Add the jit.opt.start line to each script (same comment block as 3o2, or factor into a tiny bench/_setup.lua included from each).\n2) Re-run bench/bench.lua --baseline to refresh bench/baseline.json.\n3) Re-run bench/bench.lua --print and update the MB/s tables in bench/COMPARISON.md if any number moved \u003e5%.\n4) bench/COMPARISON.md notes 'Numbers will drift run-to-run by 5–10%' — verify that variance band shrinks after the fix.\n\nalloc_probe.lua doesn't need it (allocator counters don't depend on JIT), but adding the line costs nothing and keeps the bench/ scripts uniform.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:29:35Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:29:35Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"tarantool-protobuf-auj","title":"Quantify and address Person_decode multi-byte varint side-trace bridge (intermittent ~1/5 runs)","description":"Trace-topology finding from bench/jit_trace.lua. The 'full/Person_decode multi-byte varint' check intermittently reports one bridge — a side trace whose linktype=interpreter, costing interp dispatch per multi-byte tag/length-prefix on the hot decode path.\n\nCaptured output (1–2 runs out of 10):\n\n [ OK ] full/Person_decode multi-byte varint (stops=10, bridges=1)\n info: bridge tr9 side-of tr5 hello_pb.lua:997 pc=55\n\ntr5 is the Person_decode while loop (entry at hello_pb.lua:997). pc=55 falls inside the wire.decode_tag inlined fast path: the guard 'if b \u003c 0x80' fails on a 2+ byte tag, exits to side trace tr9, which contains the multi-byte continuation loop but can't self-link back to the parent — drops to the interpreter to walk the rest of the dispatcher and re-enter on next iteration.\n\nExisting context:\n- wire.lua duplicates the 1-byte varint fast path at every hot decode call site precisely because LuaJIT side traces can't stitch returns from an inlined helper frame. That works for the 1-byte case. The 2+ byte case still calls decode_varint() (the multi-byte fallback) which has its own internal while loop.\n- gcy (inline nested decode) and kot (localize wire.* upvalues) are the structurally related items already filed; they don't cover this specific bridge though.\n\nWhy P3 (not P2):\n- Intermittent (~1/5 runs in the gate). The trace topology is mostly stable.\n- The multi-byte tag path is \u003c 5% of typical RPC payloads (field IDs 1..15 = 1-byte tag, lengths \u003c 128 = 1-byte length). Larger impact would require \u003e127-byte fields or field IDs \u003e= 16.\n- The 'multi-byte varint' fixture in bench/jit_trace.lua (200-byte name + lucky_numbers including 200000, 500000) was added specifically to expose this — and it does, intermittently. The intermittency is the JIT settling on different trace shapes across runs.\n\nConcrete approaches to investigate:\n1) Inline a 2-byte varint fast path inside decode_tag (and decode_string LEN prefix, etc.) — 'if b \u003c 0x80 then ... elseif b2 \u003c 0x80 then ...' — keeping the 3+ byte case in the fallback. Covers field IDs up to 4095 and length prefixes up to 16383, which is almost all real payloads.\n2) Profile-driven: run bench/jit_trace.lua 100x with a fixed seed, collect bridges by location, and decide whether the intermittency rate justifies (1) at all.\n\nAcceptance: 50 consecutive runs of bench/jit_trace.lua report bridges=0 for full/Person_decode multi-byte varint, OR a measured throughput improvement on the bench at the multi-byte-varint fixture.","status":"open","priority":3,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-05-18T17:27:36Z","created_by":"Eugene Blikh","updated_at":"2026-05-18T17:27:36Z","dependency_count":0,"dependent_count":0,"comment_count":0}
M cmd/protoc-gen-tarantool/internal/gen/inline.go => cmd/protoc-gen-tarantool/internal/gen/inline.go +10 -10
@@ 815,24 815,24 @@ func emitInlineDecodeExtensionRepeated(w *writer, ext *protogen.Extension, full
// peer (typed RPC against our own encoder, JSON/text round-trips, in-process
// pipelines). The unsafe variant drops the utf8_len check at every string site
// and routes the >=128-byte fallback through wire.decode_bytes instead of
-// wire.decode_string. It also skips the C runtime dispatch because the C
-// runtime currently validates unconditionally (see runtime/pb/c/c_runtime.c
-// is_valid_utf8 use). (6bb)
+// wire.decode_string. Both variants dispatch to the C runtime when available;
+// the unsafe path calls c_runtime.decode_unsafe which gates is_valid_utf8 on
+// a per-call flag (kyt). (6bb)
func emitInlineDecode(w *writer, name string, m *protogen.Message, file *protogen.File, selfPath string, imports map[string]string, prefix string, exts []*protogen.Extension, validateUTF8 bool) {
suffix := "_decode"
+ cEntry := "decode"
if !validateUTF8 {
suffix = "_decode_unsafe"
+ cEntry = "decode_unsafe"
}
emitEmmyWrapperAnnotations(w, name, emmyMessageFullName(m), wrapperDecode)
emitLocalizedFunction(w, fmt.Sprintf("function M.%s%s(buf)", name, suffix), func() {
w.line(" local _d = M.%s_descriptor", name)
- if validateUTF8 {
- // C-acceleration: see emitInlineEncode for rationale and lazy-compile.
- w.line(" if pb.c_runtime ~= nil then")
- w.line(" local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)")
- w.line(" return pb.c_runtime.decode(_p, buf)")
- w.line(" end")
- }
+ // C-acceleration: see emitInlineEncode for rationale and lazy-compile.
+ w.line(" if pb.c_runtime ~= nil then")
+ w.line(" local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)")
+ w.line(" return pb.c_runtime.%s(_p, buf)", cEntry)
+ w.line(" end")
w.line(" if type(buf) ~= 'string' then")
w.line(" error(\"expected string for %s decode, got \" .. type(buf), 0)", m.Desc.FullName())
w.line(" end")
M examples/expected/full/c_int64/c_int64_pb.lua => examples/expected/full/c_int64/c_int64_pb.lua +4 -0
@@ 165,6 165,10 @@ end
---@return c_int64.Wide
function M.Wide_decode_unsafe(buf)
local _d = M.Wide_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_int64.Wide decode, got " .. type(buf), 0)
end
M examples/expected/full/c_nested/c_nested_pb.lua => examples/expected/full/c_nested/c_nested_pb.lua +20 -0
@@ 203,6 203,10 @@ end
---@return c_nested.L1
function M.L1_decode_unsafe(buf)
local _d = M.L1_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_nested.L1 decode, got " .. type(buf), 0)
end
@@ 360,6 364,10 @@ end
---@return c_nested.L2
function M.L2_decode_unsafe(buf)
local _d = M.L2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_nested.L2 decode, got " .. type(buf), 0)
end
@@ 517,6 525,10 @@ end
---@return c_nested.L3
function M.L3_decode_unsafe(buf)
local _d = M.L3_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_nested.L3 decode, got " .. type(buf), 0)
end
@@ 674,6 686,10 @@ end
---@return c_nested.L4
function M.L4_decode_unsafe(buf)
local _d = M.L4_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_nested.L4 decode, got " .. type(buf), 0)
end
@@ 809,6 825,10 @@ end
---@return c_nested.L5
function M.L5_decode_unsafe(buf)
local _d = M.L5_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_nested.L5 decode, got " .. type(buf), 0)
end
M examples/expected/full/c_repeated/c_repeated_pb.lua => examples/expected/full/c_repeated/c_repeated_pb.lua +8 -0
@@ 198,6 198,10 @@ end
---@return c_repeated.Inner
function M.Inner_decode_unsafe(buf)
local _d = M.Inner_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_repeated.Inner decode, got " .. type(buf), 0)
end
@@ 892,6 896,10 @@ function M.Holder_decode_unsafe(buf)
local decode_sint32 = wire.decode_sint32
local decode_uint32 = wire.decode_uint32
local _d = M.Holder_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for c_repeated.Holder decode, got " .. type(buf), 0)
end
M examples/expected/full/conformance/conformance_pb.lua => examples/expected/full/conformance/conformance_pb.lua +20 -0
@@ 351,6 351,10 @@ end
function M.TestStatus_decode_unsafe(buf)
local decode_bytes = wire.decode_bytes
local _d = M.TestStatus_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for conformance.TestStatus decode, got " .. type(buf), 0)
end
@@ 528,6 532,10 @@ end
---@return conformance.FailureSet
function M.FailureSet_decode_unsafe(buf)
local _d = M.FailureSet_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for conformance.FailureSet decode, got " .. type(buf), 0)
end
@@ 872,6 880,10 @@ function M.ConformanceRequest_decode_unsafe(buf)
local decode_varint = wire.decode_varint
local varint_to_int32 = wire.varint_to_int32
local _d = M.ConformanceRequest_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for conformance.ConformanceRequest decode, got " .. type(buf), 0)
end
@@ 1414,6 1426,10 @@ end
function M.ConformanceResponse_decode_unsafe(buf)
local decode_bytes = wire.decode_bytes
local _d = M.ConformanceResponse_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for conformance.ConformanceResponse decode, got " .. type(buf), 0)
end
@@ 1734,6 1750,10 @@ end
---@return conformance.JspbEncodingConfig
function M.JspbEncodingConfig_decode_unsafe(buf)
local _d = M.JspbEncodingConfig_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for conformance.JspbEncodingConfig decode, got " .. type(buf), 0)
end
M examples/expected/full/hello/hello_pb.lua => examples/expected/full/hello/hello_pb.lua +24 -0
@@ 361,6 361,10 @@ end
function M.Result_decode_unsafe(buf)
local decode_int32 = wire.decode_int32
local _d = M.Result_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for hello.Result decode, got " .. type(buf), 0)
end
@@ 537,6 541,10 @@ end
---@return hello.HelloRequest
function M.HelloRequest_decode_unsafe(buf)
local _d = M.HelloRequest_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for hello.HelloRequest decode, got " .. type(buf), 0)
end
@@ 690,6 698,10 @@ end
---@return hello.HelloReply
function M.HelloReply_decode_unsafe(buf)
local _d = M.HelloReply_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for hello.HelloReply decode, got " .. type(buf), 0)
end
@@ 1033,6 1045,10 @@ end
function M.Event_decode_unsafe(buf)
local decode_len = wire.decode_len
local _d = M.Event_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for hello.Event decode, got " .. type(buf), 0)
end
@@ 1297,6 1313,10 @@ end
function M.Address_decode_unsafe(buf)
local decode_bytes = wire.decode_bytes
local _d = M.Address_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for hello.Address decode, got " .. type(buf), 0)
end
@@ 1839,6 1859,10 @@ function M.Person_decode_unsafe(buf)
local decode_tag = wire.decode_tag
local skip_field = wire.skip_field
local _d = M.Person_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for hello.Person decode, got " .. type(buf), 0)
end
M examples/expected/full/proto2_basic/proto2_basic_pb.lua => examples/expected/full/proto2_basic/proto2_basic_pb.lua +40 -0
@@ 436,6 436,10 @@ end
function M.Defaults_decode_unsafe(buf)
local decode_bytes = wire.decode_bytes
local _d = M.Defaults_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.Defaults decode, got " .. type(buf), 0)
end
@@ 776,6 780,10 @@ function M.Cardinality_decode_unsafe(buf)
local decode_int32 = wire.decode_int32
local decode_len = wire.decode_len
local _d = M.Cardinality_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.Cardinality decode, got " .. type(buf), 0)
end
@@ 1009,6 1017,10 @@ end
function M.Nested_decode_unsafe(buf)
local decode_len = wire.decode_len
local _d = M.Nested_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.Nested decode, got " .. type(buf), 0)
end
@@ 1155,6 1167,10 @@ end
---@return proto2_basic.Nested.Inner
function M.Nested_Inner_decode_unsafe(buf)
local _d = M.Nested_Inner_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.Nested.Inner decode, got " .. type(buf), 0)
end
@@ 1305,6 1321,10 @@ end
---@return proto2_basic.WithGroup
function M.WithGroup_decode_unsafe(buf)
local _d = M.WithGroup_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.WithGroup decode, got " .. type(buf), 0)
end
@@ 1475,6 1495,10 @@ end
---@return proto2_basic.WithGroup.SingleGroup
function M.WithGroup_SingleGroup_decode_unsafe(buf)
local _d = M.WithGroup_SingleGroup_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.WithGroup.SingleGroup decode, got " .. type(buf), 0)
end
@@ 1625,6 1649,10 @@ end
---@return proto2_basic.WithGroup.RepGroup
function M.WithGroup_RepGroup_decode_unsafe(buf)
local _d = M.WithGroup_RepGroup_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.WithGroup.RepGroup decode, got " .. type(buf), 0)
end
@@ 1965,6 1993,10 @@ function M.BenchPayload_decode_unsafe(buf)
local decode_int32 = wire.decode_int32
local decode_len = wire.decode_len
local _d = M.BenchPayload_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.BenchPayload decode, got " .. type(buf), 0)
end
@@ 2209,6 2241,10 @@ end
function M.BenchPayload_Stats_decode_unsafe(buf)
local decode_int32 = wire.decode_int32
local _d = M.BenchPayload_Stats_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.BenchPayload.Stats decode, got " .. type(buf), 0)
end
@@ 2377,6 2413,10 @@ end
---@return proto2_basic.BenchPayload.Inner
function M.BenchPayload_Inner_decode_unsafe(buf)
local _d = M.BenchPayload_Inner_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for proto2_basic.BenchPayload.Inner decode, got " .. type(buf), 0)
end
M examples/expected/full/protobuf_test_messages/proto2/test_messages_proto2_pb.lua => examples/expected/full/protobuf_test_messages/proto2/test_messages_proto2_pb.lua +84 -0
@@ 4665,6 4665,10 @@ function M.TestAllTypesProto2_decode_unsafe(buf)
local skip_field = wire.skip_field
local varint_to_int32 = wire.varint_to_int32
local _d = M.TestAllTypesProto2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestAllTypesProto2 decode, got " .. type(buf), 0)
end
@@ 6930,6 6934,10 @@ end
---@return protobuf_test_messages.proto2.TestAllTypesProto2.NestedMessage
function M.TestAllTypesProto2_NestedMessage_decode_unsafe(buf)
local _d = M.TestAllTypesProto2_NestedMessage_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestAllTypesProto2.NestedMessage decode, got " .. type(buf), 0)
end
@@ 7085,6 7093,10 @@ end
---@return protobuf_test_messages.proto2.TestAllTypesProto2.Data
function M.TestAllTypesProto2_Data_decode_unsafe(buf)
local _d = M.TestAllTypesProto2_Data_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestAllTypesProto2.Data decode, got " .. type(buf), 0)
end
@@ 7235,6 7247,10 @@ end
---@return protobuf_test_messages.proto2.TestAllTypesProto2.MultiWordGroupField
function M.TestAllTypesProto2_MultiWordGroupField_decode_unsafe(buf)
local _d = M.TestAllTypesProto2_MultiWordGroupField_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestAllTypesProto2.MultiWordGroupField decode, got " .. type(buf), 0)
end
@@ 7375,6 7391,10 @@ end
---@return protobuf_test_messages.proto2.ForeignMessageProto2
function M.ForeignMessageProto2_decode_unsafe(buf)
local _d = M.ForeignMessageProto2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.ForeignMessageProto2 decode, got " .. type(buf), 0)
end
@@ 7516,6 7536,10 @@ end
---@return protobuf_test_messages.proto2.GroupField
function M.GroupField_decode_unsafe(buf)
local _d = M.GroupField_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.GroupField decode, got " .. type(buf), 0)
end
@@ 7766,6 7790,10 @@ function M.UnknownToTestAllTypes_decode_unsafe(buf)
local decode_int32 = wire.decode_int32
local decode_len = wire.decode_len
local _d = M.UnknownToTestAllTypes_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.UnknownToTestAllTypes decode, got " .. type(buf), 0)
end
@@ 7973,6 8001,10 @@ end
---@return protobuf_test_messages.proto2.UnknownToTestAllTypes.OptionalGroup
function M.UnknownToTestAllTypes_OptionalGroup_decode_unsafe(buf)
local _d = M.UnknownToTestAllTypes_OptionalGroup_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.UnknownToTestAllTypes.OptionalGroup decode, got " .. type(buf), 0)
end
@@ 8095,6 8127,10 @@ end
---@return protobuf_test_messages.proto2.NullHypothesisProto2
function M.NullHypothesisProto2_decode_unsafe(buf)
local _d = M.NullHypothesisProto2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.NullHypothesisProto2 decode, got " .. type(buf), 0)
end
@@ 8209,6 8245,10 @@ end
---@return protobuf_test_messages.proto2.EnumOnlyProto2
function M.EnumOnlyProto2_decode_unsafe(buf)
local _d = M.EnumOnlyProto2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.EnumOnlyProto2 decode, got " .. type(buf), 0)
end
@@ 8349,6 8389,10 @@ end
---@return protobuf_test_messages.proto2.OneStringProto2
function M.OneStringProto2_decode_unsafe(buf)
local _d = M.OneStringProto2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.OneStringProto2 decode, got " .. type(buf), 0)
end
@@ 8554,6 8598,10 @@ end
function M.ProtoWithKeywords_decode_unsafe(buf)
local decode_bytes = wire.decode_bytes
local _d = M.ProtoWithKeywords_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.ProtoWithKeywords decode, got " .. type(buf), 0)
end
@@ 9362,6 9410,10 @@ function M.TestAllRequiredTypesProto2_decode_unsafe(buf)
local decode_varint = wire.decode_varint
local varint_to_int32 = wire.varint_to_int32
local _d = M.TestAllRequiredTypesProto2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestAllRequiredTypesProto2 decode, got " .. type(buf), 0)
end
@@ 9779,6 9831,10 @@ end
function M.TestAllRequiredTypesProto2_NestedMessage_decode_unsafe(buf)
local decode_len = wire.decode_len
local _d = M.TestAllRequiredTypesProto2_NestedMessage_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestAllRequiredTypesProto2.NestedMessage decode, got " .. type(buf), 0)
end
@@ 9940,6 9996,10 @@ end
---@return protobuf_test_messages.proto2.TestAllRequiredTypesProto2.Data
function M.TestAllRequiredTypesProto2_Data_decode_unsafe(buf)
local _d = M.TestAllRequiredTypesProto2_Data_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestAllRequiredTypesProto2.Data decode, got " .. type(buf), 0)
end
@@ 10199,6 10259,10 @@ end
function M.TestLargeOneof_decode_unsafe(buf)
local decode_len = wire.decode_len
local _d = M.TestLargeOneof_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestLargeOneof decode, got " .. type(buf), 0)
end
@@ 10377,6 10441,10 @@ end
---@return protobuf_test_messages.proto2.TestLargeOneof.A1
function M.TestLargeOneof_A1_decode_unsafe(buf)
local _d = M.TestLargeOneof_A1_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestLargeOneof.A1 decode, got " .. type(buf), 0)
end
@@ 10491,6 10559,10 @@ end
---@return protobuf_test_messages.proto2.TestLargeOneof.A2
function M.TestLargeOneof_A2_decode_unsafe(buf)
local _d = M.TestLargeOneof_A2_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestLargeOneof.A2 decode, got " .. type(buf), 0)
end
@@ 10605,6 10677,10 @@ end
---@return protobuf_test_messages.proto2.TestLargeOneof.A3
function M.TestLargeOneof_A3_decode_unsafe(buf)
local _d = M.TestLargeOneof_A3_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestLargeOneof.A3 decode, got " .. type(buf), 0)
end
@@ 10719,6 10795,10 @@ end
---@return protobuf_test_messages.proto2.TestLargeOneof.A4
function M.TestLargeOneof_A4_decode_unsafe(buf)
local _d = M.TestLargeOneof_A4_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestLargeOneof.A4 decode, got " .. type(buf), 0)
end
@@ 10833,6 10913,10 @@ end
---@return protobuf_test_messages.proto2.TestLargeOneof.A5
function M.TestLargeOneof_A5_decode_unsafe(buf)
local _d = M.TestLargeOneof_A5_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto2.TestLargeOneof.A5 decode, got " .. type(buf), 0)
end
M examples/expected/full/protobuf_test_messages/proto3/test_messages_proto3_pb.lua => examples/expected/full/protobuf_test_messages/proto3/test_messages_proto3_pb.lua +20 -0
@@ 4857,6 4857,10 @@ function M.TestAllTypesProto3_decode_unsafe(buf)
local skip_field = wire.skip_field
local varint_to_int32 = wire.varint_to_int32
local _d = M.TestAllTypesProto3_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto3.TestAllTypesProto3 decode, got " .. type(buf), 0)
end
@@ 6888,6 6892,10 @@ end
---@return protobuf_test_messages.proto3.TestAllTypesProto3.NestedMessage
function M.TestAllTypesProto3_NestedMessage_decode_unsafe(buf)
local _d = M.TestAllTypesProto3_NestedMessage_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto3.TestAllTypesProto3.NestedMessage decode, got " .. type(buf), 0)
end
@@ 7023,6 7031,10 @@ end
---@return protobuf_test_messages.proto3.ForeignMessage
function M.ForeignMessage_decode_unsafe(buf)
local _d = M.ForeignMessage_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto3.ForeignMessage decode, got " .. type(buf), 0)
end
@@ 7140,6 7152,10 @@ end
---@return protobuf_test_messages.proto3.NullHypothesisProto3
function M.NullHypothesisProto3_decode_unsafe(buf)
local _d = M.NullHypothesisProto3_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto3.NullHypothesisProto3 decode, got " .. type(buf), 0)
end
@@ 7254,6 7270,10 @@ end
---@return protobuf_test_messages.proto3.EnumOnlyProto3
function M.EnumOnlyProto3_decode_unsafe(buf)
local _d = M.EnumOnlyProto3_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for protobuf_test_messages.proto3.EnumOnlyProto3 decode, got " .. type(buf), 0)
end
M examples/expected/full/quickstart/quickstart_pb.lua => examples/expected/full/quickstart/quickstart_pb.lua +4 -0
@@ 214,6 214,10 @@ end
function M.User_decode_unsafe(buf)
local decode_bytes = wire.decode_bytes
local _d = M.User_descriptor
+ if pb.c_runtime ~= nil then
+ local _p = _d.c_plan or pb.c_runtime.compile_plan(_d)
+ return pb.c_runtime.decode_unsafe(_p, buf)
+ end
if type(buf) ~= 'string' then
error("expected string for quickstart.User decode, got " .. type(buf), 0)
end
M runtime/pb/c/c_runtime.c => runtime/pb/c/c_runtime.c +36 -7
@@ 1987,6 1987,11 @@ typedef struct dec_ctx {
const uint8_t *buf;
size_t len;
size_t pos;
+ /* Opt-in skip of is_valid_utf8 on every PB_KIND_STRING payload.
+ * Set by decode_unsafe_lua for trusted producers (re-decoding our
+ * own encoder output, in-process typed RPC). Mirrors the Lua
+ * scalar_unsafe.string = scalar.bytes swap in pb.codec. (kyt) */
+ int skip_utf8;
} dec_ctx;
static uint64_t
@@ 2251,7 2256,8 @@ dec_push_kind(dec_ctx *c, uint8_t kind)
uint64_t plen = dec_varint(c);
if (c->len - c->pos < plen)
luaL_error(c->L, "truncated string/bytes payload");
- if (!is_valid_utf8(c->buf + c->pos, (size_t)plen))
+ if (!c->skip_utf8 &&
+ !is_valid_utf8(c->buf + c->pos, (size_t)plen))
luaL_error(c->L,
"invalid UTF-8 in string field at offset %d",
(int)c->pos);
@@ 3066,14 3072,18 @@ decode_body(dec_ctx *c, pb_plan *plan, int result_idx,
}
static int
-decode_lua(lua_State *L)
+decode_impl(lua_State *L, int skip_utf8)
{
pb_plan *plan = (pb_plan *)luaL_checkudata(L, 1, PB_PLAN_MT);
size_t buf_len;
const char *buf = luaL_checklstring(L, 2, &buf_len);
/* WKT override: desc.decode(buf) consumes the entire body and returns
- * whatever Lua representation the override picks (e.g. datetime). */
+ * whatever Lua representation the override picks (e.g. datetime).
+ * The override is opaque — there is no unsafe variant to thread the
+ * skip_utf8 flag into, so the safe and unsafe paths both call it as
+ * declared. WKT string payloads (Timestamp/Duration/Value) are
+ * either ASCII or not user-supplied. */
if (plan->override_decode_ref != LUA_NOREF) {
lua_rawgeti(L, LUA_REGISTRYINDEX, plan->override_decode_ref);
lua_pushlstring(L, buf, buf_len);
@@ 3085,15 3095,33 @@ decode_lua(lua_State *L)
int result_idx = lua_gettop(L);
dec_ctx c;
- c.L = L;
- c.buf = (const uint8_t *)buf;
- c.len = buf_len;
- c.pos = 0;
+ c.L = L;
+ c.buf = (const uint8_t *)buf;
+ c.len = buf_len;
+ c.pos = 0;
+ c.skip_utf8 = skip_utf8;
decode_body(&c, plan, result_idx, /* stop_group_id */ 0);
return 1;
}
+static int
+decode_lua(lua_State *L)
+{
+ return decode_impl(L, /* skip_utf8 */ 0);
+}
+
+/* Sister of decode_lua for trusted producers — skips is_valid_utf8 on
+ * every PB_KIND_STRING payload across the entire decode tree (the flag
+ * lives on dec_ctx and survives sub-message recursion). Dispatched from
+ * pb.decode_unsafe and from full-mode codegen's <Name>_decode_unsafe
+ * prologue. (kyt) */
+static int
+decode_unsafe_lua(lua_State *L)
+{
+ return decode_impl(L, /* skip_utf8 */ 1);
+}
+
/* ---------------------------------------------------------------- *
* Module entry. *
* ---------------------------------------------------------------- */
@@ 3109,6 3137,7 @@ static const struct luaL_Reg c_runtime_methods[] = {
{"plan_sub_plan", plan_sub_plan},
{"encode", encode_lua},
{"decode", decode_lua},
+ {"decode_unsafe", decode_unsafe_lua},
{NULL, NULL},
};
M runtime/pb/init.lua => runtime/pb/init.lua +23 -13
@@ 33,14 33,17 @@ end
-- on desc.c_plan and lazily compile it on first call. Lazy compile is
-- required because compile_plan eagerly chases sub-message refs and
-- codegen forward-declares descriptors (see finalize_message above).
-local pb_encode = codec.encode
-local pb_decode = codec.decode
+local pb_encode = codec.encode
+local pb_decode = codec.decode
+local pb_decode_unsafe = codec.decode_unsafe
if c_runtime ~= nil then
- local c_encode = c_runtime.encode
- local c_decode = c_runtime.decode
- local c_compile = c_runtime.compile_plan
- local lua_encode = pb_encode
- local lua_decode = pb_decode
+ local c_encode = c_runtime.encode
+ local c_decode = c_runtime.decode
+ local c_decode_unsafe = c_runtime.decode_unsafe
+ local c_compile = c_runtime.compile_plan
+ local lua_encode = pb_encode
+ local lua_decode = pb_decode
+ local lua_decode_unsafe = pb_decode_unsafe
pb_encode = function(desc, t)
local plan = desc.c_plan or c_compile(desc)
if plan ~= nil then return c_encode(plan, t) end
@@ 51,6 54,11 @@ if c_runtime ~= nil then
if plan ~= nil then return c_decode(plan, b) end
return lua_decode(desc, b)
end
+ pb_decode_unsafe = function(desc, b)
+ local plan = desc.c_plan or c_compile(desc)
+ if plan ~= nil then return c_decode_unsafe(plan, b) end
+ return lua_decode_unsafe(desc, b)
+ end
end
---@type pb.Module
@@ 61,12 69,14 @@ return {
-- Opt-in non-validating decode for trusted producers (re-decoding
-- bytes from our own encoder, JSON/text round-trips, in-process
-- typed RPC). Skips utf8_len on every string field; sub-message
- -- recursion stays on the unsafe path. C runtime dispatch is
- -- bypassed because the C path validates unconditionally today
- -- (tracked in kyt). Generated runtime-mode wrappers
- -- `M.<Name>_decode_unsafe` forward into this; full-mode codegen
- -- inlines a literal sister `_decode_unsafe` body. (6bb, 58u)
- decode_unsafe = codec.decode_unsafe,
+ -- recursion stays on the unsafe path. When the C runtime is loaded
+ -- this dispatches to c_runtime.decode_unsafe, which gates the
+ -- is_valid_utf8 call on a dec_ctx.skip_utf8 flag (kyt). Generated
+ -- runtime-mode wrappers `M.<Name>_decode_unsafe` forward into this;
+ -- full-mode codegen inlines a literal sister `_decode_unsafe` body
+ -- that itself dispatches to c_runtime.decode_unsafe at the prologue.
+ -- (6bb, 58u, kyt)
+ decode_unsafe = pb_decode_unsafe,
-- Lazy / zero-copy decode view. See runtime/pb/lazy.lua for the
-- :get / :has / :which / :iter / :names surface on the returned