codegen: CHARS[_len] lookup replaces string.char(_len) at length-prefix sites (2ri)
Profile attributed 39% of Person_encode's trace share (~28% of total) to a
single line emitting `string.char(_len)` at every inlined length-prefix
site. The `_len` argument is rarely a compile-time constant (lengths come
from user data), so the JIT can't fold the C-function call, and the cost
compounds — the 1KB Person fixture fires ~30 length-prefix sites per
encode.
Replace with a 256-entry lookup table `wire.CHARS` (built once at module
load, byte i -> string.char(i)). Codegen header now emits
`local CHARS = wire.CHARS` alongside the other hot-path localizers; the
single emit site in `emitInlineLenPrefix` swaps `string.char(_len)` for
`CHARS[_len]`. Parity is by construction — both return the same interned
1-byte string.
Tests: 752/752 pass. Bench (work.lab.local, median of 3, hello.Person
encode): 10B +4.7%, 1KB +16.8%, 10KB +21.1%, 100KB +34.2%, proto2 mid
+11.1%. Decode unchanged. See bench/PERF_LOG.md 2026-05-24 2ri entry.
Closes lkz and 86g (ffi.new buffer rewrite paths) — separate hand-spike
of that shape regressed 0.31x-0.77x across all sizes; the perceived
buffering inefficiency wasn't there, and 2ri captured the single hottest
line. Remaining encode-perf headroom is c0i (C-runtime backend).
codegen: inline proto2 extension writers/readers (qwt) + table.new(N,0) for packed lists (2sn)
qwt closes the proto2_basic.BenchPayload `min` bench's worst data point:
encode 1.89M → 3.30M msgs/s (+75%), decode 1.02M → 1.26M msgs/s (+23%).
Mechanism: collectExtsByExtendee groups all `extend Foo { ... }`
declarations by extendee FullName across input files; emitInlineEncode
emits one dedicated writer per extension instead of dispatching through
pb.codec.encode_field, and emitInlineDecode adds an elseif arm per
extension id straight into result._extensions[full_name]. Dynamic
extensions_list walk preserved for forward compat, gated on
#_elist > N so it pays one int compare when no runtime extension was
registered.
2sn pre-sizes packed-scalar repeated lists via table.new(N, 0) where
the count is recoverable from the LEN payload — exact (lim >> 2 or
lim >> 3) for fixed-width, upper bound (lim) for varint-packed. The
allocation is deferred into the `if wt == 2 then` branch so the
per-element fallback and non-packable types keep the bare-`{}`
alloc. Avoids u39's regression mode because the call cost is paid
once per repeated-field-first-occurrence, not per message decode.
Bench summary for hello.Person full mode: encode +3-5% across sizes,
decode +1-3% across sizes. Tests: 752/752 pure-Lua, 1043/1043 with
PB_ENABLE_C=1, JIT 37/37. PERF_LOG entry covers the rationale and
caveats.
codegen: localize wire.* upvalues per generated message function
Capture each _encode/_decode body, scan for wire.<name> refs, and
rewrite refs that appear >=2 times to bare locals with a
"local X = wire.X" prelude. Single-use refs stay as wire.X — without
the threshold the prelude TGETS outweighed the in-body saving on
sparse small-message decode.
Measured (median-of-3, shapes bench full mode):
- scalar-heavy: enc +38.5%, dec +39.2%
- packed-int32x100: enc +51.1%, dec +38.2%
- map-strxi32-*: enc +5.5%, dec +6-7%
- nested/oneof/wkt: +1-5% (within ~5% variance band)
- bench.lua Person small sizes: neutral
closes kot
codegen: inline 1-byte LEN fast path for string/bytes decode
Singular and repeated string/bytes fields in generated full-mode _decode
now read the length byte and dispatch in-line instead of calling
wire.decode_string / wire.decode_bytes. For length < 128 (the common
short-string RPC case), the path stays inside the parent JIT trace:
no child-trace stitch, no per-element function frame, utf8_len lookup
hoisted to a generated-file upvalue.
The original a6n design — one ffi.cast(U8CP, buf) at the top of each
_decode — was abandoned: the cdata wrapper is 24 bytes per call and
LuaJIT can't sink the allocation because the pointer local lives
across wire.decode_* call frames. Net regression at small sizes
(+11-16%) overwhelmed the single-byte-read savings.
Bench (hello.Person full-mode decode):
size ns/op before ns/op after delta
10B 419 390 -6.9%
100B 515 478 -7.2%
1KB 9004 8042 -10.7%
10KB 64644 56745 -12.2%
100KB 625638 537750 -14.0%
Zero allocation impact across all sizes. Tests: 1043 pass, 37 JIT
trace gates pass. (a6n)
c_runtime: wire encode/decode dispatch + conformance-c harness
pb.encode / pb.decode lazy-compile desc.c_plan on first call and route
to pb.c_runtime.encode/decode when PB_ENABLE_C=1 loaded the module.
Eager compile at finalize_message time fails on codegen's
forward-declared descriptors — sub-messages don't have .fields yet —
so compilation is deferred until first encode/decode, by which time
the whole module table is populated and sub-plan chase resolves.
Full-mode codegen wrappers (M.<Type>_{encode,decode}) gain the same
lazy-compile prologue, bypassing the inline body when the C runtime
is loaded. Runtime-mode wrappers already call pb.encode and pick up
dispatch centrally.
Harness side: the conformance Docker image now installs tarantool-dev
+ build-essential so the C runtime can be built in-container; a new
`just conformance-c` recipe builds runtime/pb/c_runtime.so inside the
container, runs the suite with PB_ENABLE_C=1, then cleans the .so to
keep the bind mount free of foreign-platform binaries. The runner
script also pre-populates package.loaded.pb (avoids
.rocks/lib/tarantool/pb.so from starwing lua-protobuf masking ours)
and orders package.cpath by jit.os so mixed .dylib/.so trees from
host/container interleavings don't cross-load.
Verified:
- just test → 748 pass, 291 C-conditional skipped
- PB_ENABLE_C=1 just test → 1026/1039 pass; 13 fails are strict-decode
gaps in the C decoder (bd-rc8)
- just conformance-c → 2729/2806 binary suite pass; 77 unexpected
failures match the same gap categories
(illegal wire-type 6/7, field-num 0/over,
overlong tag varint, UTF-8 rejection,
message merge for oneof/repeated)
Strict-decode parity tracked in bd-rc8; this commit closes the wiring
half of bd-43t (conformance gate for C paths).
c_runtime: repeated + packed scalar encode/decode (ra6 3e)
Add repeated dispatch to the C-runtime encode/decode loop. Encode
side: encode_repeated_field walks Lua arrays via lua_objlen + per-
index rawgeti, dispatches on element kind. Packed numerics build
their payload in a stack-backed sub-buffer then emit `tag(LEN) +
varint(len) + body`; unpacked emit `tag + value` per element via
encode_one_field with force_emit=1 to bypass zero-suppression;
strings/bytes flow through the same path (never packable); repeated
messages reuse encode_submessage_field per element.
Decode side: per-field stack-slot cache (list_stack_idx[]) tied to
list_count[] avoids the per-element lua_getfield(result, name) round
trip the c-accel spike measured at 2x slower at 100KB. On first hit
for a repeated field we lua_createtable + write result[name] AND dup-
push the list onto the stack; subsequent hits lua_rawseti through the
cached absolute stack index. Lists stay valid across recursive sub-
message decodes because each child decode_body cleans up its own
scratch back to the caller's frame.
Packed/unpacked symmetry on read: a wt==LEN payload for any packable
scalar is decoded as a packed blob regardless of the schema's packed
flag, and a per-element-tagged stream is decoded element-by-element
even on a schema that defaults to packed — per proto3 reader rules.
New test/proto/c_repeated.proto fixture carries packed + explicit-
unpacked + repeated string/bytes + repeated message branches. The
encode and decode tests round-trip at 10/100/1000 elements per
branch. The two existing "skip repeated and map" marker tests
collapse to "skip map" — only map fields remain out of scope for
3e (bd-asz / 3h lands them next). 854 → 896 passing tests.
bd-jc9