~bigbes/tarantool

tarantool-protobuf

ref: 5de883e9079faacdcdccb853011e52142cbb16b9 tarantool-protobuf/examples/expected/full/c_int64/c_int64_pb.lua -rw-r--r-- 5.6 KiB
b09c7213 — Eugene Blikh 2 months ago
codegen: inline 1-byte LEN fast path for string/bytes decode

Singular and repeated string/bytes fields in generated full-mode _decode
now read the length byte and dispatch in-line instead of calling
wire.decode_string / wire.decode_bytes. For length < 128 (the common
short-string RPC case), the path stays inside the parent JIT trace:
no child-trace stitch, no per-element function frame, utf8_len lookup
hoisted to a generated-file upvalue.

The original a6n design — one ffi.cast(U8CP, buf) at the top of each
_decode — was abandoned: the cdata wrapper is 24 bytes per call and
LuaJIT can't sink the allocation because the pointer local lives
across wire.decode_* call frames. Net regression at small sizes
(+11-16%) overwhelmed the single-byte-read savings.

Bench (hello.Person full-mode decode):

      size    ns/op before   ns/op after   delta
      10B          419            390     -6.9%
      100B         515            478     -7.2%
      1KB         9004           8042    -10.7%
      10KB       64644          56745    -12.2%
      100KB     625638         537750    -14.0%

Zero allocation impact across all sizes. Tests: 1043 pass, 37 JIT
trace gates pass. (a6n)
1eb062c1 — Eugene Blikh 2 months ago
c_runtime: wire encode/decode dispatch + conformance-c harness

pb.encode / pb.decode lazy-compile desc.c_plan on first call and route
to pb.c_runtime.encode/decode when PB_ENABLE_C=1 loaded the module.
Eager compile at finalize_message time fails on codegen's
forward-declared descriptors — sub-messages don't have .fields yet —
so compilation is deferred until first encode/decode, by which time
the whole module table is populated and sub-plan chase resolves.

Full-mode codegen wrappers (M.<Type>_{encode,decode}) gain the same
lazy-compile prologue, bypassing the inline body when the C runtime
is loaded. Runtime-mode wrappers already call pb.encode and pick up
dispatch centrally.

Harness side: the conformance Docker image now installs tarantool-dev
+ build-essential so the C runtime can be built in-container; a new
`just conformance-c` recipe builds runtime/pb/c_runtime.so inside the
container, runs the suite with PB_ENABLE_C=1, then cleans the .so to
keep the bind mount free of foreign-platform binaries. The runner
script also pre-populates package.loaded.pb (avoids
.rocks/lib/tarantool/pb.so from starwing lua-protobuf masking ours)
and orders package.cpath by jit.os so mixed .dylib/.so trees from
host/container interleavings don't cross-load.

Verified:
- just test                 → 748 pass, 291 C-conditional skipped
- PB_ENABLE_C=1 just test   → 1026/1039 pass; 13 fails are strict-decode
                              gaps in the C decoder (bd-rc8)
- just conformance-c        → 2729/2806 binary suite pass; 77 unexpected
                              failures match the same gap categories
                              (illegal wire-type 6/7, field-num 0/over,
                              overlong tag varint, UTF-8 rejection,
                              message merge for oneof/repeated)

Strict-decode parity tracked in bd-rc8; this commit closes the wiring
half of bd-43t (conformance gate for C paths).
fa57c1e4 — Eugene Blikh 2 months ago
c_runtime: 64-bit cdata fidelity tests (ra6 3l)

Adds the c_int64.Wide fixture (one singular field per 64-bit kind)
plus an 18-test luatest group that round-trips each kind past 2^53
through the C runtime in both codegen modes. Confirms encode accepts
both LuaJIT int64_t/uint64_t cdata and Lua numbers, and decode
surfaces values >DBL_INT_MAX as cdata (matching msgpackffi /
net.box / box.tuple / built-in protobuf convention).

The C runtime already had the dispatch — to_int64_at / to_uint64_at
flow through luaL_toint64 / luaL_touint64 for cdata inputs, and
dec_push_one calls luaL_pushint64 / luaL_pushuint64 for every 64-bit
kind. This change pins the behavior under acceptance.

Closes tarantool-protobuf-awv (ra6 3l)