~bigbes/tarantool

tarantool-protobuf

ref: 55f0c318230172882b5b8b2bcb7460f69578c406 tarantool-protobuf/bench/COMPARISON.md -rw-r--r-- 6.6 KiB
b93adbf0 — Eugene Blikh 2 months ago
codec: emit per-descriptor encode body for monomorphic dispatch (21d)

The runtime-mode encode loop in codec.encode_message iterated
desc.fields and called `writer(data, out)` per field. Each iteration
saw a different closure, making the call site megamorphic from the
JIT's view: trace topology fragmented into 25 stops for Person_encode
vs full mode's 8 (~3x).

compile_encode_body emits a generated function at pb.finalize_message
time with one literal call site per field, all closing over a single
`_u` table upvalue indexed by constant int. TGETI on a stable array
with a literal key specializes on trace just like direct upvalue
access — and dodges LuaJIT's 60-upvalue function limit, which
TestAllTypesProto2 (~140 fields) would otherwise hit.

Trace topology (bench/jit_trace.lua):
  runtime/Person_encode             25 -> 10  (full 8)
  runtime/Person_encode multi-byte  18 ->  7  (full 6)
  runtime/Address_encode             6 ->  5  (full 2)
  runtime/Cardinality required       6 ->  2  (full 2)

Encode throughput (hello.Person, bench/bench.lua):
  10B    10.9 -> 15.5 MB/s  (+42%)
  100B  101.1 -> 144.4 MB/s (+43%)
  1KB   170.5 -> 183.5 MB/s (+8%)
  10KB  355.5 -> 362.3 MB/s (+2%)
  100KB 373.5 -> 407.6 MB/s (+9%)

Decode untouched. Full mode unaffected (its inline _encode doesn't
go through codec.encode_message). 752+1043 tests pass; examples
unchanged.

The issue's secondary goal — runtime within 5% of full on encode at
all sizes — is not met. Residual gap is per-closure call overhead;
closing it would need writer bodies inlined into the generated body
(not just call sites), which is a larger codegen-at-runtime change.

Closes tarantool-protobuf-21d.
Files tarantool-protobuf-h8x (decode_group bimodal trace flake
surfaced during validation; pre-existing on master).
07eef11c — Eugene Blikh 3 months ago
bench: Go cross-runtime comparison harness

Adds bench/go/ — single-threaded Go benchmarks against the same proto
schemas and payload sizes as bench/bench.lua, run against
google.golang.org/protobuf v1.36 (reflective apiv2) and
planetscale/vtprotobuf v0.6 (codegen marshalers). Wired through
`just gen-go` + `just bench-go`. *.pb.go is gitignored repo-wide so
the generated outputs are regenerated locally — not committed.

bench/COMPARISON.md documents the full table per fixture / size / op:
MB/s for Lua full + Lua runtime + apiv2 + vtproto, alloc bytes/op and
allocs/op side-by-side, and ratios. proto2 BenchPayload vtproto cases
are intentionally skipped — MarshalVT drops proto2 extensions and
would understate bytes vs apiv2 / Lua.