~bigbes/tarantool

tarantool-protobuf

tarantool-protobuf/bench/bench.lua -rw-r--r-- 19.7 KiB
7b3d6901 — Eugene Blikh 2 months ago
bench: apply mcode arena hardening across all bench scripts (3qu)

Added jit.opt.start('sizemcode=64', 'maxmcode=4096') to all 9 bench
scripts. Same fix 3o2 landed in bench/jit_trace.lua, propagated to the
rest. Without it, the macOS arm64 mcode allocator intermittently fails
to find an executable page in the signed-32-bit offset window and the
bench reports interpreter-only throughput with no diagnostic — visible
as silent regressions. The risk is higher in scripts with larger codegen
footprints than the trace gate.

Files touched: bench.lua, lazy_bench.lua, profile.lua, shapes_bench.lua,
starwing_bench.lua, wire_bench.lua, alloc_probe.lua, map_bench.lua,
packed_bench.lua. All 7 non-interactive scripts run rc=0; profile and
starwing loadfile-check clean.

Also refreshed bench/baseline.json per the issue's step 2. Surprise win:
full-mode decode allocations dropped 0.5-50% as a side-effect of auj/ozn
that was only visible after the snapshot. Most notable: proto2_basic
BenchPayload mid decode 2.313 -> 1.156 KB/op (-50%), Person 1KB decode
0.977 -> 0.953 KB/op. Runtime-mode unchanged — confirms the alloc
reduction is full-mode codegen specific.

Steps 3-4 (refresh COMPARISON.md throughput tables, verify the variance
band shrinks below the documented 5-10% drift) are deferred to a
work.lab.local run — laptop variance is 50%+ on identical state, can't
trust throughput A/Bs locally.
5dd21dea — Eugene Blikh 2 months ago
c_runtime: parity gate — test-c / test-all / bench-c (43t)

Adds the C-acceleration parity gate that bd-43t mandates: every test
asserts against a reference output (golden bytes, txtpb, conformance
result), so passing the same suite under both `PB_ENABLE_C=1` and the
default Lua codec proves Lua ≡ C by transitivity. No separate diff
harness needed.

Justfile recipes:
  - `test-c`  — same luatest suite with PB_ENABLE_C=1 (1043 tests;
                unlocks the c_runtime_* groups via build-c)
  - `test-all` — `test` + `test-c` (parity gate)
  - `bench-c` — bench.lua under PB_ENABLE_C=1; runtime column is
                relabelled `c-runtime`. Also tidies build-c — the stale
                "directory doesn't exist yet" branch goes away now that
                ra6 has landed.

bench.lua: extends package.cpath so `require('pb.c_runtime')` finds
runtime/pb/c_runtime.{so,dylib} when invoked outside the Justfile.
Detects `PB_ENABLE_C=1`, renames `runtime` → `c-runtime` in the
output, and refuses --baseline / --compare (alloc shape differs
between codecs by design — would noise the gate).

README: documents the parity strategy and lists `test-all`,
`conformance-c`, `bench-c` as the dual-codec entry points.

Scope notes: c-generated column dropped (c0i is deferred per
docs/c-accel.md); starwing column already lives in
bench/starwing_bench.lua.

bd-43t
2656c977 — Eugene Blikh 3 months ago
bench,codec: pin proto2 paths, kill pairs() on extension hot path

JIT trace gate gains 12 proto2-specific checks (required encode/decode,
groups singular + repeated, extension encode/decode). The extension-encode
check fired NYI on bytecode 72 (ISNEXT) — pairs() over the new
extensions_by_full_name hash is the same trace-abort that gates map
encode. Fix: register_extension now also appends to extensions_list,
an array view. codec.encode_message, text.emit_message and
json.encode_message all switch to ipairs over the list. Hash tables
stay around for O(1) lookups in decode (extensions_by_id) and bracket-
name resolution (extensions_by_full_name).

Inline (full-mode) codegen learns to walk extensions too: before this
commit the inline encoder skipped t._extensions entirely (only the
runtime codec walked it), so a generated _encode silently dropped any
extension data set on the message. Add the array-walk after the field
loop and a decode_extension dispatch in the unknown-tag branch — the
inline path now matches runtime byte-for-byte.

bench/bench.lua parameterizes over a FIXTURES list so the baseline can
cover both hello.Person and a new proto2_basic.BenchPayload fixture
(required + group + extension + repeated). baseline.json restructured
under "schemas": [{schema, results}, …]; compare walks both. Fresh
numbers committed.

740/740 luatest cases pass; JIT gate 37/37; conformance 2806 binary+JSON
and 434 text-format, both 0 failures.
784dea4d — Eugene Blikh 3 months ago
Initial commit: protoc-gen-tarantool plugin + pb runtime

A protoc plugin (Go) and a pure-Lua + LuaJIT-FFI runtime that give
Tarantool a complete proto3 + gRPC stack. Two codegen modes (full
inline / runtime descriptor), 226-test luatest suite, 18-fixture
mainline-protoc interop corpus, JSON codec, well-known types,
gRPC client/server factories, runtime .proto parser, microbench
harness with allocation regression gate.

Covers PLAN.md M1-M5. Module is `pb` (not `protobuf`) to avoid
colliding with Tarantool's built-in encode-only `protobuf` module.