~bigbes/tarantool

tarantool-protobuf

5dd21dea0c18f5e129fc729f32cdcc56d7050c40 — Eugene Blikh 2 months ago 5cc9431
c_runtime: parity gate — test-c / test-all / bench-c (43t)

Adds the C-acceleration parity gate that bd-43t mandates: every test
asserts against a reference output (golden bytes, txtpb, conformance
result), so passing the same suite under both `PB_ENABLE_C=1` and the
default Lua codec proves Lua ≡ C by transitivity. No separate diff
harness needed.

Justfile recipes:
  - `test-c`  — same luatest suite with PB_ENABLE_C=1 (1043 tests;
                unlocks the c_runtime_* groups via build-c)
  - `test-all` — `test` + `test-c` (parity gate)
  - `bench-c` — bench.lua under PB_ENABLE_C=1; runtime column is
                relabelled `c-runtime`. Also tidies build-c — the stale
                "directory doesn't exist yet" branch goes away now that
                ra6 has landed.

bench.lua: extends package.cpath so `require('pb.c_runtime')` finds
runtime/pb/c_runtime.{so,dylib} when invoked outside the Justfile.
Detects `PB_ENABLE_C=1`, renames `runtime` → `c-runtime` in the
output, and refuses --baseline / --compare (alloc shape differs
between codecs by design — would noise the gate).

README: documents the parity strategy and lists `test-all`,
`conformance-c`, `bench-c` as the dual-codec entry points.

Scope notes: c-generated column dropped (c0i is deferred per
docs/c-accel.md); starwing column already lives in
bench/starwing_bench.lua.

bd-43t
3 files changed, 80 insertions(+), 10 deletions(-)

M Justfile
M README.md
M bench/bench.lua
M Justfile => Justfile +24 -9
@@ 71,16 71,9 @@ build-doc:
    go build -o {{doc_plugin}} ./cmd/protoc-gen-tarantool-doc

# Build the optional C-acceleration runtime into runtime/pb/c_runtime.{so,dylib}.
# Source lives under runtime/pb/c/ once bd-ra6 lands; until then this recipe
# prints a helpful error and exits 1. The runtime is opt-in via PB_ENABLE_C=1
# — see docs/specs/c_accel_build_packaging.md for the full contract.
# The runtime is opt-in via PB_ENABLE_C=1 — see docs/specs/c_accel_build_packaging.md
# for the full contract.
build-c:
    @if [ ! -d runtime/pb/c ]; then \
        echo "build-c: runtime/pb/c/ does not exist yet."; \
        echo "  The C runtime arrives with bd-ra6 (generic C codec)."; \
        echo "  See docs/specs/c_accel_build_packaging.md."; \
        exit 1; \
    fi
    make -C runtime/pb/c

# Remove built C-runtime artifacts.


@@ 180,6 173,19 @@ test: gen
test-one filter: gen
    LUA_PATH="{{lua_path}}" LUA_CPATH="{{lua_cpath}}" {{luatest}} -v test/{{filter}}

# Same as `test`, but with PB_ENABLE_C=1 so pb.encode / pb.decode dispatch
# through the C runtime (runtime/pb/c_runtime.{so,dylib}). Builds the C
# module first. Together with `test`, this is the parity gate — every
# luatest assertion is against a reference output (golden bytes, txtpb,
# conformance result), so if Lua passes and C passes, both equal the
# reference and Lua ≡ C by transitivity. See bd-43t.
test-c: build-c gen
    PB_ENABLE_C=1 LUA_PATH="{{lua_path}}" LUA_CPATH="{{lua_cpath}}" {{luatest}} -v test/

# Parity gate: run the suite under both codecs. Use before pushing C-runtime
# or codec changes.
test-all: test test-c

# ---------------------------------------------------------------------------
# Bench
# ---------------------------------------------------------------------------


@@ 188,6 194,15 @@ test-one filter: gen
bench: gen
    tarantool bench/bench.lua --print

# Same as `bench`, but with PB_ENABLE_C=1 so the runtime-mode results
# exercise the C codec. The runtime-mode column is relabelled to
# `c-runtime` in the output. Full-mode results are unchanged (full mode
# uses inline wire calls, not pb.encode). Allocation baselines (`bench-baseline`
# / `bench-compare`) remain Lua-only — C alloc shape differs by design
# and would noise the gate.
bench-c: build-c gen
    PB_ENABLE_C=1 tarantool bench/bench.lua --print

# Overwrite bench/baseline.json with current alloc-per-op numbers.
bench-baseline: gen
    tarantool bench/bench.lua --baseline

M README.md => README.md +21 -0
@@ 385,12 385,33 @@ binary, JSON, and text-format input/output, including the
`test/conformance_test.lua` exercises the runner with crafted requests on
every `just test` run.

### C-runtime parity

The optional C acceleration runtime (`runtime/pb/c_runtime.{so,dylib}`,
built via `just build-c`, activated by `PB_ENABLE_C=1`) must produce
byte-identical output to the pure-Lua codec. The parity gate reuses the
existing test suites — no separate diff harness:

```bash
just test-all       # luatest under both codecs (752 + 1043 tests)
just conformance    # Google suite, Lua codecs
just conformance-c  # Google suite, PB_ENABLE_C=1
```

Every assertion checks against a reference (golden bytes, txtpb,
conformance result). If Lua passes and C passes, both equal the
reference, so Lua ≡ C by transitivity. The interop fixtures under
`test/interop/fixtures/` are parameterized over both codegen modes;
under `PB_ENABLE_C=1` the runtime-mode iteration transparently
exercises the C codec.

[gconf]: https://github.com/protocolbuffers/protobuf/tree/main/conformance

## Benchmarks

```bash
just bench           # print throughput + alloc per op (5 sizes × 2 modes)
just bench-c         # same with PB_ENABLE_C=1 — runtime column → `c-runtime`
just bench-baseline  # overwrite bench/baseline.json (run on a quiet machine)
just bench-compare   # exit 1 if any alloc-per-op regressed >5% vs baseline
```

M bench/bench.lua => bench/bench.lua +35 -1
@@ 17,6 17,13 @@ package.path = './runtime/?.lua;./runtime/?/init.lua;'
    .. './examples/expected/?.lua;./examples/expected/?/init.lua;'
    .. package.path

-- Extend cpath so `require('pb.c_runtime')` finds runtime/pb/c_runtime.{so,dylib}
-- when PB_ENABLE_C=1. `.dylib` first matches the order in the Justfile (see
-- the comment there for why this matters when both extensions coexist).
package.cpath = './runtime/?.dylib;./runtime/?.so;'
    .. './runtime/?/init.dylib;./runtime/?/init.so;'
    .. package.cpath

local clock = require('clock')
local json  = require('json')
local fio   = require('fio')


@@ 436,9 443,36 @@ end
local args = {...}
local mode_flag = args[1] or '--print'

io.stderr:write(string.format('tarantool-protobuf bench (%s)\n', _TARANTOOL))
local C_ENABLED = (os.getenv('PB_ENABLE_C') == '1') and (require('pb').c_runtime ~= nil)

if C_ENABLED and (mode_flag == '--baseline' or mode_flag == '--compare') then
    -- The alloc-per-op baseline is Lua-only by design. The C codec has a
    -- different allocation shape (C-side scratch + a single Lua-side
    -- result string) that would noise the gate. Re-run without
    -- PB_ENABLE_C=1 for baseline/compare ops.
    io.stderr:write('bench.lua: --baseline and --compare are Lua-only; '
        .. 'unset PB_ENABLE_C and rerun\n')
    os.exit(2)
end

io.stderr:write(string.format('tarantool-protobuf bench (%s)%s\n',
    _TARANTOOL, C_ENABLED and ', C runtime ENABLED' or ''))
local doc = run_all()

-- Relabel runtime-mode results to `c-runtime` when the C dispatch is
-- active. The bench fixture iterates over generated modules from
-- examples/expected/runtime/, which call pb.encode / pb.decode — those
-- dispatch through the C codec when desc.c_plan compiles. Full-mode
-- modules inline wire calls and don't go through pb.encode, so the
-- full-mode column is unchanged from the Lua run.
if C_ENABLED then
    for _, schema in ipairs(doc.schemas) do
        for _, r in ipairs(schema.results) do
            if r.mode == 'runtime' then r.mode = 'c-runtime' end
        end
    end
end

if mode_flag == '--print' then
    io.write(render(doc))
elseif mode_flag == '--baseline' then