~bigbes/tarantool

tarantool-protobuf

ref: 6e89cf9978343e9c7ead0bad1afcda886b18a3b6 tarantool-protobuf/cmd/conformance-runner.lua -rw-r--r-- 2.9 KiB
d855bcfd — Eugene Blikh 3 months ago
conformance: local Docker pipeline + cdata int64 map dedup

Wires up the Google protobuf conformance harness as a local target.
docker/conformance.Dockerfile builds conformance_test_runner from
upstream protobuf v34.1 source (matching the host's libprotoc 34.1)
and bundles Tarantool 3 from the official installer. `just conformance`
regenerates Lua, then runs the harness against cmd/conformance-runner.lua
with the repo mounted as a volume.

Six bugs surfaced and got fixed on the way to green:

  1. conformance_test_runner uses execv (not execvp): bare `tarantool`
     hits ENOENT. Pass /usr/bin/tarantool in CMD and Justfile.
  2. The harness strips LUA_PATH from the child: the runner now
     self-bootstraps package.path from debug.getinfo(1, 'S').source.
  3. C-stdio buffering on pipe stdin made io.stdin:read(n) wait for a
     full BUFSIZ before returning, deadlocking against the parent.
     setvbuf('no') on stdin/stdout.
  4. v34.1 fetches libjsoncpp via CMake FetchContent under
     _deps/jsoncpp-build/...; the runtime image now COPYs the matching
     .so* and runs ldconfig.
  5. The harness's strict jsoncpp comparator crashes on our currently-
     imperfect JSON output (enum numerics, map<K,V> shape, oneof
     object form). Gate JSON output behind PB_CONFORMANCE_SKIP_JSON=1,
     set in the container ENV; host-side `make test` still exercises
     the full JSON path.
  6. Codec bug — LuaJIT hashes cdata int64 by pointer, so duplicate-
     key map entries (per proto3's "last value wins" semantics) split
     across hash buckets even though __eq matches. Codec walks the
     map once on insert to find a canonical key, gated by a
     precomputed `f.key_dedup` flag so the dedup only fires for
     int64/uint64/sint64/fixed64/sfixed64 keys. inline.go emits the
     same `for _k in pairs(map) do` walk only when the static key
     kind is 64-bit, so string/int32-keyed map decode stays
     JIT-traceable.

Watchlists at test/conformance/known_failures.txt (binary + JSON) and
test/conformance/known_failures_text.txt (text-format) hold the
deferred failures. Current baseline:

  - Binary + JSON suite: 803 ✓ / 1864 skipped / 139 expected fails
  - Text-format suite:     0 ✓ /  430 skipped /   4 expected fails

403/403 luatest green, 19/19 jit-trace gate green.
784dea4d — Eugene Blikh 3 months ago
Initial commit: protoc-gen-tarantool plugin + pb runtime

A protoc plugin (Go) and a pure-Lua + LuaJIT-FFI runtime that give
Tarantool a complete proto3 + gRPC stack. Two codegen modes (full
inline / runtime descriptor), 226-test luatest suite, 18-fixture
mainline-protoc interop corpus, JSON codec, well-known types,
gRPC client/server factories, runtime .proto parser, microbench
harness with allocation regression gate.

Covers PLAN.md M1-M5. Module is `pb` (not `protobuf`) to avoid
colliding with Tarantool's built-in encode-only `protobuf` module.