wire: bounds-check skip_field for I32/I64/LEN truncation
skip_field advanced pos by a fixed or length-prefixed amount with no
check against #buf, so a truncated unknown field was silently consumed:
the outer decode loop's `while pos <= len` exited without raising,
making the parser accept payloads it should have rejected.
Validates that the new position never exceeds #buf+1 for WIRE_I64,
WIRE_I32, and both WIRE_LEN fast and slow paths. WIRE_VARINT already
errored correctly via decode_varint's per-byte check.
Drops 15 entries from test/conformance/known_failures.txt
(PrematureEofBeforeUnknownValue.*, PrematureEofInsideUnknownValue.*,
PrematureEofInDelimitedDataForUnknownValue.*).
wkt: auto-register WKT descriptors so Any JSON decode resolves @type
pb.wkt exported Timestamp/Duration/Empty/FieldMask/Any/Struct/Value/
ListValue and the nine Wrapper descriptors, but the REGISTRY they live in
was empty until callers manually invoked pb.register. json_to_any looked
up @type in that empty registry, fell through to the opaque base64
fallback, and errored on every Any-of-WKT JSON payload.
Iterates M for every *_descriptor entry at module load and self-registers
it. Also calls pb.register on TestAllTypesProto3 in the conformance runner
so Any tests that embed the user message type also resolve.
Drops 10 entries from test/conformance/known_failures.txt.
wire: reject illegal wire types 6/7 in decode_tag
Wire types 6 and 7 are never assigned in the protobuf wire format; only
0 (VARINT), 1 (I64), 2 (LEN), 3/4 (SGROUP/EGROUP, proto2-only), and 5
(I32) are valid. skip_field already rejected them for unknown fields,
but the decode loop in codec.lua and inline.go dispatched to a typed
reader whenever the field ID was known — the reader ignored wt and
called the value-specific decoder anyway, accepting bytes that should
have been a parse error.
Moves the check into decode_tag so every parser path — codec, generated
inline code, lazy, and map sub-fields — rejects 6/7 uniformly.
Drops 24 entries from test/conformance/known_failures.txt.
wire: truncate varints to 32 bits in int32/uint32/sint32/enum decode
decode_int32/uint32/sint32 returned full uint64 values when the wire input
carried bits above bit 31, corrupting re-encode on 51 conformance tests that
exercise overlong or over-range varints. Per the proto3 spec the decoder
must keep only the low 32 bits (and sign-extend for signed types).
Adds wire.varint_to_int32 / varint_to_uint32 and routes every enum-varint
decode site through them: wire.lua typed decoders, codec.lua (5 enum
sites), lazy.lua (3 sites), and the generated code via inline.go (3 sites).
Drops 51 entries from test/conformance/known_failures.txt.
conformance: local Docker pipeline + cdata int64 map dedup
Wires up the Google protobuf conformance harness as a local target.
docker/conformance.Dockerfile builds conformance_test_runner from
upstream protobuf v34.1 source (matching the host's libprotoc 34.1)
and bundles Tarantool 3 from the official installer. `just conformance`
regenerates Lua, then runs the harness against cmd/conformance-runner.lua
with the repo mounted as a volume.
Six bugs surfaced and got fixed on the way to green:
1. conformance_test_runner uses execv (not execvp): bare `tarantool`
hits ENOENT. Pass /usr/bin/tarantool in CMD and Justfile.
2. The harness strips LUA_PATH from the child: the runner now
self-bootstraps package.path from debug.getinfo(1, 'S').source.
3. C-stdio buffering on pipe stdin made io.stdin:read(n) wait for a
full BUFSIZ before returning, deadlocking against the parent.
setvbuf('no') on stdin/stdout.
4. v34.1 fetches libjsoncpp via CMake FetchContent under
_deps/jsoncpp-build/...; the runtime image now COPYs the matching
.so* and runs ldconfig.
5. The harness's strict jsoncpp comparator crashes on our currently-
imperfect JSON output (enum numerics, map<K,V> shape, oneof
object form). Gate JSON output behind PB_CONFORMANCE_SKIP_JSON=1,
set in the container ENV; host-side `make test` still exercises
the full JSON path.
6. Codec bug — LuaJIT hashes cdata int64 by pointer, so duplicate-
key map entries (per proto3's "last value wins" semantics) split
across hash buckets even though __eq matches. Codec walks the
map once on insert to find a canonical key, gated by a
precomputed `f.key_dedup` flag so the dedup only fires for
int64/uint64/sint64/fixed64/sfixed64 keys. inline.go emits the
same `for _k in pairs(map) do` walk only when the static key
kind is 64-bit, so string/int32-keyed map decode stays
JIT-traceable.
Watchlists at test/conformance/known_failures.txt (binary + JSON) and
test/conformance/known_failures_text.txt (text-format) hold the
deferred failures. Current baseline:
- Binary + JSON suite: 803 ✓ / 1864 skipped / 139 expected fails
- Text-format suite: 0 ✓ / 430 skipped / 4 expected fails
403/403 luatest green, 19/19 jit-trace gate green.
Initial commit: protoc-gen-tarantool plugin + pb runtime
A protoc plugin (Go) and a pure-Lua + LuaJIT-FFI runtime that give
Tarantool a complete proto3 + gRPC stack. Two codegen modes (full
inline / runtime descriptor), 226-test luatest suite, 18-fixture
mainline-protoc interop corpus, JSON codec, well-known types,
gRPC client/server factories, runtime .proto parser, microbench
harness with allocation regression gate.
Covers PLAN.md M1-M5. Module is `pb` (not `protobuf`) to avoid
colliding with Tarantool's built-in encode-only `protobuf` module.