json: strict validation pass closes the proto3 conformance suite
Six classes of relaxation that the proto3 JSON conformance corpus
flagged are now enforced. All as Recommended.* tests; combined with
the -0.0 codec fix this empties known_failures.txt and brings the
proto3 binary+JSON suite to 1493 ✓ / 1313 skipped / 0 expected
failures / 0 unexpected failures.
1. Duplicate JSON keys. Tarantool's json.decode is hash-backed and
silently collapses `{"foo":1,"foo":2}` to one entry. A small
byte-walker `find_duplicate_json_keys` runs before json.decode,
tracks per-object brace frames and key sets, errors on the
second occurrence. Closes Recommended.FieldNameDuplicate.
2. camelCase / snake_case aliases of the same proto field appearing
side-by-side. Detected inside decode_message via a `field_seen`
set keyed by proto-name; second hit errors. Closes
FieldNameDuplicateDifferentCasing{1,2}.
3. JSON null inside repeated arrays and map values. Previously
silently dropped; now errors before decode_field_value. Closes
RepeatedField{Message,Primitive}ElementIsNull and
MapFieldValueIsNull.
4. Unknown enum *names* (not integers). decode_enum used to return
nil so callers silently dropped them; now raises by default and
returns nil only when M.decode's `ignore_unknown_fields=true`
opt is set. Conformance dispatch in cmd/conformance/core.lua
forwards this flag when req.test_category ==
JSON_IGNORE_UNKNOWN_PARSING_TEST. Closes
RejectUnknownEnumStringValueIn{Optional,Repeated,Map} and the
paired IgnoreUnknownEnumStringValueIn* tests.
5. google.protobuf.NullValue JSON canonical form. The single enum
value renders as the literal JSON `null` (not the string
"NULL_VALUE"); decode accepts either, encode emits null. The
decode_field_value null-handling path also treats a JSON null
on a NullValue-typed field as "set" rather than "absent" so a
oneof gets marked active. Closes
NullValueInOtherOneof{New,Old}Format.Validator.
6. FieldMask strict round-trip. Path validity is checked on both
sides: the snake_case wire form rejects uppercase letters,
consecutive underscores, trailing underscore, and underscore
followed by anything other than a lowercase letter — these
break the snake↔camel round-trip. The JSON form rejects any
underscore in the input (must be lowerCamelCase). Closes
FieldMask{TooManyUnderscore,PathsDontRoundTrip,
NumbersDontRoundTrip}.JsonOutput and JsonInput.FieldMaskInvalidCharacter.
The pre-existing "drop unknown enum strings" unit regressions in
test/conformance_test.lua were inverted to assert the new error
shape. New strict-validation regressions in test/json_test.lua pin
all six categories so they don't regress; the `json.strict` group
runs across both codegen modes via the shared descriptor table.
codec: preserve -0.0 for proto3 float/double scalars
Proto3 default-elision dropped any singular float/double whose value
compared equal to 0 — `-0.0 == 0.0` in IEEE so the `if v ~= 0` guard
silently elided negative zero. The wire bytes for -0 differ from +0
and the TextFormatInput conformance corpus pins that -0 must survive
a round-trip; the failure surfaced as 10 unexpected text-suite
regressions covering FloatFieldNegativeZero (3 spellings × 2 outputs)
and Neg{Float,Double}FieldLargeNegativeExponentParsesAsNegZero
(2 types × 2 outputs).
Sign-bit guard via `1/v == math.huge` (positive zero yields +inf,
negative zero yields -inf). Applied in three layers:
* runtime/pb/codec.lua: is_default_scalar + the specialized
monomorphic writer for non-optional numeric scalars
* runtime/pb/text.lua: is_proto3_default (text encoder elision)
* inline codegen: scalarNotDefaultExpr emits the guard for
float/double fields in full-mode `_encode` functions
Proto3 text-format conformance now sits at 416 ✓ / 18 skipped / 0
expected failures; binary+JSON holds at 1478 ✓. Unit-test regressions
cover both directions (preserved on encode + decode round-trip, +0
still elided) and use a runtime-computed -0.0 sentinel because LuaJIT
can constant-fold the literal `-0.0` to a sign-less zero in some
load paths.
text: add pb.text.decode and wire it into conformance dispatch
Hand-written recursive-descent parser for the textproto grammar
covering every bucket the proto3 conformance suite exercises:
decimal/hex/octal integer literals with full 32/64-bit range checks,
float specials (inf/infinity/nan any case, oversize exponents
saturating to ±inf, underflows to ±0), C-style + \u/\U string escapes
with adjacent-literal concat and surrogate rejection, aggregate {} /
<> bodies, repeated short-form `[a, b, c]`, `key: K value: V` map
entries, the `[type.googleapis.com/...]` inline Any form alongside
the direct `type_url:`/`value:` form, enum-by-name-or-number,
reserved-name silent drop, numeric-field-ID tolerance, and
duplicate-singular-field rejection.
Plugin gains a small reserved_names emitter so the parser can match
mainline TextFormat::Parser's "silently drop reserved" rule. The Any
WKT descriptor advertises its real fields (type_url + string,
value + bytes) so the generic body walker can populate it directly
when the input doesn't use the inline-URL form.
cmd/conformance/core.lua stops short-circuiting text_payload to
`skipped` and runs it through pb.text.decode. The proto3 TextFormat
input suite climbs from 8 ✓ / 426 skipped to 406 ✓ / 18 skipped / 10
expected failures. The 10 surviving failures all share one cause
(proto3 -0.0 elision in the codec, not a parser bug — documented in
test/conformance/known_failures_text.txt). Binary+JSON conformance
holds at 1478 ✓. 591 unit tests pass across both codegen modes.
Closes the text-conformance-output branch.
license: BSD 2-Clause
Matches Tarantool's own license. Closes the "TBD" placeholder in the
README.
wkt: split length-prefix emission to drop per-field concat
Apply the split-emit pattern from ac9b9c1 (inline.go + codec.lua) to the
WKT hand-rolled encoders. Replace `wire.encode_len(body)` — which
allocates `varint(#body) .. body` — with three out slots (tag, varint
length, body) wherever the result feeds a `table.concat(out)`
accumulator. For StringValue/BytesValue (whose encoders return a single
string and can't be split), inline encode_len's chain so LuaJIT folds
tag + varint + body into one multi-concat instead of two sequential
concats. Same final wire bytes.
Touched sites: struct_encode (split the outer entry wrap, inline the
inner key/value chain), list_encode, any_encode (type_url, value),
fieldmask_encode, and the WRAPPERS LEN-typed encoders.
The struct_encode inner entry stays a single chain rather than fanning
out into more `out` slots — eight slots per Struct entry regressed
B/op and gave no encode win on the wkt-event shape; three slots match
the codec.lua nested-message pattern and land the speedup.
bench/shapes (wkt-event encode, median across 4 runs):
full: 125895 -> 129000 msgs/s (+2.5%)
runtime: 115231 -> 120000 msgs/s (+4%)
make test 509/509, make jit-trace 23/23, docker conformance binary
1478/0 (unexpected), text suite clean.
docs: handoff brief for the pb.text.decode slice
Self-contained brief targeting the 408 proto3 TextFormatInput tests
still skipped after the encoder + SGROUP work. Covers existing
surface (text.lua encode-only, parser.lua schema parser, conformance
dispatch), grammar buckets the suite exercises (10 categories with
upstream test counts), files to touch, LuaJIT/cdata/box.NULL
conventions, and a verifying-locally checklist. Suggested
recursive-descent shape mirrors runtime/pb/parser.lua's cursor
mechanics.
No code changes; positions the next slice without committing to a
specific implementation schedule.
text: render captured unknown fields, tolerate SGROUP in skip
Two changes close the proto3 text-format conformance suite:
1. `wire.skip_field` learns SGROUP/EGROUP. Wire 3 recurses through
inner tags until a matching EGROUP, with field_id checked against
the SGROUP's id. Callers (codec.lua, lazy.lua, wkt.lua, generated
full-mode `_pb.lua`) now pass the tag's field_id so groups inside
unknown-field skips don't error.
2. `pb.text.encode` walks the captured `_unknown_fields` buffer when
`opts.print_unknown_fields=true` and emits each entry in
TextFormat numeric-field form:
VARINT -> "<id>: <uint64>"
I64/I32 -> "<id>: 0x<hex>"
LEN -> speculative "<id> { <recurse> }"; rolls back to
byte-string form if the inner bytes don't parse as a
sub-message
SGROUP -> "<id> { <recurse> }" through matching EGROUP
`cmd/conformance/core.lua` threads `req.print_unknown_fields` into
`pb.text.encode` so `_Drop` tests drop unknowns and `_Print` tests
render them.
Conformance: text-format suite goes from 2 ✓ / 6 expected fails to
8 ✓ / 0 expected fails. All eight regression tests in
`conformance_test.lua` (one per upstream test, plus the fixed field-1011
tag bytes that were miscomputed earlier) now assert the target output.
test: pin unknown-fields text-format conformance regressions
Adds eight tests under conformance.core mirroring the
Recommended.Proto3.ProtobufInput.*UnknownFields_*.TextFormatOutput tests
in the Google harness. Each sends the byte-exact upstream payload (field
IDs 1001..1011 from UnknownToTestAllTypes) through cmd/conformance/core
and pins current output, with the target assertion + needed runtime fix
documented in each failure message.
Two blockers surface:
1. wire.skip_field rejects SGROUP/EGROUP (wire 3/4), so the four
Group/Repeated tests fail at decode.
2. pb.text.encode doesn't walk _unknown_fields, so the four *_Print
tests serialize empty.
Lets us iterate on those fixes locally without the Docker round-trip.
text: wire pb.text into conformance runner TEXT_FORMAT output
The runner short-circuited every TEXT_FORMAT request to `skipped`, even
though pb.text.encode has been encode-capable since M7. Plug it into
cmd/conformance/core.lua so protobuf/JSON input → text output exercises
the existing encoder end-to-end. Text-format input remains deferred
(pb.text is encode-only).
Text-format suite: 0 ✓ / 430 skipped / 4 expected fails →
2 ✓ / 426 skipped / 6 expected fails (Scalar/Message
*_Print added — unknown-field rendering still missing).
bench: per-helper wire bench + shape-variety bench + multi-byte jit gates
Three additions so future perf regressions are visible at the right
granularity, not just averaged out by the single Person shape in
bench/bench.lua.
bench/wire_bench.lua (new). Microbenches every wire helper in
isolation — encode/decode for varint at 1/2/3/5 bytes, zigzag, fixed
widths, float/double, LEN, tag, skip_field, utf8 validator. Used when
tuning wire.lua to confirm a change moved the helper-level ns/op as
expected (e.g. encode_varint(150) 529 ns -> 68 ns from the 2-byte
fast path). Run via `make bench-wire`. No baseline, no regression
gate — this is a manual inspection tool.
bench/shapes_bench.lua (new). Runs encode + decode against a handful
of fundamentally different Person / Event / Result shapes —
scalar-heavy, packed-int (100 and 1000 elements), nested-friends
(10 and 100), maps (scalar-valued and message-valued), oneof, and
WKT-heavy. Each shape dials one knob up so cost attribution stays
clean. Shows alloc B/op alongside throughput. Run via
`make bench-shapes`. Caught the packed-int / multi-byte-varint
opportunity that bench/bench.lua (all 1-byte varints, fixed
shape) doesn't surface.
bench/jit_trace.lua + 4 gates. Two new fixtures per mode that
specifically exercise encode_varint_slow's 2/3-byte Lua-number paths
and decode_string's multi-byte LEN fallback (200-byte name + packed
ints in [150..500000]). If a future change pushes encode_varint_slow
past LuaJIT's inline budget the gate fires instead of the regression
landing silently in shapes_bench. Total gate count: 19 -> 23.
Makefile gains `bench-wire` and `bench-shapes` phony targets.
codegen+wire: split length-prefix emission + 2-byte varint fast path
Two complementary encode optimizations: codegen + runtime writers no
longer pay a per-field string concat for length-prefixed fields, and
encode_varint_slow now handles 2/3/4-byte values without dropping into
the uint64 cdata path.
Composite bench/bench.lua, full mode:
1KB encode: 229 K -> 332 K msgs/s (1.45x)
10KB encode: 38 K -> 56.5 K (1.49x)
100KB encode: 4.1 K -> 6.7 K (1.64x)
Runtime mode now matches full mode at 10KB+:
1KB encode: 203 K -> 276 K (1.36x)
10KB encode: 35 K -> 55 K (1.56x)
bench/shapes_bench.lua biggest swings (both modes):
packed-int32x1000 enc: 1.7 K -> 13 K (7.0-7.6x)
packed-int32x100 enc: 21 K -> 113 K (5.3x)
scalar-heavy enc: 582 K -> 1.0 M (1.7x)
Per-helper, bench/wire_bench.lua:
encode_varint(150) [2-byte]: 529 ns -> 68 ns (7.8x)
encode_varint(20K) [3-byte]: 841 ns -> 88 ns (9.6x)
encode_tag(16, LEN) [2-byte]: 531 ns -> 78 ns (6.8x)
encode_string(200B) [2-byte L]: 560 ns -> 109 ns (5.1x)
encode_varint(127) [1-byte]: 29 ns -> 28 ns (no regression)
1. Split length-prefix emission. Length-delimited fields (string/bytes
scalars, nested messages, packed scalars/enums) used to emit
`tag → encode_len(body)` where `encode_len(body)` returns
`encode_varint(#body) .. body`. The string concat allocated a copy
of the body per field. Now the codegen and runtime writers emit
three separate `out` slots — `tag`, `varint(#body)`, `body` — and
let `table.concat(out)` join them in one pass at the end of encode.
Applied to inline.go (full-mode codegen) for: singular and repeated
nested messages, singular and repeated string/bytes scalars, packed
scalars, packed enums. Applied to codec.lua build_writer / build_
repeated_writer for the same set. Maps still go through encode_len
pending a separate pass.
2. encode_varint_slow Lua-number fast paths. The outer encode_varint
stays at the tiny `1-byte check + tail call` shape that LuaJIT
inlines into hot traces. The slow function (not inlined into hot
traces, so its body size is unconstrained) now handles non-negative
Lua numbers up to 2^28 directly via bit.rshift / string.char with no
cdata allocation. Values in [2^28, 2^53) emit one byte and recurse
on the smaller residue. Only cdata inputs, negative Lua numbers
(sign-extended to 10-byte varint), and the rare > 2^53 case still
take the uint64 cdata loop. Net effect: every multi-byte varint
encode that fits in a Lua number — including the length prefix for
any string >= 128 bytes, every tag for field IDs >= 16, and every
negative-zigzag sint — drops from ~500 ns to ~70 ns.
497/497 luatest pass. 23/23 jit-trace gates pass (the two new
multi-byte varint gates added in the bench infra commit confirm
encode_varint_slow JIT-compiles cleanly). Conformance suite (binary
+ text) shows 1478 expected passes, 0 unexpected failures.
wire: decode wins — utf8.len validator + fast paths + FFI cast
Five focused decode optimizations, all in wire.lua, landing a 3.9-6.6x
speedup on bench/bench.lua decode and matching wins on lazy / shapes
benches. Per-helper numbers from bench/wire_bench.lua:
is_valid_utf8(32B ASCII): 545 ns -> 36 ns (15x)
is_valid_utf8(1KB ASCII): 16329 ns -> 539 ns (30x)
decode_string(32B): 140 ns -> 80 ns (1.75x)
decode_double: 299 ns -> 226 ns (1.32x)
decode_fixed64: 196 ns -> 184 ns (1.07x)
1. utf8.len swap. Pure-Lua RFC 3629 validator replaced by
`utf8.len(s) ~= nil` (ICU U8_NEXT-backed at src/lua/utf8.c:165).
Source-verified to reject every proto3 case: stray continuation,
overlong, surrogates, truncated, > U+10FFFF, 5-byte+ sequences.
Conformance suite still at 1478/1478 expected passes.
2. decode_string / decode_bytes 1-byte LEN fast path. Strings <=127
bytes (the RPC common case) skip two function-call layers
(decode_string -> decode_len -> decode_varint).
3. decode_float / decode_double via ffi.cast(uint8_t*, buf) instead
of buf:sub. Eliminates the per-call string-slice allocation.
4. decode_double Inf/NaN check via a uint32[2] union split. Bit ops
on Lua numbers don't allocate; the previous uint64 cdata path
produced 3+ intermediates per call.
5. decode_fixed64 reads via the new pb_u64_u_t union (ffi.copy +
read .u). Replaces UINT64(lo) + bit.lshift(UINT64(hi), 32) which
allocated 2-3 cdata per call.
All FFI cdef/union locals hoisted to the top of the file so the
fixed-width decoders all reference the same scratch buffers.
497/497 luatest pass. 19/19 jit-trace gates pass. Conformance
(binary + text) shows zero unexpected failures.
json: strict proto3 JSON conformance — close all Required failures
The conformance suite's Required JSON failures collapse to zero with
this pass. Local tests grow from 456 to 497 to pin every fix.
Decode side — `runtime/pb/json.lua`:
* Strict scalar validators per type. Numeric strings must match the
JSON number grammar (no leading whitespace, no partial numerics);
out-of-range values, NaN/Inf surrogates from JSON literals, and
type mismatches all become parse_error.
* Quoted exponential ints ("1e5" -> 100000) are accepted via the
JSON-number grammar path, matching Int32FieldQuotedExponentialValue.
* Timestamp parser: strict RFC 3339 (uppercase T/Z, ±HH:MM offset,
≤9 frac digits, range check). Output uses a portable Hinnant-style
epoch_to_ymdhms so year 0001 zero-pads correctly — glibc's POSIX
%Y emits "1" for that year, breaking round-trip.
* Duration parser/formatter: mandatory `s` suffix, ±10000-year range,
sign-matching nanos, 0/3/6/9-digit fractional output.
* Any: WKT-aware nesting under "value"; empty Any -> {}; Empty WKT
inside Any omits "value" (reference parser rejects {"value":{}});
@type URL without `/` rejected; empty @type with sibling fields
rejected.
* Reject NaN/Infinity in google.protobuf.Value.number_value (no JSON
literal for these).
* Duplicate oneof branches rejected; a null oneof branch does NOT
count as set, so a sibling non-null branch is unambiguous.
* Top-level JSON null rejected for messages; preserved for Value.
* Repeated/map values must be JSON array/object (not bare scalar).
Encode side — hand-rolled JSON emitter:
Tarantool's `json.encode` uses a fixed global precision so doubles
like 0.1 don't round-trip and we can't change it per-value without
polluting other users. Replace with a minimal emitter that picks the
shortest-round-tripping precision (15 -> 16 -> 17) per double and
handles NaN/Inf as quoted sentinel strings.
`runtime/pb/wkt.lua`: `timestamp_decode` now keeps invalid Timestamps as
a raw {seconds, nanos} table when `datetime.new` rejects them (negative
nanos, year > 9999, ...) so the JSON encoder can produce
serialize_error rather than the binary decoder raising parse_error.
Required by the Timestamp conformance suite.
`test/conformance_test.lua`: 41 new regression tests grouped under
Fixes 12-19, pinning every code path touched. Strict scalar
validators (one per rejection shape per type), Timestamp/Duration
strict parsing and canonical output, Any WKT/non-WKT/Empty handling,
Value NaN/Inf rejection, ValueAcceptNull round-trip, LuaJIT
NaN-boxing collision cases (0x7FFBCBA987654321, all-ones), shortest
double round-trip, oneof-null semantics.
`test/conformance/known_failures.txt`: refreshed. 15 Recommended-only
failures remain (FieldMask round-trip quirks, duplicate-field-name
detection, unknown-enum-string rejection, null-element-in-list,
NullValue oneof validator).
wire: bypass LuaJIT NaN-boxing collision in decode_float/decode_double
Reading a NaN-payload double back through `tonumber(F.d)` runs into
LuaJIT 2.1's NaN-boxed value representation: certain IEEE NaN bit
patterns collide with internal type tags (nil, function, ...) so
`tonumber` yields a non-number and the field is silently dropped
during encoding.
Inspect the raw bit pattern via the uint32_t/uint64_t aliases of the
float/double unions before reaching for the Lua-level value. When the
exponent is all-ones we resolve Inf/NaN ourselves; only normal values
hit `tonumber`.
Closes the conformance suite's DoubleFieldNormalizeSignalingNan and
FloatFieldNormalizeSignalingNan JsonOutput tests, which used bit
patterns (0x7FFBCBA987654321, 0x7FBFFFFF) that hit the collision.
json: enable conformance JSON output + close 459 tests
Three coupled changes that turn the JSON output path on for the
conformance harness:
1. string_to_int64 accepts cdata: Tarantool's json.decode parses raw
JSON integer literals outside double range as int64_t/uint64_t cdata,
not Lua numbers. The decoder errored "expected JSON string or number
for int64" on any unquoted 64-bit value (Int64FieldMaxValueNotQuoted
et al). cdata is now cast through directly, preserving precision.
2. encode_message marks output as a map: an empty proto3 message
serialized as `[]` because Tarantool's json defaults empty tables to
array shape. jsoncpp's strict comparator threw Json::LogicError and
aborted the whole suite. Setting __serialize='map' on the output
gives `{}` and unblocks all JsonOutput tests.
3. PB_CONFORMANCE_SKIP_JSON gate is opt-in by default. core.lua now
matches exactly "1" (so docker -e VAR= disables it), and the
Dockerfile no longer hard-codes "=1" — JSON output runs end-to-end
for everyone unless they re-enable the gate.
Conformance moves from 930 / 1869 / 11 to 1389 / 1313 / 79
(successes / skipped / expected fails). The 75 new expected fails
are canonical-form edge cases (Duration formatting sign handling,
Timestamp out-of-range rejection, double precision digits, NaN
canonicalization, JSON-input strict rejection) — left for a follow-up.
Drops 7 entries from test/conformance/known_failures.txt that this
change closes; adds 75 newly-visible ones.
json: canonical lowerCamelCase + NullValue WKT descriptor
Two unrelated JSON-decoder gaps captured under the conformance triage:
1. to_camel's gsub pattern '_(%w)' didn't match consecutive underscores
and didn't drop trailing underscores, so proto names like
__field_name13 / field__name4_ / field_name17__ generated JSON keys
that didn't match what protoc produces. The fix strips trailing _+
and collapses '_+%w' to a capitalized letter; a leading underscore
thus capitalizes the next character, matching the spec.
2. pb.wkt didn't export a descriptor for google.protobuf.NullValue, so
any enum field whose type is NullValue (oneof_null_value, or the
implicit one inside Value) crashed decode_enum with "attempt to
index a nil value." Adds a minimal {by_name, by_value} descriptor.
Drops 3 entries from test/conformance/known_failures.txt
(FieldNameInSnakeCase, FieldNameWithDoubleUnderscores,
NullValueInOtherOneofOldFormat). Adds 4 regression tests covering
double underscore, leading underscore, trailing underscore, and the
NullValue enum decode path.
json: treat null fields as absent (and Value's null as a real value)
Per the proto3 JSON spec, a null on any field means "use the field's
default" — encoded as missing — with the lone exception of
google.protobuf.Value, where JSON null is itself a Value carrying
NullValue.NULL_VALUE.
Three coupled bugs surfaced together:
1. decode_field_value used to fall through with v = box.NULL, leaving a
useless box.NULL sitting in the result table for scalars. Now it
returns nil for non-Value fields, PB_NULL for Value fields.
2. decode_message's repeated and map branches called `#jv` and
`pairs(jv)` unconditionally; a JSON-null on either type crashed with
"attempt to get length of 'void *'". Now both branches short-circuit
when jv is box.NULL.
3. The nil-skip checks in the decode loop (`if dv ~= nil`) and in the
codec / inline message encoders (`if v == nil then return end`)
evaluated TRUE on box.NULL because Tarantool's cdata __eq aliases
it to nil. Decode now uses rawequal(dv, nil); encode special-cases
message kind by also accepting cdata, so the Value field's
box.NULL sentinel survives all the way through to value_encode.
Drops 3 entries from test/conformance/known_failures.txt
(AllFieldAcceptNull, WrapperTypesWithNullValue, ValueAcceptNull).
Adds 5 regression tests covering scalar / repeated / map / wrapper
null treatment and the Value-NULL_VALUE exception.
wire: reject invalid UTF-8 in proto3 string fields
Per the proto3 spec, a string field's bytes must form valid UTF-8.
decode_string was aliased to decode_len, so any byte sequence was
accepted and round-tripped. Adds a pure-Lua RFC 3629 validator —
is_valid_utf8 — and routes M.decode_string through it. The bytes
type keeps the raw decode_len path so binary payloads still pass.
The validator covers stray continuation bytes, truncated sequences,
overlong encodings, UTF-16 surrogates (U+D800..U+DFFF), and code
points above U+10FFFF.
Codec, lazy, dynamic, and the generated inline code all consume the
same M.decode_string, so singular / repeated / oneof / map-key /
map-value string fields are all covered.
Drops 5 entries from test/conformance/known_failures.txt
(RejectInvalidUtf8.String.*) and adds 7 regression tests covering
each invalid form plus a positive multi-byte string round-trip and
a bytes-field control.
wire: validate field number and minimal encoding in decode_tag
Three checks added on the multi-byte path:
1. Field number must be > 0 (IllegalZeroFieldNum_Case_0/1/3).
2. Field number must fit in 29 bits per the protobuf spec
(BadTag_FieldNumberSlightlyTooHigh, BadTag_FieldNumberTooHigh).
3. Tag varint must be minimally encoded — a trailing 0 byte with more
than one byte read is overlong (BadTag_OverlongVarint).
The field-number check runs on bit ops over the uint64 cdata returned
by decode_varint rather than after tonumber, otherwise field numbers
above 2^32 alias into the valid range (e.g., fn=2^31+1 was being
recovered as fn=1).
The single-byte fast path picks up the field-zero check directly via
the b >> 3 == 0 condition.
Drops 6 entries from test/conformance/known_failures.txt and adds 4
regression tests pinning each rejection path.
codec: recursively merge repeated singular-message wire entries
Per proto3 spec, when the same singular message field (including a oneof
branch) appears twice on the wire, the two values must merge: scalar
fields last-wins, repeated fields concatenate, sub-messages merge
recursively, maps last-wins per key. The previous reader replaced the
prev value wholesale for oneof branches and overwrote repeated/nested
fields with `prev[k] = v` even outside oneofs, losing data unique to
the first occurrence.
Adds wire.codec.merge_message(desc, prev, decoded), a descriptor-driven
recursive merge, and routes both codec.lua's reader and the inline-mode
codegen through it. WKT message fields (custom decode) keep the replace
behavior because their decoded value is not a generic Lua table.
Exposes pb.codec to the generated inline code so the helper is reachable
without a per-call require.
Drops 3 entries from test/conformance/known_failures.txt:
ValidDataOneof.MESSAGE.Merge, ValidDataOneofBinary.MESSAGE.Merge,
RepeatedScalarMessageMerge. Adds 5 regression tests covering scalar
last-wins, oneof merge, recursive sub-message merge, repeated-in-
submessage concat, and oneof sibling clearing after merge.