codec,text,json: proto2 extensions end-to-end
Plugin iterates file.Extensions and per-message Extensions, emitting
\`pb.register_extension(extendee_desc, {…})\` calls. Each extension
becomes a field descriptor attached to the extendee under two
indices: extensions_by_id (wire-decode lookup) and
extensions_by_full_name (encode + textual emission ordering).
Codec encode_message walks data._extensions after the regular field
loop and calls encode_field through the extension's descriptor.
decode_message routes an unknown tag through decode_extension when
extensions_by_id matches; the helper mirrors the in-line dispatch
on field kind (scalar/enum/message/group, singular/repeated) and
stores into result._extensions[full_name].
Text decoder recognizes bracket syntax \`[pkg.ext_name]\` outside an
Any-target and looks the name up in extensions_by_full_name. Names
that don't resolve (e.g. \`[pkg.GroupField]\` for the CapitalCase
group type instead of the lowercase field name) raise a parse error
per spec. Text encoder emits each set extension as
\`[full.name]: value\` or \`[full.name] { … }\`.
JSON encoder emits set extensions under the bracketed full-name key
(\`"[pkg.ext_name]": value\`). JSON decoder recognizes the same
shape and stores into _extensions; unknown bracket-names stay
silently dropped to match the spec's tolerant behavior for unknown
JSON keys.
Extensions on google.protobuf.* descriptors (file/message/field
option extends) are skipped — those are meta-only and our WKT
module doesn't surface their descriptors.
Conformance baseline (strict --enforce_recommended, protobuf v34.1):
Binary + JSON suite: 2806 ✓ / 0 failures (was 1493 ✓ / 1313 skipped)
Text-format suite: 434 ✓ / 0 failures (was 416 ✓ / 18 skipped)
100% pass.
text,codec: proto2 group text syntax + closed-enum semantics
Text decoder now resolves a group field reference by the group's
capitalized submessage name (e.g. \`Data\`, \`MultiWordGroupField\`)
and by the ASCII-lowercase fold (\`data\`, \`multiwordgroupfield\`),
in addition to the existing lowercase field-name lookup. The \`:\`
between field and \`{\` is optional for groups (matches the
message/map rule).
Proto2 enums are closed: parse_enum_value now rejects an integer
literal that doesn't map to any declared value. The plugin emits
\`closed = true\` on every proto2 enum descriptor (driven by
protoreflect's IsClosed()); proto3 enums stay open to preserve
forward-compatibility on the wire.
Conformance text-format suite: 16 unexpected failures → 3, all in
the remaining extension-bracketed-group cases.
codec: proto2 groups (SGROUP/EGROUP wire format)
Plugin: detect protoreflect.GroupKind, emit kind='group' in the
field descriptor and a SGROUP opening tag in inline codegen. The
group body bypasses the LEN-prefix path entirely — encoder emits
start-tag, body bytes, end-tag in three slots; decoder calls
pb.codec.decode_group(desc, buf, pos, field_id) which reads inner
tags until the matching EGROUP id.
Wire layer already understood SGROUP/EGROUP for skip_field; the new
decode_group reuses the per-tag dispatch from decode_message but
stops on EGROUP instead of end-of-buffer. Repeated groups bracket
each element with its own SGROUP/EGROUP pair.
Runtime parser now desugars `optional|required|repeated group Name
= id { body }` into (a) a nested message named Name and (b) a
synthetic field of kind=group whose lowercased name is `name` — so
source-parsed proto2 schemas behave the same as build-time codegen.
Text format renders groups under the submessage's capitalized name
(`SingleGroup { ... }`) rather than the lowercase field name,
matching mainline protoc's convention.
Adds a vendored, MessageSet-stripped test_messages_proto2.proto in
test/conformance/proto/ (header comment documents the patch).
10 new luatest cases (group_*) cover singular + repeated + text +
descriptor kind, across both codegen modes. 739 tests pass.
json,text: presence-tracked fields skip proto3 default elision
Both codecs gated default-value elision on `f.optional or f.oneof` —
the proto3 implicit-presence shape. That elides proto2 `required`
fields set to zero and any other presence-tracked descriptor that
doesn't carry the proto3 explicit-optional flag.
Extend the bypass to `f.required` so a Cardinality{r=0} survives
the round-trip through pb.json.encode / pb.text.encode. 4 new
luatest cases pin the rule.
runtime: parser + dynamic accept proto2 sources
Parser now captures `required=true` and `default_value=…` from the
proto2 keywords (instead of dropping `required` silently and ignoring
field options). Dynamic descriptor builder reads `parsed.syntax`,
flips the repeated-scalar packing default for proto2, and calls
`codec.compile_writers/compile_readers` so the per-field
required-writer specialization actually fires — without that the
generic encode_field path silently elides missing required fields.
64-bit-int defaults are coerced into the appropriate cdata type
inside `coerce_default` so the codec's value-comparison rules
match what generated code emits.
Adds 7 dynamic-mode luatest cases including a static-vs-dynamic
byte-parity check. 725 tests pass.
codegen: proto2 baseline — required, optional, custom defaults
Lifts the proto3-only syntax gate in the plugin and threads three
new field-descriptor attributes through codegen and the codec:
* required=true — fields declared with the proto2 `required` keyword.
Inline codegen and the runtime codec both error when
a required field is missing on encode (vs the silent
elide that proto3 implicit-presence fields get).
* optional=true — already wired for proto3 explicit `optional`; in
proto2 every singular field carries it via the
existing HasOptionalKeyword() check, giving presence
semantics without a separate emission path.
* default_value=… — proto2 [default = X] from the field descriptor,
rendered as a Lua literal (cdata for 64-bit ints,
symbolic name for enums) so consumers can surface
it; the codec itself does not auto-materialize
defaults on decode, matching how proto3 absent
fields stay nil.
Packed-by-default already flips correctly because we ask
protoreflect's `IsPacked()`, which is syntax-aware.
Adds test/proto/proto2_basic.proto with 33 luatest cases covering
required validation, optional presence, custom defaults, the proto2
unpacked-by-default repeated rule, nested-required messages, and
full-vs-runtime mode parity. `just gen-proto2-tests` regenerates the
fixture into examples/expected/{full,runtime}/.
Out of scope: extensions, extend, group; conformance harness still
skips TestAllTypesProto2.
json: M.encode(desc, t, opts) for use_proto_names / emit_defaults / indent
Canonical protojson options were silently ignored. Plumb a single opts
table through encode_message via a CURRENT_ENCODE_OPTS state (mirroring
the decode-side CURRENT_OPTS):
* use_proto_names — emit snake_case field names (proto wire
names) instead of the spec-default
lowerCamelCase.
* emit_defaults — emit zero-valued implicit-presence scalars,
(alias: always_emit_zero_value) empty repeated lists, and empty
maps. Explicit-presence fields (optional /
oneof) and singular message fields remain
absent — matches protobuf-go's behavior.
* indent — pretty-print with the given indent string;
empty arrays/objects stay compact.
Tests parameterize over both codegen modes (descriptor-table contract is
shared between full and runtime) plus a dedicated parity group that
asserts byte-identical JSON across modes for each option.
json: emit unbroken base64 for bytes fields ({nowrap = true})
Canonical proto3 JSON expects RFC 4648 unwrapped base64, but Tarantool's
`digest.base64_encode` defaults to RFC 2045 MIME-style 76-char line
wrapping. Any `bytes` payload past ~57 bytes used to land in the JSON
string with an embedded `\n`, which breaks every spec-compliant
consumer (grpc-gateway, protojson, protobuf-go's JSON, ...).
Pass `{nowrap = true}` at the two encode sites: the per-field bytes
encoder and the `google.protobuf.Any` opaque-fallback `value` encoder.
Surfaced by tarantool-etcd's `TestJSONGatewayBytesUnwrapped` over Range
responses whose `value` is >=57 bytes; its reference grpc-gateway never
emits the wrapped form.
codegen: emit strict <Type>_fields / _oneofs constants for lazy view
Lazy-view callers passing a typo'd field name to :get / :has / :set /
:clear / :which got `nil` back, indistinguishable from a legitimately-
absent optional field. Failures surfaced as missing data downstream.
Each generated message now exports a M.<Type>_fields table mapping
each field name to itself (and M.<Type>_oneofs for oneof groups),
wrapped by a new pb.field_names() helper that errors on unknown-key
reads and on any write. Routing field-name arguments through these
tables turns a typo into a load-time error at the read site.
Eager _encode / _decode keep round-tripping plain Lua tables — the
constants table is a lazy-view contract, documented in
docs/api-modes.md. README and lazy_test.lua converted to the new
pattern; three new tests cover typo / read-only / oneof-typo errors.
json: strict validation pass closes the proto3 conformance suite
Six classes of relaxation that the proto3 JSON conformance corpus
flagged are now enforced. All as Recommended.* tests; combined with
the -0.0 codec fix this empties known_failures.txt and brings the
proto3 binary+JSON suite to 1493 ✓ / 1313 skipped / 0 expected
failures / 0 unexpected failures.
1. Duplicate JSON keys. Tarantool's json.decode is hash-backed and
silently collapses `{"foo":1,"foo":2}` to one entry. A small
byte-walker `find_duplicate_json_keys` runs before json.decode,
tracks per-object brace frames and key sets, errors on the
second occurrence. Closes Recommended.FieldNameDuplicate.
2. camelCase / snake_case aliases of the same proto field appearing
side-by-side. Detected inside decode_message via a `field_seen`
set keyed by proto-name; second hit errors. Closes
FieldNameDuplicateDifferentCasing{1,2}.
3. JSON null inside repeated arrays and map values. Previously
silently dropped; now errors before decode_field_value. Closes
RepeatedField{Message,Primitive}ElementIsNull and
MapFieldValueIsNull.
4. Unknown enum *names* (not integers). decode_enum used to return
nil so callers silently dropped them; now raises by default and
returns nil only when M.decode's `ignore_unknown_fields=true`
opt is set. Conformance dispatch in cmd/conformance/core.lua
forwards this flag when req.test_category ==
JSON_IGNORE_UNKNOWN_PARSING_TEST. Closes
RejectUnknownEnumStringValueIn{Optional,Repeated,Map} and the
paired IgnoreUnknownEnumStringValueIn* tests.
5. google.protobuf.NullValue JSON canonical form. The single enum
value renders as the literal JSON `null` (not the string
"NULL_VALUE"); decode accepts either, encode emits null. The
decode_field_value null-handling path also treats a JSON null
on a NullValue-typed field as "set" rather than "absent" so a
oneof gets marked active. Closes
NullValueInOtherOneof{New,Old}Format.Validator.
6. FieldMask strict round-trip. Path validity is checked on both
sides: the snake_case wire form rejects uppercase letters,
consecutive underscores, trailing underscore, and underscore
followed by anything other than a lowercase letter — these
break the snake↔camel round-trip. The JSON form rejects any
underscore in the input (must be lowerCamelCase). Closes
FieldMask{TooManyUnderscore,PathsDontRoundTrip,
NumbersDontRoundTrip}.JsonOutput and JsonInput.FieldMaskInvalidCharacter.
The pre-existing "drop unknown enum strings" unit regressions in
test/conformance_test.lua were inverted to assert the new error
shape. New strict-validation regressions in test/json_test.lua pin
all six categories so they don't regress; the `json.strict` group
runs across both codegen modes via the shared descriptor table.
codec: preserve -0.0 for proto3 float/double scalars
Proto3 default-elision dropped any singular float/double whose value
compared equal to 0 — `-0.0 == 0.0` in IEEE so the `if v ~= 0` guard
silently elided negative zero. The wire bytes for -0 differ from +0
and the TextFormatInput conformance corpus pins that -0 must survive
a round-trip; the failure surfaced as 10 unexpected text-suite
regressions covering FloatFieldNegativeZero (3 spellings × 2 outputs)
and Neg{Float,Double}FieldLargeNegativeExponentParsesAsNegZero
(2 types × 2 outputs).
Sign-bit guard via `1/v == math.huge` (positive zero yields +inf,
negative zero yields -inf). Applied in three layers:
* runtime/pb/codec.lua: is_default_scalar + the specialized
monomorphic writer for non-optional numeric scalars
* runtime/pb/text.lua: is_proto3_default (text encoder elision)
* inline codegen: scalarNotDefaultExpr emits the guard for
float/double fields in full-mode `_encode` functions
Proto3 text-format conformance now sits at 416 ✓ / 18 skipped / 0
expected failures; binary+JSON holds at 1478 ✓. Unit-test regressions
cover both directions (preserved on encode + decode round-trip, +0
still elided) and use a runtime-computed -0.0 sentinel because LuaJIT
can constant-fold the literal `-0.0` to a sign-less zero in some
load paths.
text: add pb.text.decode and wire it into conformance dispatch
Hand-written recursive-descent parser for the textproto grammar
covering every bucket the proto3 conformance suite exercises:
decimal/hex/octal integer literals with full 32/64-bit range checks,
float specials (inf/infinity/nan any case, oversize exponents
saturating to ±inf, underflows to ±0), C-style + \u/\U string escapes
with adjacent-literal concat and surrogate rejection, aggregate {} /
<> bodies, repeated short-form `[a, b, c]`, `key: K value: V` map
entries, the `[type.googleapis.com/...]` inline Any form alongside
the direct `type_url:`/`value:` form, enum-by-name-or-number,
reserved-name silent drop, numeric-field-ID tolerance, and
duplicate-singular-field rejection.
Plugin gains a small reserved_names emitter so the parser can match
mainline TextFormat::Parser's "silently drop reserved" rule. The Any
WKT descriptor advertises its real fields (type_url + string,
value + bytes) so the generic body walker can populate it directly
when the input doesn't use the inline-URL form.
cmd/conformance/core.lua stops short-circuiting text_payload to
`skipped` and runs it through pb.text.decode. The proto3 TextFormat
input suite climbs from 8 ✓ / 426 skipped to 406 ✓ / 18 skipped / 10
expected failures. The 10 surviving failures all share one cause
(proto3 -0.0 elision in the codec, not a parser bug — documented in
test/conformance/known_failures_text.txt). Binary+JSON conformance
holds at 1478 ✓. 591 unit tests pass across both codegen modes.
Closes the text-conformance-output branch.
wkt: split length-prefix emission to drop per-field concat
Apply the split-emit pattern from ac9b9c1 (inline.go + codec.lua) to the
WKT hand-rolled encoders. Replace `wire.encode_len(body)` — which
allocates `varint(#body) .. body` — with three out slots (tag, varint
length, body) wherever the result feeds a `table.concat(out)`
accumulator. For StringValue/BytesValue (whose encoders return a single
string and can't be split), inline encode_len's chain so LuaJIT folds
tag + varint + body into one multi-concat instead of two sequential
concats. Same final wire bytes.
Touched sites: struct_encode (split the outer entry wrap, inline the
inner key/value chain), list_encode, any_encode (type_url, value),
fieldmask_encode, and the WRAPPERS LEN-typed encoders.
The struct_encode inner entry stays a single chain rather than fanning
out into more `out` slots — eight slots per Struct entry regressed
B/op and gave no encode win on the wkt-event shape; three slots match
the codec.lua nested-message pattern and land the speedup.
bench/shapes (wkt-event encode, median across 4 runs):
full: 125895 -> 129000 msgs/s (+2.5%)
runtime: 115231 -> 120000 msgs/s (+4%)
make test 509/509, make jit-trace 23/23, docker conformance binary
1478/0 (unexpected), text suite clean.
text: render captured unknown fields, tolerate SGROUP in skip
Two changes close the proto3 text-format conformance suite:
1. `wire.skip_field` learns SGROUP/EGROUP. Wire 3 recurses through
inner tags until a matching EGROUP, with field_id checked against
the SGROUP's id. Callers (codec.lua, lazy.lua, wkt.lua, generated
full-mode `_pb.lua`) now pass the tag's field_id so groups inside
unknown-field skips don't error.
2. `pb.text.encode` walks the captured `_unknown_fields` buffer when
`opts.print_unknown_fields=true` and emits each entry in
TextFormat numeric-field form:
VARINT -> "<id>: <uint64>"
I64/I32 -> "<id>: 0x<hex>"
LEN -> speculative "<id> { <recurse> }"; rolls back to
byte-string form if the inner bytes don't parse as a
sub-message
SGROUP -> "<id> { <recurse> }" through matching EGROUP
`cmd/conformance/core.lua` threads `req.print_unknown_fields` into
`pb.text.encode` so `_Drop` tests drop unknowns and `_Print` tests
render them.
Conformance: text-format suite goes from 2 ✓ / 6 expected fails to
8 ✓ / 0 expected fails. All eight regression tests in
`conformance_test.lua` (one per upstream test, plus the fixed field-1011
tag bytes that were miscomputed earlier) now assert the target output.
codegen+wire: split length-prefix emission + 2-byte varint fast path
Two complementary encode optimizations: codegen + runtime writers no
longer pay a per-field string concat for length-prefixed fields, and
encode_varint_slow now handles 2/3/4-byte values without dropping into
the uint64 cdata path.
Composite bench/bench.lua, full mode:
1KB encode: 229 K -> 332 K msgs/s (1.45x)
10KB encode: 38 K -> 56.5 K (1.49x)
100KB encode: 4.1 K -> 6.7 K (1.64x)
Runtime mode now matches full mode at 10KB+:
1KB encode: 203 K -> 276 K (1.36x)
10KB encode: 35 K -> 55 K (1.56x)
bench/shapes_bench.lua biggest swings (both modes):
packed-int32x1000 enc: 1.7 K -> 13 K (7.0-7.6x)
packed-int32x100 enc: 21 K -> 113 K (5.3x)
scalar-heavy enc: 582 K -> 1.0 M (1.7x)
Per-helper, bench/wire_bench.lua:
encode_varint(150) [2-byte]: 529 ns -> 68 ns (7.8x)
encode_varint(20K) [3-byte]: 841 ns -> 88 ns (9.6x)
encode_tag(16, LEN) [2-byte]: 531 ns -> 78 ns (6.8x)
encode_string(200B) [2-byte L]: 560 ns -> 109 ns (5.1x)
encode_varint(127) [1-byte]: 29 ns -> 28 ns (no regression)
1. Split length-prefix emission. Length-delimited fields (string/bytes
scalars, nested messages, packed scalars/enums) used to emit
`tag → encode_len(body)` where `encode_len(body)` returns
`encode_varint(#body) .. body`. The string concat allocated a copy
of the body per field. Now the codegen and runtime writers emit
three separate `out` slots — `tag`, `varint(#body)`, `body` — and
let `table.concat(out)` join them in one pass at the end of encode.
Applied to inline.go (full-mode codegen) for: singular and repeated
nested messages, singular and repeated string/bytes scalars, packed
scalars, packed enums. Applied to codec.lua build_writer / build_
repeated_writer for the same set. Maps still go through encode_len
pending a separate pass.
2. encode_varint_slow Lua-number fast paths. The outer encode_varint
stays at the tiny `1-byte check + tail call` shape that LuaJIT
inlines into hot traces. The slow function (not inlined into hot
traces, so its body size is unconstrained) now handles non-negative
Lua numbers up to 2^28 directly via bit.rshift / string.char with no
cdata allocation. Values in [2^28, 2^53) emit one byte and recurse
on the smaller residue. Only cdata inputs, negative Lua numbers
(sign-extended to 10-byte varint), and the rare > 2^53 case still
take the uint64 cdata loop. Net effect: every multi-byte varint
encode that fits in a Lua number — including the length prefix for
any string >= 128 bytes, every tag for field IDs >= 16, and every
negative-zigzag sint — drops from ~500 ns to ~70 ns.
497/497 luatest pass. 23/23 jit-trace gates pass (the two new
multi-byte varint gates added in the bench infra commit confirm
encode_varint_slow JIT-compiles cleanly). Conformance suite (binary
+ text) shows 1478 expected passes, 0 unexpected failures.
wire: decode wins — utf8.len validator + fast paths + FFI cast
Five focused decode optimizations, all in wire.lua, landing a 3.9-6.6x
speedup on bench/bench.lua decode and matching wins on lazy / shapes
benches. Per-helper numbers from bench/wire_bench.lua:
is_valid_utf8(32B ASCII): 545 ns -> 36 ns (15x)
is_valid_utf8(1KB ASCII): 16329 ns -> 539 ns (30x)
decode_string(32B): 140 ns -> 80 ns (1.75x)
decode_double: 299 ns -> 226 ns (1.32x)
decode_fixed64: 196 ns -> 184 ns (1.07x)
1. utf8.len swap. Pure-Lua RFC 3629 validator replaced by
`utf8.len(s) ~= nil` (ICU U8_NEXT-backed at src/lua/utf8.c:165).
Source-verified to reject every proto3 case: stray continuation,
overlong, surrogates, truncated, > U+10FFFF, 5-byte+ sequences.
Conformance suite still at 1478/1478 expected passes.
2. decode_string / decode_bytes 1-byte LEN fast path. Strings <=127
bytes (the RPC common case) skip two function-call layers
(decode_string -> decode_len -> decode_varint).
3. decode_float / decode_double via ffi.cast(uint8_t*, buf) instead
of buf:sub. Eliminates the per-call string-slice allocation.
4. decode_double Inf/NaN check via a uint32[2] union split. Bit ops
on Lua numbers don't allocate; the previous uint64 cdata path
produced 3+ intermediates per call.
5. decode_fixed64 reads via the new pb_u64_u_t union (ffi.copy +
read .u). Replaces UINT64(lo) + bit.lshift(UINT64(hi), 32) which
allocated 2-3 cdata per call.
All FFI cdef/union locals hoisted to the top of the file so the
fixed-width decoders all reference the same scratch buffers.
497/497 luatest pass. 19/19 jit-trace gates pass. Conformance
(binary + text) shows zero unexpected failures.
json: strict proto3 JSON conformance — close all Required failures
The conformance suite's Required JSON failures collapse to zero with
this pass. Local tests grow from 456 to 497 to pin every fix.
Decode side — `runtime/pb/json.lua`:
* Strict scalar validators per type. Numeric strings must match the
JSON number grammar (no leading whitespace, no partial numerics);
out-of-range values, NaN/Inf surrogates from JSON literals, and
type mismatches all become parse_error.
* Quoted exponential ints ("1e5" -> 100000) are accepted via the
JSON-number grammar path, matching Int32FieldQuotedExponentialValue.
* Timestamp parser: strict RFC 3339 (uppercase T/Z, ±HH:MM offset,
≤9 frac digits, range check). Output uses a portable Hinnant-style
epoch_to_ymdhms so year 0001 zero-pads correctly — glibc's POSIX
%Y emits "1" for that year, breaking round-trip.
* Duration parser/formatter: mandatory `s` suffix, ±10000-year range,
sign-matching nanos, 0/3/6/9-digit fractional output.
* Any: WKT-aware nesting under "value"; empty Any -> {}; Empty WKT
inside Any omits "value" (reference parser rejects {"value":{}});
@type URL without `/` rejected; empty @type with sibling fields
rejected.
* Reject NaN/Infinity in google.protobuf.Value.number_value (no JSON
literal for these).
* Duplicate oneof branches rejected; a null oneof branch does NOT
count as set, so a sibling non-null branch is unambiguous.
* Top-level JSON null rejected for messages; preserved for Value.
* Repeated/map values must be JSON array/object (not bare scalar).
Encode side — hand-rolled JSON emitter:
Tarantool's `json.encode` uses a fixed global precision so doubles
like 0.1 don't round-trip and we can't change it per-value without
polluting other users. Replace with a minimal emitter that picks the
shortest-round-tripping precision (15 -> 16 -> 17) per double and
handles NaN/Inf as quoted sentinel strings.
`runtime/pb/wkt.lua`: `timestamp_decode` now keeps invalid Timestamps as
a raw {seconds, nanos} table when `datetime.new` rejects them (negative
nanos, year > 9999, ...) so the JSON encoder can produce
serialize_error rather than the binary decoder raising parse_error.
Required by the Timestamp conformance suite.
`test/conformance_test.lua`: 41 new regression tests grouped under
Fixes 12-19, pinning every code path touched. Strict scalar
validators (one per rejection shape per type), Timestamp/Duration
strict parsing and canonical output, Any WKT/non-WKT/Empty handling,
Value NaN/Inf rejection, ValueAcceptNull round-trip, LuaJIT
NaN-boxing collision cases (0x7FFBCBA987654321, all-ones), shortest
double round-trip, oneof-null semantics.
`test/conformance/known_failures.txt`: refreshed. 15 Recommended-only
failures remain (FieldMask round-trip quirks, duplicate-field-name
detection, unknown-enum-string rejection, null-element-in-list,
NullValue oneof validator).
wire: bypass LuaJIT NaN-boxing collision in decode_float/decode_double
Reading a NaN-payload double back through `tonumber(F.d)` runs into
LuaJIT 2.1's NaN-boxed value representation: certain IEEE NaN bit
patterns collide with internal type tags (nil, function, ...) so
`tonumber` yields a non-number and the field is silently dropped
during encoding.
Inspect the raw bit pattern via the uint32_t/uint64_t aliases of the
float/double unions before reaching for the Lua-level value. When the
exponent is all-ones we resolve Inf/NaN ourselves; only normal values
hit `tonumber`.
Closes the conformance suite's DoubleFieldNormalizeSignalingNan and
FloatFieldNormalizeSignalingNan JsonOutput tests, which used bit
patterns (0x7FFBCBA987654321, 0x7FBFFFFF) that hit the collision.
json: enable conformance JSON output + close 459 tests
Three coupled changes that turn the JSON output path on for the
conformance harness:
1. string_to_int64 accepts cdata: Tarantool's json.decode parses raw
JSON integer literals outside double range as int64_t/uint64_t cdata,
not Lua numbers. The decoder errored "expected JSON string or number
for int64" on any unquoted 64-bit value (Int64FieldMaxValueNotQuoted
et al). cdata is now cast through directly, preserving precision.
2. encode_message marks output as a map: an empty proto3 message
serialized as `[]` because Tarantool's json defaults empty tables to
array shape. jsoncpp's strict comparator threw Json::LogicError and
aborted the whole suite. Setting __serialize='map' on the output
gives `{}` and unblocks all JsonOutput tests.
3. PB_CONFORMANCE_SKIP_JSON gate is opt-in by default. core.lua now
matches exactly "1" (so docker -e VAR= disables it), and the
Dockerfile no longer hard-codes "=1" — JSON output runs end-to-end
for everyone unless they re-enable the gate.
Conformance moves from 930 / 1869 / 11 to 1389 / 1313 / 79
(successes / skipped / expected fails). The 75 new expected fails
are canonical-form edge cases (Duration formatting sign handling,
Timestamp out-of-range rejection, double precision digits, NaN
canonicalization, JSON-input strict rejection) — left for a follow-up.
Drops 7 entries from test/conformance/known_failures.txt that this
change closes; adds 75 newly-visible ones.
json: canonical lowerCamelCase + NullValue WKT descriptor
Two unrelated JSON-decoder gaps captured under the conformance triage:
1. to_camel's gsub pattern '_(%w)' didn't match consecutive underscores
and didn't drop trailing underscores, so proto names like
__field_name13 / field__name4_ / field_name17__ generated JSON keys
that didn't match what protoc produces. The fix strips trailing _+
and collapses '_+%w' to a capitalized letter; a leading underscore
thus capitalizes the next character, matching the spec.
2. pb.wkt didn't export a descriptor for google.protobuf.NullValue, so
any enum field whose type is NullValue (oneof_null_value, or the
implicit one inside Value) crashed decode_enum with "attempt to
index a nil value." Adds a minimal {by_name, by_value} descriptor.
Drops 3 entries from test/conformance/known_failures.txt
(FieldNameInSnakeCase, FieldNameWithDoubleUnderscores,
NullValueInOtherOneofOldFormat). Adds 4 regression tests covering
double underscore, leading underscore, trailing underscore, and the
NullValue enum decode path.