~bigbes/tarantool

tarantool-protobuf

ref: 6e7835a29a3678aa86a49dfff76c55b18d091755 tarantool-protobuf/test/proto2_test.lua -rw-r--r-- 16.5 KiB
2656c977 — Eugene Blikh 3 months ago
bench,codec: pin proto2 paths, kill pairs() on extension hot path

JIT trace gate gains 12 proto2-specific checks (required encode/decode,
groups singular + repeated, extension encode/decode). The extension-encode
check fired NYI on bytecode 72 (ISNEXT) — pairs() over the new
extensions_by_full_name hash is the same trace-abort that gates map
encode. Fix: register_extension now also appends to extensions_list,
an array view. codec.encode_message, text.emit_message and
json.encode_message all switch to ipairs over the list. Hash tables
stay around for O(1) lookups in decode (extensions_by_id) and bracket-
name resolution (extensions_by_full_name).

Inline (full-mode) codegen learns to walk extensions too: before this
commit the inline encoder skipped t._extensions entirely (only the
runtime codec walked it), so a generated _encode silently dropped any
extension data set on the message. Add the array-walk after the field
loop and a decode_extension dispatch in the unknown-tag branch — the
inline path now matches runtime byte-for-byte.

bench/bench.lua parameterizes over a FIXTURES list so the baseline can
cover both hello.Person and a new proto2_basic.BenchPayload fixture
(required + group + extension + repeated). baseline.json restructured
under "schemas": [{schema, results}, …]; compare walks both. Fresh
numbers committed.

740/740 luatest cases pass; JIT gate 37/37; conformance 2806 binary+JSON
and 434 text-format, both 0 failures.
5fa0fd00 — Eugene Blikh 3 months ago
codec: proto2 groups (SGROUP/EGROUP wire format)

Plugin: detect protoreflect.GroupKind, emit kind='group' in the
field descriptor and a SGROUP opening tag in inline codegen. The
group body bypasses the LEN-prefix path entirely — encoder emits
start-tag, body bytes, end-tag in three slots; decoder calls
pb.codec.decode_group(desc, buf, pos, field_id) which reads inner
tags until the matching EGROUP id.

Wire layer already understood SGROUP/EGROUP for skip_field; the new
decode_group reuses the per-tag dispatch from decode_message but
stops on EGROUP instead of end-of-buffer. Repeated groups bracket
each element with its own SGROUP/EGROUP pair.

Runtime parser now desugars `optional|required|repeated group Name
= id { body }` into (a) a nested message named Name and (b) a
synthetic field of kind=group whose lowercased name is `name` — so
source-parsed proto2 schemas behave the same as build-time codegen.

Text format renders groups under the submessage's capitalized name
(`SingleGroup { ... }`) rather than the lowercase field name,
matching mainline protoc's convention.

Adds a vendored, MessageSet-stripped test_messages_proto2.proto in
test/conformance/proto/ (header comment documents the patch).
10 new luatest cases (group_*) cover singular + repeated + text +
descriptor kind, across both codegen modes. 739 tests pass.
798e8db9 — Eugene Blikh 3 months ago
json,text: presence-tracked fields skip proto3 default elision

Both codecs gated default-value elision on `f.optional or f.oneof` —
the proto3 implicit-presence shape. That elides proto2 `required`
fields set to zero and any other presence-tracked descriptor that
doesn't carry the proto3 explicit-optional flag.

Extend the bypass to `f.required` so a Cardinality{r=0} survives
the round-trip through pb.json.encode / pb.text.encode. 4 new
luatest cases pin the rule.
4db0366a — Eugene Blikh 3 months ago
runtime: parser + dynamic accept proto2 sources

Parser now captures `required=true` and `default_value=…` from the
proto2 keywords (instead of dropping `required` silently and ignoring
field options). Dynamic descriptor builder reads `parsed.syntax`,
flips the repeated-scalar packing default for proto2, and calls
`codec.compile_writers/compile_readers` so the per-field
required-writer specialization actually fires — without that the
generic encode_field path silently elides missing required fields.

64-bit-int defaults are coerced into the appropriate cdata type
inside `coerce_default` so the codec's value-comparison rules
match what generated code emits.

Adds 7 dynamic-mode luatest cases including a static-vs-dynamic
byte-parity check. 725 tests pass.
4fbdb65d — Eugene Blikh 3 months ago
codegen: proto2 baseline — required, optional, custom defaults

Lifts the proto3-only syntax gate in the plugin and threads three
new field-descriptor attributes through codegen and the codec:

  * required=true   — fields declared with the proto2 `required` keyword.
                      Inline codegen and the runtime codec both error when
                      a required field is missing on encode (vs the silent
                      elide that proto3 implicit-presence fields get).
  * optional=true   — already wired for proto3 explicit `optional`; in
                      proto2 every singular field carries it via the
                      existing HasOptionalKeyword() check, giving presence
                      semantics without a separate emission path.
  * default_value=… — proto2 [default = X] from the field descriptor,
                      rendered as a Lua literal (cdata for 64-bit ints,
                      symbolic name for enums) so consumers can surface
                      it; the codec itself does not auto-materialize
                      defaults on decode, matching how proto3 absent
                      fields stay nil.

Packed-by-default already flips correctly because we ask
protoreflect's `IsPacked()`, which is syntax-aware.

Adds test/proto/proto2_basic.proto with 33 luatest cases covering
required validation, optional presence, custom defaults, the proto2
unpacked-by-default repeated rule, nested-required messages, and
full-vs-runtime mode parity. `just gen-proto2-tests` regenerates the
fixture into examples/expected/{full,runtime}/.

Out of scope: extensions, extend, group; conformance harness still
skips TestAllTypesProto2.