codegen: inline proto2 extension writers/readers (qwt) + table.new(N,0) for packed lists (2sn)
qwt closes the proto2_basic.BenchPayload `min` bench's worst data point:
encode 1.89M → 3.30M msgs/s (+75%), decode 1.02M → 1.26M msgs/s (+23%).
Mechanism: collectExtsByExtendee groups all `extend Foo { ... }`
declarations by extendee FullName across input files; emitInlineEncode
emits one dedicated writer per extension instead of dispatching through
pb.codec.encode_field, and emitInlineDecode adds an elseif arm per
extension id straight into result._extensions[full_name]. Dynamic
extensions_list walk preserved for forward compat, gated on
#_elist > N so it pays one int compare when no runtime extension was
registered.
2sn pre-sizes packed-scalar repeated lists via table.new(N, 0) where
the count is recoverable from the LEN payload — exact (lim >> 2 or
lim >> 3) for fixed-width, upper bound (lim) for varint-packed. The
allocation is deferred into the `if wt == 2 then` branch so the
per-element fallback and non-packable types keep the bare-`{}`
alloc. Avoids u39's regression mode because the call cost is paid
once per repeated-field-first-occurrence, not per message decode.
Bench summary for hello.Person full mode: encode +3-5% across sizes,
decode +1-3% across sizes. Tests: 752/752 pure-Lua, 1043/1043 with
PB_ENABLE_C=1, JIT 37/37. PERF_LOG entry covers the rationale and
caveats.
codegen: inline 1-byte LEN fast path for string/bytes decode
Singular and repeated string/bytes fields in generated full-mode _decode
now read the length byte and dispatch in-line instead of calling
wire.decode_string / wire.decode_bytes. For length < 128 (the common
short-string RPC case), the path stays inside the parent JIT trace:
no child-trace stitch, no per-element function frame, utf8_len lookup
hoisted to a generated-file upvalue.
The original a6n design — one ffi.cast(U8CP, buf) at the top of each
_decode — was abandoned: the cdata wrapper is 24 bytes per call and
LuaJIT can't sink the allocation because the pointer local lives
across wire.decode_* call frames. Net regression at small sizes
(+11-16%) overwhelmed the single-byte-read savings.
Bench (hello.Person full-mode decode):
size ns/op before ns/op after delta
10B 419 390 -6.9%
100B 515 478 -7.2%
1KB 9004 8042 -10.7%
10KB 64644 56745 -12.2%
100KB 625638 537750 -14.0%
Zero allocation impact across all sizes. Tests: 1043 pass, 37 JIT
trace gates pass. (a6n)
codegen: inline 1-byte tag fast path at decode call sites
Hoists wire.decode_tag's 1-byte fast path into every generated
M.X_decode while-loop, falling back to the helper for multi-byte
tags (field ids > 15). The 1-byte case covers every protobuf field
with id 1..15 and is the dominant decode dispatch in real payloads.
Header now localizes string.byte, bit.band, and bit.rshift so the
inlined ops compile to straight local calls.
Bench (Person full decode, msgs/s, median-of-3 vs proper post-h8v
3-run baseline): 10B +14.7%, 100B +8.9%, 1KB +7.5%, 10KB +7.0%,
100KB +8.5%. Full encode is flat to small (-0.1% to -2.6%) at large
sizes, plausibly from header-upvalue layout. JIT trace gate: 37/37,
all bridges still 0. Tests: 745/745.
Also documented in bench/PERF_LOG.md, including the methodology
note that h8v's earlier numbers used single-run baselines and are
therefore ~3-5% optimistic; medians-of-3 are the standard now.
beads-tarantool-protobuf-4kj
codec,text,json: proto2 extensions end-to-end
Plugin iterates file.Extensions and per-message Extensions, emitting
\`pb.register_extension(extendee_desc, {…})\` calls. Each extension
becomes a field descriptor attached to the extendee under two
indices: extensions_by_id (wire-decode lookup) and
extensions_by_full_name (encode + textual emission ordering).
Codec encode_message walks data._extensions after the regular field
loop and calls encode_field through the extension's descriptor.
decode_message routes an unknown tag through decode_extension when
extensions_by_id matches; the helper mirrors the in-line dispatch
on field kind (scalar/enum/message/group, singular/repeated) and
stores into result._extensions[full_name].
Text decoder recognizes bracket syntax \`[pkg.ext_name]\` outside an
Any-target and looks the name up in extensions_by_full_name. Names
that don't resolve (e.g. \`[pkg.GroupField]\` for the CapitalCase
group type instead of the lowercase field name) raise a parse error
per spec. Text encoder emits each set extension as
\`[full.name]: value\` or \`[full.name] { … }\`.
JSON encoder emits set extensions under the bracketed full-name key
(\`"[pkg.ext_name]": value\`). JSON decoder recognizes the same
shape and stores into _extensions; unknown bracket-names stay
silently dropped to match the spec's tolerant behavior for unknown
JSON keys.
Extensions on google.protobuf.* descriptors (file/message/field
option extends) are skipped — those are meta-only and our WKT
module doesn't surface their descriptors.
Conformance baseline (strict --enforce_recommended, protobuf v34.1):
Binary + JSON suite: 2806 ✓ / 0 failures (was 1493 ✓ / 1313 skipped)
Text-format suite: 434 ✓ / 0 failures (was 416 ✓ / 18 skipped)
100% pass.
text,codec: proto2 group text syntax + closed-enum semantics
Text decoder now resolves a group field reference by the group's
capitalized submessage name (e.g. \`Data\`, \`MultiWordGroupField\`)
and by the ASCII-lowercase fold (\`data\`, \`multiwordgroupfield\`),
in addition to the existing lowercase field-name lookup. The \`:\`
between field and \`{\` is optional for groups (matches the
message/map rule).
Proto2 enums are closed: parse_enum_value now rejects an integer
literal that doesn't map to any declared value. The plugin emits
\`closed = true\` on every proto2 enum descriptor (driven by
protoreflect's IsClosed()); proto3 enums stay open to preserve
forward-compatibility on the wire.
Conformance text-format suite: 16 unexpected failures → 3, all in
the remaining extension-bracketed-group cases.
codec: proto2 groups (SGROUP/EGROUP wire format)
Plugin: detect protoreflect.GroupKind, emit kind='group' in the
field descriptor and a SGROUP opening tag in inline codegen. The
group body bypasses the LEN-prefix path entirely — encoder emits
start-tag, body bytes, end-tag in three slots; decoder calls
pb.codec.decode_group(desc, buf, pos, field_id) which reads inner
tags until the matching EGROUP id.
Wire layer already understood SGROUP/EGROUP for skip_field; the new
decode_group reuses the per-tag dispatch from decode_message but
stops on EGROUP instead of end-of-buffer. Repeated groups bracket
each element with its own SGROUP/EGROUP pair.
Runtime parser now desugars `optional|required|repeated group Name
= id { body }` into (a) a nested message named Name and (b) a
synthetic field of kind=group whose lowercased name is `name` — so
source-parsed proto2 schemas behave the same as build-time codegen.
Text format renders groups under the submessage's capitalized name
(`SingleGroup { ... }`) rather than the lowercase field name,
matching mainline protoc's convention.
Adds a vendored, MessageSet-stripped test_messages_proto2.proto in
test/conformance/proto/ (header comment documents the patch).
10 new luatest cases (group_*) cover singular + repeated + text +
descriptor kind, across both codegen modes. 739 tests pass.