~bigbes/tarantool

tarantool-protobuf

ref: b840992a2366a08451b6f45b3d18b19643e4af5e tarantool-protobuf/examples/expected/full/quickstart d---------
a6eebdb3 — Eugene Blikh 3 months ago
codegen: local counter per repeated field on decode

Replaces `list[#list + 1] = val` with `_n_<fname> = _n_<fname> + 1;
list[_n_<fname>] = val` in every generated M.X_decode repeated-field
append site. One counter local per repeated non-map field, declared at
function entry. Counters survive across loop iterations so out-of-order
wire entries for the same field continue past the existing length
without re-scanning.

Profile attributed 6.2% of hello.Person 1KB decode to the `#list + 1`
re-traversal (26-email Person paid 26 list scans per decode).

Bench (Person full decode, msgs/s, median-of-3 vs post-4kj):
1KB +5.0%, 10KB +5.0%, 100KB +7.1%. Encode flat. Tests 745/745.
JIT 37/37, 0 bridges.

beads-tarantool-protobuf-cch
feecf8e0 — Eugene Blikh 3 months ago
codegen: inline 1-byte tag fast path at decode call sites

Hoists wire.decode_tag's 1-byte fast path into every generated
M.X_decode while-loop, falling back to the helper for multi-byte
tags (field ids > 15). The 1-byte case covers every protobuf field
with id 1..15 and is the dominant decode dispatch in real payloads.

Header now localizes string.byte, bit.band, and bit.rshift so the
inlined ops compile to straight local calls.

Bench (Person full decode, msgs/s, median-of-3 vs proper post-h8v
3-run baseline): 10B +14.7%, 100B +8.9%, 1KB +7.5%, 10KB +7.0%,
100KB +8.5%. Full encode is flat to small (-0.1% to -2.6%) at large
sizes, plausibly from header-upvalue layout. JIT trace gate: 37/37,
all bridges still 0. Tests: 745/745.

Also documented in bench/PERF_LOG.md, including the methodology
note that h8v's earlier numbers used single-run baselines and are
therefore ~3-5% optimistic; medians-of-3 are the standard now.

beads-tarantool-protobuf-4kj
9ee21c09 — Eugene Blikh 3 months ago
codegen: inline 1-byte varint length prefix at every LEN emit site

Eliminates the wire.encode_varint(#body) call + dispatch for every
length-delimited field in mode=full codegen. Profile flagged the
out[n] = wire.encode_varint(#_b) line as ~33% of hello.Person 1KB
encode time, with another ~17% in encode_varint dispatch — together
~50% of encode time. Lifting the 1-byte fast path (the dominant case
for proto strings and small message bodies) to the call site removes
the function frame entirely for lengths < 128.

Applied at every LEN emit site: singular/repeated message body,
singular/repeated string|bytes, packed scalar bundle, packed enum
bundle, map entry. Map value pieces (emitMapPiece message branch)
left as-is — they sit inside a single slot assignment that would
require a deeper restructure, and maps are not on the current hot
benchmark.

Results (hello.Person full encode, msgs/s): 10B +6.9%, 100B +7.9%,
1KB +25.7%, 10KB +48.1%, 100KB +31.9%. Decode flat (unchanged path).
Runtime mode flat (descriptor dispatch still calls encode_varint).
JIT trace gate: 37/37. Test suite: 745/745.

Bench history saved to bench/PERF_LOG.md with full numbers and the
workflow this iteration follows.

beads-tarantool-protobuf-h8v
2656c977 — Eugene Blikh 3 months ago
bench,codec: pin proto2 paths, kill pairs() on extension hot path

JIT trace gate gains 12 proto2-specific checks (required encode/decode,
groups singular + repeated, extension encode/decode). The extension-encode
check fired NYI on bytecode 72 (ISNEXT) — pairs() over the new
extensions_by_full_name hash is the same trace-abort that gates map
encode. Fix: register_extension now also appends to extensions_list,
an array view. codec.encode_message, text.emit_message and
json.encode_message all switch to ipairs over the list. Hash tables
stay around for O(1) lookups in decode (extensions_by_id) and bracket-
name resolution (extensions_by_full_name).

Inline (full-mode) codegen learns to walk extensions too: before this
commit the inline encoder skipped t._extensions entirely (only the
runtime codec walked it), so a generated _encode silently dropped any
extension data set on the message. Add the array-walk after the field
loop and a decode_extension dispatch in the unknown-tag branch — the
inline path now matches runtime byte-for-byte.

bench/bench.lua parameterizes over a FIXTURES list so the baseline can
cover both hello.Person and a new proto2_basic.BenchPayload fixture
(required + group + extension + repeated). baseline.json restructured
under "schemas": [{schema, results}, …]; compare walks both. Fresh
numbers committed.

740/740 luatest cases pass; JIT gate 37/37; conformance 2806 binary+JSON
and 434 text-format, both 0 failures.
7676fdc2 — Eugene Blikh 3 months ago
codegen: preserve descriptor options as `options = { ... }`

Every populated *Options message — FileOptions, MessageOptions,
FieldOptions, OneofOptions, EnumOptions, EnumValueOptions,
ServiceOptions, MethodOptions — surfaces on the generated descriptor as
a plain Lua sub-table named `options` (or `oneof_options` /
`value_options` for the per-member shapes). Standard fields use their
proto name as a bare Lua key; extensions use their fully-qualified
extension name as a bracket-quoted string key.

The walker is generic — no extension-specific code paths. Consumers
pull whatever they care about: `(google.api.http)` for REST routing,
`(versionpb.etcd_version_*)` for compatibility gates, `[deprecated =
true]` for migration tooling, and any in-house extension without pb
knowing about them. Standard fields sort alphabetically before
extensions (also alphabetical by full name) so codegen output stays
byte-identical across runs.

The `options` key is only emitted when at least one field is populated,
so option-free protos produce zero-diff output to before. Resolver
re-links *Options* messages so in-file extensions surface via the
protoreflect walker — protogen builds f.Desc before in-file extensions
are registered, and only f.Proto gets the post-pass fix-up, so we
rebuild the resolver manually.

UninterpretedOption is treated as a codegen-time error: a populated
entry means protoc couldn't resolve the extension, and emitting
opaque parser state would hide the problem.
a0005a05 — Eugene Blikh 3 months ago
docs: full reference + how-to set, migrate Makefile to Justfile

Documentation overhaul that adds the missing user-facing surface:
four reference pages (runtime-api, generated-api, cli, grpc-contract),
twelve how-tos walking from first-message through custom transports,
a troubleshooting page, and a docs/index map. Every how-to references
a runnable artifact under examples/, all of them verified end-to-end.

Build system migration: the Makefile is gone; the Justfile is now
the canonical entry point and absorbs every target. examples/Justfile
ships one recipe per runnable example, forwarded via top-level
'just examples <name>'. The 'examples are part of the documented
surface' convention is pinned in CLAUDE.md, alongside a dedicated
section on updating the conformance harness (PROTOBUF_TAG bumps,
libjsoncpp path drift, new test-category wiring).

Stale-number sweep across README/PLAN/CLAUDE: fixture count 18→10,
test count 613/130→639, wire.lua LOC dropped, M7 marked done.
Descriptor-shape block deduplicated against codegen.md as the
canonical source. gRPC transports spec status reframed from
'draft / decision deferred' to 'shipped contract; external
transports deferred'.

.gitignore picks up *.snap / *.xlog / *.vylog / *.run / *.pid /
512.lock so example state can't leak into the working tree.