~bigbes/tarantool

tarantool-protobuf

ref: b3d342d67c5e2fcce101fdafe3aae3c5120f1e73 tarantool-protobuf/test d---------
b3d342d6 — Eugene Blikh 3 months ago
codegen: protoc-gen-tarantool-doc — Markdown reference plugin

Sibling Go plugin under cmd/protoc-gen-tarantool-doc that emits one
Markdown file per input .proto. Sections (omitted when empty):

  - Header (path, package, imports)
  - Messages (per-message description + field table:
    # | Field | Type | Label | Description)
  - Enums (value table)
  - Services (method table with unary/client/server/bidi label)

Field type cells render scalar names, full type names for message/enum
references, and `map<K, V>` for maps. Synthetic map-entry messages are
skipped. Leading comments preserved via SourceCodeInfo (squashed to a
single line inside table cells).

Built via `make build-doc`; sample output committed at
examples/docs/hello.md via `make gen-docs`. Smoke tests in
test/doc_test.lua build the plugin if needed and assert the expected
sections and labels appear. 403/403 luatest green.
78d234c1 — Eugene Blikh 3 months ago
runtime: pb.from_pb — build modules from a binary FileDescriptorSet

Complements pb.parse (which consumes .proto source) by accepting the
output of `protoc --descriptor_set_out=...`. Returns
{files={[name]=module}, order, lookup} where each per-file module has
the same shape as pb.parse() output.

Pipeline: hand-built descriptor.proto descriptors in descriptor_pb.lua
decode the wire bytes via pb.codec; fileset.lua translates each
FileDescriptorProto into the AST shape pb.parser emits;
pb.dynamic.build consumes the AST.

Handles map-entry reconstruction (synthetic entry messages skipped from
nested_messages, key/value lifted to the AST map field), proto3_optional
rehydrated as optional=true rather than a synthetic oneof, oneof
grouping, nested types, WKT references.

Tests parity against the statically-generated hello module — 17 cases
covering scalars/repeated/optional/oneof/maps/self-reference/enums/WKTs
plus a full round-trip. 403/403 luatest green.
0044b163 — Eugene Blikh 3 months ago
codegen: text-format printer (pb.text.encode + <Msg>_text wrappers)

Adds a descriptor-driven text-format encoder mirroring `protoc --decode`
output: one field per line, 2-space indent, octal byte escapes,
`nan`/`inf` floats, `opts.single_line=true` for compact one-liners.
WKT-aware — Timestamp/Duration accept datetime cdata or {seconds,nanos},
wrappers print their unwrapped scalar, Struct/Value/ListValue walk the
tagged-table form, FieldMask flattens to `paths:` lines, Any stays
opaque.

Both codegen modes emit `M.<Type>_text(t, opts)`, surfaced as `pb.text`
on the public table. Encode-only; matching parser deferred.

Tests cover scalars, repeated, maps, oneof, optional presence, all WKT
types, single-line mode, and the generated wrapper across both modes
(74 cases). 403/403 luatest green, 19/19 jit-trace gate green.
77ccfc15 — Eugene Blikh 3 months ago
lazy: zero-copy decode view (decode_lazy) with passthrough re-encode

Adds pb.decode_lazy(desc, bytes) returning a MessageView/ArrayView/MapView
that indexes the wire bytes in a single pass and decodes individual
fields only on :get / :at access. Nested messages return more lazy
sub-views; WKT descriptors (those carrying desc.decode) are
eager-wrapped so the API stays uniform.

Surface (see runtime/pb/lazy.lua):
  - MessageView: :get / :has / :which / :iter / :names / :set / :encode
  - ArrayView:   :len / :at / :iter / :tolist
  - MapView:     :get / :has / :keys / :iter / :totable

:encode is three-modes: WKT delegates to desc.encode on the materialized
table; untouched views return their original bytes verbatim
(passthrough); mixed views walk fields in id order, splicing clean
segments and re-emitting dirty ones. Sub-MessageView mutations
propagate to parent encode via a flat _sub_msg_views array (walked with
ipairs, so :is_dirty stays on a single JIT trace — pairs over a hash
is NYI in LuaJIT 2.1).

Codegen emits M.<Type>_decode_lazy in both modes as a one-line
delegation to pb.decode_lazy(<desc>, b); no inline expansion.
codec.encode_field is exposed so the lazy passthrough emitter can
splice fresh bytes for a single dirty field without rebuilding the
whole message.

Tests: 40 new lazy_test.lua cases parameterized over both codegen
modes; all 11 interop fixtures round-trip byte-equal through
decode_lazy(b):encode() in both modes. Total: 300/300 luatest, up
from 226.
784dea4d — Eugene Blikh 3 months ago
Initial commit: protoc-gen-tarantool plugin + pb runtime

A protoc plugin (Go) and a pure-Lua + LuaJIT-FFI runtime that give
Tarantool a complete proto3 + gRPC stack. Two codegen modes (full
inline / runtime descriptor), 226-test luatest suite, 18-fixture
mainline-protoc interop corpus, JSON codec, well-known types,
gRPC client/server factories, runtime .proto parser, microbench
harness with allocation regression gate.

Covers PLAN.md M1-M5. Module is `pb` (not `protobuf`) to avoid
colliding with Tarantool's built-in encode-only `protobuf` module.