~bigbes/tarantool

tarantool-protobuf

ref: d4dbd2f36353f3a03fd68fcfab563b81c6520a6b tarantool-protobuf/docs/reference/cli.md -rw-r--r-- 6.1 KiB
d4dbd2f3 — Eugene Blikh bench: C-acceleration spike — measure four Lua↔C boundaries 3 months ago

#CLI reference

How to drive protoc-gen-tarantool and the sibling protoc-gen-tarantool-doc from protoc.

#Building the plugins

just build           # builds ./protoc-gen-tarantool
just build-doc       # builds ./protoc-gen-tarantool-doc

Or directly:

go build -o protoc-gen-tarantool     ./cmd/protoc-gen-tarantool
go build -o protoc-gen-tarantool-doc ./cmd/protoc-gen-tarantool-doc

Place both binaries on PATH (or pass --plugin= to protoc) and they become available as the tarantool and tarantool-doc outputs.

#protoc-gen-tarantool — the Lua codegen

#Invocation

protoc \
    -I. \
    -Ioptions \
    --tarantool_out=<output_dir> \
    --tarantool_opt=<key>=<value>,<key>=<value> \
    path/to/file.proto

output_dir is the root the plugin writes under. Per-file output path follows the Lua require path (see prefix and (tarantool.lua_package) below).

-Ioptions is needed when any input proto uses the custom (tarantool.lua_package) option — it imports tarantool/tarantool.proto from options/tarantool/.

#--tarantool_opt

Comma-separated key=value pairs:

Option Values Default Meaning
mode full / runtime full full inlines _encode / _decode bodies; runtime emits one-line delegations to pb.encode / pb.decode. See api-modes.md.
prefix any Lua require path empty Prepended to every generated module's require path and on-disk subpath.

#prefix

prefix=foo.bar rewrites every generated module name from <orig_path> to foo.bar.<orig_path>. Affects both the require path and the on-disk location.

Without prefix With prefix=apps.myapp
hello/hello_pb.lua apps/myapp/hello/hello_pb.lua
require('hello.hello_pb') require('apps.myapp.hello.hello_pb')

The Justfile uses this to emit full/ and runtime/ copies side by side for the parametrized test suite:

protoc --tarantool_opt=mode=full,prefix=full         ...   # examples/expected/full/...
protoc --tarantool_opt=mode=runtime,prefix=runtime   ...   # examples/expected/runtime/...

#(tarantool.lua_package) — per-file override

Defined in options/tarantool/tarantool.proto. Overrides the require path for a single .proto:

syntax = "proto3";
package my.app;

import "tarantool/tarantool.proto";
option (tarantool.lua_package) = "myapp.proto.foo";

message Foo { ... }
Without lua_package With lua_package
my/app/foo_pb.lua myapp/proto/foo_pb.lua
require('my.app.foo_pb') require('myapp.proto.foo_pb')

prefix= and (tarantool.lua_package) compose: prefix is prepended to the final per-file path regardless of which scheme produced it.

#Path-resolution rules

For each input .proto, the plugin picks a Lua require path in this order:

  1. option (tarantool.lua_package) = "pkg"; — wins outright. The plugin writes pkg as a /-joined path with _pb.lua appended (or just pkg if pkg already ends in _pb).
  2. package declaration + filenamepackage my.app; and a file foo.proto produce my/app/foo_pb.lua, require('my.app.foo_pb').
  3. No package — the filename alone: foo.protofoo_pb.lua, require('foo_pb').

WKT proto files (google/protobuf/*.proto) are not generated as Lua modules. References to WKT types route to pb.wkt.<Name>_descriptor at codegen time; the runtime side ships them in runtime/pb/wkt.lua.

#What gets emitted

See generated-api.md for the per-message / per-enum / per-service surface. Both modes always emit:

  • M.<Msg>_descriptor, M.<Msg>_fields, M.<Msg>_oneofs (when applicable)
  • M.<Msg>_new, M.<Msg>_encode, M.<Msg>_decode
  • M.<Msg>_decode_lazy, M.<Msg>_text (mode-independent wrappers)
  • M.<Msg>_has_<field> / M.<Msg>_clear_<field> (proto3 explicit-optional)
  • M.<Enum>_descriptor, M.<Enum> (alias for by_name)
  • M.<Service>_service, M.<Service>_client, M.<Service>_server
  • EmmyLua / lua-language-server annotations
  • Reserved-name table (reserved_names) for the text-format parser

#Stdin/stdout protocol

The plugin reads a CodeGeneratorRequest from stdin and writes a CodeGeneratorResponse to stdout, per protoc's standard plugin interface. It advertises FEATURE_PROTO3_OPTIONAL so protoc surfaces explicit-optional fields; without this flag, protoc omits them.

Proto2 input is rejected at the top of GenerateFile. See codegen.md → proto2 deferral.

#protoc-gen-tarantool-doc — the Markdown reference generator

Same input as the Lua codegen; emits one Markdown file per input .proto summarizing messages, enums, services.

protoc \
    -I. \
    -Ioptions \
    --tarantool-doc_out=<output_dir> \
    path/to/file.proto

Example: examples/docs/hello.md is produced by just gen-docs from examples/proto/hello.proto.

No options today. The output template is minimal:

  • Per message: a table of fields (number, name, type, label, description from the leading comment).
  • Per enum: a table of values.
  • Per service: a table of methods with streaming kind.

Lua call signatures (Foo_encode, Foo_decode_lazy, etc.) are not yet in the doc template — they're documented in generated-api.md for now.

#Driving from make

The Justfile is the canonical entry point for development:

just build         # build the codegen plugin
just build-doc     # build the doc plugin
just gen           # build + regen examples/expected/{full,runtime}/...
just gen-docs      # build-doc + regen examples/docs/hello.md
just test          # run the luatest suite (parametrized over both modes)
just goldens       # regenerate test/interop/fixtures/*.bin via protoc --encode
just bench         # alloc + throughput per op
just jit-trace     # assert hot paths stay on the JIT trace

For end-user projects not using this repo's Makefile, see the planned howto 12: Build integration (protoc, Justfile, buf, CMake recipes).