From a5ba47c793d7f1177e823122c06ce8c290fa4603 Mon Sep 17 00:00:00 2001 From: Eugene Blikh Date: Mon, 25 May 2026 07:27:41 +0300 Subject: [PATCH] docs: document int64_as_number flag (5y9) Adds end-to-end coverage of the int64_as_number plugin option that landed in 8b5bfc0: - docs/reference/cli.md: row in the --tarantool_opt table + a dedicated subsection with the workload tradeoff table, the runtime-side wire helpers it exposes (wire.decode__n), and the PB_ENABLE_C=1 no-op note. - docs/codegen.md: same row in its own CLI-options table + a shorter subsection (the deep version lives in reference/cli.md). - README.md: status row for the feature; paragraph at the end of the "Migrating from a Lua proto library that auto-down-casts int64" section pitching the flag for callers who want number-on-fits semantics with cdata fallback, plus a pointer to the cli.md table. No code change. --- README.md | 21 ++++++++++++++++++ docs/codegen.md | 51 +++++++++++++++++++++++++++++++++++++++++++ docs/reference/cli.md | 48 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 120 insertions(+) diff --git a/README.md b/README.md index 5b8073a1850d24cfc045f4b3f83ff9c3dca0f6e4..5a15a10414c8abc6ee9d6c997cdb1d2e21f13e09 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,7 @@ copy of `test_messages_proto2.proto`. | Cross-file imports | ✅ | | Enums (open semantics) | ✅ | | 64-bit integers as LuaJIT cdata | ✅ | +| Opt-in 64-bit-as-Lua-number decode (`int64_as_number=true`) | ✅ | | Two codegen modes (full + runtime) | ✅ | | Zero-copy lazy decode views | ✅ | | `map` (scalar/message values) | ✅ | @@ -278,6 +279,26 @@ boundary is *Lua primitives that expect a number*. Past 2^53 (≈ 9e15), `tonumber()` silently truncates; if your IDs can be that large, keep them as cdata or stringify with `tostring(id):gsub('U?LL$', '')`. +If your schema's 64-bit values are dominated by the small-and-positive +case (counter-derived IDs, small sequence numbers, byte counts under +16 KB, integer enums, second-resolution timestamps in a restricted +range), the codegen can decode those as Lua numbers up front: + +```bash +protoc --tarantool_opt=mode=full,int64_as_number=true ... +``` + +Decoded fields come back as Lua numbers when the value fits +`[-2^53, 2^53]` and cdata otherwise — same precision guarantee, none +of the `tonumber()` calls. The type at decode time becomes value- +dependent (arithmetic works transparently across both, but +`type(v) == 'cdata'` no longer holds for small values). Workload- +specific tradeoff: cdata-allocation saving on the 1-byte-varint fast +path measured at ~17 % on a 5-field 64-bit schema, but multi-byte +varints regress ~10 % because both paths still allocate the +intermediate cdata before the size check. Full table and the per- +size measurements live in [docs/reference/cli.md](docs/reference/cli.md#int64_as_number). + **Three API modes live side-by-side.** Same descriptor, three call shapes — the inline (full) generated API is the default, the descriptor-driven runtime API is for dynamic schemas, and the lazy API is a zero-copy view for sparse diff --git a/docs/codegen.md b/docs/codegen.md index b2483b28eda65da30cae9052a5e106ff9e692d21..a3e65277e44ee10682609389ff76a84c1440a885 100644 --- a/docs/codegen.md +++ b/docs/codegen.md @@ -41,6 +41,7 @@ Passed via `protoc --tarantool_opt==,...`: |---|---|---| | `mode` | `full` (default) or `runtime` | Inline `_encode`/`_decode` vs descriptor-delegating wrappers. See [api-modes.md](api-modes.md). | | `prefix` | any Lua-require path | Prepended to every generated module's require path **and** its on-disk subpath. The Justfile uses this to generate both modes side-by-side into `examples/expected/{full,runtime}/`. | +| `int64_as_number` | `true` or `false` (default) | `mode=full` only. Opt-in: decode `int64`/`uint64`/`sint64`/`fixed64`/`sfixed64` fields as Lua numbers when the value fits `[-2^53, 2^53]`, cdata otherwise. Workload-dependent tradeoff — see [int64\_as\_number](#int64_as_number) below. | The `(tarantool.lua_package)` file option (defined in `options/tarantool/tarantool.proto`) overrides the per-file Lua module @@ -55,6 +56,56 @@ Without that option the path mirrors `package` (e.g. `package my.app; foo.proto` → `my/app/foo_pb.lua`, `require('my.app.foo_pb')`). +### int64_as_number + +By default every 64-bit scalar field decodes to LuaJIT `int64_t` / +`uint64_t` cdata — the same convention as Tarantool's `msgpackffi`, +`net.box`, `box.tuple`, and the built-in `protobuf` module. That keeps +precision past 2^53 and matches what application code already does +with 64-bit IDs. + +`int64_as_number=true` swaps the decoder for one that returns a Lua +number when the decoded value fits `[-2^53, 2^53]` (inclusive — both +endpoints are powers of two and exact as doubles) and cdata otherwise. +The return type becomes value-dependent: arithmetic (`+`/`-`/`*`/`==`) +works transparently across the two, but `type(v)` is no longer stable, +so call sites that branch on `type(v) == 'cdata'` need updating. + +``` +protoc --plugin=./protoc-gen-tarantool \ + --tarantool_out=out \ + --tarantool_opt=mode=full,int64_as_number=true \ + myschema.proto +``` + +When to use it. Workload-specific tradeoff measured on a 5-field 64-bit +schema (`c_int64.Wide`, full mode, no PB_ENABLE_C): + +| Per-field magnitude | Default (cdata) | `int64_as_number` | Delta | +|---|---|---|---| +| 1-byte varints (<128) | 2050 ns/op | 1700 ns/op | **−17%** | +| 3-byte varints | 3220 ns/op | 3600 ns/op | +11% | +| past 2^53 | 7575 ns/op | 7750 ns/op | +2% (noise) | + +Enable when 64-bit fields typically carry small values that hit the +1-byte varint path — request IDs from a counter, small sequence +numbers, byte counts under 16 KB, integer enums, second-resolution +timestamps in restricted ranges. The win comes from skipping the +cdata allocation entirely on the fast path. For workloads dominated +by full-range int64 / fixed64 (epoch-nanosecond timestamps, large +hash IDs, file-size byte counts), leave it off — the comparison and +`tonumber()` call add net overhead on top of the cdata that's allocated +anyway. + +Restrictions: + +- `mode=full` only. `mode=runtime` errors at plugin startup; supporting + it would require a descriptor flag wired through `pb.codec`. +- Under `PB_ENABLE_C=1` the flag is a no-op: the C runtime makes its + own number-vs-cdata decision via `luaL_pushint64` (Tarantool's small- + fits-in-double convention) before the codegen-emitted decoder runs. + See [c-accel.md](c-accel.md). + ## What gets emitted per .proto ```lua diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 02bc043a96c794d846a4a29adeb1e1a497a2146e..5993d4ae6272667d5998042f650e810cd565ad0c 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -49,6 +49,7 @@ Comma-separated `key=value` pairs: |---|---|---|---| | `mode` | `full` / `runtime` | `full` | `full` inlines `_encode` / `_decode` bodies; `runtime` emits one-line delegations to `pb.encode` / `pb.decode`. See [api-modes.md](../api-modes.md). | | `prefix` | any Lua require path | empty | Prepended to every generated module's require path and on-disk subpath. | +| `int64_as_number` | `true` / `false` | `false` | `mode=full` only. Decode 64-bit scalar fields as Lua numbers when the value fits `[-2^53, 2^53]`, cdata otherwise. Workload-dependent — see [`int64_as_number`](#int64_as_number) below. | ### `prefix` @@ -69,6 +70,53 @@ protoc --tarantool_opt=mode=full,prefix=full ... # examples/expected/f protoc --tarantool_opt=mode=runtime,prefix=runtime ... # examples/expected/runtime/... ``` +### `int64_as_number` + +Decode `int64` / `uint64` / `sint64` / `fixed64` / `sfixed64` fields as +a Lua `number` when the decoded value fits `[-2^53, 2^53]` (inclusive +on both endpoints — both are powers of two and exact as doubles), as +LuaJIT `int64_t` / `uint64_t` cdata otherwise. `mode=full` only; +`mode=runtime` errors at plugin startup. Default off. + +```bash +protoc --tarantool_out=out \ + --tarantool_opt=mode=full,int64_as_number=true \ + schema.proto +``` + +The return type becomes value-dependent under this flag. Arithmetic +(`+`/`-`/`*`/`/`/`==`) works transparently across number and cdata, +but call sites that branch on `type(v) == 'cdata'` or rely on +cdata-only operators need updating. + +Workload-dependent tradeoff measured on a 5-field 64-bit schema +(`c_int64.Wide`, `mode=full`, no `PB_ENABLE_C`): + +| Field-value magnitude | Default (always cdata) | `int64_as_number` | Delta | +|---|---|---|---| +| 1-byte varints (`< 128`) | 2050 ns/op | 1700 ns/op | **−17 %** | +| 3-byte varints | 3220 ns/op | 3600 ns/op | +11 % | +| past 2^53 | 7575 ns/op | 7750 ns/op | +2 % (noise) | + +Enable when the schema's 64-bit fields typically carry small values +(counter-derived IDs, small sequence numbers, byte counts < 16 KB, +integer enums, second-resolution timestamps in restricted ranges). +The win is from skipping the cdata header allocation entirely on the +fast path. For workloads dominated by full-range int64 / fixed64 +(epoch-nanosecond timestamps, large hash IDs, file-size byte counts), +leave it off — both decoders allocate a cdata for the intermediate +varint, and the comparison + `tonumber()` adds net overhead. + +Under `PB_ENABLE_C=1` the flag is a no-op: the C runtime makes its +own number-vs-cdata decision via `luaL_pushint64` (Tarantool's small- +fits-in-double convention) before the codegen-emitted decoder runs. +See [c-accel.md](../c-accel.md). + +The `runtime/` shipped Lua module exposes the matching wire helpers +(`wire.decode_int64_n`, `decode_uint64_n`, `decode_sint64_n`, +`decode_fixed64_n`, `decode_sfixed64_n`) for hand-rolled callers that +want the same semantics without going through codegen. + ### `(tarantool.lua_package)` — per-file override Defined in `options/tarantool/tarantool.proto`. Overrides the require