#Spec: C-acceleration compatibility contract

Status: contract locked, implementation pending. This spec formalizes the compatibility boundaries that the C-acceleration work (docs/c-accel.md) must preserve. It is the reference that bd-mq7 (plan compiler), bd-ra6 (generic C runtime), bd-c0i (deferred codegen-C), and bd-43t (parity gate) consume to decide what is — and isn't — allowed to change.

Closes bd-47e.

#Principle

The C path is an opt-in performance enhancement, not a new mode. Every Lua-visible behavior must be byte-equal between PB_ENABLE_C unset and PB_ENABLE_C=1. A user who never sets the variable should never observe any difference — including memory layout of returned values, error messages, error types, iteration order of returned tables, or the structure of generated _pb.lua modules.

If a C-path optimization conflicts with this principle, the C path loses. There are no exceptions and no per-feature carve-outs.

#Activation

Exactly one switch: the environment variable PB_ENABLE_C.

• Read once at module load (runtime/pb/init.lua), via os.getenv('PB_ENABLE_C'). Value '1' enables; anything else (including unset, '0', 'true', '') leaves the C path off.
• No Lua-side toggle (pb.use_c_runtime = true is not part of the surface). No per-call override. No per-message override. The knob is out-of-band by design — keeping it out of Lua makes accidental activation impossible and prevents the C path from becoming a feature with its own API surface.
• When PB_ENABLE_C=1 but require('pb.c_runtime') fails (no .so built, ABI mismatch, etc.), runtime/pb/init.lua falls back to the pure-Lua path silently. No warning, no error. A failed pcall is indistinguishable from PB_ENABLE_C unset for every downstream observer. Rationale: a noisy fallback turns a missing build artifact into a runtime error for users who didn't ask for C anyway (rockspec installed without compiler available).

#Public Lua surface — preserved

The following modules and their public functions must keep their exact signatures, argument types, return types, and error behavior in both paths:

#runtime/pb/init.lua (the pb module)

Generated modules (mode=full and mode=runtime):

#What the C plan may not change

• The on-disk format of examples/expected/{full,runtime}/*_pb.lua. Files in version control must stay byte-equal across this work. C-plan compilation happens at module load, in memory, from descriptor tables — no codegen-time change.
• The structure of desc tables documented in docs/codegen.md. Plan compilation reads desc; it does not mutate it except for attaching the opaque c_plan userdata.
• Generated module return value (return M). No new exports.

#64-bit integers

int64, uint64, sint64, fixed64, sfixed64 are LuaJIT int64_t / uint64_t cdata in both paths.

• Encode accepts cdata, Lua number (within ±2^53), or string of digits (existing behavior in pb.codec). The C path must accept all three. Implementation hook: bd-3l.
• Decode always returns cdata. The pure-Lua decoder already returns cdata; the C decoder must use luaT_pushint64 / luaT_pushuint64 (Tarantool's module.h extensions) so the value lands on the Lua stack as cdata, not as a Lua number.
• tostring of a returned 64-bit value must produce the same string in both paths. (Lua's default cdata __tostring already guarantees this.)

This rule is load-bearing for downstream users. msgpackffi, net.box, box.tuple, IProto all consume cdata. Narrowing to double silently corrupts IDs and timestamp nanos past 2^53. The pure-Lua path holds this line today, and the C path must hold it tomorrow.

#WKT shapes

google.protobuf.* types keep their hand-rolled shapes (runtime/pb/wkt.lua):

• Timestamp / Duration accept and return a datetime cdata, with {seconds=, nanos=} table accepted on encode for ergonomics.
• Empty is box.NULL.
• Wrappers (Int32Value, StringValue, …) accept and return the wrapped scalar directly.
• Any, Struct, Value, ListValue keep their existing representations.

Mechanism: WKT descriptors carry desc.encode / desc.decode function fields. The C plan, when finalizing a descriptor, must detect those overrides and arrange to call the Lua functions for that descriptor's encode/decode instead of walking fields in C. This is the same composition point that lets WKT plug into the pure-Lua codec today (runtime/pb/codec.lua short-circuits on desc.encode / desc.decode).

Implementation hook: bd-3k.

#Unknown fields

The pure-Lua decoder preserves unknown fields in t._unknown_fields (a string of raw wire bytes), and the encoder appends them after the known fields. The C path must round-trip identically:

• Decode of a message with unknown wire bytes returns a table whose _unknown_fields is the same byte string the pure-Lua decoder would produce.
• Encode of a table with _unknown_fields set appends those bytes verbatim after known-field output.

Conformance gates this — every Required.Proto3.ProtobufInput.ValidDataUnknown.* test exercises this round-trip. The C path runs the same conformance suite, so regressions surface immediately. Implementation hook: bd-3j.

#Extensions (proto2)

Extension representation in tables: t[<ext_name>] where <ext_name> is the fully-qualified extension field name. Same in both paths.

Extension range handling on decode: unknown extension numbers fall through to _unknown_fields (above). Known extension numbers populate t[<ext_name>].

Implementation hook: bd-3i.

#Lazy view (pb.decode_lazy)

The lazy view is out of scope for C acceleration in the initial ra6 ship. pb.decode_lazy continues to use the pure-Lua path in runtime/pb/lazy.lua regardless of PB_ENABLE_C.

Rationale: the lazy view's value is not doing a full decode. Its hot operations are MessageView:get(name) and :has(name), which do a single field scan over the wire bytes — already O(field_pos) in pure Lua, not the per-message dispatch cost C-accel targets. Adding a C entry point per :get call would re-introduce the boundary cost the architecture sketches were trying to avoid.

Future work (deferred, no BD issue yet): a lazy_c variant could provide a single C entry point for :get_many({names}) if a workload surfaces. Until then, the lazy view stays Lua-only.

The lazy view's public API is unchanged either way — :get(name), :has(name), :which(oneof), :set, :clear, :iter, :names, :to_table, plus ArrayView and MapView methods. Users observe no difference.

#Errors

The C path raises Lua errors via luaL_error, producing the same shape as error(string) in pure Lua. Specifically:

• Type mismatches (wrong field type in input table) → string error with the same template as runtime/pb/codec.lua ("field X expected Y, got Z" or close).
• Wire-format errors on decode → string error matching the pure-Lua message for the same input.
• 64-bit cdata required but Lua number out of range → same message as wire.encode_int64 raises today.

This is expected-output compatibility, not source-text compatibility. The exact prose of error messages may shift slightly between paths during ra6 implementation; the contract is that error type (Lua error vs. silent return) and approximate message content (mentions the field name and the type involved) must match. Tests that pattern-match on error strings should use string.find with anchor patterns, not full-string equality.

Implementation hook: covered cross-cutting in ra6, not a separate BD issue.

#Installation paths

Two paths to install:

#Pure-Lua install (no compiler)

tt rocks install tarantool-protobuf

• Rockspec's optional C build skips when no cc available.
• Install succeeds. Only Lua files land.
• pb.encode / pb.decode work; pb.c_runtime is absent.
• Setting PB_ENABLE_C=1 is silently ignored.
• This is the default install path for everyone who doesn't ask for C.

#C-enabled install

tt rocks install tarantool-protobuf
PB_ENABLE_C=1 tarantool app.lua

• Same rockspec, same tt rocks install command. The rockspec detects cc and builds pb/c_runtime.so (or .dylib on macOS). The same install command produces a C-enabled install when a compiler is present.
• User sets PB_ENABLE_C=1 to activate.

Implementation hook: bd-wky.

#ABI and version skew

The C module's exposed symbols (essentially pb.c_runtime.encode and pb.c_runtime.decode, plus the plan-compilation entry point) are versioned via a single pb.c_runtime._abi_version integer string. runtime/pb/init.lua checks it on load and treats a mismatch as a load failure (same fallback as missing module).

The plan userdata is not ABI. It's process-local, opaque, and recompiled at every module load. Cross-process plan sharing is out of scope (and would also require desc to be serializable, which it isn't today).

Implementation hook: covered in bd-wky packaging discussion.

#Conformance and parity

No new Lua-vs-C diff harness. The existing test infrastructure asserts against reference outputs (golden byte strings, txtpb fixtures, conformance results). When both paths run the same suite and both pass, byte equality is implied by transitivity.

CI matrix runs the full suite twice — PB_ENABLE_C unset and PB_ENABLE_C=1. Both must pass for every merge to master.

Implementation hook: bd-43t.

#What changes

To be unambiguous about what this work does alter:

• runtime/pb/init.lua gains a load-time pcall(require, 'pb.c_runtime') guarded on PB_ENABLE_C=1. (~5 lines.)
• pb.finalize_message(desc) gains a load-time compile step that attaches desc.c_plan when the C runtime is loaded. (~5 lines in the existing finalize function; the actual compile lives in pb.c_runtime.)
• Each generated M.<Type>_encode / M.<Type>_decode wrapper gains a one-line dispatch check. The pure-Lua body underneath is untouched.
• New file runtime/pb/c_runtime.c (or split across several files under runtime/pb/c/) — the actual C implementation.
• New file runtime/pb/c_runtime.lua — a thin Lua wrapper around the C module that performs ABI check and exposes plan compilation and encode/decode functions. Could also be inlined into the C module itself; deferred to bd-mq7 / bd-ra6.
• Justfile gains build-c recipe.
• Rockspec gains optional C build hook.
• Sourcehut .build.yml gains a PB_ENABLE_C=1 job.

That's the complete delta. Nothing else in the runtime tree, nothing in the codegen tree, nothing in examples/expected/, nothing in test/.

#Out of scope

• Replacing the pure-Lua path. mode=full and mode=runtime remain the default forever.
• C-acceleration for pb.decode_lazy (see above).
• C-acceleration for pb.parse (Lua parser, not on hot path).
• C-acceleration for JSON / text codecs beyond what they inherit by composing on top of pb.encode / pb.decode.
• Per-call activation, per-message activation, runtime toggle.
• Cross-process plan caching.

#References

• docs/c-accel.md — architecture
• docs/specs/c_accel_strategy.md — C-level design
• docs/codegen.md — descriptor contract
• bd-pf6 (closed) — architecture decision
• bd-mq7, bd-ra6, bd-43t, bd-wky — downstream work