From 314f642ebb83a814d2f092d421cf6c3720a6efca Mon Sep 17 00:00:00 2001
From: Eugene Blikh <bigbes@gmail.com>
Date: Sat, 16 May 2026 19:55:46 +0300
Subject: [PATCH] docs: refresh README + PLAN; add api-modes and codegen notes
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

README.md
  * Status table reflects current state: text-format encode + decode,
    -0.0 preservation, strict FieldMask, all-green proto3 conformance.
  * Conformance baseline jumped to 1493 / 416 / 0 failures (was
    1389 / 0 / 79 in the old table); explains the 1313+18 skipped
    tests are all TestAllTypesProto2, deferred separately.
  * Layout walks the full runtime/pb/ tree (lazy, text, json, wkt,
    grpc, parser, dynamic, fileset, descriptor_pb).
  * Generated API section calls out the three-mode design and links
    to docs/api-modes.md and docs/codegen.md.

PLAN.md
  * Section 2 ("Current state") rewritten — no longer claims M0;
    lists what's actually in the codebase.
  * M1, M2, M3 marked done with [x] checkboxes (had stale [ ]
    markers across items that have been shipping for months).
  * M5 conformance numbers updated to current baseline; calls out
    the three commits that closed the proto3 suite (text decode,
    -0 codec, JSON strict pass).
  * M7 text-format parser entry updated: 416 ✓ / 0 failures (was
    406 / 10 expected before the codec -0 fix).
  * Added an M7 entry for the JSON strict-validation pass (six
    classes of relaxation now enforced).

docs/api-modes.md (new)
  * When to use full vs runtime (descriptor / reflect) vs lazy.
  * Concrete code shapes for each, plus what the generated
    Person_encode actually looks like in full mode.
  * Descriptor-shape contract that ties all three together.
  * Lazy: SoA index rationale, sparse-read vs dense-read trade-offs,
    cross-over points from bench numbers.

docs/codegen.md (new)
  * Pipeline diagram, CLI options, what gets emitted per .proto.
  * Walk through the inline (full) mode emission with annotated
    generated code.
  * Runtime mode: pb.finalize_message's per-field writer/reader
    closures and why they exist.
  * Hot-path rules the generated code observes (no pairs(), 64-bit
    as cdata, SoA over AoS for large index structures, keep hot
    helpers small).
  * Plugin source map; where to extend for a new wire type.
  * Proto2 deferral section: what it would take and why we punt.
---
 PLAN.md           | 180 +++++++++++++--------
 README.md         | 165 ++++++++++++-------
 docs/api-modes.md | 265 +++++++++++++++++++++++++++++++
 docs/codegen.md   | 391 ++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 882 insertions(+), 119 deletions(-)
 create mode 100644 docs/api-modes.md
 create mode 100644 docs/codegen.md

diff --git a/PLAN.md b/PLAN.md
index 4e3a58bb5d98805016d27f379f629a70d5f5a5a7..d4859c4145c0af55afbabd4841e26f2dc0471b51 100644
--- a/PLAN.md
+++ b/PLAN.md
@@ -17,66 +17,81 @@ A first-class Protocol Buffers + gRPC stack for Tarantool that:
 - Is comfortable to use from real Tarantool apps: clean integration with
   `net.box`, `fiber`, `box.session`, and the 3.x config framework.
 
-## 2. Current state (M0)
+## 2. Current state (post-M7)
 
 See `README.md` for the user-visible summary. Internally:
 
-- Plugin: Go, single mode (descriptor + runtime dispatch).
-- Runtime: pure Lua + LuaJIT FFI; ~260 LOC wire layer, ~250 LOC dispatch.
-- Tests: 25 hand-rolled assertions in one file, no luatest.
-- Demo proto: 1 file, 11 fields, no map/oneof/services.
-
-This is the floor we're building on.
+- **Plugin**: Go, two emission modes (`mode=full` inline + `mode=runtime`
+  descriptor-delegating wrappers). Sibling `protoc-gen-tarantool-doc`
+  emits Markdown reference per `.proto`.
+- **Runtime**: pure Lua + LuaJIT FFI. `wire.lua` (~660 LOC) + `codec.lua`
+  + `lazy.lua` (zero-copy view) + `text.lua` (encode + decode) +
+  `json.lua` (strict proto3 JSON) + `wkt.lua` (all 9 well-known types) +
+  `grpc.lua` (transport + loopback) + `parser.lua` / `dynamic.lua` /
+  `fileset.lua` (three runtime descriptor producers — `.proto` source,
+  AST, FileDescriptorSet bytes).
+- **Tests**: 613 luatest assertions across 19 groups, parametrized over
+  both codegen modes where applicable.
+- **Conformance**: proto3 binary+JSON suite **1493 ✓ / 0 failures**;
+  proto3 text-format suite **416 ✓ / 0 failures** (Google's
+  `conformance_test_runner` v34.1).
+- **Bench**: `bench/baseline.json` tracks allocation/op (regression
+  gate at 5%); `make jit-trace` pins LuaJIT trace stability.
+
+The remaining unfinished bullets in section 3 are mostly M8 (release
+engineering) and a handful of optional perf items in M6.
 
 ## 3. Phased roadmap
 
 Each milestone ends with a green CI run, an updated README, and a tagged
 release on sourcecraft.dev.
 
-### M1 — Two codegen modes + luatest harness  *(in progress)*
+### M1 — Two codegen modes + luatest harness  *(done)*
 
-- [ ] Promote scalar encode/decode to typed helpers in `pb.wire`.
+- [x] Promote scalar encode/decode to typed helpers in `pb.wire`.
       `wire.encode_int32(v)`, `wire.decode_string(buf, pos)`, etc., for all
       15 scalar proto types. Both modes consume the same primitives.
-- [ ] **Full (inline) codegen**: emit per-message `_encode` / `_decode`
+- [x] **Full (inline) codegen**: emit per-message `_encode` / `_decode`
       functions with no descriptor lookup. Tag bytes precomputed at gen
       time as Lua string literals. This is the JIT-friendly hot path.
-- [ ] **Runtime codegen**: keep current behavior. Useful for introspection,
+- [x] **Runtime codegen**: keep current behavior. Useful for introspection,
       schema registries, and forward-compat with descriptor-only consumers.
-- [ ] Plugin parameter `mode=full|runtime` (default: `full`).
-- [ ] Migrate tests to luatest groups. Every behavior tested against **both**
+- [x] Plugin parameter `mode=full|runtime` (default: `full`).
+- [x] Migrate tests to luatest groups. Every behavior tested against **both**
       generated modules to prove parity.
-- [ ] Generate side-by-side outputs in `examples/expected/{full,runtime}/`
+- [x] Generate side-by-side outputs in `examples/expected/{full,runtime}/`
       for visual diffing.
 
-**Done when**: same `.proto` produces two modules; identical wire bytes; one
-luatest run covers both.
-
-### M2 — Composite types
+### M2 — Composite types  *(done)*
 
-- [ ] **`map<K,V>`**: emit as repeated synthetic `*Entry` messages with
+- [x] **`map<K,V>`**: emit as repeated synthetic `*Entry` messages with
       `key`/`value` fields per spec. Decode merges into a Lua table; encode
       iterates with `pairs`. Key types limited to scalars + string per spec.
-- [ ] **`oneof`**: descriptor includes `oneof_index` per field. Encode
+- [x] **`oneof`**: descriptor includes `oneof_index` per field. Encode
       emits at most one branch (last assignment wins). Decode clears prior
-      oneof siblings on assignment.
-- [ ] **proto3 explicit `optional`**: respect presence — emit field even
+      oneof siblings on assignment. Hot-path lookup goes through
+      `desc.oneofs_list` (flat array) to keep the trace JIT-stable.
+- [x] **proto3 explicit `optional`**: respect presence — emit field even
       when value equals scalar default. Generated descriptor exposes
       `has_<name>(t)` / `clear_<name>(t)` helpers.
 
-**Done when**: a "composite" example proto with map<string,int32>, a oneof
-with 3 cases, and an explicit-optional bool round-trips through both modes
-with parity-against-protoc.
-
-### M3 — Well-known types
-
-- [ ] `google.protobuf.Timestamp` ↔ Tarantool `datetime` module
-      (epoch + nsec mapping).
-- [ ] `google.protobuf.Duration` ↔ `interval` (or seconds+nanos table).
-- [ ] Wrappers (Int32Value, StringValue, BoolValue, …) with sugar:
-      pass plain Lua value → auto-wrap; decode → auto-unwrap or keep.
-- [x] `Empty`, `FieldMask`. FieldMask is a `repeated string`; JSON mapping
-      is the canonical comma-joined `lowerCamelCase` form.
+### M3 — Well-known types  *(done)*
+
+- [x] `google.protobuf.Timestamp` ↔ Tarantool `datetime` module
+      (epoch + nsec mapping). Out-of-spec inputs (negative nanos,
+      year > 9999) keep the raw `{seconds, nanos}` table so JSON
+      serialization can reject them with `serialize_error` instead of
+      crashing on decode.
+- [x] `google.protobuf.Duration` ↔ `{seconds, nanos}` table (interval
+      proved a poor fit — it carries months/days that don't map
+      cleanly).
+- [x] Wrappers (Int32Value, StringValue, BoolValue, …) with sugar:
+      pass plain Lua value → auto-wrap; decode → auto-unwrap.
+- [x] FieldMask: `repeated string`. Strict round-trip validation —
+      snake_case paths must use only `[a-z0-9_]`, no leading/trailing
+      `_`, no `__`, and `_` must precede a lowercase letter (not a
+      digit). JSON form rejects any `_` (must be lowerCamelCase).
+- [x] `Empty`.
 - [x] `Any`: opaque `{type_url, value}` form by default; `pb.register(desc)`
       + `pb.any.pack(desc, t)` / `pb.any.unpack(any_t)` for typed
       round-trips. JSON canonical mapping emits the flat `{"@type": ...}`
@@ -125,41 +140,59 @@ with parity-against-protoc.
 fiber and bridges client ↔ handler via `fiber.channel`. All four flavors
 (unary + 3 streaming) are exercised by parameterized luatest groups.
 
-### M5 — Conformance + interop  *(runner shipped; pass-rate tracking pending)*
+### M5 — Conformance + interop  *(proto3 closed; CI wire-up + proto2 deferred)*
 
 - [x] Wire up Google's [protobuf conformance test runner][conformance].
       `cmd/conformance-runner.lua` is the testee: reads length-prefixed
-      `ConformanceRequest` on stdin, runs it through our codec / JSON,
-      writes a length-prefixed `ConformanceResponse` on stdout. Loops to
-      EOF. Core dispatch is factored into `cmd/conformance/core.lua` and
-      exercised directly by `test/conformance_test.lua` (10 dispatch
-      cases + 3 stdin/stdout framing cases).
+      `ConformanceRequest` on stdin, runs it through our codec / JSON /
+      text codecs, writes a length-prefixed `ConformanceResponse` on
+      stdout. Loops to EOF. Core dispatch lives in
+      `cmd/conformance/core.lua` and is exercised directly by
+      `test/conformance_test.lua` so the inner dev loop doesn't need
+      Docker.
 - [x] Run the canonical `conformance_test_runner` binary locally and
       track the pass-rate as a numeric metric (regression gate).
       `docker/conformance.Dockerfile` builds `conformance_test_runner`
       from upstream protobuf v34.1 source (matching the host's
-      `libprotoc 34.1`) and includes Tarantool 3 from the official deb;
+      `libprotoc 34.1`) and bundles Tarantool 3 from the official deb;
       `just conformance` regenerates Lua then runs the harness against
       `cmd/conformance-runner.lua` with the repo mounted as a volume.
-      Watchlists at `test/conformance/known_failures.txt` (main suite)
-      and `test/conformance/known_failures_text.txt` (text-format
-      suite). Current baseline (2026-05-16):
-        - Binary+JSON suite: 1478 ✓ / 1313 skipped /  15 expected fails
-        - Text-format suite:    8 ✓ /  426 skipped /   0 expected fails
-      JSON output runs end-to-end; remaining expected fails are
-      Recommended-only edge cases (FieldMask round-trip,
-      duplicate-field-name rejection, null-in-collection rejection,
-      unknown-enum-name rejection, NullValue oneof validator).
-      Text-format output is wired through `pb.text.encode` and the
-      proto3 text suite is fully clean: SGROUP/EGROUP are tolerated in
-      `wire.skip_field` and `pb.text` renders captured unknown bytes in
-      numeric field-ID form under `opts.print_unknown_fields`. The 426
-      still-skipped tests are proto2/editions message types we don't
-      register. Text-format *input* parsing is still deferred. The
-      `PB_CONFORMANCE_SKIP_JSON=1` env var still short-circuits JSON
-      output if a new encoder bug crashes jsoncpp.
-      CI wire-up pending — the image build is the long pole (~10–15 min
-      on a clean cache).
+      Watchlists at `test/conformance/known_failures.txt` (binary +
+      JSON suite) and `test/conformance/known_failures_text.txt`
+      (text-format suite); **both are empty for the proto3 suites as
+      of 2026-05-16.**
+
+      Current baseline:
+        - Binary+JSON suite: **1493 ✓ / 1313 skipped / 0 failures**
+        - Text-format suite:  **416 ✓ /   18 skipped / 0 failures**
+
+      The 1313 + 18 skipped all target
+      `protobuf_test_messages.proto2.TestAllTypesProto2`. Proto2 codegen
+      is a separate slice — see [docs/codegen.md](../docs/codegen.md)
+      for what it would take.
+
+      Strict-validation closures landed across three commits on the
+      `text-conformance-output` branch:
+        - `pb.text.decode` — full grammar coverage (recursive-descent
+          parser, ~580 LOC; see
+          [docs/text_format_parser_brief.md](../docs/text_format_parser_brief.md)).
+        - `codec` -0.0 preservation — float/double `is_default_scalar`
+          and the inline-codegen elision both gained a sign-bit guard
+          (`1/v == math.huge`).
+        - `pb.json` strict-validation pass — duplicate-key rejection
+          (literal + camel/snake alias detection via a byte-walking
+          pre-scan), null-in-container rejection, unknown-enum-name
+          rejection (with `ignore_unknown_fields` opt for the
+          `JSON_IGNORE_UNKNOWN_PARSING_TEST` conformance category),
+          `google.protobuf.NullValue` round-trip as JSON `null`,
+          strict FieldMask round-trip validation.
+
+      The `PB_CONFORMANCE_SKIP_JSON=1` env var still exists to
+      short-circuit JSON output if a future encoder bug starts
+      crashing jsoncpp.
+- [ ] CI wire-up. The Docker image build is the long pole (~10–15 min
+      on a clean cache); a registry push from a scheduled job would
+      let CI runs reuse a warm cache.
 - [x] Cross-impl interop: 18-fixture corpus in `test/interop/fixtures/`
       produced by mainline `protoc --encode`; tests assert byte-for-byte
       equality.
@@ -282,10 +315,27 @@ fiber and bridges client ↔ handler via `fiber.channel`. All four flavors
       numeric-field-ID tolerance. Range-checks 32/64-bit ints, rejects
       duplicate singular fields, and threads through the conformance
       runner — `cmd/conformance/core.lua` no longer skips `text_payload`.
-      Proto3 text-format conformance suite: **8 ✓ / 426 skipped → 406 ✓
-      / 18 skipped / 10 expected failures** (the 10 are `-0` float/double
-      preservation; their root cause is in the codec, not the parser —
-      see `test/conformance/known_failures_text.txt`).
+      Plugin gained a small `reserved_names` emitter so the parser can
+      match mainline TextFormat::Parser's "silently drop reserved" rule.
+      Proto3 text-format conformance suite: **8 ✓ / 426 skipped → 416
+      ✓ / 18 skipped / 0 failures** (the 18 are the proto2 message-type
+      bucket; everything in scope passes).
+- [x] JSON strict-validation pass. Six classes of relaxation that the
+      proto3 JSON conformance corpus flagged are now enforced — together
+      with the -0 codec fix this empties `known_failures.txt`:
+        1. Duplicate JSON keys (`{"foo":1,"foo":2}`) rejected via a
+           byte-walking pre-scan that runs before `json.decode`.
+        2. camelCase / snake_case aliases of the same proto field
+           rejected via a per-message `field_seen` set.
+        3. JSON `null` inside repeated arrays and map values rejected.
+        4. Unknown enum *names* rejected by default; the
+           `ignore_unknown_fields=true` opt silently drops them (and
+           the conformance dispatch forwards this flag when
+           `req.test_category == JSON_IGNORE_UNKNOWN_PARSING_TEST`).
+        5. `google.protobuf.NullValue` JSON canonical form: literal
+           `null`, not the string `"NULL_VALUE"`. Null on a
+           NullValue-typed oneof member marks the oneof active.
+        6. Strict FieldMask round-trip (see M3 entry).
 - [x] `protoc-gen-tarantool-doc`: sibling Go plugin under
       `cmd/protoc-gen-tarantool-doc/` that emits one Markdown file per
       input `.proto`. Sections: header (package + imports), messages
diff --git a/README.md b/README.md
index 5942ebd20bcb33cd971af93b69407a76c857e5a2..4e7a811ebff1a589359b67849986bf95b963a417 100644
--- a/README.md
+++ b/README.md
@@ -15,7 +15,8 @@ path. This project fills those gaps with:
 
 ## Status
 
-MVP — proto3 messages and enums, end-to-end round-trip verified.
+Proto3 conformance is closed: every `Required.*` and `Recommended.*` test in
+both the binary+JSON and text-format suites passes.
 
 | Feature                          | State        |
 |----------------------------------|--------------|
@@ -26,9 +27,11 @@ MVP — proto3 messages and enums, end-to-end round-trip verified.
 | Enums (open semantics)           | ✅           |
 | 64-bit integers as LuaJIT cdata  | ✅           |
 | Two codegen modes (full + runtime) | ✅         |
+| Zero-copy lazy decode views      | ✅           |
 | `map<K,V>` (scalar/message values) | ✅         |
 | `oneof`                          | ✅           |
-| proto3 explicit `optional` + has_/clear_ | ✅   |
+| proto3 explicit `optional` + `has_*`/`clear_*` | ✅ |
+| `-0.0` preserved for float/double | ✅          |
 | gRPC service stubs (unary)       | ✅           |
 | gRPC streaming (server / client / bidi) | ✅      |
 | Loopback / multiplex transport   | ✅           |
@@ -36,17 +39,18 @@ MVP — proto3 messages and enums, end-to-end round-trip verified.
 | WKT: Duration, Empty, wrappers   | ✅           |
 | WKT: Struct, Value, ListValue    | ✅           |
 | WKT: Any (opaque + registry pack/unpack) | ✅   |
-| WKT: FieldMask                   | ✅           |
-| Byte-for-byte interop with `protoc` | ✅ (18 fixtures) |
-| Google conformance runner (`cmd/conformance-runner.lua`) | ✅ |
-| Runtime `.proto` parsing (`pb.parse`) | ✅           |
-| Runtime `FileDescriptorSet` ingest (`pb.from_pb`) | ✅           |
-| Markdown doc generator (`protoc-gen-tarantool-doc`) | ✅           |
-| proto3 JSON (`pb.json.encode`/`.decode`) | ✅       |
-| Text format printer (`pb.text.encode` / `<Msg>_text`) | ✅ (encode-only) |
-| Unknown-field passthrough (`_unknown_fields`) | ✅      |
+| WKT: FieldMask (strict round-trip) | ✅         |
+| Byte-for-byte interop with `protoc` (18 fixtures) | ✅ |
+| **Google conformance suite — proto3 binary+JSON**  | **1493 ✓ / 0 failures** |
+| **Google conformance suite — proto3 text format**  | **416 ✓ / 0 failures** |
+| Runtime `.proto` parsing (`pb.parse`) | ✅       |
+| Runtime `FileDescriptorSet` ingest (`pb.from_pb`) | ✅ |
+| Markdown doc generator (`protoc-gen-tarantool-doc`) | ✅ |
+| proto3 JSON (`pb.json.encode`/`.decode`) | ✅    |
+| Text format (`pb.text.encode` / `pb.text.decode`) | ✅ |
+| Unknown-field passthrough (`_unknown_fields`)    | ✅ |
 | Microbenchmark + alloc regression gate (`make bench`) | ✅ |
-| proto2 / editions                | ❌ out of scope |
+| proto2 / editions                | ❌ deferred (separate slice; see [docs/codegen.md](docs/codegen.md)) |
 
 ## Quick start
 
@@ -74,14 +78,20 @@ For each message `Foo` the plugin emits:
 ```lua
 local M = require('myapp.proto.foo')
 
-M.Foo_descriptor    -- the descriptor table consumed by the runtime
-M.Foo_new(t)        -- returns t (or {}); placeholder for future validation
-M.Foo_encode(t)     -- table -> wire bytes (string)
-M.Foo_decode(b)     -- wire bytes (string) -> table
-M.Foo_decode_lazy(b) -- wire bytes -> MessageView (zero-copy view)
-M.Foo_text(t, opts) -- table -> protoc-style text format (debug printer)
+M.Foo_descriptor       -- the descriptor table consumed by the runtime
+M.Foo_new(t)           -- returns t (or {}); placeholder for future validation
+M.Foo_encode(t)        -- table -> wire bytes (string)
+M.Foo_decode(b)        -- wire bytes (string) -> table
+M.Foo_decode_lazy(b)   -- wire bytes -> MessageView (zero-copy view)
+M.Foo_text(t, opts)    -- table -> protoc-style text format (debug printer)
+M.Foo_has_<field>(t)   -- only emitted for proto3 explicit-`optional` fields
+M.Foo_clear_<field>(t) -- same
 ```
 
+Text-format **decoding** is exposed on the runtime as `pb.text.decode(desc,
+text, opts)` (no per-message wrapper — it's used from a few places, like
+the conformance runner, and didn't warrant codegen surface).
+
 For each enum `Color`:
 
 ```lua
@@ -94,23 +104,64 @@ Repeated fields are Lua arrays (1-based, contiguous). 64-bit integers
 `uint64_t` cdata — lossless and the same convention used by Tarantool's
 `net.box`, `msgpack`, and built-in `protobuf` modules.
 
+**Three API modes live side-by-side.** Same descriptor, three call shapes —
+the inline (full) generated API is the default, the descriptor-driven runtime
+API is for dynamic schemas, and the lazy API is a zero-copy view for sparse
+reads and proxy / router workloads. See [docs/api-modes.md](docs/api-modes.md)
+for when to pick which, with measured trade-offs.
+
 ## Layout
 
 ```
-cmd/protoc-gen-tarantool/   # Go plugin
-  main.go                   # reads CodeGeneratorRequest, hands off to gen
-  internal/gen/             # codegen package
-runtime/pb/                 # Lua runtime (require('pb'))
-  init.lua                  # public surface
-  wire.lua                  # varint / zigzag / fixed / float / LEN
-  codec.lua                 # descriptor-driven encode/decode
-options/tarantool/          # custom proto options
-  tarantool.proto           # (tarantool.lua_package) file option
-examples/proto/             # demo .proto inputs
-examples/expected/          # generated output (committed for inspection)
-test/                       # roundtrip test
+cmd/
+  protoc-gen-tarantool/      Go plugin (the codegen)
+    main.go                  reads CodeGeneratorRequest, hands off to gen
+    internal/gen/            per-message emission for both modes
+  protoc-gen-tarantool-doc/  separate Go plugin that emits Markdown docs
+  conformance/               Lua conformance dispatch (loaded by the runner)
+  conformance-runner.lua     stdin/stdout framing for `conformance_test_runner`
+
+runtime/pb/                  pure-Lua runtime (`require('pb')`)
+  init.lua                   public surface
+  wire.lua                   varint / zigzag / fixed / float / LEN primitives
+  codec.lua                  descriptor-driven encode/decode
+  lazy.lua                   zero-copy MessageView / ArrayView / MapView
+  text.lua                   text format encode + decode (proto3)
+  json.lua                   proto3 JSON encode + decode (strict)
+  wkt.lua                    Timestamp / Duration / Empty / Wrappers /
+                             Struct / Value / ListValue / Any / FieldMask
+  grpc.lua                   transport interface + loopback / multiplex
+  parser.lua                 pure-Lua proto3 schema parser (.proto → AST)
+  dynamic.lua                AST → descriptor module
+  fileset.lua                FileDescriptorSet bytes → descriptor module
+  descriptor_pb.lua          hand-built descriptors of descriptor.proto
+
+options/tarantool/           custom proto file options
+  tarantool.proto            (tarantool.lua_package) — Lua require path override
+
+examples/proto/              demo .proto inputs
+examples/expected/           generated output for full + runtime modes
+                             (both committed for inspection + parametrized tests)
+test/                        luatest groups, conformance regressions, fixtures
+docs/                        codegen notes, API mode comparison, design briefs
+bench/                       per-helper bench + JIT-trace gate + alloc baseline
 ```
 
+## Documentation
+
+- **[docs/api-modes.md](docs/api-modes.md)** — when to use `Foo_encode` (full),
+  `pb.encode(desc, t)` (runtime / reflect), or `pb.decode_lazy(desc, b)` (lazy
+  view). Measured allocation + throughput trade-offs.
+- **[docs/codegen.md](docs/codegen.md)** — plugin internals, descriptor shape
+  contract, how to add a new wire type or scalar, the LuaJIT hot-path rules
+  the generated code observes.
+- **[docs/text_format_parser_brief.md](docs/text_format_parser_brief.md)** —
+  retrospective brief on the text-format parser slice (`pb.text.decode`).
+- **PLAN.md** — phased roadmap and per-feature design notes.
+- **CLAUDE.md** — invariants and conventions enforced across the codebase
+  (no `pairs()` on hot paths, 64-bit ints as cdata, SoA over AoS for large
+  index structures, etc.).
+
 ## Conformance
 
 `cmd/conformance-runner.lua` speaks the [Google protobuf conformance
@@ -132,30 +183,29 @@ just conformance
 ```
 
 (Mounts the repo into the container — generated Lua from `make gen` on the
-host is what gets tested.) Known failures live in `test/conformance/known_failures.txt` (binary +
-JSON suite) and `test/conformance/known_failures_text.txt` (text-format
-suite); the runner exits zero only when actual failures match those
-lists exactly.
-
-Current baseline (2026-05-15, protobuf v34.1):
-
-| Suite | Successes | Skipped | Expected failures |
-|-------|-----------|---------|-------------------|
-| Binary + JSON | 1389 | 1313 | 79 |
-| Text-format   |    0 |  430 |  4 |
-
-JSON output runs end-to-end through the conformance harness. The
-remaining failures are mostly canonical-form edge cases — Duration /
-Timestamp serialization rules, double-precision formatting, NaN
-canonicalization, JSON-input rejection rules — listed in
-`test/conformance/known_failures.txt`. Set `PB_CONFORMANCE_SKIP_JSON=1`
-to short-circuit JSON output requests with `skipped` if a new encoder
-bug starts crashing jsoncpp again.
-
-The runner currently supports `protobuf_test_messages.proto3.TestAllTypesProto3`
-in both protobuf and JSON formats; proto2 / editions / JSPB / text-format
-cases return `skipped`. The self-test in `test/conformance_test.lua`
-exercises the runner with crafted requests on every `make test` run.
+host is what gets tested.) Known failures live in
+`test/conformance/known_failures.txt` (binary + JSON suite) and
+`test/conformance/known_failures_text.txt` (text-format suite); both are
+empty for the proto3 suites as of 2026-05-16.
+
+Current baseline (2026-05-16, protobuf v34.1):
+
+| Suite | Successes | Skipped | Expected failures | Unexpected |
+|-------|-----------|---------|-------------------|------------|
+| Binary + JSON | **1493** | 1313 | 0 | 0 |
+| Text-format   |  **416** |   18 | 0 | 0 |
+
+The 1313 + 18 skipped tests all target
+`protobuf_test_messages.proto2.TestAllTypesProto2`, which we don't generate
+Lua for. Proto2 support is a separate slice — see
+[docs/codegen.md#proto2-deferral](docs/codegen.md) for what it would take.
+
+The runner supports `protobuf_test_messages.proto3.TestAllTypesProto3` in
+binary, JSON, and text-format input/output, including the
+`JSON_IGNORE_UNKNOWN_PARSING_TEST` category (forwarded as
+`ignore_unknown_fields=true` to `pb.json.decode`). The self-test in
+`test/conformance_test.lua` exercises the runner with crafted requests on
+every `make test` run.
 
 [gconf]: https://github.com/protocolbuffers/protobuf/tree/main/conformance
 
@@ -181,6 +231,13 @@ Current baseline (LuaJIT 2.1, hello.Person):
 | 10 KB   | 47.5 / 48.6 KB                | 59.5 / 59.6 KB                |
 | 100 KB  | 444 / 445 KB                  | 568 / 568 KB                  |
 
+Lazy decode trades a much higher *index-build* cost (one MessageView
+table + four SoA arrays) for near-zero allocation on subsequent
+field reads — best when you touch a small fraction of fields, or when
+you re-encode mostly-unchanged messages (proxy / router workloads). See
+[docs/api-modes.md](docs/api-modes.md) for the bench numbers and the
+cross-over point.
+
 ## Why named `pb` instead of `protobuf`?
 
 Tarantool's loader prefers the built-in `require('protobuf')` over any
diff --git a/docs/api-modes.md b/docs/api-modes.md
new file mode 100644
index 0000000000000000000000000000000000000000..2984736685ba0716eddb3b226eebaba6dc8efa75
--- /dev/null
+++ b/docs/api-modes.md
@@ -0,0 +1,265 @@
+# API modes: full, runtime, lazy
+
+The same descriptor table feeds three encode/decode call shapes. They are
+**not exclusive** — every generated `_pb.lua` exposes all three (`_encode`,
+`_decode`, `_decode_lazy`), and `pb.encode` / `pb.decode` / `pb.decode_lazy`
+work directly against any descriptor. Picking is per-call-site, not
+per-message and not per-build.
+
+## TL;DR
+
+| Mode | Call shape | Best for | Pays |
+|---|---|---|---|
+| **Full** (inline codegen) | `M.Foo_encode(t)` / `M.Foo_decode(b)` | Hot RPC paths, anything you call >1k times/sec on a known type | Larger generated `_pb.lua`; one wrapper per message |
+| **Runtime** (descriptor-driven / reflect) | `pb.encode(desc, t)` / `pb.decode(desc, b)` | Dynamic schemas (`pb.parse`, `pb.from_pb`, reflection); cases where you only have a descriptor at runtime | One extra indirection per field (descriptor table dispatch) |
+| **Lazy** (zero-copy view) | `M.Foo_decode_lazy(b)` / `pb.decode_lazy(desc, b)` | Proxy / router shapes that touch a few fields and re-encode; sparse reads from large payloads | Higher fixed cost per decode (index build); per-field GC pressure on dense reads |
+
+If you don't know which to use: **start with full**. Switch to lazy when
+profiling shows you're decoding more than you read; switch to runtime when
+the schema isn't known at build time.
+
+## Full (inline codegen)
+
+The default, generated by `mode=full` (the codegen default). For each
+message the plugin emits dedicated `_encode` and `_decode` functions with
+the wire calls and tag bytes inlined.
+
+```lua
+local hello = require('myapp.proto.hello')
+
+local bytes = hello.Person_encode({
+    name    = 'Alice',
+    user_id = require('ffi').cast('uint64_t', 42),
+    emails  = {'a@x', 'b@x'},
+})
+local p = hello.Person_decode(bytes)
+```
+
+Under the hood the generated `Person_encode` looks like:
+
+```lua
+function M.Person_encode(t)
+    if type(t) ~= 'table' then
+        error("expected table for hello.Person, got " .. type(t), 0)
+    end
+    local out, n = {}, 0
+    local v
+    v = t.name
+    if v ~= nil and v ~= '' then
+        n = n + 1; out[n] = "\x0a"           -- precomputed tag bytes
+        n = n + 1; out[n] = wire.encode_varint(#v)
+        n = n + 1; out[n] = v
+    end
+    v = t.user_id
+    if v ~= nil and v ~= 0 then
+        n = n + 1; out[n] = "\x10"
+        n = n + 1; out[n] = wire.encode_uint64(v)
+    end
+    -- ... one branch per field ...
+    return table.concat(out)
+end
+```
+
+No descriptor table lookups, no kind dispatch, no `pairs()` over a hash
+on the hot path. Tag bytes are precomputed Lua string literals. The LuaJIT
+trace compiler sees a flat sequence of monomorphic wire calls and
+specializes the whole encode into a single trace.
+
+This is what `make bench` measures by default. The
+`bench/baseline.json` numbers under the "full" column are this path.
+
+## Runtime (descriptor-driven / reflect)
+
+The plugin's `mode=runtime` emits thin wrappers that delegate to
+`pb.encode(desc, t)` and `pb.decode(desc, b)`:
+
+```lua
+function M.Person_encode(t) return pb.encode(M.Person_descriptor, t) end
+function M.Person_decode(b) return pb.decode(M.Person_descriptor, b) end
+```
+
+You can also call those directly with any descriptor — that's the
+"reflect" use case:
+
+```lua
+local pb = require('pb')
+
+-- Parse a .proto file at runtime; no codegen, no Go plugin.
+local mod  = pb.parse(io.open('hello.proto'):read('*a'))
+local desc = mod.Person_descriptor
+
+-- Same encode / decode surface, just bound to a runtime-built descriptor.
+local bytes = pb.encode(desc, {name = 'Alice'})
+local p     = pb.decode(desc, bytes)
+
+-- Or ingest a binary FileDescriptorSet from protoc --descriptor_set_out:
+local set  = pb.from_pb(io.open('build/all.pb', 'rb'):read('*a'))
+local desc = set.lookup('hello.Person')
+local p    = pb.decode(desc, bytes)
+```
+
+The descriptor table is the contract:
+
+```lua
+{
+    name = 'hello.Person',
+    fields = {
+        {name='name', id=1, kind='scalar', proto_type='string'},
+        {name='user_id', id=2, kind='scalar', proto_type='uint64'},
+        {name='emails', id=3, kind='scalar', proto_type='string', repeated=true},
+        {name='status', id=4, kind='enum', enum=<enum_descriptor>},
+        {name='address', id=5, kind='message', message=<other_descriptor>},
+        {name='ages_by_nickname', id=13, kind='map',
+         key={kind='scalar', proto_type='string'},
+         value={kind='scalar', proto_type='int32'}},
+        -- modifiers: repeated, packed, oneof, optional
+    },
+    field_by_id   = {[1]=<field>, ...},  -- filled by pb.finalize_message
+    field_by_name = {name=<field>, ...},
+    oneofs        = {<oneof_name> = {<field_names>}},
+    reserved_names = {[<name>]=true},     -- for text-format parser
+}
+```
+
+Three producers emit this shape — generated codegen (runtime mode),
+`pb.parse` (from .proto source), `pb.from_pb` (from
+FileDescriptorSet bytes). All three flow through the same `pb.codec`.
+
+The `pb.finalize_message(desc)` helper builds `field_by_id` /
+`field_by_name` / `oneofs_list` and attaches per-field `_writer` /
+`_reader` specializations. Always call it on hand-rolled descriptors.
+
+**Cost vs full**: per-field table lookup + kind dispatch. Allocation
+overhead is one extra `pairs()` walk in encode and a small per-field
+function-call cost; on `make bench` numbers the runtime mode is ~5-15%
+slower for encode and within noise for decode (the decoder fast paths
+are shared via `f._reader` closures).
+
+WKTs (`google.protobuf.Timestamp` etc) carry their own `desc.encode` /
+`desc.decode` overrides; the runtime codec dispatches to them in place
+of the generic field walk. That's the extension point for any
+descriptor that wants custom handling without a special case in the
+codec.
+
+## Lazy (zero-copy view)
+
+`pb.decode_lazy(desc, bytes)` returns a `MessageView` instead of a Lua
+table. Nothing past the field index gets decoded eagerly: scalar
+materialization happens on `:get`, repeated/map fields become
+`ArrayView` / `MapView` sub-views, sub-messages become nested
+`MessageView`s on access.
+
+```lua
+local view = hello.Person_decode_lazy(bytes)
+
+-- Direct field reads (decoded on demand, then cached on the view).
+local name = view:get('name')
+local uid  = view:get('user_id')
+
+-- Repeated field: ArrayView.
+local emails = view:get('emails')   -- ArrayView, no per-element table
+for i, e in emails:iter() do
+    print(i, e)
+end
+print('count:', emails:len())
+
+-- Map field: MapView.
+local ages = view:get('ages_by_nickname')
+print(ages:get('alice'))
+
+-- Sub-message: a nested MessageView.
+local addr = view:get('address')
+print(addr:get('city'))
+
+-- Presence + oneof which-branch checks without forcing a decode.
+view:has('user_id')        -- bool
+view:which('outcome')      -- string?
+
+-- Iterate fields actually present on the wire (skip-aware).
+for name, val in view:iter() do print(name, val) end
+
+-- Mutate, then re-encode. Untouched fields are spliced byte-for-byte
+-- from the original payload; only dirty fields run through encode.
+view:set('user_id', require('ffi').cast('uint64_t', 99))
+local new_bytes = view:encode()
+
+-- Force a fully-materialized table when you actually need one.
+local t = view:totable()
+```
+
+### When lazy wins
+
+- **Sparse reads** — payload is large, you only need a few fields.
+  Index build is ~one Lua table + four SoA int arrays sized to
+  `<wire-segment-count>`. Per-field reads are O(1) after the index.
+- **Mostly-passthrough re-encode** — proxy or router shapes that
+  decode, touch a couple of fields, and re-encode. The byte-splice
+  path skips encode entirely for untouched fields; only dirty fields
+  pay the encode cost. On the 100 KB hello.Person bench this puts the
+  mutate-then-reencode workload at **1.09-1.26× of eager** despite the
+  extra index build.
+- **Avoiding intermediate table churn** when iterating large repeated
+  fields: `ArrayView:iter()` doesn't allocate a flat array; values
+  flow through one at a time.
+
+### When lazy loses
+
+- **Dense reads** — if you read every field, the index build is wasted
+  work and each `:get` adds a Lua-call boundary the eager decoder
+  avoided. Eager wins by ~10-30% on full-field walks.
+- **Tiny messages** — for a 10 B payload the index build dominates.
+  The cross-over (on hello.Person) is around 1 KB.
+- **Anything that wants the result as a plain Lua table** — `:totable()`
+  reverses the win.
+
+### Implementation note (why SoA)
+
+The lazy index is structure-of-arrays, not array-of-structs: for
+N wire segments the indexer keeps four int arrays of size N (`id`,
+`tag_start`, `val_start`, `next_start`) instead of N tiny tables of 4
+keys each. Tiny per-entry tables in LuaJIT carry header overhead +
+hash dispatch costs that dominate for large N. Switching from AoS to
+SoA on the protobuf-lazy slice took sparse-read from 0.66× of eager
+back to 1.0×-1.16×.
+
+If you write your own index-style structure with many small entries,
+follow the same pattern. See `runtime/pb/lazy.lua`'s `index_bytes`
+for the reference shape.
+
+## Mixing modes
+
+All three modes interoperate freely:
+
+- A `MessageView` from `decode_lazy` can be `:set(...)` and `:encode()`'d
+  back to wire bytes that `pb.decode` (eager) parses identically.
+- A descriptor from `pb.parse` (runtime mode) plugs into
+  `pb.decode_lazy` exactly like a generated descriptor.
+- The codec's `desc.encode` / `desc.decode` override mechanism lets WKT
+  and user-registered descriptors short-circuit any of the three paths
+  without special cases in the codec.
+
+The contract is the descriptor table — the codec, both codegen modes,
+the dynamic parser, the JSON codec, the text codec, and the lazy view
+all consume the same shape. That's the design property that lets these
+three APIs coexist without forks.
+
+## Choosing per call site, not per build
+
+Both codegen modes always emit `Foo_decode_lazy` and `Foo_text`
+wrappers. The plugin's `mode=full`/`mode=runtime` switch only changes
+how `Foo_encode` / `Foo_decode` are emitted — everything else lives in
+the shared runtime. So you can:
+
+- Generate everything with `mode=full` (the default) and still call
+  `pb.encode(desc, t)` against the descriptor when you need a
+  descriptor-driven path.
+- Generate everything with `mode=runtime` if you want minimum
+  generated-code size, and pay the modest descriptor-dispatch overhead
+  uniformly.
+- Reach for lazy on a per-call basis when the workload shape calls for
+  it — typically in handlers that operate on large messages but only
+  inspect a few fields.
+
+The test suite runs every behavior test against both codegen modes
+(`full` and `runtime`) via the same descriptor; cross-mode parity is
+pinned by `parity.full_vs_runtime.*` test groups.
diff --git a/docs/codegen.md b/docs/codegen.md
new file mode 100644
index 0000000000000000000000000000000000000000..da5c1b18a4d8a1099d733db1b24bddbd370b5453
--- /dev/null
+++ b/docs/codegen.md
@@ -0,0 +1,391 @@
+# Codegen notes
+
+How `protoc-gen-tarantool` works, what it emits, and where to extend it.
+
+This is a companion to [docs/api-modes.md](api-modes.md). API-modes
+covers the *user-facing* difference between full / runtime / lazy; this
+doc covers what the plugin does to produce the code those modes rely on.
+
+## The pipeline
+
+```
+.proto sources
+    │
+    ▼  protoc (mainline binary, host-installed)
+CodeGeneratorRequest (proto descriptor bytes on stdin)
+    │
+    ▼  cmd/protoc-gen-tarantool/main.go
+    │     advertises FEATURE_PROTO3_OPTIONAL so protoc surfaces
+    │     explicit-optional fields (without this flag, it omits them)
+    │
+    ▼  cmd/protoc-gen-tarantool/internal/gen.GenerateFile
+    │
+    ├─ ModeFull    → inline `_encode` / `_decode` (the default)
+    └─ ModeRuntime → wrappers that delegate to pb.encode / pb.decode
+
+one `<lua_pkg>.lua` per input `.proto`
+```
+
+The plugin only handles `syntax = "proto3"`. Proto2 input is rejected
+at the top of `GenerateFile`. See [proto2 deferral](#proto2-deferral)
+for the rationale.
+
+## CLI options
+
+Passed via `protoc --tarantool_opt=<key>=<value>,...`:
+
+| Option | Values | Meaning |
+|---|---|---|
+| `mode` | `full` (default) or `runtime` | Inline `_encode`/`_decode` vs descriptor-delegating wrappers. See [api-modes.md](api-modes.md). |
+| `prefix` | any Lua-require path | Prepended to every generated module's require path **and** its on-disk subpath. The Makefile uses this to generate both modes side-by-side into `examples/expected/{full,runtime}/`. |
+
+The `(tarantool.lua_package)` file option (defined in
+`options/tarantool/tarantool.proto`) overrides the per-file Lua module
+path:
+
+```proto
+import "tarantool/tarantool.proto";
+option (tarantool.lua_package) = "myapp.proto.foo";
+```
+
+Without that option the path mirrors `package` (e.g.
+`package my.app; foo.proto` → `my/app/foo_pb.lua`,
+`require('my.app.foo_pb')`).
+
+## What gets emitted per .proto
+
+```lua
+-- Code generated by protoc-gen-tarantool. DO NOT EDIT.
+-- source: hello.proto
+-- syntax: proto3
+
+local pb = require('pb')
+local wire = pb.wire
+
+local M = {}
+
+-- 1. Enums first (no forward-reference problems).
+M.Status_descriptor = pb.enum('hello.Status', { ACTIVE=0, BANNED=1, ... })
+M.Status            = M.Status_descriptor.by_name
+
+-- 2. Predeclare all message descriptor tables. Self-references and
+--    mutual recursion resolve here because every descriptor is named
+--    before any field references it.
+M.Person_descriptor = {name = 'hello.Person'}
+M.Address_descriptor = {name = 'hello.Address'}
+M.Result_descriptor = {name = 'hello.Result'}
+
+-- 3. Fill in fields[] per message, then call pb.finalize_message.
+M.Person_descriptor.fields = {
+    {name='name',    id=1, kind='scalar', proto_type='string'},
+    {name='user_id', id=2, kind='scalar', proto_type='uint64'},
+    -- ...
+}
+M.Person_descriptor.oneofs        = { outcome = {'text', 'code', 'details'} }
+M.Person_descriptor.reserved_names = { ['old_field'] = true }
+pb.finalize_message(M.Person_descriptor)
+
+-- 4. EmmyLua / lua-language-server type annotations.
+---@class hello.Person
+---@field name string
+---@field user_id integer
+-- ...
+
+-- 5. Wrappers: _new / _encode / _decode / _decode_lazy / _text
+--    + _has_<field> / _clear_<field> for explicit-optional fields.
+function M.Person_new(t) return t or {} end
+function M.Person_encode(t) -- inline body in full mode, delegation in runtime mode
+function M.Person_decode(b) -- same
+function M.Person_decode_lazy(b) return pb.decode_lazy(M.Person_descriptor, b) end
+function M.Person_text(t, opts) return pb.text.encode(M.Person_descriptor, t, opts) end
+
+-- 6. Services (mode-independent — they reuse the per-message wrappers).
+M.Greeter_service = { ... }
+function M.Greeter_client(transport) ... end
+function M.Greeter_server(impl) ... end
+
+return M
+```
+
+Generation order matters: enums and predeclared descriptors come before
+field tables so cross-references inside `fields = {...}` resolve in one
+pass.
+
+## The descriptor table — the contract
+
+The shape consumed by the runtime is:
+
+```lua
+{
+    name = 'pkg.Foo',
+    fields = {
+        {name='x', id=1, kind='scalar', proto_type='int32'},
+        {name='y', id=2, kind='message', message=<other_descriptor>},
+        {name='z', id=3, kind='enum',    enum=<enum_descriptor>},
+        {name='m', id=4, kind='map',     key=<sub_field>, value=<sub_field>},
+        -- modifiers: repeated, packed, oneof, optional
+    },
+    -- Filled in by pb.finalize_message:
+    field_by_id   = {[1]=<field>, ...},
+    field_by_name = {x=<field>, ...},
+    oneofs_list   = {{name=<n>, members={<names>}}, ...},
+    -- Optional, set by the codegen:
+    oneofs         = {<name> = {<member-names>}},   -- input form, flattened above
+    reserved_names = {[<name>] = true},              -- for text-format parser
+    encode = <fn>,  decode = <fn>,                   -- override hook (WKT use this)
+    text   = <fn>,                                   -- text-format encode override
+}
+```
+
+**Three producers emit this exact shape.** They live in different
+codepaths but consume the same runtime:
+
+1. `cmd/protoc-gen-tarantool/internal/gen` — build-time codegen.
+2. `runtime/pb/dynamic.lua` — runtime synthesis from an AST that
+   `runtime/pb/parser.lua` produces from `.proto` source.
+3. `runtime/pb/fileset.lua` — runtime synthesis from a
+   `FileDescriptorSet` binary (`protoc --descriptor_set_out=...`).
+
+Hand-rolled WKT descriptors in `runtime/pb/wkt.lua` use the same shape
+but additionally set `desc.encode` / `desc.decode` to take over the wire
+path entirely (Timestamp, Duration, Wrappers, Struct, Value, ListValue,
+Any, FieldMask, Empty).
+
+**Pinning this shape is what lets us run every behavior test against
+both codegen modes.** The test harness in `test/protobuf_test.lua` and
+friends parametrizes by mode (`{'full', 'runtime'}`) and runs the same
+assertions against `require('full.hello.hello_pb')` and
+`require('runtime.hello.hello_pb')`.
+
+## Inline (full) mode — what it looks like
+
+`inline.go` walks each field and emits Lua statements that call into
+`runtime/pb/wire.lua` primitives directly. Tag bytes are precomputed at
+codegen time as Lua string literals so the runtime never re-encodes
+them.
+
+```lua
+function M.Person_encode(t)
+    if type(t) ~= 'table' then
+        error("expected table for hello.Person, got " .. type(t), 0)
+    end
+    local out, n = {}, 0
+    local v
+
+    -- field 1: name (string, singular)
+    v = t.name
+    if v ~= nil and v ~= '' then               -- proto3 default elision
+        n = n + 1; out[n] = "\x0a"             -- tag(1, LEN), precomputed
+        n = n + 1; out[n] = wire.encode_varint(#v)
+        n = n + 1; out[n] = v
+    end
+
+    -- field 11: weight_kg (double, singular)
+    v = t.weight_kg
+    if v ~= nil and (v ~= 0 or 1/v == -math.huge) then  -- -0 preserved
+        n = n + 1; out[n] = "\x59"
+        n = n + 1; out[n] = wire.encode_double(v)
+    end
+
+    -- ... oneof pre-pass picks active branch; map fields walk pairs(); etc.
+
+    return table.concat(out)
+end
+```
+
+The decoder is similarly inlined:
+
+```lua
+function M.Person_decode(b)
+    local result = {}
+    local pos, len = 1, #b
+    while pos <= len do
+        local id, wt
+        id, wt, pos = wire.decode_tag(b, pos)
+        if id == 1 then                         -- name
+            result.name, pos = wire.decode_string(b, pos)
+        elseif id == 2 then                     -- user_id
+            result.user_id, pos = wire.decode_uint64(b, pos)
+        -- ... if-elseif chain on field id ...
+        else
+            pos = wire.skip_field(b, pos, wt, id)   -- unknown
+        end
+    end
+    return result
+end
+```
+
+**Why an `if-elseif` chain rather than a dispatch table?** LuaJIT
+compiles short chains into a tight branch sequence on a single trace.
+The crossover where a jump table would win is at field counts higher
+than anything we hit in practice; pinned by `bench/jit_trace.lua`.
+
+## Runtime mode — the thin wrappers
+
+`mode=runtime` skips the inline emission and produces:
+
+```lua
+function M.Person_encode(t) return pb.encode(M.Person_descriptor, t) end
+function M.Person_decode(b) return pb.decode(M.Person_descriptor, b) end
+```
+
+`pb.encode` / `pb.decode` (in `runtime/pb/codec.lua`) walk the
+descriptor table to pick wire primitives at runtime. Slower per
+field — a hash lookup + closure call instead of a hard-coded branch —
+but smaller generated output (no per-message `_encode` / `_decode`
+bodies) and useful when the same code path needs to handle dynamic
+descriptors.
+
+`pb.finalize_message` (in `runtime/pb/init.lua`) builds two
+specialization layers on top of the descriptor at finalize-time:
+
+1. **Per-field writer closures** (`f._writer`): pre-bound to the
+   field's tag, encoder, and default predicate. The encode loop calls
+   `f._writer(data, out)` per field instead of dispatching on
+   `f.kind` / `f.proto_type` inside the loop.
+2. **Per-field reader closures** (`f._reader`): same idea for decode,
+   handling typed value extraction + repeated bookkeeping + nested
+   merge + oneof sibling clearing in one closure per field.
+
+These specializations bridge most of the gap between runtime mode and
+full mode for the *decoder*; encode is still meaningfully slower in
+runtime mode because the field walk goes through one extra Lua-function
+boundary per field.
+
+## The hot-path rules the generated code observes
+
+These come from CLAUDE.md and are enforced across both the codegen and
+the runtime. New code (codegen output or hand-written) needs to follow
+them or the LuaJIT trace compiler bails out.
+
+- **No `pairs()` on hot paths.** `pairs()` over a hash compiles to
+  bytecode `ISNEXT`, which is NYI in Tarantool's LuaJIT 2.1 fork —
+  the trace aborts and the code path falls off the JIT. Map fields are
+  the one accepted exception; the JIT-trace gate in `bench/jit_trace.lua`
+  pins the limitation. When a descriptor needs hash-shaped lookup state
+  (e.g. oneof grouping), build a parallel array view at
+  `pb.finalize_message` time (`desc.oneofs_list`) and iterate the array
+  on the hot path.
+- **64-bit integers stay as cdata.** `int64`/`uint64`/`sint64`/
+  `fixed64`/`sfixed64` are LuaJIT `int64_t` / `uint64_t` cdata
+  everywhere. Same convention as Tarantool's `msgpackffi`, `net.box`,
+  `box.tuple`, and built-in `protobuf`. Don't narrow through
+  `tonumber` — precision loss past 2^53 silently corrupts IDs and
+  timestamp nanos.
+- **`ipairs` / `for i=1,#t`, never `pairs()`, on generated arrays.**
+  Repeated fields are 1-based contiguous Lua arrays. Generated encode
+  walks them with `for i = 1, #v do`.
+- **Keep hot helpers small.** Adding "more fast paths" to a helper that
+  the JIT already inlines (e.g. extending `encode_varint`'s 1-byte
+  fast path with 2/3/4-byte branches) can push the function past the
+  LuaJIT inline budget — parent traces stop inlining it and the
+  dominant 1-byte case regresses by ~30%. Multi-byte fast paths live
+  in the *slow* helper, not the inlined one.
+- **SoA over AoS for many-entry index structures.** For per-element
+  records where N is in the thousands, use parallel int arrays, not
+  one small Lua table per entry. Each per-entry table is a separate
+  allocation with header overhead and hash dispatch; for large N the
+  allocations dominate. See `runtime/pb/lazy.lua`'s `index_bytes`
+  for the reference shape.
+- **WKT routing.** When codegen sees a field referencing a
+  `google.protobuf.*` type, it emits a reference to
+  `pb.wkt.<Name>_descriptor` instead of treating it as an ordinary
+  cross-file import. The plugin does NOT generate a Lua module for
+  imported WKT `.proto` files — those are fulfilled by
+  `runtime/pb/wkt.lua`.
+
+## Plugin source layout
+
+```
+cmd/protoc-gen-tarantool/
+  main.go              # reads CodeGeneratorRequest from stdin, sets up
+                       # the plugin options + FEATURE_PROTO3_OPTIONAL,
+                       # dispatches to gen.GenerateFile per .proto
+  internal/gen/
+    gen.go             # per-file orchestration + runtime-mode wrappers,
+                       # imports collection, enum + message emission,
+                       # descriptor field tables, oneofs, reserved_names
+    inline.go          # full-mode inline _encode / _decode body emission,
+                       # field-kind dispatch, repeated / map / oneof shapes
+    service.go         # gRPC client / server factories
+    name.go            # Lua name + path mangling rules
+    types.go           # protoreflect.Kind → scalar name mapping
+    options.go         # (tarantool.lua_package) file option lookup
+    emmylua.go         # EmmyLua / lua-language-server type annotation
+                       # emission (---@class, ---@field, ---@param)
+```
+
+**Adding a new wire type or scalar** means touching three places:
+
+1. `runtime/pb/wire.lua` — add the `encode_xxx` / `decode_xxx`
+   primitives + a `TYPE_INFO[xxx]` entry. That's the single source of
+   truth: the codec, both codegen modes, and the dynamic parser all
+   pick it up from there.
+2. `cmd/protoc-gen-tarantool/internal/gen/types.go` —
+   `scalarName(k)` mapping for the new `protoreflect.Kind`.
+3. `cmd/protoc-gen-tarantool/internal/gen/inline.go` — emission for
+   the new type (singular + repeated + packed branches).
+
+If the new type is `LEN`-typed (string/bytes-shaped), check the
+existing string/bytes paths for the split-emission pattern
+(`tag + varint(#v) + body` as three separate `out` slots instead of
+one concatenated `wire.encode_len(v)` call) — that avoids a per-field
+string allocation.
+
+## Sibling plugin: protoc-gen-tarantool-doc
+
+`cmd/protoc-gen-tarantool-doc/` is a second Go plugin that consumes
+the same `CodeGeneratorRequest` and emits one Markdown file per input
+`.proto`. Build with `make build-doc`; sample docs land in
+`examples/docs/` via `make gen-docs`. It's deliberately separate from
+the Lua codegen to keep the codegen plugin small.
+
+## Proto2 deferral
+
+The plugin rejects `syntax = "proto2"` outright. The conformance
+suites' 1331 skipped tests are all `TestAllTypesProto2`.
+
+Proto2 support is genuinely a separate slice. The wire format is
+identical to proto3 (with the one exception of `group`s — wire-types 3
+and 4, SGROUP/EGROUP, which we already tolerate on the skip path but
+don't encode/decode). The schema features that *don't* exist in proto3
+are what would need work:
+
+- **`required` fields** — codec must error on missing required at both
+  encode and decode (we currently treat absence as default).
+- **`optional` everywhere** — proto2 fields are presence-tracked by
+  default. proto3's explicit-optional path already handles this; just
+  needs the plugin to mark every proto2 field as `optional=true`.
+- **Extensions** (`extensions 100 to 199;` + `extend Foo {...}`)
+  — proto2-only mechanism for adding fields to messages defined
+  elsewhere. Round-trip overlaps with our existing
+  `_unknown_fields` machinery but explicit support needs a separate
+  descriptor table per extension and a way to expose it on user
+  messages.
+- **Groups** (`optional group Foo = 1 { ... }`) — actual encode/decode
+  rather than the skip-only tolerance we have today, plus a different
+  text-format rendering (`Foo { ... }` with the group's capitalized
+  name instead of the submessage's field name).
+- **Custom defaults** (`[default = X]`) — only matter when callers
+  query "what's the default value of field X"; for round-trips they
+  can be ignored.
+- **`MessageSet`** wire format — a few specific tests.
+
+Realistic phasing (if you take this on):
+
+1. Drop the proto3-only gate in the plugin; mark proto2 fields as
+   `optional=true`; skip extensions / groups / required harmlessly
+   (warn but emit the rest of the message). Re-run conformance —
+   should unlock most of the 1007 binary tests and ~200 of the JSON
+   tests immediately because the wire format is the same.
+2. Implement group encode/decode + the text-format group rendering.
+   Picks up the remaining text tests and a handful of binary tests.
+3. Required-field validation. Small, targeted at the
+   `TestAllRequiredTypesProto2` cases.
+4. Extensions, MessageSet, custom defaults — long tail, optional
+   depending on demand.
+
+This is a meaningfully larger slice than the typical addition here —
+plan on a few sessions, not an evening. The trade-off is "proto2 is
+mostly legacy" (see the README's status table for who actually uses
+it) vs "1331 extra ✓ in the conformance numbers looks good".