~bigbes/tarantool

tarantool-protobuf

ref: d7b0364a82bd5beb8c2fb9d643a191a1aa42e5ab tarantool-protobuf/test/json_test.lua -rw-r--r-- 21.8 KiB
7e9e3e43 — Eugene Blikh 3 months ago
json: M.encode(desc, t, opts) for use_proto_names / emit_defaults / indent

Canonical protojson options were silently ignored. Plumb a single opts
table through encode_message via a CURRENT_ENCODE_OPTS state (mirroring
the decode-side CURRENT_OPTS):

  * use_proto_names         — emit snake_case field names (proto wire
                              names) instead of the spec-default
                              lowerCamelCase.
  * emit_defaults           — emit zero-valued implicit-presence scalars,
    (alias: always_emit_zero_value)  empty repeated lists, and empty
                              maps. Explicit-presence fields (optional /
                              oneof) and singular message fields remain
                              absent — matches protobuf-go's behavior.
  * indent                  — pretty-print with the given indent string;
                              empty arrays/objects stay compact.

Tests parameterize over both codegen modes (descriptor-table contract is
shared between full and runtime) plus a dedicated parity group that
asserts byte-identical JSON across modes for each option.
76140b72 — Eugene Blikh 3 months ago
json: emit unbroken base64 for bytes fields ({nowrap = true})

Canonical proto3 JSON expects RFC 4648 unwrapped base64, but Tarantool's
`digest.base64_encode` defaults to RFC 2045 MIME-style 76-char line
wrapping. Any `bytes` payload past ~57 bytes used to land in the JSON
string with an embedded `\n`, which breaks every spec-compliant
consumer (grpc-gateway, protojson, protobuf-go's JSON, ...).

Pass `{nowrap = true}` at the two encode sites: the per-field bytes
encoder and the `google.protobuf.Any` opaque-fallback `value` encoder.

Surfaced by tarantool-etcd's `TestJSONGatewayBytesUnwrapped` over Range
responses whose `value` is >=57 bytes; its reference grpc-gateway never
emits the wrapped form.
37a18dad — Eugene Blikh 3 months ago
json: strict validation pass closes the proto3 conformance suite

Six classes of relaxation that the proto3 JSON conformance corpus
flagged are now enforced. All as Recommended.* tests; combined with
the -0.0 codec fix this empties known_failures.txt and brings the
proto3 binary+JSON suite to 1493 ✓ / 1313 skipped / 0 expected
failures / 0 unexpected failures.

  1. Duplicate JSON keys. Tarantool's json.decode is hash-backed and
     silently collapses `{"foo":1,"foo":2}` to one entry. A small
     byte-walker `find_duplicate_json_keys` runs before json.decode,
     tracks per-object brace frames and key sets, errors on the
     second occurrence. Closes Recommended.FieldNameDuplicate.

  2. camelCase / snake_case aliases of the same proto field appearing
     side-by-side. Detected inside decode_message via a `field_seen`
     set keyed by proto-name; second hit errors. Closes
     FieldNameDuplicateDifferentCasing{1,2}.

  3. JSON null inside repeated arrays and map values. Previously
     silently dropped; now errors before decode_field_value. Closes
     RepeatedField{Message,Primitive}ElementIsNull and
     MapFieldValueIsNull.

  4. Unknown enum *names* (not integers). decode_enum used to return
     nil so callers silently dropped them; now raises by default and
     returns nil only when M.decode's `ignore_unknown_fields=true`
     opt is set. Conformance dispatch in cmd/conformance/core.lua
     forwards this flag when req.test_category ==
     JSON_IGNORE_UNKNOWN_PARSING_TEST. Closes
     RejectUnknownEnumStringValueIn{Optional,Repeated,Map} and the
     paired IgnoreUnknownEnumStringValueIn* tests.

  5. google.protobuf.NullValue JSON canonical form. The single enum
     value renders as the literal JSON `null` (not the string
     "NULL_VALUE"); decode accepts either, encode emits null. The
     decode_field_value null-handling path also treats a JSON null
     on a NullValue-typed field as "set" rather than "absent" so a
     oneof gets marked active. Closes
     NullValueInOtherOneof{New,Old}Format.Validator.

  6. FieldMask strict round-trip. Path validity is checked on both
     sides: the snake_case wire form rejects uppercase letters,
     consecutive underscores, trailing underscore, and underscore
     followed by anything other than a lowercase letter — these
     break the snake↔camel round-trip. The JSON form rejects any
     underscore in the input (must be lowerCamelCase). Closes
     FieldMask{TooManyUnderscore,PathsDontRoundTrip,
     NumbersDontRoundTrip}.JsonOutput and JsonInput.FieldMaskInvalidCharacter.

The pre-existing "drop unknown enum strings" unit regressions in
test/conformance_test.lua were inverted to assert the new error
shape. New strict-validation regressions in test/json_test.lua pin
all six categories so they don't regress; the `json.strict` group
runs across both codegen modes via the shared descriptor table.
784dea4d — Eugene Blikh 3 months ago
Initial commit: protoc-gen-tarantool plugin + pb runtime

A protoc plugin (Go) and a pure-Lua + LuaJIT-FFI runtime that give
Tarantool a complete proto3 + gRPC stack. Two codegen modes (full
inline / runtime descriptor), 226-test luatest suite, 18-fixture
mainline-protoc interop corpus, JSON codec, well-known types,
gRPC client/server factories, runtime .proto parser, microbench
harness with allocation regression gate.

Covers PLAN.md M1-M5. Module is `pb` (not `protobuf`) to avoid
colliding with Tarantool's built-in encode-only `protobuf` module.