Repository navigation
docs: pipeline: outputs: kafka: document Protobuf serialization - #2717
Conversation
- Add `protobuf` to the `format` row's list of available formats.
- Add the `protobuf_message` row to the Configuration parameters
table, between `otlp_logs_partition_by_resource` and
`queue_full_retries`.
- Rescope `schema_id` and `schema_registry_subject` from Avro-only to
Avro or Protobuf.
- Rewrite the `schema_str` row: it is the inline Avro alternative to a
registry and is not supported with `format: protobuf`.
- Add a Protobuf support section covering the
`-DFLB_PROTOBUF_ENCODER=On` build option and its Protobuf 3.12
requirement, the mandatory `schema_registry_url`, recursive
`references` resolution and its 64-file / 4 MiB limits,
`protobuf_message` selection rules, the JSON-then-encode record
path, Confluent wire framing, and YAML and classic examples.
- Promote "Resolve schemas from a registry" from h4 to h3 and rewrite
its intro: the registry is enabled by either FLB_AVRO_ENCODER or
FLB_PROTOBUF_ENCODER, not by Avro support alone.
- Promote "Kafka configuration file with raw format" to h3 so it stays
a sibling rather than nesting under the registry section, and add
the missing space in `raw`format.
- Fix the MSK build requirements table delimiter row, which tripped
markdownlint MD060 four times.
Signed-off-by: Eric D. Schabell <eric@schabell.org>
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review. 📝 WalkthroughWalkthroughThe Kafka output documentation adds Protobuf format and message configuration, build requirements, schema registry behavior, encoding details, and examples. It also updates Avro support descriptions and adjusts section headings and table formatting. ChangesKafka Protobuf documentation
Priority: ⬇️ Low Estimated code review effort: 2 (Simple) | ~10 minutes Change: Other Suggested reviewers: Merge Risk: ⚪ Minimal · up to No concrete issue was identified that would block merging the Kafka Protobuf documentation. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@patrick-stephens ready for review! |
…elease builds
- Avro support: state it isn't in any official release packages or
container images and needs a source build (was "isn't enabled by
default for releases")
- Protobuf support: same rewording
- Registry schema resolution: "neither of which is enabled by default"
becomes "neither of which is included in official release builds"
Signed-off-by: Eric D. Schabell <eric@schabell.org>
protobufto theformatrow's list of available formats.protobuf_messagerow to the Configuration parameterstable, between
otlp_logs_partition_by_resourceandqueue_full_retries.schema_idandschema_registry_subjectfrom Avro-only toAvro or Protobuf.
schema_strrow: it is the inline Avro alternative to aregistry and is not supported with
format: protobuf.-DFLB_PROTOBUF_ENCODER=Onbuild option and its Protobuf 3.12requirement, the mandatory
schema_registry_url, recursivereferencesresolution and its 64-file / 4 MiB limits,protobuf_messageselection rules, the JSON-then-encode recordpath, Confluent wire framing, and YAML and classic examples.
its intro: the registry is enabled by either FLB_AVRO_ENCODER or
FLB_PROTOBUF_ENCODER, not by Avro support alone.
a sibling rather than nesting under the registry section, and add
the missing space in
rawformat.markdownlint MD060 four times.
Signed-off-by: Eric D. Schabell eric@schabell.org
Summary by CodeRabbit
protobuf_messageoption.schema_stris unsupported with this format.