buffa

A pure-Rust Protocol Buffers implementation with first-class protobuf editions support. Written by Claude ❣️

Why buffa?

The Rust ecosystem lacks an actively maintained, pure-Rust library that supports protobuf editions. Buffa fills that gap with a ground-up design that treats editions as the core abstraction. It passes all current binary and JSON protobuf serialization conformance tests.

Features

• Editions-first. Proto2 and proto3 are understood as feature presets within the editions model. One code path, parameterized by resolved features.

• Two-tier owned/borrowed types. Each message generates both MyMessage (owned, heap-allocated) and MyMessageView<'a> (zero-copy from the wire). OwnedView<V> wraps a view with its backing Bytes buffer for use across async boundaries.

• MessageField<T>. Optional message fields deref to a default instance when unset -- no Option<Box<T>> unwrapping ceremony.

• EnumValue<T>. Type-safe open enums with proper Rust enum types and preservation of unknown values, instead of raw i32.

• Linear-time serialization. Cached encoded sizes prevent the exponential blowup that affects libraries without a size-caching pass.

• Unknown field preservation. Round-trip fidelity for proxy and middleware use cases.

• no_std + alloc. The core runtime works without std, including JSON serialization via serde. Enabling std adds std::io integration, std::time conversions, and thread-local JSON parse options.

Wire formats

buffa supports binary and JSON protobuf encodings:

• Binary wire format -- full support for all scalar types, nested messages, repeated/packed fields, maps, oneofs, groups, and unknown fields.

• Proto3 JSON -- canonical protobuf JSON mapping via optional serde integration. Includes well-known type serialization (Timestamp as RFC 3339, Duration as "1.5s", int64/uint64 as quoted strings, bytes as base64, etc.).

Text format (textproto) is not supported and is not planned.

Unsupported features

These are intentionally out of scope:

• Text format (textproto) — not planned. Binary and JSON are the wire formats that matter for RPC and storage.
• Runtime reflection (DynamicMessage, descriptor-driven introspection) — not planned for 0.1. Buffa is a codegen-first library; if you need schema-agnostic processing, consider preserving unknown fields or using Any.
• Proto2 optional-field getter methods — [default = X] on optional fields does not generate fn field_name(&self) -> T unwrap-to-default accessors. Custom defaults are applied only to required fields via impl Default. Optional fields are Option<T>; use pattern matching or .unwrap_or(X).
• Scoped JsonParseOptions in no_std — serde's Deserialize trait has no context parameter, so runtime options must be passed through ambient state. In std builds, with_json_parse_options provides per-closure, per-thread scoping via a thread-local. In no_std builds, set_global_json_parse_options provides process-wide set-once configuration via a global atomic. The two APIs are mutually exclusive. The no_std global supports singular-enum accept-with-default but not repeated/map container filtering (which requires scoped strict-mode override).

Known limitations

These are gaps we intend to address in future releases:

• Closed-enum unknown values in packed-repeated view decode are silently dropped (not routed to unknown fields). The owned decoder handles this correctly; the view decoder handles singular, optional, oneof, and unpacked repeated correctly. Packed blobs have no per-element tag to borrow, so the zero-copy UnknownFieldsView<'a> has no span to reference.
• Closed-enum unknown values in map values are silently dropped (not routed to unknown fields). The proto spec requires the entire map entry (key + value) to go to unknown fields, which requires re-encoding. This affects proto2 schemas with map<K, ClosedEnum> where an evolved sender adds new enum values.

Semver and API stability

Buffa is pre-1.0. We follow the Rust community convention for 0.x crates: breaking changes increment the minor version (0.1.x → 0.2.0), additive changes increment the patch version (0.1.0 → 0.1.1). Pin to a minor version (buffa = "0.1") to avoid surprises.

The generated code API (struct shapes, Message trait, MessageView trait, EnumValue, MessageField) is considered the primary stability surface. Internal helper modules marked #[doc(hidden)] (__private, __buffa_* fields) may change at any time.

Quick start

Using buf generate (recommended)

Install buf and the protoc plugins, then create a buf.gen.yaml:

version: v2
plugins:
  - local: protoc-gen-buffa
    out: src/gen
  - local: protoc-gen-buffa-packaging
    out: src/gen
    strategy: all

buf generate

Using buffa-build in build.rs

Alternatively, use buffa-build for a build.rs-based workflow (requires protoc on PATH):

// build.rs
fn main() {
    buffa_build::Config::new()
        .files(&["proto/my_service.proto"])
        .includes(&["proto/"])
        .compile()
        .unwrap();
}

Encoding and decoding

use buffa::Message;

// Encode
let msg = MyMessage { id: 42, name: "hello".into(), ..Default::default() };
let bytes = msg.encode_to_vec();

// Decode (owned)
let decoded = MyMessage::decode_from_slice(&bytes).unwrap();

// Decode (zero-copy view)
let view = MyMessageView::decode_view(&bytes).unwrap();
println!("name: {}", view.name); // &str, no allocation

// Decode (owned view — zero-copy + 'static, for async/RPC use)
let owned_view = OwnedView::<MyMessageView>::decode(bytes.into()).unwrap();
println!("name: {}", owned_view.name); // still zero-copy, but 'static + Send

JSON serialization (with json feature)

let json = serde_json::to_string(&msg).unwrap();
let decoded: MyMessage = serde_json::from_str(&json).unwrap();

Documentation

• User Guide — comprehensive guide to buffa's API, generated code shape, encoding/decoding, views, JSON, well-known types, and editions support.
• Migrating from prost — step-by-step migration guide with before/after code examples.
• Migrating from protobuf — migration guide covering both stepancheg v3 and Google official v4.

Workspace layout

Crate	Purpose
buffa	Core runtime: Message trait, wire format codec, no_std support
buffa-types	Well-known types: Timestamp, Duration, Any, Struct, wrappers, etc.
buffa-descriptor	Protobuf descriptor types (FileDescriptorProto, DescriptorProto, ...)
buffa-codegen	Code generation from protobuf descriptors
buffa-build	build.rs helper for invoking codegen via protoc
protoc-gen-buffa	protoc plugin binary

Performance

Throughput comparison across four representative message types, measured on an Intel Xeon Platinum 8488C (x86_64). Cross-implementation benchmarks run in Docker for toolchain consistency (task bench-cross). Higher is better.

Binary decode

Raw data (MiB/s)

Message	buffa	buffa (view)	prost	protobuf‑v4	Go
ApiResponse	762	1,245 (+63%)	777 (+2%)	720 (−5%)	277 (−64%)
LogRecord	689	1,772 (+157%)	692 (+0%)	882 (+28%)	251 (−64%)
AnalyticsEvent	188	307 (+63%)	258 (+37%)	364 (+93%)	92 (−51%)
GoogleMessage1	801	1,093 (+36%)	1,001 (+25%)	659 (−18%)	351 (−56%)

Binary encode

Raw data (MiB/s)

Message	buffa	prost	protobuf‑v4	Go
ApiResponse	2,637	1,755 (−33%)	1,050 (−60%)	570 (−78%)
LogRecord	4,149	3,163 (−24%)	1,717 (−59%)	309 (−93%)
AnalyticsEvent	671	369 (−45%)	516 (−23%)	162 (−76%)
GoogleMessage1	2,543	1,866 (−27%)	882 (−65%)	366 (−86%)

JSON encode

Raw data (MiB/s)

Message	buffa	prost	Go
ApiResponse	869	776 (−11%)	119 (−86%)
LogRecord	1,335	1,099 (−18%)	144 (−89%)
AnalyticsEvent	781	768 (−2%)	52 (−93%)
GoogleMessage1	1,047	840 (−20%)	129 (−88%)

JSON decode

Raw data (MiB/s)

Message	buffa	prost	Go
ApiResponse	721	299 (−59%)	71 (−90%)
LogRecord	780	694 (−11%)	112 (−86%)
AnalyticsEvent	272	239 (−12%)	47 (−83%)
GoogleMessage1	635	253 (−60%)	74 (−88%)

Message types: ApiResponse (~200 B, flat scalars), LogRecord (~1 KB, strings + map + nested message), AnalyticsEvent (~10 KB, deeply nested + repeated sub-messages), GoogleMessage1 (standard protobuf benchmark message).

Libraries: prost 0.13 + pbjson 0.7, protobuf‑v4 (Google Rust/upb, v4.33.1), Go google.golang.org/protobuf v1.36.6. protobuf-v4 JSON is not included as it does not provide a JSON codec.

Owned decode trade-offs: buffa's owned decode is typically within ±10% of prost, trading a small throughput cost for features prost omits: unknown-field preservation by default, typed EnumValue<E> wrappers (not raw i32), and a type-stable decode loop that supports recursive message types without manual boxing. The zero-copy view path (MyMessageView::decode_view) sidesteps allocation entirely and is the recommended fast decode path. protobuf-v4's decode advantage on deeply-nested messages comes from upb's arena allocator — all sub-messages are bump-allocated in one arena rather than individually boxed.

Conformance

buffa passes the protobuf binary and JSON conformance test suite (v33.5, editions up to 2024). Both std and no_std builds pass the full suite including JSON. Run with task conformance.

Compiler compatibility

buf is the recommended way to compile .proto files. The buf CLI has its own built-in compiler, so no separate protoc install is needed — just install buf and protoc-gen-buffa.

protoc is also fully supported. protoc-gen-buffa and buffa-build work with protoc v21.12 and later. The minimum version varies by feature:

Feature	Minimum protoc
Proto2 + proto3	v21.12
Editions 2023	v27.0
Editions 2024	v33.0

Note that Linux distro packages (Debian Bookworm, Ubuntu 24.04) ship protoc v21.12, which does not support editions. Install protoc v27+ from GitHub releases or use buf if you need editions support.

Compatibility is tested against protoc v21.12, v22.5, v25.5, v27.3, v29.5, and v33.5 (task protoc-compat).

Minimum supported Rust version

1.85

License

Apache-2.0