How Serde's Serialization Model Works
Every Rust project that talks to an API, reads a config file, or persists state eventually reaches for serde, but most developers use it for months without understanding why it works the way it does.
Serde is not a JSON library, not a single format, and not a runtime reflection system - it is a pair of trait interfaces that any Rust type can implement and any data format can drive, and everything else in the ecosystem (serde_json, bincode, toml, serde_yaml) is a plugin that sits behind those traits.
Understanding this trait boundary is the single idea that makes the rest of serde's behavior predictable: what #[derive] actually generates, why switching formats needs zero changes to your struct, and where the real cost and limits of the system live.
Summary
- Serde separates "how to walk a Rust type" (
Serialize/Deserialize) from "how to encode/decode a wire format" (Serializer/Deserializer), and generates the first half at compile time. - Insight: Without this split, every crate that needed JSON, YAML, and bincode support would need three separate hand-written encoders per type, and adding a new format would mean touching every type in the ecosystem.
- Key Concepts: Serialize, Deserialize, Serializer, Deserializer, derive macro, visitor pattern, data model, self-describing format.
- When to Use: Any time a Rust value needs to cross a boundary - HTTP body, config file, database blob, IPC message, cache entry - and you want format flexibility without rewriting conversion code.
- Limitations/Trade-offs: Derived impls add compile time and binary size per type/format combination, and non-self-describing formats (like bincode) lose the flexibility that self-describing formats (like JSON) get for free.
- Related Topics: proc macros, trait objects vs generics, zero-copy parsing, schema evolution.
Foundations
At its core, serde defines two pairs of traits: Serialize/Deserialize for your data, and Serializer/Deserializer for the format.
A type that implements Serialize knows how to describe itself in terms of serde's abstract data model - a fixed set of primitives like structs, sequences, maps, and enums that every format must be able to represent in some way.
A Serializer is implemented once per wire format and knows how to turn those abstract primitives into concrete bytes, whether that means JSON text, a compact binary blob, or a YAML document.
The relationship is symmetric on the read side: a Deserialize implementation knows what shape of data it expects, and a Deserializer knows how to read that shape out of a specific format's bytes.
Crucially, your User struct's Serialize impl never mentions JSON, YAML, or any other format by name - it only calls generic trait methods, and the format is plugged in by whichever Serializer the caller passes in.
This is why adding serde = { features = ["derive"] } to a Cargo.toml and adding serde_json as a separate dependency works: one crate defines the traits, another crate implements a backend for them, and your derived code is the glue that runs generically over any backend.
The #[derive(Serialize, Deserialize)] macros are the mechanism that writes this glue automatically instead of asking you to hand-implement it for every struct and enum in your codebase.
Mechanics & Interactions
The most common misunderstanding about serde is imagining that #[derive(Serialize)] inspects your struct at runtime the way reflection-based serializers in other languages do.
In reality, the derive macro runs during compilation as a proc macro, reads your struct's field names and types from the token stream, and emits an ordinary impl Serialize for YourType block containing real, monomorphized Rust code.
That generated code is generic over S: Serializer, so the compiler produces a specialized version for every concrete Serializer type it is actually used with, and there is no dictionary lookup or type inspection happening when your program runs.
On the deserialize side the mechanism is a bit more involved because a deserializer cannot simply hand back a value the way a serializer can push bytes forward - it needs somewhere to put the data as it reads it, and that somewhere is supplied by the visitor pattern.
Your derived Deserialize impl builds a Visitor whose methods (visit_map, visit_seq, visit_str, and so on) know how to assemble a YourType from pieces, and then it hands that visitor to the format's Deserializer, which calls back into the visitor's methods as it walks its own input.
This inversion of control is what lets one Deserializer implementation serve every possible target type: the deserializer never needs to know about YourType in advance, it just drives whatever visitor it is given.
// Conceptual shape of what #[derive(Serialize)] generates (simplified)
impl Serialize for User {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer, // generic over the format - JSON, bincode, YAML, anything
{
let mut state = serializer.serialize_struct("User", 2)?;
state.serialize_field("id", &self.id)?;
state.serialize_field("name", &self.name)?;
state.end()
}
}The snippet above is real (simplified) shape of generated code, and the key detail is the S: Serializer bound: this same function body compiles to a specialized, inlined path for every format backend it is used with, so there is no runtime branch on "which format am I."
Field order, presence checks, and struct naming are decided once, at compile time, based on your struct definition and any #[serde(...)] attributes you added.
Advanced Considerations & Applications
The payoff of this design shows up clearest when a project needs to support more than one wire format for the same types, which is common once a service grows beyond a single HTTP+JSON boundary.
A struct derived once can serialize to serde_json for an API response, to bincode for an internal cache, and to serde_yaml for a config file, without a single line of format-specific code in the struct itself - only the Serializer/Deserializer backend changes at the call site.
This flexibility is not free, and the cost surfaces in two places: compile time and format capability mismatches.
Every combination of derived type and format backend that your program actually exercises gets its own monomorphized code path, so large codebases with many types and many formats can see real increases in build time and binary size.
The second cost is more subtle: serde's data model assumes formats can represent structs, maps, sequences, and enums, but not every format is self-describing in the same way JSON is, where type and field names travel with the data.
Formats like bincode are non-self-describing - they rely entirely on both sides agreeing on struct shape and field order ahead of time, which is fast and compact but means schema changes (renaming, reordering, or type changes) can silently corrupt data instead of failing with a helpful error.
This is also where hand-written impls (implementing Serialize/Deserialize manually instead of deriving) still earn their place: when the wire shape genuinely diverges from your Rust type's shape, no derive attribute combination will bridge that gap cleanly.
| Approach | Strength | Weakness | Best Fit |
|---|---|---|---|
#[derive(Serialize, Deserialize)] | Zero boilerplate, compiler-checked, works across all formats | Wire shape must roughly match Rust struct shape | The vast majority of structs and enums |
Derive + #[serde(...)] attributes | Covers renaming, defaults, flattening, skipping | Attribute combinations can get dense on complex types | APIs with naming mismatches or optional fields |
Manual Serialize/Deserialize impl | Full control over wire representation | Hand-written visitor code, more surface for bugs | Wire formats that structurally diverge from the type |
serde_json::Value / dynamic types | No struct needed, handles arbitrary shapes | Loses compile-time field checking | Truly dynamic or unknown-shape JSON |
Common Misconceptions
- "Serde uses reflection like Java or Python serializers do." - It does not;
#[derive]generates concrete trait implementations at compile time, and there is zero runtime type inspection, which is exactly why serde has no measurable "reflection tax." - "Serde is a JSON library." - Serde itself defines only the traits and the data model;
serde_jsonis a separate crate that implements aSerializer/Deserializerbackend, and dozens of other crates implement backends for other formats using the same traits. - "Generic derived code must be slower than a hand-written parser." - The
S: Serializergeneric bound gets monomorphized per format, so the compiler produces a specialized, often fully inlined code path with no dynamic dispatch in the common case. - "Serializer and Deserializer are data formats." - They are traits; JSON, bincode, and YAML are concrete implementations of those traits provided by separate crates, not part of serde's core.
- "Any type that works with JSON will work with any other format serde supports." - Non-self-describing binary formats depend on both sides agreeing on exact struct shape, so changes safe in JSON (like field reordering under most encodings) can break bincode compatibility silently.
FAQs
What problem does serde actually solve?
It removes the need to hand-write a separate encoder/decoder pair for every combination of Rust type and wire format by putting a stable trait boundary between the two, so format crates and derived type impls can be developed independently and still work together.
Why does serde use two crates - serde and serde_json - instead of one?
serdedefines only theSerialize/Deserialize/Serializer/Deserializertraits and the abstract data model.serde_json(andbincode,toml,serde_yaml, etc.) implement the format-specificSerializer/Deserializerside.- This split means adding a new format never requires changes to serde itself or to any type that already derives
Serialize/Deserialize.
How does #[derive(Serialize)] actually generate code?
It runs as a procedural macro during compilation, reads your struct or enum definition from the token stream, and emits a real impl Serialize for YourType block that calls generic Serializer trait methods - you can see the expanded output yourself with cargo expand.
How does deserialization work without knowing the target type in advance?
The deserializer is handed a Visitor (built by your derived Deserialize impl) whose callback methods know how to assemble the target type, and the deserializer drives those callbacks as it walks its own input - this inversion of control is the visitor pattern, and it is what lets one Deserializer implementation serve arbitrarily many target types.
Is there a performance cost to using derive instead of a hand-written parser?
Generated code is monomorphized per Serializer/Deserializer type, so the compiler typically inlines and specializes it the same way it would hand-written generic code; the more realistic costs are compile time and binary size, not runtime overhead.
When should I write a manual Serialize/Deserialize impl instead of deriving?
Reach for a manual impl when the wire representation structurally diverges from your Rust type - for example encoding an enum as a single scalar value, or parsing a format where one JSON key maps to multiple struct fields - because no #[serde(...)] attribute combination can bridge a genuine shape mismatch.
Why does switching from serde_json to bincode sometimes break things that worked fine before?
JSON is self-describing (field names and types travel with the data), so it tolerates certain schema drift gracefully, while bincode is non-self-describing and depends entirely on both sides agreeing on exact struct shape and field order, so a change that is harmless in JSON can desync bincode readers and writers.
Does adding serde derive to every type in a large codebase have a real cost?
Yes - every distinct combination of derived type and Serializer/Deserializer backend that gets used produces its own monomorphized code, so codebases with many types across many formats can see meaningfully longer compile times and larger binaries, which is a real trade-off against the convenience.
What exactly is serde's "data model"?
It is the fixed set of abstract shapes - things like struct, sequence, map, enum variant, and the primitive scalar types - that any Serialize implementation describes itself in terms of, and that any Serializer implementation must know how to render; it is the contract that makes format-independence possible.
Can a single struct support formats that represent enums very differently?
Usually yes, because the Serialize/Deserialize side only calls the abstract "this is an enum variant" methods and leaves the concrete tagging strategy (internally tagged, externally tagged, untagged) up to #[serde(tag = ...)] attributes and the format backend, though some combinations of representation and format do not round-trip cleanly and should be tested explicitly.
Isn't the visitor pattern just an implementation detail I can ignore?
You can ignore it for derived types, but it becomes directly relevant the moment you write a manual Deserialize impl or debug a confusing deserialization error, since serde's error messages and custom deserializer code are both expressed in visitor terms.
Does serde support serializing trait objects or dynamic types?
Not directly through #[derive], since the macro needs a concrete, known-at-compile-time shape; dynamic or arbitrary-shape data is better handled with serde_json::Value (or the equivalent dynamic value type in other format crates) at the boundary, then converted into concrete types afterward.
Related
- serde Basics - hands-on walkthrough of derive with 9 worked examples
- Custom (De)serialization - writing manual Serialize/Deserialize impls by hand
- Other Formats - the same derived impls used across TOML, YAML, MessagePack, and bincode
- Zero-Copy Deserialization - how the visitor pattern enables borrowing instead of allocating
- serde Best Practices - compatibility and schema-evolution guidance built on this model
Stack versions: This page was written for Rust 1.97.0 (edition 2024), Tokio 1.x, Axum 0.8, serde 1.0, sqlx 0.8, clap 4, and Polars 0.46+.