Polars

The fast DataFrame library for Rust - lazy query optimization, vectorized execution, and Arrow-backed memory layout.

Recipe

Quick-reference recipe card - copy-paste ready.

use polars::prelude::*;
 
fn top_spenders(path: &str) -> PolarsResult<DataFrame> {
    LazyCsvReader::new(path.into())
        .with_has_header(true)
        .finish()?
        .group_by([col("customer_id")])
        .agg([col("amount").sum().alias("total")])
        .sort(["total"], SortMultipleOptions::default())
        .limit(10)
        .collect()
}

When to reach for this:

• Replacing pandas scripts that bottleneck on single-threaded Python loops
• ETL where you need lazy predicate pushdown on CSV or Parquet
• In-process analytics without standing up Spark or DuckDB as a service
• Rust services that already use Arrow and need a DataFrame API

Working Example

use polars::prelude::*;
 
fn main() -> PolarsResult<()> {
    let df = df! {
        "region" => ["west", "east", "west", "north"],
        "amount" => [120i64, 80, 200, 50],
    }?;
 
    let summary = df
        .lazy()
        .filter(col("amount").gt(lit(60)))
        .group_by([col("region")])
        .agg([
            col("amount").sum().alias("total"),
            col("amount").mean().alias("avg"),
        ])
        .sort(["total"], SortMultipleOptions::default())
        .collect()?;
 
    println!("{}", summary);
    Ok(())
}

What this demonstrates:

• Building a small frame in memory for tests and prototypes
• Switching to lazy mode with .lazy() for optimized execution
• Group-by aggregations with multiple expressions in one plan

Deep Dive

How It Works

• Polars stores columns in Arrow-compatible buffers for cache-friendly scans.
• Lazy API builds a logical plan; the optimizer reorders filters and projections.
• collect() executes the plan - that is your memory and CPU spike boundary.
• Eager DataFrame methods run immediately - simpler mental model, fewer optimizations.

Lazy vs Eager

Rust Notes

// Prefer explicit aliases in agg - unnamed exprs are hard to spot in explain plans.
.col("amount").sum().alias("total")
 
// Use .explain() on lazy plans during development.
println!("{}", lf.explain(true)?);

Gotchas

• Collecting too early - calling collect() after every step defeats lazy optimization. Fix: chain transforms, collect once at the end.
• String-typed numeric columns - CSV inference defaults to Utf8. Fix: pass with_dtypes or cast after load.
• Join key dtype mismatch - i32 vs i64 keys silently drop rows. Fix: cast both sides to the same type before join.
• Assuming pandas index semantics - Polars has no hidden index; joins use explicit keys. Fix: add an id column when you need stable row identity.
• OOM on wide Parquet - select * on 500 columns loads everything. Fix: project only needed columns in the lazy scan.

Alternatives

FAQs

Related

• Data Basics - section orientation
• Apache Arrow - columnar memory under Polars
• CSV/Parquet I/O - I/O tuning
• Streaming & Large Data - chunked processing

Stack versions: This page was written for Rust 1.97.0 (edition 2024), Tokio 1.x, Axum 0.8, serde 1.0, sqlx 0.8, clap 4, and Polars 0.46+.