Reference: A High-Throughput Axum Service

A reference layout for a latency-sensitive HTTP API: Axum 0.8, Tokio 1.x, sqlx 0.8, serde 1.0, and observability from day one - targeting thousands of RPS per pod with predictable pool behavior and safe rollouts.

Recipe

Quick-reference recipe card - copy-paste ready.

orders-api/
  crates/
    orders-domain/     # pure logic, no axum/sqlx in public API
    orders-infra/      # sqlx, redis clients
    orders-api/        # binary: routes, middleware, main
  migrations/
  deploy/
    Dockerfile         # rust:1.97.0, edition 2024
    helm/

When to reach for this:

• Greenfield high-RPS service
• Teaching squad the fleet HTTP standard
• RFC baseline for new API ADR
• Performance regression comparison

Working Example

// crates/orders-api/src/main.rs
use axum::{routing::get, Router};
use sqlx::postgres::PgPoolOptions;
use std::net::SocketAddr;
use tower_http::trace::TraceLayer;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};
 
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    tracing_subscriber::registry()
        .with(tracing_subscriber::EnvFilter::from_default_env())
        .with(tracing_subscriber::fmt::layer().json())
        .init();
 
    let pool = PgPoolOptions::new()
        .max_connections(20)
        .connect(&std::env::var("DATABASE_URL")?)
        .await?;
 
    let app = Router::new()
        .route("/health/ready", get(|| async { "ok" }))
        .route("/v1/orders/:id", get(orders_infra::get_order))
        .layer(TraceLayer::new_for_http())
        .with_state(pool);
 
    let addr: SocketAddr = "0.0.0.0:8080".parse()?;
    let listener = tokio::net::TcpListener::bind(addr).await?;
    axum::serve(listener, app)
        .with_graceful_shutdown(shutdown_signal())
        .await?;
    Ok(())
}
 
async fn shutdown_signal() {
    tokio::signal::ctrl_c().await.ok();
}

# Deploy notes
docker build -t orders-api:sha-$(git rev-parse --short HEAD) .
# Canary 10% with p95 < 120ms guardrail; pool_acquire_p95 < 30ms

Deep Dive

Architecture Choices

• Crate split - Domain logic testable without HTTP server.
• Connection pool - Size from load test: (core_count * 2) + spare upper bound, capped by Postgres max_connections.
• Middleware - TraceLayer for request IDs; compression only if measured win.
• Shutdown - with_graceful_shutdown plus K8s preStop sleep for LB drain.

Performance Checklist

• Avoid per-request allocations in hot path (Bytes, reuse buffers).
• spawn_blocking for CPU-heavy work only.
• Index queries validated with EXPLAIN in staging load test.

Operations

• Readiness: DB ping + optional Redis.
• Liveness: process up only (avoid DB flap kills).
• Dashboards: RPS, p95, pool acquire, panic counter, deploy SHA annotation.

Gotchas

• Pool per worker misconception - sqlx pool is shared across tasks on process. Fix: One pool per process, tune max_connections.
• Debug build in prod - Throughput collapse. Fix: cargo build --release with LTO profile considered.
• No graceful shutdown - 502 during rollouts. Fix: Drain + preStop.
• Giant JSON responses - Memory spikes. Fix: Pagination and streaming where needed.

Alternatives

FAQs

Related

• Axum Routing - handlers
• Connection Pooling - pool tuning
• Graceful Shutdown - deploy safety
• Latency & Throughput Benchmarks - measurement
• Delivery Best Practices - release

Stack versions: This page was written for Rust 1.97.0 (edition 2024), Tokio 1.x, Axum 0.8, serde 1.0, sqlx 0.8, clap 4, and Polars 0.46+.