Networking Best Practices
Rules for reliable outbound calls, secure transport, and operable network behavior in Rust services.
How to Use This List
- Apply to every new external dependency integration.
- Encode retry and timeout policy in shared client factory.
- Review during incidents involving upstream outages.
A - HTTP Clients
- One shared
reqwest::Clientper process. Connection pool reuse. - Set connect and total timeouts on every client. No infinite waits.
- Identify service with
User-Agent. Ops can trace callers in logs. - Map upstream errors to domain errors. Do not leak raw bodies to end users.
- Stream large downloads. Avoid loading multi-GB responses into RAM.
B - Retries & Resilience
- Retry only idempotent operations. POST needs
Idempotency-Key. - Exponential backoff with jitter. Cap max delay and attempt count.
- Honor
Retry-Afteron 429. Respect upstream rate limits. - Circuit break sustained failures. Fail fast when dependency is down.
- Bulkhead concurrent calls per host. Prevent one slow API stalling all tasks.
C - TLS & Security
- Use rustls in Rust binaries. Avoid shipping OpenSSL unless required.
- Never disable cert verification in production. Custom roots for corporate CA only.
- Rotate TLS certs before expiry. Automate with cert-manager or LB.
- mTLS for service-to-service when policy requires. Short-lived client certs.
- Do not log Authorization headers or cookies.
D - Protocol Choice
- REST/JSON for public north-south APIs. Broad client support.
- gRPC for internal east-west RPC. Strong contracts and streaming.
- WebSocket only when server push or bidirectional needed. Not for simple polling.
- Raw TCP/UDP only with documented framing. Prefer HTTP unless latency demands otherwise.
E - Observability
- Propagate trace context (
traceparent). Link outbound spans to inbound request. - Metric: latency, error rate, retry count per dependency. Dashboard per upstream.
- DNS failures alert separately. Distinct from HTTP 5xx.
- Document upstream SLAs in runbook. Timeout values derived from SLA.
- Load test with fault injection. Verify backoff does not amplify outages.
FAQs
reqwest or hyper?
reqwest for application HTTP; hyper for custom stacks.
Default timeout?
10-30s total, 3-5s connect - tune per dependency.
gRPC vs REST?
gRPC internal; REST external unless API gateway translates.
IPv6?
Test dual-stack in staging; happy eyeballs handled by OS/runtime.
Proxy all egress?
Corporate environments - configure HTTP_PROXY on client builder.
HTTP/2 to upstream?
Enabled by default; verify server supports before forcing.
Keep-alive idle timeout?
Align client pool idle with LB idle timeout to avoid stale sockets.
Webhook verification?
HMAC signature + timestamp tolerance on inbound, not outbound.
Chaos testing?
toxiproxy or mesh fault injection in staging.
Single biggest mistake?
No timeouts on outbound HTTP blocking Tokio workers under upstream stalls.
Related
- HTTP Clients (reqwest) - Client setup
- Retries, Timeouts & Backoff - Policies
- TLS with rustls - Encryption
- Web Backends Best Practices - Server side
- APIs & Security Best Practices - Auth baseline
Stack versions: This page was written for Rust 1.97.0 (edition 2024), Tokio 1.x, Axum 0.8, serde 1.0, sqlx 0.8, clap 4, and Polars 0.46+.