Pipes & Text Processing

Compose CLI tools to analyze build output, JSON logs, benchmark CSVs, and git stats without opening a spreadsheet.

Recipe

cargo test 2>&1 | tee test.log
rg "FAILED" test.log | wc -l
cargo tree -e no-dev | sort | uniq -c | sort -rn | head

When to reach for this:

• Summarizing test failures from CI download
• Finding duplicate dependencies in cargo tree
• Quick log filtering during incident

Working Example

# Slowest tests from nextest output (example pattern)
cargo nextest run 2>&1 | rg "PASS|FAIL" | sort | uniq -c
 
# JSON log lines (tracing JSON subscriber)
rg '"level":"ERROR"' /var/log/my-service.log | jq -r '.message'
 
# Top 10 largest files in src
fd . src -t f | xargs wc -l | sort -n | tail -10
 
# Authors by commit count last month
git log --since="1 month ago" --format='%an' | sort | uniq -c | sort -rn | head

What this demonstrates:

• tee preserves output while saving file
• jq parses structured Rust logs
• Standard Unix sort/uniq patterns

Deep Dive

Pipeline Patterns

# Extract crate names from clippy warnings
cargo clippy 2>&1 | rg 'warning:.*\((.*)\)' -r '$1' | sort -u
 
# Filter benchmark CSV
awk -F, '$3 > 100 {print $1,$3}' bench.csv

Rust-Specific

Use cargo metadata --format-version=1 | jq for scripting workspace graphs.

Gotchas

• Broken pipe errors - benign when head closes early. Fix: ignore or | head || true.
• Whitespace in filenames - xargs splits wrong. Fix: fd -0 | xargs -0.
• jq on non-JSON lines - failures. Fix: filter with rg first.
• Locale sort surprises - odd ordering. Fix: LC_ALL=C sort.
• Huge log files in memory - cat overload. Fix: stream with rg or less.

Alternatives

FAQs

Related

• Search & regex
• Process & resource management
• Observability
• Linux CLI Best Practices

Stack versions: This page was written for Rust 1.97.0 (edition 2024), Tokio 1.x, Axum 0.8, serde 1.0, sqlx 0.8, clap 4, and Polars 0.46+.