Intelligent Model Routing: Why Tokenmaxxing Is a Routing Problem, Not a Rationing Problem

The Concept

Most teams pick their LLM model tier once and never revisit. Someone reads a blog post, runs a quick benchmark, picks a model, and that choice calcifies into the codebase forever. Three months later, pricing has changed, new models have launched, and the team is overpaying by 40% without knowing it.

Tokenmaxxing, the practice of optimizing token spend, is widely misunderstood as "use the cheapest model for everything." That framing misses the point entirely. The real insight is that cost optimization is a routing problem, not a rationing problem.

Think of it like shipping logistics. FedEx doesn't send every package by overnight express, and they don't send everything by ground freight. They route each package based on urgency, weight, and destination. A shipping company that uses one rate for everything is either overpaying on simple deliveries or underdelivering on urgent ones. LLM calls work the same way: a classification task needs a different level of intelligence than a creative writing task, and paying the same rate for both is leaving money on the table.

The key shift is moving from static model selection ("we use Model X") to dynamic model routing ("each task gets the cheapest model that can handle it reliably").

When to apply this: Any system making more than a few hundred LLM calls per day across mixed task types. If every call goes to the same model, you're overspending on simple tasks or underpowering complex ones. The bigger your call volume, the more routing saves.

Common pitfalls:

Benchmarking once. Model pricing and capabilities change quarterly. A routing decision made 6 months ago based on a blog post is almost certainly stale. Your system needs to re-evaluate continuously.
Routing by model name. "Use Haiku for cheap tasks" ignores that a different provider's model may deliver the same quality at half the price. Route by capability tier, not brand name.
No feedback loop. Routing without governance means you never learn whether your tier assignments are actually optimal. The governance report should feed directly back into routing decisions, creating a closed loop that self-optimizes over time.

Quick Win: Claude Skill

Here's a prompt you can use right now to analyze your current LLM spend and get a model routing recommendation:

You are an LLM cost optimization engineer. I will provide a log of recent LLM API calls with the model used, task type, and cost.

Analyze the log and produce a Model Routing Recommendation with these sections:

1. **Current Spend by Tier**: Group tasks by complexity (trivial, standard, complex, creative) and show total spend per group
2. **Routing Table**: For each task type, recommend the optimal model tier and specific model. Format as a table with columns: Task Type | Current Model | Recommended Model | Current Cost/Call | Projected Cost/Call | Confidence
3. **Savings Estimate**: Total projected monthly savings if all recommendations are adopted
4. **Migration Priority**: Rank the top 3 task types to migrate first, ordered by (savings * confidence)
5. **Risk Flags**: Any task types where downgrading the model might hurt quality (low confidence recommendations)

Be specific about model names and pricing. Use current market rates.

Here is my usage log:
[PASTE YOUR LOG HERE - format: timestamp, model, task_type, input_tokens, output_tokens, cost_usd]

Example input (paste after the prompt):

2026-05-20T08:00:00Z, claude-sonnet-4, intent_classification, 800, 200, 0.006
2026-05-20T08:01:00Z, claude-sonnet-4, triage_summary, 2000, 1000, 0.018
2026-05-20T08:05:00Z, claude-sonnet-4, meeting_brief, 4500, 2500, 0.042
2026-05-20T08:10:00Z, claude-sonnet-4, code_generation, 8000, 4000, 0.072
2026-05-20T08:15:00Z, claude-sonnet-4, status_check, 300, 100, 0.002
2026-05-20T08:20:00Z, claude-sonnet-4, post_draft, 3000, 2000, 0.030
2026-05-20T08:25:00Z, claude-sonnet-4, auto_advance, 500, 150, 0.004
2026-05-20T08:30:00Z, claude-sonnet-4, spec_generation, 5000, 3000, 0.048

What you'll get back: A routing table showing that intent_classification, status_check, and auto_advance should move to a nano-tier model (saving ~95% on those calls), that triage_summary and post_draft should move to a fast-tier model (saving ~60%), and that only code_generation and spec_generation justify the premium model. Expected monthly savings: 40-65% depending on call volume.

Iteration tips:

Add your actual monthly call volumes per task type to get dollar-amount savings instead of percentages.
Run this monthly after pricing changes or new model releases to catch re-routing opportunities your competitors will miss.

Full System Specification

Problem Statement

Engineering teams using LLMs in production lack automated, data-driven model routing. Most default to a single model for all tasks, resulting in 40-70% overspend on simple workloads. Manual model selection is based on intuition rather than measured performance, and routing decisions are never revisited as pricing and model capabilities evolve.

Architecture

Four components, each independently deployable:

Intent Classifier -- A lightweight pre-router that analyzes each incoming request and assigns a complexity tier (trivial, standard, analytical, creative). Uses the cheapest available model (or a heuristic) since classification itself should cost nearly nothing. Tags each request with the assigned tier before forwarding.
Model Router -- Maps complexity tiers to specific model + provider combinations. Maintains a routing table with primary and fallback models per tier. Supports A/B testing: route 10% of a tier's traffic to a candidate model and compare quality scores. The routing table is the single source of truth for which model handles which task.
Cost Tracker -- Middleware that intercepts every LLM call and logs: model name, provider, input/output tokens, cost (calculated from current pricing), latency, task type, and assigned tier. This is the raw data that governance analyzes.
Governance Engine -- Runs on a weekly schedule. Compares actual spend against theoretical optimal spend (what if every task used the cheapest model in its tier?). Generates re-routing suggestions when a cheaper model could handle a task type with comparable quality. Flags anomalies where spend exceeds 2x the tier baseline.

Data Model

RoutingRule {
  id, tier, model, provider, priority,
  fallbackModel, maxTokens, isActive, updatedAt
}

RoutingLog {
  id, timestamp, taskType, assignedTier, model,
  inputTokens, outputTokens, cost, latencyMs
}

GovernanceRecommendation {
  id, reportId, taskType, currentModel, suggestedModel,
  currentCostPerCall, projectedCostPerCall,
  estimatedMonthlySavings, confidence, status
}

Key Design Decisions

Route by tier, not model name. Models come and go. If your routing table says "use claude-haiku" and Anthropic discontinues it, you update one row. If it's hardcoded in 50 places, you have a migration project.
Fallback chains. Every tier needs a fallback model. If the primary is unavailable or rate-limited, the router should degrade to the next cheapest option in the same tier, not fail.
Confidence scoring. Not all governance suggestions are equal. A recommendation backed by 10,000 calls at 98% quality match is high-confidence. One backed by 50 calls is speculative. Surface confidence so engineers can prioritize safely.

Phased Rollout

MVP: Manual routing table (a config file mapping task types to models) plus cost logging middleware. No automation, but you now have visibility into what each task type costs. Run for 2 weeks to build a baseline.

v1: Intent classifier that auto-assigns tiers. Weekly governance report comparing actual spend vs. optimal. Re-routing suggestions generated but require manual approval. A/B testing framework for evaluating candidate models.

v2: Auto-routing with approval workflow (suggestions auto-apply after 7 days if not rejected). Live pricing integration (fetch current rates from provider APIs daily). Quality regression detection (if a cheaper model starts producing lower-quality outputs, auto-revert and alert). Cross-provider routing (same tier can use different providers based on real-time pricing and availability).

Technology Recommendations

Classifier: A simple keyword/regex classifier works for MVP. Graduate to an LLM-based classifier only if task types become ambiguous.
Router: A configuration-driven router (JSON/YAML config) for MVP. Graduate to a database-backed router when you need A/B testing.
Cost tracker: Middleware in your LLM client wrapper. Log to your existing database (PostgreSQL recommended). Partition the log table by month since it grows fast.
Governance: A weekly cron job that runs SQL aggregations and calls an LLM to generate human-readable suggestions. Deliver via Slack/email webhook.