The Routing pattern uses an LLM to analyze incoming requests and direct them to the appropriate specialized services or models. This approach allows for efficient handling of diverse user inputs by dynamically determining which downstream processes should handle each request, optimizing for both performance and accuracy.
How it works
Request Analysis: The LLM examines the user's request to identify intent, complexity, and domain
Classifier Decision: A routing model determines the appropriate handler based on classification
Specialized Processing: The request is forwarded to the matched service or model
Response Synthesis: Results are combined and returned to the user
Key considerations
Routing adds a classification step but enables faster specialist models
Cost optimization by routing simple queries to cheaper models
Specialists typically perform better on their domain-specific tasks
Routing rules require updates as capabilities evolve
Use cases
Multi-service applications where different user intents require different processing pipelines
Virtual assistants that need to handle diverse tasks (booking, searching, answering questions)
Content moderation systems that route different types of content to specialized analyzers
Enterprise systems that need to connect user requests with the right internal tools or databases
Build This Pattern
Copy this prompt and paste it into Claude Code, OpenCode, Codex, or Cursor to implement this pattern.
Build me an LLM routing system that classifies user input and routes it to the appropriate handler. Architecture: use a router pattern with a classifier module that calls an LLM to categorize input into predefined categories (technical support, billing, sales, general). Each category maps to a specialized handler function defined in a route registry. Use a strategy pattern for handlers so new routes can be added via config without modifying router logic. Include a confidence threshold: if classification confidence is below 0.7, route to a fallback human handoff handler. Support adding new routes dynamically via config. Error handling: handle unclassified inputs by routing to a default unknown category handler. Implement fallback chain: try primary route, then lower-confidence route, then default handler. Log classification failures with raw input for manual review. Edge cases: handle ambiguous inputs near threshold by routing to a disambiguation sub-agent. Normalize input before classification (trimming whitespace, detecting encoding). Route empty or nonsensical inputs directly to human handoff. Best practices: log routing decisions with input hash, classification, confidence, and route taken. Use structured schemas for route definitions with validation on load. Implement rate limiting per route. Testing: unit test classifier with predefined inputs and expected categories. Test fallback and threshold behavior with confidence values at boundaries. Integration test full routing flow from input to handler response. Use TypeScript with a router pattern.