Back to AI Recipes

Routing

WorkflowLow latency

Workflow

Route incoming requests to specialized services based on intent.

Summary

The Routing pattern uses an LLM to analyze incoming requests and direct them to the appropriate specialized services or models. This approach allows for efficient handling of diverse user inputs by dynamically determining which downstream processes should handle each request, optimizing for both performance and accuracy.

How it works

  1. Request Analysis: The LLM examines the user's request to identify intent, complexity, and domain
  2. Classifier Decision: A routing model determines the appropriate handler based on classification
  3. Specialized Processing: The request is forwarded to the matched service or model
  4. Response Synthesis: Results are combined and returned to the user

Key considerations

  • Routing adds a classification step but enables faster specialist models
  • Cost optimization by routing simple queries to cheaper models
  • Specialists typically perform better on their domain-specific tasks
  • Routing rules require updates as capabilities evolve

Use cases

  • Multi-service applications where different user intents require different processing pipelines
  • Virtual assistants that need to handle diverse tasks (booking, searching, answering questions)
  • Content moderation systems that route different types of content to specialized analyzers
  • Enterprise systems that need to connect user requests with the right internal tools or databases
Code View

Routing Implementation

// Routing recipe using OpenAI
// Install: bun add openai

import OpenAI from "openai";

async function main() {
  const input = "Add your prompt here.";
  const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
  const system = "You are a senior AI engineer and technical writer. Apply the agent recipe to structure the reasoning and produce a useful result. Recipe: Routing. Description: Route incoming requests to specialized services based on intent. Focus: Workflow Provide actionable, implementation-ready guidance.";
  const user = `Request: ${input}`;

  const openaiResponse = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [
      { role: "system", content: system },
      { role: "user", content: user },
    ],
  });

  const openaiText = openaiResponse.choices[0]?.message?.content?.trim() ?? "";

  console.log(openaiText);
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});