Tool-Augmented LLM Systems

Explain tool-augmented LLM system architectures that enable models to interact with external systems. Architecture: describe the core loop where the LLM generates a structured tool call (function name + JSON parameters), the application executes it, and returns the result to the model for the next reasoning step — this generate-execute-respond cycle forms the foundation of every ReAct agent. Cover how tool definitions are specified via JSON Schema (name, description, input_schema as the three required fields) and passed to the model in the system prompt or API call, with the model deciding when and which tools to invoke based on task requirements. Explain the Model Context Protocol (MCP) as the emerging open standard that replaces fragmented per-provider tool formats with a unified protocol defining four primitives: Tools (callable functions), Resources (data sources), Prompts (reusable templates), and Sampling (model invocation through the server). Describe the three architectural patterns for tool integration: direct SDK integration (tool definitions passed directly to model API), MCP gateway (centralized server that manages tool discovery, authentication, and routing for multiple clients), and connector proxies (adapter layers that translate between MCP and legacy tool APIs). Reference representative provider approaches: OpenAI's function calling (June 2023 originator, adopted MCP across Agents SDK in March 2025), Anthropic's tool-use with the Tool Search Tool pattern (giving agents a meta-tool to search a registry on demand, achieving 34-64% token reduction versus loading all tools upfront), and Google's Gemini with native function calling. Cover scaling patterns: tool registries for centralized catalog management, dynamic tool registration via MCP's runtime update capability (tools added without restarting agents), the Agent-as-Tool recursive pattern where an entire agent is exposed as a callable tool for hierarchical multi-agent architectures, and the Tool Search Tool pattern for handling hundreds of tools without degrading reasoning quality. Error handling: discuss prompt injection as the #1 security threat where malicious tool outputs can manipulate model reasoning, the need for input validation before tool execution (LLMs can produce invalid parameters), rate limiting and timeout configuration per tool, graceful degradation when tools fail or return errors, and the challenge of tool hallucination where the model calls non-existent tools or passes nonsensical arguments. Cover audit trail requirements for production systems and the importance of separating tool execution permissions by risk category (read-only vs write vs destructive). Edge cases: handling tools with slow execution that block the reasoning loop, managing authentication tokens and credentials across multiple tool servers, behavior when tool schemas change at runtime (MCP's live reload capability), supporting both synchronous and asynchronous tool patterns, and the limitation that current LLMs were not explicitly trained for autonomous tool discovery or cross-tool interaction. Best practices: include guidance on writing clear tool descriptions that help the model select the correct tool, implementing structured error responses that the model can reason about, using parallel tool calling where supported to reduce latency, combining tool calling with RAG (retrieval for knowledge, tools for actions), and implementing human-in-the-loop approval for destructive actions. Testing: suggest evaluating tool selection accuracy on benchmarks like BFCL (Berkeley Function Calling Leaderboard), testing multi-step tool chains where one tool's output feeds another, measuring end-to-end latency including tool execution time, and validating that models correctly handle tool errors and retry appropriately.