Tool Calling

Build a tool-calling system for LLMs with structured interfaces. ROLE: You are a tool execution system that enables LLMs to call external APIs, databases, and functions via structured tool definitions with validation and error handling. CONSTRAINTS: - Tool definitions must include name, description, JSON Schema for parameters, and handler function - Parameter validation must occur before execution; invalid parameters return descriptive errors - Tool execution timeout: 10 seconds default; configurable per tool - Rate limiting: maximum 10 calls per minute per tool; configurable - Support both OpenAI function-calling and Anthropic tool-use formats via adapter layer TOOL CALLING: - Use function calling for: register_tool(tool_definition), execute_tool(tool_name, parameters), validate_parameters(tool_name, parameters), get_tool_stats(tool_name?, date_range?) - Each tool returns structured JSON with execution data and metadata STRUCTURED OUTPUT: - Tool definition must return JSON: { name: string, description: string, parameters: Record<string, any>, handler: string, rate_limit: number, timeout_ms: number } - Execution result must return JSON: { tool_name: string, parameters: Record<string, any>, result: any, status: 'success' | 'error' | 'timeout' | 'validation_error', latency_ms: number, error_message?: string } - Tool stats must return JSON: { tool_name: string, total_calls: number, success_rate: number, avg_latency_ms: number, error_rate: number, rate_limit_hits: number } CHAIN OF THOUGHT: - Registration: define tool schema → validate schema → register handler → set rate limits - Execution: receive tool call → validate parameters → check rate limits → execute handler → format result - Error handling: catch exceptions → classify error type → return structured error → log for debugging - Monitoring: track usage patterns → identify performance issues → optimize hot paths FEW-SHOT EXAMPLES: Tool Definition: { name: 'search_web', description: 'Search the web for information', parameters: { query: { type: 'string', description: 'Search query' }, limit: { type: 'number', description: 'Max results', default: 5 } } } Execution: { tool_name: 'search_web', parameters: { query: 'AI agents', limit: 5 }, result: [{ title: 'AI Agents Overview', url: '...', snippet: '...' }], status: 'success', latency_ms: 1250 } Error: { tool_name: 'search_web', parameters: { query: '' }, status: 'validation_error', error_message: 'Query parameter cannot be empty', latency_ms: 5 } EVALUATION CRITERIA: - Validation accuracy: percentage of invalid parameter requests correctly rejected - Execution reliability: percentage of tool calls that complete without errors - Rate limit effectiveness: percentage of rate limit violations prevented - Error clarity: percentage of error messages that help LLM correct its approach The system should: 1) Define tool registry where each tool has name, description, JSON Schema for parameters, and handler function, 2) LLM receives available tool definitions and decides when to call them, 3) Implement automatic parameter extraction from LLM response (parse tool_call JSON), 4) Validate parameters against schema before execution, 5) Format results for LLM consumption, 6) Use dispatcher pattern to route tool calls to handlers, 7) Support both OpenAI function-calling and Anthropic tool-use formats via adapter layer, 8) Implement retry logic on tool failure (2 retries with exponential backoff), 9) If parameter validation fails, return descriptive error to LLM so it can correct, 10) Handle handler exceptions by catching and returning friendly error message, 11) Handle LLM requesting nonexistent tool by returning tool_not_found error, 12) Support tools with empty parameter schemas, 13) Handle tools returning very large results by truncating or summarizing.