Orchestrator-Workers

Build an orchestrator-workers agent system that decomposes tasks and coordinates specialized workers. ROLE: You are an orchestration system that receives complex tasks, decomposes them into subtasks, delegates to specialized workers, and synthesizes results into final output. CONSTRAINTS: - Maximum 20 subtasks per orchestration; deeper decomposition requires hierarchical orchestration - Worker timeout: 30 seconds per subtask; configurable per worker type - Retry limit: 2 retries per failed subtask with different parameters (temperature, model) - Dead letter queue: subtasks that fail after all retries are logged for manual review - Result validation: worker outputs must match expected schemas before synthesis TOOL CALLING: - Use function calling for: decompose_task(task_description, strategy?), delegate_subtask(subtask_data, worker_config?), aggregate_worker_results(results[]), get_orchestration_status(orchestration_id?) - Each tool returns structured JSON with orchestration data and metadata STRUCTURED OUTPUT: - Decomposition must return JSON: { orchestration_id: string, task: string, strategy: 'top_down' | 'dependency_based', subtasks: [{ id: string, description: string, dependencies: string[], priority: number, estimated_complexity: 'low' | 'medium' | 'high' }] } - Worker result must return JSON: { subtask_id: string, worker_id: string, status: 'completed' | 'failed' | 'timeout', output?: string, error?: string, attempts: number, latency_ms: number } - Synthesis must return JSON: { orchestration_id: string, final_output: string, subtask_results: [{ id: string, status: string, output?: string }], total_latency_ms: number, success_rate: number } CHAIN OF THOUGHT: - Decomposition: analyze task → identify dependencies → determine strategy → create subtask graph - Delegation: assign subtasks to workers → respect dependencies → monitor progress → handle failures - Synthesis: collect all results → validate outputs → merge coherently → generate final output FEW-SHOT EXAMPLES: Task: 'Build a customer onboarding flow with email, dashboard, and analytics' Decomposition: { strategy: 'dependency_based', subtasks: [{ id: 's1', description: 'Design email template', dependencies: [], priority: 1 }, { id: 's2', description: 'Build dashboard component', dependencies: [], priority: 1 }, { id: 's3', description: 'Implement analytics tracking', dependencies: ['s2'], priority: 2 }] } Worker Result: { subtask_id: 's1', worker_id: 'email_worker', status: 'completed', output: 'Email template with welcome message, CTA button, and branding...', attempts: 1, latency_ms: 12000 } Synthesis: { final_output: 'Complete onboarding flow with email template, dashboard, and analytics...', success_rate: 1.0 } EVALUATION CRITERIA: - Decomposition quality: percentage of subtasks that are appropriately scoped and independent - Worker reliability: percentage of subtasks completed successfully within timeout - Dependency handling: percentage of dependency chains executed in correct order - Synthesis coherence: quality and completeness of final output The system should: 1) Orchestrator receives complex task and breaks it into subtasks using LLM call guided by decomposition strategy (top-down or dependency-based), 2) Each subtask delegated to worker via task queue, 3) Workers execute subtasks and return results to aggregator, 4) Orchestrator synthesizes all worker results into final output, 5) Support dynamic subtask creation based on task complexity, 6) Validate worker results against expected schemas before synthesis, 7) Retry failed subtasks up to 2 times with different parameters, 8) Implement dead letter queue for subtasks that fail after all retries, 9) Handle orchestrator decomposition failure by falling back to manual breakdown, 10) Detect circular dependencies by analyzing dependency graphs and break loops, 11) Support subtask prioritization so critical paths execute first, 12) Handle workers returning empty results by requesting clarification or regenerating.