Self-Consistency is a decoding strategy that samples multiple reasoning paths from the LLM and selects the most frequent answer. By aggregating diverse reasoning traces, this technique reduces errors from single-path reasoning and improves answer reliability. It builds on Chain of Thought by sampling multiple CoT outputs rather than using greedy decoding.
How it works
Generate multiple paths: Sample N reasoning traces for the same problem
Extract answers: Parse final answers from each trace
Vote/aggregate: Select the most common answer
Return result: Output the consensus answer
Key considerations
Sample count: More samples increase accuracy but cost
Aggregation: Majority vote, weighted, or confidence-based
Diversity: Temperature and sampling parameters affect variety
Speed: Parallel generation can mitigate latency
When to use
Tasks where multiple reasoning paths exist
Applications requiring high reliability
Scenarios where cost-latency trade-off is acceptable
Math, logic, and factual reasoning tasks
Build This Pattern
Copy this prompt and paste it into Claude Code, OpenCode, Codex, or Cursor to implement this pattern.
Build a self-consistency system for LLM reasoning.
ROLE: You are a reasoning ensemble system that generates multiple independent reasoning paths and aggregates them into a consensus answer.
CONSTRAINTS:
- Default N: 5 reasoning paths; configurable per task
- Temperature: 0.7 for diversity; configurable per path
- Agreement threshold: below 0.3 triggers low-confidence warning
- Aggregation strategy: majority voting (multiple-choice), median (numeric), theme clustering (free-text)
- Answer type detection must automatically select aggregation strategy
TOOL CALLING:
- Use function calling for: generate_paths(problem, n?, temperature?), aggregate_paths(paths[], answer_type?), detect_answer_type(sample_answer), get_consensus_stats(problem_id?)
- Each tool returns structured JSON with reasoning data and metadata
STRUCTURED OUTPUT:
- Reasoning path must return JSON: { path_id: string, reasoning: string, answer: string, confidence: number, temperature: number, generation_time_ms: number }
- Aggregation result must return JSON: { problem: string, answer_type: string, consensus_answer: string, agreement_rate: number, individual_answers: [{ path_id: string, answer: string }], conflicting_answers?: string[], strategy_used: string }
- Consensus stats must return JSON: { problem_id: string, total_paths: number, agreement_rate: number, confidence_score: number, processing_time_ms: number, paths_with_no_answer: number }
CHAIN OF THOUGHT:
- Path generation: create N independent prompts → generate with different seeds → collect reasoning and answers
- Type detection: analyze answer format → classify as multiple-choice/numeric/free-text → select strategy
- Aggregation: group answers by type → apply strategy → calculate agreement → determine consensus
- Validation: check consensus against individual paths → identify conflicts → assess confidence
FEW-SHOT EXAMPLES:
Problem: 'What is the capital of France?'
Paths: [
{ path_id: 'p1', answer: 'Paris', confidence: 0.98 },
{ path_id: 'p2', answer: 'Paris', confidence: 0.95 },
{ path_id: 'p3', answer: 'Paris', confidence: 0.97 },
{ path_id: 'p4', answer: 'Lyon', confidence: 0.45 },
{ path_id: 'p5', answer: 'Paris', confidence: 0.92 }
]
Consensus: { consensus_answer: 'Paris', agreement_rate: 0.8, confidence_score: 0.96 }
Problem: 'Estimate the population of Tokyo'
Paths: [{ answer: '14 million' }, { answer: '13.5 million' }, { answer: '15 million' }, { answer: '14.2 million' }, { answer: '13.8 million' }]
Consensus: { consensus_answer: '14 million', strategy_used: 'median', agreement_rate: 0.6 }
EVALUATION CRITERIA:
- Agreement accuracy: correlation between agreement rate and answer correctness
- Consensus quality: percentage of consensus answers that are correct
- Path diversity: variance in reasoning approaches across paths
- Confidence calibration: alignment between confidence scores and actual accuracy
The system should: 1) Implement parallel generation module that creates N independent reasoning paths (default 5, temperature 0.7) with different random seeds, 2) Each path produces structured output, 3) Aggregation module combines answers using type-specific strategies: majority voting for multiple-choice, median with variance for numeric answers, theme clustering for free-text, 4) Include answer type detection module that automatically selects aggregation strategy, 5) Handle paths producing no answer by excluding them from aggregation, 6) If all paths disagree (agreement rate below 0.3), return all answers with low-confidence warning, 7) Handle empty or invalid path outputs gracefully, 8) Handle ties in majority voting by using secondary criteria (path confidence), 9) Handle numeric answers with units by normalizing before aggregation, 10) Support answers that are lists or sets by using Jaccard similarity for clustering.