A prompt caching-aware layout arranges prompt sections so that the prefix of the prompt -- the part that is identical across many requests -- is contiguous and maximized. This allows API providers to cache the processed representation of the static prefix, reducing both latency and cost.
How it works
Static first -- place system instructions, tool definitions, and few-shot examples at the very beginning.
Semi-dynamic next -- session-level context that changes infrequently (conversation summary, user profile).
Dynamic last -- the current user input and immediate context go at the end.
Cache strategy
Maximize prefix reuse across requests by keeping the static portion as large as possible and ensuring it does not change between turns. Avoid interleaving static and dynamic content.
Benefits
Lower latency: Cached prefixes skip re-computation on subsequent requests.
Reduced cost: Many providers charge less for cached prompt tokens.
Higher throughput: Cache hits reduce per-request processing time on the server side.
Build This Pattern
Copy this prompt and paste it into Claude Code, OpenCode, Codex, or Cursor to implement this pattern.
Build a prompt layout optimizer for caching efficiency.
ROLE: You are a prompt optimization system that structures prompts in three sections (static, semi-dynamic, dynamic) to maximize cache hit rates and reduce latency.
CONSTRAINTS:
- Static section: system instructions, tool definitions, fixed examples (must be identical across requests)
- Semi-dynamic section: per-session context, conversation history (changes per session, not per request)
- Dynamic section: current user input, fresh tool outputs (changes every request)
- Static content must come first for cache prefix matching
- Monitor cache hit rate; alert when below threshold
TOOL CALLING:
- Use function calling for: optimize_layout(prompt_sections), measure_cache_performance(layout_id?), analyze_cache_misses(layout_id, time_range?)
- Each tool returns structured JSON with layout data and metadata
STRUCTURED OUTPUT:
- Optimized layout must return JSON: { layout_id: string, static_section: string, semi_dynamic_section: string, dynamic_section: string, total_tokens: number, static_tokens: number, cacheable_percent: number }
- Cache performance must return JSON: { layout_id: string, total_requests: number, cache_hits: number, cache_misses: number, hit_rate: number, avg_latency_ms: number, latency_savings_ms: number }
- Cache analysis must return JSON: { layout_id: string, misses_by_section: { static: number, semi_dynamic: number, dynamic: number }, miss_reasons: [{ reason: string, count: number }], optimization_suggestions: string[] }
CHAIN OF THOUGHT:
- Layout analysis: identify prompt sections → classify as static/semi-dynamic/dynamic → measure token distribution
- Optimization: reorder sections → maximize static prefix → minimize dynamic content → validate structure
- Performance measurement: track cache hits/misses → calculate hit rate → measure latency improvements
- Improvement: analyze misses → identify causes → suggest optimizations → retest
FEW-SHOT EXAMPLES:
Layout: { static_section: 'You are a helpful assistant. Available tools: search, calculate...', semi_dynamic_section: 'User context: Company is Acme Corp, Plan: Pro', dynamic_section: 'User query: What is our revenue?'
Cacheable: 65%
Performance: { total_requests: 1000, cache_hits: 850, hit_rate: 0.85, avg_latency_ms: 120, latency_savings_ms: 350 }
Analysis: { misses_by_section: { static: 0, semi_dynamic: 120, dynamic: 30 }, miss_reasons: [{ reason: 'session_context_changed', count: 120 }], optimization_suggestions: ['Move more content to static section'] }
EVALUATION CRITERIA:
- Cache hit rate: percentage of requests that hit cache
- Latency improvement: percentage reduction in average response time
- Cost savings: reduction in token costs from cache hits
- Layout effectiveness: percentage of prompt that is cacheable
The system should: 1) Structure prompts in three sections: STATIC (system instructions, tool definitions, fixed examples), SEMI-DYNAMIC (per-session context, conversation history), DYNAMIC (current user input, fresh tool outputs), 2) Put static content first, 3) Detect when cache hit rate drops below threshold, 4) Handle very long static sections, 5) Handle mixed cache provider behavior, 6) Measure cache hit rate and latency improvement, 7) Verify cache hit rate with repeated identical prefixes.