Back to Patterns

Memory Tiering

Agent Patterns

Summary

Memory tiering organizes agent state into three tiers with distinct persistence and access rules. Working memory holds current-turn context, session memory spans the active conversation, and long-term memory persists across sessions.

How it works

  1. Three tiers -- each tier has its own storage backend, retention policy, and access semantics.
  2. Read/write rules -- agents can only access tiers appropriate to the current operation.
  3. Promotion and eviction -- data moves between tiers based on usage patterns and importance scores.

Tiers

  • Working memory: Current turn state, scratchpad, intermediate results. Volatile, cleared after each response.
  • Session memory: Conversation history, user preferences for the session, task queue. Retained for the session duration.
  • Long-term memory: User identity, learned facts, persistent preferences. Stored in durable storage across sessions.

Operations

  • Read/write rules per tier: Tier 1 is read-write for the current turn only. Tier 2 is read-write for the session. Tier 3 is append-mostly with limited mutation.
  • Promotion/demotion: Frequently accessed session facts may be promoted to long-term. Stale long-term facts may be demoted or archived.
  • Eviction: Least recently used data is evicted first when tier capacity is reached. Eviction triggers a compaction or archival step.

Build This Pattern

Copy this prompt and paste it into Claude Code, OpenCode, Codex, or Cursor to implement this pattern.

Build a memory tiering system for efficient context management. ROLE: You are a memory management system that organizes state into three tiers (working, session, long-term) with size limits and eviction policies. CONSTRAINTS: - Working memory: current turn context only; maximum 4000 tokens - Session memory: conversation/task history for active session; maximum 50,000 tokens - Long-term memory: persistent knowledge across sessions; maximum 100,000 tokens per project - Eviction: LRU for working memory; time-based for session; importance-scored for long-term - Promotion/demotion between tiers based on access frequency and relevance TOOL CALLING: - Use function calling for: store_memory(content, tier, metadata?), retrieve_memory(query, tier?, limit?), promote_memory(memory_id, target_tier), demote_memory(memory_id, target_tier), get_memory_stats() - Each tool returns structured JSON with memory data and metadata STRUCTURED OUTPUT: - Memory entry must return JSON: { id: string, content: string, tier: 'working' | 'session' | 'long_term', metadata: Record<string, any>, created_at: string, accessed_at: string, access_count: number, size_tokens: number } - Retrieval results must return JSON: { query: string, results: [{ id: string, content: string, tier: string, score: number }], total_results: number, token_count: number } - Memory stats must return JSON: { working: { count: number, size_tokens: number, utilization: number }, session: { count: number, size_tokens: number, utilization: number }, long_term: { count: number, size_tokens: number, utilization: number }, promotions_today: number, demotions_today: number } CHAIN OF THOUGHT: - Storage: determine appropriate tier → check size limits → evict if necessary → store with metadata - Retrieval: search across tiers → rank by relevance and recency → return top results within token budget - Promotion: identify frequently accessed memories → move to higher tier → update access patterns - Demotion: identify stale memories → move to lower tier → archive or delete if below threshold FEW-SHOT EXAMPLES: Store: { content: 'User prefers dark mode', tier: 'session', metadata: { topic: 'preferences' } } Retrieve: { query: 'user preferences', results: [{ id: 'mem_123', content: 'User prefers dark mode', tier: 'session', score: 0.95 }] } Promotion: { id: 'mem_456', content: 'Project uses Next.js 14', tier: 'working', access_count: 15 } Promote to: long_term (frequently accessed across sessions) Stats: { working: { count: 12, size_tokens: 3200, utilization: 0.8 }, session: { count: 45, size_tokens: 28000, utilization: 0.56 }, long_term: { count: 234, size_tokens: 89000, utilization: 0.89 } } EVALUATION CRITERIA: - Tier utilization: percentage of each tier's capacity used without overflow - Retrieval relevance: percentage of retrieved memories that are useful for the query - Eviction appropriateness: percentage of evicted memories that were truly stale - Promotion accuracy: percentage of promoted memories that are frequently accessed The system should: 1) Three tiers: WORKING (current turn context), SESSION (conversation/task history for active session), LONG-TERM (persistent knowledge across sessions), 2) Each tier has size limits and eviction policies, 3) Handle memory corruption across tiers, conflicting memories, 4) Handle memory that spans tiers, promotion/demotion between tiers, 5) Log memory operations with tier, size, access frequency, 6) Verify correct memory retrieval from the appropriate tier.