Architecture Patterns
13 patterns
Transformer Architecture
The foundation model architecture using multi-head self-attention.
Decoder-only Models
GPT-style autoregressive models optimized for text generation.
Encoder-only Models
BERT-style bidirectional models optimized for understanding tasks.
Encoder-Decoder Models
T5-style sequence-to-sequence models for translation and summarization.
Mixture of Experts
Sparse activation architecture for scaling model capacity efficiently.
Sparse Attention Mechanisms
Reduce quadratic attention complexity for long sequence processing.
KV-Cache Optimization
Cache key-value projections to speed up autoregressive generation.
Architecture Comparison
Compare transformer architectures across use cases and trade-offs.
State Space Models
Sequence architectures such as Mamba that trade quadratic attention for linear-time recurrence.
Multimodal Foundation Models
Unified models that reason across text, image, audio, and video instead of stitching separate systems together.
Reasoning Models and Test-Time Compute
Architectures that improve performance by allocating more inference-time reasoning to hard problems.
LLM Inference Serving Systems
Serving-layer architecture behind real-world AI products: paged attention, continuous batching, prefix caching, speculative decoding.
Tool-Augmented LLM Systems
Architectures that combine the model with tools, retrieval, execution environments, and standard context interfaces.