Glossary

Key AI, LLM, and agent terms defined in plain language. Bookmark this page as a quick reference while building production AI systems.

Agent: An AI system that can autonomously plan, use tools, and take actions to accomplish a goal. Agents combine an LLM with tool-calling, memory, and control flow.; See also: Internal AI OS
RAG: Retrieval-Augmented Generation. A technique that retrieves relevant documents or data before generating an answer, grounding LLM outputs in factual sources.; See also: RAG Over Company Docs
Embedding: A numerical vector representation of text that captures semantic meaning. Embeddings enable similarity search and are the foundation of vector databases.; See also: RAG Over Company Docs
Fine-tuning: The process of further training a pre-trained LLM on domain-specific data to improve its performance on a particular task or style.
Prompt Engineering: The practice of designing input prompts to elicit desired outputs from LLMs. Includes techniques like few-shot examples, chain-of-thought, and system instructions.
Chain-of-Thought: A prompting technique that asks the LLM to reason step-by-step before giving a final answer, improving accuracy on complex tasks.
Few-Shot: A prompting strategy that includes a small number of example input-output pairs in the prompt to guide the model's behavior.
Zero-Shot: Asking an LLM to perform a task without providing any examples, relying entirely on its pre-trained knowledge and instruction following.
Tool Use: The ability of an LLM to call external tools, APIs, or functions during generation. The model outputs a structured tool call, and the runtime executes it.
Function Calling: A structured mechanism where the LLM outputs a function name and arguments as JSON, which the application executes and feeds results back to the model.
Context Window: The maximum number of tokens an LLM can process in a single request, including both input and output. Larger windows allow more data but cost more.
Token: The basic unit of text processed by an LLM. A token is roughly 3/4 of a word in English. Models charge per token for both input and output.
Temperature: A parameter controlling randomness in LLM outputs. Lower values (0.0-0.3) produce more deterministic outputs; higher values (0.7-1.0) increase creativity.
Top-P: Also called nucleus sampling. A parameter that limits token selection to the smallest set whose cumulative probability exceeds P, controlling diversity.
Hallucination: When an LLM generates plausible-sounding but factually incorrect or fabricated information. A major challenge addressed by grounding and RAG.; See also: RAG Over Company Docs
Grounding: Techniques that anchor LLM outputs to real, verifiable sources. Includes RAG, citation requirements, and tool-based fact-checking.; See also: RAG Over Company Docs
Vector Database: A database optimized for storing, indexing, and querying high-dimensional vectors (embeddings). Enables fast similarity search for RAG systems.
Semantic Search: Search based on meaning rather than exact keyword matching. Uses vector embeddings to find conceptually similar content.; See also: RAG Over Company Docs
Chunking: The process of splitting documents into smaller segments for embedding and retrieval. Chunk size and overlap affect retrieval quality.; See also: RAG Over Company Docs
Knowledge Graph: A structured representation of entities and their relationships. Can augment RAG systems by providing structured context alongside vector search.
RLHF: Reinforcement Learning from Human Feedback. A training method where human preferences are used to fine-tune an LLM's behavior via a reward model.
DPO: Direct Preference Optimization. A simpler alternative to RLHF that fine-tunes models directly from preference data without a separate reward model.
LoRA: Low-Rank Adaptation. A fine-tuning technique that adds small trainable matrices to a frozen model, drastically reducing the resources needed for customization.
Quantization: Reducing the precision of model weights (e.g., from FP16 to INT4) to shrink model size and memory requirements with minimal quality loss.
GGUF: GPT-Generated Unified Format. A file format for quantized models optimized for CPU inference with llama.cpp. Popular for local deployment.
KV Cache: A memory optimization that stores previously computed key-value attention pairs, avoiding redundant computation during autoregressive generation.
Attention Mechanism: The core operation in transformers that allows each token to attend to all other tokens, learning which parts of the input are relevant to each output.
Transformer: The foundational neural network architecture for modern LLMs. Uses self-attention to process sequences in parallel, enabling scalable training.
Mixture of Experts: A model architecture where different sub-networks (experts) specialize in different inputs, and a gating network routes each input to the most relevant experts.
Agent Loop: The core execution pattern of an AI agent: observe the environment, think about next steps, take an action, and repeat until the task is complete.; See also: Internal AI OS