Back to Patterns

Structured Outputs for LLMs

Prompt Patterns

What structured output guarantees

Structured output is a response format that constrains a model's answer to a machine-readable shape. With a provider's schema-constrained mode, the response is constrained to a supported JSON Schema. This reduces format and missing-field errors, but it does not prove that extracted facts are true, authorized, or safe to apply.

Keep four checks separate:

  1. Format: did the provider return the expected response type?
  2. Schema: does the parsed value match the application's types?
  3. Meaning: are values plausible and supported by the input?
  4. Permission: is the application allowed to use the result for this action?

For example, a valid invoice object can still contain the wrong amount. Keep authorization and high-impact business rules in application code.

Choose the right output method

Method Use it when What it guarantees
Provider schema-constrained output The API and model support the schema you need Output follows the supported schema, subject to refusal and incomplete-response handling
JSON mode Schema-constrained output is unavailable, but a JSON response is useful Syntactically valid JSON in normal completion cases; it does not guarantee your schema
Prompted format only The result is for a human or a low-risk prototype No machine-enforced shape; parse and validate before application use
Function calling The model needs to request an application tool or action A structured tool request; the application still decides whether to execute it

In OpenAI's API, use Structured Outputs with response_format for a structured answer and function calling when the model should request an application action. The API supports a subset of JSON Schema, and compatible models vary. Check the current Structured Outputs guide before selecting a model or schema. It documents that JSON mode guarantees JSON syntax, while Structured Outputs also enforces supported schemas.

TypeScript example with Zod

This example uses the OpenAI JavaScript SDK's Chat Completions parser. gpt-4o-mini supports Structured Outputs. All fields in a strict schema are required; represent an absent value with a nullable field instead of an optional property.

import OpenAI from "openai";
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const openai = new OpenAI();

const ProductSchema = z.object({
  name: z.string(),
  price: z.number(),
  category: z.enum(["electronics", "clothing", "food"]),
  inStock: z.boolean(),
  notes: z.string().nullable(),
});

type Product = z.infer<typeof ProductSchema>;

async function extractProduct(text: string): Promise<Product> {
  const completion = await openai.chat.completions.parse({
    model: "gpt-4o-mini",
    messages: [
      { role: "system", content: "Extract product details from the input." },
      { role: "user", content: text },
    ],
    response_format: zodResponseFormat(ProductSchema, "product"),
  });

  const choice = completion.choices[0];
  if (!choice) throw new Error("The model returned no choice.");
  if (choice.finish_reason === "length") {
    throw new Error("The response was incomplete; do not use it as a result.");
  }

  const message = choice.message;
  if (message.refusal) {
    throw new Error(`The model refused this request: ${message.refusal}`);
  }
  if (!message.parsed) {
    throw new Error("The model returned no parsed product.");
  }

  return message.parsed;
}

The SDK's parser validates the structured response against the supplied schema. Keep a separate application check for business constraints, such as a non-negative price, before saving or acting on the result. The API documentation describes refusals and length-limited responses as cases the caller must handle.

JSON mode fallback

If a provider or model does not support schema-constrained output, JSON mode can provide valid JSON without enforcing your shape. Parse it and validate it yourself; reject invalid data rather than silently filling in fields.

const raw = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    { role: "system", content: "Return JSON with name and category fields." },
    { role: "user", content: text },
  ],
  response_format: { type: "json_object" },
});

const content = raw.choices[0]?.message.content;
if (!content) throw new Error("The model returned no JSON content.");

let decoded: unknown;
try {
  decoded = JSON.parse(content);
} catch {
  throw new Error("The model returned invalid JSON.");
}

const product = ProductSchema.safeParse(decoded);
if (!product.success) throw new Error("JSON did not match the product schema.");

In production, also catch JSON parse failures and handle refusals or truncated output according to the provider's response contract. Do not retry refusals as if they were formatting errors. For retryable failures, cap attempts and log the failure category without logging sensitive input.

Implementation checklist

  • Choose schema-constrained output when the model supports your schema; otherwise treat JSON mode as syntax-only.
  • Keep schemas small and within the provider's supported JSON Schema subset.
  • For strict OpenAI schemas, mark fields required; use null in a union when a value may be absent.
  • Distinguish refusal, incomplete output, parsing failure, and schema mismatch.
  • Validate domain rules, evidence, permissions, and side effects in application code.
  • Test representative inputs, missing information, malformed source text, boundary values, refusals, and token limits.
  • Keep retries bounded. Do not turn a valid shape into permission to perform a consequential action.

Build This Pattern

Copy this prompt and paste it into Claude Code, OpenCode, Codex, or Cursor to implement this pattern.

Build a structured output system for LLMs with schema validation. ROLE: You are a type-safe output system that constrains LLM outputs to valid JSON matching defined schemas, with validation and retry logic. CONSTRAINTS: - Use Zod for schema definition and runtime validation - Use provider-native schema-constrained output only when the selected API/model supports the schema subset - Treat JSON mode as syntax-only; parse and validate against the application schema - Handle refusals, incomplete responses, schema errors, and provider errors as distinct outcomes - Bound retries for retryable failures; never retry a refusal as a formatting error - Handle nested objects within the provider's documented schema limits TOOL CALLING: - Use function calling for: define_schema(schema_data), validate_output(output, schema_id?), generate_structured_output(prompt, schema_id, mode?), get_validation_stats(schema_id?) - Each tool returns structured JSON with schema data and metadata STRUCTURED OUTPUT: - Schema definition must return JSON: { schema_id: string, name: string, schema: Record<string, any>, zod_string: string, created_at: string, validation_count: number } - Validation result must return JSON: { valid: boolean, output: any, errors: [{ path: string, message: string, received: string }], schema_id: string, validation_time_ms: number } - Structured generation must return JSON: { output: unknown, schema_id: string, mode: 'schema_constrained' | 'json_mode', status: 'success' | 'refusal' | 'incomplete' | 'schema_error', validated: boolean, attempts: number, generation_time_ms: number } - Validation stats must return JSON: { schema_id: string, total_validations: number, success_rate: number, avg_validation_time_ms: number, common_errors: [{ error: string, count: number }] } CHAIN OF THOUGHT: - Schema definition: define structure → convert to Zod → generate TypeScript types → store - Output generation: check provider/model/schema support → select schema-constrained output or JSON mode - Validation: parse output → check shape and domain rules separately → collect errors → return structured result - Recovery: classify refusal, incomplete output, provider error, or validation error → retry only bounded retryable failures FEW-SHOT EXAMPLES: Schema: { name: 'User', schema: { name: 'string', age: 'number', email: 'string (email format)' } } Zod: z.object({ name: z.string(), age: z.number().positive(), email: z.string().email() }) Output: { name: 'John Doe', age: 30, email: 'john@example.com' } Validation: { valid: true, output: { name: 'John Doe', age: 30, email: 'john@example.com' }, errors: [] } Invalid Output: { name: 'John', age: -5, email: 'not-an-email' } Validation: { valid: false, errors: [{ path: 'age', message: 'Number must be positive', received: '-5' }, { path: 'email', message: 'Invalid email', received: 'not-an-email' }] } EVALUATION CRITERIA: - Validation accuracy: percentage of outputs correctly validated against schema - Retry success rate: percentage of retries that produce valid output - Schema coverage: percentage of common data types supported - Error clarity: percentage of validation errors that help LLM correct its output The system should: 1) Define provider-compatible schemas with Zod and derive TypeScript types, 2) Use schema-constrained output only when the provider, model, and schema are supported, 3) Treat JSON mode as valid-JSON formatting without schema guarantees and validate it separately, 4) Handle refusals and incomplete responses explicitly, 5) Validate domain rules and permissions in application code, 6) Retry only bounded retryable failures and do not retry refusals as formatting errors, 7) Explain provider-specific schema limits and nullable representations, 8) Test valid, invalid, boundary, refusal, and incomplete cases.