Why Structured Output Is Necessary
LLMs generate free-form text, but real applications need data in a defined format. API responses must be JSON, and database storage requires exact fields.
Using a restaurant analogy, “give me something delicious” (free text) versus “please fill out this order form” (structured output) — the latter gets you a more accurate result.
Common situations requiring structured output:
| Situation | Required Format | Example |
|---|---|---|
| API response generation | JSON | {"name": "John", "age": 30} |
| Data extraction | Schema-based object | Extracting schedule info from emails |
| Function calling | Function arguments | search(query="weather", city="Seoul") |
| Classification tasks | Enum | "sentiment": "positive" |
| Workflow control | Next step decision | {"action": "search", "params": {...}} |
Method 1: JSON Mode
Major APIs including OpenAI and Anthropic support JSON mode. It guarantees the response is always valid JSON.
from openai import OpenAI
client = OpenAI()
# Enable JSON mode
response = client.chat.completions.create(
model="gpt-4o",
response_format={"type": "json_object"}, # Enable JSON mode
messages=[
{
"role": "system",
"content": "Analyze the user's request and respond in JSON."
},
{
"role": "user",
"content": "Create a profile for John Smith, age 30, living in Gangnam, Seoul"
}
]
)
import json
data = json.loads(response.choices[0].message.content)
print(json.dumps(data, ensure_ascii=False, indent=2))
# Output:
# {
# "name": "John Smith",
# "age": 30,
# "address": {
# "city": "Seoul",
# "district": "Gangnam"
# }
# }Limitations of JSON mode: It guarantees valid JSON, but does not guarantee schema compliance. The age field could be a string or a number.
Method 2: Structured Outputs (Schema Enforcement)
OpenAI’s Structured Outputs generates responses that 100% comply with the schema when you define a JSON Schema.
from openai import OpenAI
from pydantic import BaseModel
client = OpenAI()
# Define output schema with Pydantic
class UserProfile(BaseModel):
name: str
age: int
email: str
skills: list[str]
is_active: bool
# Structured Outputs -- schema enforcement
response = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{"role": "system", "content": "Generate a user profile."},
{"role": "user", "content": "Jane Lee, age 25, who works with Python and JavaScript"}
],
response_format=UserProfile # Specify schema with Pydantic model
)
# Returns a type-guaranteed object
user = response.choices[0].message.parsed
print(f"Name: {user.name}") # Output: Name: Jane Lee
print(f"Age: {user.age}") # Output: Age: 25 (guaranteed int)
print(f"Skills: {user.skills}") # Output: Skills: ['Python', 'JavaScript']
print(f"Active: {user.is_active}") # Output: Active: True (guaranteed bool)Complex Schema Example
from pydantic import BaseModel, Field
from enum import Enum
class Sentiment(str, Enum):
positive = "positive"
negative = "negative"
neutral = "neutral"
class Entity(BaseModel):
name: str = Field(description="Entity name")
type: str = Field(description="Entity type (person, place, organization, etc.)")
class TextAnalysis(BaseModel):
"""Text analysis result schema"""
summary: str = Field(description="Text summary (1-2 sentences)")
sentiment: Sentiment = Field(description="Sentiment analysis result")
entities: list[Entity] = Field(description="List of extracted entities")
keywords: list[str] = Field(description="Key keywords (max 5)")
confidence: float = Field(description="Analysis confidence (0.0-1.0)")
response = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{"role": "system", "content": "Analyze the text."},
{"role": "user", "content": "Samsung Electronics announced a new AI semiconductor at an event in Gangnam, Seoul. Market response is very positive."}
],
response_format=TextAnalysis
)
analysis = response.choices[0].message.parsed
print(f"Sentiment: {analysis.sentiment}") # Output: Sentiment: positive
print(f"Entities: {analysis.entities}") # Output: [Entity(name='Samsung Electronics', type='organization'), ...]
print(f"Confidence: {analysis.confidence}") # Output: Confidence: 0.92Method 3: Function Calling (Tool Use)
Function Calling is a mechanism where the LLM decides which function to call and what arguments to pass. It’s the core of the agent pattern.
from openai import OpenAI
import json
client = OpenAI()
# Define tools (functions)
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Looks up the current weather for a specific city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name (e.g., Seoul, Busan)"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit"
}
},
"required": ["city"]
}
}
},
{
"type": "function",
"function": {
"name": "search_restaurants",
"description": "Searches for nearby restaurants",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "Search location"},
"cuisine": {"type": "string", "description": "Type of cuisine"},
"max_results": {"type": "integer", "description": "Maximum number of results"}
},
"required": ["location"]
}
}
}
]
# LLM selects the appropriate function
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "user", "content": "What's the weather in Seoul? Also recommend Italian restaurants near Gangnam"}
],
tools=tools,
tool_choice="auto" # LLM automatically selects tools
)
# Process function call results
for tool_call in response.choices[0].message.tool_calls:
func_name = tool_call.function.name
func_args = json.loads(tool_call.function.arguments)
print(f"Function called: {func_name}")
print(f"Arguments: {func_args}")
# Output:
# Function called: get_weather
# Arguments: {"city": "Seoul"}
# Function called: search_restaurants
# Arguments: {"location": "Gangnam", "cuisine": "Italian", "max_results": 5}Anthropic Claude’s Tool Use
The Claude API provides similar tool use capabilities.
import anthropic
client = anthropic.Anthropic()
# Define tools (Claude format)
tools = [
{
"name": "calculate",
"description": "Performs mathematical calculations",
"input_schema": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Expression to calculate (e.g., 2 + 3 * 4)"
}
},
"required": ["expression"]
}
}
]
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=tools,
messages=[
{"role": "user", "content": "What is 15 squared minus 100?"}
]
)
# Check tool use result
for block in response.content:
if block.type == "tool_use":
print(f"Tool: {block.name}")
print(f"Input: {block.input}")
# Output:
# Tool: calculate
# Input: {'expression': '15**2 - 100'}Method Comparison
| Criteria | JSON Mode | Structured Outputs | Function Calling |
|---|---|---|---|
| Schema guaranteed | No (valid JSON only) | Yes (100%) | Yes (argument schema) |
| Use case | Simple JSON responses | Complex data extraction | External function integration |
| Model support | Most models | GPT-4o and select models | Most models |
| Pydantic integration | Manual parsing | Auto parsing | Manual parsing |
| Nested structures | Possible | Fully supported | Possible |
Practical Tips
- Use Structured Outputs first: Since schema compliance is 100% guaranteed, you don’t need parsing error handling code.
- Actively use Pydantic models: You get type hints and validation simultaneously, plus IDE autocomplete support.
- Write specific function descriptions: The
descriptionfield is the key criterion the LLM uses to select the right tool. Include both “what it does” and “when to use it.” - Specify
requiredfields: Without specifying required arguments, the LLM may arbitrarily omit them. - Use Enums to constrain choices: Instead of free text, constraining allowed values with
enumproduces consistent output. - Have a fallback strategy: For models that don’t support Structured Outputs, use a JSON mode + Pydantic validation combination as an alternative.