Chapter 6: Structured Output Prompting — Getting Reliable JSON, Tables, and Schemas

The Production Output Problem

In development, you read the model's response yourself. It looks good. You extract what you need. You move on.

In production, code reads the model's response. If the format is even slightly off — an extra word before the JSON, a missing closing brace, a field name that is slightly different — your parser breaks, your feature fails, and your users see an error.

Structured output prompting is the discipline of making AI output reliable enough to parse programmatically, consistently, across thousands of requests.

---

The Four Approaches (Ranked by Reliability)

Approach 1: Native JSON Mode (Most Reliable)

The most reliable approach is not a prompting technique at all — it is using the model's native JSON output mode, which constrains the model's output grammar to valid JSON.

OpenAI:

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You extract information and return it as JSON.' },
    { role: 'user', content: 'Extract the key info from this job posting: ' + jobText }
  ],
  response_format: { type: 'json_object' }, // Guarantees valid JSON
});
const data = JSON.parse(response.choices[0].message.content);

OpenAI Structured Outputs (Schema-enforced):

const response = await client.beta.chat.completions.parse({
  model: 'gpt-4o-2024-08-06',
  messages: [...],
  response_format: zodResponseFormat(JobPostingSchema, 'job_posting'),
});
const job = response.choices[0].message.parsed; // Fully typed, guaranteed schema

Gemini:

const response = await model.generateContent({
  contents: [...],
  generationConfig: {
    responseMimeType: 'application/json',
    responseSchema: {
      type: 'OBJECT',
      properties: {
        title: { type: 'STRING' },
        salary: { type: 'STRING' },
        requirements: { type: 'ARRAY', items: { type: 'STRING' } },
      },
      required: ['title', 'requirements'],
    },
  },
});

Use native JSON mode whenever available. It is unambiguously the most reliable approach.

Approach 2: Explicit Schema in Prompt (High Reliability)

When native JSON mode is unavailable, provide the exact schema as part of your prompt:

``` Extract information from the following job posting and return it as a JSON object with EXACTLY this structure. Do not include any text before or after the JSON.

Required schema: { "title": string, "company": string, "location": string, "salary_range": { "min": number | null, "max": number | null, "currency": string

}, "employment_type": "full-time" | "part-time" | "contract" | "freelance", "requirements": string[], "benefits": string[], "remote_policy": "remote" | "hybrid" | "on-site" | "not-specified" }

Rules:

• Use null for fields that are not mentioned
• requirements and benefits must be arrays (use [] if none mentioned)
• Do not add fields not in the schema
• Do not change field names

This approach fails less than 2% of the time with GPT-4 class models when the schema is clear.

### Approach 3: Delimiters for Extraction (Medium Reliability)

For simpler use cases where you need to extract a specific piece of structured content:

[your structured findings here]

Each bug on a separate line in format: LINE_NUMBER | SEVERITY | DESCRIPTION

Code: [paste code]

### Approach 4: Post-Processing Extraction (Lowest Reliability, But Flexible)

For extraction from freeform text, use a second model call dedicated to extraction:

// Step 2: Extract structure from the analysis const extraction = await client.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'system', content: 'Extract structured data from text and return JSON.' }, { role: 'user', content: 'Extract all bugs from this analysis. Return JSON: { bugs: [{ line: number|null, severity: string, description: string }] }\n\nAnalysis:\n' + analysis } ], response_format: { type: 'json_object' }, });

---

## Designing Your Schema for Reliability

Poorly designed schemas cause extraction failures even when the model is trying to comply.

**Use explicit enums instead of open strings:**

**Make nullable fields explicit:**

**Use arrays for variable-length data:**

**Avoid deeply nested schemas (max 3 levels):**
Deep nesting increases the likelihood of structural errors. Flatten where possible.

---

## Validation and Recovery

Never trust parsed JSON from an AI model without validation. Use Zod:

const JobSchema = z.object({ title: z.string(), company: z.string(), salary_range: z.object({ min: z.number().nullable(), max: z.number().nullable(), currency: z.string(), }), employment_type: z.enum(['full-time', 'part-time', 'contract', 'freelance']), requirements: z.array(z.string()), });

function parseJobPosting(rawJson) { try { const parsed = JSON.parse(rawJson); return JobSchema.parse(parsed); } catch (err) { return repairAndParse(rawJson, err.message); } } ```

This "repair loop" recovers from the rare cases where JSON mode still produces invalid schema.