40. Advanced Structured Output

The Complexity Cliff

Getting a flat JSON object is easy. Getting a recursive tree of objects with polymorphic types (where `items` can be `TextItem` or `ImageItem`) is hard. Models often mess up nesting depth or forget required fields in deep children.

Expert Strategies

• Schema Decomposition: Don't ask for the whole tree. Ask for the top level, then ask for the children of Node A, then Node B. Assemble it in code.
• Grammar-Constrained Decoding: Using tools that force the model's output tokens to strictly adhere to a BNF grammar (some providers support this natively).
• Error Reflection: If the JSON fails to parse, feed the error message back to the model: "You generated invalid JSON. Error at line 5: ... Fix it."

Where to go next

• 40.1 Deep schemas: nested objects and arrays
• 40.2 “Explain errors as JSON” for debuggability
• 40.3 Schema evolution and backwards compatibility
• 40.4 Internationalization schemas (multi-locale outputs)
• 40.5 Contract tests for schema compliance