OpenAI’s API offers powerful features like JSON Mode and Structured Outputs to ensure that the model’s responses are in a structured and predictable format. When you’re designing complex workflows, automating business processes, or integrating AI into enterprise software, these features are game changers.

JSON Mode ensures the flexibility of machine-readable responses, while Structured Outputs guarantee consistency by adhering to a strict schema. Together, they offer a solid solution for developers who need to trust that the data generated by the AI will fit into their applications.

But why JSON?

JSON is a lightweight format that is easy for humans to read and write, and easy for machines to parse and generate.

JSON Mode ensures that the model’s outputs are valid JSON. This is beneficial for applications that require structured data, as it reduces the chances of errors in data formatting. However, JSON Mode does not guarantee that the output will match a specific schema.

This mode is useful in scenarios where the structure of the data is not strictly defined, like asking a chatbot to generate JSON.

This example showcases JSON mode, ensuring the model’s output is valid JSON:

curl https://api.openai.com/v1/chat/completions \
 -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
 -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
 {"role": "system", "content": "You are a gaming bot. Respond in JSON format with these keys: 'game_type', 'platform', and 'popularity'."},
 {"role": "user", "content": "Tell me about a popular battle royale game."}
 ],
    "response_format": {"type": "json_object"}
 }'

• response_format: Determines the format of the output.
• type: "json_object": Enables JSON mode for the response.
• Explicit Instructions: Either the system message or user message must explicitly mention json when using JSON mode.

Structured Outputs

Structured Outputs take this a step further by ensuring that the model’s responses exactly match a developer-supplied JSON schema ensuring that the data returned by the model is consistent every time. This is crucial for applications that rely on predictable data formats.

As of today, the OpenAI models supported are:

• gpt-4o-mini-2024-07-18 and later
• gpt-4o-2024-08-06 and later

When using Azure OpenAI  only the gpt-4o model is currently supported

curl https://api.openai.com/v1/chat/completions \
 -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
 -d '{
    "model": "gpt-4o-2024-08-06",
    "messages": [
 {"role": "system", "content": "Extract the game name, genre, and price of this game."},
 {"role": "user", "content": "The game 'Legend of the Arena' is an action RPG and costs $59.99."}
 ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "type": "object",
        "properties": {
          "game_name": {"type": "string"},
          "genre": {"type": "string"},
          "price": {"type": "number"}
 },
        "required": ["game_name", "genre", "price"]
 }
 }
 }'

• response_format: This parameter is crucial for Structured Outputs.
• type: "json_schema": Indicates that you’re providing a JSON schema.
• json_schema: This field would contain your JSON schema defining the structure of the desired output.

When to use JSON Mode vs Structured Outputs?

  Structured Outputs are the recommended mode whenever possible. It is best for integrating the model’s response with business processes where a specific JSON schema is required. This ensures that the data fits seamlessly into existing workflows and systems.

JSON Mode can be used for user-facing applications where the user needs a JSON response, such as chatbots, or when the model doesn’t support Structured Outputs.

Limitations of Structured Outputs

Structured Outputs, while offering advantages over JSON mode, still have edge cases and limitations you should be aware of:

• Incomplete Responses: Situations like reaching the max_tokens limit or the model’s output being cut off due to stop tokens can lead to incomplete JSON responses.

• Model Refusals: When dealing with user-generated input, OpenAI models might refuse to fulfill requests deemed unsafe. These refusals may not conform to your defined schema. The API response includes a “refusal” field to indicate this. You should incorporate logic to handle refusals gracefully, potentially displaying the refusal message in your UI or adjusting the request based on the refusal reason.

•  Hallucinations: While Structured Outputs aim for schema adherence, the model might still hallucinate values, particularly if the input is significantly unrelated to the schema.

You can check out the exact details listed on OpenAI docs

Conclusion

OpenAI’s JSON Mode and Structured Outputs are powerful features that significantly enhance the reliability and usability of model outputs.   By ensuring that responses are in valid JSON format and conform to specific schemas, these features provide developers with the structured data they need for building robust and efficient applications.

Structured Outputs simplify the process of integrating AI-generated data into various systems, making it easier to develop APIs, handle data interchange, and create seamless third-party integrations.