Using Structured Outputs with Semantic Kernel

Semantic Kernel is a .NET framework for working with LLMs (large language models). This post covers the new Structured Output option when using Semantic Kernel with OpenAI or Azure OpenAI models.

We’ll focus on:

Using JSON schema mode
Working with Structured outputs

These features help convert unstructured text into organized data formats, making the output more reliable and easier to process.

Introduction to Structured Outputs

Structured Outputs is the next evolution over the JSON schema option. While JSON schema guarantees a valid JSON response, Structured outputs take it a step further guaranteeing a consistent schema.

When working with AI models, getting structured, predictable responses is crucial for building reliable applications. Semantic Kernel makes this easy by combining C# classes with JSON schema mode. These capabilities are invaluable when you need to extract structured information from unstructured text, making data processing and analysis significantly more reliable.

Setting Up the Kernel

Let’s start by creating a basic Semantic Kernel setup in a .NET console application.

First, ensure you have the necessary Azure OpenAI credentials. Alternatively, you could also use the OpenAI models using the AddOpenAIChatCompletion method.

var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
    "your-deployment-name",  // The name of your model deployment
    "your-endpoint",         // Your Azure OpenAI endpoint
    "your-api-key"          // Your Azure OpenAI API key
);
var kernel = builder.Build();
var chatService = kernel.GetRequiredService<IChatCompletionService>();

Important Note: Before proceeding, verify that your Azure OpenAI model supports JSON mode and Structured outputs. Not all models have this capability. You can check the compatibility of your model at the Azure OpenAI Models Documentation.

Working with JSON Schema Mode

Let’s explore how to use JSON Schema Mode with a practical example. We’ll process a product review and convert it into structured JSON data.

First, configure the execution settings to enable JSON mode:

var executionSettings = new AzureOpenAIPromptExecutionSettings
{
    ResponseFormat = "json_object"
};

Now, let’s process a sample product review:


var reviewText = @"Neha Verma
5.0 out of 5 stars Sony WH-1000XM4 are the best headphones I've ever used
Reviewed in India on 1 October 2024
Verified Purchase
These headphones are simply amazing. The noise cancellation is top-notch, and the sound quality is superb. 
Battery life lasts 30 hours. Bit expensive but worth it. Comfortable for long wear.
25 people found this helpful";

var response = await chatService.GetChatMessageContentAsync($"Convert the given review to Json REVIEW:{reviewText}",executionSettings);

Console.WriteLine(response.ToString());

The model will return a JSON response similar to this:

{
  "reviewerName": "Neha Verma",
  "rating": 5.0,
  "title": "Sony WH-1000XM4 are the best headphones I've ever used",
  "reviewDate": "1 October 2024",
  "location": "India",
  "styleName": "WH-1000XM4",
  "verifiedPurchase": true,
  "reviewText": "These headphones are simply amazing. The noise cancellation is top-notch, and the sound quality is superb. They are very comfortable to wear, even for long periods. The battery life is excellent, lasting up to 30 hours on a single charge. They are a bit expensive, but worth every penny. Highly recommend for anyone looking for high-quality headphones!",
  "helpfulCount": 25
}

Note that since we haven’t mentioned anything in the prompt this structure will change every time we make a new call. Of course, we can provide a structure in the prompt but this still doesn’t guarantee we would always get the same structure every time. This is where Structured Outputs come in handy.

Using Structured Outputs

Now, let’s see how we can use Structured Outputs for the same input.

Semantic Kernel provides two approaches for implementing Structured Outputs. Let’s see both methods:

Method 1: Using C# Classes

First, define a C# class that represents your desired data structure:

public class ProductReview
{
    public string ReviewerName { get; set; }
    public decimal Rating { get; set; }
    public string ProductName { get; set; }
    public string ReviewText { get; set; }
    public int HelpfulVotes { get; set; }
    public string Location { get; set; }
    public bool IsVerifiedPurchase { get; set; }
    public List<string> KeyFeatures { get; set; }
}

Then we can directly pass the model type to our response format for enabling structured outputs.

var executionSettings = new AzureOpenAIPromptExecutionSettings
{
    ResponseFormat = typeof(ProductReview)
};

Method 2: Using JSON Schema

Alternatively, you can define a JSON schema that specifies the exact structure you want without defining a new model.

string chatResponseFormat = """
 {
 "type": "object",
 "properties": {
 "ProductReviews": {
 "type": "array",
 "items": {
 "type": "object",
 "properties": {
 "ReviewerName": { "type": "string" },
 "Rating": { "type": "number" },
 "ProductName": { "type": "string" },
 "ReviewText": { "type": "string" },
 "HelpfulVotes": { "type": "integer" },
 "Location": { "type": "string" },
 "IsVerifiedPurchase": { "type": "boolean" },
 "KeyFeatures": { "type": "array", "items": { "type": "string" } }
 },
 "required": [
 "ReviewerName", 
 "Rating", 
 "ProductName", 
 "ReviewText", 
 "HelpfulVotes", 
 "Location", 
 "IsVerifiedPurchase", 
 "KeyFeatures"
 ],
 "additionalProperties": false
 }
 }
 },
 "required": ["ProductReviews"],
 "additionalProperties": false
 }
 """

To use the schema, create a chat response format and apply it to your execution settings:

ChatResponseFormat chatResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat(
    jsonSchemaFormatName: "product_review",
    jsonSchema: BinaryData.FromString(productReviewSchema),
    jsonSchemaIsStrict: true);
    
var executionSettings = new AzureOpenAIPromptExecutionSettings
{
    ResponseFormat = chatResponseFormat
};

Now we can call our model using the selected approach.

var response = await chatService.GetChatMessageContentAsync($"Convert the given review to Json REVIEW:{reviewText}", executionSettings);

Console.WriteLine(response.ToString());

The resulting output will consistently match your defined structure:

{
  "ReviewerName": "Neha Verma",
  "Rating": 5,
  "ProductName": "Sony WH-1000XM4",
  "ReviewText": "These headphones are simply amazing. The noise cancellation is top-notch, and the sound quality is superb. They are very comfortable to wear, even for long periods. The battery life is excellent, lasting up to 30 hours on a single charge. They are a bit expensive, but worth every penny. Highly recommend for anyone looking for high-quality headphones!",
  "HelpfulVotes": 25,
  "Location": "India",
  "IsVerifiedPurchase": true,
  "KeyFeatures": [
    "Noise cancellation",
    "Superb sound quality",
    "Comfortable for long wear",
    "Excellent battery life"
 ]
}

Conclusion

Structured Outputs in Semantic Kernel provide a reliable way to work with AI-generated content by ensuring consistent and predictable responses. Whether you use C# classes or JSON schemas, these tools allow you to define the exact structure your application needs, making it easier to process and manage data. This eliminates the unpredictability of unstructured outputs and helps you build more robust applications.

Make sure to stay updated with Azure OpenAI’s documentation to take full advantage of the latest features and model updates as they are released.