In the previous blog, we looked at what function calling is and how we can use it in our applications with the help of auto function calling. Now, let’s see how to use the manual mode.

We will be working with the same example from the previous blog but with one key difference. Instead of ToolCallBehavior.AutoInvokeKernelFunctions:

OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };

We use ToolCallBehavior.EnableKernelFunctions to manually handle function calling in our application. This setting allows us to manually control how to handle function calls in our application.

settings = new OpenAIPromptExecutionSettings() { ToolCallBehavior = ToolCallBehavior.EnableKernelFunctions };

So now, when can we invoke the method GetChatMessageContentAsync along with these settings and plugin.

var manualResult = (OpenAIChatMessageContent)await chatService.GetChatMessageContentAsync(chat, settings, kernel);

We have to manually check for requested function calls through the LLM response json. The toolCalls property contains the list of functions required by the AI models to complete a response along with any arguments that the function needs.

The AI model is usually smart enough to be able to extract arguments from the prompt or the chat history to pass them as parameters in our functions. In some cases, a model can call one or more functions. We can handle these function calls manually by iterating over the ToolCalls property of the result:

List<ChatCompletionsFunctionToolCall> toolCalls = manualResult.ToolCalls.OfType<ChatCompletionsFunctionToolCall>().ToList();

foreach (var toolCall in toolCalls)
{
	string content = kernel.Plugins.TryGetFunctionAndArguments(toolCall, out KernelFunction? function, out KernelArguments? arguments)
					? JsonSerializer.Serialize((await function.InvokeAsync(kernel, arguments)).GetValue<object>())
					: "No Function Found";

	chat.Add(new ChatMessageContent(AuthorRole.Tool, content, metadata: new Dictionary<string, object?>(1) { { OpenAIChatMessageContent.ToolIdProperty, toolCall.Id } }));
}

Do note that the model do not know if a function exists or not in our application. Nor does it have any idea how the function works internally. The function calls that the model makes are based on the descriptions of the function and parameters provided to the model.

So if we pass such nonexistent functions to the kernel, the model will call these functions expecting a response. Along with this although rare, the model may also hallucinate functions that don’t exist.

Through manual function calling we can add our business logic like authorizing or validating the arguments before actually calling the function. This way, we have more granular control over each function call, improving the overall execution flow of our application.

Here’s the combined code for using the manual mode in semantic kernel from building a kernel to handling any function call requested by the AI model.

IKernelBuilder builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion("<Model-Name>", "<API-Key>");
Kernel kernel = builder.Build();

kernel.ImportPluginFromFunctions("TimePlugin", "Get time related info", new[] {
	kernel.CreateFunctionFromMethod(GetCurrentTime, nameof(GetCurrentTime), "Retrieves the current time in UTC.")
});

string prompt = "What time it is?";

ChatHistory chat = new ChatHistory();
chat.AddUserMessage(prompt);

IChatCompletionService chatService = kernel.GetRequiredService<IChatCompletionService>();
OpenAIPromptExecutionSettings settings = new OpenAIPromptExecutionSettings() { ToolCallBehavior = ToolCallBehavior.EnableKernelFunctions };

while (true)
{
var manualResult = (OpenAIChatMessageContent)await chatService.GetChatMessageContentAsync(chat, settings, kernel);

if (manualResult.Content is not null)
{
	Console.WriteLine(manualResult.Content);
}

List<ChatCompletionsFunctionToolCall> toolCalls = manualResult.ToolCalls.OfType<ChatCompletionsFunctionToolCall>().ToList();
if (toolCalls.Count == 0)
{
	break;
}

chat.Add(manualResult);
foreach (var toolCall in toolCalls)
{
	string content = kernel.Plugins.TryGetFunctionAndArguments(toolCall, out KernelFunction? function, out KernelArguments? arguments)
					? JsonSerializer.Serialize((await function.InvokeAsync(kernel, arguments)).GetValue<object>())
					: "No Function Found";

	chat.Add(new ChatMessageContent(AuthorRole.Tool, content, metadata: new Dictionary<string, object?>(1) { { OpenAIChatMessageContent.ToolIdProperty, toolCall.Id } }));
}
}

Wrapping up

As more and more advanced base models are released we will see more such skills available in these AI models. But rather than trying to fit everything in one model, we can expect a more modular approach where we could customize our model mixing and matching various modules to fit our needs. Semantic kernel follows a similar modular approach allowing developers to use exactly what we need.