The Semantic Kernel Agent Framework provides abstractions for building autonomous AI agents capable of taking actions, making decisions, and working together to achieve specific goals. Agents can be configured with different instructions, capabilities, and plugins to specialize them for particular tasks. An agent in the Semantic Kernel Agent Framework is built around a kernel instance that provides access to LLM capabilities, function execution, and plugin management.

Learn about:

• Building AI Agent using Semantic Kernel Agent Framework
• Building Multi‑Agent AI Workflows with Semantic Kernel Agent Framework in .NET

The Problem: Unintended Plugin Sharing Between Agents

When building multiple agents with Semantic Kernel, developers often encounter an unexpected issue: agents sharing access to all plugins regardless of explicit assignments. The problem occurs when multiple agents are created using the same kernel instance. Let’s illustrate this with a simplified example. Imagine we’re building a restaurant application with two agents:

• A MenuAgent that answers questions about food items
• A ToolAgent that manages kitchen equipment

// Create a single kernel
var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
    deploymentName: deploymentName,
    endpoint: endpoint,
    apiKey: apiKey);
Kernel kernel = builder.Build();

// Create first agent (MenuAgent)
ChatCompletionAgent MenuAgent = new() {
    Instructions = "Answer questions about the menu.",
    Name = "Menu_Agent",
    Kernel = kernel,
    Arguments = new KernelArguments(new PromptExecutionSettings() {
        FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
    }),
};

// Add menu-related plugin to MenuAgent
KernelPlugin menuPlugin = KernelPluginFactory.CreateFromType<MenuPlugin>();
MenuAgent.Kernel.Plugins.Add(menuPlugin);

// Create second agent (ToolAgent)
ChatCompletionAgent ToolAgent = new() {
    Instructions = "An agent that is configured to auto-invoke plugins.",
    Name = "Tool_Agent", 
    Kernel = kernel,
    Arguments = new KernelArguments(new PromptExecutionSettings() {
        FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
    }),
};

// Add kitchen tools plugin to ToolAgent
KernelPlugin widgetFactory = KernelPluginFactory.CreateFromType<WidgetFactory>();
ToolAgent.Kernel.Plugins.Add(widgetFactory);

What We Expected:

• MenuAgent would only have access to the MenuPlugin
• ToolAgent would only have access to the WidgetFactory plugin

What Actually Happens:

• MenuAgent has access to BOTH MenuPlugin AND WidgetFactory
• ToolAgent has access to BOTH WidgetFactory AND MenuPlugin

Why Does This Happen?

By using the same kernel instance for both agents, we’ve created a situation where they share the same plugin collection. When we add the menuPlugin to the menuAgent.Kernel.Plugins collection, we’re actually adding it to the shared kernel’s plugin collection. Similarly, when we add the widgetFactory to the toolAgent.Kernel.Plugins collection, it’s added to the same shared collection.

The result? Both agents gain access to all plugins from both domains. The Menu Agent can now access widget factory functions, and the Tool Agent can access menu-related functions. This occurs because the Kernel property of each agent references the same kernel object in memory and the same plugin collection. It’s a subtle but significant issue that can lead to unexpected behavior, as agents may invoke functions they weren’t intended to access.

Solutions: Isolating Your Agents

There are two effective approaches to solve this problem:

Solution 1: Clone the Kernel for Each Agent

Semantic Kernel provides a Clone() method that creates a copy of the kernel with the same AI service configurations but with separate plugin collections.

// Create a base kernel with AI service configurations
var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
    deploymentName: deploymentName,
    endpoint: endpoint,
    apiKey: apiKey);
Kernel baseKernel = builder.Build();

// Create first agent with a cloned kernel
ChatCompletionAgent MenuAgent = new() {
    Instructions = "Answer questions about the menu.",
    Name = "Menu_Agent",
    Kernel = baseKernel.Clone(),
    Arguments = new KernelArguments(new PromptExecutionSettings() {
        FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
    }),
};

// Add menu-related plugin to MenuAgent
KernelPlugin menuPlugin = KernelPluginFactory.CreateFromType<MenuPlugin>();
MenuAgent.Kernel.Plugins.Add(menuPlugin);

// Create second agent with a different cloned kernel
ChatCompletionAgent ToolAgent = new() {
    Instructions = "An agent that is configured to auto-invoke plugins.",
    Name = "Tool_Agent", 
    Kernel = baseKernel.Clone(),
    Arguments = new KernelArguments(new PromptExecutionSettings() {
        FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
    }),
};

// Add kitchen tools plugin to ToolAgent
KernelPlugin widgetFactory = KernelPluginFactory.CreateFromType<WidgetFactory>();
ToolAgent.Kernel.Plugins.Add(widgetFactory);

This approach ensures each agent has its own separate kernel instance while maintaining the same configuration for AI services.

Solution 2: Create Entirely New Kernel Instances

Alternatively, you can create completely separate kernel instances for each agent:

// Create first kernel and agent
var builder1 = Kernel.CreateBuilder();
builder1.AddAzureOpenAIChatCompletion(
    deploymentName: deploymentName,
    endpoint: endpoint,
    apiKey: apiKey);
Kernel kernel1 = builder1.Build();

ChatCompletionAgent MenuAgent = new() {
    Instructions = "Answer questions about the menu.",
    Name = "Menu_Agent",
    Kernel = kernel1,
    Arguments = new KernelArguments(new PromptExecutionSettings() {
        FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
    }),
};

// Add menu-related plugin to MenuAgent
KernelPlugin menuPlugin = KernelPluginFactory.CreateFromType<MenuPlugin>();
MenuAgent.Kernel.Plugins.Add(menuPlugin);

// Create second kernel and agent
var builder2 = Kernel.CreateBuilder();
builder2.AddAzureOpenAIChatCompletion(
    deploymentName: deploymentName,
    endpoint: endpoint,
    apiKey: apiKey);
Kernel kernel2 = builder2.Build();

ChatCompletionAgent ToolAgent = new() {
    Instructions = "An agent that is configured to auto-invoke plugins.",
    Name = "Tool_Agent", 
    Kernel = kernel2,
    Arguments = new KernelArguments(new PromptExecutionSettings() {
        FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
    }),
};

// Add kitchen tools plugin to ToolAgent
KernelPlugin widgetFactory = KernelPluginFactory.CreateFromType<WidgetFactory>();
ToolAgent.Kernel.Plugins.Add(widgetFactory);

This approach gives you complete separation between agents, though it requires duplicating the kernel configuration code.

Conclusion

When building multiple agents with Microsoft Semantic Kernel, it’s crucial to understand that the plugins are associated with the kernel, not directly with the agents. To maintain proper isolation between agents and their respective plugins:

1. Never share a single kernel instance across multiple agents if you want them to have different plugin sets
2. Use either the Clone() method or create entirely new kernel instances for each agent
3. Be explicit about which plugins should be available to which agents

By following these best practices, you’ll avoid unexpected behavior in your Semantic Kernel applications and ensure your agents perform exactly as intended, with access only to the plugins they need.