Building Persistent Azure AI Agents: Setup, Usage & Best Practices

The Microsoft Agent Framework for Azure AI represents a significant leap forward in building production-ready AI agents. Unlike stateless API calls, this framework enables you to create persistent agents that maintain conversation context, integrate multiple tools seamlessly, and scale effortlessly on Azure infrastructure.

If you're building chatbots, virtual assistants, or any application requiring multi-turn conversations with tool integration, this skill is essential for your Azure AI toolkit.

What This Skill Does

The agent-framework-azure-ai-py skill provides a Python SDK for building agents on Azure AI Foundry. At its core, it abstracts away the complexity of managing conversation state, tool orchestration, and streaming responses while giving you full control over agent behavior.

The framework introduces the AzureAIAgentsProvider, which serves as your gateway to Azure's hosted agent service. Unlike managing agent state in your application code, the provider delegates persistence to Azure, ensuring your agents can handle thousands of concurrent conversations without memory overhead on your servers.

One of the framework's standout features is its unified approach to tools. Whether you're exposing Python functions, leveraging hosted capabilities like code execution and web search, or integrating Model Context Protocol (MCP) servers, the SDK treats them all as first-class citizens. You simply pass functions directly to the tools parameter, and the framework handles serialization, validation, and execution routing automatically.

Getting Started

Installation is straightforward via pip. You can install the full framework or just the Azure-specific package depending on your needs:

pip install agent-framework --pre

Before running any code, you'll need an Azure AI Foundry project. Set your project endpoint and model deployment name as environment variables. For web search capabilities, you'll also need a Bing connection configured in your project.

Authentication follows Azure's standard patterns. During development, use AzureCliCredential after running az login. For production deployments, switch to DefaultAzureCredential, which automatically discovers credentials from managed identities, environment variables, or other configured sources.

The simplest agent requires just a few lines of asynchronous code. You create a provider with your credentials, define an agent with a name and instructions, and call agent.run() with your query. The framework handles everything else: sending the request to Azure, processing the response, and returning clean text output.

Key Features

Persistent Conversation Threads: The framework's AgentThread system enables true multi-turn conversations. Create a thread with agent.get_new_thread(), then pass it to subsequent run() calls. The thread maintains full conversation history on Azure's servers, allowing your agent to reference earlier messages naturally without manual context management.

Function Tools Integration: Defining custom tools is remarkably clean. Write standard Python functions with type hints and docstrings, then pass them directly to the agent's tools array. The framework introspects your function signatures and automatically generates OpenAI function calling schemas. Use Pydantic's Annotated and Field for precise parameter descriptions that help the model understand when to invoke your functions.

Hosted Capabilities: Azure AI Foundry provides three powerful hosted tools out of the box. HostedCodeInterpreterTool lets agents write and execute Python code safely in sandboxed containers—perfect for data analysis or calculations. HostedWebSearchTool integrates Bing search without managing API keys yourself. HostedFileSearchTool enables semantic search over uploaded document collections using vector stores managed by Azure.

Streaming Responses: For user-facing applications, streaming is critical. The run_stream() method returns an async iterator yielding response chunks as they're generated. This enables real-time UI updates and significantly improves perceived latency for long responses.

Structured Outputs: When you need data, not prose, define a Pydantic model and pass it to the response_format parameter. The agent will return JSON conforming to your schema, which you can parse and validate in one line. This is invaluable for building agents that feed into downstream systems or databases.

Usage Examples

A weather agent demonstrates function tool integration beautifully. Define a get_weather() function that takes a location parameter with an Annotated type hint describing its purpose. When users ask about weather, the agent automatically invokes your function and incorporates the result into its response.

For research tasks, combine multiple hosted tools. An agent with code interpreter, web search, and MCP tools can search the internet for information, analyze data with Python, and query specialized documentation servers—all in a single conversation flow.

Multi-turn conversations showcase thread persistence. Create a thread, ask about Seattle's weather, then follow up with "What about Portland?" The agent understands the context from the thread without repeating the original query.

Best Practices

Always use async context managers (async with) when creating providers and MCP tools. This ensures proper resource cleanup and connection pooling.

For function parameters, be explicit with type hints and descriptions. The more context you provide through Field(description=...), the better the model understands when and how to use your functions.

When building production agents, implement proper error handling around agent creation and execution. Network issues, quota limits, and API changes can all cause failures that should be handled gracefully.

Store conversation thread IDs in your application's database if you need to resume conversations across user sessions. The thread ID is the only state you need to persist on your side.

Start with simple, single-purpose agents and add complexity gradually. Multi-tool agents require careful instruction design to prevent the model from invoking tools unnecessarily or in the wrong sequence.

When to Use This Skill

This skill excels when building customer support chatbots, virtual assistants, or any application requiring stateful conversations with tool access. If your use case involves multi-turn interactions where context matters, Azure AI Agents solve the persistence problem elegantly.

It's also ideal when you need Azure's hosted tools. Code execution, file search, and web search are production-ready capabilities that would take significant effort to build and secure yourself.

For teams already invested in Azure, this framework provides native integration with Azure AI services, managed identities, and enterprise security features like virtual networks and private endpoints.

When NOT to Use This Skill

For simple, stateless question-answering, calling Azure OpenAI directly is more straightforward and cheaper. The agent framework's value comes from persistence and tool orchestration—features you don't need for one-shot queries.

If you require sub-second response times, the framework's network round trips to Azure may introduce too much latency. Consider local inference or custom orchestration for latency-critical applications.

Teams working exclusively in languages other than Python should explore the .NET or Java agent SDKs instead. While the patterns are similar, this particular skill is Python-specific.

Related Skills

• azure-ai-projects-py - Core Azure AI Projects SDK for broader service integration
• azure-ai-openai-dotnet - .NET developers building on Azure OpenAI
• azure-ai-projects-java - Java implementation of Azure AI project management

Source

This skill is maintained by Microsoft. View on GitHub