Build an AI agent
Check whether a task needs an agent at all, then choose the lightest Microsoft route that fits: a ready-made agent, Agent Builder, Copilot Studio, Foundry Agent Service, or code with Microsoft Agent Framework.
The problem
Teams often start building an agent before asking whether the task needs one. Agents add moving parts: a model, tools, memory, identity, monitoring and cost per call. For many requests a fixed workflow, a search box over documents or an existing Microsoft agent would do the job with less risk.
When an agent is justified, the Microsoft options overlap. Agent Builder, Copilot Studio, Foundry Agent Service, Microsoft Agent Framework and the Microsoft 365 Agents SDK differ in who builds, where the agent runs, how much code you own and how usage is billed.12
The approach
Start with Microsoft's own filter. The Cloud Adoption Framework says that clear, rule-based steps are better served by regular code or nongenerative AI, and that answering questions from a fixed set of documents is a classic retrieval (RAG) job that does not need an agent. Agents fit work with multistep decisions, many tools or systems, or messy input. The Agent Framework docs make the same point: if a function can handle the task, write the function.
Next, ask whether a ready-made agent already covers the need, such as Researcher or Workflows in Microsoft 365 Copilot, GitHub Copilot agents, Fabric data agents, Dynamics 365 agents or Security Copilot agents. Build only when none fits.
Build options, from least to most code:
- Agent Builder: no-code declarative agents inside Microsoft Copilot.
- Copilot Studio: low-code agents and workflows with prebuilt and custom connectors; the chosen harness affects reasoning, capabilities and billing.
- Foundry Agent Service: prompt agents defined by configuration, or hosted agents that run your own code (Agent Framework, LangGraph, OpenAI Agents SDK, Anthropic Agent SDK, GitHub Copilot SDK) as a container. Microsoft states the cost model as per-call inference plus tool usage, plus container compute for hosted agents.
- Microsoft 365 Agents SDK: custom engine agents with your own orchestration and models across Microsoft 365 and other channels.
- GPUs and containers: when you run your own model weights; Microsoft suggests Azure Container Apps with serverless GPU for small models and AKS for large ones.
Microsoft suggests time-boxed prototypes of one to two weeks per option and says to proceed with low-code if it meets functional and security requirements. Start with a single agent unless the use case crosses security or compliance boundaries, involves several teams or has known growth.13456
Conceptual architecture
Diagram as a list
Copilot & business apps
- Channels: Microsoft Copilot, Teams, website or appWhere users talk to the agentConnects to Agent runtime (Copilot Studio or Foundry Agent Service)
Agents & automation
- Agent runtime (Copilot Studio or Foundry Agent Service)Runs conversations, tool calls and agent lifecycle
AI models & platform
- Foundry ModelsModels the agent reasons with
Data & analytics
- Knowledge: Foundry IQ, SharePoint or other sourcesGrounding content with citations
Developer tools
- Microsoft Agent Framework (optional)Code-first agents and graph-based workflows for hosted agentsConnects to Agent runtime (Copilot Studio or Foundry Agent Service)
- Tools: MCP servers, connectors, OpenAPI, Azure FunctionsActions the agent can take in other systems
- Tracing, evaluations and monitoringShows each decision and measures qualityConnects to Agent runtime (Copilot Studio or Foundry Agent Service)
Security, identity & governance
- Microsoft Entra agent identity and RBACScoped access without shared credentialsConnects to Agent runtime (Copilot Studio or Foundry Agent Service)
Technologies and their roles
Microsoft Copilot Studio6
Low-code build path
Makers build agents and workflows with connectors and publish to Teams, Microsoft Copilot and websites.
Microsoft Foundry Agent Service5
Managed agent runtime
Hosts prompt agents and code-based hosted agents with identity, networking, tracing and publishing.
Microsoft Agent Framework3
Code framework
Open-source SDK for agents and multi-agent workflows; successor to Semantic Kernel and AutoGen.
Microsoft 365 Agents SDK2
Custom engine agents in Microsoft 365
For pro-code agents with their own orchestration that reach Microsoft 365 Copilot, Teams and other channels.
Microsoft Foundry Models5
Model choice
Catalog models can be swapped without changing agent code.
What you need first14
- A use case that passed the "do we need an agent" check, with baseline measures to compare against
- Named owners for the agent, its data sources and its tools
- Access to the systems the agent must read or act on, through connectors, APIs or MCP servers
- For Foundry: an Azure subscription and a Foundry project, with standard setup if private networking is required
- An evaluation set of realistic questions and expected outcomes
Risks and how to reduce them
- An agent is built for work that a function or simple search would handle1
- Apply the Cloud Adoption Framework checks first and pilot the hardest steps before investing.
- Prompt injection or unsafe output5
- Use Foundry content filters and guardrails, limit tools to what the task needs and keep humans in the loop for consequential actions.
- Costs grow with usage
- Model cost per call and per tool, monitor with service metrics and set decision gates based on business results.
- Quality drifts after changes5
- Trace every run, keep an evaluation set and rerun evaluations before each new version.
- Multi-agent designs add complexity too early4
- Start with one agent and move to multiple agents only for security boundaries, separate teams or known growth.
Documented examples
Novo Nordisk A/S
ScalingNovo Nordisk: a governed reasoning agent on clinical trial data
50+ now, up from 5 to 10 (measured)
Life sciences, Denmark
C.H. Robinson
In productionC.H. Robinson: AI agents for freight quotes, orders and tracking
from hours to an average of 32 seconds per quote (measured)
Logistics and transportation, United States (global operations)
Chow Tai Fook Jewellery Group
In productionChow Tai Fook: 400+ AI agents for jewelry retail staff
over 400 agents (measured)
Retail and consumer goods, Hong Kong (story dateline); store network across several markets
Related
Sources
- Business plan for AI agents (Cloud Adoption Framework) (opens in a new tab)
- Choose development tools for your Microsoft 365 Copilot plugin (opens in a new tab)
- Microsoft Agent Framework overview (opens in a new tab)
- Technology plan for AI agents (Cloud Adoption Framework) (opens in a new tab)
- What is Microsoft Foundry Agent Service? (opens in a new tab)
- Overview: Microsoft Copilot Studio (opens in a new tab)
Thank you. Your correction was sent.
The editors check it against the sources. If you left an email address, they may reply about it.