IntelligentHQ
IntelligentHQ
Skip to content

Build an AI agent

Check whether a task needs an agent at all, then choose the lightest Microsoft route that fits: a ready-made agent, Agent Builder, Copilot Studio, Foundry Agent Service, or code with Microsoft Agent Framework.

Try the Agent Platform Chooser

The problem

Teams often start building an agent before asking whether the task needs one. Agents add moving parts: a model, tools, memory, identity, monitoring and cost per call. For many requests a fixed workflow, a search box over documents or an existing Microsoft agent would do the job with less risk.

When an agent is justified, the Microsoft options overlap. Agent Builder, Copilot Studio, Foundry Agent Service, Microsoft Agent Framework and the Microsoft 365 Agents SDK differ in who builds, where the agent runs, how much code you own and how usage is billed.12

The approach

Start with Microsoft's own filter. The Cloud Adoption Framework says that clear, rule-based steps are better served by regular code or nongenerative AI, and that answering questions from a fixed set of documents is a classic retrieval (RAG) job that does not need an agent. Agents fit work with multistep decisions, many tools or systems, or messy input. The Agent Framework docs make the same point: if a function can handle the task, write the function.

Next, ask whether a ready-made agent already covers the need, such as Researcher or Workflows in Microsoft 365 Copilot, GitHub Copilot agents, Fabric data agents, Dynamics 365 agents or Security Copilot agents. Build only when none fits.

Build options, from least to most code:

  • Agent Builder: no-code declarative agents inside Microsoft Copilot.
  • Copilot Studio: low-code agents and workflows with prebuilt and custom connectors; the chosen harness affects reasoning, capabilities and billing.
  • Foundry Agent Service: prompt agents defined by configuration, or hosted agents that run your own code (Agent Framework, LangGraph, OpenAI Agents SDK, Anthropic Agent SDK, GitHub Copilot SDK) as a container. Microsoft states the cost model as per-call inference plus tool usage, plus container compute for hosted agents.
  • Microsoft 365 Agents SDK: custom engine agents with your own orchestration and models across Microsoft 365 and other channels.
  • GPUs and containers: when you run your own model weights; Microsoft suggests Azure Container Apps with serverless GPU for small models and AKS for large ones.

Microsoft suggests time-boxed prototypes of one to two weeks per option and says to proceed with low-code if it meets functional and security requirements. Start with a single agent unless the use case crosses security or compliance boundaries, involves several teams or has known growth.13456

Conceptual architecture

Build an AI agent: conceptual architectureCopilot & businessappsAgents & automationAI models & platformData & analyticsDeveloper toolsSecurity, identity &governanceChannels: Microsoft Copilot, Teams, website or app: Where users talk to the agentChannels: Microsoft Copilot,Teams, website or appAgent runtime (Copilot Studio or Foundry Agent Service): Runs conversations, tool calls and agent lifecycleAgent runtime (CopilotStudio or Foundry Agent…Foundry Models: Models the agent reasons withFoundry ModelsKnowledge: Foundry IQ, SharePoint or other sources: Grounding content with citationsKnowledge: Foundry IQ,SharePoint or other sourcesMicrosoft Agent Framework (optional): Code-first agents and graph-based workflows for hosted agentsMicrosoft Agent Framework(optional)Tools: MCP servers, connectors, OpenAPI, Azure Functions: Actions the agent can take in other systemsTools: MCP servers,connectors, OpenAPI, Azure…Tracing, evaluations and monitoring: Shows each decision and measures qualityTracing, evaluations andmonitoringMicrosoft Entra agent identity and RBAC: Scoped access without shared credentialsMicrosoft Entra agentidentity and RBAC
Diagram as a list

  1. Copilot & business apps

    • Channels: Microsoft Copilot, Teams, website or appWhere users talk to the agentConnects to Agent runtime (Copilot Studio or Foundry Agent Service)

  2. Agents & automation

    • Agent runtime (Copilot Studio or Foundry Agent Service)Runs conversations, tool calls and agent lifecycle

  3. AI models & platform

    • Foundry ModelsModels the agent reasons with

  4. Data & analytics

    • Knowledge: Foundry IQ, SharePoint or other sourcesGrounding content with citations

  5. Developer tools

    • Microsoft Agent Framework (optional)Code-first agents and graph-based workflows for hosted agentsConnects to Agent runtime (Copilot Studio or Foundry Agent Service)
    • Tools: MCP servers, connectors, OpenAPI, Azure FunctionsActions the agent can take in other systems
    • Tracing, evaluations and monitoringShows each decision and measures qualityConnects to Agent runtime (Copilot Studio or Foundry Agent Service)

  6. Security, identity & governance

    • Microsoft Entra agent identity and RBACScoped access without shared credentialsConnects to Agent runtime (Copilot Studio or Foundry Agent Service)
Conceptual: one common way to arrange the parts, not a required design.5

Technologies and their roles

  • Microsoft Copilot Studio6

    Low-code build path

    Makers build agents and workflows with connectors and publish to Teams, Microsoft Copilot and websites.

  • Microsoft Foundry Agent Service5

    Managed agent runtime

    Hosts prompt agents and code-based hosted agents with identity, networking, tracing and publishing.

  • Microsoft Agent Framework3

    Code framework

    Open-source SDK for agents and multi-agent workflows; successor to Semantic Kernel and AutoGen.

  • Microsoft 365 Agents SDK2

    Custom engine agents in Microsoft 365

    For pro-code agents with their own orchestration that reach Microsoft 365 Copilot, Teams and other channels.

  • Microsoft Foundry Models5

    Model choice

    Catalog models can be swapped without changing agent code.

What you need first14

  • A use case that passed the "do we need an agent" check, with baseline measures to compare against
  • Named owners for the agent, its data sources and its tools
  • Access to the systems the agent must read or act on, through connectors, APIs or MCP servers
  • For Foundry: an Azure subscription and a Foundry project, with standard setup if private networking is required
  • An evaluation set of realistic questions and expected outcomes

Risks and how to reduce them

An agent is built for work that a function or simple search would handle1
Apply the Cloud Adoption Framework checks first and pilot the hardest steps before investing.
Prompt injection or unsafe output5
Use Foundry content filters and guardrails, limit tools to what the task needs and keep humans in the loop for consequential actions.
Costs grow with usage
Model cost per call and per tool, monitor with service metrics and set decision gates based on business results.
Quality drifts after changes5
Trace every run, keep an evaluation set and rerun evaluations before each new version.
Multi-agent designs add complexity too early4
Start with one agent and move to multiple agents only for security boundaries, separate teams or known growth.

Documented examples

Related

Sources

  1. Business plan for AI agents (Cloud Adoption Framework) (opens in a new tab)Microsoft · Vendor-reported
  2. Choose development tools for your Microsoft 365 Copilot plugin (opens in a new tab)Microsoft · Vendor-reported
  3. Microsoft Agent Framework overview (opens in a new tab)Microsoft · Vendor-reported
  4. Technology plan for AI agents (Cloud Adoption Framework) (opens in a new tab)Microsoft · Vendor-reported
  5. What is Microsoft Foundry Agent Service? (opens in a new tab)Microsoft · Vendor-reported
  6. Overview: Microsoft Copilot Studio (opens in a new tab)Microsoft · Vendor-reported

Fill out the form below to request your copy.

Name(Required)