IntelligentHQ
IntelligentHQ
Skip to content

Build custom AI models

Pick the least custom option that meets quality: prebuilt Foundry Tools, fine-tuning foundation models in Microsoft Foundry, or full custom training with Azure Machine Learning or Azure Databricks on GPU infrastructure.

The problem

Off-the-shelf models may miss domain language, required output formats or specific prediction tasks such as forecasting and defect detection. Teams then jump to training their own model, which takes data, skills, GPUs and an operations process for versions, deployment and monitoring.

Often a prebuilt service, better prompts or retrieval would have closed the gap at a fraction of the effort.1

The approach

Microsoft's architecture guide suggests working down a ladder:

  1. Prebuilt APIs: Foundry Tools (Speech, Translator, Language, Content Understanding, Document Intelligence, Vision and others). Several support customization, such as custom neural and template models and classifiers in Document Intelligence.
  2. Fine-tuning foundation models: Microsoft Foundry offers managed fine-tuning (generally available) with supervised fine-tuning, direct preference optimization and reinforcement fine-tuning, on Foundry-managed infrastructure. Microsoft notes fine-tuning does not replace retrieval for current information and that benefits are not guaranteed. Training types differ in data residency: the developer tier has no data residency guarantees, and data zone training currently supports only the US data zone.
  3. Custom training and MLOps: Azure Machine Learning for classical and deep learning, experiment tracking, pipelines, CI/CD and managed online and batch endpoints. Azure Databricks for lakehouse-scale Spark training with MLflow and Unity Catalog.
  4. Infrastructure: Azure GPU virtual machines, with the ND family aimed at deep learning training and inference and the NC family at training and HPC.

Evaluate at every step with Foundry evaluators for quality and safety, and use the AI red teaming agent before release. Note that prompt flow in Foundry and Azure Machine Learning retires on 20 April 2027, with Microsoft Agent Framework as the migration target.

When a simpler route is enough: prompt engineering plus retrieval often meets the need without changing model weights.1234567

Conceptual architecture

Build custom AI models: conceptual architectureCopilot & businessappsAI models & platformData & analyticsDeveloper toolsSecurity, identity &governanceCloud & hybridinfrastructureApplications and agents: Use the model in business workflowsApplications and agentsFoundry Tools (prebuilt and customizable APIs): First option before custom modelsFoundry Tools (prebuilt andcustomizable APIs)Microsoft Foundry fine-tuning (SFT, DPO, RFT): Adapts foundation model weights to a taskMicrosoft Foundryfine-tuning (SFT, DPO, RFT)Training data in OneLake, lakehouse or storage: Labeled examples and featuresTraining data in OneLake,lakehouse or storageAzure Databricks: Spark-scale training with MLflow and Unity CatalogAzure DatabricksAzure Machine Learning: Custom training, pipelines, registry and MLOpsAzure Machine LearningFoundry evaluations and red teaming: Quality and safety checks before and after releaseFoundry evaluations and redteamingAzure GPU virtual machines (ND, NC): Compute for training and inferenceAzure GPU virtual machines(ND, NC)Managed endpoints and deployments: Serve models to applicationsManaged endpoints anddeployments
Diagram as a list

  1. Copilot & business apps

    • Applications and agentsUse the model in business workflows

  2. AI models & platform

    • Foundry Tools (prebuilt and customizable APIs)First option before custom modelsConnects to Applications and agents
    • Microsoft Foundry fine-tuning (SFT, DPO, RFT)Adapts foundation model weights to a taskConnects to Managed endpoints and deployments

  3. Data & analytics

    • Training data in OneLake, lakehouse or storageLabeled examples and features
    • Azure DatabricksSpark-scale training with MLflow and Unity CatalogConnects to Managed endpoints and deployments

  4. Developer tools

    • Azure Machine LearningCustom training, pipelines, registry and MLOpsConnects to Managed endpoints and deployments

  5. Security, identity & governance

    • Foundry evaluations and red teamingQuality and safety checks before and after releaseConnects to Managed endpoints and deployments

  6. Cloud & hybrid infrastructure

    • Azure GPU virtual machines (ND, NC)Compute for training and inference
    • Managed endpoints and deploymentsServe models to applicationsConnects to Applications and agents
Conceptual: one common way to arrange the parts, not a required design.1

Technologies and their roles

  • Foundry Tools1

    Prebuilt and customizable AI services

    Microsoft suggests these before building a custom model.

  • Microsoft Foundry1

    Fine-tuning and evaluation

    Managed fine-tuning plus evaluators, guardrails and observability for generative models.

  • Azure Machine Learning5

    Custom training and MLOps

    Training, pipelines, MLflow, CI/CD and managed endpoints for classical and deep learning.

  • Azure Databricks1

    Lakehouse-scale machine learning

    Spark training with MLflow and Unity Catalog governance.

  • Azure GPU virtual machines6

    Training and inference compute

    ND and NC families target deep learning, training and HPC.

What you need first4

  • A baseline using prompts and retrieval, so gains from customization can be measured
  • Enough high-quality, rights-cleared training data for the chosen method
  • An evaluation set and agreed quality and safety thresholds
  • Data residency requirements checked against the training type
  • An owner for model versions, monitoring and retirement

Risks and how to reduce them

Fine-tuning used to supply facts that change4
Use retrieval for current information; Microsoft says fine-tuning does not replace it.
Training data leaves the required region4
Pick a training type that matches residency needs; the developer tier has no residency guarantees.
Model quality or safety regresses after retraining7
Run evaluators and red teaming before each release and continuous evaluation in production.
Dependence on retiring tooling5
Plan to move prompt flow assets before 20 April 2027.

Related

Sources

  1. AI and machine learning products (Azure Architecture Center) (opens in a new tab)Microsoft · Vendor-reported
  2. What are Foundry Tools? (opens in a new tab)Microsoft · Vendor-reported
  3. Azure Document Intelligence overview (opens in a new tab)Microsoft · Vendor-reported
  4. Fine-tuning in Microsoft Foundry (opens in a new tab)Microsoft · Vendor-reported
  5. What is Azure Machine Learning? (opens in a new tab)Microsoft · Vendor-reported
  6. Virtual machine sizes overview (opens in a new tab)Microsoft · Vendor-reported
  7. Observability in generative AI (opens in a new tab)Microsoft · Vendor-reported

Fill out the form below to request your copy.

Name(Required)