Skip to content

Build a smart factory

Combine a physically accurate digital twin of the plant, vision AI on the line and simulation-trained robots to plan layouts, catch defects and test changes before they reach the real factory floor.

The problem

Changing a production line is expensive to get wrong. A new layout, robot cell or product variant can create bottlenecks, safety hazards or quality problems that only appear after installation. Manual visual inspection misses defects, and plant data is spread across CAD, automation and quality systems that do not share a model of the factory.

A smart factory project connects these pieces: a shared 3D model of the plant that planners, engineers and AI systems can test against, cameras that check quality and safety in real time, and robots trained and validated in simulation before they move on the floor.

The approach

Start with one decision, such as rebalancing a line or adding a robot cell, and build only the twin needed for it. CAD and layout data are converted to OpenUSD, a scene format created by Pixar and developed through the Alliance for OpenUSD, and assembled with Omniverse libraries for rendering, physics and asset validation. Isaac Sim adds robot simulation, synthetic data and software-in-the-loop testing, and Cosmos models can widen the variety of synthetic training data.

On the line, Metropolis covers automated visual inspection: models customized with TAO run in DeepStream pipelines on edge GPUs such as Jetson. Results feed back into the twin and into plant systems.

Not every question needs a 3D twin. Discrete-event simulation software, a spreadsheet capacity model or a commercial inspection camera may answer a narrower question faster. A full twin pays off when layouts change often, robots are involved or many teams need the same model.123456

Conceptual architecture

Build a smart factory: conceptual architectureApplications &solutionsModels & frameworksInference & runtimesoftwareAcceleratedcomputingPlant data sources (CAD, layouts, automation, quality): Provide geometry, process and quality dataPlant data sources (CAD,layouts, automation,…Simulation (Omniverse libraries, Isaac Sim): Tests layouts, robot cells and flows virtuallySimulation (Omniverselibraries, Isaac Sim)Planning and quality dashboards: Show results to engineers and feed decisions back to the twinPlanning and qualitydashboardsOpenUSD plant model: Shared scene description of the plantOpenUSD plant modelSynthetic data (Isaac Sim, Cosmos): Generates labeled images and video for training inspection and robot modelsSynthetic data (Isaac Sim,Cosmos)Inspection models in DeepStream pipelines: Detect defects and safety events on line camerasInspection models inDeepStream pipelinesEdge GPUs on the line (Jetson): Run inspection close to the camerasEdge GPUs on the line(Jetson)RTX workstations or servers: Run simulation and renderingRTX workstations or servers
Diagram as a list
  1. Applications & solutions

    • Plant data sources (CAD, layouts, automation, quality)Provide geometry, process and quality dataConnects to OpenUSD plant model
    • Simulation (Omniverse libraries, Isaac Sim)Tests layouts, robot cells and flows virtuallyConnects to Synthetic data (Isaac Sim, Cosmos), Planning and quality dashboards
    • Planning and quality dashboardsShow results to engineers and feed decisions back to the twin
  2. Models & frameworks

    • OpenUSD plant modelShared scene description of the plantConnects to Simulation (Omniverse libraries, Isaac Sim)
    • Synthetic data (Isaac Sim, Cosmos)Generates labeled images and video for training inspection and robot modelsConnects to Inspection models in DeepStream pipelines
  3. Inference & runtime software

    • Inspection models in DeepStream pipelinesDetect defects and safety events on line camerasConnects to Edge GPUs on the line (Jetson)
  4. Accelerated computing

    • Edge GPUs on the line (Jetson)Run inspection close to the camerasConnects to Planning and quality dashboards
    • RTX workstations or serversRun simulation and renderingConnects to Simulation (Omniverse libraries, Isaac Sim)
Conceptual: one common way to arrange the parts, not a required design.2

Technologies and their roles

  • NVIDIA Omniverse7

    Digital twin foundation

    Libraries and tools for OpenUSD scenes, RTX rendering, physics and SimReady asset validation.

  • NVIDIA Isaac3

    Robot simulation and training

    Isaac Sim tests robots in physically based scenes and generates synthetic data; Isaac Lab trains policies.

  • NVIDIA Metropolis4

    Visual inspection

    Vision AI platform that lists automated visual inspection in manufacturing among its uses.

  • NVIDIA DeepStream SDK6

    Real-time video pipelines

    Open source streaming analytics toolkit with hardware-accelerated plug-ins for multi-camera inspection.

  • NVIDIA Cosmos8

    Synthetic data and visual reasoning

    World foundation models that generate synthetic video and reason over images and video for inspection.

  • NVIDIA Jetson9

    Edge compute on the line

    Compact modules that run inspection and robot software near the equipment.

What you need first10

  • CAD and layout data with clear ownership and a way to keep it current
  • One defined first decision for the twin to support
  • 3D, OpenUSD and Python skills, or an integration partner who has them
  • Labeled defect images, or a plan to generate synthetic ones
  • RTX-class GPUs for simulation; Isaac Sim does not support GPUs without RT Cores such as A100 or H100

Risks and how to reduce them

The twin drifts away from the real plant
Tie twin updates to engineering change processes and check key dimensions against site surveys.
Simulation results are trusted without validation
Compare simulated cycle times and robot behavior with pilot runs before acting on them.
Worker privacy with line cameras
Point cameras at products rather than people where possible, inform staff and agree retention rules with employee representatives where they exist.
Pre-release software and mixed licenses11
Several Omniverse libraries are labeled pre-release and not enterprise-supported; check the license terms for each library before use.
Robot safety on a shared floor
Keep certified safety systems in place; simulation-trained behavior must still pass physical safety validation.

Documented examples

  • PepsiCo, with Siemens

    Pilot

    PepsiCo and Siemens: plant and warehouse twins with Digital Twin Composer on Omniverse

    PepsiCo is converting selected US plants and warehouses into 3D digital twins with Siemens Digital Twin Composer, which Siemens builds on NVIDIA Omniverse libraries. PepsiCo and Siemens report a 20 percent throughput gain on the first deployment; the program is in early pilots.

    Food and beverage manufacturing and logistics

  • BMW Group

    Scaling

    BMW Group: planning car plants in an Omniverse-based Virtual Factory

    BMW Group plans and checks its car plants in a Virtual Factory built on NVIDIA Omniverse and OpenUSD. BMW says digital twins now cover more than 30 production sites, cut collision checks for new models from almost four weeks to about three days, and are projected to lower planning costs by up to 30 percent.

    Automotive manufacturing

  • Foxconn (Hon Hai Technology Group)

    In production

    Foxconn: digital twins for new server plants with Omniverse, Isaac and Metropolis

    Foxconn uses NVIDIA Omniverse digital twins to plan production lines, Isaac to simulate robots and Metropolis for camera-based monitoring, from Hsinchu to new server plants in Mexico and the US. Most published results are expectations, such as a forecast energy cut of over 30 percent in Mexico.

    Electronics manufacturing

Related

Sources

  1. Alliance for OpenUSD (opens in a new tab)Alliance for OpenUSD · Third-party reporting
  2. NVIDIA Omniverse (product page) (opens in a new tab)NVIDIA · Vendor-reported
  3. NVIDIA Isaac Sim (developer page and FAQ) (opens in a new tab)NVIDIA · Vendor-reported
  4. NVIDIA Metropolis (product page) (opens in a new tab)NVIDIA · Vendor-reported
  5. NVIDIA Metropolis for developers (opens in a new tab)NVIDIA · Vendor-reported
  6. NVIDIA DeepStream SDK (developer page) (opens in a new tab)NVIDIA · Vendor-reported
  7. NVIDIA Omniverse for developers (opens in a new tab)NVIDIA · Vendor-reported
  8. NVIDIA Cosmos (product page and FAQ) (opens in a new tab)NVIDIA · Vendor-reported
  9. NVIDIA Jetson embedded systems page (opens in a new tab)NVIDIA · Vendor-reported
  10. Isaac Sim documentation: requirements (opens in a new tab)NVIDIA · Vendor-reported
  11. NVIDIA Omniverse documentation index (opens in a new tab)NVIDIA · Vendor-reported

Fill out the form below to request your copy.

Name(Required)