Robotics
How robot makers and robotics integrators use NVIDIA technologies to build products: training skills in simulation and moving them to real hardware, onboard perception and motion planning, foundation models for humanoids, and fleets that scale across customer sites, with notes on when classical robotics is enough.
The problem
A robot company sells hardware that must work in buildings it has never seen. Each new customer site brings different lighting, parts, floors and people, and a skill that worked in the lab can fail on the first shift. Teaching every variation by hand does not scale once a company ships hundreds of units instead of a dozen.
Simulation promises faster development, but policies trained in a virtual world often stumble on real friction, sensor noise and contact forces. Closing that sim-to-real gap is a core engineering cost for arm, mobile and humanoid makers alike, and humanoids add balance and whole-body control on top.
Onboard compute is a product decision with a long tail: it fixes power draw, cost, heat and which software can run for the life of the robot. Products that move near people also need safety certification against standards for their class of machine, and every change to perception or planning software can reopen that work. Integrators, meanwhile, want a supported stack rather than a research demo.
The approach
NVIDIA's robotics page describes three computers: DGX systems to train models, Omniverse and Cosmos on RTX PRO servers to simulate, and Jetson modules on the robot for real-time inference and control. Within Isaac, Isaac Sim provides physically based simulation with sensor models, Isaac Lab trains and evaluates robot policies, Isaac ROS offers CUDA-accelerated ROS 2 packages for perception and trajectory planning, and Isaac GR00T supplies foundation models for humanoids. The same page names Agility Robotics using Isaac Sim and Isaac Lab to train its Digit humanoid, and Skild AI using Omniverse, Isaac Lab and Cosmos to train robot foundation models.
Universal Robots shows the product route for an arm maker: its AI Accelerator, which UR's product page lists as available for order, bundles a Jetson AGX Orin box, a 3D camera and Isaac libraries for pose estimation and motion planning, and NVIDIA says UR trained gear assembly policies in Isaac Lab that run on real arms without fine-tuning. Its sister company MiR runs pallet detection for the MiR1200 Pallet Jack on Jetson AGX Orin.
Not every robot needs this stack. Fixed pick-and-place with known parts is usually programmed on the teach pendant, open source planners such as MoveIt handle many arm tasks on a CPU, and mobile robots in mapped, stable sites run well on lidar navigation with modest compute. GPU modules and simulation-trained policies pay off when parts vary, scenes change, contact-rich skills are involved, or a fleet must adapt to many customer sites.1234
Conceptual architecture
Diagram as a list
Applications & solutions
- Real robot logs, CAD of robots and customer cellsSupplies kinematics, sensor recordings and site layoutsConnects to Simulation and synthetic data (Isaac Sim, Omniverse, Cosmos)
- Simulation and synthetic data (Isaac Sim, Omniverse, Cosmos)Builds randomized virtual scenes and generates training dataConnects to Policy and model training (Isaac Lab on DGX)
Models & frameworks
- Robot models and libraries (Isaac ROS, Isaac Manipulator, GR00T)Packaged perception, motion planning and humanoid foundation modelsConnects to Onboard computer (Jetson AGX Orin or Thor)
Inference & runtime software
- Onboard computer (Jetson AGX Orin or Thor)Runs perception and control on the robot in real timeConnects to Certified safety controller and sensors
Operations & orchestration
- Certified safety controller and sensorsEnforces speed, zone and stop functions independent of the AI stackConnects to Fleet management and field telemetry
- Fleet management and field telemetryTracks robots at customer sites and returns failure cases for retrainingConnects to Real robot logs, CAD of robots and customer cells
Accelerated computing
- Policy and model training (Isaac Lab on DGX)Trains perception, manipulation and locomotion policies at scaleConnects to Robot models and libraries (Isaac ROS, Isaac Manipulator, GR00T)
Technologies and their roles
NVIDIA Isaac3
Simulation, robot learning and accelerated ROS 2 packages
Agility Robotics trains Digit with Isaac Sim and Isaac Lab, and Universal Robots builds its AI Accelerator on Isaac Manipulator, Isaac ROS and Isaac Lab.
NVIDIA Jetson5
Onboard AI computer
UR's AI Accelerator and MiR's pallet jack run perception on Jetson AGX Orin, and NVIDIA's robotics page places Jetson on the robot for real-time control.
NVIDIA Cosmos1
World models for synthetic data
NVIDIA's robotics page says Skild AI uses Cosmos with Omniverse and Isaac Lab to train robot foundation models.
NVIDIA Omniverse1
Physically based virtual scenes
NVIDIA's three-computer description runs Omniverse on RTX PRO servers for robot simulation, and Skild AI uses it in training.
NVIDIA DGX1
Training compute
NVIDIA's robotics page assigns model training to DGX systems in its three-computer approach.
What you need first
- Accurate robot models (kinematics, dynamics, sensor specifications) that simulation can use
- Real-world recordings from target sites to calibrate simulation and check sim-to-real transfer
- A defined task scope and success metrics per application, agreed with integrators and pilot customers
- Robotics software skills in ROS 2, Python and reinforcement learning, in house or through partners
- A safety architecture and certification plan for the product class and target markets
- A compute roadmap that covers module supply, power budget and software support over the product's life
- Fleet telemetry to collect failures from the field and feed them into retraining
Risks and how to reduce them
- Safety certification of learned behavior4
- Keep certified safety functions independent of AI perception and planning, document how software updates are validated, and budget for recertification; MiR, for example, designs its pallet jack to ISO 3691-4.
- Sim-to-real failures at customer sites
- Randomize simulation widely, test on real hardware in representative cells, and stage rollouts with monitored pilot sites before fleet-wide updates.
- Camera and sensor data from customer premises
- Robots record people and proprietary processes; agree with customers what is stored, anonymize before data leaves the site, and restrict use for retraining to what contracts allow.
- Speed-up claims without baselines3
- Published figures such as 50 to 100 times faster planning lack stated baselines; benchmark candidate stacks on your own tasks and hardware.
Documented examples
Universal Robots (Teradyne Robotics) · Robotics (collaborative robot maker)
Universal Robots: building the AI Accelerator for cobots on NVIDIA Isaac and Jetson
Universal Robots, the cobot maker in Teradyne Robotics, sells the AI Accelerator: a Jetson AGX Orin compute box, a 3D camera and NVIDIA Isaac libraries that add perception and motion planning to its arms. UR's product page says it can be ordered; partners have shown inspection, tending and assembly uses.
In production
Related
Sources
- NVIDIA robotics (industry page) (opens in a new tab)
- AI Accelerator for UR cobots (product page) (opens in a new tab)
- Universal Robots accelerates cobot development with NVIDIA (customer story) (opens in a new tab)
- MiR1200 Pallet Jack launch (MiR blog and press release) (opens in a new tab)
- Universal Robots unveils its AI Accelerator (opens in a new tab)
Thank you. Your correction was sent.
The editors check it against the sources. If you left an email address, they may reply about it.