Vision AI & Smart Environments
Video analytics for cities, factories, stores and warehouses: the Metropolis platform, DeepStream pipelines, the video search and summarization blueprint, TAO model customization, and Jetson or RTX PRO hardware for deployment.
3 technology profiles in this category
Overview
Many organizations have cameras but no one to watch them all. Vision AI turns video into structured events such as counts, safety alerts or answers to plain-language questions. This category covers NVIDIA tools for building those systems.
Metropolis is NVIDIA's vision AI application platform and partner ecosystem, from edge to cloud. The AI Blueprint for video search and summarization (VSS) builds agents that answer questions about live or recorded video. DeepStream is an open-source toolkit based on GStreamer for real-time pipelines in C/C++ or Python. TAO customizes vision foundation models, which can then be deployed as NIM. Deployment targets include Jetson Thor, DGX Spark and RTX PRO 6000 Blackwell GPUs.
Typical users are system integrators, city operators, manufacturers and retailers.
Video analytics raises privacy questions, so check local law on recording and retention first. A single-camera counting task may be handled by an off-the-shelf smart camera without custom AI.12
Problems it addresses
Too much video, too few people1
The VSS blueprint lets operators query live or archived video in natural language.
Real-time multi-camera processing2
DeepStream provides more than 40 hardware-accelerated GStreamer plug-ins for multi-camera pipelines.
Models that miss local conditions1
TAO customizes vision foundation models with site data.
Traffic and transit monitoring1
Metropolis is used for multi-camera tracking at transit corridors and intersections.
Manual inspection on production lines1
Automated visual inspection is a listed Metropolis use in factories.
A typical workflow
Define events and privacy rules
List the events or questions that matter and the retention rules that apply.
Build the stream pipeline2
Ingest camera streams and run inference with DeepStream.
Adapt the models1
Customize vision models with TAO and serve them as NIM.
Add search and summaries3
Deploy the VSS blueprint for natural-language queries over video.
Deploy at edge or center1
Run on Jetson Thor near the cameras or on RTX PRO servers centrally.
Technologies
-
Framework
NVIDIA DeepStream SDK
DeepStream is NVIDIA's toolkit for building real-time video and multi-sensor analytics pipelines on GPUs and Jetson devices. It is based on GStreamer and part of Metropolis. Its source code has been on GitHub under Apache-2.0 since version 9.0, while the prebuilt runtime libraries stay under an NVIDIA license.
- Applications
- Inference
- Compute
-
Platform
NVIDIA Metropolis
NVIDIA Metropolis is a vision AI application platform and partner ecosystem. It bundles models, libraries and blueprints, such as the Video Search and Summarization (VSS) blueprint, DeepStream and TAO, for building video analytics agents that turn camera streams into events, alerts, search and reports.
- Applications
- Models
- Inference
-
Platform
NVIDIA Cosmos
NVIDIA Cosmos is an open platform of world foundation models and tools for physical AI. Cosmos 3 reasons over images and video and generates video, sound and robot actions; Curator, Evaluator and Cosmos Framework cover data, scoring and post-training. Licenses differ by model (OpenMDW 1.1 for Cosmos 3).
- Models
- Inference
No technology in this category matches. Clear the filter to see all.
Use cases
Documented implementations
Aigen
In production
Aigen: solar-powered weeding robots trained with NVIDIA Cosmos and run on Jetson
Aigen builds solar-powered Element robots that pull weeds mechanically in row crops. NVIDIA says Aigen post-trains Cosmos models to generate synthetic field data and runs its newest robots on three Jetson Orin NX modules. Aigen reports more than 100 robots built and offers four-arm fleets for the 2027 season.
Agriculture (autonomous mechanical weeding)
Kaohsiung City Government, with integrator Linker Vision
Scaling
Kaohsiung City Government: city video agents and a digital twin built by Linker Vision
Linker Vision built a vision AI platform for Kaohsiung City Government on NVIDIA Metropolis, Omniverse and Cosmos, with video agents that flag events such as roadway flooding and alert the right agencies. No measured results with a stated baseline have been published.
City government and public infrastructure
Foxconn (Hon Hai Technology Group)
In production
Foxconn: digital twins for new server plants with Omniverse, Isaac and Metropolis
Foxconn uses NVIDIA Omniverse digital twins to plan production lines, Isaac to simulate robots and Metropolis for camera-based monitoring, from Hsinchu to new server plants in Mexico and the US. Most published results are expectations, such as a forecast energy cut of over 30 percent in Mexico.
Electronics manufacturing
Digital Twin Opportunity Lab
Find the first digital twin worth building for your site.
Next steps
Run a privacy impact assessment for the cameras you plan to use.
Start with recorded video from two or three cameras and one event type.
Try the video-search-and-summarization code from the NVIDIA-AI-Blueprints GitHub organization on sample footage.
Use the Digital Twin Opportunity Lab if you also want to simulate the site layout.
Connected categories
Sources
Thank you. Your correction was sent.
The editors check it against the sources. If you left an email address, they may reply about it.