NVIDIA Metropolis
NVIDIA Metropolis is a vision AI application platform and partner ecosystem. It bundles models, libraries and blueprints, such as the Video Search and Summarization (VSS) blueprint, DeepStream and TAO, for building video analytics agents that turn camera streams into events, alerts, search and reports.1
Also known as NVIDIA Vision AI platform, Metropolis VSS blueprint, Video Search and Summarization (VSS)
At a glance
- What is it?
- Metropolis is NVIDIA's collection of building blocks for video analytics. NVIDIA describes it both as a vision AI application platform and as an ecosystem of partner companies that sell solutions built on it. The developer stack includes vision language and vision foundation models (including Cosmos), vision NIM microservices, embedding models, the DeepStream streaming analytics toolkit, TAO for fine-tuning, and the VSS blueprint, which is a set of reference architectures for vision agents.2
- What does it do?
- Metropolis helps developers go from raw video to answers. In the VSS blueprint, real-time services detect and track objects across cameras, create searchable embeddings, and use vision language models to caption video and flag incidents. Downstream services track objects over time, apply rules such as tripwires or restricted zones, and verify alerts against the video. An agent then answers questions, searches footage, summarizes long recordings and writes reports, and its tools are exposed through the Model Context Protocol (MCP).3
- Who needs it?
- Developers, integrators and software vendors who build camera-based analytics for factories, warehouses, retail stores, airports, cities and roads. Operations teams that need to search or summarize large amounts of video, or verify alerts, while keeping video on premises.
- What does it need?4
- A validated GPU: H100, RTX PRO 6000 Blackwell, L40S, DGX Spark, IGX Thor or AGX Thor (RTX PRO 4500 Blackwell for the alerts profile only)
- Ubuntu 22.04 or 24.04 on x86, or DGX OS / Jetson Linux on Spark and Thor, with an R580-branch NVIDIA driver
- Docker 28.3.3 or later (before 29.5.0), Docker Compose v2.39.1+, NVIDIA Container Toolkit 1.17.8+ and NGC CLI 4.10.0+
- A trusted, isolated network with authentication and TLS provided by your infrastructure
- Camera streams (for example RTSP) and camera calibration for multi-camera tracking
- A privacy and data-retention policy for the video being analyzed
- What it is not
- Metropolis is not a camera, a video management system or a finished app you install; it is a developer platform, and finished products come from partners or your own team. DeepStream is one part of Metropolis, not the whole of it. The VSS blueprint is not meant to face untrusted networks: NVIDIA states it assumes authentication, TLS and access control are supplied by the surrounding infrastructure. Privacy, consent and legal compliance for camera analytics remain the deployer's responsibility.3
Availability and licensing. Per the VSS documentation, the blueprint's deployment files and scripts are under Apache 2.0, while its microservices are under the NVIDIA Software and Model Evaluation License Agreement; bundled models, NIM microservices, Elasticsearch and sample data each carry their own license. We recommend checking production terms with NVIDIA before going live. NVIDIA states VSS should run in a trusted, isolated network.5
The problem it solves
Organizations record far more video than people can watch. Finding an event means scrolling through hours of footage, and simple motion or line-crossing alerts produce many false alarms. Building a custom system to detect, track, search and explain events across many cameras involves model training, streaming pipelines, message brokers and storage that most teams would rather not assemble from scratch.
Metropolis provides those parts as models, microservices and reference blueprints that run on NVIDIA GPUs at the edge, on premises or in the cloud.
How it works
The VSS blueprint is organized in three layers:
- Real-time video intelligence. RT-CV uses DeepStream with models such as RT-DETR, Grounding DINO and Sparse4D for detection, classification and multi-camera tracking. RT-Embedding creates semantic embeddings with Cosmos-Embed1. RT-VLM applies vision language models such as Cosmos Reason or Qwen3-VL to caption video and flag anomalies. Results go to a message broker.
- Downstream analytics. Behavior Analytics reads metadata from Kafka, Redis Streams or MQTT, tracks objects over time and raises incidents from rules such as tripwires, restricted zones and proximity. The Alerts microservice pulls the matching clip and asks a vision language model to confirm or reject each alert.
- Agent and offline processing. An agent uses MCP to reach analytics data, incidents and vision tools for question answering, search, summarization and reports.
Video ingest, storage and replay are handled by the Video IO and Storage (VIOS) microservices. Models can be fine-tuned with TAO and trained on synthetic data from Cosmos and Isaac Sim.3
Diagram as a list
Applications & solutions
- Cameras and sensors (RTSP)Live and recorded video sourcesConnects to Video IO and Storage (VIOS)
- Behavior Analytics and AlertsRules, incidents and VLM alert verificationConnects to VSS agent with MCP gateway
- VSS agent with MCP gatewaySearch, Q&A, summaries and reports
Models & frameworks
- RT-VLM (Cosmos Reason, Qwen3-VL)Captions, incidents and anomaliesConnects to Message broker (Kafka, Redis, MQTT)
- RT-Embedding (Cosmos-Embed1)Embeddings for video searchConnects to VSS agent with MCP gateway
- TAO fine-tuningAdapts models to site dataConnects to RT-CV on DeepStream, RT-VLM (Cosmos Reason, Qwen3-VL)
Inference & runtime software
- RT-CV on DeepStreamDetection, classification and multi-camera trackingConnects to Message broker (Kafka, Redis, MQTT)
Operations & orchestration
- Video IO and Storage (VIOS)Ingests, records and replays streamsConnects to RT-CV on DeepStream, RT-VLM (Cosmos Reason, Qwen3-VL), RT-Embedding (Cosmos-Embed1)
- Message broker (Kafka, Redis, MQTT)Carries metadata to analyticsConnects to Behavior Analytics and Alerts
Accelerated computing
- GPUs: H100, L40S, RTX PRO, DGX Spark, Jetson/IGX ThorRuns the pipeline at edge or data centerConnects to RT-CV on DeepStream, RT-VLM (Cosmos Reason, Qwen3-VL)
Capabilities
VSS blueprint for vision agents35
Reference architectures that combine accelerated vision microservices, vision language models and LLMs into agents for search, summarization, alert verification and real-time alerts.
Why it matters: Gives a working starting point instead of assembling a video agent from generic parts.
Limits: Must run in a trusted, isolated network; the microservices are under an NVIDIA evaluation license.
Real-time detection and multi-camera tracking3
RT-CV uses DeepStream with RT-DETR, Grounding DINO and Sparse4D to detect, classify and track objects across single or multiple cameras.
Why it matters: Turns raw streams into structured events in real time.
Limits: Accuracy depends on camera placement, calibration and how close your scenes are to the training data.
Vision language models on video3
RT-VLM applies models such as Cosmos Reason or Qwen3-VL to caption video, detect incidents and identify anomalies.
Why it matters: Lets operators ask questions about video in natural language.
Limits: VLM answers can be wrong and need verification for important decisions.
Behavior analytics and alert verification3
Rules for tripwires, regions of interest, proximity and restricted zones generate incidents, and a VLM marks each alert as confirmed, rejected or unverified.
Why it matters: Reduces the number of false alarms people must review.
Limits: Rules and thresholds must be tuned per site.
Fine-tuning with TAO2
TAO provides agent skills and tools for fine-tuning vision AI models, including LoRA post-training of Cosmos 3, with natural language prompts.
Why it matters: Adapts models to site-specific objects or defects.
Limits: Needs labeled domain data and GPU time.
Synthetic data for vision models2
Agent skills for defect image generation and video augmentation, Cosmos models, and Isaac Sim event and actor generation create extra training data.
Why it matters: Covers rare defects and events that are hard to record.
Limits: Synthetic data must be tested against real footage.
Edge and data center deployment profiles34
On validated data center GPUs the pipeline can run fully locally on NIM microservices and self-hosted models; on DGX Spark, IGX Thor and AGX Thor, VSS currently needs a remote LLM and supports only the base and alerts profiles. Helm-based Kubernetes deployment is also documented.
Why it matters: On data center GPUs, video can stay on site for sovereignty, latency or cost reasons.
Limits: You run and maintain the GPU infrastructure yourself; NVIDIA says fully local deployment of all workflows on Spark and Thor is planned for a future release.
Partner ecosystem1
NVIDIA states the Metropolis ecosystem includes over 1,000 companies, including system integrators, application providers and system builders.
Why it matters: Buyers can choose a finished partner solution instead of building one.
Limits: Partner products vary; evaluate each on its own merits.
Practical use cases
A warehouse wants to catch near misses between people and forklifts.
- Approach
- Deploy the VSS warehouse operations example for people and forklift detection and tracking, with alert verification of near-miss events.
- Role of NVIDIA Metropolis
- Provides detection, tracking, rules and VLM verification.
- Data, infrastructure and skills
- Calibrated cameras covering aisles, a validated GPU, and a safety team to act on alerts.
- Type of benefit
- Improved safety visibility
- Caveats
- Worker privacy and works council or legal approval may be required.
- First step
- Run the warehouse example with its sample data before adding your own cameras.
A city traffic team needs to find and confirm collisions across many cameras.
- Approach
- Use the VSS smart city example for vehicle and person tracking and event verification of collisions.
- Role of NVIDIA Metropolis
- Supplies the reference pipeline and agent.
- Data, infrastructure and skills
- Access to traffic camera streams, GPUs and an incident workflow.
- Type of benefit
- Faster incident detection
- Caveats
- Camera angles and weather affect accuracy; keep a person in the loop.
- First step
- Deploy the smart city example with Docker Compose and its sample data.
A manufacturer needs to detect visual defects that are rare in its data.
- Approach
- Fine-tune a vision model with TAO, add synthetic defect images from the defect image generation skill, and deploy it with DeepStream.
- Role of NVIDIA Metropolis
- Provides models, fine-tuning tools and the streaming pipeline.
- Data, infrastructure and skills
- Labeled defect samples, a line camera and GPU hardware near the line.
- Type of benefit
- Higher inspection coverage
- Caveats
- Validate on real defects; synthetic images alone are not proof of accuracy.
- First step
- Try the defect image generation skill on one defect type.
Investigators spend hours scrolling through recorded video.
- Approach
- Use a VSS developer profile for video search and summarization over archived footage with natural language queries.
- Role of NVIDIA Metropolis
- Provides embeddings, VLM summaries and the search agent.
- Data, infrastructure and skills
- Archived video, storage, and two validated data center GPUs (H100, RTX PRO 6000 Blackwell or L40S); the search profile is not listed for DGX Spark or Thor.
- Type of benefit
- Less manual review time
- Caveats
- Summaries can miss or misdescribe events; check key findings in the source video.
- First step
- Deploy the search developer profile and index one day of footage.
Sources 4
Who uses it
Foxconn (Hon Hai Technology Group) · Electronics manufacturing
Foxconn: digital twins for new server plants with Omniverse, Isaac and Metropolis
Foxconn uses NVIDIA Omniverse digital twins to plan production lines, Isaac to simulate robots and Metropolis for camera-based monitoring, from Hsinchu to new server plants in Mexico and the US. Most published results are expectations, such as a forecast energy cut of over 30 percent in Mexico.
In production
Works with
Optional integration
- NVIDIA CosmosVSS uses Cosmos Reason and Cosmos-Embed models for captions, alerts and search.
- NVIDIA NIMVision NIM microservices and locally hosted NIMs serve models in the pipeline.
Complementary tools
- NVIDIA JetsonJetson AGX Thor is a validated edge platform for the VSS blueprint.
- NVIDIA CosmosCosmos Reason and Cosmos-Embed models run inside the Metropolis VSS blueprint.
Same family
- NVIDIA DeepStream SDKDeepStream is the streaming analytics toolkit inside Metropolis and powers RT-CV in VSS.
Relationship labels follow NVIDIA's documentation. "Alternative approaches" does not mean one is better: each profile says when it fits.
Getting started
Match your hardware to the validated list
Compare your GPU and OS with the VSS prerequisites; non-validated GPUs use an experimental OTHER configuration.
Check: Your GPU appears on the validated list, or you accept experimental status.
Install the software prerequisites
Install the listed NVIDIA driver, Docker and Docker Compose, NVIDIA Container Toolkit and NGC CLI, and apply the documented kernel settings.
Check: docker and the NGC CLI report the required versions.
Deploy a developer profile
Start with the basic video agent developer profile using Docker Compose, or use the Brev launchable for a preconfigured environment.
Check: The agent answers a question about a sample video.
Add your own camera
Register a camera stream through the VIOS sensor management API and run summarization or search on it.
Check: Events from your camera appear in search results.
Try an industry example
Deploy the smart city or warehouse operations example with its sample data, prompts and report templates.
Check: Alerts are verified as confirmed, rejected or unverified.
Official resources
Could this technology help you?
Describe your project to the Solution Architect. It starts with NVIDIA Metropolis as context but recommends independently, including when you do not need it.
Sources
Each statement above links to the source it comes from. Labels say who reported it.
- NVIDIA Metropolis (product page) (opens in a new tab)
- NVIDIA Metropolis for developers (opens in a new tab)
- NVIDIA VSS Blueprint documentation: Introduction (opens in a new tab)
- NVIDIA VSS Blueprint documentation: Prerequisites (opens in a new tab)
- NVIDIA VSS Blueprint documentation: License Information (opens in a new tab)
Thank you. Your correction was sent.
The editors check it against the sources. If you left an email address, they may reply about it.