Skip to content

Vision AI & Smart Environments

Video analytics for cities, factories, stores and warehouses: the Metropolis platform, DeepStream pipelines, the video search and summarization blueprint, TAO model customization, and Jetson or RTX PRO hardware for deployment.

3 technology profiles in this category

Overview

Many organizations have cameras but no one to watch them all. Vision AI turns video into structured events such as counts, safety alerts or answers to plain-language questions. This category covers NVIDIA tools for building those systems.

Metropolis is NVIDIA's vision AI application platform and partner ecosystem, from edge to cloud. The AI Blueprint for video search and summarization (VSS) builds agents that answer questions about live or recorded video. DeepStream is an open-source toolkit based on GStreamer for real-time pipelines in C/C++ or Python. TAO customizes vision foundation models, which can then be deployed as NIM. Deployment targets include Jetson Thor, DGX Spark and RTX PRO 6000 Blackwell GPUs.

Typical users are system integrators, city operators, manufacturers and retailers.

Video analytics raises privacy questions, so check local law on recording and retention first. A single-camera counting task may be handled by an off-the-shelf smart camera without custom AI.12

Problems it addresses

  • Too much video, too few people1

    The VSS blueprint lets operators query live or archived video in natural language.

  • Real-time multi-camera processing2

    DeepStream provides more than 40 hardware-accelerated GStreamer plug-ins for multi-camera pipelines.

  • Models that miss local conditions1

    TAO customizes vision foundation models with site data.

  • Traffic and transit monitoring1

    Metropolis is used for multi-camera tracking at transit corridors and intersections.

  • Manual inspection on production lines1

    Automated visual inspection is a listed Metropolis use in factories.

A typical workflow

  1. Define events and privacy rules

    List the events or questions that matter and the retention rules that apply.

  2. Build the stream pipeline2

    Ingest camera streams and run inference with DeepStream.

  3. Adapt the models1

    Customize vision models with TAO and serve them as NIM.

  4. Add search and summaries3

    Deploy the VSS blueprint for natural-language queries over video.

  5. Deploy at edge or center1

    Run on Jetson Thor near the cameras or on RTX PRO servers centrally.

Technologies

  • Framework

    NVIDIA DeepStream SDK

    DeepStream is NVIDIA's toolkit for building real-time video and multi-sensor analytics pipelines on GPUs and Jetson devices. It is based on GStreamer and part of Metropolis. Its source code has been on GitHub under Apache-2.0 since version 9.0, while the prebuilt runtime libraries stay under an NVIDIA license.

    • Applications
    • Inference
    • Compute
  • Platform

    NVIDIA Metropolis

    NVIDIA Metropolis is a vision AI application platform and partner ecosystem. It bundles models, libraries and blueprints, such as the Video Search and Summarization (VSS) blueprint, DeepStream and TAO, for building video analytics agents that turn camera streams into events, alerts, search and reports.

    • Applications
    • Models
    • Inference
  • Platform

    NVIDIA Cosmos

    NVIDIA Cosmos is an open platform of world foundation models and tools for physical AI. Cosmos 3 reasons over images and video and generates video, sound and robot actions; Curator, Evaluator and Cosmos Framework cover data, scoring and post-training. Licenses differ by model (OpenMDW 1.1 for Cosmos 3).

    • Models
    • Inference

Use cases

Documented implementations

  • Aigen

    In production

    Aigen: solar-powered weeding robots trained with NVIDIA Cosmos and run on Jetson

    Aigen builds solar-powered Element robots that pull weeds mechanically in row crops. NVIDIA says Aigen post-trains Cosmos models to generate synthetic field data and runs its newest robots on three Jetson Orin NX modules. Aigen reports more than 100 robots built and offers four-arm fleets for the 2027 season.

    Agriculture (autonomous mechanical weeding)

  • Kaohsiung City Government, with integrator Linker Vision

    Scaling

    Kaohsiung City Government: city video agents and a digital twin built by Linker Vision

    Linker Vision built a vision AI platform for Kaohsiung City Government on NVIDIA Metropolis, Omniverse and Cosmos, with video agents that flag events such as roadway flooding and alert the right agencies. No measured results with a stated baseline have been published.

    City government and public infrastructure

  • Foxconn (Hon Hai Technology Group)

    In production

    Foxconn: digital twins for new server plants with Omniverse, Isaac and Metropolis

    Foxconn uses NVIDIA Omniverse digital twins to plan production lines, Isaac to simulate robots and Metropolis for camera-based monitoring, from Hsinchu to new server plants in Mexico and the US. Most published results are expectations, such as a forecast energy cut of over 30 percent in Mexico.

    Electronics manufacturing

Digital Twin Opportunity Lab

Find the first digital twin worth building for your site.

Open the lab

Next steps

  1. Run a privacy impact assessment for the cameras you plan to use.

  2. Start with recorded video from two or three cameras and one event type.

  3. Try the video-search-and-summarization code from the NVIDIA-AI-Blueprints GitHub organization on sample footage.

  4. Use the Digital Twin Opportunity Lab if you also want to simulate the site layout.

Connected categories

Sources

  1. NVIDIA Metropolis (product page) (opens in a new tab)NVIDIA · Vendor-reported
  2. NVIDIA DeepStream SDK (developer page) (opens in a new tab)NVIDIA · Vendor-reported
  3. NVIDIA AI Blueprints GitHub organization (opens in a new tab)NVIDIA · Vendor-reported

Fill out the form below to request your copy.

Name(Required)