Agriculture
How growers and agricultural technology makers use AI and NVIDIA technologies: field robots that recognize crops and weeds on board, synthetic field imagery from world models to train them, and camera analytics for livestock, with notes on when guidance systems and services already do the job.
The problem
Farming decisions are tied to a calendar that cannot be moved. Weeds must be controlled in a window of a few weeks, a missed pass cannot be repeated next month, and every field differs in soil, crop stage and weather. Herbicide-resistant weeds are spreading, chemical rules are tightening, and labor for hand weeding or scouting is scarce and seasonal.
Machines that act on individual plants need perception that tells crop from weed under harsh sun, dust and shadow, yet the training images have to be gathered and labeled in the same short season, and a model trained on one crop or region often fails on the next. Robots in the field run on batteries or solar power, far from reliable connectivity, so the computing has to happen on board within a tight energy budget.
Livestock operations face their own counting and monitoring problems: animals move, cluster and look alike, and manual counts at gates or in pens are slow and error-prone.1
The approach
Aigen, a maker of solar-powered weeding robots, shows the pattern for field robotics. NVIDIA reports that Aigen tunes Cosmos world models with video from its own machines to produce labeled synthetic scenes of crops, weeds and soil, and that a weeding perception model built with only one percent real imagery drove the robots autonomously in the field. On the machines, NVIDIA says, a trio of Jetson Orin NX boards handles sensing, driving and the weeding arms. Aigen itself counts over 100 robots produced and is selling four-arm fleets for 2027.
Earlier examples from NVIDIA's 2021 roundup of agricultural startups include Greeneye and Bilberry, whose weed-targeting sprayers run on Jetson, and Plainsight, a Metropolis partner whose video analytics counted livestock for a global food company. Those figures are several years old and were not re-checked against current deployments.
Many farms will not need custom AI. RTK guidance and auto-steer, camera-guided row cultivators sold by implement makers, satellite and drone scouting services, and herbicide programs that still work cover much of the need. On-board GPU perception and synthetic training data make sense for companies building machines that act on single plants, or for large operations where labor, resistance or regulation make current methods untenable.123
Conceptual architecture
Diagram as a list
Applications & solutions
- Field robots and implements with cameras and depth sensorsCapture plant-level video, position and action data on every passConnects to Field data store with crop, weed and soil labels
- Field data store with crop, weed and soil labelsHolds real footage by crop, region and growth stageConnects to World model for synthetic field imagery (Cosmos)
- Farm management and agronomy recordsPlans passes, records treatments and feeds results back to data teamsConnects to Field data store with crop, weed and soil labels
Models & frameworks
- World model for synthetic field imagery (Cosmos)Generates labeled variations of crops, weeds, lighting and depthConnects to Perception model training on GPUs
Inference & runtime software
- On-robot inference (Jetson modules)Runs detection, navigation and tool control within a solar or battery budgetConnects to Field robots and implements with cameras and depth sensors, Farm management and agronomy records
- Barn and gate cameras with video analytics (Metropolis)Counts and tracks animals in pens and at transfer pointsConnects to Farm management and agronomy records
Accelerated computing
- Perception model training on GPUsTrains crop and weed detectors on mixed real and synthetic dataConnects to On-robot inference (Jetson modules)
Technologies and their roles
NVIDIA Jetson1
On-board computer for field robots
According to NVIDIA, Aigen's latest robots split perception, navigation and arm control across three Jetson Orin NX boards.
NVIDIA Cosmos4
Synthetic field data
Aigen post-trains Cosmos models on fleet footage to generate labeled images of crops and weeds, according to NVIDIA and Aigen's own Cosmos Cookbook recipe.
NVIDIA Metropolis3
Livestock video analytics
NVIDIA's 2021 roundup describes Plainsight, a Metropolis partner, counting livestock with video analytics for a global food company.
What you need first
- Representative field footage for each crop, region and growth stage, captured by your own machines
- Agronomists who define what counts as a weed or defect and review model errors
- A power and thermal budget for on-board computing that matches the machine's duty cycle
- Clear data ownership and sharing terms with growers whose fields are recorded
- Field trial plots and a season plan for validating models before commercial passes
- Local service capacity to repair and update machines during the growing window
Risks and how to reduce them
- Autonomous machines near people and animals
- Fit obstacle detection and emergency stops that work independently of the AI, define operating zones, and follow machinery safety rules for autonomous equipment in each market.
- Farm data ownership and privacy
- Field imagery, yields and locations are commercially sensitive for growers; agree in writing who owns the data, whether it may train shared models, and how long it is kept.
- Models that fail on new crops, regions or seasons
- Validate on fields not used in training, track performance per crop and stage, and keep a fallback method for the treatment window.
- Unverified and dated performance figures5
- The Aigen figures come from NVIDIA without published accuracy data and the sources disagree on its onboard processor, while the Greeneye, Bilberry and Plainsight figures date from 2021; run your own field trials.
Documented examples
Aigen · Agriculture (autonomous mechanical weeding)
Aigen: solar-powered weeding robots trained with NVIDIA Cosmos and run on Jetson
Aigen builds solar-powered Element robots that pull weeds mechanically in row crops. NVIDIA says Aigen post-trains Cosmos models to generate synthetic field data and runs its newest robots on three Jetson Orin NX modules. Aigen reports more than 100 robots built and offers four-arm fleets for the 2027 season.
In production
Related
Sources
- Agriculture robotics and edge AI: Aigen (customer story) (opens in a new tab)
- Aigen launches Element Gen 2 X4 (opens in a new tab)
- Food for thought: startups harness AI to nurture the future of agriculture (opens in a new tab)
- Generate photorealistic agricultural images for robot perception training (Cosmos Cookbook recipe) (opens in a new tab)
- How Aigen transformed agricultural robotics for sustainable farming with Amazon SageMaker AI (opens in a new tab)
Thank you. Your correction was sent.
The editors check it against the sources. If you left an email address, they may reply about it.