Secure AI agents
Let AI agents work with real tools and data while limiting what they can reach: sandboxed execution, default-deny network and file access, scoped credentials, guardrails on inputs and outputs, and a complete audit trail.
The problem
Agents that can run code, browse, read files and call APIs act with real permissions. A prompt injection hidden in a web page or document, a confused plan or a compromised tool can lead an agent to leak data, delete records or spend money. Traditional application security assumes predictable code paths, but an agent chooses its actions at run time.
Security teams need controls that hold even when the model misbehaves: isolation, least-privilege access, policy enforced outside the agent, and logs detailed enough to investigate what happened.
The approach
Start with a threat model for each agent: which data, tools and actions it needs and what could go wrong. Then layer the controls.
- Runtime isolation: OpenShell, an open source runtime from NVIDIA, runs each agent in a sandbox with no direct network access, permits nothing by default, enforces policy outside the agent process, supplies credentials only where policy allows and logs every allow and deny decision. Its documentation lists version v0.1.2.
- Content controls: NeMo Guardrails adds programmable safety and topic rules, and Nemotron Safety models check prompts and replies for unsafe content, topic drift and jailbreaks.
- Infrastructure: on shared clusters, BlueField DPUs isolate tenants and enforce zero-trust policies outside the host.
Containers with strict egress rules, a separate service account per tool and human approval for sensitive actions form a sound baseline with any stack and may be enough for low-risk internal agents.1234
Conceptual architecture
Diagram as a list
Applications & solutions
- User or event triggerStarts an agent taskConnects to Gateway and policy control plane
- Agent sandbox (OpenShell)Runs the agent without privileges and with limited file accessConnects to Policy-checked network and tool access, Inference router to approved models
- Enterprise tools and dataSystems the agent is permitted to use
Models & frameworks
- Model with guardrails and safety checksGenerates actions; inputs and outputs are screened
Inference & runtime software
- Inference router to approved modelsForwards permitted model requests onlyConnects to Model with guardrails and safety checks
Operations & orchestration
- Gateway and policy control planeAuthenticates users, manages sandboxes and delivers policies and credentialsConnects to Agent sandbox (OpenShell)
- Audit log and security monitoringStores every allow and deny decision for investigation
Networking, power & facilities
- Policy-checked network and tool accessAllows only approved destinations, methods and pathsConnects to Enterprise tools and data
Technologies and their roles
openshell1
Agent runtime isolation
Sandboxes agents, denies by default, routes inference through policy and logs every decision.
nemo5
Guardrails
Applies configurable rules that keep model and agent responses within safety and topic limits.
nemotron3
Safety models
Nemotron Safety models screen prompts and replies for unsafe content, topic drift and jailbreaks.
bluefield4
Infrastructure isolation
DPUs enforce zero-trust policies and tenant isolation in hardware on shared clusters.
ai-enterprise67
Patched software supply chain
NVIDIA describes extended-lifetime production branches, a secure software supply chain and vulnerability mitigation; continuous CVE patching is stated in the NIM LLM documentation for NIM production branches.
What you need first1
- A threat model per agent listing data, tools and allowed actions
- Identity and secrets management for service credentials
- A container or Kubernetes platform; OpenShell supports Docker, Podman and Kubernetes
- Security monitoring that can ingest agent audit logs
- Red-team prompts and prompt-injection test cases
Risks and how to reduce them
- Prompt injection leads to data exfiltration
- Deny network egress by default, allow-list destinations per agent and treat retrieved content as untrusted.
- Over-broad credentials
- Issue short-lived, task-scoped credentials and keep them outside the agent process.
- Early-version tooling2
- OpenShell is at version 0.1.x; pin versions, test upgrades and keep compensating controls.
- Guardrails create false confidence
- Combine guardrails with isolation and human approvals, and red-team regularly.
- Incomplete audit trail
- Log every tool call and policy decision centrally and review incidents.
Related
Sources
- NVIDIA OpenShell (opens in a new tab)
- NVIDIA OpenShell documentation (opens in a new tab)
- NVIDIA Nemotron foundation models (opens in a new tab)
- NVIDIA BlueField Platform (opens in a new tab)
- NVIDIA NeMo documentation hub (opens in a new tab)
- NVIDIA AI Enterprise product page (opens in a new tab)
- NVIDIA NIM for LLM and VLM documentation: overview (opens in a new tab)
Thank you. Your correction was sent.
The editors check it against the sources. If you left an email address, they may reply about it.