Nvidia launches Scale-In to secure agentic AI factories

Securing the AI Factory

Nvidia recently introduced a new pillar to its AI networking infrastructure called Scale-In. This architecture is specifically designed to manage and operate what the company calls 'agentic AI factories.' These factories are environments where AI agents assume increasingly large roles in performing tasks.

The primary goal of Scale-In is to provide secure access and high-performance data movement for these agents. It aims to deliver tenant isolation and more predictable performance as AI systems become more autonomous. By creating a dedicated framework, Nvidia wants to ensure that agents can operate efficiently without compromising the underlying system.

Gilad Shainer, senior vice president of networking at Nvidia, noted that agentic AI represents a different use case than previous models. He explained that because many agents need to run on the same infrastructure, there is a critical need to keep them contained in a sandbox. Scale-In is the infrastructure built to ensure that security is maintained across memory, storage, and other components.

Hardware-Based Security

The Scale-In architecture is powered by Nvidia BlueField-4 and Nvidia DOCA, connected over Nvidia Spectrum-X Ethernet. BlueField-4 is a data processing unit (DPU) built for large-scale infrastructure needs. It serves as a networking platform that handles the heavy lifting of agentic AI security.

BlueField-4 is essential because it contains in-silicon security engines that work out-of-band. This means the security monitoring happens independently of the main processing flow. It collects telemetry from various sources, including GPUs, CPUs, memory, and storage, to understand exactly what the agents are doing.

To make this hardware useful, Nvidia uses the DOCA software platform. DOCA provides the APIs and framework that allow developers to create software-defined, cloud-native solutions. It enables the rapid creation of apps that offload and accelerate data center workloads while maintaining the isolation provided by the DPU.

The OpenShell Framework

Nvidia has also reinforced its strategy with an open-source runtime called OpenShell. This tool is designed to govern how agents execute their tasks, how they access data, and how they route inference. It acts as a management layer for the behavior of the AI agents within the factory.

The company chose to combine OpenShell with BlueField-4 because they wanted a device completely separated from the agents. By using BlueField-4, Nvidia can monitor the environment from a position that no agent can influence. This separation ensures that even if an agent behaves unexpectedly, the monitoring system remains secure and unaffected.

This combination brings together all the security elements required for generative AI. By pairing the runtime controls of OpenShell with the hardware-based monitoring of BlueField-4, Nvidia creates a secure infrastructure. This setup allows for the deployment of agents across enterprise systems with a higher degree of oversight.

Monitoring and Quarantine

The architecture also incorporates the OpenShell Reference Architecture, which integrates several key components. This includes Vera Rubin infrastructure, Vera CPUs, and the BlueField-4 networking platform. Together, these pieces support secure agent execution across large-scale enterprise systems.

A key part of this expansion is Nvidia Sentry, which runs on BlueField-4 using DOCA software. Sentry provides a continuous monitoring service for agent behavior. It is designed to identify when an agent attempts to move outside of its defined software boundaries.

If an agent tries to breach its sandbox, Sentry can quarantine that agent automatically. This provides an independent enforcement layer that works alongside the runtime controls of OpenShell. The result is a multi-layered security approach that protects the data center from unauthorized actions by autonomous AI agents.