Orange Business and NVIDIA pilot distributed AI for real-time voice

A new model for AI inference

Orange Business and NVIDIA are teaming up to develop a distributed voice AI inference system. Inference is the process where an AI model actually performs a task, such as generating a response or translating a sentence, rather than just being trained. By distributing this process, the companies aim to help enterprises deploy conversational agents that feel natural and respond in real time.

The primary goal of this collaboration is to solve common bottlenecks in AI performance. When all AI requests go to one central location, it can cause delays or high costs. This new approach allows workloads to be spread across different locations, which improves the overall responsiveness and capacity of the system. It also helps optimize how the company uses its GPU hardware and network bandwidth.

As reported by The Fast Mode, this setup provides businesses with more control over where their AI services actually run. This is particularly important for companies that have strict rules about where their data can be stored or processed. By moving away from a purely centralized model, Orange and NVIDIA are creating a more flexible framework for enterprise AI deployment.

The wildfire emergency pilot

To test this technology in a high-stakes environment, Orange and NVIDIA are piloting the system during a wildfire emergency scenario. In this demonstration, the AI acts as a bridge between a caller speaking a foreign language and an emergency operator who speaks a different language. The system ensures that the translation is accurate and happens quickly enough to be useful in a crisis.

Beyond just translating the spoken words, the AI also handles several other administrative tasks for the emergency services. It transcribes the live conversation and prepares summaries that capture the most important information from the call. This allows the operators to understand the situation faster and take the necessary actions to address the emergency without missing key details.

The technology relies on Orange’s secure multi-LLM platform, known as Live Intelligence. This platform manages the different large language models while the Distributed Inference platform handles the actual movement of workloads. Together, they ensure that the AI can handle complex tasks while maintaining the security and reliability required for public safety operations.

How the distributed network works

The technical backbone of this pilot involves a mix of specialized hardware and smart routing. Orange Business has deployed NVIDIA Blackwell chips at its central cloud locations to handle heavy computing needs. For the distributed parts of the network, they are using the NVIDIA Aerial RAN Computer, which is designed for edge computing and regional sites.

A key component of this architecture is integrated routing, which acts like a traffic controller for AI requests. This system balances the flow of data between different sites to ensure no single location becomes overloaded. If one site goes offline or becomes too busy, the router can automatically redirect the requests to another available location to keep the service running.

Orange is also integrating NVIDIA Nemotron models into its platform. These are open weight models, which means they provide a level of transparency and flexibility for customers. Because they are open weight, businesses can audit the models, tailor them to specific needs, and deploy them in ways that satisfy their unique operational requirements.

Expanding the use cases

While the current pilot focuses on emergency calls, the distributed voice AI is built to support a wide variety of other applications. This includes live multilingual translation for general business use and advanced conversational agents for enterprise contact centers. It can also be applied to more complex visual tasks like video analytics and computer vision.

The technology is also being positioned for industrial and security applications. For example, it could be used for drone detection or to monitor industrial IoT devices. By providing a framework that handles both voice and data across a distributed network, Orange and NVIDIA are creating a versatile tool for many different industries.

Orange is also looking into how this fits into the broader telecommunications landscape. They are currently assessing AI-RAN with Nokia and NVIDIA to see how it affects radio spectral efficiency and network capacity. The goal is to see if shared infrastructure can support both traditional radio functions and new AI services like sensing and drone detection simultaneously.

The future of the AI factory

This collaboration marks a shift toward what NVIDIA describes as the next frontier of the AI factory. As the demand for edge computing and real-time inference grows, telecommunications companies are becoming essential infrastructure providers. They are moving beyond just providing a connection to providing the actual compute power needed for modern AI applications.

Orange Business is positioning itself to lead this transition by combining its network footprint with advanced model management. By offering a solution that includes compute, connectivity, and orchestration, they can help customers deploy AI with better resilience. This allows for

The future of the AI factory

This collaboration marks a shift toward what NVIDIA describes as the next frontier of the AI factory. As the demand for edge computing and real-time inference grows, telecommunications companies are becoming essential infrastructure providers. They are moving beyond just providing a connection to providing the actual compute power needed for modern AI applications.

Orange Business is positioning itself to lead this transition by combining its network footprint with advanced model management. By offering a solution that includes compute, connectivity, and orchestration, they can help customers deploy AI with better resilience. This allows for