Orange and NVIDIA pilot distributed AI for real-time voice translation

Solving the latency hurdle

Orange Business and NVIDIA are teaming up to tackle one of the biggest hurdles in enterprise AI: latency. When a user interacts with a voice AI, any delay in processing can break the flow of a natural conversation. To solve this, the companies are developing distributed AI inference, which spreads the workload across different geographic locations.

Instead of relying solely on a single central cloud, this method places AI models at regional Points of Presence (PoP). A PoP is a physical location where a network provider connects different networks to provide services to users. By moving the compute closer to the end user, the system can deliver faster responses while still maintaining high performance.

This distributed approach also helps optimize how companies use their hardware. It allows for better management of GPU capacity and bandwidth across the network. As reported by The Fast Mode, this setup ensures that conversational agents can provide a seamless experience for users who require immediate interaction.

A high-stakes emergency pilot

To prove the technology works in demanding environments, Orange and NVIDIA are piloting the system during a wildfire emergency scenario. In this simulation, the AI acts as a real-time translator between a foreign-language caller and an emergency operator. The goal is to ensure that critical information is communicated accurately and instantly without any lag in the conversation.

Beyond just translating spoken words, the AI also performs several other tasks to assist the operator. It transcribes the live conversation and prepares summaries that capture key details from the call. These summaries help emergency services take faster actions to address the situation on the ground. The system is designed to provide the specific accuracy and response times needed for life-saving operations.

The pilot utilizes specific hardware from both companies to achieve these results. Orange Business has deployed NVIDIA Blackwell at their central cloud sites, while using NVIDIA Aerial RAN Computers at distributed locations. This combination allows the system to handle heavy processing tasks at the core while maintaining a distributed presence at the edge of the network.

Control and data sovereignty

For many large enterprises, where data is processed is just as important as how fast it is processed. The distributed inference platform gives customers more control over data residency, which refers to the physical location where data is stored and processed. By keeping voice data local where required, the system helps companies meet strict regulatory and sovereignty requirements.

The solution integrates with Live Intelligence, which is Orange’s secure platform for managing multiple Large Language Models (LLMs). Within this framework, Orange is incorporating NVIDIA Nemotron models. These are open weight models, meaning they provide customers with the flexibility to audit, customize, and deploy the AI according to their specific operational needs.

The platform also includes integrated routing to manage traffic across the various sites. This routing system balances the load between different locations and can automatically redirect requests if a specific site becomes overloaded or goes offline. This ensures that the AI service remains reliable and available even during peak usage or technical issues.

Future infrastructure and use cases

While the current focus is on conversational AI and emergency communications, the architecture is built to support a wide variety of industries. The service is being designed to handle live multilingual translation for enterprise contact centers, as well as video analytics and computer vision. It could also be used for drone detection and industrial IoT, where real-time data processing is essential.

Orange is also exploring the potential of AI-RAN, which is a concept where radio network functions and AI inference share the same infrastructure. They are currently assessing this with Nokia and NVIDIA to see if it can improve radio spectral efficiency and network capacity. This research looks at whether shared hardware can support both standard mobile functions and advanced AI services simultaneously.

The ultimate goal is to create a new class of real-time intelligence applications that require deterministic performance and scale. By combining connectivity with distributed compute, Orange aims to provide a complete end-to-end service. They are currently gathering feedback from customers on real-world use cases to refine the architecture's performance and resilience in trusted environments.