NVIDIA optimizes FLUX 3 image generation for Blackwell GPUs

Accelerating high-resolution models

Generating high-quality images with diffusion models often requires multiple sequential passes through a system with billions of parameters. As the resolution of the output increases, the computational cost of 'attention'—the mechanism the model uses to understand relationships between different parts of an image—grows significantly.

NVIDIA Developer reported that these bottlenecks can slow down production workflows for designers and image editors. To address this, NVIDIA has optimized the FLUX 3 Image Klein model from Black Forest Labs specifically for its Blackwell GPU architecture.

The goal of these optimizations is to reduce inference latency, which is the time it takes for the model to produce a result. By using the TensorRT LLM VisualGen runtime, developers can achieve much faster results without sacrificing the fidelity of the generated images.

How the optimizations work

Several distinct technologies work together to achieve these speed gains. One primary method is NVFP4, a low-precision data format supported by Blackwell GPUs. This allows the model to process information more quickly than the standard BF16 format while maintaining high output quality.

The system also uses 'fused kernels' and 'epilogue fusion' to combine multiple mathematical operations into single steps. Additionally, 'Sol-Attention' uses sparsity to reduce the amount of computation needed for the attention mechanism, while 'Cache-DiT' reuses intermediate transformer outputs across different denoising steps.

For multi-GPU setups, the runtime uses Ulysses sequence parallelism to split the workload across multiple cards. This allows the system to handle the massive demands of 2048x2048 images by distributing the processing power more efficiently across a cluster of GPUs.

Benchmarking the results

The performance gains were tested on a single NVIDIA B200 GPU using 2048x2048 images. The combined optimizations resulted in a 3.46x speedup for text-to-image tasks and a 4.45x speedup for image-to-image tasks compared to uncompiled inference.

The results show that 'Cache-DiT' alone was able to nearly halve the latency for text-to-image generation at certain resolutions. For image-to-image tasks, the speedup was particularly notable, reducing the time required to transform a reference image into a new scene significantly.

NVIDIA also tested these optimizations on the RTX PRO 6000 Blackwell Server Edition GPU. On that hardware, the system delivered generation speedups ranging from 1.73x to 3.25x, showing that the benefits extend beyond the largest data center chips to professional workstation cards.

Impact on visual production

These improvements are specifically designed for developers building design tools and visual production pipelines. By reducing the time it takes to generate or edit an image, these tools become more responsive for human users who need to iterate quickly on creative concepts.

The optimizations handle different types of inputs, such as text prompts, noisy target images, and reference images. Because the model processes these streams independently before combining them, it can handle complex editing requests, like changing a scene's contents while preserving the original shape of objects.

For companies running these models at scale, the ability to use multi-GPU parallelism means they can serve more users simultaneously. The guide includes specific configurations to help these users set up OpenAI-compatible endpoints for online image generation services.

Deployment and next steps

NVIDIA has provided a comprehensive tutorial including deployment instructions and YAML configurations. This allows developers to easily swap out standard configurations for the optimized versions, such as enabling VC-Attention FP8 or Sol-Attention depending on their specific needs.

The tutorial also highlights how different settings can balance speed and quality. For example, users can choose different 'tau' values in Sol-Attention to prioritize either the speed of the denoising process or the final visual fidelity of the image.

Moving forward, the availability of these pre-quantized checkpoints and optimized kernels makes it easier for the community to adopt FLUX 3. Developers can now start by using the provided Docker containers to test these speedups on their own Blackwell hardware.