NVIDIA releases Granary-v2 speech labels for multilingual AI training

The scope of Granary-v2

NVIDIA recently published Granary-v2, a project designed to supply generated labels and instruction-target pairs for multilingual speech training. The public release includes labels for 463,062 hours of source speech across 48 different languages. This release serves as a subset of a much larger processed collection managed by NVIDIA's curation system.

It is important for developers to distinguish between the total processed collection and the public download. While the curation system processed approximately 843,600 unique audio hours, the public release is specifically limited to the 463,062-hour figure. This distinction ensures that users understand exactly what data is bundled for immediate use versus what was handled during the broader curation process.

The dataset is designed to support multiple representations and tasks for each recording. This means a single piece of audio can be associated with various types of data, such as normalized transcripts or translations. By providing these diverse labels, NVIDIA aims to give developers a versatile starting point for different types of speech AI applications.

How to access the data

The Granary-v2 data is hosted on Hugging Face, but the repository does not contain the actual audio files. Instead, it provides compressed JSONL label manifests. These manifests contain an audio_filepath for each record, which acts as a unique identifier to help users locate the recordings in their original source corpora.

To actually use the data for training, developers must identify the correct manifest and then obtain the audio separately. The dataset card identifies the specific distributors for each recording, but users must follow those distributors' individual access requirements and terms. This two-part preparation process is necessary because the labels and the source audio are managed under different rules.

Licensing for the released material varies depending on the specific subset chosen. Most of the material follows the OpenMDW 1.1 terms, but the People’s Speech subset is governed by CC BY-SA 4.0. This exception also applies when accessing records through the combined English configuration, requiring developers to be diligent about which license applies to their specific use case.

Data types and quality

Granary-v2 can attach a wide variety of targets to a single recording, including captions, code-switched text, and speech questions. It also supports translations and inverse-normalized transcripts. However, the coverage of these fields varies from record to record, meaning a training data loader must be able to handle missing values gracefully.

The dataset card includes specific schema notes to help developers filter the data correctly. For example, a translation_raw field might be an empty string if a candidate was rejected, so users should check for a non-empty value rather than just the presence of the key. Similarly, a SKIP value indicates that a speech question-answer target is not applicable to that specific recording.

NVIDIA also includes a high_quality flag to indicate whether a recording passed the pipeline's internal quality gate. Approximately 165,300 hours in the public release met this standard. The documentation notes that this flag is a result of the pipeline's own filtering and does not serve as a guarantee of human verification for every individual target.

Training and tooling

The project overview links to a curation pipeline built with NeMo Curator and includes model-training recipes. While the pipeline source is currently public, NVIDIA has stated that full end-to-end runs and per-stage scripts will follow at a later time. This means developers have the foundation now but may need to build out some of the specific execution steps themselves.

One specific feature of the release is the preference_instructions, which is a derived training view. This view packages existing fields into prompt-target pairs without adding any new information to the dataset. Developers training a standard speech-recognition or speech-translation model do not need to treat these instruction pairs as an additional underlying dataset.

Because translation targets reuse audio that is already counted in the speech-recognition totals, they do not count as extra source-audio hours. This is a key detail for developers who need to accurately estimate the amount of material available for specific tasks. The release is structured to provide a clear view of the data while maintaining the integrity of the original audio counts.