Block unveils Dessa AI trained exclusively on spoken conversations

A more human dialogue

Block, the company founded by Jack Dorsey, recently showcased a new AI model called Dessa. The demonstrations highlight an AI capable of engaging in incredibly fluid conversations that mimic human behavior closely. In one example, the AI identifies a movie title before the user even finishes describing it.

Observers noted that the interaction feels remarkably alive compared to other current models. It does not simply wait for a user to stop speaking before it begins its response. Instead, it participates in the flow of the conversation with natural timing and reactions.

The model is designed to reproduce the specific imperfections of human speech. As reported by numerama.com, it can laugh in the middle of a sentence or speak louder when asked. These behaviors make the experience feel less like a machine and more like a real person on the other end of the line.

Learning through sound

The core differentiator for Dessa is its training methodology. While most large language models are trained on massive amounts of text, Dessa was trained exclusively on spoken conversations. The developers claim the model has never 'read' a single word during its learning process.

The team compares this approach to how a child learns to speak before they learn to read. By focusing on audio, the model captures nuances that are typically stripped away during transcription. This includes specific intonations, sighs, pauses, and the emotional weight behind certain words.

For example, the words 'I'm fine' can mean very different things depending on the tone of voice. Because Dessa learns from the raw audio, it can theoretically pick up on these differences. The goal is for the AI to understand the underlying sentiment of a conversation rather than just the literal meaning of the words used.

Real-time interaction

Dessa utilizes a technology known as full duplex communication. This allows the AI to listen to and speak at the same time, which is essential for natural dialogue. Because it can process incoming audio while it is still talking, it can react to things the user says in real-time.

This capability enables the AI to perform actions like nodding with a verbal 'yes' during a long explanation. It can also interrupt the user or be interrupted without breaking the flow of the conversation. This mimics the way humans overlap and weave their words together during a standard chat.

While other models like Moshi have explored simultaneous voice exchanges, they often rely on underlying text models. Block's approach seeks to bypass that text layer entirely. By staying in the audio domain, the model aims to maintain a more seamless and less mechanical experience for the user.

Security and local power

The developers believe that training on audio is also a matter of safety and utility. An assistant that can hear distress in a person's voice might be able to react appropriately even if the words spoken do not clearly express a problem. This focus on emotional awareness is a key part of the project's vision.

However, the model is currently in a research phase and is not yet available to the general public. Block is providing early access only to a small group of AI safety researchers. Their task is to identify potential risks and unintended behaviors that the development team may have missed.

Technically, the model is designed to run locally rather than in the cloud. According to the source, it can operate on a MacBook Pro equipped with an M4 Max chip or something more powerful. This local execution offers a different path for privacy and speed compared to traditional cloud-based AI services.

What comes next

Because Dessa is currently a research preview, there is no official release date or pricing information available. The focus remains on refining the model's ability to handle complex human nuances without the guidance of text-based training. The team is looking for feedback from safety experts to ensure the technology is handled responsibly.

The success of Dessa will likely be measured by how well it handles these 'human' elements in practical scenarios. If it can successfully recognize and react to emotional cues, it could change how we interact with digital assistants. It moves the needle closer to the type of seamless interaction seen in science fiction.

For now, the public can only view the demonstrations provided by Block. These clips serve as a proof of concept for a new way of building AI. The next step involves rigorous testing to see if these natural interactions can be scaled safely and effectively for wider use.

Technical details

Dessa's architecture is built to handle the complexities of human speech patterns. By avoiding a pre-trained text model, the developers aim to preserve the integrity of the audio signal. This includes the subtle variations in pitch and speed that define human communication.

The full duplex system is what allows for the overlapping speech that makes conversations feel real. In standard models, there is often a distinct pause between the user finishing and the AI beginning. Dessa removes this barrier, allowing for a continuous stream of interaction.

The hardware requirements for local operation are significant, requiring high-end chips like the M4 Max. This suggests that while the model is powerful, it is currently optimized for high-performance personal computers. This local approach may be a key factor in how the company plans to handle data and privacy in the future.

The role of emotion

One of the most striking features of Dessa is its ability to convey and interpret emotion. The developers want the AI to understand that a sigh or a laugh is a piece of information just as important as a word. By training on raw audio, the model learns these non-verbal cues as part of the language itself.

This is particularly important for applications where the AI might need to detect a user's state of mind. If a user sounds frustrated or upset, a model that understands tone can provide a more appropriate response. This goes beyond simple keyword recognition and into the realm of emotional intelligence.

While the current demos are curated by Block, they show a high level of sophistication in how the AI handles these nuances. The goal is to create a system that doesn't just follow instructions but understands the context of the human experience. This represents a shift toward more empathetic and responsive AI technology.

Comparison to others

Dessa stands out from other recent voice models, such as those released by Gemini or ChatGPT. While those models have made strides in voice quality, they are often perceived as more mechanical in their interaction style. They frequently follow a strict turn-taking structure that can feel less natural in a fast-paced conversation.

The French laboratory Kyutai also developed a model called Moshi that handles simultaneous voice exchanges. However, Moshi still relies on a language model trained on text. Block's decision to skip the text phase is what they claim makes Dessa unique in its ability to capture the 'soul' of spoken language.

By focusing on the audio-only path, Block is attempting to solve the problem of 'lost in translation' errors. When speech is converted to text and back to speech, the original nuance is often destroyed. Dessa aims to keep that nuance intact by staying entirely within the sound domain from start to finish.

The future of speech AI

The development of Dessa signals a move toward more immersive and human-centric AI. As the technology matures, we may see a shift away from typing into boxes and toward constant, fluid verbal interaction. This could change how we use personal assistants, educational tools, and communication software.

The current focus on safety research is a necessary step before any wide release. Ensuring that an AI that can detect emotion doesn't also create new risks is a primary concern for the researchers. They need to understand how the model handles ambiguous emotional signals before it reaches the public.

Ultimately, Dessa represents a gamble on the idea that audio is a richer medium for learning than text. If successful, it could lead to a new generation of AI that feels less like a tool and more like a companion. It is a significant step toward the goal of making AI interactions feel truly human.

Local vs Cloud

A notable technical choice for Dessa is its ability to run locally on a MacBook Pro. This is a departure from many large models that require massive cloud infrastructure to function. Running locally can offer significant advantages in terms of speed and user privacy.

By using a chip like the M4 Max, the model can process complex audio in real-time without the latency of sending data to a remote server. This is crucial for a full duplex system where every millisecond of delay can make a conversation feel awkward or broken.

This hardware-centric approach suggests that Block is prioritizing a high-quality, low-latency experience for the end user. It also means that the model is optimized to be efficient enough to run on consumer-grade high-end hardware. This could pave the way for more private and responsive AI interactions in the future.

Training nuances

The training data for Dessa is described as being purely human conversations. This means the model learns from the way people actually talk, including their slang, their hesitations, and their unique styles. It doesn't have the 'polished' feel of a model trained on written literature or Wikipedia entries.

This distinction is important because written language is often structured and formal. Spoken language is messy, repetitive, and full of interruptions. By training on the latter, Dessa learns to navigate the messiness of human interaction without getting confused by it.

The developers see this as a way to capture the 'flavor' of a conversation. When you speak with a friend, you aren't just exchanging information; you are sharing an experience. Dessa is being built to capture that experience by learning the sounds of human connection rather than just the rules of human grammar.

The role of interruptions

One of the most human aspects of conversation is the ability to interrupt. In many current AI systems, if you speak while the AI is talking, it might stop or ignore you. Dessa is designed to handle these overlaps naturally, just as people do in a lively debate or a casual chat.

This is achieved through its full duplex architecture, which allows it to process your voice even while it is generating its own audio. It can react to a sudden 'wait' or a quick question without needing to finish its current thought first. This makes the interaction feel much more dynamic and less like a series of commands.

These interruptions are what give the AI its 'living' quality. They allow for a back-and-forth flow where the AI can slide in a quick 'yeah' or a laugh. It is these small, non-verbal cues that make the difference between a functional tool and a natural-sounding conversationalist.

Safety and detection

The project includes a significant focus on safety, specifically regarding the detection of human distress. Because the model learns from audio, it can potentially identify when a user is upset or in trouble based on the sound of their voice alone. This could be a vital feature for emergency services or mental health support.

However, the team is being cautious about how this works. They are currently working with safety researchers to ensure the model doesn't misinterpret emotions or react in an inappropriate way. The goal is to create a system that is helpful and safe, not one that causes confusion or alarm.

This research phase is a critical step in the development of any AI that interacts with human emotions. By identifying risks early, the team hopes to build a model that can be trusted in sensitive situations. It shows that the goal of Dessa is not just to sound human, but to be a responsible and useful part of human life.

The future of local AI

The fact that Dessa is designed to run on a MacBook Pro points to a growing trend in local AI. As hardware becomes more powerful, the need for constant cloud connectivity for many tasks is decreasing. This allows for faster responses and better privacy for the user.

For a full duplex conversation, low latency is non-negotiable. If the AI takes a second to think before responding, the flow of conversation is ruined. Running on a high-end local chip like the M4 Max ensures that the interaction remains snappy and real-time.

This local approach also means that the data from these conversations doesn't necessarily have to leave the user's device. For a model that is designed to be intimate and human-like, this could be a major selling point for privacy-conscious users. It combines high-end performance with a more secure way of interacting with AI.

Summary of the approach

Block's approach with Dessa is a bold experiment in machine learning. By choosing to ignore text and focus entirely on audio, they are trying to capture the essence of human communication. It is a move toward a more organic, less structured way of interacting with artificial intelligence.

The model's ability to handle interruptions, laughter, and emotional tone sets it apart from current market leaders. While it is still in the research phase, the demonstrations suggest a high level of sophistication in how it mimics human speech patterns. It represents a significant step toward making AI feel less like a machine and more like a conversational partner.

As the safety researchers begin their work, the next few months will be crucial for determining the model's viability. If it can be made safe and reliable, it could redefine how we think about voice assistants. For now, it serves as a fascinating glimpse into a future where AI speaks not just with words, but with a human-like touch.

The impact of audio-only training

The decision to train only on spoken conversations has profound implications for how the AI understands the world. It means the AI's 'knowledge' is filtered through the lens of human speech. It learns what people say to each other, rather than what people write in books or on the internet.

This can lead to a more colloquial and relatable AI. It will understand the way people actually talk in their daily lives, including the shortcuts and emotional expressions they use. This makes the AI more suitable for casual, everyday use where formal language might feel out of place.

However, it also means the model's knowledge base is limited to what has been spoken in its training data. The developers are essentially trying to replicate the way humans learn to communicate. It is a specialized approach that prioritizes the 'how' of speaking over the 'what' of information retrieval.

The goal of natural dialogue

The ultimate goal of Dessa is to create a conversation that feels natural. This means moving beyond the 'input-output' loop that defines most current AI interactions. In a natural conversation, things happen simultaneously, and the flow is often unpredictable.

Dessa's ability to react to interruptions and use non-verbal cues like laughter is a direct attempt to achieve this. It aims to create a sense of presence, where the AI feels like it is actually listening and participating in the moment. This is what makes the demonstrations so striking to the public.

As the technology develops, this natural dialogue could become the standard for all voice-based AI. Instead of a robot that waits for its turn, we may see systems that can engage in the messy, beautiful, and complex way that humans do. Dessa is one of the first major steps toward that reality.

Hardware and performance

The technical requirements for Dessa highlight the power needed for real-time audio processing. To achieve full duplex communication, the AI must be able to analyze incoming audio and generate outgoing audio at the same time with minimal delay.

By targeting the MacBook Pro with an M4 Max chip, Block is ensuring that the model has the necessary computational power. This allows for a high-fidelity experience that doesn't rely on the variable speeds of the internet. Local execution provides a consistent and reliable experience for the user.

This hardware focus also suggests that the model is being optimized for high-performance personal computing. It is a move toward making powerful AI tools accessible on high-end consumer devices. This could lead to a new category of 'pro' AI experiences that are both powerful and private.

The research phase

Because Dessa is currently in a research phase, the company is taking a cautious approach to its rollout. They are not looking to release it to the general public immediately. Instead, they are focusing on the core technology and its safety implications.

The involvement of AI safety researchers is a critical part of this process. They are tasked with finding the 'edge cases' where the model might behave unexpectedly. This is a necessary step for any AI that is designed to interact so closely with human emotions and speech patterns.</