Skipping the text step
Most voice assistants today first turn what you say into text, hand it to a language model, then synthesize the answer back into speech. Even end-to-end speech models usually sit on a base pretrained on huge amounts of text. Dessa, which Block unveiled on October 23, takes a different route: its official page says it was "trained from scratch on nothing but human conversation", with no text and no pretrained models, much as a baby learns to talk by listening long before it learns to read.
Block's argument is that converting speech to text throws away a lot of information: tone, pauses, breathing, sarcasm, a voice cracking, and the way the same sentence changes meaning depending on which word is stressed. Dessa is a full-duplex speech-to-speech model, with audio in and audio out, so it can listen and talk at the same time. From real conversation it learned when to cut in and when to backchannel with "yeah" or "got it" to show it is following. Block says inference runs faster than real time and the model runs locally on a MacBook Pro M4 Max or better.
One rack, hundreds of times fewer words
Data efficiency is the hard part of learning language from sound alone. Block's page explains that the same sentence said twice still produces different data, and that a single spoken word takes thousands of times more bytes than the same word as text. According to Block, Dessa learned language from hundreds of times fewer words than large language models, a comparison Block itself says is not quite fair, since LLMs also have to learn things like C++. On compute, Block says the whole model was trained on a single NVIDIA GB200 NVL72.
The page includes more than a dozen live recordings of people talking with Dessa. Block says nobody steers it: the model picks what to say, when and how, down to its own voice and personality, and subtitles were added afterward. One clip has Dessa talking directly to GPT-Live-1, another company's voice model; the page notes that its developer is not affiliated with Block and has not reviewed the demo.
Who built it, and can you use it
Dessa comes from a "(very) small" AI research lab inside Block whose members came from Dessa, the Toronto deep learning startup the model is named after. The team says it has built generative speech models for more than eight years and reports directly to Block cofounder Jack Dorsey. Lab lead Ragavan Thurairatnam called the release "a GPT3 moment for speech" in a post on X (as quoted by Unite.AI).
For now Dessa is a research preview. The public can join a waitlist, and a small group of AI safety researchers has early access to look for risks the team has not anticipated. The page offers no public API, model weights or downloadable app. Block also lists open research questions: how efficiently a speech-only model learns facts compared with text, how much data it needs for a new language, and what kinds of in-context learning it can do.
Keep in mind that all the recordings were selected and published by Block, and both "trained only on speech" and "hundreds of times fewer words" are Block's own claims; the page does not disclose the size or sources of the training data. Its factual knowledge, stability over long conversations and safety behavior will need independent assessment once researchers have access.
via: Block: Dessa - the Conversation Model trained only on speech, Unite.AI: Block Unveils Dessa, a Full-Duplex Speech Model Trained on Conversation