In late 2022, while everyone was using Stable Diffusion to generate images, two engineers had a wild idea in their spare time: could we use an image-generation model to "paint" music? The concept was so ingenious it bordered on eerie—since sound can be visualized as a spectrogram (an image with time on the x-axis and frequency on the y-axis), why not have SD generate new spectrograms and then convert those images back into audio? They actually did it, open-sourced the project, and it instantly went viral in the tech community. Riffusion thus became a landmark "hacker-style innovation" in the history of AI creation: using an image model to handle audio tasks, proving that generative AI capabilities can be transferred across modalities.
The story has a sequel: The Riffusion team later pivoted to more specialized audio models, launching serious AI music products—but it is that initial "painting music with SD"creative concept (brainstorm/idea) that cemented its name in tech history.
The brand and owner have changed three times (checked 2026-09-06): riffusion.com → producer.ai → flowmusic.app, both hops being 301 redirects. The timeline: Riffusion became Producer.ai in mid-2025; Google announced the acquisition in February 2026 and told users to export their work before February 20, after which past generations, sessions, and models stopped being accessible; and in April 2026 it was renamed Google Flow Music, with the team folded into Google Labs and DeepMind and the underlying stack switched to Google's Lyria music models.
Two consequences: the interface and workflow described below reflect the Riffusion era and no longer apply, and anything generated today falls under Google's terms — re-check the licence before using it commercially rather than relying on what was true before.
What is Riffusion?
Riffusion (riffusion.com) initially used Stable Diffusion to generate spectrograms as images and convert them into audio. It produced clips from style descriptions, with open-source code for studying that approach.
The key to understanding it lies in its positioning: it is primarily a technical experiment, secondarily a tool—which means both its charm and its limitations stem from the same source.
Core Features
Text-to-Music
Input a style description ("acoustic guitar, folk, relaxing" or "electronic dance, energetic") to generate corresponding snippets. The style coverage is extremely diverse, ranging from folk to electronic, jazz to hip-hop. However, the quality carries a distinct "experimental feel"—it is hit-or-miss and unstable. This is precisely the side effect of using an image model for audio: its "painting" of spectrograms doesn't always translate back into pleasing sound. This unpredictability is seen as a flaw by those seeking finished products, but as a surprise by those experimenting.
Style Interpolation: A Signature Feature
Riffusion’s most interesting and rarest capability is performing "interpolation" between two styles—gradually transitioning from "calm piano" to "intense metal," generating a continuous musical passage. This directly benefits from its image-model DNA (latent space interpolation in SD is a classic technique for image generation, applied here verbatim to music). Mainstream AI music tools largely lack this kind of style fusion and creative exploration; it is the most shining aspect of Riffusion’s technical aesthetics.
Image-to-Sound
Because it fundamentally processes spectrograms, any image can be "read" by it as sound—a photo converted into audio. This is a purely experimental, artisticgameplay (way of playing/usage). It may be useless in daily life, but it holds unique exploratory value in sound art and cross-media creation.
Open Source and Local Deployment
The code is fully open-source, allowing for local execution and secondary development—this is the foundation of its enduring influence on the research community and the best footnote to its identity as an "experimental project."
Comparison with Similar Tools
vs Suno/Udio: The current kings of finished AI music products (full songs, vocals, stable quality); in terms of producing final products, Riffusion is no match. But while Suno and Udio are "products," Riffusion is an "experiment"—if you need usable songs, go to the former; if you want to play with technical boundaries and style interpolation, go to the latter. They are not on the same track.
vs Stable Audio: The "regular army version" of a similar lineage—Stability AI uses specialized audio models for generation, offering sound quality and stability far surpassing Riffusion’s brute-force adaptation of image models. In a sense, Stable Audio follows the path inspired by Riffusion but is more professional.
vs Meta AudioCraft/MusicGen: Both are open-source audio generators; AudioCraft uses a specialized audio architecture for more stable quality. Both are research-friendly open-source options, with Riffusion winning on its unique technical story and ready-made online interface.
vs Mubert: Mubert focuses on functional background music streams, serving a completely different scenario.
Who Should Use Riffusion?
Those curious about AI technology: The very idea of "painting music with an image model" is worth understanding—it is the most vivid case study for grasping "cross-modal transfer in generative AI," offering greaterpopular science (popular science/educational) value than practical utility.
Experimental creators in music and sound: Features like style interpolation and image-to-sound offer uniqueness that mainstream tools cannot provide in sound art and creative exploration—for those who do not seek finished products but rather possibilities.
Developers and researchers: The open-source code serves as an excellent reference for learning AI music generation and secondary development, acting as a perennial textbook in the tech community.
Nostalgia and history enthusiasts: Those who want to experience what the landmark project of the "Year One of AI Music" looked like.
Limitations
To be honest: As a production tool, its quality is unstable, snippets are short, and it cannot generate complete songs—it was never designed to "produce usable music." For stable final products, Suno/Udio are the answer; don’t force Riffusion to do what it wasn’t built for.
Like many research projects, updates and user support are less proactive than commercial products; issues often require self-help. Moreover, its original form of "SD painting spectrograms" has been surpassed by the team’s own newer products and more specialized solutions.
Pricing
The online version offers free trials; the open-source version can be deployed locally for free. The team’s subsequent commercial products have their own pricing, subject to the current state of their official website.
Riffusion’s experiment represented audio as spectrograms, generated those images with an image model and converted them back to sound. It demonstrated one route from image generation to audio; music-tool selection still depends on output quality and editing controls.