AI startup Stability AI has released Secure Audio Open Small, a “stereo” audio-generating AI model that the corporate claims is the quickest on the market — and environment friendly sufficient to run on smartphones.
Secure Audio Open Small is the fruit of a collaboration between Stability AI and Arm, the chipmaker that produces lots of the processors inside tablets, telephones, and different cellular gadgets. Whereas quite a lot of AI-powered apps can generate audio, like Suno and Udio, most rely on cloud processing, which means that they can’t be used offline.
Stability additionally claims that Secure Audio Open Small’s coaching set is made up fully of songs from the royalty-free audio libraries Free Music Archive and Freesound. That’s versus the coaching units of the aforementioned Suno and Udio, which reportedly include copyrighted content material, posing an IP danger.
Secure Audio Open Small is 341 million parameters in measurement and optimized to run on Arm CPUs. (Parameters, typically known as weights, are the inner parts of a model that information its conduct.) Designed for rapidly producing quick audio samples and sound results (e.g., drum and instrument riffs), Secure Audio Open Small can produce as much as 11 seconds of audio on a smartphone in lower than 8 seconds, claims Stability AI.
Right here’s a pattern generated by Secure Audio Open Small:
And right here’s one other one:
The model isn’t with out its limitations. Secure Audio Open Small solely helps prompts written in English, and Stability notes in its documentation that the model can’t generate real looking vocals or high-quality songs. The model additionally doesn’t carry out equally nicely throughout musical types, Stability warns — a consequence of its Western-biased coaching information.
In one other potential wrinkle for devs, Secure Audio Open Small has considerably restrictive utilization phrases. It’s free to make use of for researchers, hobbyists, and companies with lower than $1 million in annual income, however builders and organizations making over $1 million in income should pay for Stability’s enterprise license.
Stability, the beleaguered agency behind the favored picture era model Secure Diffusion, raised new money final yr as buyers, together with Eric Schmidt and Napster founder Sean Parker, sought to show the enterprise round. Emad Mostaque, Stability’s co-founder and ex-CEO, reportedly mismanaged Stability into monetary destroy, main employees to resign, a partnership with Canva to fall by way of, and buyers to develop involved concerning the firm’s prospects.
In the previous couple of months, Stability has employed a brand new CEO, appointed Titanic director James Cameron to its board of administrators, and launched a number of new picture era fashions.
