by: adminPosted on:

AI Music Generation just got a whole lot better


The day when you can entirely generate your own songs using AI, ready for a club play, is drawing nearer.

Last week, researchers from Meta introduced a new AI model for music generation named Audio Craft- and it’s impressively good. This marks a significant stride towards the future of AI DJing, demonstrating a considerable improvement in quality. Let’s review some examples and explore how you can use it yourself.

In my last article , “The Golden Age of DJing,” I envisioned a future where DJing and production merge into a single generative process. The tech examples were a bit raw then, but now we have a better proof of concept that you can try for free.

Two models have shown the most promise in generative music AI for producers and DJs:

  1. Google’s “MusicLM” –
    – Limited to 12 seconds of generation
    – Requires waitlist sign-up to use
  2. Meta’s “Audio Craft”
    – Public version is limited to 12 seconds (However, you can generate longer pieces of music using code forks.)
    – Available for anyone to use
    – Generally, it seems to produce superior results.

Let’s delve deeper into the second one and see what it can do.

Meta Researchers Release “Audio Craft” AI Music Generation

Although it’s not quite studio ready, this latest model is producing some genuinely intriguing audio outputs.

There’s one catch: each generation round using the demo bellow produces only 12 seconds of audio. You could run the tool several times, accumulate various outputs, and stitch them all together for something original.

Try it yourself with the free Hugging Face demo of Meta’s new AI Music generation model, Audio Craft.

Kinda basic? Don’t worry – there is a better version for you to try later in the article.

IS OPEN SOURCE AI MUSIC GENERATION THE FUTURE?

The AudioCraft model, developed by a team of researchers from Meta, is fully open source. Within days, numerous forks of the original code had been created, generating impressive enhancements. This progress makes me confident that we will witness rapid developments in music generation by AI in the coming months.

There are several forks of the AudioCraft code on GitHub, which offer much longer run times and a host of interesting features. One particular version by Granddaddy Shmax can be run on your computer or Google Colab (with a good GPU).

Colab: https://colab.research.google.com/github/camenduru/MusicGen-colab/blob/main/MusicGen_ClownOfMadness_plus_colab.ipynb

Repo: https://github.com/GrandaddyShmax/audiocraft_plus/tree/plus

This custom version features:

  • sample and melody inputs
  • long run times (up to 2/3 minutes)
  • ability to generate sections which blend into each other using different prompts (parts of the song)

I generated a track using the following sequence of prompts and the melody of Eleanor Rigby from The Beatles.

This is the result of the above prompts. Not exactly what I had in mind – but musically interesting!

Here is another version my friend Ryan Lucero made using the simple prompt: “Deep house 124bpm.”

GENERATING AN AI “REMIX”

The promise of AI for DJs lies in generating new genres and combinations we have never seen before. As a DJ, there are so many precise subtle micro-genre’s we recognize and look for, but no one seems to have the time or interest in producing all of our whimsical fantasies. Theoretically, generative AI music solves this problem.

One example is songs with a strange, funky swing – rare but very effective when you find them. Frank Sinatra is a perfect example – so I thought, what if Skrillex and Frank Sinatra had a production baby? Here are the steps I took to answer that really bad question.

  • Purchased Frank Sinatra’s "The Way You Look Tonight" on iTunes
  • Imported into Ableton and created a short loop of the melody I wanted
  • Entered that in the Hugging Face AI generation demo with the following description:

“An electronic drum and bass song, BPM:128, with the swing and musical style of Frank Sinatra’s ‘The Way You Look Tonight.’”

The first results, admittedly, were pretty bad:

• BPM was correct.

• Key was right

• The swing was off

• Music quality generated by the AI was decent

• Far from a loop, it was an awkward mix of parts.

• This took about 1 minute to generate.

Then I tried again with slightly new prompts

"An electronic drum and bass song with minimal musicality, BPM:128, with the swing of Frank Sinatra's 'The Way You Look Tonight.'"

Closer, but far from being something I could DJ with. Cheesy and weird in a good way.

Final try with a new prompt that gave me something rather delightful.

"A minimal drum and bass song that's dark and moody, BPM:128, with the swing of Frank Sinatra's 'The Way You Look Tonight.'"

SO, IS AI MUSIC GENERATION READY FOR DJS?

No, it’s not there yet

  • The outputs have low audio quality (32kbps sample rate) and are widely varied in terms of music quality.
  • The BPMs generated are sometimes inconsistent and off the grid, making editing very difficult.

Along with the above issues, the model seems to have very little adherence to strict musical structures and will jump around a lot. They are, however, incredible writing and creative tools. Want to generate some crazy loops or samples to incorporate into a new song quickly? This could be an invaluable asset to your workflow, but you will need to correct the timing of the samples and try a lot of rolls.

More than anything, I see this development as further proof that music generation AI is going to be a game-changer. Stay tuned. I am captivated by this field and will be writing about it a lot more in the future.



Source link