AI Music Generation: From Prompt to Production

Goal: For the workshop to be interactive, for it to be fun, and for you to walk away with something tangible.

• Scaling sounds: is AI Music commercially viable?

  • How do we define AI Music?
    • Compartments
      • Style Transfer: Audio Filter, Voice conversion
        • AI Weeknd & AI Drake - Heart on my Sleeve
        • Randy Travis - Where That Came From
        • Kanye West AI Vocal Process
      • Text to Music
        • Meme Songs
          • BBL Drizzy by King Willonius (1.5M)
          • Started To Sing by ElevenLabs
        • AI Samples
          • Sound FX
          • One Shots
          • Loops
            • piano
            • drums
      • Audio to Music
        • Beatbox to real drums using Stable Audio 2.0
        • Cute dog LoFi Remix using Suno
        • Finger drumming to punk rock guitar using Udio
    • Misnomer
      • is it really AI Music if AI is a tool the human is using to create?
      • maybe AI Assisted Music is more accurate
        • akin to Photographers reaction to Photoshop
        • or musicians and drum machines
  • Using the music commercially
    • Terms of Service
      • Udio
        • FAQs
        • TOS points
          • Very broad license to your inputs and outputs
          • Peculiar attribution (recently walked back)
          • Only 30 days to opt out of the arbitration clause
      • Suno
        • FAQs
        • TOS points
          • Very broad license to your inputs and outputs
          • Allows for Midjourney style display of outputs
          • You can only use the song commercially if you are a paid subscriber
    • Defending your IP
      • Works created solely by AI are not eligible for copyright protection
      • Human authorship is a fundamental requirement for copyright
        • If the "traditional elements of authorship" (like melody, harmony, rhythm in music) are determined by AI without human creative direction, the work is not copyrightable.
        • Simply providing a prompt to an AI system is not enough to establish human authorship.
        • The human must have creative control over the output, not just influence over the general topic or style.
      • What qualifies as sufficient human input?
        • A human creatively selecting and arranging AI-generated musical segments could be eligible for copyright protection.
        • A human significantly modifying AI-generated music to add original expression could make the modified work copyrightable.
      • Best case scenario: You go viral. Anyone going viral using your idea gives you credit.
    • Sync license for content creation
      • Services
        • Artlist
        • StoryBlocks
        • Track Club
      • Related: Trailer Music
    • Distributing music as an artist
      • Services
        • TuneCore
        • United Masters
        • CD Baby
  • Perspective
    • I think the current use case of Udio and Suno as meme song generators is a lot of fun but
      • Producers want more control. Will create at a greater pace given more tools.
      • Side note: compulsory mechanical license for music covers
    • This will make music production more accessible. Will be seen as tools just like a DAW or a plugin. Many integrations to look forward to.
    • The human connection in music triumphs.
    • The US Copyright Office's stance right now is AI gen cannot be copyrighted. This has not been held up in the Supreme Court yet. We'll see how it plays out in the future.
  • Where we are headed
    • an AI assisted DAW
    • text to music as Figma
      • as Canva
      • as Photoshop

• Prompting

  • Udio Tips
  • Suno Tips
  • GPT Prompt
    • [Song Description - A brief description of the song's theme and mood]
    • "< Description >"
    • [Style Tags - Keywords that describe the music and feel of the song]
    • <Tag 1>, <Tag 2>, <Tag 3>, ...
    • [Intro - Musical and lyrical setup of the song - ]
    • < Lyrics >
    • [Verse 1 - Development of the song's story and themes - <Verse 1 music and vocals description>]
    • < Lyrics >
    • [Chorus - The catchy and repetitive segment that outlines the song's main themes - ]
    • < Lyrics >
    • [Instrumental Break - A non-vocal segment that highlights the instrumental prowess - ]
    • [Verse 2 - Further development of the story/theme - <Verse 2 music and vocals description>]
    • < Lyrics >
    • [Chorus - Repetition of the chorus to reinforce the song's themes - ]
    • < Lyrics >
    • [Outro - Concluding the song - ]
    • < Lyrics >
  • Music Model Prompts
    • a children's birthday sing-along about Alice visiting the zoo
    • Torchy-lounge song from the POV of a piano player working at a speak-easy who wants to go home because "It's Closing Time"
    • Sultry RnB
    • Upbeat Country, Female vocal
    • Mississipi Blues, sparse harmonica, acoustic guitar, stomp
    • bittersweet Hindustani Electro-pop, melodic sarad, ornamental singing
  • A prompt master at work
    • https://www.udio.com/songs/5Rt22swvR7hpLWfsgJtU7i
    • https://www.udio.com/songs/mqogi2xJvNywqaT46soSwU
    • "Elgareth" in the Udio discord

• List of research sources

  • Podcasts
    • Making Transformers Sing on Latent Space
    • Can AI Make Music That's Worth Listening To
    • How A.I. Has Changed Music, and What’s Coming Next
    • Why AI Can Never Truly Replace All Musicians feat. Rick Beato
    • Is AI Music for the Lazy and Untalented on The AI Music Podcast
  • Videos
    • Music Attorney discusses Udio TOS
    • AI's Threat To Music
  • Articles
    • Harvard Law on Heart on my Sleeve (AI Weeknd & AI Drake)
    • ELVIS Act becomes law in TN
    • RIAA vs Suno, Udio
  • Govt Policy
    • State of AI and Copyright Law in 2023
    • USCO Letter on AI and Copyright
    • Copyright Office Request for Comments
    • Copyright Registration Guidance for AI Works
  • Court Cases
    • Thaler v. Perlmutter
    • Anderson v. Stability AI
    • Getty v. Stability AI
    • RIAA v. Suno
    • RIAA v. Udio
  • Community Discussions
    • Potential Suno Workflow
    • Generate samples over songs
    • Argument against more control for casuals
  • AI Music Papers and Models
    • AudioLM
      • Paper
      • Blog
      • Examples
      • IEEE
      • AudioLM Implementation in PyTorch
    • MusicLM
      • Paper
      • Examples
      • MusicLM Implementation in PyTorch
    • SoundStream
      • Paper
      • Blog
      • Follow up: SoundStorm
    • AudioCraft
      • Code
      • AudioGen Paper

        Diffusion

      • MusicGen Paper

        Language Model

    • Machine Learning and Audio
      • https://wandb.ai/wandb_gen/audio/reports/A-Technical-Guide-to-Diffusion-Models-for-Audio-Generation--VmlldzoyNjc5ODIx
      • https://www.reddit.com/r/MachineLearning/comments/j9d0tj/d_good_reference_for_audio_processing_and_deep/
      • https://www.reddit.com/r/audioengineering/comments/18xn905/opinions_on_machine_learning_within_audio_plugin/
      • https://www.reddit.com/r/MachineLearning/comments/razoa5/d_why_is_audio_so_far_behind_other_ml_application/
      • https://www.reddit.com/r/learnmachinelearning/comments/1c2sbf4/how_do_ai_music_generation_tools_work/
    • Audio Classification
      • https://ieeexplore.ieee.org/document/10258355
    • ASR Papers
      • https://github.com/jonnor/machinehearing
    • List of machine learning audio papers
      • https://github.com/GuitarML/mldsp-papers

  AIE Summit Talk 2023

  Visualizing Datasets