We help rightsholders solve some pretty technical problems, like finding all versions of a song online even if it’s been sped up, chopped up, or mashed up. We get into the hairy details of cover versions and publisher splits. But, sometimes the problem plaguing rightsholders the most is something much simpler. Something like, does this file contain music? And if it does, where in the file is the music? In order to answer these seemingly simple questions, you need technology to first understand what is considered music, and what is considered other types of audio or noise. Then, you need a system to actually identify and label the sounds in a file. With our latest Pex Search release, we’ve done exactly that, and rightsholders or platforms can now check any file to determine which segments are music, speech, or silence. We call this content classification and will explain more about how it works below.
Content classification in Pex Search
What IS music? When we ask that question, do we mean full songs with verses, bridges, and choruses? Or do we mean any amount of instrumentation, like a chord progression or drum beat? Based on rightsholder and platform needs, we built our technology to classify certain audio sounds as music and others as non-music or speech.
Examples of sounds classified as music
- A track, or part of a track, containing identifiable music
- Comedy with musical elements involved
- Musical prayers, religious songs, and chants
- Quiet background music (e.g. in the background of a fire video)
- Musical box tune
- Spa and meditation music
- Rapping containing rhythmic elements
Examples of sounds classified as non-music
- Non-rhythmic speech, without musical work in it (comedy, ebooks, poetry)
- Nature sounds
- Pure bird song, not performed by humans
- Videos with background music
- Vocal technique demonstration videos
- Meditation noises with no or minimal pitch changes
- Singular sine waves with no pitch changes
- DTMF and random dial signals
- Any song that is fully hosted on a DSP but contains only silence
Examples of sounds classified as speech
- Human vocalizations (conversation, narration, whispering, child speech)
- Expressive tones (shouting, laughing, screaming)
- AI-generated voices, synthetic speech
To better understand, check out these videos on YouTube which represent audio we classify as music or containing music.
A track containing music
Comedy with musical elements involved
Musical prayers, religious songs, and chants
Rapping/vocals containing rhythmic elements
We followed a similar approach to train our models on what is considered speech and what is considered silence. With these three things determined, we added content classification to Pex Search, so rightsholders and platforms can identify what audio is present in a file.
Content classification use cases
While there are certainly more technical and granular problems facing the music industry, determining whether or not music is present in a file has several valuable use cases.
- Checking podcasts for music: For podcast hosting platforms and streaming services, content classification can quickly verify that there’s no copyrighted music present in a podcast and that it’s safe to publish. Or, it can identify where there is music present as well as what song is being used.
- Determining where music is used in a live stream or broadcast: Even when live streams or broadcasts aren’t about music or concerts, they often contain clips of music. Since these files tend to be longer and primarily contain speech, content classification can easily determine if music is used and where.
- Filtering and verifying large amounts of content or reports: Social media and user-generated content platforms often create reports of content on their sites that contain music versus content that doesn’t. Based on the content with music, the platforms identify rightsholders or may pay licenses based on market share. Content classification can verify these reports are accurate and that non-music files truly contain no music.
- Checking social media uploads for copyright risk: Companies with a social media presence often want to ensure their uploads are free from copyright infringement, or that any content their employees and influencers share doesn’t contain music. If licensed music is allowed, brands can use Pex Search to ensure the correct music is being shared. If no music is allowed at all, brands can use content classification to block any posts containing music.
Get started with Pex Search and content classification
Whether the music problems you need to solve are big or small, Pex identification technology can help. With Pex Search, you can find modified audio and cover versions, audit social media content, ensure licensed music is used, or check files for infringement before distributing them. Now, with content classification, you can determine if any file contains music and know where it is, even if it occurs in multiple spliced segments.
Reach out to our team today for more information or to get started with Pex Search.
