Python Audio Separation Libraries allowing you to isolate individual tracks

Python offers a variety of libraries for audio separation, allowing you to isolate individual instruments or vocals from a mixed audio track. Popular choices include Librosa, a powerful audio analysis and manipulation library with functions for source separation using techniques like Non-negative Matrix Factorization (NMF) and Deep Neural Networks (DNNs), and Spleeter, a deep learning-based audio source separation tool that can separate vocals, drums, bass, and other instruments with impressive accuracy. These libraries offer pre-trained models and flexible APIs, enabling you to easily incorporate audio separation into your Python projects for music production, analysis, and more. 
 

Here’s a list of high-quality audio separation libraries in Python, along with examples for each. These libraries are designed to handle audio with intricate vocal and instrumental components, delivering precise separation results.

1. Demucs

Demucs, developed by Facebook AI Research, is a state-of-the-art model for music source separation. It uses deep convolutional and recurrent layers, which significantly improve performance for complex mixtures. Demucs is especially good at separating vocals from intricate background music.

Installation

pip install demucs

Example

import subprocess
from moviepy.editor import VideoFileClip, AudioFileClip

def separate_with_demucs(input_audio_path):
    command = f'demucs --two-stems=vocals {input_audio_path}'
    subprocess.call(command, shell=True)

# Step 1: Extract audio from video
video = VideoFileClip("input_video.mp4")
audio_path = "input_audio.wav"
video.audio.write_audiofile(audio_path)

# Step 2: Run Demucs to separate vocals and accompaniment
separate_with_demucs(audio_path)

# Step 3: Load separated vocals and replace in video
vocals_path = 'separated/vocals.wav'  # Demucs output folder
vocals = AudioFileClip(vocals_path)
final_video = video.set_audio(vocals)
final_video.write_videofile("output_video.mp4", codec="libx264", audio_codec="aac")

2. X-UMX (Open-Unmix)

Open-Unmix, or X-UMX, is an open-source deep learning-based audio separation library for source separation. This library can separate vocals and accompaniment or even drums and bass if needed. It's easy to customize and works well on audio files with distinct frequency patterns for each source.

Installation

pip install openunmix

Example

from openunmix import predict
from moviepy.editor import VideoFileClip, AudioFileClip

# Step 1: Extract audio from video
video = VideoFileClip("input_video.mp4")
audio_path = "input_audio.wav"
video.audio.write_audiofile(audio_path)

# Step 2: Use Open-Unmix to separate vocals
output_path = 'output_vocals.wav'
predict.separate(audio_path, targets=['vocals'], output=output_path)

# Step 3: Replace audio in video with vocal-only track
vocals = AudioFileClip(output_path)
final_video = video.set_audio(vocals)
final_video.write_videofile("output_video.mp4", codec="libx264", audio_codec="aac")

3. Spleeter

Spleeter, developed by Deezer, is widely used for quick audio separation. While not as high quality as Demucs for complex tracks, Spleeter remains effective for general use and is known for its ease of installation and operation.

Installation

pip install spleeter

Example

from spleeter.separator import Separator
from moviepy.editor import VideoFileClip, AudioFileClip

# Step 1: Extract audio from video
video = VideoFileClip("input_video.mp4")
audio_path = "input_audio.wav"
video.audio.write_audiofile(audio_path)

# Step 2: Use Spleeter to separate vocals
separator = Separator('spleeter:2stems')  # separate into 'vocals' and 'accompaniment'
separator.separate_to_file(audio_path, 'output')

# Step 3: Load separated vocals and replace in video
vocals_path = 'output/input_audio/vocals.wav'  # Spleeter output folder
vocals = AudioFileClip(vocals_path)
final_video = video.set_audio(vocals)
final_video.write_videofile("output_video.mp4", codec="libx264", audio_codec="aac")

4. UVR (Ultimate Vocal Remover)

UVR (Ultimate Vocal Remover) is an advanced open-source tool that combines several separation models like Demucs and MDX-Net to provide multiple source separation options. It’s highly flexible, allowing users to experiment with different models and find the best one for their audio.

Installation

1. Install UVR’s dependencies:

   pip install demucs
   

2. Download UVR from its GitHub page for more advanced options.

Example (using Demucs through UVR)

Since UVR integrates other models like Demucs, this example shows how to call Demucs via UVR’s shell command:

import subprocess
from moviepy.editor import VideoFileClip, AudioFileClip

# Step 1: Run Demucs through UVR for separation
def separate_with_uvr(input_audio):
    command = f'demucs --two-stems=vocals {input_audio}'
    subprocess.call(command, shell=True)

# Step 2: Process video audio
video = VideoFileClip("input_video.mp4")
audio_path = "input_audio.wav"
video.audio.write_audiofile(audio_path)

# Step 3: Run separation
separate_with_uvr(audio_path)

# Step 4: Load separated vocals and replace in video
vocals_path = 'separated/vocals.wav'  # UVR/Demucs output folder
vocals = AudioFileClip(vocals_path)
final_video = video.set_audio(vocals)
final_video.write_videofile("output_video.mp4", codec="libx264", audio_codec="aac")

5. Lalal.ai API (Cloud-Based Service)

Lalal.ai is a high-quality, cloud-based audio separation tool. It’s highly effective at separating vocals and background music even in complex audio. However, it’s a paid service with a REST API for integration.

Example (using requests library)

First, sign up on Lalal.ai and get an API key.

import requests
import json
from moviepy.editor import VideoFileClip, AudioFileClip

# Step 1: Upload audio and request separation
def separate_with_lalalai(api_key, input_audio_path):
    url = "https://api.lalal.ai/upload"
    headers = {"Authorization": f"Bearer {api_key}"}
    files = {"file": open(input_audio_path, "rb")}
    response = requests.post(url, headers=headers, files=files)
    data = response.json()
    return data['vocals_url']

# Step 2: Process video
video = VideoFileClip("input_video.mp4")
audio_path = "input_audio.wav"
video.audio.write_audiofile(audio_path)

# Step 3: Separate audio using Lalal.ai API
api_key = "your_api_key_here"
vocal_url = separate_with_lalalai(api_key, audio_path)

# Step 4: Download separated vocals and replace audio in video
vocals = AudioFileClip(vocal_url)
final_video = video.set_audio(vocals)
final_video.write_videofile("output_video.mp4", codec="libx264", audio_codec="aac")

Summary

These libraries provide high-quality audio separation solutions depending on your needs, hardware, and willingness to customize. Let me know if you’d like further help with any!