Abd756/StemSense

1# StemSense: AI Audio Analysis & Separation Workflow2 3This document outlines the architecture, tools, and implementation plan for **StemSense**, a Python-based workflow that fetches audio from YouTube (via URL or Search), separates it into stems (Vocals, Drums, Bass, Other), analyzes musical metadata (BPM, Key, Loudness), and packages the results.4 5## System Architecture6 7```mermaid8graph TD9    A[User Input: Song Name or YouTube URL] --> B[YouTube Downloader Engine]10    B -->|Audio File: MP3/WAV| C[ML Stem Separator]11    C -->|Vocals, Drums, Bass, Other| D[Audio Analysis Engine]12    D -->|BPM, Key, Loudness| E[Data Aggregator]13    E --> F[Packaging Engine]14    F -->|ZIP: Original + Stems + JSON Metadata| G[Final Output]15```16 17## Project Structure18 19```text20StemSense/21├── main.py                 # Orchestrator (CLI Application)22├── config.py               # Constants, Paths, and Model Configs23├── core/                   # Processing logic24│   ├── __init__.py25│   ├── downloader.py       # Module 1: YouTube Downloader (yt-dlp based)26│   ├── stems.py            # Module 2: ML Stem Separator (Demucs)27│   ├── analyzer.py         # Module 3: Audio Feature Extractor28│   └── packager.py         # Module 4: ZIP Packager29├── tests/                  # Unit and integration tests30├── data/                   # Temporary file storage (ignored by git)31│   ├── downloads/32│   ├── stems/33│   └── exports/34├── requirements.txt35├── .gitignore36└── .env37```38 39## Modular Implementation Plan40 41### [Module 1] Audio Downloader (`core/downloader.py`)42*   **Action**: Accept a YouTube URL or a Song Name string. Use `yt-dlp` to search and download the high-quality audio file.43*   **Metadata Extraction**: Capture basic YouTube metadata (Title, Uploader, Duration) to seed the analysis.44*   **Test Case**: Verify that providing a song name results in a valid audio file in `data/downloads/`.45 46### [Module 2] ML Stem Separator (`core/stems.py`)47*   **Action**: Use **Demucs** (Meta Research) to split the audio into 4 stems: Vocals, Drums, Bass, and Other.48*   **Test Case**: Verify existing output directories for each stem and ensure WAV files are generated.49 50### [Module 3] Audio Feature Analyzer (`core/analyzer.py`)51*   **Action**: Analyze the original audio file to detect:52    *   **BPM**: Beats per minute.53    *   **Musical Key**: (e.g., C Minor, G Major).54    *   **Loudness**: Integrated LUFS/RMS.55*   **Test Case**: Compare results with known track data for accuracy.56 57### [Module 4] Packaging & Orchestration (`core/packager.py` & `main.py`)58*   **Action**: Collect all generated files and metadata, create a `metadata.json`, and bundle everything into a timestamped ZIP file.59*   **Test Case**: Ensure the final ZIP contains: `original.mp3`, `vocals.wav`, `drums.wav`, `bass.wav`, `other.wav`, and `metadata.json`.60 61---62 63## Technical Considerations64 65> [!IMPORTANT]66> **FFmpeg**: Must be globally installed and accessible for audio conversion and processing.67 68> [!NOTE]69> **GPU Support**: Demucs runs significantly faster if a CUDA-enabled GPU is available.70 71---72 73## Next Steps74 751. **Update Requirements**: Swap Spotify-specific libraries for `yt-dlp`.762. **Implement Downloader**: Build the search-and-download logic.773. **Verify Pipeline**: Run a full end-to-end test from URL to ZIP.78