CoolFace
Apppublic

Dyen/Rovin

sourceHugging Faceupdated 9mo agoView on Hugging Face
0likes
App README

Rovin

🎬 Transform long videos into engaging shorts — A fully local, open-source tool that automatically generates short clips with English captions from videos up to 40 minutes long.

LocalShortsLab Python License

Features

  • —100% Local Processing — No cloud APIs, no data leaving your machine
  • —Automatic Highlight Detection — Uses AI to find the most engaging moments
  • —Scene-Based Segmentation — Smart scene detection with PySceneDetect
  • —English Captions — Automatic transcription with Whisper.cpp
  • —Incremental Results — Shorts appear as they're generated, no waiting for the full job
  • —Modern Web UI — Clean React interface with drag-and-drop upload

Tech Stack

ComponentTechnology
BackendPython 3.11+, FastAPI
Video ProcessingFFmpeg
Scene DetectionPySceneDetect
TranscriptionWhisper.cpp
AI ScoringSmolVLM2 (HuggingFace)
FrontendReact + Vite

Prerequisites

Before installing LocalShortsLab, ensure you have:

1. FFmpeg

bash
# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt install ffmpeg

# Windows (with Chocolatey)
choco install ffmpeg

2. Whisper.cpp

bash
# Clone and build
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
make

# Download the model (base English, ~150MB)
./models/download-ggml-model.sh base.en

3. Python 3.11+

bash
# Check version
python3 --version

Installation

1. Clone the Repository

bash
git clone <repo-url>
cd LocalShortsLab

2. Set Up Python Environment

bash
# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

3. Configure Environment

bash
# Copy example config
cp .env.example .env

# Edit .env with your paths:
# - WHISPER_BIN_PATH: Path to whisper.cpp binary
# - WHISPER_MODEL_PATH: Path to ggml model file

Example .env:

env
WHISPER_BIN_PATH=/path/to/whisper.cpp/main
WHISPER_MODEL_PATH=/path/to/whisper.cpp/models/ggml-base.en.bin

4. Set Up Frontend

bash
cd frontend
npm install

Running the Application

Start Backend

bash
# From project root
source venv/bin/activate
uvicorn backend.main:app --reload --port 8000

Start Frontend (separate terminal)

bash
cd frontend
npm run dev

Access the App

Open http://localhost:5173 in your browser.

Usage

  1. 1.Upload — Drag and drop a video file (up to 40 minutes)
  2. 2.Wait — The system will:
  3. 3.Extract audio
  4. 4.Detect scene boundaries
  5. 5.Transcribe speech
  6. 6.Score scenes for engagement
  7. 7.Select and render shorts
  8. 8.Download — Shorts appear incrementally with:
  9. 9.Raw MP4
  10. 10.SRT captions
  11. 11.MP4 with burned-in subtitles

API Endpoints

EndpointMethodDescription
/api/uploadPOSTUpload video file
/api/jobs/{job_id}/statusGETGet job progress
/api/jobs/{job_id}/shortsGETList generated shorts
/api/jobs/{job_id}/shorts/{short_id}/videoGETDownload short MP4
/api/jobs/{job_id}/shorts/{short_id}/srtGETDownload captions
/api/jobs/{job_id}/shorts/{short_id}/subtitledGETDownload captioned video

Configuration

All settings can be configured via environment variables or .env file:

VariableDefaultDescription
MAX_INPUT_DURATION2400Maximum video length (seconds)
MIN_SHORT_DURATION15Minimum short length (seconds)
MAX_SHORT_DURATION120Maximum short length (seconds)
SCORE_THRESHOLD6.5Minimum score for highlight
MAX_SHORTS10Maximum shorts to generate
SCENE_DETECTION_THRESHOLD27.0PySceneDetect sensitivity
NUM_THREADS4CPU threads for inference

Troubleshooting

"Whisper.cpp not found"

Ensure WHISPER_BIN_PATH in .env points to the main binary in your whisper.cpp directory.

"CUDA out of memory"

SmolVLM2 will automatically fall back to CPU if GPU memory is insufficient.

Slow processing

  • —Reduce NUM_THREADS if system becomes unresponsive
  • —Use a smaller Whisper model (tiny.en instead of base.en)

License

MIT License — See LICENSE for details.

Acknowledgments