CoolFace
Apppublic

Coastline6/transcript-agent-v2

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes
App README

<div align="center">

๐ŸŽค Transcript Agent

AI-powered transcription, interview coaching, and video analysis โ€” local-first.

![Version](https://github.com/jayuan101/transcript-agent/releases) ![Docker Hub](https://hub.docker.com/r/sushi0934/transcript-agent) ![HuggingFace](https://huggingface.co/spaces/Coastline6/transcript-agent-v2) ![License](LICENSE)

**Live Demo** ยท **Docker Hub** ยท **Releases** ยท **Changelog**

</div>


Two editions: Production vs Development

This repo ships two editions on two branches. They share the same Python engine (transcript_agent.py, video_analyzer.py, interview_vision.py) but use a different UI.

๐ŸŸข Production (`main`)๐Ÿงช Development (`dev`)
UIGradio (app.py)React + PrimeReact (frontend/), served by api.py
Default Docker entrypointpython app.pypython api.py (UI_MODE=react)
๐Ÿ”ด Live Interview (real-time webcam)โŒ Not availableโœ… Yes
Video delivery analysis (uploaded file)โœ… Yesโœ… Yes
Transcription, coaching, exports, historyโœ… Yesโœ… Yes
Legacy Gradio UI(it is the UI)available via UI_MODE=gradio
Build-from-source composedocker-compose.prod.yml (pull image)docker-compose.yml (local build)
TL;DR: If you want the stable Docker Hub release, use Production. If you need the new React UI and the Live Interview webcam feature, use Development (dev branch).

What it does

Core

FeatureEditionDetails
๐ŸŽคTranscriptionboth9 STT engines โ€” Whisper (local/offline), OpenAI, Groq, Deepgram, AssemblyAI, Google Cloud, Azure, ElevenLabs Scribe, Rev.ai
๐Ÿค–AI Analysisboth13 providers โ€” Claude, OpenAI, Gemini, Groq, Mistral, Together AI, Perplexity, xAI Grok, DeepSeek, OpenRouter, Cerebras, Cohere, Ollama (local)
๐Ÿ—ฃ๏ธ37+ LanguagesbothAuto-detect or choose, with regional dialect variants; translate output to any language
๐Ÿ’ฌLive TranscriptionbothReal-time audio capture and transcription (Deepgram by default, local Whisper fallback)
๐Ÿ‘คSpeaker DiarizationbothWhisperX-powered multi-speaker detection with automatic fragment merging (fixes split-speaker issues)
๐Ÿ‘คSpeaker NamesbothOn-screen participant-name OCR (Teams / Meet / Zoom / Nextcloud) maps diarized speakers to real names
๐Ÿ“คExportsboth.txt .docx .pdf .srt .vtt .json with readable download filenames
๐Ÿ“ŠReports & HistorybothSummary, key points, action items, speaker profiles, token spend + cost; trash/restore for deleted sessions

Interview Coaching

FeatureEditionDetails
๐ŸŽฏPer-Question Scoringboth10-point score per question, coaching tips, ideal answers, deflection detection
๐Ÿ“‹Questions TabbothDedicated tab with markdown export and per-question breakdown
๐Ÿ’ปCoding Challenge AnalysisbothAuto-detects coding/algorithm questions; shows candidate's answer vs. optimal solution; detects language (Java/Python/SQL/etc.) and frameworks (PySpark/React/Spring Boot/etc.); separate coding score
๐ŸŽญRound-Type WeightingbothAuto-detects interview round type (behavioral, technical, coding, system design) and weights scoring accordingly
๐Ÿ“ˆScore BannerbothOverall Score, Advance Likelihood %, Deflection Rate at the top of every report
๐Ÿ“–How Numbers Are CalculatedbothTransparent explanation of scoring methodology in every report

Video & Body Language Analysis

FeatureEditionDetails
๐ŸŽฅVideo Delivery Analysis (uploaded)bothEmotion, eye contact, head pose, posture, body language โ€” per-person score cards + annotated video
๐Ÿ’ชBody Language TransparencybothPer-cue breakdown: smiling, eye contact, open posture, forward lean, composure โ€” how each is measured and how it helps/hurts advancement chances
๐ŸŒŽCultural AnalysisbothAmerican Interview Standard score + Indian-to-American adaptation coaching
๐Ÿ”ดLive Interview (webcam)dev onlyReal-time webcam analysis โ€” records 5s clips, scores update live

Platform & Updates

FeatureEditionDetails
๐Ÿ”„OTA Auto-UpdatebothWindows installer + Mac dmg auto-check GitHub on every launch; silent download and relaunch
๐Ÿ–ฅ๏ธGPU AccelerationbothNVIDIA CUDA, Apple MPS, AMD DirectML โ€” auto-detected at startup with in-app GPU badge
๐ŸŒNetwork MonitorbothLive upload/download speed, ping latency, connection type โ€” always visible
๐Ÿ“‚Large File SupportbothPaste file paths for files >500 MB (no upload timeout); merge two files into one transcript
๐Ÿ›ŒSleep PreventionbothMachine stays awake while the app is running (Windows SetThreadExecutionState / Mac caffeinate)

Supported formats

TypeFormats
Audiomp3 wav m4a flac ogg aac wma
Videomp4 mov avi mkv webm m4v
Documentspdf docx txt md srt vtt

๐ŸŸข Production (main)

Stable Gradio UI, distributed as a pre-built Docker Hub image โ€” no git clone, no build.

Requirement: Docker (Desktop or Engine).
bash
mkdir transcript-agent && cd transcript-agent

# Add your API keys (at least one LLM key) โ€” see "Configuration" below
cp .env.example .env

# Pull and start the production image
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d

Open http://localhost:7860 (Gradio UI) and http://localhost:8000/docs (REST API).

CommandAction
docker compose -f docker-compose.prod.yml up -dStart in the background
docker compose -f docker-compose.prod.yml pullUpdate to the latest image
docker compose -f docker-compose.prod.yml logs -fFollow logs
docker compose -f docker-compose.prod.yml downStop and remove the container

Image tags: sushi0934/transcript-agent:latest (auto-updates) ยท sushi0934/transcript-agent:2.5.17 (pinned). The prod compose file sets pull_policy: always, restart: unless-stopped, and a healthcheck. The container runs python app.py (Gradio), which grafts the REST API onto its own server.

Run production without Docker

bash
git clone -b main https://github.com/jayuan101/transcript-agent.git
cd transcript-agent

python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt
pip install torch --index-url https://download.pytorch.org/whl/cpu

python app.py                   # Gradio UI + REST API on port 7860

On Windows you can instead double-click `run.bat`, which detects your GPU and opens the browser automatically.


๐Ÿงช Development (dev)

Adds the React + PrimeReact UI (served by api.py) and the ๐Ÿ”ด Live Interview webcam feature. The legacy Gradio UI is still available via UI_MODE=gradio.

bash
git clone -b dev https://github.com/jayuan101/transcript-agent.git
cd transcript-agent
cp .env.example .env            # add your keys

# Build & run the React UI + REST API (UI_MODE=react is the default)
docker compose build
docker compose up -d            # http://localhost:7860

Run the React UI from source

bash
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt
pip install torch --index-url https://download.pytorch.org/whl/cpu

# Build the React UI once (output frontend/dist is served by api.py at "/")
cd frontend && npm install && npm run build && cd ..

python api.py                   # React UI + REST API (port 8000, override with API_PORT)

For live front-end development with hot reload:

bash
cd frontend
npm run dev                     # Vite dev server, proxies to a running `python api.py`

Dev launcher (Windows, Gradio on 7861)

`launch_dev.bat` runs the Gradio app in dev mode on port 7861 (isolated from a production instance on 7860), with TA_DEV_MODE=1 showing a [DEV] banner:

bash
launch_dev.bat
Note: launch_dev.bat runs the Gradio UI, which does not include the Live Interview tab. To develop Live Interview, use python api.py + npm run dev above.

Promote dev to production

  • โ€”Windows desktop: `deploy_to_prod.bat` stops the running prod app, copies the updated .py source into the installed app's _internal folder, and relaunches it.
  • โ€”Docker Hub: `push_to_dockerhub.bat` builds and pushes a new image; production then pulls it via docker-compose.prod.yml.

Configuration (.env)

Used by both editions. Copy `.env.example` to .env and fill in at least one LLM key. Keys are mounted read-only into the container and never leave your machine.

dotenv
# AI / LLM providers (pick one or more)
ANTHROPIC_API_KEY=          # Claude  โ€” https://console.anthropic.com/keys
OPENAI_API_KEY=             # GPT     โ€” https://platform.openai.com/api-keys
GEMINI_API_KEY=             # Gemini  โ€” https://aistudio.google.com/app/apikey
GROQ_API_KEY=               # Groq    โ€” https://console.groq.com/keys

# Speech-to-Text engines (optional โ€” Whisper runs locally for free)
DEEPGRAM_API_KEY=
ASSEMBLYAI_API_KEY=
ELEVENLABS_API_KEY=
REV_AI_ACCESS_TOKEN=

# App settings
TZ=America/New_York         # your timezone
You can also enter keys directly in the UI sidebar โ€” they are saved in your browser only.

GPU acceleration (NVIDIA)

Local Whisper and emotion detection run much faster on a GPU. Uncomment the deploy.resources block in the compose file:

yaml
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
  • โ€”Linux: sudo apt install nvidia-container-toolkit && sudo systemctl restart docker
  • โ€”Windows: WSL2 + NVIDIA driver >= 510 + Docker Desktop >= 4.13

Auto-update in production

The latest tag plus pull_policy: always means docker compose -f docker-compose.prod.yml up -d always re-pulls the newest image. For hands-off updates, uncomment the Watchtower service in the prod compose file to re-pull hourly:

yaml
  watchtower:
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --interval 3600 --cleanup transcript-agent
    restart: unless-stopped

REST API

Available in both editions. Runs on port 8000 by default (and on 7860 inside the container). Interactive Swagger docs: /docs.

MethodEndpointDescription
POST/api/transcribeStart async transcription โ€” returns a job_id immediately
POST/api/transcribe/syncTranscribe and wait for the result
GET/api/jobs/{job_id}Get job status and results
GET/api/jobs/{job_id}/logStream live processing log
GET/api/jobs/{job_id}/download/{name}Download a generated result file
POST/api/jobs/{job_id}/cancelCancel a running job
POST/api/jobs/{job_id}/regenerateRegenerate PDF & DOCX (optionally in another language)
POST/api/transcribe-clipQuick live transcription of a short clip (used by Live Interview)
POST/api/analyze-videoAnalyze interview video delivery (body language, emotion, eye contact)
GET/api/historyPast runs with token spend + cost
DELETE/api/history/{entry_id}Delete a run (moves it to trash)
GET/api/trash ยท POST /api/trash/{id}/restore ยท POST /api/trash/emptyTrash management
GET/api/devicesAvailable compute device (GPU/CPU) for local Whisper
GET/api/update-checkCheck GitHub for a newer release
GET/healthHealth check (used by the container healthcheck)
GET/docsSwagger UI

Usage

  1. 1.Open http://localhost:7860
  2. 2.Enter an API key in the sidebar (or set it in .env) and pick your STT engine + AI provider
  3. 3.Upload a file, paste a path, or paste a URL โ€” click Analyze
  4. 4.Enable Interview Mode for per-question scoring, or use the Video Analysis tab
  5. 5.(dev only) Open the Live Interview tab for real-time webcam coaching

Ollama โ€” run AI locally (no API key)

  1. 1.Install Ollama from ollama.ai, then ollama pull gemma3:27b
  2. 2.In the app, select Ollama (Local) as the provider โ€” models are discovered dynamically
RAMRecommended model
48 GB+llama3.3, qwen2.5:72b, or deepseek-r1:70b
16-24 GBgemma3:27b (default), qwen3:32b
10-16 GBphi4, qwen3:14b, or gemma3:12b
8 GBgemma3:12b or llama3.2
In Docker, the app reaches Ollama on the host via host.docker.internal:11434 automatically.

Project structure

transcript_agent.py  โ€” STT engines, LLM analysis, report & export generation   (both)
video_analyzer.py    โ€” Emotion, eye contact, posture, body language, cultural   (both)
interview_vision.py  โ€” On-screen participant-name OCR + speaker mapping         (both)
api.py               โ€” FastAPI REST API (+ serves the React UI on dev)          (both)
app.py               โ€” Gradio UI                                                (production UI)
frontend/            โ€” React + PrimeReact web UI, incl. Live Interview          (dev only)
entrypoint.sh        โ€” Docker entrypoint (app.py on main, api.py on dev)
docker-compose.prod.yml  โ€” Pull pre-built image from Docker Hub                 (production)
docker-compose.yml       โ€” Build from source                                    (dev)
requirements.txt     โ€” Python dependencies

Support the project

If this tool saves you time, consider buying me a coffee

![Donate via PayPal](https://paypal.me/jay247616)


<div align="center"> <sub>Transcript Agent ยท Transcription by OpenAI Whisper ยท Analysis by Anthropic Claude ยท <a href="CHANGELOG.md">Changelog</a></sub> </div>