Axeom/tribe-engage

Quantis by Axeom — Video Engagement Intelligence

Predict how well a video will engage its audience using Meta TRIBE v2, an open-source brain encoding model that predicts fMRI responses to video, audio, and text.

How It Works

1. 1.Upload a social media video (MP4, MOV, WebM, etc.)
2. 2.TRIBE v2 processes the video through its multimodal transformer (V-JEPA2 for video, Wav2Vec-BERT for audio, LLaMA 3.2 for language)
3. 3.Brain predictions are generated across ~20,000 cortical vertices on the fsaverage5 mesh
4. 4.Engagement scoring maps brain region activations to 7 engagement dimensions (attention, emotion, social, visual, language, narrative, audio)
5. 5.Dashboard shows your overall score, second-by-second timeline, and brain region breakdown

Architecture

┌──────────────────┐                     ┌──────────────────────┐
│  React Frontend  │  REST API (JWT)     │  FastAPI Backend      │
│  Cloudflare Pages│ ──────────────────▶ │  HuggingFace Spaces   │
│                  │ ◀────────────────── │  (Docker + GPU)       │
└──────────────────┘                     │  ┌────────────────┐  │
                                         │  │ TRIBE v2 Model │  │
                                         │  ├────────────────┤  │
                                         │  │ SQLite History │  │
                                         │  └────────────────┘  │
                                         └──────────────────────┘

Quick Start

Prerequisites

• —Python 3.10+
• —Node.js 18+
• —GPU recommended (TRIBE v2 runs on CPU but is slow)

Option 1: Quick script

chmod +x run.sh
./run.sh

Option 2: Manual

Backend:

cd backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --port 8000

Frontend:

cd frontend
npm install
npm start

Open http://localhost:3000 in your browser.

Option 3: Docker (backend only)

docker build -t tribe-engage .
docker run -p 8000:8000 tribe-engage

Demo Mode

If tribev2 is not installed (e.g., no GPU available), the backend automatically runs in demo mode with synthetic brain predictions. This lets you develop and test the full UI pipeline without the model.

API

All endpoints except /api/health and /api/auth/login require a Bearer token.

Engagement Dimensions

Deployment

Backend (HuggingFace Spaces)

1. 1.Create a new Space with SDK: Docker
2. 2.Push this repo (HF reads the root Dockerfile)
3. 3.Set these secrets in Space Settings:
4. 4.SECRET_KEY — random string for JWT signing
5. 5.APP_PASSWORD — your login password
6. 6.ALLOWED_ORIGINS — your Cloudflare Pages URL (e.g. https://tribe-engage.pages.dev)
7. 7.DATABASE_PATH — /data/tribe.db
8. 8.MODEL_CACHE_DIR — /data/cache

Frontend (Cloudflare Pages)

1. 1.Connect your repo in Cloudflare Pages
2. 2.Build command: cd frontend && npm install && npm run build
3. 3.Build output: frontend/build
4. 4.Environment variable: REACT_APP_API_URL=https://<your-space>.hf.space

Project Structure

tribe-engage/
├── backend/
│   ├── app/
│   │   ├── main.py              # FastAPI routes & middleware
│   │   ├── config.py            # Environment-driven settings
│   │   ├── auth.py              # JWT authentication
│   │   ├── database.py          # SQLite history storage
│   │   ├── rate_limiter.py      # Per-IP rate limiting
│   │   ├── tribe_service.py     # TRIBE v2 model loading & inference
│   │   ├── engagement_scorer.py # Brain → engagement score mapping
│   │   └── brain_regions.py     # Region definitions & atlas mapping
│   ├── requirements.txt
│   └── .env.example
├── frontend/
│   ├── src/
│   │   ├── App.js
│   │   ├── api.js               # API client with auth & progress
│   │   ├── components/
│   │   │   ├── LoginPage.js
│   │   │   ├── UploadPanel.js
│   │   │   ├── ScoreCard.js
│   │   │   ├── TimelineHeatmap.js
│   │   │   ├── DimensionBreakdown.js
│   │   │   ├── RegionBreakdown.js
│   │   │   ├── HistorySidebar.js
│   │   │   └── Toast.js
│   │   └── styles.css
│   └── package.json
├── Dockerfile
├── run.sh
└── README.md

License

This project uses Meta's TRIBE v2 model, released under CC BY-NC. Your application code can be licensed as you wish.