atharv-1447/META-TRIBE-v2-api

META-TRIBE-v2-api

A REST API wrapping Meta's TRIBE v2 brain encoding model. Given any text content, it predicts which regions of the human brain would activate — returning scores for language processing, visual imagery, attention capture, emotional engagement, and a composite viral potential score.

Live Space: https://atharv-1447-meta-tribe-v2-api.hf.space

What TRIBE v2 Does

TRIBE v2 (Transformers for In-silico Brain Encoding v2) is Meta's multimodal foundation model that predicts fMRI brain responses to naturalistic stimuli. It combines:

• —LLaMA 3.2-3B — text encoding
• —V-JEPA2 — video encoding
• —Wav2Vec-BERT — audio encoding

Output: predicted fMRI BOLD signal on the fsaverage5 cortical surface (~20,484 vertices).

This Space exposes a text-based API. Image and video are supported in the Mavrx backend via a preprocessing pipeline (Claude Vision for images, Whisper + keyframe for video) before scoring.

Auth Requirements

Every request requires a HuggingFace Read token passed in the request body or Authorization header. The token must belong to an account that has accepted the license for:

• —`meta-llama/Llama-3.2-3B` — required for the text encoder
• —`facebook/tribev2` — public, no acceptance needed

No token is stored in this Space. Users supply their own.

API Reference

GET /health

Returns model status.

{ "status": "ok", "model_loaded": true, "model_loading": false, "error": null }

POST /warmup

Triggers model load. Required on first use after cold start (~90s to load).

{ "hf_token": "hf_..." }

POST /predict/text

{
  "text": "Your content here (5–5000 chars)",
  "hf_token": "hf_..."
}

Response:

{
  "status": "ok",
  "modality": "text",
  "scores": {
    "language_processing": 52.0,
    "visual_imagery": 50.8,
    "attention_capture": 59.4,
    "emotional_valence": 52.1,
    "overall_brain_engagement": 50.1,
    "viral_potential": 52.8,
    "activation_timeline": [0.14, 0.16, 0.19, ...],
    "n_timesteps": 27,
    "n_vertices": 20484,
    "dominant_hemisphere": "left",
    "word_count": 57,
    "overall_mean_activation": 0.193791
  }
}

POST /predict/image

Multipart form — image file + token.

curl -X POST https://atharv-1447-meta-tribe-v2-api.hf.space/predict/image \
  -F "file=@image.jpg" \
  -F "hf_token=hf_..."

POST /predict/video

Multipart form — video file + token.

curl -X POST https://atharv-1447-meta-tribe-v2-api.hf.space/predict/video \
  -F "file=@video.mp4" \
  -F "hf_token=hf_..."

POST /predict/base64

Send file as base64 JSON.

{
  "data": "<base64 string>",
  "type": "image",
  "ext": ".jpg",
  "hf_token": "hf_..."
}

Auth via header (alternative to body field)

Authorization: Bearer hf_...

Score Reference

All scores are on a 0–100 scale. 50 = average human response. Differences of 5–10 points are meaningful. Scores rarely reach extremes (0 or 100).

Known Behaviours

• —Aggressive/salesy copy scores lower than natural storytelling — the brain pattern-matches it as noise
• —Visual descriptive language produces the highest brain engagement
• —Generic aspirational copy ("be the best version of yourself") scores below average — treated as a template
• —Left hemisphere dominant = narrative/emotional language. Right hemisphere dominant = data, urgency, non-English
• —Cold starts take ~90s — the LLaMA 3.2 tokenizer downloads on first predict call. Subsequent requests are faster

Build History & Issues Resolved

This Space went through multiple iterations. Documented here for future reference.

Current status: Text endpoint fully working. Image and video endpoints exist but V-JEPA2 requires GPU — handled via Mavrx backend preprocessing (Claude Vision + Whisper → text → score).

CPU Patches Applied in app.py

tribev2 was designed for GPU. Running on CPU requires several patches:

# 1. WhisperX: replace float16 with int8 in subprocess command
subprocess.run = _patched_run  # intercepts uvx whisperx calls

# 2. torch.nn.Module.to: redirect cuda → cpu
torch.nn.Module.to = _cpu_module_to

# 3. transformers BatchEncoding.to: redirect cuda → cpu
BatchEncoding.to = _cpu_batch_to

# 4. All neuralset pydantic extractor classes: force device="cpu" after init
# Applied to: base, audio, video, text extractors
cls.model_post_init = _make_cpu_post(original)

# 5. HuggingFaceText._load_model: force device="cpu" before weight loading
HuggingFaceText._load_model = _cpu_load

# 6. ExtractWordsFromAudio._get_transcript_from_audio: full CPU-compatible override
# Uses --compute_type int8 --device cpu --batch_size 4

# 7. Environment
os.environ["CUDA_VISIBLE_DEVICES"] = ""  # also set in Dockerfile ENV
torch.cuda.is_available = lambda: False

Local Development

# Clone
git clone https://huggingface.co/spaces/atharv-1447/META-TRIBE-v2-api
cd META-TRIBE-v2-api

# Build
docker build -t tribe-v2-api .

# Run (requires HF token with LLaMA access)
docker run -p 7860:7860 -e HF_TOKEN=hf_... tribe-v2-api

# Test
curl http://localhost:7860/health
curl -X POST http://localhost:7860/warmup -d '{"hf_token":"hf_..."}' -H "Content-Type: application/json"
curl -X POST http://localhost:7860/predict/text \
  -H "Content-Type: application/json" \
  -d '{"text": "Your content here", "hf_token": "hf_..."}'

Mavrx Integration

This Space is called from the Mavrx backend via backend/core/tribe_client.py.

Endpoints exposed to Mavrx frontend:

• —POST /api/v1/brain/analyze/text — direct text scoring
• —POST /api/v1/brain/analyze/media — image or video (preprocessed via Claude Vision / Whisper before scoring)

Config: Set TRIBE_HF_TOKEN=hf_... in backend/.env.

License

Model weights: CC-BY-NC-4.0 — non-commercial use only. API wrapper code: MIT.