jasonfan/youtube-fashion-vton

YouTube Fashion Analyzer + Multi-Agent Virtual Try-On

End-to-end pipeline: YouTube video → outfit identification → product matching → AI virtual try-on, iteratively improved by a 120-round multi-agent debate (kimi-k2.5 × gpt-5.3-codex × gpt-5.4 × Claude Opus arbiter).

What This Is

This project analyzes the YouTube video "The Most UNHINGED Asian Reality Dating Show" by Jimmy Zhang, identifies the 4 male contestants' outfits, finds matching Uniqlo/New Balance products online, and generates AI virtual try-on images using FASHN VTON v1.5.

The entire design was then debated and refined by 3 AI agents over 120 rounds, with Claude Opus serving as arbiter.

Pipeline Overview

YouTube Video
    │
    ▼ yt-dlp sb0 storyboard
324 frames (9.7s interval, 320×180px)
    │
    ▼ Claude Vision (claude-sonnet-4-6)
4 male contestants identified + outfit analysis
    │
    ▼ WebSearch + Uniqlo image URL scraping
Product matches (tops/bottoms/shoes)
    │
    ▼ FASHN VTON v1.5 (local CPU, nice=20, 4 threads, 15 steps)
Virtual try-on images
    │
    ▼ PIL composite cards
[Video frame | Product image | AI try-on]

Outputs

The 4 Outfits (Budget ≤ $300 each, purchasable online)

Male A — 纹身街头极简暗系 ($294)

• —Linen Blend Open Collar SS Shirt Black $39.90
• —U Ribbed Tank Top White $14.90
• —Slim Fit Jeans Indigo Blue $49.90
• —New Balance 2002R Black ~$130
• —Gold chain necklace ~$20, Casio MTP-V004G ~$40

Male B — 全黑西装白衬衫 ($289)

• —Men's AirSense Blazer Black $149.90
• —Extra Fine Cotton Broadcloth Shirt White $29.90
• —Satin Slim Tie Black $19.90
• —AirSense Slim Pants Black $79.90
• —Silver stud earring (single) ~$10

Male C — 清爽亚麻白×海滩风 ($229)

• —Premium Linen Oversized SS Shirt White $49.90
• —Pleated Wide Ankle Pants Beige $49.90
• —New Balance 550 White $99.99
• —Square sunglasses ~$20, braided bracelet ~$10

Male D — 韩系Boxy极简 ($274)

• —Boxy Shirt SS Black $39.90
• —Relaxed Fit Straight Jeans Black $49.90
• —New Balance 9060 Rain Cloud $139.99
• —Silver Cuban Link Chain 6mm ~$30, minimalist ring ~$15

Code

Multi-Agent Debate (120 Rounds)

Three agents debated the design for 120 rounds:

• —Agent-K (kimi-k2.5): Fashion/UX/product perspective (Chinese)
• —Agent-C (gpt-5.3-codex): ML engineering / inference optimization
• —Agent-G (gpt-5.4): Architecture / devil's advocate

Claude Opus served as arbiter at Round 20, Round 70 (mid-point), and Round 120 (final).

Full debate log: debate/rounds_1_20_debate.md

Opus Final Verdict (Round 20 summary)

"The system proves the video→identify→try-on pipeline can work, but it's overconfident where it shouldn't be. Priority: build product truth first, fix input quality second, conditional rendering last. Honesty builds more trust than showing off."

Opus scores: Technical 6.5/10 | Fashion accuracy 4.5/10 | UX 4/10 | Cost 5.5/10

Top 3 improvements (Opus ordered):

1. 1.✅ Build closed catalog (Uniqlo/NB SKUs, asset_score ≥ 0.85) → code/catalog.py
2. 2.✅ Replace sb0 storyboard with keyframe mining (yt-dlp 720p + person Re-ID) → code/keyframe_extractor.py
3. 3.✅ Intent-gated rendering (shoppable card by default, VTON only on user click) → code/intent_gate.py

Setup

# 1. Clone FASHN VTON
git clone https://github.com/fashn-AI/fashn-vton-1.5.git
cd fashn-vton-1.5

# 2. Patch for Mac CPU (no onnxruntime-gpu)
sed -i '' 's/onnxruntime-gpu/onnxruntime/' pyproject.toml

# 3. Install (Python 3.10+)
python3 -m venv .venv && source .venv/bin/activate
pip install -e .

# 4. Download weights (~2GB)
python scripts/download_weights.py --weights-dir ./weights

# 5. Run optimized try-on
python run_tryon_optimized.py
# ~10 min/garment on CPU (M-series Mac), ~30s on GPU

Target User Profile

• —Height: 174cm, Weight: 65kg, Asian, slim build
• —Location: San Jose CA 95124
• —Nearest stores: UNIQLO Oakridge (~2mi), New Balance Union Ave (~1.5mi)

Related Projects

• —FASHN VTON v1.5 — virtual try-on model used
• —OOTDiffusion — alternative (6.5k stars)
• —IDM-VTON — ECCV 2024, best quality
• —CatVTON — ICLR 2025, most lightweight

License

Code: MIT | FASHN VTON model weights: Apache-2.0 | Debate logs: CC BY 4.0