Zhongzhi1228/synthesis-demo

FAC-Synthesis Demo

Interactive demo for "Less is Enough: Synthesizing Diverse Data in Feature Space of LLMs"

Quick Start

Local Testing

cd demo
pip install -r requirements.txt
streamlit run app.py

Open http://localhost:8501 in your browser.

Deploy to Public (Recommended: Streamlit Cloud)

Step 1: Push to GitHub

cd demo
git init
git add .
git commit -m "Initial commit"
git remote add origin https://github.com/YOUR_USERNAME/fac-synthesis-demo.git
git branch -M main
git push -u origin main

Step 2: Deploy on Streamlit Cloud

1. 1.Go to https://share.streamlit.io
2. 2.Sign in with your GitHub account (free)
3. 3.Click "New app"
4. 4.Select your repository: YOUR_USERNAME/fac-synthesis-demo
5. 5.Set main file path: app.py
6. 6.Click "Deploy"

Done! Your app will be live at: https://YOUR-USERNAME-fac-synthesis-demo.streamlit.app

Why Streamlit Cloud?

• —Completely free
• —5-minute setup
• —Auto-deploy on git push
• —Built-in HTTPS
• —No server management

Features

SAE Configuration

• —Load SAE weights from default path, Hugging Face, or upload files
• —Adjust activation threshold (0.0 to 4.0)
• —Support for LLaMA, Mistral, Qwen models

Data Synthesizer

• —Choose from LLaMA-3.1-8B, Mistral-7B, Qwen2-7B
• —Optional GPT-4o-mini (no GPU required, API key needed)

Core Functions

• —Feature Analysis: Analyze text and visualize SAE activations
• —Targeted Synthesis: Generate text that activates specific features
• —FAC Coverage: Compute coverage metrics for datasets
• —Batch Synthesis: Generate multiple samples efficiently

Advanced

• —Load custom datasets from Hugging Face
• —Use your own SAE checkpoints
• —Extend to new tasks

Configuration

GPU Requirements

• —Local models (LLaMA/Mistral/Qwen): 16GB+ VRAM recommended
• —GPT-4o-mini: No GPU needed (cloud API)

SAE Weight Options

Option 1: Default (Local) Place weights in:

demo/default_weights/Llama-3.1-8B-Instruct/sae_l16.pt

Option 2: Hugging Face (Recommended) In the app:

1. 1.Select "Hugging Face" as source
2. 2.Enter repository ID: username/sae-weights
3. 3.Enter filename: sae_l16.pt
4. 4.Click "Load"

Option 3: Upload Upload .pt or .pth files directly (max 200MB)

Troubleshooting

CUDA Out of Memory

• —Switch to GPT-4o-mini synthesizer
• —Use smaller model (Mistral-7B)
• —Close other GPU applications

SAE Weights Not Found

• —Use Hugging Face option
• —Check file path in default_weights/

Import Errors

pip install -r requirements.txt --upgrade

Upload Your SAE to Hugging Face

from huggingface_hub import HfApi

api = HfApi()
api.upload_file(
    path_or_fileobj="path/to/sae_l16.pt",
    path_in_repo="sae_l16.pt",
    repo_id="your-username/sae-weights",
    repo_type="model"
)

See upload_to_hf_example.py for more details.

Citation

@article{less-is-enough-2026,
  title={Less is Enough: Synthesizing Diverse Data in Feature Space of LLMs},
  author={...},
  year={2026}
}

License

MIT License