Alirezakzt/Concept-Segmentation

1---2title: SAM 3.1 Concept Segmentation3emoji: 🎯4colorFrom: indigo5colorTo: blue6sdk: gradio7app_file: app.py8pinned: false9license: other10short_description: Language-driven segmentation with Meta SAM 3.1.11---12 13# SAM 3.1 · Concept Segmentation14 15A live, **language-driven** segmentation demo built on Meta's **Segment Anything16Model 3.1**. Type a short noun phrase (`horse`, `saddle`, `person`) and the model17finds and segments **every matching instance** in the image — no boxes, no clicks,18no retraining.19 20> Segmentation is fully driven by language prompts — no retraining required.21 22## What's inside23 24- **Text-prompt segmentation** via SAM 3.1's Promptable Concept Segmentation (PCS).25- **Three result views** in tabs — *Overlay* (semi-transparent, color-coded per26  instance), *Mask only*, and *Original*.27- **Instant feedback** — match count + inference latency shown after every run.28- **Advanced mode** — confidence threshold, optional boxes/scores, and a29  **multiple-prompts** runner that reuses a single vision pass for speed.30- **Prompt history** — a gallery of your recent prompts and their overlays.31 32## Deploy on Hugging Face Spaces33 341. **Create a Space** → SDK: **Gradio**. Upload `app.py`, `requirements.txt`,35   and this `README.md` (or push the repo).362. **Request model access.** SAM 3 / 3.1 weights are gated. Open the model page37   (e.g. <https://huggingface.co/facebook/sam3.1>) and accept the license. Do the38   same for `facebook/sam3` if you want the fallback.393. **Add your token.** In **Settings → Variables and secrets**, add a secret named40   **`HF_TOKEN`** with a read token from <https://huggingface.co/settings/tokens>.414. **Pick hardware.** SAM 3.1 is an ~848M-parameter GPU model.42   - **ZeroGPU** (free, recommended for public demos) — works out of the box; the43     app uses `@spaces.GPU`.44   - or a small **GPU Space** (e.g. T4 / A10G) for an always-on demo.45   - CPU works but is slow; fine only for smoke-testing.465. **Open the Space.** The first request downloads the weights (cold start takes a47   bit); subsequent prompts are fast.48 49### Run locally50 51```bash52pip install torch            # from https://pytorch.org for your platform/CUDA53pip install -r requirements.txt54export HF_TOKEN=hf_...        # token with access to the gated weights55python app.py56```57 58## Configuration (environment variables)59 60| Variable             | Default          | Purpose                                              |61|----------------------|------------------|------------------------------------------------------|62| `MODEL_ID`           | `facebook/sam3.1`| Primary checkpoint to load.                          |63| `FALLBACK_MODEL_ID`  | `facebook/sam3`  | Used automatically if the primary fails to load.     |64| `HF_TOKEN`           | —                | Access token for the gated SAM 3 / 3.1 weights.      |65 66To force SAM 3 instead of 3.1, set `MODEL_ID=facebook/sam3`.67 68## Notes & tips69 70- **Short noun phrases win.** PCS is tuned for concepts like `horse` or `saddle`.71  Long descriptive phrases are less reliable — for reins, prefer `reins` or72  `bridle` over *"object used for riding control."* The example dropdown keeps the73  descriptive phrase so you can see the difference for yourself.74- **Threshold.** Lower it (Advanced) to surface more instances; raise it to keep75  only high-confidence matches.76- **Speed.** On GPU the forward pass runs in bfloat16. The multi-prompt runner77  computes vision features once and reuses them across prompts in a single call.78- **Example images.** Drop files into an `examples/` folder (`horse.jpg`,79  `street.jpg`, `kitchen.jpg`) to enable one-click examples. The app runs fine80  without them.81 82## About the model83 84Built on Meta's Segment Anything Model 3.1 (released March 2026), which extends85SAM 3's Promptable Concept Segmentation with the *Object Multiplex* tracker for86faster multi-object video. This demo uses the **image** PCS path through 🤗87Transformers (`Sam3Model` / `Sam3Processor`). The SAM 3 / 3.1 weights are88distributed under Meta's SAM License — review it before any production use.89