nvidia/Cosmos3-Super-Text2Image

1# Agentic Prompt Upsampling2 3This repository includes a standalone text-to-image agentic prompt upsampler for Cosmos3-Super-Text2Image.4 5The loop:6 71. Upsamples the user prompt into a structured Cosmos3 T2I JSON prompt.82. Generates an image through a vLLM-Omni `/v1/images/generations` endpoint.93. Scores the image with a VLM critic.104. Rewrites both the positive JSON prompt and generator-side negative prompt from the critic feedback.115. Repeats up to the configured iteration limit and returns the best scored image.12 13## Install14 15From the repository root:16 17```bash18python -m pip install requests pillow19```20 21Recommended vLLM-Omni serving configuration for `nvidia/Cosmos3-Super-Text2Image` on 4xH200 is:22 23```bash24vllm serve nvidia/Cosmos3-Super-Text2Image \25  --omni \26  --cfg-parallel-size 2 \27  --ulysses-degree 2 \28  --tensor-parallel-size 129```30 31With the no-offload configuration above, 1024x1024 image generation with 50 steps is expected to take roughly 5 seconds server-side per request.32 33## Default Models34 35The default prompt upsampler and rewriter are OpenAI GPT-5.5 through the public OpenAI chat completions API:36 37```text38endpoint: https://api.openai.com/v139model: gpt-5.540extra body: {"reasoning_effort": "low"}41env var: OPENAI_API_KEY42```43 44The default critic is Gemini 3.1 Pro Preview through Google's OpenAI-compatible chat completions endpoint:45 46```text47endpoint: https://generativelanguage.googleapis.com/v1beta/openai/48model: gemini-3.1-pro-preview49env var: GEMINI_API_KEY50```51 52Set credentials:53 54```bash55export OPENAI_API_KEY=...56export GEMINI_API_KEY=...57```58 59If your vLLM-Omni generation endpoint requires auth:60 61```bash62export AGENTIC_UPSAMPLING_GENERATION_AUTH_KEY=...63```64 65## Run One Prompt66 67```bash68python -m agentic_upsampling.run \69  --prompt "a cinematic photo of a glass greenhouse at sunrise" \70  --output-dir outputs/agentic_greenhouse \71  --generation-endpoint https://YOUR_VLLM_OMNI_ENDPOINT72```73 74The generation call is a standard vLLM-Omni image request:75 76```text77POST /v1/images/generations78model: nvidia/Cosmos3-Super-Text2Image79size: 1024x102480response_format: b64_json81num_inference_steps: 5082guidance_scale: 4.083flow_shift: 3.084negative_prompt: ""85extra_args: {"guardrails": false, "use_resolution_template": false}86```87 88## Run A Batch89 90Text file, one prompt per non-empty line:91 92```bash93python -m agentic_upsampling.run \94  --prompts prompts.txt \95  --output-dir outputs/agentic_batch \96  --generation-endpoint https://YOUR_VLLM_OMNI_ENDPOINT97```98 99JSONL rows can be strings or objects with `prompt` and optional `id`:100 101```json102{"id": "greenhouse", "prompt": "a glass greenhouse at sunrise"}103{"id": "city", "prompt": "a clean futuristic city plaza after rain"}104```105 106CSV files must include a `prompt` or `Prompt` column and may include an `id` column.107 108## Useful Options109 110```bash111python -m agentic_upsampling.run \112  --prompt "a precise product photo of a transparent mechanical keyboard" \113  --output-dir outputs/keyboard \114  --generation-endpoint https://YOUR_VLLM_OMNI_ENDPOINT \115  --max-iterations 2 \116  --samples-per-iteration 3 \117  --seed-base 42 \118  --size 1024x1024 \119  --guidance 4.0 \120  --flow-shift 3.0121```122 123- `--max-iterations` controls total prompt stages. The default is `2`, meaning the initial upsample plus up to two rewrites.124- `--samples-per-iteration` runs a best-of-N seed search for each prompt stage. Generation requests for those seeds are submitted concurrently within the iteration.125- `--seed-base` makes seeds deterministic. Sample seeds are `seed_base + sample_index`.126- `--size` is the vLLM-Omni image size in `WIDTHxHEIGHT` format.127- `--guidance` sets `guidance_scale`; the default is `4.0`.128- `--flow-shift` sets `flow_shift`; the default is `3.0`.129- `--generation-extra-args` overrides the default vLLM-Omni generation `extra_args` JSON object.130- Early stopping is enabled by default when the critic score clears the strict threshold. Use `--disable-early-stop` to always run every iteration.131- Reruns resume from completed artifacts by default. Use `--overwrite` to regenerate them.132 133## Output Layout134 135```text136output_dir/137  run_config.json138  summary.json139  manifest.jsonl140  failures.jsonl141  0001/142    best.json143    iter_00/144      prompt.json145      negative_prompt.json146      image.jpg147      generation_meta.json148      analysis.json149      samples.json150      meta.json151    iter_01/152      ...153```154 155For `--samples-per-iteration N`, each iteration contains `sample_00/`, `sample_01/`, and so on.156 157## Export Best Images158 159Copy the selected best image for every completed prompt into one folder:160 161```bash162python -m agentic_upsampling.extract_best \163  --output-dir outputs/agentic_batch \164  --export-dir outputs/agentic_batch_best \165  --overwrite166```167 168The exporter writes:169 170```text171best_generations.jsonl172best_generations.csv173images/174```175