mantrakp/component-studio-geometry

Component Studio

A new Gradio + uv pipeline targeting Hugging Face ZeroGPU. It accepts an image, a prompt, or both and produces a textured GLB with individually editable components. Add an animation prompt to run learned humanoid rigging, NVIDIA Kimodo motion generation, retargeting, video export, full-frame collision checks and Astra visual review.

The animated workflow is under production verification. A completed export is not a quality pass: rejected models or animations retain their artifacts with needs_review status.

Inputs

Pipeline stages

1. 1.Validate inputs and prepare a reference. Images keep EXIF orientation and alpha.
2. 2.TRELLIS.2 generates geometry at its 1024 cascade setting and a 1024px PBR atlas. Texture sampling uses twelve steps before component upscaling, giving refinement more source detail.
3. 3.PartField predicts learned 3D features. Cluster those features into the requested number of parts, partition the original faces, crop component texture bounds and remap UVs. This is learned part segmentation, not merely splitting disconnected islands. The target count controls granularity; semantic correctness is not guaranteed.
4. 4.Real-ESRGAN x4plus upscales each component's base color, with tiled overlap. Profile a maximal tile, inspect free RAM, CUDA memory and cgroup CPU limits, and admit the maximum workers fitting 90% of those resources. Recalculate each wave. Minimum concurrency is one; if one cannot fit safely, shrink tiles or fail instead of overriding the reserve. CUDA OOM halves concurrency and then tile size. GPU leases and requests are serialized; component workers run concurrently within the lease. Alpha is interpolated separately. Normal, metallic, roughness, occlusion and emission maps preserve their data and UVs.
5. 5.Astra (openai/gpt-6-astra) reviews the reference and rendered views. An embedded fx agent inspects per-run evidence inside a fresh just-bash virtual filesystem and returns a validated repair plan. Its tools cannot access the host filesystem, shell or network. The host applies bounded repairs, renders again and preserves the original on regression. Quality acceptance is a separate decision from non-regression.
6. 6.For an animation prompt, learn the humanoid skeleton and weights, transfer them onto the original textured components, and generate the requested motion with pinned native NVIDIA Kimodo. Retarget it in Blender and export the actual animated GLB and MP4.
7. 7.Measure every exported frame for new triangle intersections and ground penetration; render chronological front/side evidence and ask Astra to judge the requested action, deformation, texture fidelity and contacts. A failing review remains needs_review.
8. 8.Package the original inputs, candidate meshes, textures, rig, source motion, GLB, video, audit evidence and provenance. Downloads belong to one unique run directory.

Local development

uv sync --locked
uv run pytest
uv run app.py

On a Mac this starts the interface without loading CUDA models. Generation reports that the GPU runtime is required; it does not substitute sample meshes. PORT=7861 uv run app.py starts another local instance. No public sharing is enabled.

Hugging Face ZeroGPU deployment

The production layout uses one coordinator and five Gradio ZeroGPU workers:

Set STUDIO_WORKER_ROLE to the matching suffix on each worker. Leave it unset on the coordinator. STUDIO_WORKERS optionally overrides the stage-to-Space mapping as JSON; see studio/remote.py for defaults. Every stage creates a fresh authenticated Gradio request. Each worker loads only its own model, and each GPU request uses one lease.

Add HF_TOKEN as a secret to the coordinator and workers. Geometry requires approved DINOv3 and BRIA RMBG-2.0 access; motion requires approved Meta Llama 3 access. Add OPENROUTER_API_KEY only to the coordinator. Reviews use openai/gpt-6-astra without substitution. Model access approval and provider billing remain account requirements.

Deploy with scripts/deploy_space.py --space OWNER/SPACE --role ROLE --message MESSAGE. Use role coordinator for portable CPU requirements and the other five role names for CUDA requirements. The script uploads an explicit runtime file list and records hashes; it excludes credentials, generated outputs and caches. Select free CPU hardware for the coordinator and ZeroGPU for workers. Provision hardware separately before deployment.

Coordinator startup installs the pinned Node/libfx runtime. Workers download pinned model assets without installing fx or starting Blender. CUDA workers target Linux x86_64, Python 3.12, torch 2.11 and CUDA 13. These are Gradio Spaces, not Docker Spaces.

For a matching local NVIDIA machine:

uv sync --locked
uv pip install -r requirements-gpu.txt
uv run --no-sync scripts/prepare_runtime.py
STUDIO_NATIVE=1 uv run --no-sync app.py

Use --no-sync after adding the GPU requirements; normal uv sync intentionally manages only the portable development environment. GPU requirements are kept separate because the upstream CUDA wheels cannot install on macOS. Startup requires substantial model storage and RAM; downloads are cached. ZeroGPU quota expiry, unavailable models and memory failures are surfaced with retained intermediate files, never reported as completed runs.

Refinement uses libfx with an explicit host-owned OpenRouter transport and a fresh in-memory just-bash filesystem for each attempt. Only the run's review evidence is mounted; credentials remain outside tools. Tool calls, model calls, memory, output size and elapsed time are bounded. Runtime dependencies and the upstream cleanup skill are pinned.

Model choices and known limits

See model decisions for sources, revision pins and the quality/latency tradeoffs. “Newest” is not a claim of superior texture reconstruction. Neural upscaling cannot recover details that the source model never generated.

Astra receives rendered evidence through a host-owned OpenRouter transport. Its fx tools operate only inside the run's in-memory filesystem and return a validated repair plan. Technical validation and a model verdict do not guarantee visual quality.

The original code is in stash@{0} (Archive pre-rebuild forge3d and Mercy experiments). Ignored old outputs, weights and local secret pointers were moved to the sibling directory ../.3dgen-legacy-20260916/. They are not runtime dependencies of this implementation. Use git stash show --stat stash@{0} to inspect the archived source without restoring it.

See verification status for executed checks and the remaining ZeroGPU smoke test. Regenerate the Linux lock with:

uv pip compile requirements-hf.in --python-version 3.12 --python-platform x86_64-manylinux_2_28 --emit-index-url -o requirements-hf.lock
sed -e '/^-e \.$/d' -e 's/^torch==2.11.0+cu130$/torch==2.11.0/' -e 's/^torchvision==0.26.0+cu130$/torchvision==0.26.0/' requirements-hf.lock > requirements.txt

Archived source stash object: 0899c5725eddd965045cf645af45fbf9a60e10c2.

Continue an interrupted generation

If a later GPU lease fails, retain source.glb and reference.png from that run. The Continue an existing textured mesh panel accepts those files and resumes PartField, texture upscaling, review and packaging in a fresh request. Its API is /continue_asset(image, source_mesh, prompt, parts, seed). The prompt supplies the review brief; it does not regenerate the reference. The continued run records continued_from_source and preserves the supplied GLB bytes.

The September 18 astronaut experiment is in outputs/animated-astronaut/. scripts/animate_kimodo_astronaut.py fits that specific character to an exported Kimodo NPZ/BVH pair in Blender; it is not a generic auto-rigger. Its delivered GLB retains eight components and a 21-bone skin. Revision 2 projects the reference into a 4096px front texture and constrains the Kimodo motion to a smaller elbow-led gesture. All 180 exported frames have zero new triangle intersections relative to rest; seven static-mesh intersections remain. See docs/verification.md and outputs/animated-astronaut/final/collision-comparison.json for the exact scope.