CoolFace
Modelpublic

OzzyGT/depth_pro_custom_block

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes
Model Card

Depth Pro Estimator Block

A custom Modular Diffusers block for monocular depth estimation using Apple's Depth Pro model. Supports both images and videos.

Features

  • —Metric depth estimation in real-world meters using Depth Pro
  • —Image and video input support
  • —Grayscale or turbo colormap visualization
  • —Inverse depth normalization (following Apple's reference implementation) for robust handling of outdoor/sky scenes

Installation

bash
# Using uv
uv sync

# Using pip
pip install -r requirements.txt

Quick Start

Load the block

python
from diffusers import ModularPipelineBlocks
import torch

blocks = ModularPipelineBlocks.from_pretrained(
    "your-username/depth-pro-estimator",  # or local path "."
    trust_remote_code=True,
)
pipeline = blocks.init_pipeline()
pipeline.load_components(torch_dtype=torch.float16)
pipeline.to("cuda")

Single image - grayscale depth

python
from PIL import Image

image = Image.open("photo.jpg")
output = pipeline(image=image)

# Save depth map
output.depth_image.save("photo_depth.png")

# Access raw metric depth tensor (in meters)
print(output.predicted_depth.shape)  # (H, W)
print(output.field_of_view)          # estimated FOV
print(output.focal_length)           # estimated focal length

Single image - turbo colormap

python
output = pipeline(image=image, colormap="turbo")
output.depth_image.save("photo_depth_turbo.png")

Video - grayscale depth

python
from block import save_video

output = pipeline(video_path="input.mp4", colormap="grayscale")
save_video(output.depth_frames, output.fps, "output_depth.mp4")

Video - turbo colormap

python
output = pipeline(video_path="input.mp4", colormap="turbo")
save_video(output.depth_frames, output.fps, "output_depth_turbo.mp4")

Inputs

ParameterTypeDefaultDescription
imagePIL.Image-Image to estimate depth for
video_pathstr-Path to input video. When provided, image is ignored
colormapstr"grayscale""grayscale" or "turbo" (colormapped)

Outputs

Image mode

OutputTypeDescription
depth_imagePIL.ImageNormalized depth visualization
predicted_depthtorch.TensorRaw metric depth in meters (H x W)
field_of_viewfloatEstimated horizontal FOV
focal_lengthfloatEstimated focal length

Video mode

OutputTypeDescription
depth_framesList[PIL.Image]Per-frame depth visualizations
fpsfloatSource video frame rate

Depth Normalization

Depth visualization uses inverse depth clipped to [0.1m, 250m], following Apple's reference implementation. This prevents sky/infinity values (clamped at 10,000m by the model) from crushing near-field detail into a binary mask.

  • —Bright = close, dark = far (grayscale)
  • —Warm (red/yellow) = close, cool (blue) = far (turbo)