Skip to main content
A Depth Pro 2 monocular device, with a sharp lens, held by a hand, against a blurred background, emphasizing its ability to c

Editorial illustration for Depth Pro 2 Aims for Sharpness and Consistency in Monocular Depth

Depth Pro 2 Improves Monocular Depth Mapping Speed

Depth Pro 2 Aims for Sharpness and Consistency in Monocular Depth

• 2 min read

A team led by Peiyun Hu and Vladlen Koltun has released Depth Pro 2, an update to the metric monocular depth model the same group put out in October 2024. The original Depth Pro could turn a single photo into a dense, high-resolution depth map in about 0.3 seconds on a standard GPU, without needing camera metadata to get absolute scale right. That made it useful for anything from robotics to 3D reconstruction, but single images only go so far when the input is video.

Depth Pro 2 extends the approach to full-resolution video, aiming to keep the sharpness of the original while fixing a problem that plagues most depth estimators: flicker between frames, where depth estimates jitter even when the scene barely changes. The authors, who also include Marko Mihajlovic, Lyndon Duong, Lahav Lipson, Wei Dong, Amaël Delaunoy, and Stephan R. Richter, are releasing both code and model weights alongside the paper. The work sits next to a related release from the same lab, SHARP, which handles single-image view synthesis using 3D Gaussian representations rendered from a single feedforward pass.

Depth Pro 2 produces depth maps that are metrically accurate, sharp, and temporally consistent all at once, outperforming prior work in all of these dimensions.

Why this matters

For anyone building with video or 3D, the sharpness-versus-consistency trade-off in monocular depth has been a real tax: pick single-frame models and your boundaries look clean but your geometry jitters between frames; pick multi-frame models and you get stability at the cost of crisp edges. Depth Pro 2, from a team at Apple including Vladlen Koltun, claims to collapse that trade-off into one scalable network handling both single images and video at full resolution. If the numbers hold up outside the paper, that's a genuine win for robotics, AR, and video-editing pipelines that currently stitch together separate models or post-process depth maps to smooth out flicker.

We're inclined to treat "metric" depth claims with some caution until independent benchmarks confirm accuracy across diverse scenes, not just the authors' test set. Still, a single network that doesn't force engineers to choose between sharp edges and temporal stability is worth watching closely. The next thing to check: how it performs on occlusion-heavy, fast-motion footage, where both sharpness and consistency usually break down together.

Common Questions Answered

How does Depth Pro 2 improve upon the original Depth Pro model?

Depth Pro 2 extends the original model's capabilities from single images to video sequences while maintaining metric accuracy and sharp detail. The key advancement is achieving temporal consistency across video frames without sacrificing edge sharpness, addressing a fundamental trade-off that previous monocular depth models struggled with.

What was the main limitation of using monocular depth models for video applications?

Previous monocular depth models forced developers to choose between sharpness and consistency: single-frame models produced clean boundaries but caused geometry to jitter between frames, while multi-frame models provided stability at the cost of blurry edges. Depth Pro 2 claims to eliminate this trade-off by delivering both sharp edges and temporal consistency simultaneously.

What are the practical applications of Depth Pro 2's metric monocular depth capabilities?

Depth Pro 2 is useful for robotics, 3D reconstruction, and video processing applications that require accurate depth information from single images or video sequences. The model's ability to produce metrically accurate depth maps without requiring camera metadata makes it accessible for a wide range of computer vision tasks.

How fast can Depth Pro 2 process depth maps on standard hardware?

The original Depth Pro could generate dense, high-resolution depth maps in approximately 0.3 seconds on a standard GPU. Depth Pro 2 maintains this efficiency while extending functionality to handle video at full resolution through a single scalable network architecture.

Does Depth Pro 2 require camera metadata to determine absolute scale?

No, Depth Pro 2 produces metrically accurate depth maps without needing camera metadata to establish absolute scale, similar to its predecessor. This independence from camera calibration information makes the model more practical for real-world applications where such metadata may not be available.

LIVE20:21Depth Pro 2 Aims for Sharpness and Consistency in Monocular Depth