Learned priors
Learned priors supply initial 3D pose, shape, and camera translation from a single RGB image.
Metric 3D human pose from optional depth and ground geometry.
Anonymous submission · 2026
Paper and code will be released after publication.
TL;DR: Camera-frame 3D pose must recover true physical scale and distance—quantities that monocular RGB cannot disambiguate alone. MetricPose resolves this by unifying learned RGB priors with optional depth and ground geometry in a single model, using metric sensing when available without requiring it.
Per-frame results on dynamic sequences not included in the training dataset, comparing our model against SAM 3D Body and Anny-Fit to evaluate scale disambiguation and metric tracking against ground truth.
Every method lands on the same pixels, but RGB-only models treat the child as a smaller adult standing much farther away. Ground geometry and depth recover the true distance.
Camera, floor and depth point cloud seen from the side, with each model’s body at its predicted size and distance.
Distance to the person, frame by frame, for every method against the depth camera’s own reading of the person.
Where each model places the person relative to the camera and the scene’s point cloud.
A single unified model that dynamically adapts to whichever metric cues are available at test time.
Learned priors supply initial 3D pose, shape, and camera translation from a single RGB image.
Floor plane and gravity constraints anchor the body to correct metric height and distance without depth.
Metric depth directly fixes physical scale and camera distance while refining joint placement.
Up to 51% reduction in camera-frame error across four standard benchmarks, demonstrating consistent gains over baseline SAM 3D Body.
| Camera-local MPJPE (mm) ↓ | BEDLAM2 | CMU Panoptic | HuMMan | EgoBody |
|---|---|---|---|---|
| Baseline | ||||
| SAM 3D Body | 266.7 | 122.9 | 76.7 | 97.4 |
| SAM 3D Body + depth correction | 184.5 | 102.0 | 52.7 | 70.2 |
| One model, three modes | ||||
| Ours (RGB) | 188.2 | 104.6 | 81.2 | 90.1 |
| Ours (RGB + Ground) | 147.0 | 78.5 | 50.1 | 96.0 |
| Ours (RGB-D) | 166.1 | 98.5 | 39.7 | 47.5 |
Camera-local MPJPE in mm (lower is better). Ground geometry provides the strongest gains on RGB benchmarks (BEDLAM2, CMU), while calibrated depth achieves sub-50 mm accuracy on depth benchmarks (HuMMan, EgoBody).
Predicted 3D meshes compared against ground-truth outlines on HuMMan and RICH frames. Panel labels denote signed optical-axis depth error (Δz in mm, positive = too far).
Systematic evaluation of depth drift across varying camera heights and distances. Ground geometry significantly dampens baseline scale confusion, while RGB-D remains invariant across all tested ranges.
Camera-frame 3D human pose is fundamentally an information decoupling problem: learned RGB priors recover body articulation, while geometric constraints anchor physical scale and sensor placement. By unifying these roles in a single model, MetricPose delivers high metric precision whenever depth or ground cues exist, while retaining full functionality on standard RGB alone.