MetricPose

Metric 3D human pose from optional depth and ground geometry.

Anonymous submission · 2026

Paper and code will be released after publication.

TL;DR: Camera-frame 3D pose must recover true physical scale and distance—quantities that monocular RGB cannot disambiguate alone. MetricPose resolves this by unifying learned RGB priors with optional depth and ground geometry in a single model, using metric sensing when available without requiring it.

Overview: from the image, the same person can be a smaller body close to the camera or a larger one farther away; depth and ground geometry resolve the ambiguity.
The same image evidence supports different body scales and camera distances. Depth and ground contact add the constraints that monocular RGB lacks.

Qualitative Experimental Results

Per-frame results on dynamic sequences not included in the training dataset, comparing our model against SAM 3D Body and Anny-Fit to evaluate scale disambiguation and metric tracking against ground truth.

Scene 1 · Camera Perspective

A child and an adult in one view

Every method lands on the same pixels, but RGB-only models treat the child as a smaller adult standing much farther away. Ground geometry and depth recover the true distance.

Ground Truth SAM 3D Body Ours (RGB) Ours (RGB + Ground) Ours (RGB-D) Anny-Fit
Scene 1 · Metric 3D Geometry

The same scene in metric 3-D

Camera, floor and depth point cloud seen from the side, with each model’s body at its predicted size and distance.

Ground Truth SAM 3D Body Ours (RGB) Ours (RGB + Ground) Ours (RGB-D) Anny-Fit
Scene 2 · Moving Robot Tracking

A robot following a walking adult

Distance to the person, frame by frame, for every method against the depth camera’s own reading of the person.

D455 reading SAM 3D Body Ours (RGB) Ours (RGB + Ground) Ours (RGB-D) Anny-Fit
Scene 2 · Metric 3D Geometry

The same recording as 3-D geometry

Where each model places the person relative to the camera and the scene’s point cloud.

D455 reading SAM 3D Body Ours (RGB) Ours (RGB + Ground) Ours (RGB-D) Anny-Fit

One model, three operating modes

A single unified model that dynamically adapts to whichever metric cues are available at test time.

RGB

Learned priors

Learned priors supply initial 3D pose, shape, and camera translation from a single RGB image.

Needs a single RGB image
RGB + Ground

A floor and gravity cue

Floor plane and gravity constraints anchor the body to correct metric height and distance without depth.

Adds a floor plane
RGB-D

Measured surface depth

Metric depth directly fixes physical scale and camera distance while refining joint placement.

Adds a depth map
Diagram of the RGB-D reconstruction and metric-correction pipeline
Architecture of the RGB-D reconstruction and metric-correction pipeline.

Quantitative Evaluation

Up to 51% reduction in camera-frame error across four standard benchmarks, demonstrating consistent gains over baseline SAM 3D Body.

BEDLAM2
−45%
266.7 → 147.0 mm
RGB + Ground
CMU Panoptic
−36%
122.9 → 78.5 mm
RGB + Ground
HuMMan
−48%
76.7 → 39.7 mm
RGB-D
EgoBody
−51%
97.4 → 47.5 mm
RGB-D
Camera-local MPJPE (mm) ↓ BEDLAM2CMU PanopticHuMManEgoBody
Baseline
SAM 3D Body 266.7 122.9 76.7 97.4
SAM 3D Body + depth correction 184.5 102.0 52.7 70.2
One model, three modes
Ours (RGB) 188.2 104.6 81.2 90.1
Ours (RGB + Ground) 147.0 78.5 50.1 96.0
Ours (RGB-D) 166.1 98.5 39.7 47.5

Camera-local MPJPE in mm (lower is better). Ground geometry provides the strongest gains on RGB benchmarks (BEDLAM2, CMU), while calibrated depth achieves sub-50 mm accuracy on depth benchmarks (HuMMan, EgoBody).

Qualitative Mesh Comparisons

Predicted 3D meshes compared against ground-truth outlines on HuMMan and RICH frames. Panel labels denote signed optical-axis depth error (Δz in mm, positive = too far).

Side-by-side comparison of SAM 3D Body, Anny-Fit and the three modes of our model against ground truth on HuMMan and RICH frames.
Columns: (a) Input, (b) SAM 3D Body, (c) Anny-Fit, (d–f) Our three modes. While baselines suffer from severe depth drift, our ground and depth cues tightly align the mesh with ground truth.

Controlled Distance Evaluation

Systematic evaluation of depth drift across varying camera heights and distances. Ground geometry significantly dampens baseline scale confusion, while RGB-D remains invariant across all tested ranges.

Camera-distance error across camera heights and subject distances
Camera-distance error across prescribed ground distances. Ground geometry and depth act as complementary metric anchors to eliminate scale-distance collapse.

Key Takeaway

Camera-frame 3D human pose is fundamentally an information decoupling problem: learned RGB priors recover body articulation, while geometric constraints anchor physical scale and sensor placement. By unifying these roles in a single model, MetricPose delivers high metric precision whenever depth or ground cues exist, while retaining full functionality on standard RGB alone.