Markerless human motion capture (mocap) from multiple RGB cameras is a widelystudied problem. Existing methods either need calibrated cameras or calibratethem relative to a static camera, which acts as the reference frame for themocap system. The calibration step has to be done a priori for every capturesession, which is a tedious process, and re-calibration is required whenevercameras are intentionally or accidentally moved. In this paper, we propose amocap method which uses multiple static and moving extrinsically uncalibratedRGB cameras. The key components of our method are as follows. First, since thecameras and the subject can move freely, we select the ground plane as a commonreference to represent both the body and the camera motions unlike existingmethods which represent bodies in the camera coordinate. Second, we learn aprobability distribution of short human motion sequences (\sim1sec) relativeto the ground plane and leverage it to disambiguate between the camera andhuman motion. Third, we use this distribution as a motion prior in a novelmulti-stage optimization approach to fit the SMPL human body model and thecamera poses to the human body keypoints on the images. Finally, we show thatour method can work on a variety of datasets ranging from aerial cameras tosmartphones. It also gives more accurate results compared to thestate-of-the-art on the task of monocular human mocap with a static camera. Ourcode is available for research purposes onhttps://github.com/robot-perception-group/SmartMocap.
@inproceedings{SmartMocap,title={SmartMocap: Joint Estimation of Human and Camera Motion using Uncalibrated RGB Cameras},author={Saini, Nitin and Huang, Chun-hao P. and Black, Michael J. and Ahmad, Aamir},year={2022},tags={mocap, pose-estimation, uncalibrated},}