hand-pose
Papers with tag hand-pose
2022
- UmeTrack: Unified multi-view end-to-end hand tracking for VRShangchen Han, Po-chen Wu, Yubo Zhang, Beibei Liu, Linguang Zhang, Zheng Wang, Weiguang Si, Peizhao Zhang, Yujun Cai, Tomas Hodan, and 6 more authorsIn 2022
Real-time tracking of 3D hand pose in world space is a challenging problemand plays an important role in VR interaction. Existing work in this space arelimited to either producing root-relative (versus world space) 3D pose or relyon multiple stages such as generating heatmaps and kinematic optimization toobtain 3D pose. Moreover, the typical VR scenario, which involves multi-viewtracking from wide \acfov cameras is seldom addressed by these methods. Inthis paper, we present a unified end-to-end differentiable framework formulti-view, multi-frame hand tracking that directly predicts 3D hand pose inworld space. We demonstrate the benefits of end-to-end differentiabilty byextending our framework with downstream tasks such as jitter reduction andpinch prediction. To demonstrate the efficacy of our model, we further presenta new large-scale egocentric hand pose dataset that consists of both real andsynthetic data. Experiments show that our system trained on this datasethandles various challenging interactive motions, and has been successfullyapplied to real-time VR applications.
Multi-view hand pose tracking. 用VR眼镜的相机预测3D hand pose in world space. 用网络预测. 提出了一个数据集.
hand-pose@inproceedings{UmeTrack, title = {UmeTrack: Unified multi-view end-to-end hand tracking for VR}, author = {Han, Shangchen and Wu, Po-chen and Zhang, Yubo and Liu, Beibei and Zhang, Linguang and Wang, Zheng and Si, Weiguang and Zhang, Peizhao and Cai, Yujun and Hodan, Tomas and Cabezas, Randi and Tran, Luan and Akbay, Muzaffer and Yu, Tsz-Ho and Keskin, Cem and Wang, Robert}, year = {2022}, tags = {hand-pose}, sida = {Multi-view hand pose tracking. 用VR眼镜的相机预测3D hand pose in world space. 用网络预测. 提出了一个数据集.}, }