2dpose

2022

1. Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation

   Zhehan Kan, Shuoshuo Chen, Zeng Li, and Zhihai He

   In ECCV 2022

   Abs Bib HTML

   We observe that human poses exhibit strong group-wise structural correlationand spatial coupling between keypoints due to the biological constraints ofdifferent body parts. This group-wise structural correlation can be explored toimprove the accuracy and robustness of human pose estimation. In this work, wedevelop a self-constrained prediction-verification network to characterize andlearn the structural correlation between keypoints during training. During theinference stage, the feedback information from the verification network allowsus to perform further optimization of pose prediction, which significantlyimproves the performance of human pose estimation. Specifically, we partitionthe keypoints into groups according to the biological structure of human body.Within each group, the keypoints are further partitioned into two subsets,high-confidence base keypoints and low-confidence terminal keypoints. Wedevelop a self-constrained prediction-verification network to perform forwardand backward predictions between these keypoint subsets. One fundamentalchallenge in pose estimation, as well as in generic prediction tasks, is thatthere is no mechanism for us to verify if the obtained pose estimation orprediction results are accurate or not, since the ground truth is notavailable. Once successfully learned, the verification network serves as anaccuracy verification module for the forward pose prediction. During theinference stage, it can be used to guide the local optimization of the poseestimation results of low-confidence keypoints with the self-constrained losson high-confidence keypoints as the objective function. Our extensiveexperimental results on benchmark MS COCO and CrowdPose datasets demonstratethat the proposed method can significantly improve the pose estimation results.

   @inproceedings{SCIO,
     title = {Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation},
     author = {Kan, Zhehan and Chen, Shuoshuo and Li, Zeng and He, Zhihai},
     year = {2022},
     tags = {2dpose, top-down},
     booktitle = {ECCV},
   }

   2dpose   top-down  
2. Poseur: Direct Human Pose Regression with Transformers

   Weian Mao, Yongtao Ge, Chunhua Shen, Zhi Tian, Xinlong Wang, Zhibin Wang, and Anton Hengel

   In ECCV 2022

   Abs Bib HTML

   We propose a direct, regression-based approach to 2D human pose estimationfrom single images. We formulate the problem as a sequence prediction task,which we solve using a Transformer network. This network directly learns aregression mapping from images to the keypoint coordinates, without resortingto intermediate representations such as heatmaps. This approach avoids much ofthe complexity associated with heatmap-based approaches. To overcome thefeature misalignment issues of previous regression-based methods, we propose anattention mechanism that adaptively attends to the features that are mostrelevant to the target keypoints, considerably improving the accuracy.Importantly, our framework is end-to-end differentiable, and naturally learnsto exploit the dependencies between keypoints. Experiments on MS-COCO and MPII,two predominant pose-estimation datasets, demonstrate that our methodsignificantly improves upon the state-of-the-art in regression-based poseestimation. More notably, ours is the first regression-based approach toperform favorably compared to the best heatmap-based pose estimation methods.

   @inproceedings{Poseur,
     title = {Poseur: Direct Human Pose Regression with Transformers},
     author = {Mao, Weian and Ge, Yongtao and Shen, Chunhua and Tian, Zhi and Wang, Xinlong and Wang, Zhibin and van den Hengel, Anton},
     year = {2022},
     tags = {2dpose, top-down, transformer},
     booktitle = {ECCV},
   }

   2dpose   top-down   transformer  

2021

1. Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation

   William McNally, Kanav Vats, Alexander Wong, and John McPhee

   In ECCV 2021

   Abs Qing Bib HTML

   In keypoint estimation tasks such as human pose estimation, heatmap-basedregression is the dominant approach despite possessing notable drawbacks:heatmaps intrinsically suffer from quantization error and require excessivecomputation to generate and post-process. Motivated to find a more efficientsolution, we propose to model individual keypoints and sets of spatiallyrelated keypoints (i.e., poses) as objects within a dense single-stageanchor-based detection framework. Hence, we call our method KAPAO (pronounced"Ka-Pow"), for Keypoints And Poses As Objects. KAPAO is applied to the problemof single-stage multi-person human pose estimation by simultaneously detectinghuman pose and keypoint objects and fusing the detections to exploit thestrengths of both object representations. In experiments, we observe that KAPAOis faster and more accurate than previous methods, which suffer greatly fromheatmap post-processing. The accuracy-speed trade-off is especially favourablein the practical setting when not using test-time augmentation. Source code:https://github.com/wmcnally/kapao.

   把关键点当做物体来直接用yolo回归

   @inproceedings{KAPAO,
     title = {Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation},
     author = {McNally, William and Vats, Kanav and Wong, Alexander and McPhee, John},
     year = {2021},
     tags = {2dpose, bottom-up},
     booktitle = {ECCV},
   }

   2dpose   bottom-up  

2020

1. Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation

   Zhengxiong Luo, Zhicheng Wang, Yan Huang, Tieniu Tan, and Erjin Zhou

   In 2020

   Abs Qing Bib HTML

   Heatmap regression has become the most prevalent choice for nowadays humanpose estimation methods. The ground-truth heatmaps are usually constructed viacovering all skeletal keypoints by 2D gaussian kernels. The standard deviationsof these kernels are fixed. However, for bottom-up methods, which need tohandle a large variance of human scales and labeling ambiguities, the currentpractice seems unreasonable. To better cope with these problems, we propose thescale-adaptive heatmap regression (SAHR) method, which can adaptively adjustthe standard deviation for each keypoint. In this way, SAHR is more tolerant ofvarious human scales and labeling ambiguities. However, SAHR may aggravate theimbalance between fore-background samples, which potentially hurts theimprovement of SAHR. Thus, we further introduce the weight-adaptive heatmapregression (WAHR) to help balance the fore-background samples. Extensiveexperiments show that SAHR together with WAHR largely improves the accuracy ofbottom-up human pose estimation. As a result, we finally outperform thestate-of-the-art model by +1.5AP and achieve 72.0AP on COCO test-dev2017, whichis com-arable with the performances of most top-down methods. Source codes areavailable at https://github.com/greatlog/SWAHR-HumanPose.

   均衡不同距离的heatmap的高斯核大小

   @inproceedings{2012.15175,
     title = {Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation},
     author = {Luo, Zhengxiong and Wang, Zhicheng and Huang, Yan and Tan, Tieniu and Zhou, Erjin},
     year = {2020},
     tags = {human, image, 2dpose},
   }

   human   image   2dpose  

2019

2018

2017