Abstract:Touch contact and pressure are essential for understanding how humans interact with and manipulate objects, insights which can significantly benefit applications in mixed reality and robotics. However, estimating these interactions from an egocentric camera perspective is challenging, largely due to the lack of comprehensive datasets that provide both accurate hand poses on contacting surfaces and detailed annotations of pressure information. In this paper, we introduce EgoPressure, a novel egocentric dataset that captures detailed touch contact and pressure interactions. EgoPressure provides high-resolution pressure intensity annotations for each contact point and includes accurate hand pose meshes obtained through our proposed multi-view, sequence-based optimization method processing data from an 8-camera capture rig. Our dataset comprises 5 hours of recorded interactions from 21 participants captured simultaneously by one head-mounted and seven stationary Kinect cameras, which acquire RGB images and depth maps at 30 Hz. To support future research and benchmarking, we present several baseline models for estimating applied pressure on external surfaces from RGB images, with and without hand pose information. We further explore the joint estimation of the hand mesh and applied pressure. Our experiments demonstrate that pressure and hand pose are complementary for understanding hand-object interactions. ng of hand-object interactions in AR/VR and robotics research. Project page: \url{this https URL}.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2409.02224 [cs.CV] |
| (or arXiv:2409.02224v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2409.02224 arXiv-issued DOI via DataCite |
Submission history
From: Yiming Zhao [view email]
[v1]
Tue, 3 Sep 2024 18:53:32 UTC (20,903 KB)
[v2]
Wed, 4 Dec 2024 10:24:43 UTC (44,146 KB)