| Consumer Depth Cameras for Computer Vision | 3 |
---|
| Foreword | 5 |
| Working on Human Pose Estimation for Kinect | 6 |
| Beyond Entertainment | 7 |
| Looking to the Future | 8 |
| Preface | 9 |
| Contents | 12 |
| Acronyms | 14 |
| Part I: 3D Registration and Reconstruction | 16 |
---|
| Chapter 1: 3D with Kinect | 18 |
| 1.1 Introduction | 18 |
| 1.2 Kinect as a 3D Measuring Device | 19 |
| 1.2.1 IR Image | 20 |
| 1.2.2 RGB Image | 21 |
| 1.2.3 Depth Image | 21 |
| 1.2.4 Depth Resolution | 21 |
| 1.3 Kinect Geometrical Model | 23 |
| 1.3.1 Shift Between IR Image and Depth Image | 24 |
| 1.3.2 Identi?cation of the IR Projector Geometrical Center | 25 |
| 1.3.3 Identi?cation of Effective Depth Resolutions of the IR Camera and Projector Stereo Pair | 26 |
| 1.4 Kinect Calibration | 29 |
| 1.4.1 Learning Complex Residual Errors | 30 |
| 1.5 Validation | 31 |
| 1.5.1 Kinect Depth Models Evaluation on a 3D Calibration Object | 34 |
| 1.5.2 Comparison of Kinect, SLR Stereo and 3D TOF | 35 |
| 1.5.3 Combining Kinect and Structure from Motion | 36 |
| 1.6 Conclusion | 39 |
| References | 39 |
| Chapter 2: Real-Time RGB-D Mapping and 3-D Modeling on the GPU Using the Random Ball Cover | 41 |
| 2.1 Introduction | 42 |
| 2.2 Related Work | 43 |
| 2.3 Methods | 45 |
| 2.3.1 Data Preprocessing on the GPU | 46 |
| Nomenclature | 46 |
| Landmark Extraction | 47 |
| 2.3.2 Photogeometric ICP Framework | 47 |
| 2.3.3 6-D Nearest Neighbor Search Using RBC | 48 |
| 2.4 Implementation Details | 50 |
| 2.4.1 Details Regarding the ICP Framework | 50 |
| 2.4.2 RBC Construction and Queries on the GPU | 51 |
| RBC Construction | 51 |
| RBC Nearest Neighbor Queries | 53 |
| 2.5 Experiments and Results | 53 |
| 2.5.1 Qualitative Results | 53 |
| 2.5.2 Performance Study | 56 |
| Preprocessing Pipeline | 56 |
| ICP Using RBC | 56 |
| 2.5.3 Approximate RBC | 57 |
| 2.6 Discussion and Conclusions | 59 |
| References | 60 |
| Chapter 3: A Brute Force Approach to Depth Camera Odometry | 63 |
| 3.1 Introduction | 63 |
| 3.2 Related Work | 64 |
| 3.3 Proposed Method | 65 |
| 3.3.1 Algorithm Overview | 66 |
| 3.3.2 Practical Issues | 67 |
| Feature Extraction | 67 |
| Score Evaluation | 67 |
| 3.3.3 Implementation Details | 67 |
| 3.4 Experimental Results | 68 |
| 3.4.1 Qualitative Evaluation | 68 |
| 3.4.2 Precision Analysis | 69 |
| 3.4.3 Comparison with the ICP Method | 72 |
| 3.5 Conclusion and Future Work | 73 |
| References | 73 |
| Part II: Human Body Analysis | 75 |
---|
| Chapter 4: Key Developments in Human Pose Estimation for Kinect | 77 |
| 4.1 Introduction: The Challenge | 77 |
| 4.2 Body Part Classi?cation-The Natural Markers Approach | 78 |
| 4.2.1 Generating the Training Data | 79 |
| 4.2.2 Randomized Forests for Classi?cation | 79 |
| 4.3 Random Forest Regression-The Voting Approach | 80 |
| 4.4 Context-Sensitive Pose Estimation-Conditional Regression Forests | 81 |
| 4.5 One-Shot Model Fitting: The Vitruvian Manifold | 82 |
| 4.6 Directions for Future Work | 83 |
| References | 83 |
| Chapter 5: A Data-Driven Approach for Real-Time Full Body Pose Reconstruction from a Depth Camera | 85 |
| 5.1 Introduction | 86 |
| Contributions | 87 |
| 5.2 Related Work | 88 |
| Intensity-Image-Based Tracking | 88 |
| Depth-Image-Based Tracking | 88 |
| 5.3 Acquisition and Data Preparation | 90 |
| 5.3.1 Depth Data | 90 |
| 5.3.2 Model of the Actor | 91 |
| 5.3.3 Pose Database | 92 |
| 5.3.4 Normalization | 93 |
| 5.4 Pose Reconstruction Framework | 94 |
| 5.4.1 Local Optimization | 95 |
| 5.4.2 Feature Computation | 95 |
| 5.4.3 Database Lookup | 101 |
| 5.4.4 Hypothesis Selection | 102 |
| 5.5 Experiments | 103 |
| 5.5.1 Feature Extraction | 103 |
| 5.5.2 Quantitative Evaluation | 103 |
| 5.5.3 Run Time | 105 |
| 5.5.4 Qualitative Evaluation | 105 |
| 5.5.5 Limitations | 108 |
| 5.6 Conclusions | 109 |
| References | 109 |
| Chapter 6: Home 3D Body Scans from a Single Kinect | 113 |
| 6.1 Introduction | 114 |
| 6.2 Related Work | 116 |
| 6.3 Sensor and Preprocessing | 117 |
| Intrinsic Calibration | 118 |
| Stereo Calibration | 118 |
| Depth Calibration | 118 |
| Ground Plane | 118 |
| Segmentation | 118 |
| 6.4 Body Model and Fitting | 119 |
| 6.4.1 SCAPE Body Model | 119 |
| 6.4.2 Pose Initialization | 120 |
| 6.4.3 Depth Objective | 121 |
| 6.4.4 Silhouette Objective | 121 |
| 6.4.5 Optimization | 124 |
| 6.5 Results | 124 |
| From Bodies to Measurements | 127 |
| Accuracy Relative to Laser Scans | 127 |
| Linear Measurement Accuracy | 129 |
| 6.6 Conclusions | 129 |
| References | 130 |
| Chapter 7: Real Time Hand Pose Estimation Using Depth Sensors | 132 |
| 7.1 Introduction | 132 |
| 7.1.1 Related Work | 134 |
| 7.1.1.1 Hand Pose Estimation | 134 |
| 7.1.1.2 Hand Shape Recognition
|