Abstract

I will first present a model-based approach to 3D hand tracking from monocular video. The 3D hand pose, the hand texture and the illuminant are dynamically estimated through minimisation of an objective function. Derived from an inverse problem formulation, the objective function enables explicit use of texture temporal continuity and shading information, while handling important self-occlusions and time-varying illumination. The minimisation is done efficiently using a quasi-Newton method, for which we propose a rigorous derivation of the objective function gradient. Particular attention is given to terms related to the change of visibility near self-occlusion boundaries that are neglected in existing formulations. In doing so we introduce new occlusion forces and show that using all gradient terms greatly improves the performance of the method. Experimental results demonstrate the potential of the formulation.

I will then briefly present a more recent work for the 3D reconstruction of a piecewise-planar watertight surface from range images, typically indoor laser scans with millions of points. We formulate the surface reconstruction problem as a integer programming problem where we assign a binary label to elements of a partition of the scene volume that is based on detected and hypothesised planes. Our major contributions is a regularisation of the reconstructed surface w.r.t. the length of edges and the number of corners. Compared to classical area-based regularisation, it better captures surface complexity and is therefore better suited for man-made environments, such as buildings. We approximately solve the integer program using a simple continuous relaxation. Experiments show that it is fast and reaches near-optimal solution.

References: M. de la Gorce, N. Paragios and David Fleet. Model-Based Hand Tracking with Texture, Shading and Self-occlusions. IEEE Conference in Computer Vision and Pattern Recognition (CVPR), Anchorage 2008 http://www.gigovery.com/research_page/publications/CVPR_2008_de_La_Gorce.pdf

Alexandre Boulch, Martin de La Gorce, Renaud Marlet Piecewise-Planar 3D Reconstruction with Edge and Corner Regularization. Comput. Graph. Forum http://onlinelibrary.wiley.com/doi/10.1111/cgf.12431/pdf