# Monocular deformable 3D reconstruction using differential geometry

- Casillas Pérez, David

- Daniel Pizarro Pérez Director
- Manuel Ramón Mazo Quintas Co-director

Defence university: Universidad de Alcalá

Fecha de defensa: 18 October 2019

- Javier Macías Guarasa Chair
- José Miguel Buenaposada Biencinto Secretary
- Alessio del Bue Committee member

Type: Thesis

## Abstract

The present thesis is about 3D reconstruction of deformable objects. It gives contributions in both the most studied reconstruction problems: Shape-from-Template (SfT) and Non-Rigid Structure-fromMotion (NRSfM). The former reconstructs the deformed surface from one of its views and a 3D model of the surface before being transformed, called the template. The later retrieves the 3D object shape from multiple views where it undergoes deformations. This thesis contributes to SfT for non-isometric deformations. In particular, it studies equiareal deformations, a kind of deformation characterized by preserving the intrinsic areas. Equiareal deformations include isometries and are a special case of the linear-elastic deformations when the Young modulus is fixed to 1. We first deduce the equiareal equation, a quadratic first-order Partial Differential Equation (PDE) in three variables: the depth and its derivatives, which characterizes the reconstruction problem. We show that in the absence of initial conditions, equiareal SfT is not a well-posed problem. We prove the existence and uniqueness of one solution with a given initial strip and two solutions if the given initial data is a curve. This thesis uses Monge’s theory to study well-posedness in equiareal SfT. Also based on this theory, we provide an algorithm which solves the Monge’s characteristic system through conventional Ordinary Differential Equation (ODE) solvers. The obtained characteristic curves are integrated to give the solution surface. We evaluate our algorithm in experiments with synthetic and real data and compare our method with the best non-isometric SfT state-of-art-methods. Our method outperforms their reconstruction results even considering non-equiareal deformations. Registration functions, also called warps, are a key element in isometric SfT. The recently discovered analytic expressions for solving this problem involves a warp and its derivatives in their calculations. The present thesis contributes to improve the quality of warps, opening a new strategy for solving isometric SfT based on the paradigm of registration for reconstruction. In this sense, we give contributions to isometric SfT trough the study of isowarps. Isowarps are warps between images of objects which have undergone isometric deformations. As a result, isowarps intrinsically inherit the 3D geometric properties of the deformation model. We first derive the called isowarp equations, a set of three quadratic second-order PDEs that characterize isowarps. We show that forcing warps to fulfill these equations to become isowarps results in better reconstructions, achieving comparable and even outperforming refinement methods. We provide an algorithm which enforces isowarp equations to the warps, computed from features correspondences. We thoroughly evaluate our reconstruction strategy in synthetic and real experiments comparing it with the recent state-of-the-art methods. In planar curve reconstruction, we provide a theoretic study of the so-called planar perspective equation. It is a quadratic first-order ODE which describes the geometric problem of reconstructing a plane curve from one of its views, taken with a 1D perspective camera, and given the modulus of its velocity vector. This is also known as isometric 1DSfT. We show that the problem of solving the ODE is not wellposed. We prove that only two local solutions exist and a maximum of two analytical solutions to the initial value problem with regular and critical initial conditions, respectively. The classification of points into regular and critical solutions is important. We show that relaxing the analytic constraint produces convergent cones, which are regions of the plane where a dense set of solutions appear. We establish the so-called maximal depth solution problem, which consists in finding the farthest solution from the image plane. This curve is also the smoothest of all solutions and is a is well-posed problem. This curve is commonly retrieved from numerous state-of-the-art methods. We provide robust algorithms for computing the local solutions in both regular and critical initial conditions, and the maximal depth solution. We evaluate them in experiments with synthetic date testing all possible cases. Finally, we contribute to the isometric NRSfM problem. Under the differential geometric framework, we derive the isometric NRSfM equations, a set of 3 quadratic second-order PDEs per each pair of views in 6 fixed variables. This system is solvable from at least three views. This is the first time that such a system is derived, which constitutes an important breakthrough in the NRSfM field. The system variables are the depth and its derivatives of one chosen reference view . No approximation is assumed for inferring the system, in contrast with the recent state-of-the-art methods which uses infinitesimal planarity or inextensibility approximations. We provide an efficient and robust global algorithm based on establishing an optimization problem where the recently discovered isometric NRSfM equations are involved. We evaluate our algorithm in experiments with synthetic and real data and make a comparison with the recent state-of-the-art methods. We show that our algorithm obtains competitive results with the state-of-the-art methods.