BibTeX entry

@PHDTHESIS{200908Mukta_Prasad,
  AUTHOR={Mukta Prasad},
  TITLE={Class-based Single View Reconstruction},
  SCHOOL={Oxford University},
  MONTH=Aug,
  YEAR=2009,
  URL={http://www.bmva.org/theses/2009/2009-prasad.pdf},
}

Abstract

The aim of this thesis is to construct a realistic, freeform 3D model of an object from a single view, given the knowledge of the class it belongs to. “Classes” can be fruits (oranges, apples etc.), flowers (lilies, hibiscus etc.), faces etc. Single view reconstruction (SVR) is a severely under-constrained problem and relies on cues like shading, texture, occluding contour etc. In this thesis, first the projective properties of image silhouettes are exploited to effectively constrain the problem. We show how intuitive user-input and other image-based cues such as multi-local singularities (cusps) and creases can be incorporated to add more definition to the model. The problem is then, to find the smoothest surface, given the set of constraints. All the above constraints are incorporated as linear constraints in the optimization of a quadratic objective. The resultant framework is convex and can be solved easily. The parametric surface representation used here can model objects of any topology: genus 0, 1 or higher. An object class is strongly bound by characteristic shape, texture and a family of deformations. Thus, class information is an important cue for SVR, like texture and silhouette. Therefore, the problem of simultaneously reconstructing and learning classspecific shape models from photo collections is addressed next. Only a single view of each object instance is available, so this is an extension of the class-based SVR problem. Object classes which can be represented as wireframes are addressed, e.g. lily petals. We show that Non-Rigid Structure from Motion (NRSfM) can be extended to the scenario where each image is of a different object instance. However, this requires a novel method of defining correspondences. Instead of first finding correspondences and then employing existing Non-Rigid Structure from Motion (NRSfM) techniques, we frame this problem as one joint objective which integrates the correspondence finding problem with the other variables of NRSfM. This is solved jointly and analytically with effective bundle adjustment. A specialization of this method to stereo is shown to be an improvement over existing stereo approaches and methods in fitting 3D active shape models. Work of this nature depends on learning from training data and often requires userdriven annotation. Minimal, intuitive annotation is used during the course of this work. However, we also show how class-based information can also be used for the automatic detection and segmentation of class instances. Local image-based information is used to learn a classifier. This helps to differentiate between image edges lying on object class boundaries, from others. This learning is used to improve detection and segmentation techniques such as Chamfer matching and ObjCut. As a result an end-to-end class-based SVR system is built, which automatically detects, segments and subsequently reconstructions an object class instance with the above-mentioned techniques.