BibTeX entry

@PHDTHESIS{200702William_A._P._Smith,
  AUTHOR={William A. P. Smith},
  TITLE={Statistical Methods For Facial Shape-from-shading and Recognition},
  SCHOOL={University of York},
  MONTH=Feb,
  YEAR=2007,
  URL={http://www.bmva.org/theses/2007/2007-smith.pdf},
}

Abstract

This thesis presents research aimed at improving the quality of facial shape information that can be recovered from single intensity images using shape-from-shading, with the aim of exploiting this information for the purposes of face recognition and view synthesis. The common theme throughout this thesis is the use of statistical methods to offer enhanced accuracy and robustness over existing techniques for facial shape-from-shading. The work presented goes some way to reinstating shape-from-shading as a viable means to recover facial shape from single, real world images. In Chapter 2 we thoroughly survey the existing literature in the areas of face recognition, shape recovery and skin reflectance modelling. We draw from this review a number of important observations. The first is that existing solutions to the general shape-from-shading problem prove incapable of recovering accurate facial shape from real world images. The second is that statistical models have been shown to be highly effective in modelling facial appearance and shape variation and have been applied successfully to the problem of face recognition. Finally, we highlight the complex nature of light interaction with skin and note that previous attempts to apply shape-from-shading to real world face images have, almost exclusively, discounted these effects. Chapter 3 presents our first contribution, which is to explore the idea of incorporating a statistical model within an iterative shape-from-shading framework. In order to do so, we first show how a statistical model can be constructed in the domain of fields of surface normals. We overcome problems of modelling directional data using ideas borrowed from directional statistics and cartography. We use the model as a regularisation constraint within a shape-from-shading algorithm which imposes satisfaction of Lambert’s law as a hard constraint. We show how the approach provides both a model-based and data-driven solution and how the model-based solution can be used to estimate facial albedo. We use the estimated shape and albedo information for the purposes of novel view synthesis. In Chapter 4 we extend the ideas presented in Chapter 3 in a number of ways. We begin by reformulating the statistical model for fields of surface normals in terms of a distribution of points on a spherical manifold. We call on techniques from differential geometry and arrive at a model formulation which is more elegant and allows the whole shape-from-shading process to be couched in terms of operations on the tangent plane to the unit sphere. Our second contribution in this chapter is to show how robust statistics can be used to reduce the impact of regions of low albedo and cast shadows. This approach allows us to identify regions in which the image intensity obeys our simple local illumination model. In those regions which do not, we can use the statistical model to complete the surface. The result is improved performance under significantly non-frontal lighting and reduced sensitivity to albedo variation. We explore the use of the recovered shape and albedo information for face recognition. In Chapter 5 we expand our consideration of facial shape recovery into the domain of surface height. We present two statistical approaches to the problem of recovering surface height from fields of surface normals (the surface integration problem). The first is based on learning the relationship between the parameters of statistical models for the two representations. The second shows how the parameters of a statistical surface height model can be recovered from the field of surface normals directly. We extend this second approach further by showing how a statistical surface height model can be used to provide a constraint on the estimated field of surface normals within a shape-from-shading framework. The resulting algorithm retains the advantages of the techniques described in the preceding two chapters (strict satisfaction of local irradiance constraints) but yields a height map directly without having to integrate the surface normals. Our final contribution in Chapter 6 is to relax the assumptions made about the reflectance properties of skin. We show how an arbitrary radiance function can be estimated as part of a shape-from-shading algorithm, using the surface height constraint developed in the previous chapter. Further, we show how spatially varying reflectance properties can be accounted for by estimating a local albedo term as part of the iterative process. By fitting a parametric reflectance model to the recovered data we are able to extrapolate the reflectance properties beyond those present in the input image. We also demonstrate how the method can be applied to colour images and how this provides a route to facial colour constancy. The work in this thesis suggests that constraints provided by statistical models of face shape render the facial shape-from-shading tractable, even when complex, non-Lambertian reflectance effects are considered. The results suggest that information useful for illumination and pose insensitive face recognition may be recovered from one training image.