Fitting models to data: Accuracy, Speed, Robustness

Andrew Fitzgibbon

Abstract

In vision and machine learning, almost everything we do may be considered to be a form of model fitting. Whether estimating the parameters of a convolutional neural network, target tracking, computing lowdimensional representations of datasets, computing structure and motion from image collections, estimating parameters for an inference model such as Markov random fields, or extracting shape spaces such as active appearance models, it almost always boils down to minimizing an objective containing some parameters of interest as well as some latent or nuisance parameters. This tutorial will describe several tools and techniques for solving such optimization problems, covering a broad syllabus: vector and matrix calculus; nonlinear optimization algorithms such as gradient descent and its many variants, Quasi-Newton and Gauss-Newton derivates such as LBFGS and Levenberg Marquardt; modelling options for curves and surfaces; how to deal with missing data; how to deal with outliers, including how to optimize functions with robust kernels. There will be maths, and lots of it, but I hope to make it clear enough that anyone with decent high-school linear algebra and calculus can benefit, while nevertheless showing mathematical experts a few useful tricks.

Session

Tutorial

Files

Paper (PDF, 132K)

DOI

10.5244/C.29.1
https://dx.doi.org/10.5244/C.29.1

Citation

Video Part 1

Video Part 2

Video Part 3

Video Part 4

Bibtex

@inproceedings{BMVC2015_1,
	title={Fitting models to data: Accuracy, Speed, Robustness},
	author={Andrew Fitzgibbon},
	year={2015},
	month={September},
	pages={1.1-1.1},
	articleno={1},
	numpages={1},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.1},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.1}
}