The classical optical flow approach generates
motion fields
between successive images. Rather than motion, the generalised
formulation presented here computes the
displacement field
which defines the change in position of a pixel
from frame
t
to some arbitrary time
in the past or future. Thus the position of a pixel
at time
is given by
Assuming constant intensity over short sequences of images, the
estimated displacement field should warp a pixel of a particular
greylevel to one of the same greylevel in another image. The accuracy of
this motion can be measured by an
error term
which compares the greylevel
at a point
in the image at time
t
with the greylevel of the image at time
at the displaced pixel location,
i.e.
the error term is defined as
As the above term is not sufficiently constrained to compute the displacement field at each pixel, a motion model defining motion over a local neighbourhood of pixels is required to provide the additional constraint. The displacement field may now be related to any as yet unspecified linear motion model
where
are the motion parameters for the current image. Thus we can rewrite the
error term as
Combining this motion model expression with a first order Taylor series
expansion of the error term generates the usual optical flow constraint
which may be used to construct a
least squares functional
in terms of the motion parameters
where the
frame difference
is given by
.
A problem arises for all methods using the Taylor series expansion of
the
intensity constancy
constraint: the magnitude of detectable changes in displacement is
effectively restricted to the width of the spatial derivative operator
. Larger motions are likely to be corrupted by
aliasing
. The common use of Gaussian smoothing to suppress noise generates a
derivative operator whose inter-peak width is
. Thus the magnitude of detectable motions is determined by the standard
deviation
of the smoothing operation. Multi-resolution copes with this problem to
some extent by providing an ordered and overlapping set of detectable
velocity ranges.
Recent work has shown that a new iterative formulation of the intensity constancy constraint handles arbitrary large motions [ 7 , 8 ]. Given initial estimates of the optical flow, this approach described in full below can currently cope with remarkably large displacements provided the visual motions are not undergoing large accelerations. In this context, the width of the spatial derivative operator acts only as a limit on the detectable rate of change in velocity.
To generate a suitable iterative estimator, the error function of
equation
4
must be linearized around the previous
motion estimate
where
is the
motion compensated
frame
and
is the update to the previous motion parameters. The
displaced frame difference
is given by
where
is the displacement field generated by the
motion parameter
estimate using equation
3
.
Any errors in the
motion compensated
frame are compounded when computing its spatial derivatives. For this
reason, we use the spatial derivatives
as a good approximation to
. The error term may now be rewritten to relate the next estimate of the
motion parameters
to the current estimate of the displacement field
Such a formulation enables accurate motion estimation even in the
presence of large velocities given an accurate initial estimate of the
displacement field
.
A
neighbourhood
of pixels,
, consisting of a rectangular region in the current image, is used to
define a spatial support for a least-squares formulation. The temporal
support
(where
) is the set of images over which the minimisation is to be established.
The set
therefore provides the required spatio-temporal support. Using the error
term of equation
7
, an error functional in terms of the current motion estimate
may be defined as
Setting to zero the partial derivatives of the above functional with
respect to
generates the following iterative estimator for
where the first parameter estimate
can be computed from the initial displacement field
.
Graeme Jones
