3 Temporal Stretch Correlation

The previous section shows that for any given stereo matching algorithm we can minimise the probability of mismatch by reducing the search area. However, in the absence of information regarding the range of expected disparities, a suitable minimum search area cannot be established. One way to acquire this information is to encapsulate the matching algorithm within a temporal loop where disparities from the previous frame are used to seed matching in the current frame. In this way the search ranges can be kept as small as possible with the matching algorithm being used to track disparities from frame to frame, thus reducing the probability of a mismatch.

To achieve this within the SC algorithm the search range along an epi-polar for a given block is based on the disparities recovered locally on the previous frame. The maximum and minimum disparities from the neighbourhood surrounding the current block are used to direct the search range.

3.1 Stochastic search

When attempting to design an algorithm to exploit temporal information we have to be careful not to construct a system which converges on the wrong solution. A suitable analogy is with simulated annealing algorithms. Such algorithms perform well because the probability of remaining with the global minimum (once located) is greater than the probability of switching to a local minima. Also the probability of leaving a local minima is greater than that of leaving the global minimum.

Relying on the previous disparity results in order to guide the current matching, permits the propagation of incorrect matches, possibly recovered at the start of the sequence. It is possible to limit the extent of these mismatches by introducing a stochastic full epi-polar search into the matching algorithm. This is achieved by randomly performing a full epi-polar search on each of the correlation blocks in the image. This check looks for stronger correlation matches along the entire right epi-polar. Coupled with global constraints this provides the flexibility for the temporal algorithm to open up previously unmatched or mismatched regions. The temporal algorithm searches disparities from a surrounding neighbourhood, therefore full epi-polar searches are only necessary on 1 in 49 (2%) of the blocks in the image in order to recover an effective `full' search coverage. Thus we achieve algorithmic robustness in a manner analogous with simulated annealing algorithms.

3.2 Temporal consistency

The integration of temporal information increases robustness against artifacts, caused by effects such as illumination variations, local occlusions or self similarity, which can introduce `fake' minima into the search space. With temporal consistency the size of the search area need only reflect the expected inter-frame motion of the scene and any expected location inaccuracies. Further, boot-strapping the matching algorithm in this way seeds the matcher with the most probable hypothesis for the match location given the prior evidence. Given the transitory nature of most image artifacts, temporal integration is more likely to return the global minima.