We wish to calculate u and v, but unfortunately the above constraint gives us only one equation per pixel for two unknowns, so this is not enough by itself. To see why, consider the images in Fig. 43.
Fig. 43 Ambiguity in determining optical flow
The lines are contours of
equal intensity in the two images, which are taken at times
apart.
The difficulty is in telling whether the part of the scene represented by point A in the first image has moved to point B, or B', or indeed any other point of the same intensity in the second image.
Thus further information is required allow determination of u and v.
A key observation is that except near the edges of moving objects, the motion that is observed in adjacent pixels will be quite similar.
A measure of how much the optical flow deviates from this smoothly
varying ideal can be calculated by evaluating the following
integral 
over the whole image,
where the derivatives like 
measure how rapidly the
velocity is changing on going across the image.
David Marshall 1994-1997
