We wish to calculate u and v, but unfortunately the above constraint gives us only one equation per pixel for two unknowns, so this is not enough by itself. To see why, consider the images in Fig. 43.
Fig. 43 Ambiguity in determining optical flow
The lines are contours of equal intensity in the two images, which are taken at times apart.
The difficulty is in telling whether the part of the scene represented by point A in the first image has moved to point B, or B', or indeed any other point of the same intensity in the second image.
Thus further information is required allow determination of u and v.
A key observation is that except near the edges of moving objects, the motion that is observed in adjacent pixels will be quite similar.
A measure of how much the optical flow deviates from this smoothly
varying ideal can be calculated by evaluating the following
over the whole image, where the derivatives like measure how rapidly the velocity is changing on going across the image.