Planar Homography Estimation from Traffic Streams via Energy Functional Minimization

Abstract

The 3x3 homography matrix specifies the mapping between two images of the same plane as viewed by a pinhole camera. Knowledge of the matrix allows one to remove the perspective distortion and apply any similarity transform, effectively making possible the measurement of distances and angles on the image. A rectified road scene for instance, where vehicles can be segmented and tracked, gives rise to ready estimates of their velocities and spacing or categorization of their type.

Typical road scenes render the classical approach to homography estimation difficult. The Direct Linear Transform is highly susceptible to noise and usually requires refining via an further nonlinear penalty minimization. Additionally, the penalty is a function of the displacement between measured and calibrated coordinates, a quantity unavailable in a scene for which we have no knowledge of the road coordinates. We propose instead to achieve metric rectification via the minimization of an energy that measures the violation of two constraints: the divergence-free nature of the traffic flow and the orthogonality of the flow and transverse directions under the true transform.

Given that an homography is only determined up to scale, the minimization is performed on the Lie group $SL(3)$, for which we develop a gradient descent algorithm. While easily expressed in the world frame, the energy must be computed from measurements made in the image and thus must be pulled back using standard differential geometric machinery to the image frame. We develop an enhancement to the algorithm by incorporating optical flow ideas and apply it to both a noiseless test case and a suite of real-world video streams to demonstrate its efficacy and convergence. Finally, we discuss the extension to a 3D-to-planar mapping for vehicle height inference and an homography that is allowed to vary over the image, invoking a minimization on Diff$(SL(3))$.