The OpenVX Specification  a73e458
Optical Flow Pyramid (LK)

Detailed Description

Computes the optical flow using the Lucas-Kanade method between two pyramid images.

The function is an implementation of the algorithm described in [1] [R00086]. The function inputs are two vx_pyramid objects, old and new, along with a vx_array of vx_keypoint_t structs to track from the old vx_pyramid. Both pyramids old and new pyramids must have the same dimensionality [R00087]. VX_SCALE_PYRAMID_HALF pyramidal scaling must be supported [R00088].
The function outputs a vx_array of vx_keypoint_t structs that were tracked from the old vx_pyramid to the new vx_pyramid [R00089]. Each element in the vx_array of vx_keypoint_t structs in the new array may be valid or not [R00090]. The implementation shall return the same number of vx_keypoint_t structs in the new vx_array that were in the older vx_array [R00091].

In more detail: The Lucas-Kanade method finds the affine motion vector \( V \) for each point in the old image tracking points array, using the following equation:

\[ \begin{bmatrix} V_x \\ V_y \end{bmatrix} = \begin{bmatrix} \sum_{i}{I_x}^2 & \sum_{i}{I_x*I_y} \\ \sum_{i}{I_x*I_y} & \sum_{i}{I_y}^2 \end{bmatrix}^{-1} \begin{bmatrix} -\sum_{i}{I_x*I_t} \\ -\sum_{i}{I_y*I_t} \end{bmatrix} \]

Where \( I_x \) and \( I_y \) are obtained using the Scharr gradients on the input image:

\[ G_x = \begin{bmatrix} +3 & 0 & -3 \\ +10 & 0 & -10 \\ +3 & 0 & -3 \end{bmatrix} \]

\[ G_y = \begin{bmatrix} +3 & +10 & +3 \\ 0 & 0 & 0 \\ -3 & -10 & -3 \end{bmatrix} \]

\( I_t \) is obtained by a simple difference between the same pixel in both images. \( I \) is defined as the adjacent pixels to the point \( p(x,y) \) under consideration. With a given window size of \( M \), \( I \) is \( M^2 \) points. The pixel \( p(x,y) \) is centered in the window. In practice, to get an accurate solution, it is necessary to iterate multiple times on this scheme (in a Newton-Raphson fashion) until:

Clients are responsible for editing the output vx_array of vx_keypoint_t structs array before applying it as the input vx_array of vx_keypoint_t structs for the next frame. For example, vx_keypoint_t structs with tracking_status set to zero may be removed by a client for efficiency.

This function changes just the x, y, and tracking_status members of the vx_keypoint_t structure and behaves as if it copied the rest from the old tracking vx_keypoint_t to new image vx_keypoint_t [R00094].

Functions

vx_node VX_API_CALL vxOpticalFlowPyrLKNode (vx_graph graph, vx_pyramid old_images, vx_pyramid new_images, vx_array old_points, vx_array new_points_estimates, vx_array new_points, vx_enum termination, vx_scalar epsilon, vx_scalar num_iterations, vx_scalar use_initial_estimate, vx_size window_dimension)
 [Graph] Creates a Lucas Kanade Tracking Node. More...
 

Function Documentation

◆ vxOpticalFlowPyrLKNode()

vx_node VX_API_CALL vxOpticalFlowPyrLKNode ( vx_graph  graph,
vx_pyramid  old_images,
vx_pyramid  new_images,
vx_array  old_points,
vx_array  new_points_estimates,
vx_array  new_points,
vx_enum  termination,
vx_scalar  epsilon,
vx_scalar  num_iterations,
vx_scalar  use_initial_estimate,
vx_size  window_dimension 
)

[Graph] Creates a Lucas Kanade Tracking Node.

Parameters
[in]graphThe reference to the graph [R00377].
[in]old_imagesInput of first (old) image pyramid in VX_DF_IMAGE_U8 [R00378].
[in]new_imagesInput of destination (new) image pyramid VX_DF_IMAGE_U8 [R00379].
[in]old_pointsAn array of key points in a vx_array of VX_TYPE_KEYPOINT [R00380]; those key points are defined at the old_images high resolution pyramid.
[in]new_points_estimatesAn array of estimation on what is the output key points in a vx_array of VX_TYPE_KEYPOINT [R00381]; those keypoints are defined at the new_images high resolution pyramid.
[out]new_pointsAn output array of key points in a vx_array of VX_TYPE_KEYPOINT [R00382]; those key points are defined at the new_images high resolution pyramid.
[in]terminationThe termination can be VX_TERM_CRITERIA_ITERATIONS or VX_TERM_CRITERIA_EPSILON or VX_TERM_CRITERIA_BOTH [R00383].
[in]epsilonThe vx_float32 error for terminating the algorithm [R00384].
[in]num_iterationsThe number of iterations. Use a VX_TYPE_UINT32 scalar [R00385].
[in]use_initial_estimateUse a VX_TYPE_BOOL scalar [R00386].
[in]window_dimensionThe size of the window on which to perform the algorithm [R00387]. See VX_CONTEXT_OPTICAL_FLOW_MAX_WINDOW_DIMENSION
Returns
vx_node [R00388].
Return values
vx_nodeA node reference. Any possible errors preventing a successful creation should be checked using vxGetStatus