The OpenVX Specification
dba1aa3

Computes the optical flow using the LucasKanade method between two pyramid images.
The function is an implementation of the algorithm described in [1]. The function inputs are two vx_pyramid
objects, old and new, along with a vx_array
of vx_keypoint_t
structs to track from the old vx_pyramid
. Both pyramids old and new pyramids must have the same dimensionality. VX_SCALE_PYRAMID_HALF
pyramidal scaling must be supported.
The function outputs a vx_array
of vx_keypoint_t
structs that were tracked from the old vx_pyramid
to the new vx_pyramid
. Each element in the vx_array
of vx_keypoint_t
structs in the new array may be valid or not. The implementation shall return the same number of vx_keypoint_t
structs in the new vx_array
that were in the older vx_array
.
In more detail: The LucasKanade method finds the affine motion vector \( V \) for each point in the old image tracking points array, using the following equation:
\[ \begin{bmatrix} V_x \\ V_y \end{bmatrix} = \begin{bmatrix} \sum_{i}{I_x}^2 & \sum_{i}{I_x*I_y} \\ \sum_{i}{I_x*I_y} & \sum_{i}{I_y}^2 \end{bmatrix}^{1} \begin{bmatrix} \sum_{i}{I_x*I_t} \\ \sum_{i}{I_y*I_t} \end{bmatrix} \]
Where \( I_x \) and \( I_y \) are obtained using the Scharr gradients on the input image:
\[ G_x = \begin{bmatrix} +3 & 0 & 3 \\ +10 & 0 & 10 \\ +3 & 0 & 3 \end{bmatrix} \]
\[ G_y = \begin{bmatrix} +3 & +10 & +3 \\ 0 & 0 & 0 \\ 3 & 10 & 3 \end{bmatrix} \]
\( I_t \) is obtained by a simple difference between the same pixel in both images. \( I \) is defined as the adjacent pixels to the point \( p(x,y) \) under consideration. With a given window size of \( M \), \( I \) is \( M^2 \) points. The pixel \( p(x,y) \) is centered in the window. In practice, to get an accurate solution, it is necessary to iterate multiple times on this scheme (in a NewtonRaphson fashion) until:
vx_true_e
is given as the input to use_initial_estimates
, the algorithm starts by calculating \( I_t \) as the difference between the old image and the pixel with the initial estimated coordinates in the new image. The input vx_array
of vx_keypoint_t
structs with tracking_status
set to zero (lost) are copied to the new vx_array
.Clients are responsible for editing the output vx_array
of vx_keypoint_t
structs array before applying it as the input vx_array
of vx_keypoint_t
structs for the next frame. For example, vx_keypoint_t
structs with tracking_status
set to zero may be removed by a client for efficiency.
This function changes just the x, y, and tracking_status members of the vx_keypoint_t
structure and behaves as if it copied the rest from the old tracking vx_keypoint_t
to new image vx_keypoint_t
.
Functions  
vx_node VX_API_CALL  vxOpticalFlowPyrLKNode (vx_graph graph, vx_pyramid old_images, vx_pyramid new_images, vx_array old_points, vx_array new_points_estimates, vx_array new_points, vx_enum termination, vx_scalar epsilon, vx_scalar num_iterations, vx_scalar use_initial_estimate, vx_size window_dimension) 
[Graph] Creates a Lucas Kanade Tracking Node. More...  
vx_status VX_API_CALL  vxuOpticalFlowPyrLK (vx_context context, vx_pyramid old_images, vx_pyramid new_images, vx_array old_points, vx_array new_points_estimates, vx_array new_points, vx_enum termination, vx_scalar epsilon, vx_scalar num_iterations, vx_scalar use_initial_estimate, vx_size window_dimension) 
[Immediate] Computes an optical flow on two images. More...  
vx_node VX_API_CALL vxOpticalFlowPyrLKNode  (  vx_graph  graph, 
vx_pyramid  old_images,  
vx_pyramid  new_images,  
vx_array  old_points,  
vx_array  new_points_estimates,  
vx_array  new_points,  
vx_enum  termination,  
vx_scalar  epsilon,  
vx_scalar  num_iterations,  
vx_scalar  use_initial_estimate,  
vx_size  window_dimension  
) 
[Graph] Creates a Lucas Kanade Tracking Node.
[in]  graph  The reference to the graph. 
[in]  old_images  Input of first (old) image pyramid in VX_DF_IMAGE_U8 . 
[in]  new_images  Input of destination (new) image pyramid VX_DF_IMAGE_U8 . 
[in]  old_points  An array of key points in a vx_array of VX_TYPE_KEYPOINT ; those key points are defined at the old_images high resolution pyramid. 
[in]  new_points_estimates  An array of estimation on what is the output key points in a vx_array of VX_TYPE_KEYPOINT ; those keypoints are defined at the new_images high resolution pyramid. 
[out]  new_points  An output array of key points in a vx_array of VX_TYPE_KEYPOINT ; those key points are defined at the new_images high resolution pyramid. 
[in]  termination  The termination can be VX_TERM_CRITERIA_ITERATIONS or VX_TERM_CRITERIA_EPSILON or VX_TERM_CRITERIA_BOTH . 
[in]  epsilon  The vx_float32 error for terminating the algorithm. 
[in]  num_iterations  The number of iterations. Use a VX_TYPE_UINT32 scalar. 
[in]  use_initial_estimate  Use a VX_TYPE_BOOL scalar. 
[in]  window_dimension  The size of the window on which to perform the algorithm. See VX_CONTEXT_OPTICAL_FLOW_MAX_WINDOW_DIMENSION 
vx_node
. vx_node  A node reference. Any possible errors preventing a successful creation should be checked using vxGetStatus 
vx_status VX_API_CALL vxuOpticalFlowPyrLK  (  vx_context  context, 
vx_pyramid  old_images,  
vx_pyramid  new_images,  
vx_array  old_points,  
vx_array  new_points_estimates,  
vx_array  new_points,  
vx_enum  termination,  
vx_scalar  epsilon,  
vx_scalar  num_iterations,  
vx_scalar  use_initial_estimate,  
vx_size  window_dimension  
) 
[Immediate] Computes an optical flow on two images.
[in]  context  The reference to the overall context. 
[in]  old_images  Input of first (old) image pyramid in VX_DF_IMAGE_U8 . 
[in]  new_images  Input of destination (new) image pyramid in VX_DF_IMAGE_U8 
[in]  old_points  an array of key points in a vx_array of VX_TYPE_KEYPOINT those key points are defined at the old_images high resolution pyramid 
[in]  new_points_estimates  an array of estimation on what is the output key points in a vx_array of VX_TYPE_KEYPOINT those keypoints are defined at the new_images high resolution pyramid 
[out]  new_points  an output array of key points in a vx_array of VX_TYPE_KEYPOINT those key points are defined at the new_images high resolution pyramid 
[in]  termination  termination can be VX_TERM_CRITERIA_ITERATIONS or VX_TERM_CRITERIA_EPSILON or VX_TERM_CRITERIA_BOTH 
[in]  epsilon  is the vx_float32 error for terminating the algorithm 
[in]  num_iterations  is the number of iterations. Use a VX_TYPE_UINT32 scalar. 
[in]  use_initial_estimate  Can be set to either vx_false_e or vx_true_e . 
[in]  window_dimension  The size of the window on which to perform the algorithm. See VX_CONTEXT_OPTICAL_FLOW_MAX_WINDOW_DIMENSION 
vx_status_e
enumeration. VX_SUCCESS  Success 
*  An error occurred. See vx_status_e . 