OpenVX

Portable, Power-efficient Vision Processing

OpenVX is an open, royalty-free standard for cross platform acceleration of computer vision applications. OpenVX enables performance and power-optimized computer vision processing, especially important in embedded and real-time uses cases such as face, body and gesture tracking, smart video surveillance, advanced driver assistance systems (ADAS), object and scene reconstruction, augmented reality, visual inspection, robotics and more.

OpenVX 1.0 Specification

OpenVX Participants and Timeline

  • Finalized OpenVX 1.0 specification released October 2014
    • www.khronos.org/openvx
  • Full conformance test suite and Adopters Program immediately available
    • $20K Adopters fee ($15K for members) – working group reviews submitted results
    • Test suite exercises graph framework and functionality of each OpenVX 1.0 node
    • Approved Conformant implementations can use the OpenVX trademark
OpenVX Participants

OpenVX defines a higher level of abstraction for execution and memory models than compute frameworks such as OpenCL™, enabling significant implementation innovation and efficient execution on a wide range of architectures while maintaining a consistent vision acceleration API for application portability.

Power efficient vision processing with OpenVX

OpenVX – Power Efficient Vision Acceleration

  • Out-of-the-Box vision acceleration framework
    • Enables low-power, real-time applications
    • Targeted at mobile and embedded platforms
  • Functional Portability
    • Tightly defined specification
    • Full conformance tests
  • Performance portability across diverse HW
    • Higher-level abstraction hides hardware details
    • ISPs, Dedicated hardware, DSPs and DSP arrays, GPUs, Multi-core CPUs …
  • Enables low-power, always-on acceleration
    • Can run solely on dedicated vision hardware
    • Does not require full SOC CPU/GPU complex to be powered on
“Increasingly powerful and efficient processors and image sensors are enabling engineers to incorporate visual intelligence into a wide range of systems and applications,” said Jeff Bier, founder of the Embedded Vision Alliance. “A key challenge for engineers is efficiently mapping complex algorithms onto the processor best suited to the application. OpenVX is an important step towards easing this challenge.”

An OpenVX developer expresses a connected graph of vision nodes that an implementer can execute and optimize through a wide variety of techniques such as: acceleration on CPUs, GPUs, DSPs or dedicated hardware, compiler optimizations, node coalescing, and tiled execution to keep sections of processed images in local memories. This architectural agility enables OpenVX applications on a diversity of systems optimized for different levels of power and performance, including very battery-sensitive, vision-enabled, wearable displays.

OpenVX Graphs

  • Vision processing directed graphs for power and performance efficiency
    • Each Node can be implemented in software or accelerated hardware
    • Nodes may be fused by the implementation to eliminate memory transfers
    • Tiling extension enables user nodes (extensions) to also run in local memory
  • VXU Utility Library for access to single nodes
    • Easy way to start using OpenVX
  • EGLStreams can provide data and event interop with other APIs
    • BUT use of other Khronos APIs are not mandated

OpenVX Graphs

Stereo Machine Vision Example

Stereo Machine Vision with OpenVX

The precisely defined specification and conformance tests for OpenVX make it ideal for deployment in production systems, where cross-vendor consistency and reliability are essential. OpenVX is complementary to the popular OpenCV open source vision library that is also used for application prototyping but is not so tightly defined and lacks OpenVX graph optimizations. Khronos has defined the VXU™ utility library to enable developers to call individual OpenVX nodes as standalone functions for efficient code migration from traditional vision libraries such as OpenCV.

OpenVX and OpenCV are Complementary

  OpenCV OpenVX
Governance Open Source
Community Driven
No formal specification
Formal specification and conformance tests
Implemented by hardware vendors
Scope Very wide
1000s of functions of imaging and vision
Multiple camera APIs/interfaces
Tight focus on hardware accelerated functions for mobile vision
Use external camera API
Conformance No Conformance testing
Every vendor implements different subset
Full conformance test suite / process
Reliable acceleration platform
Use Case Rapid prototyping Production deployment
Efficiency Memory-based architecture
Each operation reads and writes memory
Graph-based execution
Optimizable computation, data transfer
Portability APIs can vary depending on processor Hardware abstracted for portability

As any Khronos specification, OpenVX is extensible to enable nodes to be defined and deployed to meet customer needs, ahead of being integrated into the core specification.

OpenVX 1.0 Function Overview

  • Core data structures
    • Images and Image Pyramids
    • Processing Graphs, Kernels, Parameters
  • Image Processing
    • Arithmetic, Logical, and statistical operations
    • Multichannel Color and BitDepth Extraction and Conversion
    • 2D Filtering and Morphological operations
    • Image Resizing and Warping
  • Core Computer Vision
    • Pyramid computation
    • Integral Image computation
  • Feature Extraction and Tracking
    • Histogram Computation and Equalization
    • Canny Edge Detection
    • Harris and FAST Corner detection
    • Sparse Optical Flow
Several Khronos APIs can be used for vision processing - developers can select the API that best meets their needs. Also, often several Khronos APIs can be used to construct an accelerated vision stack e.g. the OpenCL low-level compute API will be used to implement OpenVX nodes on programmable systems.

Khronos APIs for Vision Processing

OpenGL ES and OpenGL

GPU Compute Shaders (OpenGL 4.X and OpenGL ES 3.1)
Pervasively available on almost any mobile device or OS
Easy integration into graphics apps – no vision/compute API interop needed

Program in GLSL not C
Limited to acceleration on a single GPU

OpenCL

General Purpose Heterogeneous Programming Framework
Flexible, low-level access to any devices with OpenCL compiler
Single programming and run-time framework for CPUs, GPUs, DSPs, hardware
Open standard for any device or OS – being used as backed by many languages and frameworks

Needs full compiler stack and IEEE precision

OpenVX

Out of the Box Vision Framework - Operators and graph framework library
Can run some or all modes on dedicated hardware – no compiler needed
Higher-level abstraction means easier performance portability to diverse hardware
Graph optimization opens up possibility of low-power, always-on vision acceleration

Fixed set of operators – but can be extended


It is possible to use OpenCL or GLSL to build OpenVX Nodes on programmable devices!

OpenVX is a member of the Khronos API Ecosystem that are designed to operate standalone or together to enable advanced visual computing applications such as Augmented Reality

Khronos APIs for Augmented Reality

Khronos APIs for Augmented Reality chart
safety