OpenVX is an open, royalty-free standard for cross platform acceleration of computer vision applications. OpenVX enables performance and power-optimized computer vision processing, especially important in embedded and real-time use cases such as face, body and gesture tracking, smart video surveillance, advanced driver assistance systems (ADAS), object and scene reconstruction, augmented reality, visual inspection, robotics and more.
OpenVX 1.2 was released on 1st May 2017
OpenVX Neural Network Extension
The OpenVX™ working group has released an extension to enable Convolutional Neural Network topologies to be represented as OpenVX graphs and mixed with traditional vision functions. Learn more about the new extension below.
Read the press release
OpenVX – Vision Acceleration
Royalty-free open standard API
- Reliably accelerated by hardware vendors
- Tightly defined conformance tests
Targeted at low-power, real-time applications
- Mobile and embedded platforms
Portability across diverse heterogeneous processors
- ISPs, Dedicated hardware, DSPs and DSP arrays, GPUs, Multi-core CPUs …
Doesn’t require high-power CPU/GPU Complex
- Low-power host can setup and manage frame-rate vision processing pipeline
OpenVX Graphs – The Key to Efficiency
OpenVX developers express a graph of image operations (‘Nodes’)
- Nodes can be on any hardware or processor coded in any language
Graph enables implementations to optimize for power and performance
- E.g. Nodes may be fused by the implementation to eliminate memory transfers
- E.g. Processing can be tiled to keep data entirely in local memory/cache
Minimizes host interaction during frame-rate graph execution
- Host processor can setup graph which can then execute almost autonomously
OpenVX Framework Efficiency..
Layered Vision Processing Ecosystem
OpenVX 1.1 Specification released 2nd May 2016 at Embedded Vision Summit
- 18 months after OpenVX 1.0 in October 2014
- Expands node functionality AND enhances graph framework
OpenVX 1.0 open source sample implementation and conformance tests
- Will be updated to OpenVX 1.1 in 1H16
- Significantly broaden node functionality – including in-graph neural nets
- OpenVX SC – refining OpenVX for markets requiring API safety certification
OpenVX and OpenCV are Complementary
||Community driven open source library
||Open standard API designed to be implemented by hardware vendors
||Extensive OpenCV Test Suite but
no formal Adopters program
|Implementations must pass defined conformance test suite to use trademark
||Available functions can vary depending on implementation / platform
||All core functions must be available in all conformant implementations
||Very wide 1000s of imaging and vision functions Multiple camera APIs/interfaces
||Tight focus on core hardware accelerated functions for mobile vision – but extensible Uses external/native camera API
||Memory-based architecture Each operation reads and writes to memory
||Graph-based execution Optimizable computation and data transfer
|Typical Use Case
||Rapid experimentation and
prototyping - especially on desktop
|Production development & deployment on mobile and embedded devices
OpenVX Neural Network Extension
The OpenVX™ working group has release an extension to enable Convolutional Neural Network topologies to be represented as OpenVX graphs and mixed with traditional vision functions.
Neural Network technology has seen recent explosive progress in solving pattern matching tasks in computer vision such as object recognition, face identification, image search, image to text, and is also playing a key part in enabling driver assistance and autonomous driving systems. Convolutional Neural Networks (CNN) are computationally expensive, and so many companies are actively developing mobile and embedded processor architectures to accelerate neural network-based inferencing at high speed and low power. As a result of such rapid progress, the market for embedded neural network processing is in danger of fragmenting, creating barriers for developers seeking to configure and accelerate inferencing engines across multiple platforms.
The OpenVX Neural Network extension specifies an architecture for executing CNN-based inference in OpenVX graphs. The extension defines a multi-dimensional tensor object data structure which can be used to connect neural network layers, represented as OpenVX nodes, to create flexible CNN topologies. OpenVX neural network layers types include convolution, pooling, fully-connected, normalization, soft-max and activation – with nine different activation functions. The extension enables neural network inferencing to be mixed with traditional vision processing operations in the same OpenVX graph.
Today, OpenVX has also released an Import/Export extension that complements the Neural Network extension by defining an API to import and export OpenVX objects, such as: traditional computer vision nodes, data objects of a graph or partial graph, and CNN objects including network weights and biases or complete networks.
The high-level abstraction of OpenVX enables implementers to accelerate a dataflow graph of vision functions across a diverse array of hardware and software acceleration platforms. The inclusion of neural net inferencing functionality in OpenVX enables the same portable, processor independent, expression of functionality with significant freedom and flexibility in how that inferencing is actually accelerated. The OpenVX Neural Net extension is released in provisional form to enable developers and implementers to provide feedback before finalization and industry feedback is welcomed at the OpenVX Forums.
Khronos is coordinating its neural network activities, and expects that NNEF files will be able to represent all aspects of an OpenVX neural network graph, and that OpenVX will enable import of network topologies via NNEF files through the Import/Export extension, once the NEFF format definition is complete.
Convolutional Neural Network topologies can be represented as OpenVX graphs
- Layers are represented as OpenVX nodes
- Layers connected by multi-dimensional tensors objects
- Layers include convolution, normalization, pooling, fully-connected, soft-max
- Also activation layers – with nine different activation functions
- CNN nodes can be mixed with traditional vision nodes
- Efficient handling of network weights/biases or complete networks
The specification is provisional