OpenVX

Portable, Power-efficient Vision Processing

OpenVX is an open, royalty-free standard for cross platform acceleration of computer vision applications. OpenVX enables performance and power-optimized computer vision processing, especially important in embedded and real-time use cases such as face, body and gesture tracking, smart video surveillance, advanced driver assistance systems (ADAS), object and scene reconstruction, augmented reality, visual inspection, robotics and more.

OpenVX 1.1 was released on 2nd May 2016

OpenVX Neural Network Extension

The OpenVX™ working group has released an extension to enable Convolutional Neural Network topologies to be represented as OpenVX graphs and mixed with traditional vision functions. Learn more about the new extension below.

Read the press release

OpenVX 1.1 Specification

OpenVX 1.0.1 Specification

OpenVX Neural Network Extension

OpenVX – Vision Acceleration

Royalty-free open standard API

  • Reliably accelerated by hardware vendors
  • Tightly defined conformance tests

Targeted at low-power, real-time applications

  • Mobile and embedded platforms

Portability across diverse heterogeneous processors

  • ISPs, Dedicated hardware, DSPs and DSP arrays, GPUs, Multi-core CPUs …

Doesn’t require high-power CPU/GPU Complex

  • Low-power host can setup and manage frame-rate vision processing pipeline

OpenVX Graphs – The Key to Efficiency

OpenVX developers express a graph of image operations (‘Nodes’)

  • Nodes can be on any hardware or processor coded in any language

Graph enables implementations to optimize for power and performance

  • E.g. Nodes may be fused by the implementation to eliminate memory transfers
  • E.g. Processing can be tiled to keep data entirely in local memory/cache

Minimizes host interaction during frame-rate graph execution

  • Host processor can setup graph which can then execute almost autonomously

OpenVX Framework Efficiency..

OpenVX Framework Efficiency..

Layered Vision Processing Ecosystem

Layered Vision Processing Ecosystem

OpenVX Status

OpenVX 1.1 Specification released 2nd May 2016 at Embedded Vision Summit

  • 18 months after OpenVX 1.0 in October 2014
  • Expands node functionality AND enhances graph framework

OpenVX 1.0 open source sample implementation and conformance tests

  • Will be updated to OpenVX 1.1 in 1H16

Roadmap discussions

  • Significantly broaden node functionality – including in-graph neural nets
  • OpenVX SC – refining OpenVX for markets requiring API safety certification

OpenVX Status

OpenVX and OpenCV are Complementary

  OpenCV OpenVX
Implementation Community driven open source library Open standard API designed to be implemented by hardware vendors
Conformance Extensive OpenCV Test Suite but
no formal Adopters program
Implementations must pass defined conformance test suite to use trademark
Consistency Available functions can vary depending on implementation / platform All core functions must be available in all conformant implementations
Scope Very wide 1000s of imaging and vision functions Multiple camera APIs/interfaces Tight focus on core hardware accelerated functions for mobile vision – but extensible Uses external/native camera API
Efficiency Memory-based architecture Each operation reads and writes to memory Graph-based execution Optimizable computation and data transfer
Typical Use Case Rapid experimentation and
prototyping - especially on desktop
Production development & deployment on mobile and embedded devices
Embedded
Deployment
Re-usable code Callable library

OpenVX Neural Network Extension

The OpenVX™ working group has release an extension to enable Convolutional Neural Network topologies to be represented as OpenVX graphs and mixed with traditional vision functions.

Neural Network technology has seen recent explosive progress in solving pattern matching tasks in computer vision such as object recognition, face identification, image search, image to text, and is also playing a key part in enabling driver assistance and autonomous driving systems. Convolutional Neural Networks (CNN) are computationally expensive, and so many companies are actively developing mobile and embedded processor architectures to accelerate neural network-based inferencing at high speed and low power. As a result of such rapid progress, the market for embedded neural network processing is in danger of fragmenting, creating barriers for developers seeking to configure and accelerate inferencing engines across multiple platforms.

The OpenVX Neural Network extension specifies an architecture for executing CNN-based inference in OpenVX graphs. The extension defines a multi-dimensional tensor object data structure which can be used to connect neural network layers, represented as OpenVX nodes, to create flexible CNN topologies. OpenVX neural network layers types include convolution, pooling, fully-connected, normalization, soft-max and activation – with nine different activation functions. The extension enables neural network inferencing to be mixed with traditional vision processing operations in the same OpenVX graph.

Today, OpenVX has also released an Import/Export extension that complements the Neural Network extension by defining an API to import and export OpenVX objects, such as: traditional computer vision nodes, data objects of a graph or partial graph, and CNN objects including network weights and biases or complete networks.

The high-level abstraction of OpenVX enables implementers to accelerate a dataflow graph of vision functions across a diverse array of hardware and software acceleration platforms. The inclusion of neural net inferencing functionality in OpenVX enables the same portable, processor independent, expression of functionality with significant freedom and flexibility in how that inferencing is actually accelerated. The OpenVX Neural Net extension is released in provisional form to enable developers and implementers to provide feedback before finalization and industry feedback is welcomed at the OpenVX Forums.

Khronos is coordinating its neural network activities, and expects that NNEF files will be able to represent all aspects of an OpenVX neural network graph, and that OpenVX will enable import of network topologies via NNEF files through the Import/Export extension, once the NEFF format definition is complete.

Convolutional Neural Network topologies can be represented as OpenVX graphs

  • Layers are represented as OpenVX nodes
  • Layers connected by multi-dimensional tensors objects
  • Layers include convolution, normalization, pooling, fully-connected, soft-max
  • Also activation layers – with nine different activation functions
  • CNN nodes can be mixed with traditional vision nodes

Import/Export Extension

  • Efficient handling of network weights/biases or complete networks

The specification is provisional

OpenVX NNEF Extension
safety