OpenVX Hardware acceleration API for Computer Vision applications and libraries

OpenVX enables performance and power optimized computer vision algorithms for use cases such as face, body and gesture tracking, smart video surveillance, automatic driver assistance systems, object and scene reconstruction, augmented reality, visual inspection, robotics and more. The provisional release of the specification enables developers and implementers to provide feedback before specification finalization, which is expected within six months.

OpenVX 1.0 Provisional Specification

OpenVX enables significant implementation innovation while maintaining a consistent API for developers. An OpenVX application expresses vision processing holistically as a graph of function nodes. An OpenVX implementer can optimize graph execution through a wide variety of techniques such as: acceleration of nodes on CPUs, GPUs, DSPs or dedicated hardware, compiler optimizations, node coalescing, and tiled execution to keep sections of processed images in local memories as they flow through the graph. Khronos has released a provisional tiled execution extension alongside the main OpenVX specification to enable user custom kernels to exploit this style of optimization. Additionally, Khronos has released the VXU™ utility library to enable developers using OpenVX to call individual nodes as standalone functions for easy code migration.

OpenVX can be used directly by applications or to accelerate higher-level middleware, such as the popular OpenCV open source vision library that is often used for application prototyping. OpenVX will have extensive conformance tests to complement a focused and tightly defined finalized specification for consistent and reliable operation across multiple vendors and platforms making OpenVX an ideal foundation for shipping production vision applications. Finally, as any Khronos specification, OpenVX is extensible to enable nodes to be deployed to meet customer needs, ahead of being integrated into the core specification.

OpenVX – Power Efficient Vision Processing

Power efficient vision processing with OpenVX
  • Acceleration API for real-time vision
    • Focus on mobile and embedded systems
  • Diversity of efficient implementations
    • From programmable processors, through GPUs to dedicated hardware pipelines
  • Tightly specified API with conformance
    • Portable, production-grade vision functions
  • Complementary to OpenCV
    • Which is great for prototyping

OpenVX Graphs

  • Vision processing directed graphs for power and performance efficiency
    • Each Node can be implemented in software or accelerated hardware
    • Nodes may be fused by the implementation to eliminate memory transfers
    • Tiling extension enables user nodes (extensions) to also run in local memory
  • VXU Utility Library for access to single nodes
    • Easy way to start using OpenVX
  • EGLStreams can provide data and event interop with other APIs
    • BUT use of other Khronos APIs are not mandated
OpenVX Graphs

OpenVX 1.0 Function Overview

  • Core data structures
    • Images and Image Pyramids
    • Processing Graphs, Kernels, Parameters
  • Image Processing
    • Arithmetic, Logical, and statistical operations
    • Multichannel Color and BitDepth Extraction and Conversion
    • 2D Filtering and Morphological operations
    • Image Resizing and Warping
  • Core Computer Vision
    • Pyramid computation
    • Integral Image computation
  • Feature Extraction and Tracking
    • Histogram Computation and Equalization
    • Canny Edge Detection
    • Harris and FAST Corner detection
    • Sparse Optical Flow

OpenVX Participants and Timeline

  • Aiming for specification finalization by mid-2014
  • Itseez is working group chair
  • Qualcomm and TI are specification editors
OpenVX Participants

OpenVX and OpenCV are Complementary

  OpenCV OpenVX
Governance Open Source
Community Driven
No formal specification
Formal specification and conformance tests
Implemented by hardware vendors
Scope Very wide
1000s of functions of imaging and vision
Multiple camera APIs/interfaces
Tight focus on hardware accelerated functions for mobile vision
Use external camera API
Conformance No Conformance testing
Every vendor implements different subset
Full conformance test suite / process
Reliable acceleration platform
Use Case Rapid prototyping Production deployment
Efficiency Memory-based architecture
Each operation reads and writes memory
Graph-based execution
Optimizable computation, data transfer
Portability APIs can vary depending on processor Hardware abstracted for portability

OpenVX and OpenCL are Complementary

  OpenCL OpenVX
Use Case General Heterogeneous programming Domain targeted - vision processing
Architecture Language-based
– needs online compilation
- no online compiler required
Target Hardware ‘Exposed’ architected memory model – can impact performance portability Abstracted node and memory model - diverse implementations can be optimized for power and performance
Precision Full IEEE floating point mandated Minimal floating point requirements – optimized for vision operators
Ease of Use Focus on general-purpose math libraries with no built-in vision functions Fully implemented vision operators and framework ‘out of the box’

Stereo Machine Vision

Stereo Machine Vision with OpenVX

Khronos APIs for Augmented Reality

Khronos APIs for Augmented Reality chart