The goal of NNEF is to enable data scientists and engineers to easily transfer trained networks from their chosen training framework into a wide variety of inference engines. A stable, flexible and extensible standard that equipment manufacturers can rely on is critical for the widespread deployment of neural networks onto edge devices, and so NNEF encapsulates a complete description of the structure, operations and parameters of a trained neural network, independent of the training tools used to produce it and the inference engine used to execute it.
Convolutional Neural Networks (CNN) are computationally expensive, and so many companies are actively developing mobile and embedded processor architectures to accelerate neural net-based inferencing at high speed and low power. As a result of such rapid progress, the market for embedded neural net processing is in danger of fragmenting, creating barriers for developers seeking to configure and accelerate inferencing engines across multiple platforms.
Today, most neural net toolkits and inference engines use proprietary formats to describe the trained network parameters, making it necessary to construct many proprietary importers and exporters to enable a trained network to be executed across multiple inference engines.
NNEF has been designed to be reliably exported and imported across tools and engines such as Torch, Caffe, TensorFlow, Theano, Chainer, Caffe2, PyTorch, and MXNet. The NNEF 1.0 Specification covers a wide range of use-cases and network types with a rich set of operations and a scalable design that borrows syntactical elements from existing languages with formal elements to aid in correctness. NNEF includes the definition of custom compound operations that offers opportunities for sophisticated network optimizations. Future work will build on this architecture in a predictable way so that NNEF tracks the rapidly moving field of machine learning while providing a stable platform for deployment.
Released as a stable version after getting industry feedback based on provisional version
Initial focus on passing trained frameworks to embedded inference engines
Support deployable range of network topologies
Split Structure and Data files
Active NNEF roadmap development
NNEF Working Group Participants
NNEF welcomes the participation of the machine learning community to make NNEF useful for their own workflows. In addition, NNEF is working closely with the Khronos OpenVX™ working group to enable ingestion of NNEF files. The OpenVX Neural Network extension enables OpenVX 1.2 to act as a cross-platform inference engine, combining computer vision and deep learning operations in a single graph.
|Embedded Inferencing Import||Training Interchange|
|Defined Specification||Open Source Project|
|Multi-company Governance at Khronos||Initiated by Facebook & Microsoft|
|Stability for hardware deployment||Software stack flexibility|
ONNX and NNEF are Complementary
ONNX moves quickly to track authoring framework updates
NNEF provides a stable bridge from training into edge inferencing engines