The NNEF™ (Neural Network Exchange Format) working group at Khronos® has released bidirectional converters that enable easy import/export of NNEF files containing trained neural networks into and out of the TensorFlow and Caffe2 machine learning training frameworks. These open source tools are available now on the NNEF GitHub under the Apache 2.0 license and were developed in partnership with Au-Zone Technologies. The NNEF standard enables networks trained in tools such as TensorFlow or Caffe2 to be effectively exported and optimized for acceleration on a wide range of embedded inferencing engines.
The two new NNEF converters, along with a Caffe converter previously developed by the Khronos Group, take trained protobuf files from the training frameworks. These converters are easy-to-use since the original script isn’t needed and conversion can be done with simply the saved file. If its necessary to save compound operations by TensorFlow, Khronos has a previously developed TensorFlow converter, which reads and writes the Python script that builds the TensorFlow graphs, which is able to capture higher level functions in the script and export them as compound operations if needed.
Import works by converting an NNEF file back to the file format of the framework, reversing the conversion process. It can also generate Python script, if needed, for TensorFlow. This is a separate tool independent of the frameworks, just like the exporter. The primary use case is validation of the export. This demonstrates the exchange capabilities of NNEF.
The NNEF standard is focused on passing trained frameworks to embedded inference engines, addressing the ever-widening range of applications in the machine learning and inferencing space that need low-cost inferencing solutions. NNEF reduces industry fragmentation by enabling a rich mix of neural network training tools to be used to feed inference engines across a diverse mix of devices and platforms. Think of it as a “PDF for neural networks.”
The Khronos Group announced the ratification and the public release of the NNEF1.0 specification in August 2018 after positive industry feedback on the provisional specification released six months earlier.
NNEF significantly reduces deployment friction in the industry by eliminating the need for hardware inferencing engines to support multiple import formats and by enabling processor vendors to focus their resources on a single format designed to support effective optimization of quantized, variable-precision inferencing operations.
Looking forward, the NNEF working group is working on expanding and merging multiple format converters into a unified tool that can handle a wide range of supported formats with a unified interface, which will be published as a Python package.
Finally, the NNEF and OpenVX™ Working Groups are working closely within Khronos to develop open-source importers to enable OpenVX to ingest and execute NNEF files, enabling OpenVX to act as an open, cross-platform inference engine, melding computer vision and deep learning operations in a single graph description for highly-optimized hardware acceleration.
Khronos will be working to steadily expand range of NNEF tools, and member companies of the NNEF working group within Khronos Group, such as Almotive and AMD, are already developing tools that leverage the power of the NNEF standard.