Intel Compute Sticks 2

I recently got the Intel Compute Stick 2 (NCS2). It can do neural network inference. The process is to start with a frozen tensorflow file (.pb), then convert it to IR format (which the NCS2 can understand). You need to use OpenVINO for it. There was an earlier API which is now defunct. Here is my experience with it.

Summary:

Installation : Getting Started Guide
Official Examples : ncappzoo
My (mpkuse’s) Standalone MNIST Demo: https://github.com/mpkuse/my_try_NCS2

Installation

I have a fresh install of Ubuntu 16.04 on Intel i7 computer. I followed the official getting started (https://software.intel.com/en-us/articles/get-started-with-neural-compute-stick). I was able to install OpenVINO and could run the sample program to get the output as shown in the getting started page.

Additional Examples

After the installation, I am trying to learn how to use it. I am exploring the repo: https://github.com/movidius/ncappzoo. Particularly I am using the following app, apps/simple_classifier_py. You are encouraged to try more.

# Before you run any examples, remember to setup the environment: 
$ source ~/intel/openvino/bin/setupvars.sh

$ cd ncappzoo/apps/simple_classifier_py
$ make run
simple_classifier_py: Compiling the model to IR...
- Googlenet IR files found. No need to compile.

simple_classifier_py: Making default models...

simple_classifier_py: Making dependencies...

simple_classifier_py: Making the required data...
make[1]: Entering directory '/home/dji/Downloads/ncappzoo/data/ilsvrc12'

Making all for ilsvrc data... 
make[1]: Leaving directory '/home/dji/Downloads/ncappzoo/data/ilsvrc12'

simple_classifier_py: Running the sample...
Intel OpenVINO environment is already set!
python3 run.py --ir=googlenet-v1.xml --image=../../data/images/nps_electric_guitar.png --labels=../../data/ilsvrc12/synset_labels.txt 

Starting application...
   - Plugin:       Myriad
   - IR File:      googlenet-v1.xml
   - Input Shape:  [1, 3, 224, 224]
   - Output Shape: [1, 1000]
   - Labels File:  ../../data/ilsvrc12/synset_labels.txt
   - Mean File:    None
   - Image File:    ../../data/images/nps_electric_guitar.png

 **********  Results  ***********

 Prediction is 99.6% electric guitar

High Level Overview

The process to get your neural networks working on NCS2 is the following:

Tensorflow frozen network (.pb) —> IR (Intel’s format). Then IR format can be executed on the compute stick 2.

Keras/Tensorflow model to Frozen Graph (.pb)

I had described how to do this in an earlier post when I tried to setup inference with Nvidia’s TX2.

Frozen Graph (.PB) to Intel IR Format

Use the utility (from intel) mo_tf.py. This is available in the openvino bin. Setup bash vars for openVINO and they mo_tf.py should be in path

# Before you run any examples, remember to setup the environment: 
$ source ~/intel/openvino/bin/setupvars.sh

# Assume in the current folder there is a .pb (tensorflow frozen file)
$ mo_tf.py --input_model mobilenet_v1_1.0_224_frozen.pb
Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/home/dji/Downloads/my_try_NCS2/mobilenet_v1_1.0_224_frozen.pb
	- Path for generated IR: 	/home/dji/Downloads/my_try_NCS2/.
	- IR output name: 	mobilenet_v1_1.0_224_frozen
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	False
TensorFlow specific parameters:
	- Input model in text protobuf format: 	False
	- Path to model dump for TensorBoard: 	None
	- List of shared libraries with TensorFlow custom layers implementation: 	None
	- Update the configuration file with input/output node names: 	None
	- Use configuration file used to generate the model with Object Detection API: 	None
	- Operations to offload: 	None
	- Patterns to offload: 	None
	- Use the config file: 	None
Model Optimizer version: 	2019.3.0-375-g332562022

[ SUCCESS ] Generated IR model.
[ SUCCESS ] XML file: /home/dji/Downloads/my_try_NCS2/./mobilenet_v1_1.0_224_frozen.xml
[ SUCCESS ] BIN file: /home/dji/Downloads/my_try_NCS2/./mobilenet_v1_1.0_224_frozen.bin
[ SUCCESS ] Total execution time: 7.19 seconds. 

Execute on NCS2

There is a Python as well as C++ API for openVINO inference engine. Look at an example in the ncappzoo demo. The API Reference is available here: https://docs.openvinotoolkit.org/latest/ie_python_api.html

try:
    from openvino.inference_engine import IENetwork, IECore
except:
    print('\nPlease make sure your OpenVINO environment variables are set by sourcing the `setupvars.sh` script found in <your OpenVINO install location>/bin/ folder.\n')
    exit(1)


## You need the 'xml' and 'bin' file (IR Format)
ir = './output_model.xml'
ie = IECore()
net = IENetwork(model = ir, weights = ir[:-3] + 'bin')

## Input and Output
input_blob = next(iter(net.inputs))
output_blob = next(iter(net.outputs))
input_shape = net.inputs[input_blob].shape
output_shape = net.outputs[output_blob].shape

## Load the network and get the network shape information
exec_net = ie.load_network(network = net, device_name ="MYRIAD" )

## Inference, uu is a numpy array of shape='input_shape'
res = exec_net.infer({input_blob: uu })
output_logits = res[output_blob][0]

MNIST Demo

I have created a standalone github repo to demo how to train a mnist model in keras and run it on Intel Compute Stick 2. See https://github.com/mpkuse/my_try_NCS2 .

Advertisement