Tutorial: Streaming Object Detection on a Raspberry Pi 3 with the Intel Movidius Neural Compute Stick
Total build time: around 5 hours, most of it is waiting for the Raspberry Pi (RPi) to install packages.
Note: Intel/Movidius have released the Intel Movidius Neural Compute Stick 2 in the meantime, so note the difference.
If you don’t care about converting your own models and just want to run a pre-trained streaming object detector on an RPi, skip right to step 2 and 3. I won’t judge.
When a company comes along and tells you you no longer need to carry a Geforce GTX-1080Ti GPU PC with you to demo model inference for streaming object detection, you bite! Let’s give this Edge Computing thing a shot.
Some time ago (these writeups sure take a while) my colleagues at my employer and my client and myself have played around with the Intel Movidius Neural Compute Stick (NCS). When I say played around with, I mean mostly Googling and crying in a corner (ahh, the data science life). The main reason being t̵h̵e̵ ̵e̵m̵p̵t̵i̵n̵e̵s̵s̵ ̵i̵n̵ ̵m̵y̵ ̵s̵o̵u̵l̵ that the information and documentation were…not in any particularly concentrated place. There was not one single source I could follow to get it to work start to finish for my not-so-specific case (Macbook + RPi + NCS). When even the AI Vlogmeister himself can’t provide answers, you know it’s serious. So I figured: might as well aggregate everything so anyone can follow it and hopefully save some headaches. Even if you don’t know the first thing about neural networks and deep learning but are semi-handy with computers, you can get it to work!
Disclaimer though: these steps worked for me and my colleagues after lots of trial and error…but if they don’t work for you, well at least I got you up to speed.
As for my specific use case: lots of data scientists, AI researchers and machine learning engineers (or whatever term currently ranks higher on Gartner’s Hype Cycle) carry around Macbooks… So imagine my disappointment when I found out that the NCS development kit (NCSDK) given by Intel only works on Ubuntu or Raspbian Stretch. Of course, you can launch a cloud instance, or dual boot, or switch to a full Ubuntu laptop. But before you do something so drastic, why not work with what you have and just launch a Virtual Machine as Apple+Virtualbox intended?
So here are the three main steps:
- Install the full NCSDK on an Ubuntu VM, running on a Mac host. (needed to create model graphs, why no one hosts the most-used NCS graphs remains a mystery)
- Install the API-mode of NCSDK on a RPi. (meaning it only does existing model inference, no training of new models)
- Configure streaming object detection on the RPi.
For this we’ll need:
- a Mac running High Sierra OSX (make sure it has a USB3 port or if you’re stuck with USB-C ports, like me: use an adapter cable with USB3 port). I have not tried NCSDK with Mojave yet.
- a Raspberry Pi Model 3 B (there are 2 versions, for our demo the a22082 seems to run more stably)
- (optionally: a fancy case, fan and bluetooth keyboard+mouse for the RPi)
- a micro SD card with at least 32 GB (to install Stretch on the RPi)
- a good old USB webcam
- and the Neural Compute Stick itself of course.
Step 1: Install NCSDK on Ubuntu VM on a Mac host
Important: choose one and the same version of the NCSDK. At the time of writing there are 2 versions. By default ncsdk from github gives the 1.x version, choosing ncsdk2 requires manual input. For this tutorial we’ll choose v1. Why, you ask? Because it’s the first one we got to work.
Also important: running the NCSDK on a RPi means TensorFlow is not supported, only caffe models (PyTorch anyone?). If you want to test your luck with tensorflow, go for the 2.X install that builds a nightly tensorflow 1.16 for your RPi.
We’ll start by installing an Ubuntu VM on a Virtualbox on you Mac. This allows you to essentially run a different OS inside your current one. For this there are a couple of steps:
- Download and Install the latest virtual box.
- Download and Install the latest virtual box extension pack.
- Download the Ubuntu 16.04 LTS ISO file and install Ubuntu on Virtualbox. 18.04 broke it for us, so try that one at your own risk. Open Virtualbox and follow the GUI to add a new machine.
- Upgrade the Ubuntu VM via terminal using ‘sudo apt-get upgrade’.
- Run these commands in a terminal window on your Ubuntu VM.
mkdir -p ~/workspace
cd ~/workspace
git clone https://github.com/movidius/ncsdk.git
cd ~/workspace/ncsdk
make install - Setup USB Settings for the NCS stick to be attached to the VM
USB2 Device Filter Details ==> Vendor ID == “03e7”
USB3 Device Filter Details ==> Vendor ID == “040e” - Attach the Neural Compute Stick to your computer and the Ubuntu VM should see it. Verify using ‘lsusb’ in terminal to see if you see a usb device with Vendor ID == 03e7.
- The USB controller setting should be set to USB3.0(XHCI). Make sure you turn off the VM before editing the USB devices and set it to the 3.0 controller.
- If you followed step 8, when plugging in the NCS and clicking on ‘Devices’ ==> ‘USB’ in the Virtualbox menu when you have the Ubuntu VM running, it will find a mysterious loopback device in your USB devices list. Click and add every single device that magically appeared to make them available in the Ubuntu VM (for me there were 3 of them).
- Reboot the VM after adding these 3 usb devices. Important one to overlook.
- Now try ‘make examples’ in ncsdk workspace folder. Ironically, this is just step 3 from the official Movidius start tutorial. It gives me flashbacks to everytime in university when math+stats books the authors said “It’s easy to see that…” and skipped the derivation.
- If it runs: success! You are now able to convert all the object detection models you’ve trained in Caffe to the native NCS format. If it doesn’t run…well in the meantime they’ve released NCS 2 sticks, which might work easier.
Step 2 and 3: running real-time object detection models on an RPi
Lucky you! Since I first set out to writing this, my colleague at my current client who has way better developer skills bundled all these steps I was typing out for you into a smoothly working github project. Check it out and we’ll look forward to your pull requests!
The result
It’s pretty cool. Not the most stable thing, but it runs at around 15 fps for a limited number of object classes. Which is less than the 25 fps from a 1080Ti desktop beast running something like Darknet’s YoloV3. But it’s quite respectable given that it opens up a plethora of new business cases with it’s small and lightweight setup. (Face detecting drones, anyone?) In less than a year, our setup at the Dutch Big Data Expo went from the full desktop GPU you saw at the beginning, to this byte-sized (pun intended) configuration:
List of used sources (every single one of these has been essential in getting it to work, hence my urge to aggregate the information):
Pretty sure this is an important face since he shows up three times.
My first ever Medium blog/tutorial, let me know whether it helped you in any way or if you found shortcuts in the steps we took :)
