Arduino IDE 1.18

Impulses can be deployed as an Arduino library. This packages all of your signal processing blocks, configuration and learning blocks up into a single package. You can include this package in your own sketches to run the impulse locally. In this tutorial, you'll export an impulse, and integrate the impulse in a sketch to classify sensor data.

This tutorial should work on most Arm-based Arduino development boards with at least 64K of RAM, like the Arduino Nano 33 BLE Sense, Arduino Portenta H7 + Vision shield and the Arduino Nicla Vision.

In October 2022, we also added support for ESP32 boards. It has been tested with the ESP-EYE and the ESP32-CAM AI Thinker

Prerequisites

Make sure you followed one of the following tutorials, and have a trained impulse:

• Recognizing sounds from audio.

• Responding to your voice.

• Adding sight to your sensors.

• Building a continuous motion recognition system (only for boards with built-in IMU sensor).

• FOMO: Object detection for constrained devices.

Also install the following software:

• Arduino IDE.

Boards manager

Make sure to install the right development board under Tools->Boards->Boards Manager. We officially support the Arduino Nano 33 BLE Sense, Arduino Portenta H7 + Vision shield, Arduino Nicla Vision, Arduino Nicla Sense ME and the Espressif ESP-EYE (ESP32).

• Arduino Nano 33 BLE Sense

  For the Arduino Nano 33 BLE Sense, install the following board:

  
• Arduino Portenta H7

  For the Arduino Portenta H7, make sure to install the Arduino Mbed OS Portenta Boards v2.8.0 and to select the Arduino Portenta H7 (M7 core) board and the Flash Split 2 MB M7 + M4 in SDRAM:

  

  
• Arduino Nicla boards

  For the Arduino Nicla Vision and the Arduino Nicla Sense ME, install the following board:

  
• Espressif ESP32

  For the ESP32 boards, we officially support the ESP-EYE. Other boards have been tested such as the ESP32-CAM AI Thinker. To install ESP32 boards, go to Arduino->Preferences and add the following link to the additional boards manager URLs: https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json. Then install the ESP32 boards from the board manager menu.

  

  

And make sure you can compile sketches for your development board.

Deploying your impulse

Head over to your Edge Impulse project, and go to Deployment. From here you can create the full library which contains the impulse and all external required libraries. Select Arduino library and click Build to create the library. Then download and extract the .zip file.

Then to add the library and open an example, open the Arduino IDE and:

1. Choose Sketch > Include Library > Add .ZIP library....

   
2. Find the folder (do not go inside the folder), and select Choose.

   
3. Then, load an example by going to File > Examples > Your project name - Edge Impulse > static_buffer.

   
4. Voila. You now have an example application that loads your impulse.

   

Running the impulse

With the project ready it's time to verify that the application works. Head back to the studio and click on Live classification. Then load a validation sample, and click on a row under 'Detailed result'.

To verify that the local application classifies the same, we need the raw features for this timestamp. To do so click on the 'Copy to clipboard' button next to 'Raw features'. This will copy the raw values from this validation file, before any signal processing or inferencing happened.

In the sketch paste the raw features inside the static const float features[] definition, for example:

static const float features[] = {
    -19.8800, -0.6900, 8.2300, -17.6600, -1.1300, 5.9700, ...
};

Then, select the port that your board is connected to in the Arduino IDE menu: Tools > Port > your Arduino board.

Then click Upload in the Arduino IDE to build and flash the application.

Seeing the output

To see the output of the impulse, open the serial monitor from the Arduino IDE via Tools > Serial monitor, and selecting baud rate 115,200.

This will run the signal processing pipeline, and then classify the output:

Edge Impulse standalone inferencing (Arduino)
Running neural network...
Predictions (time: 0 ms.):
idle:   0.015319
snake:  0.000444
updown: 0.006182
wave:   0.978056
Anomaly score (time: 0 ms.): 0.133557
run_classifier_returned: 0
[0.01532, 0.00044, 0.00618, 0.97806, 0.134]

Which matches the values we just saw in the studio. You now have your impulse running on your Arduino development board!

Connecting sensors?

A demonstration on how to plug sensor values into the classifier can be found here: Data forwarder - classifying data (Arduino).

Troubleshooting

Multiple libraries were found for ...

Exported libraries are automatically versioned, but it's possible that the Arduino IDE gets confused on which version to use leading to an error like: Multiple libraries were found for .... You can delete old versions of the libraries to mitigate this. The libraries are located at:

• Windows: My Documents > Arduino > libraries

• macOS: ~/Documents/Arduino/libraries/

• Linux: ~/sketchbook/libraries

error: reference to 'SerialUSB' is ambiguous

This error shows up on the Arduino Nano 33 BLE Sense when using the Arduino Core v1.1.6 for the target. We've filed a bug with Arduino (here) but until that is resolved you can revert the Arduino core version back to v1.1.4 to compile your sketch.

No such file or directory: include <arm_math.h>

~/Documents/Arduino/libraries/ei-accelerometer-impulse/src/edge-impulse-sdk/CMSIS/NN/Source/PoolingFunctions/arm_max_pool_s8.c:32:10: fatal error: arm_math.h: No such file or directory
 #include <arm_math.h>

Some users have indicated that this issue can be solved by reinstalling the Arduino IDE.

macro "min" passed 3 arguments, but takes just 2

If you're compiling on a SAMD21-based target, you'll see the above error. This is a known bug in the Arduino core for the SAMD21. If you apply the following patch the issue will go away.

error: 'va_start' was not declared in this scope

This error can be seen while compiling for a SAMD21-based target. You will need to add the standard library in your sketch:

#include <cstdarg>

Empty array when printing results

If the predictions are not properly printed, e.g.:

Edge Impulse standalone inferencing (Arduino)
run_classifier returned: 0
Predictions (DSP: 369 ms., Classification: 3 ms., Anomaly: 4 ms.): 
[, , , , ]

Then your printing library does not support printing floating point numbers (this happens for example on the Arduino MKR WAN 1300). You can get around this by converting the predictions to integers. E.g.:

    ei_printf("Predictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
        result.timing.dsp, result.timing.classification, result.timing.anomaly);

    // print the predictions
    ei_printf("[");
    for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
        ei_printf("%d", static_cast<int>(result.classification[ix].value * 100));
#if EI_CLASSIFIER_HAS_ANOMALY == 1
        ei_printf(", ");
#else
        if (ix != EI_CLASSIFIER_LABEL_COUNT - 1) {
            ei_printf(", ");
        }
#endif
    }
#if EI_CLASSIFIER_HAS_ANOMALY == 1
    ei_printf("%d", static_cast<int>(result.anomaly));
#endif
    ei_printf("]\n");

Slow DSP operations

Where possible the signal processing code utilizes the vector extensions on your platform, but these are not enabled on all platforms. If these are not enabled we fall back to a software implementation which is slower. We don't enable these on all platforms because the wide variety of platform and core versions Arduino supports, but you can see enable them for your platform by adding the following code on the first line of your sketch, before any includes (only works on Arm cores):

#define EIDSP_USE_CMSIS_DSP		        1
#define EIDSP_LOAD_CMSIS_DSP_SOURCES    1

If this still does not work you need to edit the src/edge-impulse-sdk/dsp/config.hpp file in the library and add, before #ifndef EIDSP_USE_CMSIS_DSP:

#define EIDSP_USE_CMSIS_DSP		        1
#define EIDSP_LOAD_CMSIS_DSP_SOURCES    1

If you have a target that ships on old versions of CMSIS Core (like the SAMD21 targets) you might need to also declare some new macros, e.g.:

#define __STATIC_FORCEINLINE                   __attribute__((always_inline)) static inline
#define __SSAT(ARG1, ARG2) \
__extension__ \
({                          \
  int32_t __RES, __ARG1 = (ARG1); \
  __ASM volatile ("ssat %0, %1, %2" : "=r" (__RES) :  "I" (ARG2), "r" (__ARG1) : "cc" ); \
  __RES; \
 })

Code compiling fails under Windows OS

fork/exec C:\Users\MYUSER\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++.exe: The filename or extension is too long.

This error is usually thrown when the list of object files to compile exceeds Windows max number of characters (32k) in a command line.

Note: This issue is reportedly fixed in Arduino CLI v0.14 and in the Arduino IDE 1.8.15 and above.

If updating your Arduino IDE does not work you can overcome this issue by downloading the platform.local.txt below for your target:

• mbed platform.local.txt. Copy this file under the Arduino mbed directory, i.e: C:\Users\MYUSER\AppData\Local\Arduino15\packages\arduino\hardware\mbed\1.1.4\ or C:\Users\MYUSER\AppData\Local\Arduino15\packages\arduino\hardware\mbed_nano\2.1.0\

• SAMD21 platform.local.txt. Copy this file under the Arduino SAMD directory, i.e: C:\Users\MYUSER\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.9\

• SAMD51 (Adafruit) platform.local.txt. Copy this file under the Arduino Adafruit SAMD directory, i.e: C:\Users\MYUSER\AppData\Local\Arduino15\packages\adafruit\hardware\samd\1.6.3\

• ESP32 platform.local.txt. Copy this file under the Arduino ESP32 directory, i.e: C:\Users\MYUSER\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\

• STM32 platform.local.txt. Copy this file under the Arduino STM32 directory, i.e: C:\Users\MYUSER\AppData\Local\Arduino15\packages\esp32\hardware\stm32\1.9.0\

Failed to connect to COM3 (Arduino Portenta H7)

[SER] Connecting to COM3
[SER] Failed to connect to COM3 retrying in 5 seconds Opening COM3: Access denied
[SER] You might need `sudo` or set up the right udev rules
[SER] Failed to connect to COM3 retrying in 5 seconds Opening COM3: Access denied
[SER] You might need `sudo` or set up the right udev rules
[SER] Failed to connect to COM3 retrying in 5 seconds Opening COM3: Access denied
[SER] You might need `sudo` or set up the right udev rules

Make sure the vision shield is present.

No DFU capable USB device available (Arduino Portenta H7)

arduino:mbed_portenta 2.6.1     2.6.1  Arduino Mbed OS Portenta Boards                                                  
Finding Arduino Mbed core OK
Finding Arduino Portenta H7...
Finding Arduino Portenta H7 OK at Arduino
dfu-util 0.10-dev

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2021 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Warning: Invalid DFU suffix signature
A valid DFU suffix will be required in a future dfu-util release
No DFU capable USB device available
Error during Upload: uploading error: uploading error: exit status 74
Flashing failed. Here are some options:
If your error is 'incorrect FQBN' you'll need to upgrade the Arduino core via:
     $ arduino-cli core update-index
     $ arduino-cli core install arduino:mbed_portenta@2.6.1
Otherwise, double tap the RESET button to load the bootloader and try again
Press any key to continue . . .

You need to put the board in its bootloader mode. Double press and the RESET button before flashing the board.