1 of 12

C++ library

The provided methods package all your signal processing blocks, configuration and learning blocks up into a single package. You can include this package in your own application to run the impulse locally.

C++ Libraries

Impulses can be deployed as a C++ library. The library does not have any external dependencies and can be built with any C++11 compiler, see Running your impulse as a C++ library.

We have end-to-end guides for:

Running your impulse on your desktop
Running your impulse on Zephyr on a Nordic semiconductor development board
Running your impulse in Simplicity Studio on the TB Sense 2
Running your impulse on STM32 using STM32Cube.MX
Running your impulse on the Himax WE-I Plus
Running your impulse on the Espressif ESP-EYE (ESP32)
Running your impulse on the Raspberry Pi RP2040
Running your impulse on the Sony Spresense
Running your impulse on the Syntiant TinyML Board
Running your impulse on the TI LaunchPad using GCC and the SimpleLink SDK

We also have tutorials for:

Using Arduino IDE

Running your impulse in an Arduino sketch

Using OpenMV IDE

Running your impulse on the OpenMV Cam H7 Plus

On Linux-based devices

Running your impulse on a Linux system with our C++, Node.js, Python or Go SDKs.

Using the DRP-AI library

Running your impulse on the Renesas RZ/V2L

Using WebAssembly

These tutorials show you how to run your impulse, but you'll need to hook in your sensor data yourself. We have a number of examples on how to do that in the Data forwarder documentation, or you can use the full firmware for any of the fully supported development boards as a starting point - they have everything (including sensor integration) already hooked up. Or keep reading for documentation about the sensor format and inputs that we expect.

Did you know?

You can build binaries for supported development boards straight from the studio. These will include your full impulse. See Edge Impulse Firmwares

Input to the run_classifier function

The input to the run_classifier function is always a signal_t structure with raw sensor values. This structure has two properties:

total_length - the total number of values. This should be equal to EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (from model_metadata.h). E.g. if you have 3 sensor axes, 100Hz sensor data, and 2 seconds of data this should be 600.
get_data - a function that retrieves slices of data required by the DSP process. This is used in some DSP algorithms (like all audio-based ones) to page in the required data, and thus saves memory. Using this function you can store (f.e.) the raw data in flash or external RAM, and page it in when required.

F.e. this is how you would page in data from flash:

// this is placed in flash
const float features[300] = { 0 };

// function that pages the data in
int raw_feature_get_data(size_t offset, size_t length, float *out_ptr) {
    memcpy(out_ptr, features + offset, length * sizeof(float));
    return 0;
}

int main() {
    // construct the signal
    signal_t signal;
    signal.total_length = 300;
    signal.get_data = &raw_feature_get_data;
    // ... rest of the application

If you have your data already in RAM you can use the signal_from_buffer function to construct the signal:

float features[30] = { 0 };
signal_t signal;
numpy::signal_from_buffer(features, 30, &signal);
// ... rest of the application

The get_data function expects floats to be returned, but you can use the int8_to_float and int16_to_float helper functions if your own buffers are int8_t or int16_t (useful to save memory). E.g.:

const int16_t features[300] = { 0 };

int raw_feature_get_data(size_t offset, size_t length, float *out_ptr) {
    return numpy::int16_to_float(features + offset, out_ptr, length);
}

int main() {
    signal_t signal;
    signal.total_length = 300;
    signal.get_data = &raw_feature_get_data;
    // ... rest of the application

Signal layout for time-series data

Signals are always a flat buffer, so if you have multiple sensor data you'll need to flatten it. E.g. for sensor data with three axes:

Input data:
Axis 1:  9.8,  9.7,  9.6
Axis 2:  0.3,  0.4,  0.5
Axis 3: -4.5, -4.6, -4.8

Signal: 9.8, 0.3, -4.5, 9.7, 0.4, -4.6, 9.6, 0.5, -4.8

Signal layout for image data

The signal for image data is also flattened, starting with row 1, then row 2 etc. And every pixel is a single value in HEX format (RRGGBB). E.g.:

Input data (3x2 pixel image):
BLACK RED  RED
GREEN BLUE WHITE

Signal: 0x000000, 0xFF0000, 0xFF0000, 0x00FF00, 0x0000FF, 0xFFFFFF

We do have an end-to-end example on constructing a signal from a frame buffer in RGB565 format, which is easily adaptable to other image formats, see: example-signal-from-rgb565-frame-buffer.

Directly quantize image data

If you're doing image classification and have a quantized model, the data is automatically quantized when reading the data from the signal to save memory. This is automatically enabled when you call run_impulse. To control the size of the buffer that's used to read from the signal in this case you can set the EI_DSP_IMAGE_BUFFER_STATIC_SIZE macro (which also allocates the buffer statically).

Static allocation

To statically allocate the neural network model, set this macro:

EI_CLASSIFIER_ALLOCATION_STATIC=1

You can easily control where the tensor arena is allocated by defining the EI_TENSOR_ARENA_LOCATION macro, specifying .where_to_allocate. This is particularly useful for large size requirements and when the target has external RAM:

For example:

EI_TENSOR_LOCATION="<.where_to_allocate>" - Here, <.where_to_allocate> can be a memory region such as ".sram," depending on your target's linker file.

Additionally we support full static allocation for quantized image models. To do so set this macro:

EI_DSP_IMAGE_BUFFER_STATIC_SIZE=1024

Static allocation is not supported for other DSP blocks at the moment.

As a generic C++ library

While Edge Impulse supports a number of boards that make gathering data and deploying your model easy, we know that many people will want to run edge machine learning on their own board. This tutorial will show you how to export an impulse and how to include it as a C++ library.

An impulse is a combination of any preprocessing code necessary to extract features from your raw data along with inference using your trained machine learning model. You provide the library with raw data from your sensor, and it will return the output of your model. It performs feature extraction and inference, just as you configured in the Edge Impulse Studio!

We recommend working through the steps in this guide to see how to run an impulse on a full operating system (macOS, Linux, or Windows) first. Once you understand how to include the impulse as a C++ library, you can port it to any build system or integrated development environment (IDE) you wish.

Knowledge required

This guide assumes you have some familiarity with C and the GNU Make build system. We will demonstrate how to run an impulse (e.g. inference) on Linux, macOS, or Windows using a C program and Make. We want to give you a starting point for porting the C++ library to your own build system.

details the available macros, structs, variables, and functions for the C++ SDK library.

A working demonstration of this project can be found .

Prerequisites

You will need a C compiler, a C++ compiler, and Make installed on your computer.

Linux

Install , , and . If you are using a Debian-based system, this can be done with:

macOS

Install and . If you are using , you can run the following commands:

Windows

Download the C++ Library from Edge Impulse

Head to the Deployment page for your project. Select C++ library. Scroll down, and click Build. Note that you must have a fully trained model in order to download any of the deployment options.

Your impulse will download as a C++ library in a .zip file.

Create a Project

The easiest way to test the impulse library is to use raw features from one of your test set samples. When you run your program, it should print out the class probabilities that match those of the test sample in the Studio.

Create a directory to hold your project (e.g. my-motion). Unzip the C++ library file into the project directory. Your directory structure should look like the following:

Explanation of C++ Library

total_length - total number of values, which should be equal to EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (from model-parameters/model_metadata.h). For example, if you have an accelerometer with 3 axes sampling at 100 Hz for 2 seconds, total_length would be 600.

model_variables.h holds some additional information about the model and preprocessing steps. Most importantly, you might want ei_classifier_inferencing_categories[] if you need the labels for your categories in string form.

tflite-model/ contains the actual trained model stored in an array. You should not need to access any of the variables or functions in these files, as inference is handled by the impulse library.

Signal Structure

Time-series data with multiple axes are flattened so that the value from each axis is listed from each time step before moving on to the next time step. For example, here is how sensor data with 3 axes would be flattened:

Image data is flattened by listing row 1, row 2, etc. Each pixel is given in HEX format (0xRRGGBB). For example:

Static Allocation

By default, the trained model resides mostly in ROM and is only pulled into RAM as needed. You can force a static allocation of the model by defining:

If you are doing image classification with a quantized model, the data is automatically quantized when read from the signal. This is automatically enabled when you call run_impulse. If you want to adjust the size of the buffer that is used to read from the signal in this case, you can set EI_DSP_IMAGE_BUFFER_STATIC_SIZE, which also allocates the buffer statically. For example, you might set:

Create an Application

Open main.cpp in your editor of choice. Paste in the following code:

We’re going to copy raw features from one of our test samples. This process allows us to test that preprocessing and inference works without needing to connect a real sensor to our computer or board.

Go back to your project in the Edge Impulse Studio. Click on Model testing. Find a sample (I’ll use one of the samples labeled “wave”), click the three dots (kebab menu) next to the sample, and click Show classification.

A new tab will open, and you can see a visualization of the sample along with the raw features, expected outcome (ground truth label), and inference results. Feel free to slide the window to any point in the test sample to get the raw features from that window. The raw features are the actual values that are sent to the impulse for preprocessing and inference.

I’ll leave the window at the front of the sample for this example. Click the Copy features button next to the Raw features. This will copy only the raw features under the given window to your clipboard. Additionally, make a note of the highlighted Detailed result. We will want to compare our local inference output to these values (e.g. wave should be close to 0.99 and the other labels should be close to 0.0). Some rounding error is expected.

Paste the list of raw feature values into the input_buf array. Note that this buffer is constant for this particular program. However, it demonstrates how you can fill an array with floating point values from a sensor to pass to the impulse SDK library.

For performing inference live, you would want to fill the features[] array with values from a connected sensor.

Save your main.cpp.

Explanation of Main Application

Before moving on to the Makefile, let’s take a look at the important code sections in our application.

To use the C++ library, we really only need to include one header file to use the impulse SDK:

The ei_run_classifier.h file includes any other files we might need from the library and gives us access to the necessary functions.

signal.total_length is the number of array elements in the input buffer. For our case, it should match the expected total number of elements (EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE).

signal.get_data must be set to a callback function. run_classifier() will use this callback function to grab data from our buffer as needed. It is up to you to create this function. Let’s take a look at the simplest form of this callback:

This function copies data from our input buffer (a static global array) plus a memory offset into an array provided by the caller. We don’t know exactly what offset and length will be for any given call, but we must be ready with valid data. We do know that this function will not attempt to index beyond the provided signal.total_length amount.

The callback structure is used here so that data can be paged in from any location (e.g. RAM or ROM), which means we don't necessarily need to save the entire sample in RAM. This process helps save precious RAM space on resource-constrained devices.

With our signal_t struct configured, we can call our inference function:

We print out the time it took (in milliseconds) to perform preprocessing (“dsp”), classification, and any anomaly detection we had enabled:

Finally, we print inference results to the console:

We can access the individual classification results for each class with result.classification[i].value where i is the index of our label. Labels are stored in alphabetical order in ei_classifier_inferencing_categories[]. Each prediction value must be between 0.0 and 1.0. Additionally, thanks to the softmax function at the end of our neural network, all of the predictions should add up to 1.0.

Functions That Require Definition

Throughout the library, you will find these functions being called. However, no definitions are provided because every platform is different in how these functions are implemented. For example, you may want to print debugging information to a console (stdout) or over a UART serial port.

If you were to try to build this project for another platform (e.g. a microcontroller), the process would fail, as you are missing these definitions. If your platform is supported by the Edge Impulse C++ Inference SDK, you may include that folder in your C++ sources. A Makefile example of including support for TI implementations might be:

If your platform is not supported or you would like to create custom definitions, you may do so in your own code. The following functions must be defined for your platform (the reference guide linked to by each function provides several examples on possible implementations):

Create a Makefile

Due to the number of files we must include from the library, it can be quite difficult to call the compiler and linker manually. As a result, we will use a Makefile script and the Make tool to compile all the necessary source code, link the object files, and generate a single executable file for us.

Copy the following into your Makefile:

Save your Makefile. Ensure that it is in the top level directory (for this particular project).

This Makefile should serve as an example of how to import and compile the impulse SDK library. The particular build system or IDE for your platform may not use Make, so I recommend reading the next section to see what files and flags must be included. You can use this information to configure your own build system.

Explanation of the Makefile

Near the top, we define where the compiler(s) can find the necessary header files:

We need to point this -I flag to the directory that holds edge-impulse-sdk/, model-parameters/, and tflite-model/ so that the build system can find the required header files. If you unzipped your C++ library into a lib/ folder, for example, this flag should be -Ilib/.

We then define a number of compiler flags that are set by both the C and the C++ compiler. What each of these do has been commented in the script:

In addition to including the header files, we also need to tell the compiler(s) where to find source code. To do that, we create separate lists of all the .c, .cpp, and .cc files:

Note the directory locations given in these lists. Many IDEs will ask you for the location of source files to include in the build process. You will want to include these directories (such as edge-impulse-sdk/CMSIS/DSP/Source/TransformFunctions/, etc.).

If you unzipped the C++ library into a different location (e.g. into a separate lib/ directory), then all of these source locations should be updated to reflect that. For example, tflite-model/*.cpp would become lib/tflite-model/*.cpp.

While TFLM is a great generic package for many target platforms, it is not as efficient as TFLite for some, such as Linux and Android. As a result, you will likely see a performance boost if you use TFLite (instead of TFLM) on Linux.

The rest of the Makefile compiles each of the source files to object files (.o) before combining and linking them into a standalone executable file. This particular Makefile places the executable (app) in the build/ directory.

Build and Run

At this point, you’re ready to build your application and run it! Open a terminal (MinGW Shell, if you’re on Windows), navigate to your project directory, and run the make command. You can use the -j [jobs] command to have Make use multiple threads to speed up the build process (especially if you have multiple cores in your CPU):

This may take a few minutes, so be patient. When the build process is done, run your application:

Note that this may be build/app.exe on Windows.

Take a look at the output predictions–they should match the predictions we saw earlier in the Edge Impulse Studio!

Going Further

This guide should hopefully act as a starting point to use your trained machine learning models on nearly any platform (as long as you have access to C and C++ compilers).

On your desktop computer

Impulses can be deployed as a C++ library. This packages all your signal processing blocks, configuration and learning blocks up into a single package. You can include this package in your own application to run the impulse locally. In this tutorial you'll export an impulse, and build a desktop application to classify sensor data.

Even though this is a C++ library you can link to it from C applications. See 'Using the library from C' below.

Knowledge required

This tutorial assumes that you know how to build C++ applications, and works on macOS, Linux and Windows. If you're unfamiliar with these tools you can build binaries directly for your development board from the Deployment page in the studio.

Note: This tutorial provides the instructions necessary to build the C++ SDK library locally on your desktop. If you would like a full explanation of the Makefile and how to use the library, please see the deploy your model as a C++ library tutorial.

Looking for examples that integrate with sensors? See the Edge Impulse C++ SDK for Linux.

Prerequisites

Make sure you followed the Continuous motion recognition tutorial, and have a trained impulse. Also install the following software:

macOS, Linux

GNU Make - to build the application. make should be in your PATH.
A modern C++ compiler. The default LLVM version on macOS works, but on Linux upgrade to LLVM 9 (installation instructions).

Windows

MinGW-W64 which includes both GNU Make and a compiler. Make sure mingw32-make is in your PATH.

Cloning the base repository

We created an example repository which contains a Makefile and a small CLI example application, which takes the raw features as an argument, and prints out the final classification. Clone or download this repository at example-standalone-inferencing.

Deploying your impulse

Head over to your Edge Impulse project, and go to Deployment. From here you can create the full library which contains the impulse and all external required libraries. Select C++ library, and click Build to create the library.

Download the .zip file and place the contents in the 'example-standalone-inferencing' folder (which you downloaded above). Your final folder structure should look like this:

example-standalone-inferencing
|_ build.bat
|_ build.sh
|_ CMakeLists.txt
|_ edge-impulse-sdk/
|_ LICENSE
|_ Makefile
|_ model-parameters/
|_ README.md
|_ README.txt
|_ source/
|_ tflite-model/

Add data sample to main.cpp

To get inference to work, we need to add raw data from one of our samples to main.cpp. Head back to the studio and click on Live classification. Then load a validation sample, and click on a row under 'Detailed result'. Make a note of the classification results, as we want our local application to produce the same numbers from inference.

To verify that the local application classifies the same, we need the raw features for this timestamp. To do so click on the 'Copy to clipboard' button next to 'Raw features'. This will copy the raw values from this validation file, before any signal processing or inferencing happened.

Open source/main.cpp in an editor of your choice. Find the following line:

// Raw features copied from test sample
static const float features[] = {
    // Copy raw features here (e.g. from the 'Model testing' page)
};

Paste in your raw sample data where you see // Copy raw features here:

// Raw features copied from test sample
static const float features[] = {
    -19.8800, -0.6900, 8.2300, -17.6600, -1.1300, 5.9700, ...
};

Note: the raw features will likely be longer than what I listed here (the ... won't compile--I just wanted to demonstrate where the features would go).

In a real application, you would want to make the features[] buffer non-const. You would fill it with samples from your sensor(s) and call run_classifier() or run_classifier_continuous(). See deploy your model as a C++ library tutorial for more information.

Save and exit.

Running the impulse

Open a terminal or command prompt, and build the project:

macOS, Linux

$ sh build.sh

Windows

$ build.bat

This will first build the inferencing engine, and then build the complete application. After building succeeded you should have a binary in the build/ directory.

Then invoke the local application by calling the binary name:

macOS, Linux

./build/app

Windows

build\app

This will run the signal processing pipeline using the values you provided in the features[] buffer and then give you the classification output:

run_classifier_returned: 0
Timing: DSP 0 ms, inference 0 ms, anomaly 0 ms
Predictions (time: 0 ms.):
  idle:   0.015319
  snake:  0.000444
  updown: 0.006182
  wave:   0.978056
Anomaly score (time: 0 ms.): 0.133557

Which matches the values we just saw in the studio. You now have your impulse running locally!

Using the library from C

Even though the impulse is deployed as a C++ application, you can link to it from C applications. This is done by compiling the impulse as a shared library with the EIDSP_SIGNAL_C_FN_POINTER=1 and EI_C_LINKAGE=1 macros, then link to it from a C application. The run_classifier can then be invoked from your application. An end-to-end application that demonstrates this and can be used with this tutorial is under example-standalone-inferencing-c.

On your Zephyr-based Nordic Semiconductor development board

A working Zephyr RTOS build environment is required

This tutorial assumes that you're already familiar with building applications for the nRF52840DK or other Zephyr RTOS supported board, and that you have your environment set up to compile applications for this platform. For this tutorial, you can use the nRF Connect SDK v1.6.0 or higher.

Prerequisites

Make sure you followed the Continuous motion recognition tutorial, and have a trained impulse. Also, make sure you have a working Zephyr build environment, including the following tools:

Either the nRF Connect SDK which includes Zephyr and all its dependencies (v1.6.0 or higher), or a manual installation of the Zephyr build environment.
The GNU ARM Embedded Toolchain (version 9-2019-q4-major).
Optional: The nRF comand line tools and Segger J-Link tools. These command line tools are required if you use the west command line interface to upload firmware to your target board.

Cloning the base repository

We created an example repository which contains a small application that compliments the Continuous motion recognition tutorial. This application can take raw, hard-coded inputs as an argument, and print out the final classification to the serial port so it can be read from your development computer. You can either download the application or import the repository using Git:

git clone https://github.com/edgeimpulse/example-standalone-inferencing-zephyr.git

Fully featured open source repos are also available

If you are looking for sample projects showcasing all sensors and features supported by Edge Impulse out of the box, we have public firmware repos available for the Nordic Semiconductor nRF52840, nRF5340 and nRF9160 development kits as well as for the Thingy:91 and Thingy:53. See

Deploying your impulse

Head over to your Edge Impulse project, and go to the Deployment page. From here you can obtain a packaged library containing the Edge Impulse C++ SDK, your impulse, and all required external dependencies. Select C++ library and click Build to create the library.

Download the .zip file and extract the contents. Now copy the following directories to the 'example-standalone-inferencing-zephyr' folder (which you downloaded above).

edge-impulse-sdk
model-parameters
tflite-model

Your final folder structure should look like this:

 example-standalone-inferencing-zephyr
 ├── CMakeLists.txt
 ├── edge-impulse-sdk
 ├── model-parameters
 ├── prj.conf
 ├── README.md
 ├── sample.yaml
 ├── src
 ├── tflite-model
 └── utils

Running the impulse

With the project ready it's time to verify that the application works. Head back to the studio and click on Live classification in the project you created for the continuous motion recognition tutorial, then load a testing sample, and click on a row under 'Detailed result'.

To verify that the Zephyr application performs the same classification when running locally on your board, we need to use the same raw inputs as those provided to the Live classification for any given timestamp. To do so, click on the 'Copy to clipboard' button next to 'Raw features'. This will copy the raw values from this validation file, before any signal processing or inferencing happened.

Next, open src/main.cpp in the example directory and paste the raw features inside the static const float features[] definition. For example:

static const float features[] = {
    -19.8800, -0.6900, 8.2300, -17.6600, -1.1300, 5.9700, ...
};

And use west or your usual method to build the application:

# nRF52840 DK
$ west build -b nrf52840dk_nrf52840

# nRF5340 DK
$ west build -b nrf5340dk_nrf5340_cpuapp

# nRF9160DK
$ west build -b nrf9160dk_nrf9160ns

# Thingy:91
$ west build -b thingy91_nrf9160

Invalid choice: 'build'

If you try to build the application but it throws an 'invalid choice' error like:

$ west build -b nrf52840dk_nrf52840
usage: west [-h] [-z ZEPHYR_BASE] [-v] [-V] <command> ...
west: error: argument <command>: invalid choice: 'build' (choose from 'init', 'update', 'list', 'manifest', 'diff', 'status', 'forall', 'help', 'config', 'topdir', 'selfupdate')

You'll need to set up your environment variables correctly (more info). You can do so by opening a command prompt or terminal window and running the commands below from the zephyr parent directory: On Windows

$ zephyr\zephyr-env.cmd

On macOS / Linux

$ source zephyr/zephyr-env.sh

If you have set up the Segger J-LINK tools and the board that comes with J-LINK debug probe, you can also flash this application with:

$ west flash

otherwise if your board shows up as a mass storage device, you can find the build/zephyr/zephyr.bin file and drag it to the JLINK USB mass-storage device in the same way you do with a USB flash drive.

For the nRF9160DK, you also have to make sure the board controller has been flashed at least once.

Boards such as Thingy:91 and Thingy:53 do not come with in built J-LINK debug probe, and cannot be used with west flash directly. They do include connector that enable users to connect with external J-LINK debug probe and take advantage of west flash command.

Seeing the output

To see the output of the impulse, connect to the development board over a serial port on baud rate 115,200 and reset the board. You can do this with your favourite serial monitor or with the Edge Impulse CLI:

$ edge-impulse-run-impulse --raw

This will show you the output of the signal processing pipeline and the results of the classification:

Edge Impulse standalone inferencing (Zephyr)
Running neural network...
Predictions (time: 1 ms.):
idle:   0.015319
snake:  0.000444
updown: 0.006182
wave:   0.978056
Anomaly score (time: 0 ms.): 0.133557
Predictions (DSP: 18 ms., Classification: 1 ms., Anomaly: 0 ms.): 
[0.01532, 0.00044, 0.00618, 0.97806, 0.134]

The output should match the values you just saw in the studio. If it does, you now have your impulse running on your Zephyr development board!

Connecting live sensors?

Now that you have verified that the impulse works with hard-coded inputs, you should be ready to plug live sensors from your board. A demonstration on how to plug sensor values into the classifier can be found here: Data forwarder - classifying data (Zephyr).

On your Himax WE-I Plus

Knowledge required

This tutorial assumes that you're familiar with building applications for the Himax WE-I Plus. If you're unfamiliar with either of these you build binaries directly for your development board from the Deployment page in the studio.

Note: Are you looking for an example that has all sensors included? The Edge Impulse firmware for the Himax WE-I Plus has that. See edgeimpulse/firmware-himax-we-i-plus.

Prerequisites

Make sure you've followed one of the tutorials and have a trained impulse. Also install the following software:

Edge Impulse CLI - to flash the firmware.
A build toolchain, either:
- Docker desktop.
- Or, the GNU Toolchain for DesignWare ARC processors, and make sure you have arc-elf32-gcc in your PATH (Linux only).
- Or, the DesignWare ARC MetaWare Toolkit - including a valid license, and make sure you have ccac in your PATH.

If you're building with the GNU or DesignWare toolchains, also install:

CMake.
GNU Make.

Cloning the base repository

We created an example repository which contains a small application for the Himax WE-I Plus, which takes the raw features as an argument, and prints out the final classification. Download the application here, or import this repository using Git:

$ git clone https://github.com/edgeimpulse/example-standalone-inferencing-himax

Deploying your impulse

Download the .zip file and extract the directories in the 'example-standalone-inferencing-himax' folder. Make sure to not replace CMakeLists.txt in this folder. Your final folder structure should look like this:

example-standalone-inferencing-himax
|_ arc_mli_package
|_ edge-impulse-sdk
|_ image_gen_linux_v3
|_ model-parameters
|_ tflite-model
|_ main.cc

Running the impulse

With the project ready it's time to verify that the application works. Head back to the studio and click on Live classification. Then load a validation sample, and click on a row under 'Detailed result'.

Open main.cc and paste the raw features inside the static const float features[] definition, for example:

static const float features[] = {
    -19.8800, -0.6900, 8.2300, -17.6600, -1.1300, 5.9700, ...
};

Then build and flash the application to your development board:

Building the application (Docker)

Build the container:

$ docker build -t himax-build-gnu -f Dockerfile.gnu .

Then set up your build environment:

$ mkdir -p build-gnu
$ docker run --rm -it -v $PWD:/app himax-build-gnu /bin/bash -c "cd build-gnu && cmake -DCMAKE_TOOLCHAIN_FILE=toolchain.gnu.cmake .."

And build and link the application:

$ docker run --rm -it -v $PWD:/app:delegated himax-build-gnu /bin/bash -c "cd build-gnu && make -j && sh ../make-image.sh GNU"

There are instructions in the README.md file on how to build with the Metaware toolkit under Docker.

Building the application (Metaware Toolkit)

Create a build directory and initialize CMake:

$ mkdir build-mw
$ cd build-mw
$ cmake -DCMAKE_TOOLCHAIN_FILE=toolchain.metaware.cmake ..

Build and link the application:
```
$ make -j
$ sh ../make-image.sh MW
```

Building the application (GNU)

Create a build directory and initialize CMake:

$ mkdir build-gnu
$ cd build-gnu
$ cmake -DCMAKE_TOOLCHAIN_FILE=toolchain.gnu.cmake ..

Build and link the application:
```
$ make -j
$ sh ../make-image.sh GNU
```

Flashing

You'll need the Edge Impulse CLI v1.10 or higher. Then flash the binary with:

$ himax-flash-tool --firmware-path image_gen_linux/out.img

Seeing the output

$ edge-impulse-run-impulse --raw

This will run the signal processing pipeline, and then classify the output:

Edge Impulse standalone inferencing (Himax)
Running neural network...
Predictions (time: 0 ms.):
idle:   0.015319
snake:  0.000444
updown: 0.006182
wave:   0.978056
Anomaly score (time: 0 ms.): 0.133557
run_classifier_returned: 0
[0.01532, 0.00044, 0.00618, 0.97806, 0.134]

Which matches the values we just saw in the studio. You now have your impulse running on your Himax WE-I Plus development board!

On your Raspberry Pi Pico (RP2040) development board

Knowledge required

This tutorial assumes that you're familiar with building applications with C/C++ Pico-SDK for Raspberry Pi Pico (RP2040). If you're unfamiliar with either of these you build binaries directly for your development board from the Deployment page in the studio.

Note: Are you looking for an example that has all sensors included? The Edge Impulse firmware for Raspberry Pi Pico (RP2040) has that. See .

Prerequisites

Make sure you've followed one of the tutorials and have a trained impulse. For the purpose of this tutorial, we’ll assume you trained a model. Also install the following software:

The below instructions assume you are using Debian-based Linux distribution. Alternative instructions for those using Microsoft Windows or Apple macOS are provided in the (Sections 9.1 and 9.2).

To build the project, you will need the pico-sdk, CMake, a cross-platform tool used to build the software, and the GNU Embedded Toolchain for Arm. In Debian-based OS, you can install both these via apt from the command line.

Note: Ubuntu and Debian users might additionally need to also install libstdc++-arm-none-eabi-newlib.

You'll need the PICO SDK to compile the firmware. You can obtain it from https://github.com/raspberrypi/pico-sdk and then specify PICO_SDK_PATH environmental variable, that would point to exact PICO SDK location on your system. E.g.

Cloning the base repository

We created an example repository which contains a small application for Raspberry Pi Pico (RP2040), which takes the raw features as an argument, and prints out the final classification. Download the application as a .zip, or import this repository using Git:

Deploying your impulse

Head over to your Edge Impulse project, and go to the Deployment tab. From here you can create the full library which contains the impulse and all required libraries. Select C++ library and click Build to create the library.

Running the impulse

With the project ready, it's time to verify that the application works. Head back to the studio and click on Live classification. Then load a validation sample, and click on a row under 'Detailed result'.

To verify that the local application classifies the same, we need the raw features for this timestamp. To do so click on the Copy to clipboard button next to 'Raw features'. This will copy the raw values from this validation file, before any signal processing or inferencing happens.

Open ei_main.cpp and paste the raw features inside the static const float features[] definition, for example:

Build the application by calling make in the build directory of the project:

The fastest method to load firmware onto a RP2040-based board for the first time is by mounting it as a USB Mass Storage Device. Doing this allows you to drag a file onto the board to program the flash. Connect the Raspberry Pi Pico to your computer using a micro-USB cable, making sure that you hold down the BOOTSEL button as you do so, to force it into USB Mass Storage Mode. Drag the ei_rp2040_firmware.uf2 file from the build folder to the newly appeared USB Mass Storage device.

Seeing the output

To see the output of the impulse, connect to the development board over a serial port on baud rate 115200 and reset the board. You can do this with your favorite serial monitor or with the Edge Impulse CLI:

This will run the signal processing pipeline, and then classify the output, for example:

Which matches the values we just saw in the studio. You now have your impulse running on your Raspberry Pi Pico development board

On your Espressif ESP-EYE (ESP32) development board

Knowledge required

This tutorial assumes that you're familiar with building applications with ESP IDF development framework for ESP-EYE (ESP32). If you're unfamiliar ESP-IDF, you can download a compatible with the ESP32-EYE or download the generated directly from the Deployment page in the studio.

Note: Are you looking for an example that has sensors included? The Edge Impulse firmware for Espressif ESP-EYE (ESP32) has that. See

Prerequisites

Make sure you've followed one of the tutorials and have a trained impulse. For the purpose of this tutorial, we’ll assume you trained an model. Also install the following software:

Cloning the base repository

We created an example repository which contains a small application for Espressif ESP32, which takes the raw features as an argument, and prints out the final classification. Download the application as a .zip, or import this repository using Git:

Deploying your impulse

Download the .zip file and unzip the deployed C++ library from your Edge Impulse project and copy only the folders to the root directory of this repository folder. Your final folder structure should look like this:

Running the impulse

In your case, since you might pick a different sample, the values and classification results might be different from the screenshot above. The important thing is that classification result in Studio matches the one from the device - which we will be checking a bit later.

Open ei_main.cpp and paste the raw features inside the static const float features[] definition, for example:

Build the application with ESP IDF in the project directory:

To flash the project, in the project directory execute:

Seeing the output

This will run the signal processing pipeline, and then classify the output, for example:

Which matches the values you just saw in the studio. You now have your impulse running on your Espressif ESP32 development board.

On your Alif Ensemble Series Device

Impulses can be deployed as a C++ library. This packages all your signal processing blocks, configuration and optimized learning blocks up into a single package. You can include this package in your own application to run the impulse locally. In this tutorial you'll export an impulse, and build an impulse into a custom application using either or for your Ensemble device.

Knowledge required

This tutorial assumes that you're familiar with building applications using Alif development tools and drivers, as well as Makefile based projects. You will need make set up in your environment. If you're unfamiliar with these tools you can build binaries directly for your development board from the Deployment page in the studio.

Prerequisites

Make sure you followed the , and have a trained impulse from one of the listed tutorials.
Clone the repository to your working directory.

Deploying your impulse

Head over to your Edge Impulse project, and go to Deployment. From here you can create the full library which contains the impulse and all external required libraries. Select Ethos u55 Library and click Build to create the library. Then download and extract the .zip file.

To add the impulse to your firmware project, paste the edge-impulse-sdk/, model-parameters and tflite-model directories from the downloaded '.zip' file into the source/ directory of the repository. Make sure to overwrite any existing files in the source/ directory.

This standalone example project contains minimal code required to run the imported impulse on the device. This code is located in ei_main.cpp. In this minimal code example, inference is run from a static buffer of input feature data. To verify that our embedded model achieves the exact same results as the model trained in Studio, we want to copy the same input features from Studio into the static buffer in ei_main.cpp.

To do this, first head back to the studio and click on the Live classification tab. Then load a validation sample, and click on a row under 'Detailed result'.

To verify that the local application classifies the same result, we need the raw features for this timestamp. To do so click on the 'Copy to clipboard' button next to 'Raw features'. This will copy the raw input values from this validation file, before any signal processing or inferencing happened.

In ei_main.cpp paste the raw features inside the static const float features[] definition, for example:

The project will repeatedly run inference on this buffer of raw features once built. This will show that the inference result is identical to the Live classification tab in Studio. From this starting point, the example project is fully compatible with existing SimpleLink SDK plugins, drivers or custom firmware. Use new sensor data collected in real time on the device to fill a buffer. From there, follow the same code used in ei_main.cpp to run classification on live data.

Building the project

There are three ways to build the project. The first uses the included Docker environment, pre-configured with the ARM GCC toolchain. The other options are to build the project locally with either GCC or ARMCLANG.

When building projects for the Ensemble kit, you have the option to deploy to the 'high efficiency' or 'high performance' cores. For all build options, the core is selected via the -DTARGET_SUBSYSTEM parameter when building. The commands below all default to the high performance core, but you can easily switch cores by swapping any -DTARGET_SUBSYSTEM=HP parameter to -DTARGET_SUBSYSTEM=HE

Building with Docker

Run the Docker Desktop executable, or start the docker daemon from a terminal as shown below:

Build the application by copying the following command to build inside the container:

Windows

Linux, macOS

The compiled app.axf will now be available in the build/bin directory.

Building with ARMCLANG

With the ARMCLANG compiler set up, you can build the project via:

Building with GCC

With the GCC set up, you can build the project via:

Flash the board

Grab the app.axf from the build/bin directory, and note whether you built the application for the high performance or high efficiency core
Connect your flash programmer to your debugger of choice, and configure it to select

Flash and run app.axf

View the output

To see the output of the impulse over UART2, connect to the development board over a serial port on baud rate 115,200 and reset the board. You can do this with your favourite serial monitor or with the Edge Impulse CLI:

This will run the signal processing pipeline, and then classify the output:

Troubleshooting and optimization

Timing

Timing calculations are performed in ei_classifier_porting.cpp and make use of an interrupt attached to SysTick.

An RTOS may take over this interrupt handler, in which case you should re-implement ei_read_timer_us and _ms.
The default calculation is based on the default clock rates of the Alif dev kit (400 MHz for HP core, 160 MHz for HE core). If you change this, redefine EI_CORE_CLOCK_HZ.

Memory placement

Alif M55 processors have a private fast DTCM, and also access to a larger, but slower, chip global SRAM.

For armclang the linker file attempts to place as much as possible in DTCM, and overflows into SRAM if needed.

Known issues

With debugger attached, my device boots up directly into Bus_Fault (or possibly another fault). This can especially happen when you entered Hard Fault before your last reset.

Power cycle your board and reload your program

In Simplicity Studio on the SiLabs TB Sense 2

Knowledge required

This tutorial assumes that you're familiar with compiling applications with Simplicity Studio. If you're unfamiliar with this you can build binaries directly for your development board from the Deployment page in the studio.

Note: Are you looking for an example that has all sensors included? The Edge Impulse firmware for the SiLabs Thunderboard Sense 2 has that. See .

Prerequisites

Make sure you followed the [Continuous motion recognition] (tutorials/continuous-motion-recognition.md) tutorial, and have a trained impulse. Also install the following software:

.
Python 3.6.8 or higher.
Java 64 bit JVM 11 or higher: - available at or .

Alternatively you can build this application from the command line or through Docker, see the build instructions in .

Cloning the base repository

We created an example repository which contains a small Simplicity Studio application, which takes the raw features as an argument, and prints out the final classification. You can either or import this repository using Git:

Deploying your impulse

Download the .zip file and place the contents in the 'example-standalone-inferencing-silabs-tb-sense-2/ei-workspace/edgeimpulse' folder (which you downloaded above).

Importing the workspace into Simplicity Studio

With the model downloaded you can import the project into Simplicity Studio.

Generate the Simplicity Studio project by opening a command prompt or terminal, navigating to the 'example-standalone-inferencing-silabs-tb-sense-2 folder and running:

Open Simplicity IDE and install the Gecko SDK 3.2.x.
Create a new project via File > New > Silicon Labs Project Wizard...
In the New Project Wizard select Simplicity Studio > Silicon Labs MCU Project and click Next
Under 'board' select Thunderboard Sense 2.
Select the correct SDK you installed in #1 and click Next.

Select Empty C++ Program and click Next.

Name the project example-standalone-inferencing-silabs-tb-sense-2 (exactly this) and make sure Copy contents is selected before clicking Finish.

Under 'Project Explorer' select all files, except for Includes and delete them:

Then, navigate to the example-standalone-inferencing-silabs-tb-sense-2/ei-workspace folder (in this repository), and drag all files and folders into the 'Project explorer' window in Simplicity Studio. When prompted select Copy files and folders for this operation.
Then close, and reopen the project via: Project > Close Project, then Project > Open Project.
Double-click on example-standalone-inferencing-silabs-tb-sense-2.slcp to show the Simplicity Configurator.
Edit 'Project Generators' and disable 'IAR EMBEDDED WORKBENCH PROJECT' (if it's listed):

Click Force Generation to regenerate all links and include paths.

The project is now imported!

Running the impulse

In the example directory open main.cpp and paste the raw features inside the static const float features[] definition, for example:

With everything set up, you can now build the application using Simplicity Studio, the command line, or with Docker.

Building and flashing the application

In Simplicity Studio v5, select Project > Build Project to build the firmware.
Then, right click on the development board in the Debug adapters section of Simplicity Studio and select Upload application.

Under Application image path select the GNU ARM v10.2.1 - Default/example-standalone-inferencing-silabs-tb-sense-2.bin file and click OK to flash.

(Alternatively you can drag and drop the GNU ARM v10.2.1 - Default/example-standalone-inferencing-silabs-tb-sense-2.bin file onto the TB004 mass-storage device to flash the binary.

Seeing the output

This will run the signal processing pipeline, and then classify the output:

Which matches the values we just saw in the studio. You now have your impulse running on your Thunderboard Sense 2 development board!

On your Spresense by Sony development board

Knowledge required

This tutorial assumes that you're familiar with building applications for Sony's Spresense. If you're unfamiliar with either of these you build binaries directly for your development board from the Deployment page in the studio.

Note: Are you looking for an example that has all sensors included? The Edge Impulse firmware for Sony's Spresense has that. See .

Prerequisites

Make sure you've followed one of the tutorials and have a trained impulse. Also install the following software:

.
.
- make sure arm-none-eabi-gcc is in your PATH.

Cloning the base repository

We created an example repository which contains a small application for Sony's Spresense, which takes the raw features as an argument, and prints out the final classification. Download the application , or import this repository using Git:

Deploying your impulse

Head over to your Edge Impulse project, and go to the Deployment tab. From here you can create the full library which contains the impulse and all required libraries. Select C++ library and click Build to create the library.

Download the .zip file and extract the directories in the example-standalone-inferencing-spresense/edge_impulse/ folder. Your final folder structure should look like this:

Running the impulse

To verify that the local application classifies the same, we need the raw features for this timestamp. To do so click on the Copy to clipboard button next to 'Raw features'. This will copy the raw values from this validation file, before any signal processing or inferencing happened.

Open ei_main.cpp and paste the raw features inside the static const float features[] definition, for example:

Then build and flash the application to your development board:

Building the application (`make`)

Build the application by calling make in the root directory of the project:
Connect the board to your computer using USB.
Flash the board:

Building the application (Docker)

Build the Docker image:
Build the application by running the container as follows:
Windows
Linux, macOS
Connect the board to your computer using USB.
Flash the board:
Or if you don't have make installed:

Seeing the output

To see the output of the impulse, connect to the development board over a serial port on baud rate 115200 and reset the board. You can do this with your favourite serial monitor or with the Edge Impulse CLI:

This will run the signal processing pipeline, and then classify the output, for example:

Which matches the values we just saw in the studio. You now have your impulse running on your Spresense development board!

As a generic C++ library

Knowledge required

details the available macros, structs, variables, and functions for the C++ SDK library.

A working demonstration of this project can be found .

Prerequisites

You will need a C compiler, a C++ compiler, and Make installed on your computer.

Linux

Install , , and . If you are using a Debian-based system, this can be done with:

macOS

Install and . If you are using , you can run the following commands:

Windows

Install , which comes with GNU Make and the necessary compilers. You will need to add the mingw64\bin folder to your Path.

Download the C++ Library from Edge Impulse

You are welcome to download a C++ library from your own project, but you can also follow along using this . If you use the public project, you will need to click Clone this project in the upper-right corner to clone the project to your own account.

Your impulse will download as a C++ library in a .zip file.

Create a Project

Create a directory to hold your project (e.g. my-motion). Unzip the C++ library file into the project directory. Your directory structure should look like the following:

my-motion/
|-- edge-impulse-sdk/
|-- model-parameters/
|-- tflite-model/
|-- CMakeLists.txt
|-- Makefile
|-- main.cpp

Note: You can write in C or C++ for your main application. Because portions of the impulse library are written in C++, you must use a C++ compiler for your main application (see this for more information). A more advanced option would be to use bindings for your language of choice (e.g. ). We will stick to C for this demonstration. We highly recommend keeping your main file as a .cpp or .cc file so that it will compile as as C++ code.

Explanation of C++ Library

The CMakeLists.txt file is used as part of the build system generation process. We won’t use CMake in this demonstration, but see for such an example.

edge-impulse-sdk/ contains the full software development kit (SDK) required to run your impulse along with various optimizations (e.g. ARM’s CMSIS) for supported platforms. contains the important public functions that you will want to call. Of the functions listed in that file, you will likely only need a few:

- Basic inference: we pass it the raw features and it returns the classification results.
- Initializes necessary static variables prior to running continuous inference. You must call this function prior to calling run_classifier_continuous()
- Retains a sliding window of features so that inference may be performed on a continuous stream of data. We will not explore this option in this tutorial.

Both run_classifier() and run_classifier_continuous() expect raw data to be passed in through a struct. The definition of signal_t can be found in . This struct has two properties:

total_length - total number of values, which should be equal to EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (from model-parameters/model_metadata.h). For example, if you have an accelerometer with 3 axes sampling at 100 Hz for 2 seconds, total_length would be 600.
get_data - a callback function that retrieves slices of data as required by the preprocessing (DSP) step. Some DSP algorithms (e.g. computing for keyword spotting) page raw features in one slice at a time to save memory. This function allows you to store the raw data in other locations (e.g. internal RAM, external RAM, flash) and page it in when required. We will show how to configure this callback function later in the tutorial.

If you already have your data in RAM, you can use the C++ function numpy::signal_from_buffer() (found in to construct the signal_t for you.

model-parameters/ contains the settings for preprocessing your data (in dsp_blocks.h) and for running the trained machine learning model. In that directory, defines the many settings needed by the impulse. In particular, you’ll probably care about the following:

- Number of raw elements in the array expected by the pre-processor input
- Sampling frequency of the sensor
- Number of classifier labels

tflite-model/ contains the actual trained model stored in an array. You should not need to access any of the variables or functions in these files, as inference is handled by the impulse library.

Signal Structure

Raw data data being passed to run_classifier() or run_classifier_continuous() is known as a "signal" and is passed in through a struct. Signals are always a flat buffer, so you must flatten any sensor data to a 1-dimensional array.

Input data:
Axis 1:  9.8,  9.7,  9.6
Axis 2:  0.3,  0.4,  0.5
Axis 3: -4.5, -4.6, -4.8

Signal: 9.8, 0.3, -4.5, 9.7, 0.4, -4.6, 9.6, 0.5, -4.8

Image data is flattened by listing row 1, row 2, etc. Each pixel is given in HEX format (0xRRGGBB). For example:

Input data (3x2 pixel image):
BLACK RED  RED
GREEN BLUE WHITE

Signal: 0x000000, 0xFF0000, 0xFF0000, 0x00FF00, 0x0000FF, 0xFFFFFF

It's possible to convert other image formats into this expected signal format. See for an example that converts RGB565 into a flat signal buffer.

Static Allocation

By default, the trained model resides mostly in ROM and is only pulled into RAM as needed. You can force a static allocation of the model by defining:

EI_CLASSIFIER_ALLOCATION_STATIC=1

EI_DSP_IMAGE_BUFFER_STATIC_SIZE=1024

Create an Application

Open main.cpp in your editor of choice. Paste in the following code:

#include <stdio.h>

#include "edge-impulse-sdk/classifier/ei_run_classifier.h"

// Callback function declaration
static int get_signal_data(size_t offset, size_t length, float *out_ptr);

// Raw features copied from test sample (Edge Impulse > Model testing)
static float input_buf[] = {
    /* Paste your raw features here! */ 
};

int main(int argc, char **argv) {
    
    signal_t signal;            // Wrapper for raw input buffer
    ei_impulse_result_t result; // Used to store inference output
    EI_IMPULSE_ERROR res;       // Return code from inference

    // Calculate the length of the buffer
    size_t buf_len = sizeof(input_buf) / sizeof(input_buf[0]);

    // Make sure that the length of the buffer matches expected input length
    if (buf_len != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
        printf("ERROR: The size of the input buffer is not correct.\r\n");
        printf("Expected %d items, but got %d\r\n", 
                EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, 
                (int)buf_len);
        return 1;
    }

    // Assign callback function to fill buffer used for preprocessing/inference
    signal.total_length = EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE;
    signal.get_data = &get_signal_data;

    // Perform DSP pre-processing and inference
    res = run_classifier(&signal, &result, false);

    // Print return code and how long it took to perform inference
    printf("run_classifier returned: %d\r\n", res);
    printf("Timing: DSP %d ms, inference %d ms, anomaly %d ms\r\n", 
            result.timing.dsp, 
            result.timing.classification, 
            result.timing.anomaly);

    // Print the prediction results (object detection)
#if EI_CLASSIFIER_OBJECT_DETECTION == 1
    printf("Object detection bounding boxes:\r\n");
    for (uint32_t i = 0; i < EI_CLASSIFIER_OBJECT_DETECTION_COUNT; i++) {
        ei_impulse_result_bounding_box_t bb = result.bounding_boxes[i];
        if (bb.value == 0) {
            continue;
        }
        printf("  %s (%f) [ x: %u, y: %u, width: %u, height: %u ]\r\n", 
                bb.label, 
                bb.value, 
                bb.x, 
                bb.y, 
                bb.width, 
                bb.height);
    }

    // Print the prediction results (classification)
#else
    printf("Predictions:\r\n");
    for (uint16_t i = 0; i < EI_CLASSIFIER_LABEL_COUNT; i++) {
        printf("  %s: ", ei_classifier_inferencing_categories[i]);
        printf("%.5f\r\n", result.classification[i].value);
    }
#endif

    // Print anomaly result (if it exists)
#if EI_CLASSIFIER_HAS_ANOMALY == 1
    printf("Anomaly prediction: %.3f\r\n", result.anomaly);
#endif

    return 0;
}

// Callback: fill a section of the out_ptr buffer when requested
static int get_signal_data(size_t offset, size_t length, float *out_ptr) {
    for (size_t i = 0; i < length; i++) {
        out_ptr[i] = (input_buf + offset)[i];
    }

    return EIDSP_OK;
}

For performing inference live, you would want to fill the features[] array with values from a connected sensor.

Important! Make sure that the length of the array matches the expected length for the preprocessing block in the impulse library. This value is given by (which is 200 values * 3 axes = 600 total values for the given window in our case). Also note how the values are stored: {x0, y0, z0, x1, y1, z1, …}. You will need to construct a similar array if you are sampling live data from a sensor.

Save your main.cpp.

Explanation of Main Application

Before moving on to the Makefile, let’s take a look at the important code sections in our application.

To use the C++ library, we really only need to include one header file to use the impulse SDK:

#include "edge-impulse-sdk/classifier/ei_run_classifier.h"

The ei_run_classifier.h file includes any other files we might need from the library and gives us access to the necessary functions.

The function expects a struct as an input. So, we set the members here:

// Assign callback function to fill buffer used for processing/inference
signal.total_length = EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE;
signal.get_data = &get_signal_data;

signal.total_length is the number of array elements in the input buffer. For our case, it should match the expected total number of elements (EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE).

// Callback: fill a section of the out_ptr buffer when requested
static int get_signal_data(size_t offset, size_t length, float *out_ptr) {
    for (size_t i = 0; i &lt; length; i++) {
        out_ptr[i] = (input_buf + offset)[i];
    }
    return EIDSP_OK;
}

With our signal_t struct configured, we can call our inference function:

// Perform DSP preprocessing and inference
res = run_classifier(&signal, &result, false);

run_classifier() will perform any necessary preprocessing steps (such as computing the power spectral density) prior to running inference. The inference results are stored in the second argument (of type . The third parameter is debug, which is used to print out internal states of the preprocessing and inference steps. We leave debugging disabled in this example. The function should return a value equal to EI_IMPULSE_OK if everything ran without error.

We print out the time it took (in milliseconds) to perform preprocessing (“dsp”), classification, and any anomaly detection we had enabled:

printf("Timing: DSP %d ms, classification %d ms, anomaly %d ms\r\n", 
        result.timing.dsp, 
        result.timing.classification, 
        result.timing.anomaly);

Finally, we print inference results to the console:

printf("Predictions:\r\n");
for (uint16_t i = 0; i &lt; EI_CLASSIFIER_LABEL_COUNT; i++) {
    printf("  %s: ", ei_classifier_inferencing_categories[i]);
    printf("%.5f\r\n", result.classification[i].value);
}

If your model has anomaly detection enabled, will be set to 1. We can access the anomaly value via result.anomaly. Additionally, if you are using an object detection impulse, will be set to 1, and bounding box information will be an array stored in .

Functions That Require Definition

The C++ Inference SDK library relies on several functions to allocate memory, delay the processor, read current execution time, and print out debugging information. The SDK library provides the necessary declarations in .

By default, Edge Impulse defines these functions for several popular platforms and operating systems, which you can see . In the example throughout this guide, we include the definitions for POSIX and MinGW (refer to the to see how these definitions are included in the build process).

CXXSOURCES += $(wildcard edge-impulse-sdk/porting/ti/*.c*)

Create a Makefile

Copy the following into your Makefile:

# Tool macros
CC ?= gcc
CXX ?= g++

# Settings
NAME = app
BUILD_PATH = ./build

# Location of main.cpp (must use C++ compiler for main)
CXXSOURCES = main.cpp

# Search path for header files (current directory)
CFLAGS += -I.

# C and C++ Compiler flags
CFLAGS += -Wall						# Include all warnings
CFLAGS += -g						# Generate GDB debugger information
CFLAGS += -Wno-strict-aliasing		# Disable warnings about strict aliasing
CFLAGS += -Os						# Optimize for size
CFLAGS += -DNDEBUG					# Disable assert() macro
CFLAGS += -DEI_CLASSIFIER_ENABLE_DETECTION_POSTPROCESS_OP	# Add TFLite_Detection_PostProcess operation

# C++ only compiler flags
CXXFLAGS += -std=c++14				# Use C++14 standard

# Linker flags
LDFLAGS += -lm 						# Link to math.h
LDFLAGS += -lstdc++					# Link to stdc++.h

# Include C source code for required libraries
CSOURCES += $(wildcard edge-impulse-sdk/CMSIS/DSP/Source/TransformFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/CommonTables/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/BasicMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/ComplexMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/FastMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/SupportFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/MatrixFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/StatisticsFunctions/*.c)

# Include C++ source code for required libraries
CXXSOURCES += 	$(wildcard tflite-model/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/kissfft/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/dct/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/memory.cpp) \
				$(wildcard edge-impulse-sdk/porting/posix/*.c*) \
				$(wildcard edge-impulse-sdk/porting/mingw32/*.c*)
CCSOURCES +=

# Use TensorFlow Lite for Microcontrollers (TFLM)
CFLAGS += -DTF_LITE_DISABLE_X86_NEON=1
CSOURCES +=	edge-impulse-sdk/tensorflow/lite/c/common.c
CCSOURCES +=	$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/internal/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/memory_planner/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/core/api/*.cc)

# Include CMSIS-NN if compiling for an Arm target that supports it
ifeq (${CMSIS_NN}, 1)

	# Include CMSIS-NN and CMSIS-DSP header files
	CFLAGS += -Iedge-impulse-sdk/CMSIS/NN/Include/
	CFLAGS += -Iedge-impulse-sdk/CMSIS/DSP/PrivateInclude/

	# C and C++ compiler flags for CMSIS-NN and CMSIS-DSP
	CFLAGS += -Wno-unknown-attributes 					# Disable warnings about unknown attributes
	CFLAGS += -DEI_CLASSIFIER_TFLITE_ENABLE_CMSIS_NN=1	# Use CMSIS-NN functions in the SDK
	CFLAGS += -D__ARM_FEATURE_DSP=1 					# Enable CMSIS-DSP optimized features
	CFLAGS += -D__GNUC_PYTHON__=1						# Enable CMSIS-DSP intrisics (non-C features)

	# Include C source code for required CMSIS libraries
	CSOURCES += $(wildcard edge-impulse-sdk/CMSIS/NN/Source/ActivationFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/BasicMathFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/ConcatenationFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/ConvolutionFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/FullyConnectedFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/NNSupportFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/PoolingFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/ReshapeFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/SoftmaxFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/SVDFunctions/*.c)
endif

# Generate names for the output object files (*.o)
COBJECTS := $(patsubst %.c,%.o,$(CSOURCES))
CXXOBJECTS := $(patsubst %.cpp,%.o,$(CXXSOURCES))
CCOBJECTS := $(patsubst %.cc,%.o,$(CCSOURCES))

# Default rule
.PHONY: all
all: app

# Compile library source code into object files
$(COBJECTS) : %.o : %.c
$(CXXOBJECTS) : %.o : %.cpp
$(CCOBJECTS) : %.o : %.cc
%.o: %.c
	$(CC) $(CFLAGS) -c $^ -o $@
%.o: %.cc
	$(CXX) $(CFLAGS) $(CXXFLAGS) -c $^ -o $@
%.o: %.cpp
	$(CXX) $(CFLAGS) $(CXXFLAGS) -c $^ -o $@

# Build target (must use C++ compiler)
.PHONY: app
app: $(COBJECTS) $(CXXOBJECTS) $(CCOBJECTS)
ifeq ($(OS), Windows_NT)
	if not exist build mkdir build
else
	mkdir -p $(BUILD_PATH)
endif
	$(CXX) $(COBJECTS) $(CXXOBJECTS) $(CCOBJECTS) -o $(BUILD_PATH)/$(NAME) $(LDFLAGS)

# Remove compiled object files
.PHONY: clean
clean:
ifeq ($(OS), Windows_NT)
	del /Q $(subst /,\,$(patsubst %.c,%.o,$(CSOURCES))) >nul 2>&1 || exit 0
	del /Q $(subst /,\,$(patsubst %.cpp,%.o,$(CXXSOURCES))) >nul 2>&1 || exit 0
	del /Q $(subst /,\,$(patsubst %.cc,%.o,$(CCSOURCES))) >nul 2>&1 || exit 0
else
	rm -f $(COBJECTS)
	rm -f $(CCOBJECTS)
	rm -f $(CXXOBJECTS)
endif

Save your Makefile. Ensure that it is in the top level directory (for this particular project).

Explanation of the Makefile

We’ll look at the important lines in our example Makefile. If you are not familiar with Make, we recommend taking a look at . It will walk you through the basics of creating a Makefile and what many of the commands do.

Near the top, we define where the compiler(s) can find the necessary header files:

# Search path for header files (current directory)
CFLAGS += -I.

We then define a number of compiler flags that are set by both the C and the C++ compiler. What each of these do has been commented in the script:

# C and C++ Compiler flags
CFLAGS += -Wall						# Include all warnings
CFLAGS += -g						# Generate GDB debugger information
CFLAGS += -Wno-strict-aliasing		# Disable warnings about strict aliasing
CFLAGS += -Os						# Optimize for size
CFLAGS += -DNDEBUG					# Disable assert() macro
CFLAGS += -DEI_CLASSIFIER_ENABLE_DETECTION_POSTPROCESS_OP	# Add TFLite_Detection_PostProcess operation

Some of the functions in the library use . As a result, we must support C++11 or later. The C++14 standard is recommended, so we set that in our C++ flags:

# C++ only compiler flags
CXXFLAGS += -std=c++14				# Use C++14 standard

The SDK relies on the and libraries, which come with most GNU C/C++ installations. We need to tell the linker to include them from the standard libraries on our system:

# Linker flags
LDFLAGS += -lm 						# Link to math.h
LDFLAGS += -lstdc++					# Link to stdc++.h

In addition to including the header files, we also need to tell the compiler(s) where to find source code. To do that, we create separate lists of all the .c, .cpp, and .cc files:

# Include C source code for required libraries
CSOURCES += $(wildcard edge-impulse-sdk/CMSIS/DSP/Source/TransformFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/CommonTables/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/BasicMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/ComplexMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/FastMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/SupportFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/MatrixFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/StatisticsFunctions/*.c)

# Include C++ source code for required libraries
CXXSOURCES += 	$(wildcard tflite-model/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/kissfft/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/dct/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/memory.cpp) \
				$(wildcard edge-impulse-sdk/porting/posix/*.c*) \
				$(wildcard edge-impulse-sdk/porting/mingw32/*.c*)
CCSOURCES +=

edge-impulse-sdk/porting/posix/*.c* and edge-impulse-sdk/porting/mingw32/*.c* point to C++ files that provide implementations for the . If you are using something other than a POSIX-based system or MinGW, you will want to change these files to one of the other or to your own custom definitions for those functions.

To use pure C++ for inference on almost any target with the SDK library, we can use (TFLM). TFLM comes bundled with the downloaded library. All we need to do is include it. Once again, note the compiler flag and source files that are added to the lists:

# Use TensorFlow Lite for Microcontrollers (TFLM)
CFLAGS += -DTF_LITE_DISABLE_X86_NEON=1
CSOURCES +=	edge-impulse-sdk/tensorflow/lite/c/common.c
CCSOURCES +=	$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/internal/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/memory_planner/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/core/api/*.cc)

TFLM is efficient and works with almost any microcontroller or microprocessor target. However, it does not include all of the features and functions found in (TFLite). If you are deploying to a single board computer, smartphone, etc. with TFLite support and you wish to use such functionality, you can enable full TFLite support in the build (as opposed to TFLM).

You can also use to optimize inference for NVIDIA GPUs on boards such as the NVIDIA Jetson Nano.

To enable either TFLite or TensorRT (instead of TFLM), see this . You will need to include different source files and flags. Note that for TensorRT, you will need to install a third-party library from NVIDIA.

Build and Run

cd my-motion/
make -j 4

This may take a few minutes, so be patient. When the build process is done, run your application:

./build/app

Note that this may be build/app.exe on Windows.

Take a look at the output predictions–they should match the predictions we saw earlier in the Edge Impulse Studio!

Going Further

This guide should hopefully act as a starting point to use your trained machine learning models on nearly any platform (as long as you have access to C and C++ compilers).

The easiest method of running live inference is to fill features[] with your raw sensor data, ensure it’s the correct length and format (e.g. float), and call run_classifier(). However, we did not cover use cases where you might need to run inference on a sliding window of data. Instead of retaining a large window in memory and calling run_classifier() for each new slice of data (which will re-compute features for the whole window), you can use run_classifier_continuous(). This function will remember features from one call to the next so you just need to provide the new data. See for a demonstration on how to run your impulse continuously.

We recognize that the embedded world is full of different build systems and IDEs. While we can’t support every single IDE, we hope that this guide showed how to include the required header and source files to build your project. Additionally, here are for popular platforms to help you run your impulse locally.