LogoLogo
HomeAPI & SDKsProjectsForumStudio
  • Getting started
    • For beginners
    • For ML practitioners
    • For embedded engineers
  • Frequently asked questions (FAQ)
  • Tutorials
    • End-to-end tutorials
      • Computer vision
        • Image classification
        • Object detection
          • Object detection with bounding boxes
          • Detect objects with centroid (FOMO)
        • Visual anomaly detection
        • Visual regression
      • Audio
        • Sound recognition
        • Keyword spotting
      • Time-series
        • Motion recognition + anomaly detection
        • Regression + anomaly detection
        • HR/HRV
        • Environmental (Sensor fusion)
    • Data
      • Data ingestion
        • Collecting image data from the Studio
        • Collecting image data with your mobile phone
        • Collecting image data with the OpenMV Cam H7 Plus
        • Using the Edge Impulse Python SDK to upload and download data
        • Trigger connected board data sampling
        • Ingest multi-labeled data using the API
      • Synthetic data
        • Generate audio datasets using Eleven Labs
        • Generate image datasets using Dall-E
        • Generate keyword spotting datasets using Google TTS
        • Generate physics simulation datasets using PyBullet
        • Generate timeseries data with MATLAB
      • Labeling
        • Label audio data using your existing models
        • Label image data using GPT-4o
      • Edge Impulse Datasets
    • Feature extraction
      • Building custom processing blocks
      • Sensor fusion using embeddings
    • Machine learning
      • Classification with multiple 2D input features
      • Visualize neural networks decisions with Grad-CAM
      • Sensor fusion using embeddings
      • FOMO self-attention
    • Inferencing & post-processing
      • Count objects using FOMO
      • Continuous audio sampling
      • Multi-impulse (C++)
      • Multi-impulse (Python)
    • Lifecycle management
      • CI/CD with GitHub Actions
      • Data aquisition from S3 object store - Golioth on AI
      • OTA model updates
        • with Arduino IDE (for ESP32)
        • with Arduino IoT Cloud
        • with Blues Wireless
        • with Docker on Allxon
        • with Docker on Balena
        • with Docker on NVIDIA Jetson
        • with Espressif IDF
        • with Nordic Thingy53 and the Edge Impulse app
        • with Particle Workbench
        • with Zephyr on Golioth
    • API examples
      • Customize the EON Tuner
      • Ingest multi-labeled data using the API
      • Python API bindings example
      • Running jobs using the API
      • Trigger connected board data sampling
    • Python SDK examples
      • Using the Edge Impulse Python SDK to run EON Tuner
      • Using the Edge Impulse Python SDK to upload and download data
      • Using the Edge Impulse Python SDK with Hugging Face
      • Using the Edge Impulse Python SDK with SageMaker Studio
      • Using the Edge Impulse Python SDK with TensorFlow and Keras
      • Using the Edge Impulse Python SDK with Weights & Biases
    • Expert network projects
  • Edge Impulse Studio
    • Organization hub
      • Users
      • Data campaigns
      • Data
        • Cloud data storage
      • Data pipelines
      • Data transformation
        • Transformation blocks
      • Upload portals
      • Custom blocks
        • Custom AI labeling blocks
        • Custom deployment blocks
        • Custom learning blocks
        • Custom processing blocks
        • Custom synthetic data blocks
        • Custom transformation blocks
      • Health reference design
        • Synchronizing clinical data with a bucket
        • Validating clinical data
        • Querying clinical data
        • Transforming clinical data
    • Project dashboard
      • Select AI hardware
    • Devices
    • Data acquisition
      • Uploader
      • Data explorer
      • Data sources
      • Synthetic data
      • Labeling queue
      • AI labeling
      • CSV Wizard (time-series)
      • Multi-label (time-series)
      • Tabular data (pre-processed & non-time-series)
      • Metadata
      • Auto-labeler | deprecated
    • Impulses
    • EON Tuner
      • Search space
    • Processing blocks
      • Audio MFCC
      • Audio MFE
      • Audio Syntiant
      • Flatten
      • HR/HRV features
      • Image
      • IMU Syntiant
      • Raw data
      • Spectral features
      • Spectrogram
      • Custom processing blocks
      • Feature explorer
    • Learning blocks
      • Anomaly detection (GMM)
      • Anomaly detection (K-means)
      • Classification
      • Classical ML
      • Object detection
        • MobileNetV2 SSD FPN
        • FOMO: Object detection for constrained devices
      • Object tracking
      • Regression
      • Transfer learning (images)
      • Transfer learning (keyword spotting)
      • Visual anomaly detection (FOMO-AD)
      • Custom learning blocks
      • Expert mode
      • NVIDIA TAO | deprecated
    • Retrain model
    • Live classification
    • Model testing
    • Performance calibration
    • Deployment
      • EON Compiler
      • Custom deployment blocks
    • Versioning
    • Bring your own model (BYOM)
    • File specifications
      • deployment-metadata.json
      • ei-metadata.json
      • ids.json
      • parameters.json
      • sample_id_details.json
      • train_input.json
  • Tools
    • API and SDK references
    • Edge Impulse CLI
      • Installation
      • Serial daemon
      • Uploader
      • Data forwarder
      • Impulse runner
      • Blocks
      • Himax flash tool
    • Edge Impulse for Linux
      • Linux Node.js SDK
      • Linux Go SDK
      • Linux C++ SDK
      • Linux Python SDK
      • Flex delegates
      • Rust Library
    • Rust Library
    • Edge Impulse Python SDK
  • Run inference
    • C++ library
      • As a generic C++ library
      • On Android
      • On your desktop computer
      • On your Alif Ensemble Series Device
      • On your Espressif ESP-EYE (ESP32) development board
      • On your Himax WE-I Plus
      • On your Raspberry Pi Pico (RP2040) development board
      • On your SiLabs Thunderboard Sense 2
      • On your Spresense by Sony development board
      • On your Syntiant TinyML Board
      • On your TI LaunchPad using GCC and the SimpleLink SDK
      • On your Zephyr-based Nordic Semiconductor development board
    • Arm Keil MDK CMSIS-PACK
    • Arduino library
      • Arduino IDE 1.18
    • Cube.MX CMSIS-PACK
    • Docker container
    • DRP-AI library
      • DRP-AI on your Renesas development board
      • DRP-AI TVM i8 on Renesas RZ/V2H
    • IAR library
    • Linux EIM executable
    • OpenMV
    • Particle library
    • Qualcomm IM SDK GStreamer
    • WebAssembly
      • Through WebAssembly (Node.js)
      • Through WebAssembly (browser)
    • Edge Impulse firmwares
    • Hardware specific tutorials
      • Image classification - Sony Spresense
      • Audio event detection with Particle boards
      • Motion recognition - Particle - Photon 2 & Boron
      • Motion recognition - RASynBoard
      • Motion recognition - Syntiant
      • Object detection - SiLabs xG24 Dev Kit
      • Sound recognition - TI LaunchXL
      • Keyword spotting - TI LaunchXL
      • Keyword spotting - Syntiant - RC Commands
      • Running NVIDIA TAO models on the Renesas RA8D1
      • Two cameras, two models - running multiple object detection models on the RZ/V2L
  • Edge AI Hardware
    • Overview
    • Production-ready
      • Advantech ICAM-540
      • Seeed SenseCAP A1101
      • Industry reference design - BrickML
    • MCU
      • Ambiq Apollo4 family of SoCs
      • Ambiq Apollo510
      • Arducam Pico4ML TinyML Dev Kit
      • Arduino Nano 33 BLE Sense
      • Arduino Nicla Sense ME
      • Arduino Nicla Vision
      • Arduino Portenta H7
      • Blues Wireless Swan
      • Espressif ESP-EYE
      • Himax WE-I Plus
      • Infineon CY8CKIT-062-BLE Pioneer Kit
      • Infineon CY8CKIT-062S2 Pioneer Kit
      • Nordic Semi nRF52840 DK
      • Nordic Semi nRF5340 DK
      • Nordic Semi nRF9160 DK
      • Nordic Semi nRF9161 DK
      • Nordic Semi nRF9151 DK
      • Nordic Semi nRF7002 DK
      • Nordic Semi Thingy:53
      • Nordic Semi Thingy:91
      • Open MV Cam H7 Plus
      • Particle Photon 2
      • Particle Boron
      • RAKwireless WisBlock
      • Raspberry Pi RP2040
      • Renesas CK-RA6M5 Cloud Kit
      • Renesas EK-RA8D1
      • Seeed Wio Terminal
      • Seeed XIAO nRF52840 Sense
      • Seeed XIAO ESP32 S3 Sense
      • SiLabs Thunderboard Sense 2
      • Sony's Spresense
      • ST B-L475E-IOT01A
      • TI CC1352P Launchpad
    • MCU + AI accelerators
      • Alif Ensemble
      • Arduino Nicla Voice
      • Avnet RASynBoard
      • Seeed Grove - Vision AI Module
      • Seeed Grove Vision AI Module V2 (WiseEye2)
      • Himax WiseEye2 Module and ISM Devboard
      • SiLabs xG24 Dev Kit
      • STMicroelectronics STM32N6570-DK
      • Synaptics Katana EVK
      • Syntiant Tiny ML Board
    • CPU
      • macOS
      • Linux x86_64
      • Raspberry Pi 4
      • Raspberry Pi 5
      • Texas Instruments SK-AM62
      • Microchip SAMA7G54
      • Renesas RZ/G2L
    • CPU + AI accelerators
      • AVNET RZBoard V2L
      • BrainChip AKD1000
      • i.MX 8M Plus EVK
      • Digi ConnectCore 93 Development Kit
      • MemryX MX3
      • MistyWest MistySOM RZ/V2L
      • Qualcomm Dragonwing RB3 Gen 2 Dev Kit
      • Renesas RZ/V2L
      • Renesas RZ/V2H
      • IMDT RZ/V2H
      • Texas Instruments SK-TDA4VM
      • Texas Instruments SK-AM62A-LP
      • Texas Instruments SK-AM68A
      • Thundercomm Rubik Pi 3
    • GPU
      • Advantech ICAM-540
      • NVIDIA Jetson
      • Seeed reComputer Jetson
    • Mobile phone
    • Porting guide
  • Integrations
    • Arduino Machine Learning Tools
    • AWS IoT Greengrass
    • Embedded IDEs - Open-CMSIS
    • NVIDIA Omniverse
    • Scailable
    • Weights & Biases
  • Tips & Tricks
    • Combining impulses
    • Increasing model performance
    • Optimizing compute time
    • Inference performance metrics
  • Concepts
    • Glossary
    • Course: Edge AI Fundamentals
      • Introduction to edge AI
      • What is edge computing?
      • What is machine learning (ML)?
      • What is edge AI?
      • How to choose an edge AI device
      • Edge AI lifecycle
      • What is edge MLOps?
      • What is Edge Impulse?
      • Case study: Izoelektro smart grid monitoring
      • Test and certification
    • Data engineering
      • Audio feature extraction
      • Motion feature extraction
    • Machine learning
      • Data augmentation
      • Evaluation metrics
      • Neural networks
        • Layers
        • Activation functions
        • Loss functions
        • Optimizers
          • Learned optimizer (VeLO)
        • Epochs
    • What is embedded ML, anyway?
    • What is edge machine learning (edge ML)?
Powered by GitBook
On this page
  • Prerequisites
  • Download the C++ Library from Edge Impulse
  • Create a Project
  • Create an Application
  • Functions That Require Definition
  • Create a Makefile
  • Build and Run
  • Going Further

Was this helpful?

Export as PDF
  1. Run inference
  2. C++ library

As a generic C++ library

PreviousC++ libraryNextOn Android

Last updated 3 months ago

Was this helpful?

While Edge Impulse supports a number of boards that make gathering data and deploying your model easy, we know that many people will want to run edge machine learning on their own board. This tutorial will show you how to export an impulse and how to include it as a C++ library.

An impulse is a combination of any preprocessing code necessary to extract features from your raw data along with inference using your trained machine learning model. You provide the library with raw data from your sensor, and it will return the output of your model. It performs feature extraction and inference, just as you configured in the Edge Impulse Studio!

We recommend working through the steps in this guide to see how to run an impulse on a full operating system (macOS, Linux, or Windows) first. Once you understand how to include the impulse as a C++ library, you can port it to any build system or integrated development environment (IDE) you wish.

Knowledge required

This guide assumes you have some familiarity with C and the GNU Make build system. We will demonstrate how to run an impulse (e.g. inference) on Linux, macOS, or Windows using a C program and Make. We want to give you a starting point for porting the C++ library to your own build system.

details the available macros, structs, variables, and functions for the C++ SDK library.

A working demonstration of this project can be found .

Prerequisites

You will need a C compiler, a C++ compiler, and Make installed on your computer.

Linux

Install , , and . If you are using a Debian-based system, this can be done with:

sudo apt update
sudo apt install build-essentials

macOS

Install and . If you are using , you can run the following commands:

brew install llvm
brew install make

Windows

Download the C++ Library from Edge Impulse

Head to the Deployment page for your project. Select C++ library. Scroll down, and click Build. Note that you must have a fully trained model in order to download any of the deployment options.

Your impulse will download as a C++ library in a .zip file.

Create a Project

The easiest way to test the impulse library is to use raw features from one of your test set samples. When you run your program, it should print out the class probabilities that match those of the test sample in the Studio.

Create a directory to hold your project (e.g. my-motion). Unzip the C++ library file into the project directory. Your directory structure should look like the following:

my-motion/
|-- edge-impulse-sdk/
|-- model-parameters/
|-- tflite-model/
|-- CMakeLists.txt
|-- Makefile
|-- main.cpp

Explanation of C++ Library

  • total_length - total number of values, which should be equal to EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (from model-parameters/model_metadata.h). For example, if you have an accelerometer with 3 axes sampling at 100 Hz for 2 seconds, total_length would be 600.

model_variables.h holds some additional information about the model and preprocessing steps. Most importantly, you might want ei_classifier_inferencing_categories[] if you need the labels for your categories in string form.

tflite-model/ contains the actual trained model stored in an array. You should not need to access any of the variables or functions in these files, as inference is handled by the impulse library.

Signal Structure

Time-series data with multiple axes are flattened so that the value from each axis is listed from each time step before moving on to the next time step. For example, here is how sensor data with 3 axes would be flattened:

Input data:
Axis 1:  9.8,  9.7,  9.6
Axis 2:  0.3,  0.4,  0.5
Axis 3: -4.5, -4.6, -4.8

Signal: 9.8, 0.3, -4.5, 9.7, 0.4, -4.6, 9.6, 0.5, -4.8

Image data is flattened by listing row 1, row 2, etc. Each pixel is given in HEX format (0xRRGGBB). For example:

Input data (3x2 pixel image):
BLACK RED  RED
GREEN BLUE WHITE

Signal: 0x000000, 0xFF0000, 0xFF0000, 0x00FF00, 0x0000FF, 0xFFFFFF

Resizing images

    int res = ei::image::processing::resize_image_using_mode(
        input_buf,
        my_input_cols, // Width of native camera image, in pixels
        my_input_rows, // Height of native camera image, in pixels 
        input_buf, // destination can be same buffer as input ("in place")
        EI_CLASSIFIER_INPUT_WIDTH, // defined in header
        EI_CLASSIFIER_INPUT_HEIGHT, // defined in header
        3, // always 3 (input to run_classifier is always RGB888)
        EI_CLASSIFIER_RESIZE_MODE); // defined in header

Static Allocation

By default, the trained model resides mostly in ROM and is only pulled into RAM as needed. You can force a static allocation of the model by defining:

EI_CLASSIFIER_ALLOCATION_STATIC=1

If you are doing image classification with a quantized model, the data is automatically quantized when read from the signal. This is automatically enabled when you call run_impulse. If you want to adjust the size of the buffer that is used to read from the signal in this case, you can set EI_DSP_IMAGE_BUFFER_STATIC_SIZE, which also allocates the buffer statically. For example, you might set:

EI_DSP_IMAGE_BUFFER_STATIC_SIZE=1024

Create an Application

Open main.cpp in your editor of choice. Paste in the following code:

#include <stdio.h>

#include "edge-impulse-sdk/classifier/ei_run_classifier.h"

// Callback function declaration
static int get_signal_data(size_t offset, size_t length, float *out_ptr);

// Raw features copied from test sample (Edge Impulse > Model testing)
static float input_buf[] = {
    /* Paste your raw features here! */ 
};

int main(int argc, char **argv) {
    
    signal_t signal;            // Wrapper for raw input buffer
    ei_impulse_result_t result; // Used to store inference output
    EI_IMPULSE_ERROR res;       // Return code from inference

    // Calculate the length of the buffer
    size_t buf_len = sizeof(input_buf) / sizeof(input_buf[0]);

    // Make sure that the length of the buffer matches expected input length
    if (buf_len != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
        printf("ERROR: The size of the input buffer is not correct.\r\n");
        printf("Expected %d items, but got %d\r\n", 
                EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, 
                (int)buf_len);
        return 1;
    }

    // Assign callback function to fill buffer used for preprocessing/inference
    signal.total_length = EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE;
    signal.get_data = &get_signal_data;

    // Perform DSP pre-processing and inference
    res = run_classifier(&signal, &result, false);

    // Print return code and how long it took to perform inference
    printf("run_classifier returned: %d\r\n", res);
    printf("Timing: DSP %d ms, inference %d ms, anomaly %d ms\r\n", 
            result.timing.dsp, 
            result.timing.classification, 
            result.timing.anomaly);

    // Print the prediction results (object detection)
#if EI_CLASSIFIER_OBJECT_DETECTION == 1
    printf("Object detection bounding boxes:\r\n");
    for (uint32_t i = 0; i < EI_CLASSIFIER_OBJECT_DETECTION_COUNT; i++) {
        ei_impulse_result_bounding_box_t bb = result.bounding_boxes[i];
        if (bb.value == 0) {
            continue;
        }
        printf("  %s (%f) [ x: %u, y: %u, width: %u, height: %u ]\r\n", 
                bb.label, 
                bb.value, 
                bb.x, 
                bb.y, 
                bb.width, 
                bb.height);
    }

    // Print the prediction results (classification)
#else
    printf("Predictions:\r\n");
    for (uint16_t i = 0; i < EI_CLASSIFIER_LABEL_COUNT; i++) {
        printf("  %s: ", ei_classifier_inferencing_categories[i]);
        printf("%.5f\r\n", result.classification[i].value);
    }
#endif

    // Print anomaly result (if it exists)
#if EI_CLASSIFIER_HAS_ANOMALY == 1
    printf("Anomaly prediction: %.3f\r\n", result.anomaly);
#endif

    return 0;
}

// Callback: fill a section of the out_ptr buffer when requested
static int get_signal_data(size_t offset, size_t length, float *out_ptr) {
    for (size_t i = 0; i < length; i++) {
        out_ptr[i] = (input_buf + offset)[i];
    }

    return EIDSP_OK;
}

We’re going to copy raw features from one of our test samples. This process allows us to test that preprocessing and inference works without needing to connect a real sensor to our computer or board.

Go back to your project in the Edge Impulse Studio. Click on Model testing. Find a sample (I’ll use one of the samples labeled “wave”), click the three dots (kebab menu) next to the sample, and click Show classification.

A new tab will open, and you can see a visualization of the sample along with the raw features, expected outcome (ground truth label), and inference results. Feel free to slide the window to any point in the test sample to get the raw features from that window. The raw features are the actual values that are sent to the impulse for preprocessing and inference.

I’ll leave the window at the front of the sample for this example. Click the Copy features button next to the Raw features. This will copy only the raw features under the given window to your clipboard. Additionally, make a note of the highlighted Detailed result. We will want to compare our local inference output to these values (e.g. wave should be close to 0.99 and the other labels should be close to 0.0). Some rounding error is expected.

Paste the list of raw feature values into the input_buf array. Note that this buffer is constant for this particular program. However, it demonstrates how you can fill an array with floating point values from a sensor to pass to the impulse SDK library.

For performing inference live, you would want to fill the features[] array with values from a connected sensor.

Save your main.cpp.

Explanation of Main Application

Before moving on to the Makefile, let’s take a look at the important code sections in our application.

To use the C++ library, we really only need to include one header file to use the impulse SDK:

#include "edge-impulse-sdk/classifier/ei_run_classifier.h"

The ei_run_classifier.h file includes any other files we might need from the library and gives us access to the necessary functions.

// Assign callback function to fill buffer used for processing/inference
signal.total_length = EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE;
signal.get_data = &get_signal_data;

signal.total_length is the number of array elements in the input buffer. For our case, it should match the expected total number of elements (EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE).

signal.get_data must be set to a callback function. run_classifier() will use this callback function to grab data from our buffer as needed. It is up to you to create this function. Let’s take a look at the simplest form of this callback:

// Callback: fill a section of the out_ptr buffer when requested
static int get_signal_data(size_t offset, size_t length, float *out_ptr) {
    for (size_t i = 0; i &lt; length; i++) {
        out_ptr[i] = (input_buf + offset)[i];
    }
    return EIDSP_OK;
}

This function copies data from our input buffer (a static global array) plus a memory offset into an array provided by the caller. We don’t know exactly what offset and length will be for any given call, but we must be ready with valid data. We do know that this function will not attempt to index beyond the provided signal.total_length amount.

The callback structure is used here so that data can be paged in from any location (e.g. RAM or ROM), which means we don't necessarily need to save the entire sample in RAM. This process helps save precious RAM space on resource-constrained devices.

With our signal_t struct configured, we can call our inference function:

// Perform DSP preprocessing and inference
res = run_classifier(&signal, &result, false);

We print out the time it took (in milliseconds) to perform preprocessing (“dsp”), classification, and any anomaly detection we had enabled:

printf("Timing: DSP %d ms, classification %d ms, anomaly %d ms\r\n", 
        result.timing.dsp, 
        result.timing.classification, 
        result.timing.anomaly);

Finally, we print inference results to the console:

printf("Predictions:\r\n");
for (uint16_t i = 0; i &lt; EI_CLASSIFIER_LABEL_COUNT; i++) {
    printf("  %s: ", ei_classifier_inferencing_categories[i]);
    printf("%.5f\r\n", result.classification[i].value);
}

We can access the individual classification results for each class with result.classification[i].value where i is the index of our label. Labels are stored in alphabetical order in ei_classifier_inferencing_categories[]. Each prediction value must be between 0.0 and 1.0. Additionally, thanks to the softmax function at the end of our neural network, all of the predictions should add up to 1.0.

Functions That Require Definition

Throughout the library, you will find these functions being called. However, no definitions are provided because every platform is different in how these functions are implemented. For example, you may want to print debugging information to a console (stdout) or over a UART serial port.

If you were to try to build this project for another platform (e.g. a microcontroller), the process would fail, as you are missing these definitions. If your platform is supported by the Edge Impulse C++ Inference SDK, you may include that folder in your C++ sources. A Makefile example of including support for TI implementations might be:

CXXSOURCES += $(wildcard edge-impulse-sdk/porting/ti/*.c*)

If your platform is not supported or you would like to create custom definitions, you may do so in your own code. The following functions must be defined for your platform (the reference guide linked to by each function provides several examples on possible implementations):

Create a Makefile

Due to the number of files we must include from the library, it can be quite difficult to call the compiler and linker manually. As a result, we will use a Makefile script and the Make tool to compile all the necessary source code, link the object files, and generate a single executable file for us.

Copy the following into your Makefile:

# Tool macros
CC ?= gcc
CXX ?= g++

# Settings
NAME = app
BUILD_PATH = ./build

# Location of main.cpp (must use C++ compiler for main)
CXXSOURCES = main.cpp

# Search path for header files (current directory)
CFLAGS += -I.

# C and C++ Compiler flags
CFLAGS += -Wall						# Include all warnings
CFLAGS += -g						# Generate GDB debugger information
CFLAGS += -Wno-strict-aliasing		# Disable warnings about strict aliasing
CFLAGS += -Os						# Optimize for size
CFLAGS += -DNDEBUG					# Disable assert() macro
CFLAGS += -DEI_CLASSIFIER_ENABLE_DETECTION_POSTPROCESS_OP	# Add TFLite_Detection_PostProcess operation

# C++ only compiler flags
CXXFLAGS += -std=c++14				# Use C++14 standard

# Linker flags
LDFLAGS += -lm 						# Link to math.h
LDFLAGS += -lstdc++					# Link to stdc++.h

# Include C source code for required libraries
CSOURCES += $(wildcard edge-impulse-sdk/CMSIS/DSP/Source/TransformFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/CommonTables/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/BasicMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/ComplexMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/FastMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/SupportFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/MatrixFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/StatisticsFunctions/*.c)

# Include C++ source code for required libraries
CXXSOURCES += 	$(wildcard tflite-model/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/kissfft/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/dct/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/memory.cpp) \
				$(wildcard edge-impulse-sdk/porting/posix/*.c*) \
				$(wildcard edge-impulse-sdk/porting/mingw32/*.c*)
CCSOURCES +=

# Use LiteRT (previously Tensorflow Lite) for Microcontrollers (TFLM)
CFLAGS += -DTF_LITE_DISABLE_X86_NEON=1
CSOURCES +=	edge-impulse-sdk/tensorflow/lite/c/common.c
CCSOURCES +=	$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/internal/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/memory_planner/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/core/api/*.cc)

# Include CMSIS-NN if compiling for an Arm target that supports it
ifeq (${CMSIS_NN}, 1)

	# Include CMSIS-NN and CMSIS-DSP header files
	CFLAGS += -Iedge-impulse-sdk/CMSIS/NN/Include/
	CFLAGS += -Iedge-impulse-sdk/CMSIS/DSP/PrivateInclude/

	# C and C++ compiler flags for CMSIS-NN and CMSIS-DSP
	CFLAGS += -Wno-unknown-attributes 					# Disable warnings about unknown attributes
	CFLAGS += -DEI_CLASSIFIER_TFLITE_ENABLE_CMSIS_NN=1	# Use CMSIS-NN functions in the SDK
	CFLAGS += -D__ARM_FEATURE_DSP=1 					# Enable CMSIS-DSP optimized features
	CFLAGS += -D__GNUC_PYTHON__=1						# Enable CMSIS-DSP intrisics (non-C features)

	# Include C source code for required CMSIS libraries
	CSOURCES += $(wildcard edge-impulse-sdk/CMSIS/NN/Source/ActivationFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/BasicMathFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/ConcatenationFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/ConvolutionFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/FullyConnectedFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/NNSupportFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/PoolingFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/ReshapeFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/SoftmaxFunctions/*.c) \
				$(wildcard edge-impulse-sdk/CMSIS/NN/Source/SVDFunctions/*.c)
endif

# Generate names for the output object files (*.o)
COBJECTS := $(patsubst %.c,%.o,$(CSOURCES))
CXXOBJECTS := $(patsubst %.cpp,%.o,$(CXXSOURCES))
CCOBJECTS := $(patsubst %.cc,%.o,$(CCSOURCES))

# Default rule
.PHONY: all
all: app

# Compile library source code into object files
$(COBJECTS) : %.o : %.c
$(CXXOBJECTS) : %.o : %.cpp
$(CCOBJECTS) : %.o : %.cc
%.o: %.c
	$(CC) $(CFLAGS) -c $^ -o $@
%.o: %.cc
	$(CXX) $(CFLAGS) $(CXXFLAGS) -c $^ -o $@
%.o: %.cpp
	$(CXX) $(CFLAGS) $(CXXFLAGS) -c $^ -o $@

# Build target (must use C++ compiler)
.PHONY: app
app: $(COBJECTS) $(CXXOBJECTS) $(CCOBJECTS)
ifeq ($(OS), Windows_NT)
	if not exist build mkdir build
else
	mkdir -p $(BUILD_PATH)
endif
	$(CXX) $(COBJECTS) $(CXXOBJECTS) $(CCOBJECTS) -o $(BUILD_PATH)/$(NAME) $(LDFLAGS)

# Remove compiled object files
.PHONY: clean
clean:
ifeq ($(OS), Windows_NT)
	del /Q $(subst /,\,$(patsubst %.c,%.o,$(CSOURCES))) >nul 2>&1 || exit 0
	del /Q $(subst /,\,$(patsubst %.cpp,%.o,$(CXXSOURCES))) >nul 2>&1 || exit 0
	del /Q $(subst /,\,$(patsubst %.cc,%.o,$(CCSOURCES))) >nul 2>&1 || exit 0
else
	rm -f $(COBJECTS)
	rm -f $(CCOBJECTS)
	rm -f $(CXXOBJECTS)
endif

Save your Makefile. Ensure that it is in the top level directory (for this particular project).

This Makefile should serve as an example of how to import and compile the impulse SDK library. The particular build system or IDE for your platform may not use Make, so I recommend reading the next section to see what files and flags must be included. You can use this information to configure your own build system.

Explanation of the Makefile

Near the top, we define where the compiler(s) can find the necessary header files:

# Search path for header files (current directory)
CFLAGS += -I.

We need to point this -I flag to the directory that holds edge-impulse-sdk/, model-parameters/, and tflite-model/ so that the build system can find the required header files. If you unzipped your C++ library into a lib/ folder, for example, this flag should be -Ilib/.

We then define a number of compiler flags that are set by both the C and the C++ compiler. What each of these do has been commented in the script:

# C and C++ Compiler flags
CFLAGS += -Wall						# Include all warnings
CFLAGS += -g						# Generate GDB debugger information
CFLAGS += -Wno-strict-aliasing		# Disable warnings about strict aliasing
CFLAGS += -Os						# Optimize for size
CFLAGS += -DNDEBUG					# Disable assert() macro
CFLAGS += -DEI_CLASSIFIER_ENABLE_DETECTION_POSTPROCESS_OP	# Add TFLite_Detection_PostProcess operation
# C++ only compiler flags
CXXFLAGS += -std=c++14				# Use C++14 standard
# Linker flags
LDFLAGS += -lm 						# Link to math.h
LDFLAGS += -lstdc++					# Link to stdc++.h

In addition to including the header files, we also need to tell the compiler(s) where to find source code. To do that, we create separate lists of all the .c, .cpp, and .cc files:

# Include C source code for required libraries
CSOURCES += $(wildcard edge-impulse-sdk/CMSIS/DSP/Source/TransformFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/CommonTables/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/BasicMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/ComplexMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/FastMathFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/SupportFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/MatrixFunctions/*.c) \
			$(wildcard edge-impulse-sdk/CMSIS/DSP/Source/StatisticsFunctions/*.c)

# Include C++ source code for required libraries
CXXSOURCES += 	$(wildcard tflite-model/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/kissfft/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/dct/*.cpp) \
				$(wildcard edge-impulse-sdk/dsp/memory.cpp) \
				$(wildcard edge-impulse-sdk/porting/posix/*.c*) \
				$(wildcard edge-impulse-sdk/porting/mingw32/*.c*)
CCSOURCES +=

Note the directory locations given in these lists. Many IDEs will ask you for the location of source files to include in the build process. You will want to include these directories (such as edge-impulse-sdk/CMSIS/DSP/Source/TransformFunctions/, etc.).

If you unzipped the C++ library into a different location (e.g. into a separate lib/ directory), then all of these source locations should be updated to reflect that. For example, tflite-model/*.cpp would become lib/tflite-model/*.cpp.

# Use LiteRT (previously Tensorflow Lite) for Microcontrollers (TFLM)
CFLAGS += -DTF_LITE_DISABLE_X86_NEON=1
CSOURCES +=	edge-impulse-sdk/tensorflow/lite/c/common.c
CCSOURCES +=	$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/kernels/internal/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/kernels/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/micro/memory_planner/*.cc) \
				$(wildcard edge-impulse-sdk/tensorflow/lite/core/api/*.cc)

While TFLM is a great generic package for many target platforms, it is not as efficient as TFLite for some, such as Linux and Android. As a result, you will likely see a performance boost if you use TFLite (instead of TFLM) on Linux.

The rest of the Makefile compiles each of the source files to object files (.o) before combining and linking them into a standalone executable file. This particular Makefile places the executable (app) in the build/ directory.

Build and Run

At this point, you’re ready to build your application and run it! Open a terminal (MinGW Shell, if you’re on Windows), navigate to your project directory, and run the make command. You can use the -j [jobs] command to have Make use multiple threads to speed up the build process (especially if you have multiple cores in your CPU):

cd my-motion/
make -j 4

This may take a few minutes, so be patient. When the build process is done, run your application:

./build/app

Note that this may be build/app.exe on Windows.

Take a look at the output predictions–they should match the predictions we saw earlier in the Edge Impulse Studio!

Going Further

This guide should hopefully act as a starting point to use your trained machine learning models on nearly any platform (as long as you have access to C and C++ compilers).

Install , which comes with GNU Make and the necessary compilers. You will need to add the mingw64\bin folder to your Path.

You are welcome to download a C++ library from your own project, but you can also follow along using this . If you use the public project, you will need to click Clone this project in the upper-right corner to clone the project to your own account.

Note: You can write in C or C++ for your main application. Because portions of the impulse library are written in C++, you must use a C++ compiler for your main application (see this for more information). A more advanced option would be to use bindings for your language of choice (e.g. ). We will stick to C for this demonstration. We highly recommend keeping your main file as a .cpp or .cc file so that it will compile as as C++ code.

The CMakeLists.txt file is used as part of the build system generation process. We won’t use CMake in this demonstration, but see for such an example.

edge-impulse-sdk/ contains the full software development kit (SDK) required to run your impulse along with various optimizations (e.g. ARM’s CMSIS) for supported platforms. contains the important public functions that you will want to call. Of the functions listed in that file, you will likely only need a few:

- Basic inference: we pass it the raw features and it returns the classification results.

- Initializes necessary static variables prior to running continuous inference. You must call this function prior to calling run_classifier_continuous()

- Retains a sliding window of features so that inference may be performed on a continuous stream of data. We will not explore this option in this tutorial.

Both run_classifier() and run_classifier_continuous() expect raw data to be passed in through a struct. The definition of signal_t can be found in . This struct has two properties:

get_data - a callback function that retrieves slices of data as required by the preprocessing (DSP) step. Some DSP algorithms (e.g. computing for keyword spotting) page raw features in one slice at a time to save memory. This function allows you to store the raw data in other locations (e.g. internal RAM, external RAM, flash) and page it in when required. We will show how to configure this callback function later in the tutorial.

If you already have your data in RAM, you can use the C++ function numpy::signal_from_buffer() (found in to construct the signal_t for you.

model-parameters/ contains the settings for preprocessing your data (in dsp_blocks.h) and for running the trained machine learning model. In that directory, defines the many settings needed by the impulse. In particular, you’ll probably care about the following:

- Number of raw elements in the array expected by the pre-processor input

- Sampling frequency of the sensor

- Number of classifier labels

Raw data data being passed to run_classifier() or run_classifier_continuous() is known as a "signal" and is passed in through a struct. Signals are always a flat buffer, so you must flatten any sensor data to a 1-dimensional array.

It's possible to convert other image formats into this expected signal format. See for an example that converts RGB565 into a flat signal buffer.

We always recommend that you configure your camera driver to output images in the correct size (the input size chosen in Studio) for best performance. However, if for debug or other reasons you have a larger frame than you want to run inference on, you can use this function from the edge-impulse-sdk to resize. It can operate in place, as in this example, but you must be mindful of the buffer size. In the case of using "fit longest" mode in the view from Edge Impulse Studio, the resized buffer can actually be larger than the input buffer (because of the letterboxing resizing method). The constant parameters are all pulled from the already included edge-impulse-sdk header file, that are set by Studio when you export your project.

See this for more details.

Important! Make sure that the length of the array matches the expected length for the preprocessing block in the impulse library. This value is given by (which is 200 values * 3 axes = 600 total values for the given window in our case). Also note how the values are stored: {x0, y0, z0, x1, y1, z1, …}. You will need to construct a similar array if you are sampling live data from a sensor.

The function expects a struct as an input. So, we set the members here:

run_classifier() will perform any necessary preprocessing steps (such as computing the power spectral density) prior to running inference. The inference results are stored in the second argument (of type . The third parameter is debug, which is used to print out internal states of the preprocessing and inference steps. We leave debugging disabled in this example. The function should return a value equal to EI_IMPULSE_OK if everything ran without error.

If your model has anomaly detection enabled, will be set to 1. We can access the anomaly value via result.anomaly. Additionally, if you are using an object detection impulse, will be set to 1, and bounding box information will be an array stored in .

The C++ Inference SDK library relies on several functions to allocate memory, delay the processor, read current execution time, and print out debugging information. The SDK library provides the necessary declarations in .

By default, Edge Impulse defines these functions for several popular platforms and operating systems, which you can see . In the example throughout this guide, we include the definitions for POSIX and MinGW (refer to the to see how these definitions are included in the build process).

We’ll look at the important lines in our example Makefile. If you are not familiar with Make, we recommend taking a look at . It will walk you through the basics of creating a Makefile and what many of the commands do.

Some of the functions in the library use . As a result, we must support C++11 or later. The C++14 standard is recommended, so we set that in our C++ flags:

The SDK relies on the and libraries, which come with most GNU C/C++ installations. We need to tell the linker to include them from the standard libraries on our system:

edge-impulse-sdk/porting/posix/*.c* and edge-impulse-sdk/porting/mingw32/*.c* point to C++ files that provide implementations for the . If you are using something other than a POSIX-based system or MinGW, you will want to change these files to one of the other or to your own custom definitions for those functions.

To use pure C++ for inference on almost any target with the SDK library, we can use (TFLM). TFLM comes bundled with the downloaded library. All we need to do is include it. Once again, note the compiler flag and source files that are added to the lists:

TFLM is efficient and works with almost any microcontroller or microprocessor target. However, it does not include all of the features and functions found in (TFLite). If you are deploying to a single board computer, smartphone, etc. with TFLite support and you wish to use such functionality, you can enable full TFLite support in the build (as opposed to TFLM).

You can also use to optimize inference for NVIDIA GPUs on boards such as the NVIDIA Jetson Nano.

To enable either TFLite or TensorRT (instead of TFLM), see this . You will need to include different source files and flags. Note that for TensorRT, you will need to install a third-party library from NVIDIA.

The easiest method of running live inference is to fill features[] with your raw sensor data, ensure it’s the correct length and format (e.g. float), and call run_classifier(). However, we did not cover use cases where you might need to run inference on a sliding window of data. Instead of retaining a large window in memory and calling run_classifier() for each new slice of data (which will re-compute features for the whole window), you can use run_classifier_continuous(). This function will remember features from one call to the next so you just need to provide the new data. See for a demonstration on how to run your impulse continuously.

We recognize that the embedded world is full of different build systems and IDEs. While we can’t support every single IDE, we hope that this guide showed how to include the required header and source files to build your project. Additionally, here are for popular platforms to help you run your impulse locally.

This API Reference
here
gcc
g++
GNU Make
LLVM
GNU Make
Homebrew
MinGW-w64
public project
FAQ
calling C++ functions from Python
CMake
here
edge-impulse-sdk/classifier/ei_run_classifier.h
run_classifier()
run_classifier_init()
run_classifier_continuous()
signal_t
edge-impulse-sdk/dsp/numpy_types.h
MFCCs
edge-impulse-sdk/dsp/numpy.h
model_metadata.h
EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE
EI_CLASSIFIER_FREQUENCY
EI_CLASSIFIER_LABEL_COUNT
signal_t
here
Create Impulse
example repo
EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE
run_classifier()
signal_t
ei_impulse_result_t
EI_CLASSIFIER_HAS_ANOMALY
EI_CLASSIFIER_OBJECT_DETECTION
result.bounding_boxes[]
edge-impulse-sdk/porting/ei_classifier_porting.h
DebugLog()
ei_calloc()
ei_free()
ei_malloc()
ei_printf()
ei_printf_float()
ei_read_timer_ms()
ei_read_timer_us()
ei_run_impulse_check_canceled()
ei_sleep()
this guide
lambda functions
math
stdc++
LiteRT (previously Tensorflow Lite) for Microcontrollers
LiteRT (previously Tensorflow Lite)
TensorRT
Makefile
this tutorial
some IDE-specific guides
here
Makefile section
supported platforms
Functions That Require Definition
Studio deployment page with C++ library selected