1 of 100

Documentation

Getting Started

Welcome to Edge Impulse! We enable developers to create the next generation of intelligent device solutions with embedded Machine Learning. In the documentation you'll find user guides, tutorials and API documentation. For support, visit the forums.

If you're new to the idea of embedded machine learning, or machine learning in general, you may enjoy our quick guide: What is embedded ML, anyway?

Get started with any device

Follow these three steps to build your first embedded Machine Learning model - no worries, you can use almost any device to get started.

You'll need some data:
- If you have an existing development board or device, you can collect data with a few lines of code using the Data forwarder or the Edge Impulse for Linux SDK.
- If you want to collect live data from a supported development kit, select your board from the list of fully supported development boards and follow the instructions to connect your board to edge impulse.
- If you already have a dataset, you can upload it via the Uploader.
- If you have a mobile phone you can use it as a sensor to collect data, see Mobile phone.
Try the tutorials on continuous motion recognition, responding to your voice, recognizing sounds from audio, adding sight to your sensors or object detection. These will let you build machine learning models that detect things in your home or office.
After training your model you can run your model on your device:
- If you want to integrate the model with your own firmware or project you can export your complete model (including all signal processing code and machine learning models) to a C++ or Arduino library with no external dependencies (open source and royalty-free), see Running your impulse locally.
- If you have a fully supported development board (or your mobile phone) you can build new firmware - which includes your model - directly from the UI. It doesn't get easier than that!
- If you have a gateway, a computer or a web browser where you want to run your model, you can export to WebAssembly and run it anywhere you can run JavaScript.

Suitable for any type of embedded ML application

We have some great tutorials, but you have full freedom in the models that you design in Edge Impulse. You can plug in new signal processing blocks, and completely new neural networks. See Building custom processing blocks, or click the three dots on a neural network page and select 'Switch to Keras (expert) mode'.

API Documentation

You can access any feature in the Edge Impulse Studio through the Edge Impulse API. We also have the Ingestion service if you want to send data directly, and we have an open Remote management protocol to control devices from the Studio.

Enterprise version

For larger teams, and companies with lots of data we offer an enterprise version of Edge Impulse. The enterprise version offers team collaboration on projects, a dataset builder that makes your internal data available to your whole team, integrations with your cloud buckets, transformation blocks that let you extract ML features from thousands of files in one go, and custom processing and deployment blocks for your organization. You can find documentation under Organizations or contact us via [email protected] for more information.

API and SDK references

The API references for the ingestion service, remote management service, and the studio API; plus SDK documentation for the acquisition and inferencing libraries can be found in the .

Services

SDK documentation

SDKs for Node.js, Python, Go and C++

What is embedded ML, anyway?

A gentle introduction to the exciting field of embedded machine learning.

Machine learning (ML) is a way of writing computer programs. Specifically, it’s a way of writing programs that process raw data and turn it into information that is meaningful at an application level.

For example, one ML program might be designed to determine when an industrial machine has broken down based on readings from its various sensors, so that it can alert the operator. Another ML program might take raw audio data from a microphone and determine if a word has been spoken, so it can activate a smart home device.

Unlike normal computer programs, the rules of ML programs are not determined by a developer. Instead, ML uses specialized algorithms to learn rules from data, in a process known as training.

In a traditional piece of software, an engineer designs an algorithm that takes an input, applies various rules, and returns an output. The algorithm’s internal operations are planned out by the engineer and implemented explicitly through lines of code. To predict breakdowns in an industrial machine, the engineer would need to understand which measurements in the data indicate a problem and write code that deliberately checks for them.

This approach works fine for many problems. For example, we know that water boils at 100°C at sea level, so it’s easy to write a program that can predict whether water is boiling based on its current temperature and altitude. But in many cases, it can be difficult to know the exact combination of factors that predicts a given state. To continue with our industrial machine example, there might be various different combinations of production rate, temperature, and vibration level that might indicate a problem but are not immediately obvious from looking at the data.

To create an ML program, an engineer first collects a substantial set of training data. They then feed this data into a special kind of algorithm, and let the algorithm discover the rules. This means that as ML engineers, we can create programs that make predictions based on complex data without having to understand all of the complexity ourselves.

Through the training process, the ML algorithm builds a model of the system based on the data we provide. We run data through this model to make predictions, in a process called inference.

There are many different types of machine learning algorithms, each with their own unique benefits and drawbacks. Edge Impulse helps engineers select the right algorithm for a given task.

Where can machine learning help?

Machine learning is an excellent tool for solving problems that involve pattern recognition, especially patterns that are complex and might be difficult for a human observer to identify. ML algorithms excel at turning messy, high-bandwidth raw data into usable signals, especially combined with conventional signal processing.

For example, the average person might struggle to recognize the signs of a machine failure given ten different streams of dense, noisy sensor data. However, a machine learning algorithm can often learn to spot the difference.

But ML is not always the best tool for the job. If the rules of a system are well defined and can be easily expressed with hard-coded logic, it’s usually more efficient to work that way.

Limitations of machine learning

Machine learning algorithms are powerful tools, but they can have the following drawbacks:

They output estimates and approximations, not exact answers
ML models can be computationally expensive to run
Training data can be time consuming and expensive to obtain

It can be tempting to try and apply ML everywhere—but if you can solve a problem without ML, it is usually better to do so.

What is embedded ML?

Recent advances in microprocessor architecture and algorithm design have made it possible to run sophisticated machine learning workloads on even the smallest of microcontrollers. Embedded machine learning, also known as TinyML, is the field of machine learning when applied to embedded systems such as these.

There are some major advantages to deploying ML on embedded devices. The key advantages are neatly expressed in the unfortunate acronym BLERP, coined by Jeff Bier. They are:

Bandwidth—ML algorithms on edge devices can extract meaningful information from data that would otherwise be inaccessible due to bandwidth constraints.

Latency—On-device ML models can respond in real-time to inputs, enabling applications such as autonomous vehicles, which would not be viable if dependent on network latency.

Economics—By processing data on-device, embedded ML systems avoid the costs of transmitting data over a network and processing it in the cloud.

Reliability—Systems controlled by on-device models are inherently more reliable than those which depend on a connection to the cloud.

Privacy—When data is processed on an embedded system and is never transmitted to the cloud, user privacy is protected and there is less chance of abuse.

Learn more

The best way to learn about embedded machine learning is to see it for yourself. To train your own model and deploy it to any device, including your mobile phone, follow our Getting Started guide.

Frequently asked questions

The enterprise version of Edge Impulse offers on projects, go to Dashboard, find the Collaborators section, and click the '+' icon. If you have an interesting research or community project we can enable collaboration on the free version of Edge Impulse as well, by emailing [email protected].

You can also create a public version of your Edge Impulse project. This makes your project available to the whole world - including your data, your impulse design, your models, and all intermediate information - and can easily be cloned by anyone in the community. To do so, go to Dashboard, and click Make this project public.

What are the minimum hardware requirements to run the Edge Impulse inferencing library on my embedded device?

The minimum hardware requirements for the embedded device depends on the use case, anything from a Cortex-M0+ for vibration analysis to Cortex-M4F for audio, Cortex-M7 for image classification to Cortex-A for object detection in video, view our for more details.

What frameworks does Edge Impulse use to train the machine learning models?

We use a wide variety of tools, depending on the machine learning model. For neural networks we typically use TensorFlow and Keras, for object detection models we use TensorFlow with Google's Object Detection API, and for 'classic' non-neural network machine learning algorithms we mainly use sklearn. For neural networks you can see (and modify) the Keras code by clicking ⋮, and selecting Switch to expert mode.

Another big part of Edge Impulse are the processing blocks, as they clean up the data, and already extract important features from your data before passing it to a machine learning model. The source code for these processing blocks can be found on GitHub: (and you can build as well).

Is there a downside to enabling the EON Compiler?

The compiles your neural networks to C++ source code, which then gets compiled into your application. This is great if you need the lowest RAM and ROM possible (EON typically uses 30-50% less memory than TensorFlow Lite) but you also lose some flexibility to update your neural networks in the field - as it is now part of your firmware.

By disabling EON we place the full neural network (architecture and weights) into ROM, and load it on demand. This increases memory usage, but you could just update this section of the ROM (or place the neural network in external flash, or on an SD card) to make it easier to update.

Can I use a model that has been trained elsewhere in Edge Impulse?

You cannot import a pretrained model, but you can import your model architecture and then retrain. Add a neural network block to your impulse, go to the block, click ⋮, and select Switch to expert mode. You then have access to the full Keras API.

How does the feature explorer visualize data that has more that 3 dimensions?

Edge Impulse uses (a dimensionality reduction algorithm) to project high dimensionality input data into a 3 dimensional space. This even works for extremely high dimensionality data such as images.

Does Edge impulse integrate with other cloud services?

Yes. The enterprise version of Edge Impulse can integrate directly with your cloud service to access and transform data.

What is the typical power consumption of the Edge Impulse machine learning processes on my device?

Simple answer: To get an indication of time per inference we show performance metrics in every DSP and ML block in the Studio. Multiply this by active power consumption of your MCU to get an indication of power cost per inference.

More complicated answer: It depends. Normal techniques to conserve power still apply to ML, so try to do as little as possible (do you need to classify every second, or can you do it once a minute?), be smart about when to run inference (can there be an external trigger like a motion sensor before you run inference on a camera?), and collect data in a lower power mode (don't run at full speed when sampling low resolution data, and see if your sensor can use an interrupt to wake your MCU - rather than polling).

What is the .eim model format for Edge Impulse for Linux?

See on the Edge Impulse for Linux pages.

How is the labeling of the data performed?

Using the Edge Impulse Studio data acquisition tools (like the or ), you can collect data samples manually with a pre-defined label. If you have a dataset that was collected outside of Edge Impulse, you can upload your dataset using the , , , or . You can then utilize the Edge Impulse Studio to split up your data into labeled chunks, crop your data samples, and more to create high quality machine learning datasets.

Edge Impulse Studio

Dashboard

After creating your Edge Impulse Studio project, you will be directed to the project's dashboard. The dashboard gives a quick overview of your project such as your project ID, the number of devices connected, the amount of data collected, the preferred labeling method, among other editable properties. You can also enable some additional capabilities to your project such as collaboration, making your project public, and showcasing your public projects using Markdown READMEs as we will see.

The figure below shows the various sections and widgets of the dashboard that we will cover here.

1. Showcasing your public projects with Markdown READMEs

The project README enables you to explain the details of your project in a short way. Using this feature, you can add visualizations such as images, GIFs, code snippets, and text to your project in order to bring your colleagues and project viewers up to speed with the important details of your project. In your README you might want to add things like:

What the project does
Why the project is useful
Motivations of the project
How to get started with the project
What sensors and target deployment devices you used
How you plan to improve your project
Where users can get help with your project

To create your first README, navigate to the "about this project" widget and click "add README"

For more README inspiration, check out the public Edge Impulse project tutorials below:

2. Making your project public

To share your private project with the world, and click Make this project public.

By doing this, all of your data, block configurations, intermediate results, and final models will be shared with the world. Your project will be publicly accessible and can be cloned with a single click with the provided URL:

3. Collaboration

You can invite up to three collaborators to join and contribute to your project. To have unlimited collaborators, your project needs to be part of an to access unlimited team collaborations.

To add a collaborator, go to your project's dashboard and find the "Collaborators" widget. Click the '+' icon and type the username or e-mail address of the other user. The user will be invited to create an Edge Impulse account if it doesn't exist.

The user will be automatically added to the project and will get an email notification inviting them to start contributing to your project. To remove a user, simply click on the three dots besides the user then tap ‘Delete’ and they will be automatically removed.

4. Project info

The project info widget shows the project's specifications such as the project ID, labeling method, and latency calculations for your target device.

The project ID is a unique numerical value that identifies your project. Whenever you have any issue with your project on the studio, you can always share your project ID on the for assistance from edge Impulse staff.
On the labeling method dropdown, you need to specify the type of labeling your dataset and model expect. This can be either one label per data item or bounding boxes. Bounding boxes only work for object detection tasks in the studio. Note that if you interchange the labeling methods, learning blocks will appear to be hidden when building your impulse.
One of the amazing Edge Impulse superpowers is the latency calculation component. This is an approximate time in milliseconds that the trained model and DSP operations are going to take during inference based on the selected target device. This hardware in the loop approach ensures that the target deployment device compute resources are not underutilized or over-utilized. It also saves developers' time associated with numerous inference iterations back and forth the studio in search of optimum models.

5. Block Outputs

In the Block Output section, you can download the results of the DSP and ML operations of your impulse.

The downloadable assets include the extracted features, Tensorflow SavedModel, and both quantized and unquantized TensorFlow lite models. This is particularly helpful when you want to perform other operations to the output blocks outside the Edge Impulse studio. For example, if you need a TensorflowJS model, you will just need to download the TensorFlow saved model from the dashboard and convert it to TensorFlowJS model format to be served on a browser.

6. Performance Settings

Changing Performance Settings is only available for enterprise customers

Organizational features are only available for enterprise customers. View our for more information.

This section consists of editable parameters that directly affect the performance of the studio when building your impulse. Depending on the selected or available settings, your jobs can either be fast or slow.

The use of GPU for training and Parallel DSP jobs is currently an internal experimental feature that will be soon released.

7. Administrative Zone

To bring even more flexibility in projects, the administrative zone gives developers the power to enable other additional features that are not found in edge impulse projects by default. Most of these features are usually advanced features intended for organizations or sometimes experimental features.

To activate these features you just need to check the boxes against the specific features you want to use and click save experiments.

8. Danger Zone

The danger zone widget consists of irrevocable actions that let you to:

Delete your project. This action removes all devices, data, and impulses from your project.
Delete all data in this project.
Perform train/test split. This action re-balances your dataset by splitting all your data automatically between the training and testing set and resets the categories for all data
Launch the getting started wizard. This will remove all data, and clear out your impulse.

Devices

There is a wide variety of devices that you can connect to your Edge Impulse project. These devices can help you collect datasets for your project, test your trained ML model and even deploy your ML model directly to your development board with a pre-built binary application (for fully supported development platforms).

On the Devices tab, you'll find a list of all your connected devices and a guide on how to connect new devices that are currently supported by Edge Impulse.

To connect a new device, click on the Connect a new device button on the top right of your screen.

You will get a pop-up with multiple options of devices you can connect to your Edge Impulse project. Available options include:

Connecting a fully supported development board.
Using your mobile phone.
Using your computer (Linux/Mac).
Using the serial port to connect devices that are currently not fully supported.

Uploader

You can upload an already existing dataset to your project directly through the Edge impulse Studio. The data should be in the Data Acquisition Format (CBOR, JSON, CSV), or as WAV, JPG or PNG files.

To upload data using the uploader, go to the Data acquisition page and click on the uploader button as shown in the image below:

When uploading your data, you can choose the category you want your data to fall in i.e training set, testing set or automatically split the dataset between training and testing set. You can also choose whether to infer labels from files name or enter a label of which the files should automatically fall in.

Labeling Queue (Images)

If you are working on an object detection project, you will most likely see "labelling queue" bar on your data acquisition page. The labeling queue shows you all the data that has not been labelled in your dataset.

Can't see the labeling queue? Go to Dashboard, and under 'Project info > Labeling method' select 'Bounding boxes (object detection)'.

In object detection, labelling is the process of adding a bounding box around specific objects in an image so that your machine learning model can learn and infer from it. Edge impulse studio has an inbuilt data annotation tool with AI assisted labelling to assist you in your labelling workflows as we will see.

In the Edge Impulse studio, labelling your data is as easy as dragging a box around the object, then entering a label and saving as shown below.

However, as simple the manual labelling process might look like, it sometimes can become tedious and time consuming especially when dealing with huge datasets. To make your life easier, Edge Impulse studio has an inbuilt AI-assisted labelling feature to automatically assist you in your labelling workflows.

AI Assisted labelling

there are 3 ways you can use to perform AI assisted labelling on the Edge Impulse Studio:

Using yolov5
Using your own model
Using object tracking

Using YoloV5

By utilizing an existing library of pre-trained object detection models from YOLOv5 (trained with the COCO dataset), common objects in your images can quickly be identified and labeled in seconds without needing to write any code!

To label your objects with YOLOv5 classification, click the Label suggestions dropdown and select “Classify using YOLOv5.” If your object is more specific than what is auto-labeled by YOLOv5, e.g. “coffee” instead of the generic “cup” class, you can modify the auto-labels to the left of your image. These modifications will automatically apply to future images in your labeling queue.

Click Save labels to move on to your next raw image, and see your fully labeled dataset ready for training in minutes!

Using your own model

You can also use your own trained model to predict and label your new images. From an existing (trained) Edge Impulse object detection project, upload new unlabeled images from the Data Acquisition tab. Then, from the “Labeling queue”, click the Label suggestions dropdown and select “Classify using ”:

You can also upload a few samples to a new object detection project, train a model, then upload more samples to the Data Acquisition tab and use the AI-Assisted Labeling feature for the rest of your dataset. Classifying using your own trained model is especially useful for objects that are not in YOLOv5, such as industrial objects, etc.

Click Save labels to move on to your next raw image, and see your fully labeled dataset ready for training in minutes using your own pre-trained model!

Using Object tracking

If you have objects that are a similar size or common between images, you can also track your objects between frames within the Edge Impulse Labeling Queue, reducing the amount of time needed to re-label and re-draw bounding boxes over your entire dataset.

Draw your bounding boxes and label your images, then, after clicking Save labels, the objects will be tracked from frame to frame:

Now that your object detection project contains a fully labeled dataset, learn how to train and deploy your model to your edge device: check out our tutorial!

We are excited to see what you build with the AI-Assisted Labeling feature in Edge Impulse, please post your project on our forum or tag us on social media, @Edge Impulse!

Create Impulse

After collecting data for your project, you can now create your Impulse. A complete Impulse will consist of 3 main building blocks: input block, processing block and a learning block.

This view is one of the most important, here you will build your own machine learning pipeline.

Impulse example for movement classification using accelerometer data:

Impulse example for object detection using images:

Input block

The input block indicates the type of input data you are training your model with. This can be time series (audio, vibration, movements) or images.

Time series (audio, vibration, movements)

The input axes field lists all the axis referenced from your training dataset
The window size is the size of the raw features that is used for the training
The window increase is used to artificially create more features (and feed the learning block with more information)
The frequency is automatically calculated based on your training samples. You can modify this value but you currently cannot use values lower than 0.000016 (less than 1 sample every 60s).
Zero-pad data: Adds zero values when raw feature is missing

Below is a sketch to summarize the role of each parameters:

Images

Axes: Images
Image width & height: Most of our pre-trained models work with square images.
Resize mode: You have three options, Squash, Fit to the shortest axis, Fit to the longest axis

Processing Block

A is basically a feature extractor. It consists of DSP (Digital Signal Processing) operations that are used to extract features that our model learns on. These operations vary depending on the type of data used in your project.

You don't have much experience with DSP? No problem, Edge Impulse usually uses a star to indicate the most recommended processing block based on your input data as shown in the image below.

In the case where the available processing blocks aren't suitable for your application, you can and import into your project.

Learning blocks

After adding your , it is now time to add a to make your impulse complete. A learning block is simply a neural network that is trained to learn on your data.

vary depending on what you want your model to do. It can be , , , or . It can also be a (enterprise feature)

Learning blocks available with time-series projects:

Learning blocks available with image projects:

Learning blocks available with object detection projects:

Processing Blocks

Extracting meaningful features from your data is crucial to building small and reliable machine learning models, and in Edge Impulse this is done through processing blocks. We ship a number of processing blocks for common sensor data (such as vibration and audio):

Raw Data
Flatten
Image
Spectral features
Spectrogram
Audio MFE
Audio MFCC
Audio Syntiant
IMU Syntiant

The source code of these blocks are available in the Edge Impulse processing blocks GitHub repository.

If you have a very specific sensor, want to apply custom filters, or are implementing the latest research in digital signal processing, follow our tutorial on Building custom processing blocks.

Raw Data

The Raw Data block generates windows from data samples without any specific signal processing. It is great for signals that have already been pre-processed and if you just need to feed your data into the Neural Network block.

GitHub repository containing all DSP block code: edgeimpulse/processing-blocks.

Raw data parameters

Scaling

Scale axes: Multiplies each axis by this number. This can be used to normalize your data between 0 and 1.

How does the raw data block work?

The Raw Data block retrieves raw samples and applies the Scaling parameter.

Flatten

The Flatten block performs statistical analysis on the signal. It is useful for slow-moving averages like temperature data, in combination with other blocks.

GitHub repository containing all DSP block code: edgeimpulse/processing-blocks.

Flatten parameters

Scaling

Scale axes: Multiplies axes by this number

Method

Average: Calculates the average value for the window
Minimum: Calculates the minimum value in the window
Maximum: Calculates the maximum value in the window
Root-mean square: Calculates the RMS value of the window
Standard deviation: Calculates the standard deviation of the window
Skewness: Calculates the skewness of the window
Kurtosis: Calculates the kurtosis of the window

How does the flatten block work?

The Flatten block first rescales axes of the signal if value is different than 1. Then statistical analysis is performed on each window, computing between 1 and 7 features for each axis, depending on the number of selected methods.

Image

The Image block is dedicated to computer vision applications. It normalizes image data, and optionally reduce the color depth.

GitHub repository containing all DSP block code: edgeimpulse/processing-blocks.

Image parameters

Color depth: Color depth to use (RGB or grayscale)

How does the image block work?

The Image performs normalization, converting each pixel's channel of the image to a float value between 0 and 1. If Grayscale is selected, each pixel is converted to a single value following the ITU-R BT.601 conversion (Y' component only).

Spectral features

The Spectral features block extracts frequency and power characteristics of a signal. Low-pass and high-pass filters can also be applied to filter out unwanted frequencies. It is great for analyzing repetitive patterns in a signal, such as movements or vibrations from an accelerometer.

GitHub repository containing all DSP block code: .

Spectral analysis parameters

Scaling

Scale axes: Multiplies axes by this number to scale data from the sensor

Filter

Type: Type of filter to apply to the raw data (low-pass, high-pass, or none)
Cut-off frequency: Cut-off frequency of the filter in hertz
Order: Order of the Butterworth filter

Spectral power

FFT length: The FFT size
No. of peaks: Number of spectral power peaks to extract
Peaks threshold: Minimum threshold to extract a peak (frequency domain normalized to [0, 1])
Power edges: Splits the power spectral density in various buckets (V**2/Hz unit)

How does the spectral features block work?

The spectral analysis block generates 3 types of features per axis:

The root mean square of the filter output (1 scalar value)
The frequency and height of spectral power peaks
The average power spectral density for each bucket

The raw signal is first scaled up or down based on the Scale axes value and offset by its mean value. A Butterworth filter is then applied (except if None selected); the order of the filter indicates how steep the slope is at the cut-off frequency.

At this point the root mean square of the filter output is added to the features' list.

The filter output is then used to extract:

Spectral power peaks: after performing the FFT, the No. of peaks peaks with the highest magnitude are stored in the features' list (frequency and height). Peaks threshold can be tuned to define a minimum value to extract a peak.
PSD for each bucket: after computing the power spectral density, the signal is divided in power buckets, defined by the Power edges parameter. Each sample frequency of the PSD is added to a power bucket. The power buckets are then averaged and added to the features' list. This process allows to extract a power representation of the signal.

Number of generated features

Let's consider an input signal with 3 axis and the following parameters:

No. of peaks = 3
Power edges = 0.1, 0.5, 1.0, 2.0, 5.0

The number of generated features per axis is:

1 value for RMS of the filter output
6 values for power peaks (frequency & height for each peak)
4 values for power buckets (number of Power edges - 1)

33 features are generated in total for the input signal.

Spectrogram

The Spectrogram processing block extracts time and frequency features from a signal. It performs well on audio data for non-voice recognition use cases, or on any sensor data with continuous frequencies.

GitHub repository containing all DSP block code: .

Spectrogram parameters

Spectrogram

Frame length: The length of each frame in seconds
Frame stride: The step between successive frame in seconds
Frequency bands: The FFT size

Normalization

Noise floor (dB): signal lower than this level will be dropped

How does the spectrogram block work?

It first divides the window in multiple overlapping frames. The size and number of frames can be adjusted with the parameters Frame length and Frame stride. For example with a window of 1 second, frame length of 0.02s and stride of 0.01s, it will create 99 time frames.

Each time frame is then divided in frequency bins using an FFT (Fast Fourier Transform) and we compute its power spectrum. The number of frequency bins equals to the Frequency bands parameter divided by 2 plus 1. We recommend keeping the Frequency bands (a.k.a. FFT size) value as a power of 2 for performances purpose. Finally the Noise floor value is applied to the power spectrum.

The features generated by the Spectrogram block are equal to the number of generated time frames times the number of frequency bins.

Frequency bands and frame length

There is a correlation between the Frequency bands (FFT size) parameter and the frame length. The frame length will be cropped or padded to the Frequency bands value while applying the FFT. For example, with a 8kHz sampling frequency and a time frame of 0.02s, each time frame contains 160 samples (8k * 0.02). If your FFT size is set 128, time frames will be cropped to 128 samples. If your FFT size is set to 256, time frames will be padded with zeros.

Audio MFE

Similarly to the Spectrogram block, the Audio MFE processing block extracts time and frequency features from a signal. However it uses a non-linear scale in the frequency domain, called Mel-scale. It performs well on audio data, mostly for non-voice recognition use cases when sounds to be classified can be distinguished by human ear.

GitHub repository containing all DSP block code: edgeimpulse/processing-blocks.

Audio MFE parameters

Mel-filterbank energy features

Frame length: The length of each frame in seconds
Frame stride: The step between successive frame in seconds
Filter number: The number of triangular filters applied to the spectrogram
FFT length: The FFT size
Low frequency: Lowest band edge of Mel-scale filterbanks
High frequency: Highest band edge of Mel-scale filterbanks

Normalization

Noise floor (dB): signal lower than this level will be dropped

How does the MFE block work?

The features' extractions is similar to the Spectrogram (Frame length, Frame stride, and FFT length parameters are the same) but it adds 2 extra steps.

After computing the spectrogram, triangular filters are applied on a Mel-scale to extract frequency bands. They are configured with parameters Filter number, Low frequency and High frequency to select the frequency band and the number of frequency features to be extracted. The Mel-scale is a perceptual scale of pitches judged by listeners to be equal in distance from one another. The idea is to extract more features (more filter banks) in the lower frequencies, and less in the high frequencies, thus it performs well on sounds that can be distinguished by human ear.

The last step is to perform a local mean normalization of the signal, applying the Noise floor value to the power spectrum.

Audio MFCC

The Audio MFCC blocks extracts coefficients from an audio signal. Similarly to the Audio MFE block, it uses a non-linear scale called Mel-scale. It is the reference block for speech recognition and can also performs well on some non-human voice use cases.

GitHub repository containing all DSP block code: edgeimpulse/processing-blocks.

Audio MFCC parameters

Mel Frequency Cepstral Coefficients

Number of coefficients: Number of cepstral coefficients to keep after applying Discrete Cosine Transform
Frame length: The length of each frame in seconds
Frame stride: The step between successive frame in seconds
Filter number: The number of triangular filters applied to the spectrogram
FFT length: The FFT size
Low frequency: Lowest band edge of Mel-scale filterbanks
High frequency: Highest band edge of Mel-scale filterbanks
Window size: The size of sliding window for local cepstral mean normalization. Windows size must be odd.

Pre-emphasis

Coefficient: The pre-emphasizing coefficient to apply to the input signal (0 equals to no filtering)
Note: Shift has been removed and set to 1 for all future projects. Older & existing projects can still change this value or use an existing value.

How does the MFCC block work?

The features' extractions adds one extra step to the MFE block resulting in a compressed representation of the filterbanks. A Discrete Cosine Transform is applied on each filterbank to extract cepstral coefficients. 13 coefficients are usually retained, the rest are discarded as they represent fast changes not useful for speech recognition.

Audio Syntiant

The Audio Syntiant processing block extracts time and frequency features from a signal. It is similar to the but performs additional processing specific to the Syntiant NDP101 chip. This block can be used only with Syntiant targets.

Audio Syntiant parameters

Log Mel-filterbank energy features

Frame length: The length of each frame in seconds
Frame stride: The step between successive frame in seconds
Filter number (fixed): The number of triangular filters applied to the spectrogram
FFT length (fixed): The FFT size
Low frequency (fixed): Lowest band edge of Mel-scale filterbanks
High frequency (fixed): Highest band edge of Mel-scale filterbanks
Coefficient: Pre-emphasis coefficient

How does the Syntiant block work?

The features' extractions is a proprietary algorithm from Syntiant. However parameters are very close to the . Pre-emphasis coefficient is applied first to amplify higher frequencies. The signal is then divided in overlapping frames, defined by the Frame length and Frame stride to extract speech features.

Sampling frequency

The Audio Syntiant block only supports a 16 kHz frequency. You can adjust the sampling frequency in the "Create Impulse" section.

IMU Syntiant

The IMU Syntiant block rescales raw data to 8 bits values to match the NDP101 chip input requirements.

Parameters

Scaling

Scale 16 bits to 8 bits: Scale data to 8-bits values in the [-1, 1] range, raw data is divided by 2G (2 * 9.80665). Using Edge Impulse official firmwares, this parameter should be enabled as raw data is not rescaled. If this parameter is disabled the data samples will not be rescaled, you should disable this parameter if your raw data samples are already normalized to the [-1, 1] range.

How does the IMU Syntiant block work?

The IMU Syntiant block retrieves raw samples and applies the Scale 16 bits to 8 bits parameter.

Learning Blocks

After extracting meaningful features from the raw signal using signal processing, you can now train your model using a learning block. We provide a number of pre-defined learning blocks:

Classification (Keras).
Regression (Keras).
Anomaly Detection (K-means).
Images Classification (using Transfer Learning).
Object Detection (using MobileNetV2 SSD FPN).
Object Detection (using FOMO).
Custom Transfer Learning (Enterprise feature).

For most of the learning blocks (except K-means Anomaly Detection), you can use the Switch to expert mode button to access the full Keras API for custom architectures, rebalancing your weights, and more.

Anomaly Detection (K-means)

Neural networks are great, but they have one big flaw. They're terrible at dealing with data they have never seen before (like a new gesture). Neural networks cannot judge this, as they are only aware of the training data. If you give it something unlike anything it has seen before it'll still classify as one of the four classes.

Tutorial

Want to see the Anomaly Detection in action? Check out our Continuous Motion Recognition tutorial.

K-means clustering

This method looks at the data points in a dataset and groups those that are similar into a predefined number K of clusters. A threshold value can be added to detect anomalies: if the distance between a data point and its nearest centroid is greater than the threshold value, then it is an anomaly.

The main difficulty resides in choosing K, since data in a time series is always changing and different values of K might be ideal at different times. Besides, in more complex scenarios where there are both local and global outliers, many outliers might pass under the radar and be assigned to a cluster.

Features importance (optional)

In most of your DSP blocks, you have an option to calculate the feature importance. Edge Impulse Studio will then output a Feature Importance graphic that will help you determine which axes and values generated from your DSP block are most significant to analyze when you want to do anomaly detection.

This process of generating features and determining the most important features of your data will further reduce the amount of signal analysis needed on the device with new and unseen data.

Setting up the anomaly detection block

In your anomaly detection block, you can click on the Select suggested axes button to harness the value of the feature importance output.

Here is the process in the background:

Create X number of clusters and group all the data.
For each of these clusters we store the center and the size of the cluster.
During inference we calculate the closest cluster for a new data point, and show the distance from the edge of the cluster. If it’s within a cluster (no anomaly)you thus get a value below 0.

In the above picture, known clusters in are in blue, new classified data in orange. It's clearly outside of any known clusters and can thus be tagged as an anomaly.

Additional resources

Tutorial: Continuous Motion Recognition
Blog post: Advanced Anomaly Detection with Feature Importance

Regression (Keras)

Solving regression problems is one of the most common applications for machine learning models, especially in supervised machine learning. Models are trained to understand the relationship between independent variables and an outcome or dependent variable. The model can then be leveraged to predict the outcome of new and unseen input data, or to fill a gap in missing data.

Prerequisites

Labelling

To build a regression model you collect data as usual, but rather than setting the label to a text value, you set it to a numeric value.

Processing blocks

You can use any of the built-in signal processing blocks to pre-process your vibration, audio or image data, or use custom processing blocks to extract novel features from other types of sensor data.

Train your regression block

You have full freedom in modifying your neural network architecture - whether visually or through writing Keras code.

Number of training cycles: Each time the training algorithm makes one complete pass through all of the training data with back-propagation and updates the model's parameters as it goes, it is known as an epoch or training cycle.
Learning rate: The learning rate controls how much the models internal parameters are updated during each step of the training process. Or you can also see it as how fast the neural network will learn. If the network overfits quickly, you can reduce the learning rate
Validation set size: The percentage of your training set held apart for validation, a good default is 20%
Auto-balance dataset Mix in more copies of data from classes that are uncommon. Might help make the model more robust against overfitting if you have little data for some classes.

Test your regression model

If you want to see the accuracy of your model across your test dataset, go to Model testing. You can adjust the Maximum error percentage by clicking on "⋮" button.

Additional resources

Transfer Learning (Images)

When creating an impulse to solve an image classification problem, you will most likely want to use 'transfer learning' as learning block. This is particularly true especially when working with a relatively small dataset.

Transfer learning is the process of taking features learned from one problem and leveraging it on a new but related problem. Most of the time these features are learned from large scale datasets with common objects hence making it faster & more accurate to tune and adapt to new tasks.

To choose transfer learning as your learning block, go to create impulse and click on 'Add a Learning Block', and select 'Transfer Learning'

To choose your preferred pretrained network, go to Transfer learning on the left side of your screen and click 'choose a different model'. A pop up will appear on your screen with a list of models to choose from as shown in the image below.

Edge Impulse uses state of the art MobileNetV1 & V2 architectures trained on ImageNet dataset as it's pretrained network for you to fine-tune for your specific application. The pretrained networks comes with varying input blocks ranging from 96x96 to 320x320 and both RGB & Grayscale images for you to choose from depending on your application & target deployment hardware.

Neural Network Settings

Before you start training your model, you need to set the following neural network configurations:

Number of training cycles: Each time the training algorithm makes one complete pass through all of the training data with back-propagation and updates the model's parameters as it goes, it is known as an epoch or training cycle.
Learning rate: The learning rate controls how much the models internal parameters are updated during each step of the training process. Or you can also see it as how fast the neural network will learn. If the network overfits quickly, you can reduce the learning rate
Validation set size: The percentage of your training set held apart for validation, a good default is 20%.

You might also need to enable auto balance to prevent model bias or even enable data augmentation to increase the size of your dataset and have more diverse dataset to prevent overfitting.

The preset configurations just don't work for your model? No worries, Expert Mode is for you! The Expert mode gives you full control of your model so that you can configure it however you want. To enable the expert mode, just click on the "⋮" button and toggle the expert mode.

You can use the expert mode to change your loss function, optimizer, print your model architecture and even set an early stopping callback to prevent overfitting your model.

Object Detection (Images)

The two most common image processing problems are image classification and object detection.

Image classification takes an image as an input and outputs what type of object is in the image. This technique works great, even on microcontrollers, as long as we only need to detect a single object in the image.

On the other hand, object detection takes an image and outputs information about the class and number of objects, position, (and, eventually, size) in the image.

Edge Impulse provides two different methods to perform object detection:

Using MobileNetV2 SSD FPN
Using FOMO

Specifications

MobileNetV2 SSD FPN

FOMO

Labelling method

Bounding boxes

Bounding Boxes

Input size

320x320

Square (any size)

Image format

RGB

Greyscale & RGB

Output

Bounding boxes

Centroids

MCU

❌

✅

CPU/GPU

✅

Limitations

- Works best with big objects - Models use high compute resources (in the edge computing world) - Image size is fixed

- Works best when objects have similar sizes & shapes - The size of the objects are not available - Objects should not be too close to each other

MobileNetV2 SSD FPN

It's very hard to build a good working computer vision model from scratch, as you need a wide variety of input data to make the model generalize well, and training such models can take days on a GPU. To make this easier and faster we are using transfer learning. This lets you piggyback on a well-trained model, only retraining the upper layers of a neural network, leading to much more reliable models that train in a fraction of the time and work with substantially smaller datasets.

Tutorial

Want to see the MobileNetV2 SSD FPN-Lite models in action? Check out our Detect objects with bounding boxes tutorial.

How to get started?

To build your first object detection models using MobileNetV2 SSD FPN-Lite :

Create a new project in Edge Impulse.
Make sure to set your labelling method to 'Bounding boxes (object detection)'.
Collect and prepare your dataset as in Object detection
Resize your image to fit 320x320px
Add an 'Object Detection (Images)' block to your impulse.
Under Images, choose RGB.
Under Object detection, select 'Choose a different model' and select 'MobileNetV2 SSD FPN-Lite 320x320'
You can start your training with a learning rate of '0.15'

Click on 'Start training'

MobileNetV2 SSD FPN-Lite 320x320 is available with Edge Impulse for Linux

How does this 🪄 work?

Here, we are using the MobileNetV2 SSD FPN-Lite 320x320 pre-trained model. The model has been trained on the COCO 2017 dataset with images scaled to 320x320 resolution.

In the MobileNetV2 SSD FPN-Lite, we have a base network (MobileNetV2), a detection network (Single Shot Detector or SSD) and a feature extractor (FPN-Lite).

Base network:

MobileNet, like VGG-Net, LeNet, AlexNet, and all others, are based on neural networks. The base network provides high-level features for classification or detection. If you use a fully connected layer and a softmax layer at the end of these networks, you have a classification.

But you can remove the fully connected and the softmax layers, and replace it with detection networks, like SSD, Faster R-CNN, and others to perform object detection.

Detection network:

The most common detection networks are SSD (Single Shot Detection) and RPN (Regional Proposal Network).

When using SSD, we only need to take one single shot to detect multiple objects within the image. On the other hand, regional proposal networks (RPN) based approaches, such as R-CNN series, need two shots, one for generating region proposals, one for detecting the object of each proposal.

As a consequence, SSD is much faster compared with RPN-based approaches but often trades accuracy with real-time processing speed. They also tend to have issues in detecting objects that are too close or too small.

Feature Pyramid Network:

Detecting objects in different scales is challenging in particular for small objects. Feature Pyramid Network (FPN) is a feature extractor designed with feature pyramid concept to improve accuracy and speed.

Retrain model

Training and deploying high performing ML models is usually considered as a continuous process rather than a one time exercise. When you are validating your model and discover an overfit, you might consider adding some more diverse data then perform model retraining while maintaining the initially set DSP and Neural Network block configurations.

Also during inference If you find that the data distribution has drifted significantly from the initial training distribution, it is usually a good common practice to retrain your model on the newer data distribution to keep up with the high model performance.

The “Retrain Model” feature in the Edge Impulse Studio is usually useful when adding new data to your project. It uses known parameters from your selected DSP and ML blocks then uses them to automatically regenerate new features and retrain the Neural Network model in one single step. You can consider this a shortcut for retraining your model since you don’t need to go through all the blocks in your impulse one by one again.

To retrain your model after adding some data, navigate to "Retrain Model" on the studio and click "Train model"

Live Classification

Live classification lets you validate your model with data captured directly from any device or supported development board. This gives you a picture on how your model will perform with real world data. To achieve this, go to Live classification and connect the device or development board you want to capture data from.

Using a fully supported development board

All of your connected devices and sensors will appear under Devices as shown below. The devices can be connected through the or :

Using your mobile phone

To perform live classification using your phone, go to Devices and click Connect a new device then select "Use your mobile phone". Scan the QR code using your phone then click Switch to classification mode and start sampling.

Using your computer

To perform live classification using your computer, go to Devices and click Connect a new device then select "Use your computer". Give permissions on your computer then click Switch to classification mode and start sampling.

Model Testing

When collecting data, we split the dataset into training and testing sets. The model was trained with only the training set, and the testing set is used to validate how well the model will perform on un-seen data. This will ensure that the model has not learned to overfit the training data, which is a common occurrence.

To test your model, go to Model testing, and click Test all. The model will classify all of the test set samples and give you an overall accuracy of how your model performed.

This is also accompanied by a confusion matrix to show you how your models performs for each class.

Evaluating individual samples

To see a classification in detail, go to the individual sample you are want to evaluate and click the three dots next to it, then just select show classification. This will open a new window that will display the expected outcome, and the predicted output of your model with its accuracy. This detailed view can also give you a hint on why an item has been misclassified.

Setting confidence threshold

Every learning block has a threshold. This can be the minimum confidence that a neural network needs to have, or the maximum anomaly score before a sample is tagged as an anomaly. You can configure these thresholds to tweak the sensitivity of these learning blocks. This affects both live classification and model testing.

Organizations

Your Edge Impulse organization helps your team with the full lifecycle of your TinyML deployment. It contains tools to collect and maintain large datasets, allows your data scientists to quickly access relevant data through their familiar tools, adds versioning and traceability to your machine learning models, and lets you quickly create new Edge Impulse projects for on-device deployment.

Only available for enterprise customers

Organizational features are only available for enterprise customers. View our pricing for more information.

To get started, follow these tutorials:

Collaborating on projects - to work with your colleagues on one project.
Building your first dataset - to build up a shared dataset for your organization.
Upload portals - to allow external parties to securely contribute data to your datasets.
Creating a transformation block - to quickly extract features from your dataset.
Building deployment blocks - to create custom deployment targets for your products.
Hosting custom DSP blocks - to create and host your custom signal processing techniques and use it directly in your projects.
Adding custom transfer learning models - to use your custom neural networks architectures and load pre-trained weights.

Collaborating on projects

Within an you can work on one project with multiple people. These can be colleagues, outside researchers, or even members of the community. They will only get access to the specific data in the project, and not to any of the raw data in your organizational datasets.

Only available for enterprise customers

Organizational features are only available for enterprise customers. for more information.

To give someone access, go to your project's dashboard, and find the "Collaborators" widget. Click the '+' icon, and type the username or e-mail address of the other user. This user needs to have an Edge Impulse account already.

Upload portals

Upload portals are a secure way to let external parties upload data to your datasets. Through an upload portal they get an easy user interface to add data, but they have no access to the content of the dataset, nor can they delete any files. Data that is uploaded through the portal can be stored on-premise or in your own cloud infrastructure.

In this tutorial we'll set up an upload portal, show you how to add new data, and how to show this data in Edge Impulse for further processing.

Only available for enterprise customers

Organizational features are only available for enterprise customers. View our pricing for more information.

1. Configuring a storage bucket

Data is stored in storage buckets, which can either be hosted by Edge Impulse, or in your own infrastructure. Follow the Building your first dataset tutorial to set up your storage bucket.

2. Creating an upload portal

With your storage bucket configured you're ready to set up your first upload portal. In your organization go to Data > Upload portals and choose Create new upload portal. Here, select a name, a description, the storage bucket, and a path in the storage bucket.

Note: You'll need to enable CORS headers on the bucket. If these are not configured you'll get prompted with instructions. Talk to your user success engineer (when your data is hosted by Edge Impulse), or your system administrator to configure this.

After your portal is created a link is shown. This link contains an authentication token, and can be shared directly with the third party.

Click the link to open the portal. If you ever forget the link: no worries. Click the ⋮ next to your portal, and choose View portal.

3. Uploading data to the portal

To upload data you can now drag & drop files or folders to the drop zone on the right, or use Create new folder to first create a folder structure. There's no limit to the amount of files you can upload here, and all files are hashed, so if you upload a file that's already present the file will be skipped.

Note: Files with the same name but with a different hash are overwritten.

4. Adding the data to your dataset

To view the uploaded data in your dataset now go to your organization, and select Data. Then select Add data > Add dataset from bucket. Here, enter a name for the dataset, and select the same bucket path as you used for the portal. Then click Add data.

You now have the data in your Edge Impulse organization, ready to be applied to your next machine learning project.

You can run the 'Add dataset from bucket' step any time new data was added. Data is automatically de-duplicated and new files will be picked up. You can also automate this through our API.

5. Recap

If you need a secure way for external parties to contribute data to your datasets then upload portals are the way to go. They offer a friendly user interface, upload data directly into your storage buckets, and give you an easy way to use the data directly in Edge Impulse. 🚀

Any questions, or interested in the enterprise version of Edge Impulse? Contact us for more information.

Creating transformation blocks

Transformation blocks take raw data from your organizational datasets and convert the data into files that can be loaded in an Edge Impulse project. You can use transformation blocks to only include certain parts of individual data files, calculate long-running features like a running mean or derivatives, or efficiently generate features with different window lengths. Transformation blocks can be written in any language, and run on the Edge Impulse infrastructure.

Only available for enterprise customers

Organizational features are only available for enterprise customers. View our pricing for more information.

Transformation blocks can output data to either:

A project - here the data in your dataset is placed in an Edge Impulse project, and you'll have all the normal features from the studio available to build your machine learning models. This is great if you already have an idea what you want with the data, and are looking for a reproducible pipeline.
Back in the dataset - here data is placed back in the dataset. This is great for extracting long running features, batch jobs, or combining data from multiple sources - even when you don't want to place the data in a project yet.

For both of these you can find tutorials here:

Project transformation block
Dataset transformation block

Development Platforms

Overview

There is a list of development boards that are fully supported by Edge Impulse. These boards come with a special firmware which enables data collection from all their sensors, allows you to build new ready-to-go binaries that include your trained impulse, and come with examples on integrating your impulse with your custom firmware. These boards are the perfect way to start building Machine Learning solutions on real embedded hardware.

Different development board? No problem, you can always collect data using the Data forwarder or the Edge Impulse for Linux SDK, and deploy your model back to the device with the Running your impulse locally tutorials. Also, if you feel like porting your board, use this Porting guide.

Just want to experience Edge Impulse? You can also use your Mobile phone!

Officially supported MCU targets

Himax WE-I Plus

The Himax WE-I Plus is a tiny development board with a camera, a microphone, an accelerometer and a very fast DSP - and it's fully supported by Edge Impulse. You'll be able to sample raw data, build models, and deploy trained machine learning models directly from the studio. It's available for around 65 USD from Sparkfun.

The Edge Impulse firmware for this development board is open source and hosted on GitHub: edgeimpulse/firmware-himax-we-i-plus.

Installing dependencies

To set this device up in Edge Impulse, you will need to install the following software:

Edge Impulse CLI.
On Linux:
- GNU Screen: install for example via sudo apt install screen.

Problems installing the CLI?

See the Installation and troubleshooting guide.

Connecting to Edge Impulse

With all the software in place it's time to connect the development board to Edge Impulse.

1. Connect the development board to your computer

Use a micro-USB cable to connect the development board to your computer.

2. Update the firmware

The development board does not come with the right firmware yet. To update the firmware:

Download the latest Edge Impulse firmware, and unzip the file.
Open the flash script for your operating system (flash_windows.bat, flash_mac.command or flash_linux.sh) to flash the firmware.
Wait until flashing is complete, and press the RESET button once to launch the new firmware.

3. Setting keys

From a command prompt or terminal, run:

edge-impulse-daemon

This will start a wizard which will ask you to log in, and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

Alternatively, recent versions of Google Chrome and Microsoft Edge can collect data directly from your development board, without the need for the Edge Impulse CLI. See this blog post for more information.

4. Verifying that the device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to your Edge Impulse project, and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Looking to connect different sensors? The Data forwarder lets you easily send data from any sensor into Edge Impulse.

Troubleshooting

All licenses are in use by other developers.

If you export to the Himax WE-I Plus you could receive the error: "All licenses are in use by other developers.". Unfortunately we have a limited number of licenses for the MetaWare compiler and these are shared between all Studio users. Try again in a little bit, or export your project as a C++ Library, add it to the edgeimpulse/firmware-himax-we-i-plus project and compile locally.

COM port not detected

If no device shows up in your OS (ie: COMxx, /dev/tty.usbxx) after connecting the board and your USB cable supports data transfer, you may need to install FTDI VCP driver.

Infineon CY8CKIT-062S2 Pioneer Kit

CY8CKIT-062S2 Pioneer Kit and CY8CKIT-028-SENSE expansion kit required

This guide assumes you have the attached to a

The Infineon Semiconductor enables the evaluation and development of applications using the PSoC 62 Series MCU. This low-cost hardware platform enables the design and debug of the PSoC 62 MCU and the Murata 1LV Module (CYW43012 Wi-Fi + Bluetooth Combo Chip). The PSoC 6 MCU is Infineon' latest, ultra-low-power PSoC specifically designed for wearables and IoT products. The board features a PSoC 6 MCU, and a CYW43012 Wi-Fi/Bluetooth combo module. Infineon CYW43012 is a 28nm, ultra-low-power device that supports single-stream, dual-band IEEE 802.11n-compliant Wi-Fi MAC/baseband/radio and Bluetooth 5.0 BR/EDR/LE. When paired with the , the PSoC® 62S2 Wi-Fi® BLUETOOTH® Pioneer Kit can be used to easily interface a variety of sensors with the PSoC™ 6 MCU platform, specifically targeted for audio and machine learning applications which are fully supported by Edge Impulse! You'll be able to sample raw data as well as build and deploy trained machine learning models to your PSoC® 62S2 Wi-Fi® BLUETOOTH® Pioneer Kit, directly from the Edge Impulse Studio.

The Edge Impulse firmware for this development board is open source and hosted on GitHub: .

Installing dependencies

To set this device up with Edge Impulse, you will need to install the following software:

. A utility program we will use to flash firmware images onto the target.
The which will enable you to connect your CY8CKIT-062S2 Pioneer Kit directly to Edge Impulse Studio, so that you can collect raw data and trigger in-system inferences.

Problems installing the CLI?

See the guide.

Updating the firmware

Edge Impulse Studio can collect data directly from your CY8CKIT-062S2 Pioneer Kit and also help you trigger in-system inferences to debug your model, but in order to allow Edge Impulse Studio to interact with your CY8CKIT-062S2 Pioneer Kit you first need to flash it with our .

1. Download the base firmware image

, and unzip the file. Once downloaded, unzip it to obtain the firmware-infineon-cy8ckit-062s2.hex file, which we will be using in the following steps.

2. Connect the CY8CKIT-062S2 Pioneer Kit to your computer

Use a micro-USB cable to connect the CY8CKIT-062S2 Pioneer Kit to your development computer (where you downloaded and installed ).

3. Load the base firmware image with Infineon CyProgrammer

You can use to flash your CY8CKIT-062S2 Pioneer Kit with our . To do this, first select your board from the dropdown list on the top left corner. Make sure to select the item that starts with CY8CKIT-062S2-43012:

Then select the base firmware image file you downloaded in the first step above (i.e., the file named firmware-infineon-cy8ckit-062s2.hex). You can now press the Connect button to connect to the board, and finally the Program button to load the base firmware image onto the CY8CKIT-062S2 Pioneer Kit.

Keep Handy

will be needed to upload any other project built on Edge Impulse, but the base firmware image only has to be loaded once.

Connecting to Edge Impulse

With all the software in place, it's time to connect the CY8CKIT-062S2 Pioneer Kit to Edge Impulse.

1. Connect the development board to your computer

Use a micro-USB cable to connect the development board to your computer.

2. Setting keys

From a command prompt or terminal, run:

This will start a wizard which will ask you to log in, and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

Alternatively, recent versions of Google Chrome and Microsoft Edge can collect data directly from your development board, without the need for the Edge Impulse CLI. See for more information.

3. Verifying that the device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to , and click Devices on the left sidebar. The device will be listed there:

Next steps: Build a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Looking to connect different sensors? The lets you easily send data from any sensor into Edge Impulse.

Nordic Semi nRF5340 DK

The Nordic Semiconductor nRF5340 DK is a development board with dual Cortex-M33 microcontrollers, QSPI flash, and an integrated BLE radio - and it's fully supported by Edge Impulse. You'll be able to sample raw data, build models, and deploy trained machine learning models directly from the studio. As the nRF5340 DK does not have any built-in sensors we recommend you to pair this development board with the shield (with a MEMS accelerometer and a MEMS microphone). The nRF5340 DK is available for around 50 USD from a .

If you don't have the X-NUCLEO-IKS02A1 shield you can use the to capture data from any other sensor, and then follow the tutorial to run your impulse. Or, you can modify the example firmware (based on nRF Connect) to interact with other accelerometers or PDM microphones that are supported by Zephyr.

The Edge Impulse firmware for this development board is open source and hosted on GitHub: .

Installing dependencies

To set this device up in Edge Impulse, you will need to install the following software:

.
On Linux:
- GNU Screen: install for example via sudo apt install screen.

Problems installing the CLI?

See the guide.

Connecting to Edge Impulse

With all the software in place it's time to connect the development board to Edge Impulse.

1. Plugging in the X-NUCLEO-IKS02A1 MEMS expansion shield

Remove the pin header protectors on the nRF5340 DK and plug the X-NUCLEO-IKS02A1 shield into the development board.

Note: Make sure that the shield does not touch any of the pins in the middle of the development board. This might cause issues when flashing the board or running applications.

2. Connect the development board to your computer

Use a micro-USB cable to connect the development board to your computer. There are two USB ports on the development board, use the one on the short side of the board. Then, set the power switch to 'on'.

3. Update the firmware

The development board does not come with the right firmware yet. To update the firmware:

The development board is mounted as a USB mass-storage device (like a USB flash drive), with the name JLINK. Make sure you can see this drive.
.
Drag the nrf5340-dk.bin file to the JLINK drive.
Wait 20 seconds and press the BOOT/RESET button.

4. Setting keys

From a command prompt or terminal, run:

This starts a wizard which asks you to log in and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

The nRF5340 DK exposes multiple UARTs. If prompted, choose the bottom one:

Alternatively, recent versions of Google Chrome and Microsoft Edge can collect data directly from your development board, without the need for the Edge Impulse CLI. See for more information.

5. Verifying that the device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to , and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Looking to connect different sensors? The lets you easily send data from any sensor into Edge Impulse.

Troubleshooting

Failed to flash

If your board fails to flash new firmware (a FAIL.txt file might appear on the JLINK drive) you can also flash using nrfjprog.

Install the .
Flash new firmware via:

Nordic Semi nRF9160 DK

The Nordic Semiconductor nRF9160 DK is a development board with an nRF9160 SIP incorporating a Cortex M-33 for your application, a full LTE-M/NB-IoT modem with GPS along with 1 MB of flash and 256 KB RAM. It also includes an nRF52840 board controller with Bluetooth Low Energy connectivity. The Development Kit is fully supported by Edge Impulse. You'll be able to sample raw data, build models, and deploy trained machine learning models directly from the studio. As the nRF9160 DK does not have any built-in sensors we recommend you to pair this development board with the shield (with a MEMS accelerometer and a MEMS microphone). The nRF9160 DK is available for around 150 USD from a variety of distributors including .

The Edge Impulse firmware for this development board is open source and hosted on GitHub: .

Installing dependencies

To set this device up in Edge Impulse, you will need to install the following software:

.
On Linux:
- GNU Screen: install for example via sudo apt install screen.

Problems installing the CLI?

See the guide.

Connecting to Edge Impulse

With all the software in place it's time to connect the development board to Edge Impulse.

1. Plugging in the X-NUCLEO-IKS02A1 MEMS expansion shield

Remove the pin header protectors on the nRF9160 DK and plug the X-NUCLEO-IKS02A1 shield into the development board.

Note: Make sure that the shield does not touch any of the pins in the middle of the development board. This might cause issues when flashing the board or running applications. You can also remove the shield before flashing the board.

2. Connect the development board to your computer

3. Update the firmware

The development board does not come with the right firmware yet. To update the firmware:

The development board is mounted as a USB mass-storage device (like a USB flash drive), with the name JLINK. Make sure you can see this drive.
Install the .
.
Flash the board controller, you only need to do this once. Go to step 4 if you've performed this step before.
- Ensure that the PROG/DEBUG switch is in nRF52 position.
- Copy board-controller.bin to the JLINK mass storage device.

Flash the application:
- Ensure that the PROG/DEBUG switch is in nRF91 position.
- Run the flash script for your Operating System.
Wait 20 seconds and press the BOOT/RESET button.

4. Setting keys

From a command prompt or terminal, run:

This starts a wizard which asks you to log in and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

The nRF9160 DK exposes multiple UARTs. If prompted, choose the top one:

Alternatively, recent versions of Google Chrome and Microsoft Edge can collect data directly from your development board, without the need for the Edge Impulse CLI. See for more information.

5. Verifying that the device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to , and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Looking to connect different sensors? The lets you easily send data from any sensor into Edge Impulse.

Open MV Cam H7 Plus

The OpenMV Cam is a small and low-power development board with a Cortex-M7 microcontroller supporting MicroPython, a μSD card socket and a camera module capable of taking 5MP images - and it's fully supported by Edge Impulse. You'll be able to sample raw data, build models, and deploy trained machine learning models through the studio and the OpenMV IDE. It is available for 80 USD directly from OpenMV.

Installing dependencies

To set this device up in Edge Impulse, you will need to install the following software:

Edge Impulse CLI.
OpenMV IDE.

Problems installing the CLI?

See the installation and troubleshooting guide.

Connecting to Edge Impulse

With all the software in place it's time to connect the development board to Edge Impulse. To make this easy we've put some tutorials together which takes you through all the steps to acquire data, train a model, and deploy this model back to your device.

Adding sight to your sensors - end-to-end tutorial.
Detecting objects using FOMO.
Collecting image data with the OpenMV Cam H7 Plus - collecting datasets using the OpenMV IDE.
Running your impulse on your OpenMV camera - run your trained impulse on the OpenMV Cam H7 Plus.

Syntiant Tiny ML Board

The Syntiant TinyML Board is a tiny development board with a microphone and accelerometer, USB host microcontroller and an always-on Neural Decision Processor™, featuring ultra low-power consumption, a fully connected neural network architecture, and fully supported by Edge Impulse. You'll be able to sample raw data, build models, and deploy trained embedded machine learning models directly from the Edge Impulse studio to create the next generation of low-power, high-performance audio interfaces.

The Edge Impulse firmware for this development board is open source and hosted on GitHub.

IMU data acquisition - SD Card

An SD Card is required to use IMU data acquisition as the internal RAM of the MCU is too small. You don't need the SD Card for inferencing only or for audio projects.

Installing dependencies

To set this device up in Edge Impulse, you will need to install the following software:

Connecting to Edge Impulse

1. Download the firmware

Select one of the 2 firmwares below for audio or IMU projects:

Insert SD Card if you need IMU data acquisition and connect the USB cable to your computer. Double-click on the script for your OS. The script will flash the Arduino firmware and a default model on the NDP101 chip.

Flashing issues

0x000000: read 0x04 != expected 0x01

Some flashing issues can occur on the Serial Flash. In this case, open a Serial Terminal on the TinyML board and send the command: :F. This will erase the Serial Flash and should fix the flashing issue.

2. Connect the development board to your computer

Connect the Syntiant TinyML Board directly to your computer's USB port. Linux, Mac OS, and Windows 10 platforms are supported.

3. Setup the Syntiant TinyML Board to collect data

Audio - USB microphone (macOS/Linux only)

Check that the Syntiant TinyML enumerates as "TinyML" or "Arduino MKRZero". For example, in Mac OS you'll find it under System Preferences/Sound:

Audio acquisition - Windows OS

Using the Syntiant TinyML board as an external microphone for data collection doesn't currently work on Windows OS.

IMU

From a command prompt or terminal, run:

edge-impulse-daemon

This will start a wizard which will ask you to log in and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

That's all! Your device is now connected to Edge Impulse. To verify this, go to your Edge Impulse project, and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model and evaluate it using the Syntiant TinyML Board with this tutorial:

FAQ

How to use Arduino-CLI with macOS M1 chip? You will need to install Rosetta2 to run the Arduino-CLI. See details on Apple website.
Board is detected as MKRZero and not TinyML: when compiling using the Arduino IDE, the board name will change from TinyML to MKRZero as it automatically retrieves the name from the board type. This doesn't affect the execution of the firmware.
How to label my classes? The NDP101 chip expects one and only negative class and it should be the last in the list. For instance, if your original dataset looks like: yes, no, unknown, noise and you only want to detect the keyword 'yes' and 'no', merge the 'unknown' and 'noise' labels in a single class such as z_openset (we prefix it with 'z' in order to get this class last in the list).

Officially supported CPU/GPU targets

macOS

You can use your Intel or M1-based Mac computer as a fully-supported development environment for Edge Impulse for Linux. This lets you sample raw data, build models, and deploy trained machine learning models directly from the Studio. If you have a Macbook, the webcam and microphone of your system are automatically detected and can be used to build models.

1. Connecting to your Mac

To connect your Mac to Edge Impulse:

Install .
Install .
Open a terminal window and install the dependencies:

Last, install the Edge Impulse CLI:

Problems installing the CLI?

See the guide.

2. Connecting to Edge Impulse

With the software installed, open a terminal window and run::

This will start a wizard which will ask you to log in, and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

3. Verifying that your device is connected

That's all! Your Mac is now connected to Edge Impulse. To verify this, go to , and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Looking to connect different sensors? Our lets you easily send data from any sensor and any programming language (with examples in Node.js, Python, Go and C++) into Edge Impulse.

Deploying back to device

To run your impulse locally, just open a terminal and run:

This will automatically compile your model with full hardware acceleration, download the model to your Raspberry Pi, and then start classifying. Our has examples on how to integrate the model with your favourite programming language.

Image model?

If you have an image model then you can get a peek of what your device sees by being on the same network as your device, and finding the 'Want to see a feed of the camera and live classification in your browser' message in the console. Open the URL in a browser and both the camera feed and the classification are shown:

Linux x86_64

You can use your Linux x86_64 device or computer as a fully-supported development environment for Edge Impulse for Linux. This lets you sample raw data, build models, and deploy trained machine learning models directly from the Studio. If you have a webcam and microphone plugged into your system, they are automatically detected and can be used to build models.

Instruction set architectures

If you are not sure about your instruction set architectures, use:

1. Installing dependencies

To set this device up in Edge Impulse, run the following commands:

Ubuntu/Debian:

3. Connecting to Edge Impulse

With all software set up, connect your camera and microphone to your operating system (see 'Next steps' further on this page if you want to connect a different sensor), and run:

This will start a wizard which will ask you to log in, and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

4. Verifying that your device is connected

That's all! Your machine is now connected to Edge Impulse. To verify this, go to , and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Looking to connect different sensors? Our lets you easily send data from any sensor and any programming language (with examples in Node.js, Python, Go and C++) into Edge Impulse.

Deploying back to device

To run your impulse locally run on your Linux platform:

This will automatically compile your model with full hardware acceleration, download the model to your local machine, and then start classifying. Our has examples on how to integrate the model with your favourite programming language.

Image model?

NVIDIA Jetson Nano

The Jetson Nano is an embedded Linux dev kit featuring a GPU accelerated processor (NVIDIA Tegra) targeted at edge AI applications. You can easily add a USB external microphone or camera - and it's fully supported by Edge Impulse. You'll be able to sample raw data, build models, and deploy trained machine learning models directly from the Studio. The Jetson Nano is available from 59 USD from a wide range of distributors, including .

In addition to the Jetson Nano we recommend that you also add a camera and / or a microphone. Most popular USB webcams work fine on the development board out of the box.

Powering your Jetson

Although powering your Jetson via USB is technically supported, some users report on forums that they have issues using USB power. If you have any issues such as the board resetting or becoming unresponsive, consider powering via a 5V, 4A power supply on the DC barrel connector. Don't forget to change the jumper! is an example power supply for sale.

An added bonus to powering via the DC barrel plug, you can carry out your first boot w/o an external monitor or keyboard.

1. Setting up your Jetson Nano

Depending on your hardware, follow NVIDIA's setup instructions ( or ) for both "Write Image to SD Card" and "Setup and First Boot." When finished, you should have a bash prompt via the USB serial port, or using an external monitor and keyboard attached to the Jetson. You will also need to connect your Jetson to the internet via the Ethernet port (there is no WiFi on the Jetson). (After setting up the Jetson the first time via keyboard or the USB serial port, you can SSH in.)

2. Installing dependencies

Make sure your ethernet is connected to the Internet

Issue the following command to check:

The result should look similar to this:

Running the setup script

To set this device up in Edge Impulse, run the following commands (from any folder). When prompted, enter the password you created for the user on your Jetson in step 1. The entire script takes a few minutes to run (using a fast microSD card).

3. Connecting to Edge Impulse

With all software set up, connect your camera and microphone to your Jetson (see 'Next steps' further on this page if you want to connect a different sensor), and run:

This will start a wizard which will ask you to log in, and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

4. Verifying that your device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to , and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Looking to connect different sensors? Our lets you easily send data from any sensor and any programming language (with examples in Node.js, Python, Go and C++) into Edge Impulse.

Deploying back to device

To run your impulse locally, just connect to your Jetson again, and run:

This will automatically compile your model with full hardware acceleration, download the model to your Jetson, and then start classifying. Our has examples on how to integrate the model with your favourite programming language.

Image model?

Running models on the GPU

Due to some incompatibilities we don't run models on the GPU by default. You can enable this by following the in the C++ SDK.

Troubleshooting

edge-impulse-linux reports "[Error: Input buffer contains unsupported image format]"

This is probably caused by a missing dependency on libjpeg. If you run:

The end of the output should show support for file import/export with libjpeg, like so:

If you don't see jpeg support as "yes", rerun the setup script and take note of any errors.

edge-impulse-linux reports "Failed to start device monitor!"

If you encounter this error, ensure that your entire home directory is owned by you (especially the .config folder):

Long warm-up time and under-performance

By default, the Jetson Nano enables a number of aggressive power saving features to disable and slow down hardware that is detected to be not in use. Experience indicates that sometimes the GPU cannot power up fast enough, nor stay on long enough, to enjoy best performance. You can run a script to enable maximum performance on your Jetson Nano.

ONLY DO THIS IF YOU ARE POWERING YOUR JETSON NANO FROM A DEDICATED POWER SUPPLY. DO NOT RUN THIS SCRIPT WHILE POWERING YOUR JETSON NANO THROUGH USB.

To enable maximum performance, run:

Community boards

Seeed Wio Terminal

Community board

This is a community board by Seeed Studios, and it's not maintained by Edge Impulse. For support head to the .

The Seeed Wio Terminal is a development board from Seeed Studios with a Cortex-M4 microcontroller, motion sensors, an LCD display, and Grove connectors to easily connect external sensors. Seeed Studio has added support for this development board to Edge Impulse, so you can sample raw data and build machine learning models from the studio. The board is available for 29 USD directly from .

Setting up your development board

To set up your Seeed Wio Terminal, follow this guide: .

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with this full end-to-end course from Seeed's EDU team: .

Looking to connect different sensors? The lets you easily send data from any sensor into Edge Impulse.

Deploying back to device

With the impulse designed, trained and verified you can deploy this model back to your Wio Terminal. This makes the model run without an internet connection, minimizes latency, and runs with minimum power consumption. Edge Impulse can package up the complete impulse - including the signal processing code, neural network weights, and classification code - up in a single library that you can run on your development board.

The easiest way to deploy your impulse to the Seeed Wio Terminal is via an Arduino library. See for more information.

Arducam Pico4ML TinyML Dev Kit

Community board

This is a community board by Arducam, and it's not maintained by Edge Impulse. For support head to the Arducam support page.

The Arducam Pico4ML TinyML Dev Kit is a development board from Arducam with a RP2040 microcontroller, QVGA camera, bluetooth module (depending on your version), LCD screen, onboard microphone, accelerometer, gyroscope, and compass. Arducam has created in depth tutorials on how to get started using the Pico4ML Dev Kit with Edge Impulse, including how to collect new data and how to train and deploy your Edge Impulse models to the Pico4ML. The Arducam Pico4ML TinyML Dev Kit has two versions, the version with BLE is available for 55 USD and the version without BLE is available for 50 USD.

Setting up your development board

To set up your Arducam Pico4ML TinyML Dev Kit, follow this guide: Arducam: How to use Edge Impulse to train machine learning models for Raspberry Pico.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with the Edge Impulse continuous motion recognition tutorial.

Or you can follow Arducam's tutorial on How to build a Magic Wand with Edge Impulse for Arducam Pico4ML-BLE.

Looking to connect different sensors? The Data forwarder lets you easily send data from any sensor into Edge Impulse.

Deploying back to device

With the impulse designed, trained and verified you can deploy this model back to your Arducam Pico4ML TinyML Dev Kit. This makes the model run without an internet connection, minimizes latency, and runs with minimum power consumption. Edge Impulse can package the complete impulse - including the signal processing code, neural network weights, and classification code - up into a single library that you can run on your development board. See the end of the Arducam's How to use Edge Impulse to train machine learning models for Raspberry Pico tutorial for more information on deploying your model onto the device.

Blues Wireless Swan

Community board

This is a community board by Blues Wireless, and is not maintained by Edge Impulse. For support head to the .

The Blues Wireless Swan is a development board featuring a 120MHz ARM Cortex-M4 from STMicroelectronics with 2MB of flash and 640KB of RAM. Blues Wireless has created an on how to get started using the Swan with Edge Impulse, including how to collect new data from a triple axis accelerometer and how to train and deploy your Edge Impulse models to the Swan. For more details and ordering information, visit the Blues Wireless Swan .

Setting up your development board

To set up your Blues Wireless Swan, follow this complete guide: .

Next steps: building a machine learning model

The Blues Wireless Swan will guide you through how to create a simple classification model with an accelerometer designed to analyze movement over a brief period of time (2 seconds) and infer how the motion correlates to one of the following four states:

Idle (no motion)
Circle
Slash
An up-and-down motion in the shape of the letter "W"

For more insight into using a triple axis accelerometer to build an embedded machine learning model visit the .

Looking to connect different sensors? The lets you easily send data from any sensor into Edge Impulse.

Deploying back to device

With the impulse designed, trained and verified you can deploy this model back to your Blues Wireless Swan. This makes the model run without an internet connection, minimizes latency, and runs with minimum power consumption. Edge Impulse can package the complete impulse - including the signal processing code, neural network weights, and classification code - up into a single library that you can run on your development board. See the end of the Blues Wireless' [Using Swan with Edge Impulse] (https://dev.blues.io/get-started/swan/using-swan-with-edge-impulse) tutorial for more information on deploying your model onto the device.

Mobile Phone

You can use any smartphone with a modern browser as a fully-supported client for Edge Impulse. You'll be able to sample raw data (from the accelerometer, microphone and camera), build models, and deploy machine learning models directly from the studio. Your phone will behave like any other device, and data and models that you create using your mobile phone can also be deployed to embedded devices.

The mobile client is open source and hosted on GitHub: . As there are thousands of different phones and operating system versions we'd love to hear from you there if something is amiss.

There's also a video version of this tutorial:

Connecting to Edge Impulse

To connect your mobile phone to Edge Impulse, go to the , and head to the Devices page. Then click Connect a new device.

Select Mobile phone, and a QR code will appear. Either scan the QR code with the camera of your phone - many phones will automatically recognize the code and offer to open a browser window - or click on the link above the QR code to open the mobile client.

This opens the mobile client, and registers the device directly. On your phone you see a Connected message.

That's all! Your device is now connected to Edge Impulse. If you return to the Devices page in the studio, your phone now shows as connected. You can change the name of your device by clicking on ⋮.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

Your phone will show up like any other device in Edge Impulse, and will automatically ask permission to use sensors.

No data (using Chrome on Android)?

You might need to enable motion sensors in the Chrome settings via Settings > Site settings > Motion sensors.

Deploying back to device

With the impulse designed, trained and verified you can deploy this model back to your phone. This makes the model run without an internet connection, minimizes latency, and runs with minimum power consumption. Edge Impulse can package up the complete impulse - including the signal processing code, neural network weights, and classification code - up in a single WebAssembly package that you can straight from the browser.

To do so, just click Switch to classification mode at the bottom of the mobile client. This will first build the impulse, and then samples data from the sensor, run the signal processing code, and then classify the data:

Victory! You're now running your machine learning model locally in your browser - you can even turn on airplane mode and the model will continue running. You can also package to include in your own website or Node.js application. 🚀

Porting Guide

If you have a development board that is not officially supported by Edge Impulse, no problem. This guide contains information on connecting any device to Edge Impulse.

Collecting data

Edge Impulse can handle data from any device, whether it's coming from a new development board or from a device that has been in production for years. Just post your data to the ingestion service and it will automatically show up in the studio. You can either do this directly from your device (if it has an IP connection) or through an intermediate protocol like a phone application. To deal with data that is already collected we have the Uploader tools, which can label and import data.

A quick way of getting data from devices is using the Data forwarder. This lets you forward data collected over a serial interface to the studio. This method only works on sensors with lower sampling frequencies (f.e. no audio), does not allow sensor selection, and does not sign data on device. It's however a really easy way to collect data from existing devices with just a few lines of code.

Running impulses

The Inferencing SDK enables you to run impulses locally and on-device. The SDK contains efficient native implementations of all processing and learning blocks. The SDK was written in portable C++11 with as little dependencies as possible, and the best way of testing out whether it works on your platform is through the Deployment page in the studio. From here you can export a library with all blocks, configuration and the SDK. See the Running your impulse locally tutorials.

If you need to make changes to the SDK to get it to run on your device we welcome contributions. We also welcome contributions which add optimized code paths for your specific hardware. The SDK documentation has more information on where to add these.

Controlling the device from the studio

Devices can be controlled from the studio through the Remote management interface. This is a service where devices connect to, either over a web socket or through a serial connection (with the help of the Serial daemon). The studio lists these devices, and can instruct them to start sampling straight from the UI.

To add full support for your development board you'll need to implement the serial protocol and (if your device has an IP connection) the websocket protocol. Alternatively you can also implement the web socket protocol through an intermediate layer (like a mobile phone app). There are end-to-end integration tests available at edgeimpulse/integration-tests-firmware which validate both the serial and websocket protocols on a development board.

Devices that connect through the data forwarder can be controlled by the studio, but have a limited integration. They don't support sensor or frequency selection.

Full integration in Edge Impulse, and help porting?

Do you want help porting? Or want to get the best integration in Edge Impulse, including full studio support, and want to let users build binaries directly from the UI? Let us know at [email protected] and we'll let you know the possibilities.

Edge Impulse for Linux

Edge Impulse for Linux is the easiest way to build Machine Learning solutions on real embedded hardware. It contains tools which let you collect data from any microphone or camera, can be used with the Node.js, Python, Go and C++ SDKs to collect new data from any sensor, and can run impulses with full hardware acceleration - with easy integration points to write your own applications.

Development boards

This is a list of development boards that are fully supported by Edge Impulse for Linux. Follow the instructions to get started:

Raspberry Pi 4.
NVIDIA Jetson Nano.
Mac.
Linux x86_64 devices.

Different development board? Probably no problem! You can use the Linux x86_64 getting started guide to set up the Edge Impulse for Linux CLI tool, and you can run your impulse on any x86_64, ARMv7 or AARCH64 Linux target. For support please head to the forums.

SDKs

To build your own applications, or collect data from new sensors, you can use the high-level language SDKs. These use full hardware acceleration, and let you integrate your Edge Impulse models in a few lines of code:

Node.js.
Python.
Go.
C++.

.eim models?

Edge Impulse for Linux models are delivered in .eim format. This is an executable that contains your signal processing and ML code, compiled with optimizations for your processor or GPU (e.g. NEON instructions on ARM cores) plus a very simple IPC layer (over a Unix socket). We do this because your model file is now completely self-contained, it does not depend on anything (except glibc) and thus you don't need specific TensorFlow versions, avoid Python dependency hell, and will never have to worry about why you're not running at full native speed.

The Node.js / Python / Go SDKs talk to the model through the IPC layer to run inference, so these SDKs are very thin, and just need the ability to spawn a binary. The SDKs are open source if you want to take a look, e.g. here is the Node.js IPC client.

You can download .eim files using the Edge Impulse for Linux CLI or from the Studio (go to Dashboard, then enable 'Show Linux deploy options', and they'll be listed under Deployment). You can also build the .eim files yourself with the Edge Impulse for Linux C++ SDK, see Building .eim files.

Linux Node.js SDK

This library lets you run machine learning models and collect sensor data on machines using Node.js. The SDK is open source and hosted on GitHub: .

Installation guide

Add the library to your application via:

Collecting data

Before you can classify data you'll first need to collect it. If you want to collect data from the camera or microphone on your system you can use the Edge Impulse CLI, and if you want to collect data from different sensors (like accelerometers or proprietary control systems) you can do so in a few lines of code.

Collecting data from the camera or microphone

To collect data from the camera or microphone, follow the for your development board.

Collecting data from other sensors

To collect data from other sensors you'll need to write some code where you instantiate a DataForwarder object, write data samples, and finally call finalize() which uploads the data to Edge Impulse. .

Classifying data

To classify data (whether this is from the camera, the microphone, or a custom sensor) you'll need a model file. This model file contains all signal processing code, classical ML algorithms and neural networks - and typically contains hardware optimizations to run as fast as possible. To grab a model file:

Train your model in Edge Impulse.
.
Download the model file via:
This downloads the file into modelfile.eim. (Want to switch projects? Add --clean)

Then you can start classifying realtime sensor data. We have examples for:

- grabs data from the microphone and classifies it in realtime.
- as above, but shows how to use the moving-average filter to smooth your data and reduce false positives.
- grabs data from a webcam and classifies it in realtime.
- classifies custom sensor data.

Linux Go SDK

This library lets you run machine learning models and collect sensor data on Linux machines using Go. The SDK is open source and hosted on GitHub: edgeimpulse/linux-sdk-go.

Installation guide

Install Go 1.15 or higher.

Clone this repository:

$ git clone https://github.com/edgeimpulse/linux-sdk-go

Find the example that you want to build and run go build:
```
$ cd cmd/eimclassify
$ go build
```
Run the example:
```
$ ./eimclassify
```
And follow instructions.
This SDK is also published to pkg.go.dev, so you can pull the package from there too.

Collecting data

Collecting data from the camera or microphone

To collect data from the camera or microphone, follow the getting started guide for your development board.

Collecting data from other sensors

To collect data from other sensors you'll need to write some code to collect the data from an external sensor, wrap it in the Edge Impulse Data Acquisition format, and upload the data to the Ingestion service. Here's an end-to-end example.

Classifying data

Train your model in Edge Impulse.
Install the Edge Impulse for Linux CLI.
Download the model file via:
```
$ edge-impulse-linux-runner --download modelfile.eim
```
This downloads the file into modelfile.eim. (Want to switch projects? Add --clean)

Then you can start classifying realtime sensor data. We have examples for:

Audio - grabs data from the microphone and classifies it in realtime.
Camera - grabs data from a webcam and classifies it in realtime.
Custom data - classifies custom sensor data.

Linux Python SDK

This library lets you run machine learning models and collect sensor data on machines using Python. The SDK is open source and hosted on GitHub: .

Installation guide

Install a recent version of (>=3.7).
Install the SDK
Raspberry Pi
Jetson Nano
Other platforms
Clone this repository to get the examples:

Collecting data

Collecting data from the camera or microphone

To collect data from the camera or microphone, follow the ) for your development board.

Collecting data from other sensors

Classifying data

Train your model in Edge Impulse.
Install the .
Download the model file via:
This downloads the file into modelfile.eim. (Want to switch projects? Add --clean)

Then you can start classifying realtime sensor data. We have examples for:

- grabs data from the microphone and classifies it in realtime.
- grabs data from a webcam and classifies it in realtime.
- classifies custom sensor data.

Troubleshooting

[Errno -9986] Internal PortAudio error (macOS)

If you see this error you can re-install portaudio via:

Abort trap (6) (macOS)

This error shows when you want to gain access to the camera or the microphone on macOS from a virtual shell (like the terminal in Visual Studio Code). Try to run the command from the normal Terminal.app.

Edge Impulse CLI

Overview

This Edge Impulse CLI is used to control local devices, act as a proxy to synchronise data for devices that don't have an internet connection, and to upload and convert local files. The CLI consists of seven tools:

- configures devices over serial, and acts as a proxy for devices that do not have an IP connection.
- allows uploading and signing local files.
- a very easy way to collect data from any device over a serial connection, and forward the data to Edge Impulse.
- show the impulse running on your device.
- create organizational transformation, custom dsp, custom deployment and custom transfer learning blocks.
- to flash the Himax WE-I Plus.

Did you know you can also connect devices directly to your browser?

Recent versions of Google Chrome and Microsoft Edge can connect directly to fully-supported development boards, without the CLI. See for more information.

Serial daemon

The serial daemon is used to onboard new devices, configure upload settings, and acts as a proxy for devices without an IP connection.

Recent versions of Google Chrome and Microsoft Edge can connect directly to fully-supported development boards, without the serial daemon. See for more information.

To use the daemon, connect a fully-supported development board to your computer and run:

The daemon will ask you for the server you want to connect to, prompt you to log in, and then configure the device. If your device does not have the right firmware yet, it will also prompt you to upgrade this.

This is an example of the output of the daemon:

Note: Your credentials are never stored. When you log in these are exchanged for a token. This token is used to further authenticate requests.

Clearing configuration

To clear the configuration, run:

This resets both the daemon configuration as well as the on-device configuration. If you still run into issues, you can connect to the device using a serial monitor (on baud rate 115,200) and run AT+CLEARCONFIG. This removes all configuration from the device.

Devices without an IP connection

If your device is not connected to the remote management interface - for example because it does not have an IP connection, or because WiFi is out of range - the daemon will act as a proxy. It will register with Edge Impulse on behalf of the device, and proxy events through over serial. For this to work your device needs to support the Edge Impulse AT command set, please refer to the documentation for more information.

Silent mode

To skip any wizards (except for the login prompt) you can run the daemon in silent mode via:

This is useful in environments where there is no internet connection, as the daemon won't prompt to connect to WiFi.

Switching projects

You can use one device for many projects. To switch projects run:

And select the new project. The device will remain listed in the old project, and if you switch back will retain the same name and last seen date.

Troubleshooting

Unable to set up WiFi with ST B-L475E-IOT01A development board

If you are using the development board, you may experience the following error when attempting to connect to a WiFi network:

There is a with the firmware for this development board's WiFi module that results in a timeout during network scanning if there are more than 20 WiFi access points detected. If you are experiencing this issue, you can work around it by attempting to reduce the number of access points within range of the device, or by skipping WiFi configuration.

Impulse runner

The impulse runner shows the results of your impulse running on your development board. This only applies to ready-to-go binaries built from the studio.

You start the impulse via:

This will sample data from your real sensors, classify the data, then print the results. E.g.:

Other options

--debug - run the impulse in debug mode, this will print the intermediate DSP results. For image models, a live feed of the camera and inference results will also be locally hosted and available in your browser.
--continuous - run the impulse in continuous mode (not available on all platforms).

Himax flash tool

The Himax flash tool uploads new binaries to the over a serial connection.

You upload a new binary via:

This will yield a response like this:

Other options

--baud-rate <n> - sets the baud rate of the bootloader. This should only be used during development.
--verbose - enable debug logs, including all communication received from the device.

Deployment

On your Mbed-enabled development board

Impulses can be deployed as a C++ library. This packages all your signal processing blocks, configuration and learning blocks up into a single package. You can include this package in your own application to run the impulse locally. In this tutorial you'll export an impulse, and build an Mbed OS application to classify sensor data.

Knowledge required

This tutorial assumes that you're familiar with Mbed OS, and have installed Mbed CLI. If you're unfamiliar with these tools you can build binaries directly for your development board from the Deployment page in the studio.

Note: Are you looking for an example that has all sensors included? The Edge Impulse firmware for the ST IoT Discovery Kit has that. See edgeimpulse/firmware-st-b-l475e-iot01a.

Prerequisites

Make sure you followed the Continuous motion recognition tutorial, and have a trained impulse. Also install the following software:

Mbed CLI - make sure mbed is in your PATH.
GNU ARM Embedded Toolchain 9 - make sure arm-none-eabi-gcc is in your PATH.

Cloning the base repository

We created an example repository which contains a small Mbed OS application, which takes the raw features as an argument, and prints out the final classification. Import this repository using Mbed CLI:

$ mbed import https://github.com/edgeimpulse/example-standalone-inferencing-mbed

Deploying your impulse

Head over to your Edge Impulse project, and go to Deployment. From here you can create the full library which contains the impulse and all external required libraries. Select C++ library and click Build to create the library.

Download the .zip file and place the contents in the 'example-standalone-inferencing-mbed' folder (which you downloaded above). Your final folder structure should look like this:

example-standalone-inferencing-mbed
|_ Makefile
|_ README.md
|_ build.sh
|_ edge-impulse-sdk
|_ model-parameters
|_ source
|_ tflite-model

Running the impulse

With the project ready it's time to verify that the application works. Head back to the studio and click on Live classification. Then load a validation sample, and click on a row under 'Detailed result'.

To verify that the local application classifies the same, we need the raw features for this timestamp. To do so click on the 'Copy to clipboard' button next to 'Raw features'. This will copy the raw values from this validation file, before any signal processing or inferencing happened.

Open main.cpp and paste the raw features inside the static const float features[] definition, for example:

static const float features[] = {
    -19.8800, -0.6900, 8.2300, -17.6600, -1.1300, 5.9700, ...
};

Then build and flash the application to your development board with Mbed CLI:

$ mbed compile -t GCC_ARM -m auto -f

Seeing the output

To see the output of the impulse, connect to the development board over a serial port on baud rate 115,200 and reset the board (e.g. by pressing the black button on the ST B-L475E-IOT01A. You can do this with your favourite serial monitor or with the Edge Impulse CLI:

$ edge-impulse-run-impulse --raw

This will run the signal processing pipeline, and then classify the output:

Edge Impulse standalone inferencing (Mbed)
Running neural network...
Predictions (time: 0 ms.):
idle:   0.015319
snake:  0.000444
updown: 0.006182
wave:   0.978056
Anomaly score (time: 0 ms.): 0.133557
run_classifier_returned: 0
[0.01532, 0.00044, 0.00618, 0.97806, 0.134]

Which matches the values we just saw in the studio. You now have your impulse running on your Mbed-enabled development board!

Connecting sensors?

A demonstration on how to plug sensor values into the classifier can be found here: Data forwarder - classifying data (Mbed OS).

Documentation

Getting Started

Get started with any device

Suitable for any type of embedded ML application

API Documentation

Enterprise version

API and SDK references

Services

SDK documentation

What is embedded ML, anyway?

Where can machine learning help?

What is embedded ML?

Learn more

Frequently asked questions

How can I share my Edge Impulse project?

What are the minimum hardware requirements to run the Edge Impulse inferencing library on my embedded device?

What frameworks does Edge Impulse use to train the machine learning models?

Is there a downside to enabling the EON Compiler?

Can I use a model that has been trained elsewhere in Edge Impulse?

How does the feature explorer visualize data that has more that 3 dimensions?

Does Edge impulse integrate with other cloud services?

What is the typical power consumption of the Edge Impulse machine learning processes on my device?

What is the .eim model format for Edge Impulse for Linux?

How is the labeling of the data performed?

Edge Impulse Studio

Dashboard

1. Showcasing your public projects with Markdown READMEs

2. Making your project public

3. Collaboration

4. Project info

5. Block Outputs

6. Performance Settings

7. Administrative Zone

8. Danger Zone

Devices

Uploader

Labeling Queue (Images)

AI Assisted labelling

Using YoloV5

Using your own model

Using Object tracking

Create Impulse

Input block

Time series (audio, vibration, movements)

Images

Processing Block

Learning blocks

Processing Blocks

Raw Data

Raw data parameters

How does the raw data block work?

Flatten

Flatten parameters

How does the flatten block work?

Image

Image parameters

How does the image block work?

Spectral features

Spectral analysis parameters

How does the spectral features block work?

Number of generated features

Spectrogram

Spectrogram parameters

How does the spectrogram block work?

Audio MFE

Audio MFE parameters

How does the MFE block work?

Audio MFCC

Audio MFCC parameters

How does the MFCC block work?

Audio Syntiant

Audio Syntiant parameters

How does the Syntiant block work?

IMU Syntiant

Parameters

How does the IMU Syntiant block work?

Learning Blocks

Anomaly Detection (K-means)

Features importance (optional)

Setting up the anomaly detection block