In this tutorial, you'll learn how to set up the HR/HRV Features block within Edge Impulse Studio to process your physiological data and extract meaningful heart rate (HR) and heart rate variability (HRV) features. These features can be leveraged in various downstream machine learning tasks, such as activity classification, stress detection, and health monitoring. We will extend this section over time with more examples.
For this guide, we will be using a subset of the publicly available PPG-DaLiA dataset, which contains data collected from 15 subjects during daily activities. This dataset includes physiological signals such as photoplethysmography (PPG), accelerometer data, and heart rate ground truth, allowing us to explore how the HR/HRV Features block can be applied to real-world data.
You can download the DaLIA-PPG dataset or clone the project S1_E4 from the links below:
All users (Community, Professional and Enterprise Plan) can extract HR/HRV features using this processing block for testing purposes. However, the deployment option is only available for Enterprise users. Contact your Solution Engineer to enable it.
Heart Rate refers to the number of times your heart beats per minute (BPM). It's a straightforward metric that indicates your cardiac activity and is influenced by factors like physical activity, stress, and overall health.
Heart Rate Variability measures the variation in time intervals between successive heartbeats, known as interbeat intervals (IBIs). HRV provides deeper insights into the autonomic nervous system's regulation of the heart, reflecting your body's ability to adapt to stress, recover, and maintain homeostasis.
Key HRV Metrics:
RMSSD (Root Mean Square of Successive Differences): Reflects short-term HRV and parasympathetic activity.
SDNN (Standard Deviation of NN intervals): Indicates overall HRV.
pNN50: Percentage of intervals differing by more than 50 ms, another measure of parasympathetic activity.
Understanding both HR and HRV can provide a comprehensive view of an individual's cardiovascular and autonomic health.
Physiological Data: You can download a subset of PPG-DaLIA the DaLIA-PPG dataset S1_E4:
Ensure your CSV file is structured with the following columns:
Time: The timestamp for each data point.
X, Y, Z: Accelerometer data for motion artifact compensation (optional).
ECG/PPG: The PPG or ECG signal for heart rate estimation.
HR: Heart rate value, if available (should be uploaded as a multi-label sample for regression).
Label: The activity label (optional but useful for classification tasks).
HR Calculation: Avoid uploading data in short snippets. HR calculation relies on history and feedback to accumulate a continuous stream for better calculation and smoothing. Upload long, contiguous data for the most accurate results.
Community tier need to use single label and should keep the length to 2 seconds for the best accuracy.
You can download a subset of the PPG-DaLIA dataset S1_E4:
Log into your Edge Impulse Studio.
Navigate to Data Acquisition in the left-hand panel.
Click Upload Data and select your CSV file for each subject.
Ensure that the data is correctly parsed, and assign appropriate labels for heartrate applicable (e.g., 100, 90, etc.).
Go to Impulse Design > Create Impulse.
Add a HR and HRV Features block:
Input Axes: Select accX, accY, accZ, PPG (if using accelerometer data for motion artifact correction).
Set Window Size to 2000 ms.
Set Frequency (Hz) to 25 Hz (tolerance +/- 1 Hz) or 50 Hz (tolerance +/- 3 Hz).
Under Impulse Design, add the HR/HRV Features block.
Input Axes: Select your PPG signal for HR estimation.
HR Window Size: Set the window size for heart rate computation (e.g., 9 seconds, and no shorter than 2 seconds).
HRV Settings: Enable all HRV features to see RMSSD and other params.
Input Axes: Select your ECG signal for HR estimation.
HR Window Size: Set the window size for heart rate computation (e.g., 10 seconds and upwards of 90 seconds for best accuracy).
HRV Settings: Enable HRV features such as RMSSD or time-series statistics.
Input Axes: Select your PPG signal for HR estimation.
Accelerometer X, Y, Z: Include these axes to filter motion artifacts.
HR Window Size: Set the window size for heart rate computation (e.g., 10 seconds).
HRV Settings: Enable HRV features such as RMSSD or time-series statistics.
Click on Generate Features from the top menu.
Review the feature generation output to ensure that the raw signals are correctly processed.
Edge Impulse will generate the features for both heart rate and HRV over the specified window size.
Add a Learning Block such as a Regression if you wish to estimate HR
Ensure that HR/HRV is selected as the input feature.
Train the model using your labeled data to predict different heart rate.
After training, use Model Testing to evaluate the performance of your HR/HRV feature extraction and heart rate prediction model.
Upload test data to ensure that the heart rate is correctly estimated and any HRV features are extracted as expected.
For enterprise users, you can deploy the model to your target device and start collecting real-time heart rate and HRV data. When the model is trained and validated, you can deploy it to your target device or C++ binary for real-time heart rate and HRV monitoring.
Deployment will look as follows:
To include heart rate extraction alongside your classifier results, define the following macro in your deployment:
This macro will enable the HR/HRV Features block to calculate heart rate and HRV features in real-time.
Read on in the HR/HRV Features block documentation for more deployment details or speak with your enterprise support team for more guidance.
For Professional and Community Plan users, the deployment option is not available, contact our sales team if you want to upgrade to an Enterprise Plan:
By following this tutorial, you've successfully set up the HR/HRV Features block in Edge Impulse Studio, extracted meaningful cardiovascular features, and optionally trained a machine learning model. This foundation enables you to build robust, real-time heart rate and HRV monitoring solutions for applications like health monitoring, stress detection, and fitness tracking.
Edge Impulse Documentation: HR/HRV Features
If you have any questions or need further assistance, feel free to reach out on the Edge Impulse Forum or consult the documentation.
Neural networks are not limited to working with one type of data at a time. One of their biggest advantages is that they are incredibly flexible with the type of input data, so long as the format and ordering of that data stays the same from training to inference. As a result, we can use them to perform sensor fusion for a variety of tasks.
Sensor fusion is the process of combining data from different types of sensors or similar sensors mounted in different locations, which gives us more information to make decisions and classifications. For example, you could use temperature data with accelerometer data to get a better idea of a potential anomaly!
In this tutorial, you will learn how to use Edge Impulse to perform sensor fusion on the Arduino Nano 33 BLE Sense.
Example Project: You can find the dataset and impulse used throughout this tutorial in .
Multi-impulse vs multi-model vs sensor fusion
Running multi-impulse refers to running two separate projects (different data, different DSP blocks and different models) on the same target. It will require modifying some files in the EI-generated SDKs. See the
Running multi-model refers to running two different models (same data, same DSP block but different tflite models) on the same target. See how to run a motion classifier model and an anomaly detection model on the same device in .
Sensor fusion refers to the process of combining data from different types of sensors to give more information to the neural network. To extract meaningful information from this data, you can use the same DSP block (like in this tutorial), multiples DSP blocks, or use neural networks embeddings like this tutorial.
For this tutorial, you'll need a .
For this demo, we'll show you how to identify different environments by using a fusion of temperature, humidity, pressure, and light data. In particular, I'll have the Arduino board identify different rooms in my house as well as outside. Note that the we assume that the environment is static--if I turn out lights or the outside temperature changes, the model will not work. However, it demonstrates how we can combine different sensor data with machine learning to do classification!
As we will be collecting data from our Arduino board connected to a computer, it helps to have a laptop that you can move to different rooms.
Create a new project on the Edge Impulse studio.
Connect the Arduino Nano 33 BLE to your computer. Follow the to upload the Edge Impulse firmware to the board and connect it to your project.
Go to Data acquisition. Under Record new data, select your device and set the label to bedroom. Change Sensor to Environmental + Interactional, set the Sample length to 10000 ms and Frequency to 12.5Hz.
Stand in one of your rooms with your Arduino board (and laptop). Click Start sampling and slowly move the board around while data is collected. After sampling is complete, you should see a new data plot with a different line for each sensor.
Variations
Try to stand in different parts of each room while collecting data.
Repeat this process to record about 3 minutes of data for the bedroom class. Try to stand in a different spot in the room while collecting data--we want a robust dataset that represents the features of each room. Head to another room and repeat data collection. Continue doing this until you have around 3 minutes of data for each of the following classes:
Bedroom
Hallway
Outside
You are welcome to try other rooms or locations. For this demo, I found that my bedroom, kitchen, and living room all exhibited similar environmental and lighting properties, so the model struggled to tell them apart.
Head to Dashboard and scroll down to Danger zone. Click Perform train/test split and follow the instructions in the pop-up window to split your dataset into training and testing groups. When you're done, you can head back to Data acquisition to see that your dataset has been split. You should see about 80% of your samples in Training data and about 20% in Test data.
Head to Create impulse. Change the Window increase to 500 ms. Add a Flatten block. Notice that you can choose which environmental and interactional sensor data to include. Deselect proximity and gesture, as we won't need those to detect rooms. Add a Classification (Keras) learning block
Click Save impulse.
Head to Flatten. You can select different samples and move the window around to see what the DSP result will look like for each set of features to be sent to the learning block.
The Flatten block will compute the average, minimum, maximum, root-mean square, standard deviation, skewness, and kurtosis of each axis (e.g. temperature, humidity, brightness, etc.). With 7 axes and 7 features computed for each axis, that gives us 49 features for each window being sent to the learning block. You can see these computed features under Processed features.
Click Save parameters. On the next screen, select Calculate feature importance and click Generate features.
After a few moments, you should be able to explore the features of your dataset to see if your classes are easily separated into categories.
Interestingly enough, it looks like temperature and red light values were the most important features in determining the location of the Arduino board.
With our dataset collected and features processed, we can train our machine learning model. Click on NN Classifier. Change the Number of training cycles to 300 and click Start training. We will leave the neural network architecture as the default for this demo.
During training, parameters in the neural network's neurons are gradually updated so that the model will try to guess the class of each set of data as accurately as possible. When training is complete, you should see a Model panel appear on the right side of the page.
The Confusion matrix gives you an idea of how well the model performed at classifying the different sets of data. The top row gives the predicted label and the column on the left side gives the actual (ground-truth) label. Ideally, the model should predict the classes correctly, but that's not always the case. You want the diagonal cells from the top-left to the bottom-right to be as close to 100% as possible.
The On-device performance provides some statistics about how the model will likely run on a particular device. By default, an Arm Cortex-M4F running at 80 MHz is assumed to be your target device. The actual memory requirements and run time may vary on different platforms.
Rather than simply assume that our model will work when deployed, we can run inference on our test dataset as well as on live data.
First, head to Model testing, and click Classify all. After a few moments, you should see results from your test set.
You can click on the three dots next to an item and select Show classification. This will give you a classification result screen where you can see results information in more detail.
Additionally, we can test the impulse in a real-world environment to make sure the model has not overfit the training data. To do that, head to Live classification. Make sure your device is connected to the Studio and that the Sensor, Sample length, and Frequency match what we used to initially capture data.
Click Start sampling. A new sample will be captured from your board, uploaded, and classified. Once complete, you should see the classification results.
In the example above, we sampled 10 seconds of data from the Arduino. This data is split into 1-second windows (the window moves over 0.5 seconds each time), and the data in that window is sent to the DSP block. The DSP block computes the 49 features that are then sent to the trained machine learning model, which performs a forward pass to give us our inference results.
As you can see, the inference results from all of the windows claimed that the Arduino board was in the bedroom, which was true! This is great news for our model--it seems to work even on unseen data.
Now that we have an impulse with a trained model and we've tested its functionality, we can deploy the model back to our device. This means the impulse can run locally without an internet connection to perform inference!
Edge Impulse can package up the entire impulse (preprocessing block, neural network, and classification code) into a single library that you can include in your embedded software.
Click on Deployment in the menu. Select the library that you would like to create, and click Build at the bottom of the page.
Running your impulse locally
Well done! You've trained a neural network to determine the location of a development board based on a fusion of several sensors working in tandem. Note that this demo is fairly limited--as the daylight or temperature changes, the model will no longer be valid. However, it hopefully gives you some ideas about how you can mix and match sensors to achieve your machine learning goals.
We can't wait to see what you'll build! 🚀
This tutorial demonstrates how to implement and integrate on an Arduino Opta using Edge Impulse. Anomaly detection is a machine learning technique that identifies unusual patterns in data, making it ideal for monitoring industrial processes and equipment. By training a model to recognize normal behavior, you can detect deviations that may indicate faults or malfunctions.
In this tutorial, you will:
Collect temperature data from a sensor connected to the Opta PLC.
Train a machine learning model in Edge Impulse to detect anomalies in the data.
Deploy the model to the Opta PLC for real-time inference.
Optionally, integrate the model with Arduino Cloud for remote monitoring and visualization.
We ran a webinar on integrating Edge Impulse with Arduino Opta PLC and Blues Wireless for remote monitoring and anomaly detection. The webinar covered the following topics:
Arduino Opta PLC: Learn about the flexibility and ease of integration of the Arduino Opta micro PLC, designed for industrial environments.
Edge Impulse for Machine Learning: Understand how to train and implement ML models for anomaly detection on industrial data directly on your Opta PLC.
Blues for Wireless Connectivity: Explore how Blues' Wireless for PLC Expansion enables secure cellular connectivity, allowing your Opta PLC to communicate with cloud-based monitoring systems.
About the DIN Celsius Board
The DIN Celsius is an all-in-one temperature laboratory offering two independent heaters and a temperature sensor. It allows you to simulate heating scenarios and monitor temperature changes, making it ideal for testing our anomaly detection use case, we can introduce an anomaly by turning off one of the heaters to cause a deviation from the normal condition.
Connecting the DIN Celsius to the Opta PLC
Safety First: Before making any connections, ensure that all power sources are disconnected to prevent electric shock or short circuits.
Connections Overview:
Power Connections:
Connect the +24V and GND terminals of the DIN Celsius to the corresponding power supply pins.
Heater Control:
Connect Relay 3 (pin 2) on the Opta PLC to Input Heat 1 on the DIN Celsius.
Connect Relay 4 (pin 3) on the Opta PLC to Input Heat 2 on the DIN Celsius.
These connections will control the two independent heaters on the DIN Celsius.
Temperature Sensor Input:
Connect the Output Voltage from the DIN Celsius to the I8 input (analog pin A7) on the Opta PLC.
This connection allows the PLC to read the temperature sensor data.
Pin Definitions:
HEAT_LEFT (Relay 3) connected to pin 2
HEAT_RIGHT (Relay 4) connected to pin 3
TEMP_SENS connected to analog pin A7
BTN (User Button) on the Opta PLC
Arduino IDE Configuration
Install Necessary Libraries:
Ensure you have the latest Arduino IDE 2 installed.
Install any required libraries via the Library Manager, such as Edge Impulse SDK and Arduino_HTS221 if using temperature/humidity sensors.
Define Pin Constants:
Testing the Connections
Heater Control Test Code:
Upload the Code: Use the Arduino IDE to upload the sketch to the Opta PLC.
Verify Operation: The LEDs on the DIN Celsius should blink, indicating the heaters are being activated.
Temperature Sensor Reading Test:
Open Serial Monitor: Set the baud rate to 9600.
Observe Readings: You should see numerical values corresponding to the temperature sensor output.
Log In: Access your Edge Impulse account.
New Project: Create a project named, for example, "Opta PLC Anomaly Detection."
Upload a sketch to the Opta PLC that reads the temperature sensor and sends data to Edge Impulse.
Steps:
Upload the Data Forwarder Sketch: Use the Arduino IDE to upload the sketch to the Opta PLC.
Run Data Forwarder: In your terminal, execute the data forwarding command.
Select Serial Port: Choose the serial port corresponding to the Opta PLC.
Label the Data: As you collect data, assign labels to your data (e.g., "normal," "anomalous") based on the system's behavior.
Run Data Forwarder: In your terminal, execute:
Select Serial Port: Choose the serial port corresponding to the Opta PLC.
Label the Data: Assign labels to your data (e.g., "normal," "anomalous") as you collect it.
Add Blocks: 2. Add Blocks:
Processing Block: Select a Time Series or Spectral Analysis block based on your data characteristics.
Learning Block: Choose Anomaly Detection using the Gaussian Mixture Model (GMM).
Window Size: Set according to the data frequency (e.g., 1000 ms).
Window Increase: Set overlap (e.g., 500 ms).
Compute Features: Navigate to the Generate Features tab and run feature generation.
Training Parameters:
Epochs: Start with 100 and adjust as needed.
Learning Rate: Default of 0.005 is usually sufficient. Start Training: Monitor the accuracy and loss graphs.
Model Testing: Use a separate dataset to evaluate model performance.
Adjust if Necessary: Retrain or adjust parameters based on results.
Deployment Tab: In Edge Impulse, go to Deployment.
Select Arduino Library: Download the library tailored for your model.
Add Library to IDE: Import the downloaded library into your Arduino IDE.
Replace Your_Edge_Impulse_Inference_Library.h with the actual header file name from your downloaded library.
Set ANOMALY_THRESHOLD to an appropriate value based on your model's performance.
Compile and Upload: Use the Arduino IDE to program the Opta PLC with your new inference code.
Monitor Output: Open the Serial Monitor to observe inference results and system behavior.
This section demonstrates integrating Blues Wireless for remote connectivity and monitoring. By connecting your Opta PLC to the cloud, you can visualize data, receive alerts, and monitor system performance from anywhere.
Explore how Blues' Wireless for PLC Expansion enables seamless, secure cellular connectivity, allowing your Opta PLC to communicate with your cloud-based monitoring systems, regardless of location and without the hassle of local Wi-Fi.
In this section, we will integrate the anomaly detection model deployed on the Arduino Opta PLC with a ladder logic program. This integration will allow the machine learning inferences to interact with the PLC's native ladder logic control system, providing intelligent control responses to anomalies in temperature or motor power.
The Arduino PLC IDE allows you to combine an Arduino sketch with ladder logic using Shared Variables. These shared variables act as a bridge between the Edge Impulse inference running on the PLC and the PLC's control logic written in Ladder Diagram (LD) or other IEC-61131-3 programming languages (such as Function Block Diagram or Structured Text). This enables real-time decision-making based on the machine learning model's output.
Create Shared Variables
In the Arduino PLC IDE, navigate to the Global_vars section to create shared variables that will store the results from the Edge Impulse inference. Define a shared variable for the anomaly score or classification output of the model.
Example:
Shared variable for storing inference result (e.g., float anomaly_score).
Modify Inference Sketch
Update the Edge Impulse inference sketch to store the inference result in the shared variable. This will allow the ladder logic to access the result.
Example:
Ladder Logic Program
In the PLC IDE, create a new ladder logic program that will read the anomaly_score shared variable. The logic can then trigger actions based on the value, such as activating relays, generating alarms, or shutting down equipment in response to detected anomalies.
Example Ladder Logic:
Create a rung that monitors the anomaly_score. If the score exceeds a certain threshold, the logic can trigger an alarm (e.g., turn on an LED or activate a relay).
Add Inputs and Outputs:
Define Inputs (e.g., sensor values, inference results from the Arduino sketch) and Outputs (e.g., control signals like turning on a relay).
Click Add in the "Shared Inputs" and "Shared Outputs" sections to create global variables. These variables will allow communication between your inference sketch and ladder logic.
Set up the Inputs/Outputs:
Inputs: Define variables that will receive values from the Arduino sketch (e.g., anomaly_score).
Outputs: Define variables that will control actuators (e.g., relay_control).
Step 2: Accessing the Ladder Logic Editor
Create a New Ladder Logic Program:
Go to the Project tab (top-left section).
Right-click on Programs and select New Program.
Name the program (e.g., AnomalyDetection_LD) and select Ladder Diagram (LD) as the language.
Opening the Ladder Logic Editor:
Once the program is created, double-click it to open the Ladder Diagram editor. You will see a canvas where you can start adding blocks.
Step 3: Designing the Ladder Logic
Drag and Drop Components:
On the right panel under the Library Tree, you can see various block types, such as Comparison, Logic, and Arithmetic.
Comparison blocks will allow you to compare the input (e.g., anomaly_score) to a threshold (e.g., >=).
Creating Logic:
Input Condition: Drag a Comparison block (e.g., >=) to compare the anomaly_score to a threshold (e.g., 0.8).
Output Control: Connect the result of the comparison to an Output coil that controls the relay (e.g., relay_control).
Steps for Adding Logic:
Input: Select anomaly_score as the input to the comparison block.
Condition: Set the threshold (e.g., >= 0.8).
Output: Set the output to control a relay (e.g., activate relay_control when the condition is met).
Assigning the Ladder Logic to a Task
In the PLC IDE, assign the Arduino sketch (which runs the inference) to a task such as Fast (runs every 10ms) or Background (runs every 500ms) based on your system’s real-time requirements. Attach the ladder logic program to the same or another appropriate task to ensure it reacts to the updated shared variables in a timely manner.
Steps:
Go to the Project tab and locate the Tasks section.
Assign the ladder logic program (e.g., AnomalyDetection_LD) to an appropriate task (e.g., Fast Task for real-time control).
Monitor and Debug
Use the Watch window in the PLC IDE to monitor the shared variables and ensure the system responds correctly to anomalies detected by the machine learning model.
Upload the Arduino Sketch:
Ensure your Arduino sketch is uploaded to the Opta PLC to provide the inference results.
Run the Ladder Logic:
Start the ladder logic program from the Arduino PLC IDE.
Monitor the shared variables using the Watch window to see how the ladder logic reacts to the inference results.
Real-time Control: By integrating anomaly detection with ladder logic, you can implement real-time, intelligent control systems that take action based on data-driven decisions from the Edge Impulse model.
Easy Troubleshooting: Using ladder logic alongside machine learning allows for clear, visual representation of control logic, making it easier to debug and monitor the system's responses to anomalies.
Seamless PLC Integration: The Arduino PLC IDE provides a smooth environment for combining traditional control logic with modern machine learning, ensuring compatibility and ease of use.
Congratulations! You have successfully implemented anomaly detection on an Arduino Opta PLC using Edge Impulse. This tutorial demonstrates the seamless integration of machine learning models with industrial automation systems, enabling real-time monitoring and fault detection.
For more information on Edge Impulse and Arduino Opta PLC, visit the official websites:
Arduino Cloud Issues: We are aware of issues with Arduino Cloud .properties file format vs IDE and are working with Arduino. If you have issues try moving the .properties file to the same folder as the .ino file and re-uploading the sketch.
In this tutorial, you'll use machine learning to build a gesture recognition system that runs on a microcontroller. This is a hard task to solve using rule-based programming, as people don't perform gestures in the exact same way every time. But machine learning can handle these variations with ease. You'll learn how to collect high-frequency data from real sensors, use signal processing to clean up data, build a neural network classifier, and how to deploy your model back to a device. At the end of this tutorial, you'll have a firm understanding of applying machine learning in embedded devices using Edge Impulse.
There is also a video version of this tutorial:
You can view the finished project, including all data, signal processing and machine learning blocks here: .
If your device is connected (green dot) under Devices in the studio you can proceed:
Data ingestion
With your device connected, we can collect some data. In the studio go to the Data acquisition tab. This is the place where all your raw data is stored, and - if your device is connected to the remote management API - where you can start sampling new data.
Under Record new data, select your device, set the label to updown, the sample length to 10000, the sensor to Built-in accelerometer and the frequency to 62.5Hz. This indicates that you want to record data for 10 seconds, and label the recorded data as updown. You can later edit these labels if needed.
After you click Start sampling move your device up and down in a continuous motion. In about twelve seconds the device should complete sampling and upload the file back to Edge Impulse. You see a new line appear under 'Collected data' in the studio. When you click it you now see the raw data graphed out. As the accelerometer on the development board has three axes you'll notice three different lines, one for each axis.
Continuous movement
It's important to do continuous movements as we'll later slice up the data in smaller windows.
Machine learning works best with lots of data, so a single sample won't cut it. Now is the time to start building your own dataset. For example, use the following four classes, and record around 3 minutes of data per class:
Idle - just sitting on your desk while you're working.
Snake - moving the device over your desk as a snake.
Wave - waving the device from left to right.
Updown - moving the device up and down.
Variations
Make sure to perform variations on the motions. E.g. do both slow and fast movements and vary the orientation of the board. You'll never know how your user will use the device. It's best to collect samples of ~10 seconds each.
Prebuilt dataset
With the training set in place, you can design an impulse. An impulse takes the raw data, slices it up in smaller windows, uses signal processing blocks to extract features, and then uses a learning block to classify new data. Signal processing blocks always return the same values for the same input and are used to make raw data easier to process, while learning blocks learn from past experiences.
For this tutorial we'll use the 'Spectral analysis' signal processing block. This block applies a filter, performs spectral analysis on the signal, and extracts frequency and spectral power data. Then we'll use a 'Neural Network' learning block, that takes these spectral features and learns to distinguish between the four (idle, snake, wave, updown) classes.
In the studio go to Create impulse, set the window size to 2000 (you can click on the 2000 ms. text to enter an exact value), the window increase to 80, and add the 'Spectral Analysis' and 'Classification (Keras)' blocks. Then click Save impulse.
To configure your signal processing block, click Spectral features in the menu on the left. This will show you the raw data on top of the screen (you can select other files via the drop down menu), and the results of the signal processing through graphs on the right. For the spectral features block you'll see the following graphs:
Filter response - If you have chosen a filter (with non zero order), this will show you the response across frequencies. That is, it will show you how much each frequency will be attenuated.
After filter - the signal after applying the filter. This will remove noise.
Spectral power - the frequencies at which the signal is repeating (e.g. making one wave movement per second will show a peak at 1 Hz).
A good signal processing block will yield similar results for similar data. If you move the sliding window (on the raw data graph) around, the graphs should remain similar. Also, when you switch to another file with the same label, you should see similar graphs, even if the orientation of the device was different.
Bonus exercise: filters
Try to reason about the filter parameters. What does the cut-off frequency control? And what do you see if you switch from a low-pass to a high-pass filter?
Set the filter to low pass with the following parameters:
Once you're happy with the result, click Save parameters. This will send you to the 'Feature generation' screen. In here you'll:
Split all raw data up in windows (based on the window size and the window increase).
Apply the spectral features block on all these windows.
Calculate feature importance. We will use this later to set up the anomaly detection.
Click Generate features to start the process.
Afterward the 'Feature explorer' will load. This is a plot of all the extracted features against all the generated windows. You can use this graph to compare your complete data set. A good rule of thumb is that if you can visually identify some clusters by classes, then the machine learning model will be able to do so as well.
With all data processed it's time to start training a neural network. Neural networks are a set of algorithms, modeled loosely after the human brain, that are designed to recognize patterns. The network that we're training here will take the signal processing data as an input, and try to map this to one of the four classes.
So how does a neural network know what to predict? A neural network consists of layers of neurons, all interconnected, and each connection has a weight. One such neuron in the input layer would be the height of the first peak of the X-axis (from the signal processing block); and one such neuron in the output layer would be wave (one the classes). When defining the neural network all these connections are initialized randomly, and thus the neural network will make random predictions. During training, we then take all the raw data, ask the network to make a prediction, and then make tiny alterations to the weights depending on the outcome (this is why labeling raw data is important).
This way, after a lot of iterations, the neural network learns; and will eventually become much better at predicting new data. Let's try this out by clicking on NN Classifier in the menu.
Set 'Number of training cycles' to 1. This will limit training to a single iteration. And then click Start training.
Now change 'Number of training cycles' to 2 and you'll see performance go up. Finally, change 'Number of training cycles' to 30 and let the training finish.
You've just trained your first neural networks!
100% accuracy
You might end up with 100% accuracy after training for 100 training cycles. This is not necessarily a good thing, as it might be a sign that the neural network is too tuned for the specific test set and might perform poorly on new data (overfitting). The best way to reduce this is by adding more data or reducing the learning rate.
From the statistics in the previous step we know that the model works against our training data, but how well would the network perform on new data? Click on Live classification in the menu to find out. Your device should (just like in step 2) show as online under 'Classify new data'. Set the 'Sample length' to 10000 (10 seconds), click Start sampling and start doing movements. Afterward, you'll get a full report on what the network thought you did.
If the network performed great, fantastic! But what if it performed poorly? There could be a variety of reasons, but the most common ones are:
There is not enough data. Neural networks need to learn patterns in data sets, and the more data the better.
The data does not look like other data the network has seen before. This is common when someone uses the device in a way that you didn't add to the test set. You can add the current file to the test set by clicking ⋮, then selecting Move to training set. Make sure to update the label under 'Data acquisition' before training.
The model has not been trained enough. Up the number of epochs to 200 and see if performance increases (the classified file is stored, and you can load it through 'Classify existing validation sample').
The model is overfitting and thus performs poorly on new data. Try reducing the learning rate or add more data.
The neural network architecture is not a great fit for your data. Play with the number of layers and neurons and see if performance improves.
As you see there is still a lot of trial and error when building neural networks, but we hope the visualizations help a lot. You can also run the network against the complete validation set through 'Model validation'. Think of the model validation page as a set of unit tests for your model!
With a working model in place, we can look at places where our current impulse performs poorly.
Neural networks are great, but they have one big flaw. They're terrible at dealing with data they have never seen before (like a new gesture). Neural networks cannot judge this, as they are only aware of the training data. If you give it something unlike anything it has seen before it'll still classify as one of the four classes.
Let's look at how this works in practice. Go to 'Live classification' and record some new data, but now vividly shake your device. Take a look and see how the network will predict something regardless.
So, how can we do better? If you look at the feature explorer, you should be able to visually separate the classified data from the training data. We can use this to our advantage by training a new (second) network that creates clusters around data that we have seen before, and compares incoming data against these clusters. If the distance from a cluster is too large you can flag the sample as an anomaly, and not trust the neural network.
To add this block go to Create impulse, click Add learning block, and select 'Anomaly Detection (K-Means)'. Then click Save impulse.
To configure the clustering model click on Anomaly detection in the menu. Here we need to specify:
The number of clusters. Here use 32.
Click Start training to generate the clusters. You can load existing validation samples into the anomaly explorer with the dropdown menu.
Axes
The anomaly explorer only plots two axes at the same time. Under 'average axis distance' you see how far away from each axis the validation sample is. Use the dropdown menu's to change axes.
If you now go back to 'Live classification' and load your last sample, it should now have tagged everything as anomaly. This is a great example where signal processing (to extract features), neural networks (for classification) and clustering algorithms (for anomaly detection) can work together.
With the impulse designed, trained and verified you can deploy this model back to your device. This makes the model run without an internet connection, minimizes latency, and runs with minimum power consumption. Edge Impulse can package up the complete impulse - including the signal processing code, neural network weights, and classification code - up in a single C++ library that you can include in your embedded software.
Mobile phone
To export your model, click on Deployment in the menu. Then under 'Build firmware' select your development board, and click Build. This will export the impulse, and build a binary that will run on your development board in a single step. After building is completed you'll get prompted to download a binary. Save this on your computer.
When you click the Build button, you'll see a pop-up with text and video instructions on how to deploy the binary to your particular device. Follow these instructions. Once you are done, we are ready to test your impulse out.
We can connect to the board's newly flashed firmware over serial. Open a terminal and run:
Serial daemon
If the device is not connected over WiFi, but instead connected via the Edge Impulse serial daemon, you'll need stop the daemon. Only one application can connect to the development board at a time.
This will sample data from the sensor, run the signal processing code, and then classify the data:
Continuous movement
We trained a model to detect continuous movement in 2 second intervals. Thus, changing your movement while sampling will yield incorrect results. Make sure you've started your movement when 'Sampling...' gets printed. In between sampling, you have two seconds to switch movements.
To run the continuous sampling, run the following command:
Victory! You've now built your first on-device machine learning model.
We can't wait to see what you'll build! 🚀
An impulse is a combination of preprocessing (DSP) blocks followed by machine learning blocks. It will slice up our data into smaller windows, use signal processing to extract features, and then train a machine learning model. Because we are using environmental and light data, which are slow-moving averages, we will use the for preprocessing.
You can also look at the Feature importance section to get an idea of which features are the most important in determining class membership. You can read more about feature importance .
If you see a lot of confusion between classes, it means you need to gather more data, try different features, use a different model architecture, or train for a longer period of time (more epochs). See to learn about ways to increase model performance.
See to learn how to deploy your impulse to a variety of platforms.
If you're interested in more, see our tutorials on or . If you have a great idea for a different project, that's fine too. Edge Impulse lets you capture data from any sensor, build to extract features, and you have full flexibility in your Machine Learning pipeline with the learning blocks.
To showcase how easy it is to integrate Edge Impulse with the , we'll walk through a practical example using the Arduino DIN Celsius board that comes with the kit, but also a Motor to demonstrate this setup can be used interchangeably. This example demonstrates how to set up a temperature-controlled system, collect data, train a machine learning model, and deploy it for anomaly detection.
The webinar is now available on-demand .
(Wi-Fi version recommended)
Arduino IDE 2 ()
Edge Impulse account ()
Edge Impulse CLI (
Note: If you are new to Edge Impulse, please refer to our for detailed instructions.
If you are new to Edge Impulse please see our .
Read on
If you are new to Arduino inference code, see our Arduino inference code documentation for more information.
See the full Arduino OPTA for Arduino Cloud guide .
See the Blues Wireless .
This section will be updated following the Blues Wireless Webinar on the 30th of October
Later will will revisit this with C++ and Siemens PLCs, for now, you can explore the .
Please share your experiences and feedback with us on the .
For this tutorial, you'll need a .
Alternatively, use the either or SDK to collect data from any other development board, or your .
Edge Impulse can ingest data from many sources and any device - including embedded devices that you already have in production. See the documentation for the for more information.
Alternatively, you can load an example test set that has about ten minutes of data in these classes (but how much fun is that?). See the for more information.
See the dedicated page for the pre-processing block.
See the dedicated page for the learning block.
The axes that we want to select during clustering. Click on the Select suggested axes button to harness the results of the output. Alternatively, the data separates well on the accX RMS, accY RMS and accZ RMS axes, you can also include these axes.
See the dedicated page for the learning block. We also provide the learning block that is compatible with this tutorial.
Your mobile phone can build and download the compiled impulse directly from the mobile client. See 'Deploying back to device' on the page.
Congratulations! You have used Edge Impulse to train a machine learning model capable of recognizing your gestures and understand how you can build models that classify sensor data or find anomalies. Now that you've trained your model you can integrate your impulse in the firmware of your own embedded device, see . There are examples for Mbed OS, Arduino, STM32CubeIDE, and any other target that supports a C++ compiler.
Or if you're interested in more, see our tutorials on or . If you have a great idea for a different project, that's fine too. Edge Impulse lets you capture data from any sensor, build to extract features, and you have full flexibility in your Machine Learning pipeline with the learning blocks.
