Data forwarder

The data forwarder is used to easily relay data from any device to Edge Impulse over serial. Devices write sensor values over a serial connection, and the data forwarder collects the data, signs the data and sends the data to the ingestion service. The data forwarder is useful to quickly enable data collection from a wide variety of development boards without having to port the full remote management protocol and serial protocol, but only supports collecting data at relatively low frequencies.
To use the data forwarder, load an application (examples for Arduino, Mbed OS and Zephyr below) on your development board, and run:
1
$ edge-impulse-data-forwarder
Copied!
The data forwarder will ask you for the server you want to connect to, prompt you to log in, and then configure the device.
This is an example of the output of the forwarder:
1
Edge Impulse data forwarder v1.5.0
2
? What is your user name or e-mail address (edgeimpulse.com)? [email protected]
3
? What is your password? [hidden]
4
Endpoints:
5
    Websocket: wss://remote-mgmt.edgeimpulse.com
6
    API:       https://studio.edgeimpulse.com
7
    Ingestion: https://ingestion.edgeimpulse.com
8
​
9
[SER] Connecting to /dev/tty.usbmodem401203
10
[SER] Serial is connected
11
[WS ] Connecting to wss://remote-mgmt.edgeimpulse.com
12
[WS ] Connected to wss://remote-mgmt.edgeimpulse.com
13
? To which project do you want to add this device? accelerometer-demo-1
14
? 3 sensor axes detected. What do you want to call them? Separate the names with ',': accX, accY, accZ
15
? What name do you want to give this device? Jan's DISCO-L475VG
16
[WS ] Authenticated
Copied!
Note: Your credentials are never stored. When you log in these are exchanged for a token. This token is used to further authenticate requests.
Clearing configuration
To clear the configuration, run:
1
$ edge-impulse-data-forwarder --clean
Copied!
Overriding the frequency
To override the frequency, use:
1
$ edge-impulse-data-forwarder --frequency 100
Copied!
Overriding the baud rate
To set a different baud rate, use:
1
$ edge-impulse-data-forwarder --baud-rate 460800
Copied!
Protocol
The protocol is very simple. The device should send data on baud rate 115,200 with one line per reading, and individual sensor data should be split with either a , or a TAB. For example, this is data from a 3-axis accelerometer:
1
-0.12,-6.20,7.90
2
-0.13,-6.19,7.91
3
-0.14,-6.20,7.92
4
-0.13,-6.20,7.90
5
-0.14,-6.20,7.91
Copied!
The data forwarder will automatically determine the sampling rate and the number of sensors based on the output. If you load a new application where the sampling frequency or the number of axes changes, the data forwarder will automatically be reconfigured.
Example (Arduino)
This is an example of a sketch that reads data from an accelerometer (tested on the Arduino Nano 33 BLE):
1
#include <Arduino_LSM9DS1.h>
2
​
3
#define CONVERT_G_TO_MS2    9.80665f
4
#define FREQUENCY_HZ        50
5
#define INTERVAL_MS         (1000 / (FREQUENCY_HZ + 1))
6
​
7
static unsigned long last_interval_ms = 0;
8
​
9
void setup() {
10
    Serial.begin(115200);
11
    Serial.println("Started");
12
​
13
    if (!IMU.begin()) {
14
        Serial.println("Failed to initialize IMU!");
15
        while (1);
16
    }
17
}
18
​
19
void loop() {
20
    float x, y, z;
21
​
22
    if (millis() > last_interval_ms + INTERVAL_MS) {
23
        last_interval_ms = millis();
24
​
25
        IMU.readAcceleration(x, y, z);
26
​
27
        Serial.print(x * CONVERT_G_TO_MS2);
28
        Serial.print('\t');
29
        Serial.print(y * CONVERT_G_TO_MS2);
30
        Serial.print('\t');
31
        Serial.println(z * CONVERT_G_TO_MS2);
32
    }
33
}
Copied!
Example (Mbed OS)
This is an example of an Mbed OS application that reads data from an accelerometer (tested on the ST IoT Discovery Kit):
1
#include "mbed.h"
2
#include "stm32l475e_iot01_accelero.h"
3
​
4
static int64_t sampling_freq = 104; // in Hz.
5
static int64_t time_between_samples_us = (1000000 / (sampling_freq - 1));
6
​
7
// set baud rate of serial port to 115200
8
static BufferedSerial serial_port(USBTX, USBRX, 115200);
9
FileHandle *mbed::mbed_override_console(int fd) {
10
    return &serial_port;
11
}
12
​
13
int main()
14
{
15
    int16_t pDataXYZ[3] = {0};
16
    Timer t;
17
    t.start();
18
​
19
    BSP_ACCELERO_Init();
20
​
21
    while(1) {
22
        int64_t next_tick = t.read_us() + time_between_samples_us;
23
​
24
        BSP_ACCELERO_AccGetXYZ(pDataXYZ);
25
        pc.printf("%d\t%d\t%d\n", pDataXYZ[0], pDataXYZ[1], pDataXYZ[2]);
26
​
27
        while (t.read_us() < next_tick) {
28
            /* busy loop */
29
        }
30
    }
31
}
Copied!
There's also a complete example that samples data from both the accelerometer and the gyroscope here: edgeimpulse/example-dataforwarder-mbed.
Example (Zephyr)
This is an example of a Zephyr application that reads data from an accelerometer (tested on the Nordic Semiconductor nRF52840 DK with ST X-NUCLEO-IKS02A1 shield), based on the sensorhub example:
1
#include <zephyr.h>
2
#include <sys/printk.h>
3
#include <drivers/sensor.h>
4
#include <stdio.h>
5
#include <stdlib.h>
6
​
7
static int64_t sampling_freq = 104; // in Hz.
8
static int64_t time_between_samples_us = (1000000 / (sampling_freq - 1));
9
​
10
int main() {
11
    // output immediately without buffering
12
    setvbuf(stdout, NULL, _IONBF, 0);
13
​
14
    // get driver for the accelerometer
15
    const struct device *iis2dlpc = device_get_binding(DT_LABEL(DT_INST(0, st_iis2dlpc)));
16
    if (iis2dlpc == NULL) {
17
        printf("Could not get IIS2DLPC device\n");
18
        return 1;
19
    }
20
​
21
    struct sensor_value accel[3];
22
​
23
    while (1) {
24
        // start a timer that expires when we need to grab the next value
25
        struct k_timer next_val_timer;
26
        k_timer_init(&next_val_timer, NULL, NULL);
27
        k_timer_start(&next_val_timer, K_USEC(time_between_samples_us), K_NO_WAIT);
28
​
29
        // read data from the sensor
30
        if (sensor_sample_fetch(iis2dlpc) < 0) {
31
            printf("IIS2DLPC Sensor sample update error\n");
32
            return 1;
33
        }
34
​
35
        sensor_channel_get(iis2dlpc, SENSOR_CHAN_ACCEL_XYZ, accel);
36
​
37
        // print over stdout
38
        printf("%.3f\t%.3f\t%.3f\r\n",
39
            sensor_value_to_double(&accel[0]),
40
            sensor_value_to_double(&accel[1]),
41
            sensor_value_to_double(&accel[2]));
42
​
43
        // busy loop until next value should be grabbed
44
        while (k_timer_status_get(&next_val_timer) <= 0);
45
    }
46
}
Copied!
There's also a complete example that samples data from the accelerometer here: edgeimpulse/example-dataforwarder-zephyr.
Sensor fusion
Using the Data Forwarder, you can relay data from multiple sensors. You can check Benjamin Cabe's artificial nose for a complete example using NO2, CO, C2H5OH and VOC sensors on a WIO Terminal.
You may also have sensors with different sampling frequencies, such as:
accelerometer: 3 axis sampled at 100Hz
RMS current sensor: 1 axis sampled at 5Hz
In this case, you should first upscale to the highest frequency to keep the finest granularity: upscale RMS sensor to 100 Hz by duplicating each value 20 times (100/5). You could also smooth values over between samples.
Classifying data
To classify data you first deploy your project by following the steps in Running your impulse locally - which contains examples for a wide variety of platforms. Then, declare a features array, fill it with sensor data, and run the classifier. Here are examples for Arduino, Mbed and Zephyr - but the same applies to any other platform.
Note: These examples collect a full frame of data, then classify this data. This might not be what you want (as classification blocks the collection thread). See Continuous audio sampling for an example on how to implement continuous classification.
Classifying data (Arduino)
1
// Include the Arduino library here (something like your_project_inference.h) 
2
// In the Arduino IDE see **File > Examples > Your project name - Edge Impulse > Static buffer** to get the exact name
3
#include <your_project_inference.h>
4
#include <Arduino_LSM9DS1.h>
5
​
6
#define CONVERT_G_TO_MS2    9.80665f
7
#define FREQUENCY_HZ        EI_CLASSIFIER_FREQUENCY
8
#define INTERVAL_MS         (1000 / (FREQUENCY_HZ + 1))
9
​
10
static unsigned long last_interval_ms = 0;
11
// to classify 1 frame of data you need EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE values
12
float features[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE];
13
// keep track of where we are in the feature array
14
size_t feature_ix = 0;
15
​
16
void setup() {
17
    Serial.begin(115200);
18
    Serial.println("Started");
19
​
20
    if (!IMU.begin()) {
21
        Serial.println("Failed to initialize IMU!");
22
        while (1);
23
    }
24
}
25
​
26
void loop() {
27
    float x, y, z;
28
​
29
    if (millis() > last_interval_ms + INTERVAL_MS) {
30
        last_interval_ms = millis();
31
​
32
        // read sensor data in exactly the same way as in the Data Forwarder example
33
        IMU.readAcceleration(x, y, z);
34
​
35
        // fill the features buffer
36
        features[feature_ix++] = x * CONVERT_G_TO_MS2;
37
        features[feature_ix++] = y * CONVERT_G_TO_MS2;
38
        features[feature_ix++] = z * CONVERT_G_TO_MS2;
39
​
40
        // features buffer full? then classify!
41
        if (feature_ix == EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
42
            ei_impulse_result_t result;
43
​
44
            // create signal from features frame
45
            signal_t signal;
46
            numpy::signal_from_buffer(features, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
47
​
48
            // run classifier
49
            EI_IMPULSE_ERROR res = run_classifier(&signal, &result, false);
50
            ei_printf("run_classifier returned: %d\n", res);
51
            if (res != 0) return;
52
​
53
            // print predictions
54
            ei_printf("Predictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
55
                result.timing.dsp, result.timing.classification, result.timing.anomaly);
56
​
57
            // print the predictions
58
            for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
59
                ei_printf("%s:\t%.5f\n", result.classification[ix].label, result.classification[ix].value);
60
            }
61
        #if EI_CLASSIFIER_HAS_ANOMALY == 1
62
            ei_printf("anomaly:\t%.3f\n", result.anomaly);
63
        #endif
64
​
65
            // reset features frame
66
            feature_ix = 0;
67
        }
68
    }
69
}
70
​
71
void ei_printf(const char *format, ...) {
72
    static char print_buf[1024] = { 0 };
73
​
74
    va_list args;
75
    va_start(args, format);
76
    int r = vsnprintf(print_buf, sizeof(print_buf), format, args);
77
    va_end(args);
78
​
79
    if (r > 0) {
80
        Serial.write(print_buf);
81
    }
82
}
Copied!
Classifying data (Mbed OS)
1
#include "mbed.h"
2
#include "stm32l475e_iot01_accelero.h"
3
#include "ei_run_classifier.h"
4
​
5
static int64_t sampling_freq = EI_CLASSIFIER_FREQUENCY; // in Hz.
6
static int64_t time_between_samples_us = (1000000 / (sampling_freq - 1));
7
​
8
// to classify 1 frame of data you need EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE values
9
static float features[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE];
10
​
11
// set baud rate of serial port to 115200
12
static BufferedSerial serial_port(USBTX, USBRX, 115200);
13
FileHandle *mbed::mbed_override_console(int fd) {
14
    return &serial_port;
15
}
16
​
17
int main()
18
{
19
    int16_t pDataXYZ[3] = {0};
20
    Timer t;
21
    t.start();
22
​
23
    BSP_ACCELERO_Init();
24
​
25
    while (1) {
26
        // fill the features array
27
        for (size_t ix = 0; ix < EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE; ix += EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME) {
28
            int64_t next_tick = t.read_us() + time_between_samples_us;
29
            BSP_ACCELERO_AccGetXYZ(pDataXYZ);
30
​
31
            // copy accelerometer data into the features array
32
            features[ix + 0] = (float)pDataXYZ[0];
33
            features[ix + 1] = (float)pDataXYZ[0];
34
            features[ix + 2] = (float)pDataXYZ[0];
35
​
36
            while (t.read_us() < next_tick) {
37
                /* busy loop */
38
            }
39
        }
40
​
41
        // frame full? then classify
42
        ei_impulse_result_t result = { 0 };
43
​
44
        // create signal from features frame
45
        signal_t signal;
46
        numpy::signal_from_buffer(features, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
47
​
48
        // run classifier
49
        EI_IMPULSE_ERROR res = run_classifier(&signal, &result, false);
50
        ei_printf("run_classifier returned: %d\n", res);
51
        if (res != 0) return 1;
52
​
53
        // print predictions
54
        ei_printf("Predictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
55
            result.timing.dsp, result.timing.classification, result.timing.anomaly);
56
​
57
        // print the predictions
58
        for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
59
            ei_printf("%s:\t%.5f\n", result.classification[ix].label, result.classification[ix].value);
60
        }
61
    #if EI_CLASSIFIER_HAS_ANOMALY == 1
62
        ei_printf("anomaly:\t%.3f\n", result.anomaly);
63
    #endif
64
    }
65
}
Copied!
Classifying (Zephyr)
Before adding the classifier in Zephyr:
1.Copy the extracted C++ library into your Zephyr project, and add the following to your CMakeLists.txt file (where ./model is where you extracted the library).
1
set(CMAKE_CXX_STANDARD 11)
2
set(CMAKE_CXX_STANDARD_REQUIRED ON)
3
​
4
add_subdirectory(./model)
Copied!
1.Enable C++ and set the stack size of the main thread to at least 4K, by adding the following to prj.conf:
1
CONFIG_CPLUSPLUS=y
2
CONFIG_LIB_CPLUSPLUS=y
3
CONFIG_NEWLIB_LIBC=y
4
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
5
​
6
CONFIG_MAIN_STACK_SIZE=8192
Copied!
1.If you're on a Cortex-M target, enable hardware acceleration by adding the following defines to your CMakeLists.txt file:
1
add_definitions(-DEIDSP_USE_CMSIS_DSP=1
2
                -DEIDSP_LOAD_CMSIS_DSP_SOURCES=1
3
                -DEI_CLASSIFIER_TFLITE_ENABLE_CMSIS_NN=1
4
                -DARM_MATH_LOOPUNROLL)
Copied!
Then, run the following application:
1
#include <zephyr.h>
2
#include <sys/printk.h>
3
#include <drivers/sensor.h>
4
#include <stdio.h>
5
#include <stdlib.h>
6
#include "ei_run_classifier.h"
7
​
8
static int64_t sampling_freq = EI_CLASSIFIER_FREQUENCY; // in Hz.
9
static int64_t time_between_samples_us = (1000000 / (sampling_freq - 1));
10
​
11
// to classify 1 frame of data you need EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE values
12
static float features[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE];
13
​
14
int main() {
15
    // output immediately without buffering
16
    setvbuf(stdout, NULL, _IONBF, 0);
17
​
18
    // get driver for the accelerometer
19
    const struct device *iis2dlpc = device_get_binding(DT_LABEL(DT_INST(0, st_iis2dlpc)));
20
    if (iis2dlpc == NULL) {
21
        printf("Could not get IIS2DLPC device\n");
22
        return 1;
23
    }
24
​
25
    struct sensor_value accel[3];
26
​
27
    while (1) {
28
        for (size_t ix = 0; ix < EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE; ix += EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME) {
29
            // start a timer that expires when we need to grab the next value
30
            struct k_timer next_val_timer;
31
            k_timer_init(&next_val_timer, NULL, NULL);
32
            k_timer_start(&next_val_timer, K_USEC(time_between_samples_us), K_NO_WAIT);
33
​
34
            // read data from the sensor
35
            if (sensor_sample_fetch(iis2dlpc) < 0) {
36
                printf("IIS2DLPC Sensor sample update error\n");
37
                return 1;
38
            }
39
​
40
            sensor_channel_get(iis2dlpc, SENSOR_CHAN_ACCEL_XYZ, accel);
41
​
42
            // fill the features array
43
            features[ix + 0] = sensor_value_to_double(&accel[0]);
44
            features[ix + 1] = sensor_value_to_double(&accel[1]);
45
            features[ix + 2] = sensor_value_to_double(&accel[2]);
46
​
47
            // busy loop until next value should be grabbed
48
            while (k_timer_status_get(&next_val_timer) <= 0);
49
        }
50
​
51
        // frame full? then classify
52
        ei_impulse_result_t result = { 0 };
53
​
54
        // create signal from features frame
55
        signal_t signal;
56
        numpy::signal_from_buffer(features, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
57
​
58
        // run classifier
59
        EI_IMPULSE_ERROR res = run_classifier(&signal, &result, false);
60
        printf("run_classifier returned: %d\n", res);
61
        if (res != 0) return 1;
62
​
63
        // print predictions
64
        printf("Predictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
65
            result.timing.dsp, result.timing.classification, result.timing.anomaly);
66
​
67
        // print the predictions
68
        for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
69
            printf("%s:\t%.5f\n", result.classification[ix].label, result.classification[ix].value);
70
        }
71
    #if EI_CLASSIFIER_HAS_ANOMALY == 1
72
        printf("anomaly:\t%.3f\n", result.anomaly);
73
    #endif
74
    }
75
}
Copied!
Troubleshooting
"The execution of scripts is disabled on this system" (Windows)
If you are running the data forwarder on a Windows system, you need to update PowerShell's execution policy to allow running scripts:
1
Set-ExecutionPolicy unrestricted
Copied!
Edge Impulse CLI - Previous
Uploader
Next - Edge Impulse CLI
Impulse runner
Last modified 18d ago
Export as PDF
Copy link
Contents