1 of 16

Data

Data ingestion

Collecting image data from the Studio

This page is part of Image classification and describes how you can use development boards with an integrated camera to import image data into Edge Impulse.

First, make sure your device is connected on the Devices page in the Edge Impulse Studio. Then, head to Data acquisition, and under 'Record new data', set a label and select 'Camera' as a sensor (most devices have multiple resolutions). This shows you a nice preview of the camera. Then click Start sampling.

A few moments later - depending on the speed of the development board and the resolution - you'll now have an image collected!

Do this until you have captured 30 images per class from a variety of angles. Also make sure to vary the things you capture for the unknown class.

Collecting image data with your mobile phone

This page is part of Image classification and describes how you can use your mobile phone to import image data into Edge Impulse.

To add your phone to your project, go to the Devices page, select Connect a new device and select Use your mobile phone. A QR code will pop up. Scan this code with your phone and your phone will pop up on the devices screen.

1. Collecting images

With your phone connected to your project, it's time to start capturing some images and build our dataset. We have a special UI for collecting images quickly, on your phone choose Collecting images?.

On your phone a permission prompt will show up, and then the viewfinder will be displayed. Set the label (in the top corner) to 'lamp', point your camera at your lamp and press Capture.

Afterwards the photo shows up in the studio on the Data acquisition page.

Do this until you have captured 30 images per class from a variety of angles. Also make sure to vary the things you capture for the unknown class.

2. Alternative: upload data directly

Alternatively you can also capture your dataset directly through a different app, and then upload the data directly to Edge Impulse There are both options to do this visually (click the 'Upload' icon on the data acquisition screen), or via the CLI. You can find instructions here: Uploader. In this case it's highly recommended to you use square images, as the transfer learning model expects these; and you probably want to resize these images before uploading them to make sure training remains fast.

Collecting image data with the OpenMV Cam H7 Plus

This page is part of Image classification and describes how you can use the OpenMV Cam H7 Plus to build a dataset, and import the data into Edge Impulse.

1. Setting up your environment

To set up your OpenMV camera, and collect some data:

Install the OpenMV IDE.
Follow the OpenMV hardware setup guide to clean the sensor and focus the lens.
Connect a micro-USB cable to the camera, and open the OpenMV IDE. The camera should automatically update to the latest firmware.
Verify that the camera can capture live images, by clicking on the Connect button in the bottom left corner, then pressing Play to run the application.

A live feed from your camera will be displayed in the top right corner of the IDE.

2. Collecting images

Once your camera is up and running, it's time to start capturing some images and build our dataset.

First, set up a new dataset via Tools -> Dataset Editor, select New Dataset.

This opens the 'Dataset editor' panel on the left side, and the 'dataset capture script' in the main panel of the IDE. Here, create three classes: "plant", "lamp" and "unknown". It's important to add an unknown class that contains random images which are neither lamps nor plants.

As we'll build a model that takes in square images, change the 'Dataset capture script' to read:

import sensor, image, time

sensor.reset()
sensor.set_pixformat(sensor.RGB565) # Modify as you like.
sensor.set_framesize(sensor.QVGA) # Modify as you like.
sensor.set_windowing((240, 240)) # Modify as you like.
sensor.skip_frames(time = 2000)

clock = time.clock()

while(True):
    clock.tick()
    img = sensor.snapshot()
    print(clock.fps())

Now you can capture data for the three classes.

Click the Play icon to run the 'dataset capture script' on your OpenMV camera.
Select one of the classes by clicking on the folder name in the 'Dataset editor'.
Take a snap by clicking the Capture data (camera icon) button.

Do this until you have captured 30 images per class from a variety of angles. Also make sure to vary the things you capture for the unknown class.

3. Sending the dataset to Edge Impulse

To import the dataset into Edge Impulse go to Tools > Dataset Editor > Export > Upload to Edge Impulse project.

Then, choose the project name, and the split between training and testing data (recommended to keep this to 80/20).

A duplicate check runs when you upload new data, so you can upload your dataset multiple times (for example, when you've added new files) without adding the same data twice.

Training and testing data split

The split between training and testing data is based on the hash of the file in order to have a deterministic process. As a consequence you may not have a perfect 80/20 split between training and testing, but this process ensures samples are always placed in the same category.

Our dataset now appears under the Data acquisition section of our project.

You can now go back to the Image classification tutorial to build your machine learning model.

Using the Edge Impulse Python SDK to upload and download data

If you want to upload files directly to an Edge Impulse project, we recommend using the CLI uploader tool. However, sometimes you cannot upload your samples directly, as you might need to convert the files to one of the accepted formats or modify the data prior to model training. Edge Impulse offers data augmentation for some types of projects, but you might want to create your own custom augmentation scheme. Or perhaps you want to generate synthetic data and script the upload process.

The Python SDK offers a set of functions to help you move data into and out of your project. This can be extremely helpful when generating or augmenting your dataset. The following cells demonstrate some of these upload and download functions.

You can find the API documentation for the functions found in this tutorial here.

WARNING: This notebook will add and delete data in your Edge Impulse project, so be careful! We recommend creating a throwaway project when testing this notebook.

Note that you might need to refresh the page with your Edge Impulse project to see the samples appear.

# If you have not done so already, install the following dependencies
!python -m pip install edgeimpulse

import edgeimpulse as ei

You will need to obtain an API key from an Edge Impulse project. Log into edgeimpulse.com and create a new project. Open the project, navigate to Dashboard and click on the Keys tab to view your API keys. Double-click on the API key to highlight it, right-click, and select Copy.

Note that you do not actually need to use the project in the Edge Impulse Studio. We just need the API Key.

Paste that API key string in the ei.API_KEY value in the following cell:

# Settings
ei.API_KEY = "ei_dae2..." # Change this to your Edge Impulse API key

Upload directory

You can upload all files in a directory using the Python SDK. Note that you can set the category, label, and metadata for all files with a single call. If you want to use a different label for each file set label=None in the function call and name your files with <label>.<name>.<ext>. For example, wave.01.csv will have the label wave when uploaded. See here for more information.

The following file formats are allowed: .cbor, .json, .csv, .wav, .jpg, .png, .mp4, .avi.

from datetime import datetime

# Download image files to use as an example dataset
!mkdir -p dataset
!wget -P dataset -q \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/capacitor.01.png \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/capacitor.02.png

# Upload the entire directory
response = ei.experimental.data.upload_directory(
    directory="dataset",
    category="training",
    label=None, # Will use the prefix before the '.' on each filename for the label
    metadata={
        "date": datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
        "source": "camera",
    }
)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

# Review the sample IDs and get the associated server-side filename
# Note the lack of extension! Multiple samples on the server can have the same filename.
for id in ids:
    filename = ei.experimental.data.get_filename_by_id(id)
    print(f"Sample ID: {id}, filename: {filename}")

If you head to the Data acquisition page on your project, you should see images in your dataset.

Download files

You can download samples from your Edge Impulse project if you know the sample IDs. You can get sample IDs by calling the ei.data.get_sample_ids() function, which allows you to filter IDs based on filename, category, and label.

# Get sample IDs for everything in the "training" category
infos = ei.experimental.data.get_sample_ids(category="training")

# The SampleInfo should match what we uploaded earlier
ids = []
for info in infos:
    print(info)
    ids.append(info.sample_id)

# Download samples
samples = ei.experimental.data.download_samples_by_ids(ids)

# Save the downloaded files
for sample in samples:
    with open(sample.filename, "wb") as file:
        file.write(sample.data.read())

# View sample information
for sample in samples:
    print(
        f"filename: {sample.filename}\r\n"
        f"  sample ID: {sample.sample_id}\r\n"
        f"  category: {sample.category}\r\n"
        f"  label: {sample.label}\r\n"
        f"  bounding boxes: {sample.bounding_boxes}\r\n"
        f"  metadata: {sample.metadata}"
    )

Take a look at the files in this directory. You should see the downloaded images. They should match the images in the dataset/ directory, which were the original images that we uploaded.

Delete files

If you know the ID of the sample you would like to delete, you can call the delete_sample_by_id() function. You can also delete all the samples in your project by calling delete_all_samples().

# Delete the samples from the Edge Impulse project that we uploaded earlier
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Take a look at the data in your project. The samples that we uploaded should be gone.

Upload folder for object detection

For object detection, you can put bounding box information (following the Edge Impulse JSON bounding box format) in a file named info.labels in that same directory.

Important! The annotations file must be named exactly info.labels

# Download images and bounding box annotations to use as an example dataset
!mkdir -p dataset
!rm dataset/capacitor.01.png dataset/capacitor.02.png
!wget -P dataset -q \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/dog-ball-toy.01.png \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/dog-ball-toy.02.png \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/annotations/info.labels

# Upload the entire directory (including the info.labels file)
response = ei.experimental.data.upload_exported_dataset(
    directory="dataset",
)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

If you head to the Data acquisition page on your project, you should see images in your dataset along with the bounding box information.

# Delete the samples from the Edge Impulse project that we uploaded
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload individual CSV files

The Edge Impulse ingestion service accepts CSV files, which we can use to upload raw data. Note that if you configure a CSV template using the CSV Wizard, then the expected format of the CSV file might change. If you do not configure a CSV template, then the ingestion service expects CSV data to be in a particular format. See here for details about the default CSV format.

import csv
import io
import os

# Create example CSV data
sample_data = [
    [
        ["timestamp", "accX", "accY", "accZ"],
        [0, -9.81, 0.03, 0.21],
        [10, -9.83, 0.04, 0.27],
        [20, -9.12, 0.03, 0.23],
        [30, -9.14, 0.01, 0.25],
    ],
    [
        ["timestamp", "accX", "accY", "accZ"],
        [0, -9.56, 5.34, 1.21],
        [10, -9.43, 1.37, 1.27],
        [20, -9.22, -4.03, 1.23],
        [30, -9.50, -0.98, 1.25],
    ],
]

# Write to CSV files
filenames = [
    "001.csv",
    "002.csv"
]
for i, filename in enumerate(filenames):
    file_path = os.path.join("dataset", filename)
    with open(file_path, "w", newline="") as file:
        writer = csv.writer(file)
        writer.writerows(sample_data[i])

# Add metadata to the CSV data
my_samples = [
    {
        "filename": filenames[0],
        "data": open(os.path.join("dataset", filenames[0]), "rb"),
        "category": "training",
        "label": "idle",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
    {
        "filename": filenames[1],
        "data": open(os.path.join("dataset", filenames[1]), "rb"),
        "category": "training",
        "label": "wave",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
]

# Wrap the samples in instances of the Sample class
samples = [ei.experimental.data.Sample(**i) for i in my_samples]

# Upload samples to your project
response = ei.experimental.data.upload_samples(samples)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

If you head to the Data acquisition page on your project, you should see your time series data.

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload JSON data directly

Another way to upload data is to encode it in JSON format. See the data acquisition format specificaion for more information on acceptable key/value pairs. Note that at this time, the signature value can be set to 0.

The raw data must be encoded in an IO object. We convert the dictionary objects to a BytesIO object, but you can also read in data from .json files.

import io
import json

# Create two different example data samples
sample_data_1 = {
    "protected": {
        "ver": "v1",
        "alg": "none",
    },
    "signature": 0,
    "payload": {
        "device_name": "ac:87:a3:0a:2d:1b",
        "device_type": "DISCO-L475VG-IOT01A",
        "interval_ms": 10,
        "sensors": [
            { "name": "accX", "units": "m/s2" },
            { "name": "accY", "units": "m/s2" },
            { "name": "accZ", "units": "m/s2" }
        ],
        "values": [
            [ -9.81, 0.03, 0.21 ],
            [ -9.83, 0.04, 0.27 ],
            [ -9.12, 0.03, 0.23 ],
            [ -9.14, 0.01, 0.25 ]
        ]
    }
}
sample_data_2 = {
    "protected": {
        "ver": "v1",
        "alg": "none",
    },
    "signature": 0,
    "payload": {
        "device_name": "ac:87:a3:0a:2d:1b",
        "device_type": "DISCO-L475VG-IOT01A",
        "interval_ms": 10,
        "sensors": [
            { "name": "accX", "units": "m/s2" },
            { "name": "accY", "units": "m/s2" },
            { "name": "accZ", "units": "m/s2" }
        ],
        "values": [
            [ -9.56, 5.34, 1.21 ],
            [ -9.43, 1.37, 1.27 ],
            [ -9.22, -4.03, 1.23 ],
            [ -9.50, -0.98, 1.25 ]
        ]
    }
}

# Provide a filename, category, label, and optional metadata for each sample
my_samples = [
    {
        "filename": "001.json",
        "data": io.BytesIO(json.dumps(sample_data_1).encode('utf-8')),
        "category": "training",
        "label": "idle",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
    {
        "filename": "002.json",
        "data": io.BytesIO(json.dumps(sample_data_2).encode('utf-8')),
        "category": "training",
        "label": "wave",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
]

# Wrap the samples in instances of the Sample class
samples = [ei.data.sample_type.Sample(**i) for i in my_samples]

# Upload samples to your project
response = ei.experimental.data.upload_samples(samples)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

If you head to the Data acquisition page on your project, you should see your time series data.

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload NumPy arrays

NumPy is powerful Python library for working with large arrays and matrices. You can upload NumPy arrays directly into your Edge Impulse project. Note that the arrays are required to be in a particular format, and must be uploaded with required metadata (such as a list of labels and the sample rate).

Important! NumPy arrays must be in the shape (Number of samples, number of data points, number of sensors)

If you are working with image data in NumPy, we recommend saving those images as .png or .jpg files and using upload_directory().

import numpy as np

# Create example NumPy array with 2 time series samples
sample_data = np.array(
    [
        [ # Sample 1 ("idle")
            [-9.81, 0.03, 0.21],
            [-9.83, 0.04, 0.27],
            [-9.12, 0.03, 0.23],
            [-9.14, 0.01, 0.25],
        ],
        [ # Sample 2 ("wave")
            [-9.56, 5.34, 1.21],
            [-9.43, 1.37, 1.27],
            [-9.22, -4.03, 1.23],
            [-9.50, -0.98, 1.25],
        ],
    ]
)

# Labels for each sample
labels = ["idle", "wave"]

# Names of the sensors and units for the 3 axes
sensors = [
    { "name": "accX", "units": "m/s2" },
    { "name": "accY", "units": "m/s2" },
    { "name": "accZ", "units": "m/s2" },
]

# Optional metadata for all samples being uploaded
metadata = {
    "source": "accelerometer",
    "collection site": "desk",
}

# Upload samples to your project
response = ei.experimental.data.upload_numpy(
    data=sample_data,
    labels=labels,
    sensors=sensors,
    sample_rate_ms=10,
    metadata=metadata,
    category="training",
)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

If you head to the Data acquisition page on your project, you should see your time series data. Note that the sample names are randomly assigned, so we recommend recording the sample IDs when you upload.

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload pandas (and pandas-like) dataframes

pandas is popular Python library for performing data manipulation and analysis. The Edge Impulse library supports a number of ways to upload dataframes. We will go over each format.

Note that several other packages exist that work as drop-in replacements for pandas. You can use these replacements so long as you import that with the name pd. For example, one of:

import pandas as pd
import modin.pandas as pd
import dask.dataframe as pd
import polars as pd

import pandas as pd

The first option is to upload one dataframe for each sample (non-time series)

# Construct one dataframe for each sample (multidimensional, non-time series)
df_1 = pd.DataFrame([[-9.81, 0.03, 0.21]], columns=["accX", "accY", "accZ"])
df_2 = pd.DataFrame([[-9.56, 5.34, 1.21]], columns=["accX", "accY", "accZ"])

# Optional metadata for all samples being uploaded
metadata = {
    "source": "accelerometer",
    "collection site": "desk",
}

# Upload the first sample
ids = []
response = ei.experimental.data.upload_pandas_sample(
    df_1,
    label="One",
    filename="001",
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Upload the second sample
response = ei.experimental.data.upload_pandas_sample(
    df_2,
    label="Two",
    filename="002",
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

You can also upload one dataframe for each sample (time series). As with previous examples, we'll assume that the sample rate is 10 ms.

# Create samples (multidimensional, time series)
sample_data_1 = [ # Sample 1 ("idle")
    [-9.81, 0.03, 0.21],
    [-9.83, 0.04, 0.27],
    [-9.12, 0.03, 0.23],
    [-9.14, 0.01, 0.25],
]
sample_data_2 = [ # Sample 1 ("wave")
    [-9.56, 5.34, 1.21],
    [-9.43, 1.37, 1.27],
    [-9.22, -4.03, 1.23],
    [-9.50, -0.98, 1.25],
]

# Construct one dataframe for each sample
df_1 = pd.DataFrame(sample_data_1, columns=["accX", "accY", "accZ"])
df_2 = pd.DataFrame(sample_data_2, columns=["accX", "accY", "accZ"])

# Optional metadata for all samples being uploaded
metadata = {
    "source": "accelerometer",
    "collection site": "desk",
}

# Upload the first sample
ids = []
response = ei.experimental.data.upload_pandas_sample(
    df_1,
    label="Idle",
    filename="001",
    sample_rate_ms=10,
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Upload the second sample
response = ei.experimental.data.upload_pandas_sample(
    df_2,
    label="Wave",
    filename="002",
    sample_rate_ms=10,
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

You can upload non-time series data where each sample is a row in the dataframe. Note that you need to provide labels in the rows.

# Construct non-time series data, where each row is a different sample
data = [
    ["desk", "training", "One", -9.81, 0.03, 0.21],
    ["field", "training", "Two", -9.56, 5.34, 1.21],
]
columns = ["loc", "category", "label", "accX", "accY", "accZ"]

# Wrap the data in a DataFrame
df = pd.DataFrame(data, columns=columns)

# Upload non-time series DataFrame (with multiple samples) to the project
ids = []
response = ei.experimental.data.upload_pandas_dataframe(
    df,
    feature_cols=["accX", "accY", "accZ"],
    label_col="label",
    category_col="category",
    metadata_cols=["loc"],
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

A "wide" dataframe is one where each column represents a value in the time series data, and the rows become individual samples. Note that you need to provide labels in the rows.

# Construct time series data, where each row is a different sample
data = [
    ["desk", "training", "idle", 0.8, 0.7, 0.8, 0.9, 0.8, 0.8, 0.7, 0.8],
    ["field", "training", "motion", 0.3, 0.9, 0.4, 0.6, 0.8, 0.9, 0.5, 0.4],
]
columns = ["loc", "category", "label", "0", "1", "2", "3", "4", "5", "6", "7"]

# Wrap the data in a DataFrame
df = pd.DataFrame(data, columns=columns)

# Upload time series DataFrame (with multiple samples) to the project
ids = []
response = ei.experimental.data.upload_pandas_dataframe_wide(
    df,
    label_col="label",
    category_col="category",
    metadata_cols=["loc"],
    data_col_start=3,
    sample_rate_ms=100,
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

A DataFrame can also be divided into "groups" so you can upload multidimensional time series data.

# Create samples
sample_data = [
    ["desk", "sample 1", "training", "idle", 0, -9.81, 0.03, 0.21],
    ["desk", "sample 1", "training", "idle", 0.01, -9.83, 0.04, 0.27],
    ["desk", "sample 1", "training", "idle", 0.02, -9.12, 0.03, 0.23],
    ["desk", "sample 1", "training", "idle", 0.03, -9.14, 0.01, 0.25],
    ["field", "sample 2", "training", "wave", 0, -9.56, 5.34, 1.21],
    ["field", "sample 2", "training", "wave", 0.01, -9.43, 1.37, 1.27],
    ["field", "sample 2", "training", "wave", 0.02, -9.22, -4.03, 1.23],
    ["field", "sample 2", "training", "wave", 0.03, -9.50, -0.98, 1.25],
]
columns = ["loc", "sample_name", "category", "label", "timestamp", "accX", "accY", "accZ"]

# Wrap the data in a DataFrame
df = pd.DataFrame(sample_data, columns=columns)

# Upload time series DataFrame (with multiple samples and multiple dimensions) to the project
ids = []
response = ei.experimental.data.upload_pandas_dataframe_with_group(
    df,
    group_by="sample_name",
    timestamp_col="timestamp",
    feature_cols=["accX", "accY", "accZ"],
    label_col="label",
    category_col="category",
    metadata_cols=["loc"]
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Trigger connected board data sampling

1. Obtain an API key from your project

Your project API key can be used to enable programmatic access to Edge Impulse. You can create and/or obtain a key from your project's Dashboard, under the Keys tab. API keys are long strings, and start with ei_:

2. Connect your development kit to your project

Open a terminal and run the Edge Impulse daemon. The daemon is the service that connects your hardware with any Edge Impulse project:

edge-impulse-daemon --api-key <your project API key>

3. Obtain your project's ID

Copy your project's ID from the project's Dashboard under the Project Info section:

4. Setup API Connection

Replace the PROJECT_ID below with the ID of your project you selected and enter your API key when prompted:

import requests
import getpass
import json

URL_STUDIO = "https://studio.edgeimpulse.com/v1/api/"
PROJECT_ID = int(input('Enter your Project ID: '))
AUTH_KEY = getpass.getpass('Enter your API key: ')


def check_response(response, debug=False):
    if not response.ok:
        raise RuntimeError("⛔️ Error\n%s" % response.text)
    else:
        if debug:
            print(response)
        return response


def do_get(url, auth, debug=False):
    if debug:
        print(url)
    response = requests.get(url,
                            headers={
                                "Accept": "application/json",
                                "x-api-key": auth
                            })
    return check_response(response, debug)


def parse_response(response, key=""):
    parsed = json.loads(response.text)
    if not parsed["success"]:
        raise RuntimeError(parsed["error"])
    if key == "":
        return json.loads(response.text)
    return json.loads(response.text)[key]


def get_project(project_id, project_auth, debug=False):
    response = do_get(URL_STUDIO + str(project_id), project_auth)
    return parse_response(response, "project")


print("Project %s is accessible" % get_project(PROJECT_ID, AUTH_KEY)["name"])

5. Get the ID of the connected device

# https://studio.edgeimpulse.com/v1/api/{projectId}/devices

def get_devices(project_id, project_auth, debug=False):
    response = do_get(URL_STUDIO + str(project_id) + "/devices", project_auth)
    return parse_response(response, "devices")


device_id = ""
for device in get_devices(PROJECT_ID, AUTH_KEY):
    # if device["remote_mgmt_connected"] and device["supportsSnapshotStreaming"]:
    if device["remote_mgmt_connected"]:
        device_id = device["deviceId"]
        print("Found %s (type %s, id: %s)" %
              (device["name"], device["deviceType"], device_id))
        break
if device_id == "":
    print(
        "Could not find a connected device that supports snapshot streaming!")

6. Trigger data sampling

# https://studio.edgeimpulse.com/v1/api/{projectId}/device/{deviceId}/start-sampling

SAMPLE_CATEGORY = "testing"
SAMPLE_LENGTH_MS = 20000
SAMPLE_LABEL = "squat"

def do_post(url, payload, auth, debug=False):
    if debug:
        print(url)
    response = requests.post(url,
                             headers={
                                 "Accept": "application/json",
                                 "x-api-key": auth
                             },
                             json=payload)
    return check_response(response, debug)


def collect_sample(project_id, device_id, project_auth, debug=False):
    payload = {
        "category": SAMPLE_CATEGORY,
        # "Microphone", "Inertial", "Environmental" or "Inertial + Environmental"
        "sensor": "Inertial",
        # The inverse of frequency in Hz
        "intervalMs": 10,
        "label": SAMPLE_LABEL,
        "lengthMs": SAMPLE_LENGTH_MS
    }
    response = do_post(
        URL_STUDIO + str(project_id) + "/device/" + str(device_id) +
        "/start-sampling", payload, project_auth, debug)
    return parse_response(response, "id")


print("Sample request returned", collect_sample(PROJECT_ID, device_id, AUTH_KEY))

Ingest multi-labeled data using the API

This guide provides a quick walk-through on how to upload and update time-series data with multiple labels using the Edge Impulse API.

Getting Started

Prerequisites

API Key - Obtain from your project settings on Edge Impulse.
Example Github repository - Clone the following repository, it contains the scripts and the example files:
```
git clone git@github.com:edgeimpulse/example-multi-label-ingestion-via-api.git
```

Setup Your Environment

Export your API key to use in the upload scripts:

export EI_PROJECT_API_KEY='your_api_key_here'

Prepare Your Data

Data File: JSON formatted time-series data. See the Data acquisition format speicification
structured_labels.labels File: JSON with structured labels. See the specification format

Upload Data

Use the upload.sh script to send your data and labels to Edge Impulse:

#!/bin/bash
if [[ -z "$EI_PROJECT_API_KEY" ]]; then
    echo "Missing EI_PROJECT_API_KEY env variable"
    exit 1
fi

curl -X POST \
    -H "x-api-key: $EI_PROJECT_API_KEY" \
    -H "Content-Type: multipart/form-data" \
    -F "data=@updown.3.json" \
    -F "data=@structured_labels.labels" \
    https://ingestion.edgeimpulse.com/api/training/files

Update Existing Samples

To update a sample, run update-sample.sh with the required project and sample IDs:

sh update-sample.sh --project-id <your_project_id> --sample-id <your_sample_id>

We hope this tutorial has helped you to understand how to ingest multi-label data samples to your Edge Impulse project. If you have any questions, please reach out to us on our forum.

Synthetic data

Synthetic datasets are a collection of data artificially generated rather than being collected from real-world observations or measurements. They are created using algorithms, simulations, or mathematical models to mimic the characteristics and patterns of real data. Synthetic datasets are a valuable tool to generate data for experimentation, testing, and development when obtaining real data is challenging, costly, or undesirable.

You might want to generate synthetic datasets for several reasons:

Cost Efficiency: Creating synthetic data can be more cost-effective and efficient than collecting large volumes of real data, especially in resource-constrained environments.

Data Augmentation: Synthetic datasets allow users to augment their real-world data with variations, which can improve model robustness and performance.

Data Diversity: Synthetic datasets enable the inclusion of uncommon or rare scenarios, enriching model training with a wider range of potential inputs.

Privacy and Security: When dealing with sensitive data, synthetic datasets provide a way to train models without exposing real information, enhancing privacy and security.

You can generate synthetic data directly from Edge Impulse using the Synthetic Data tab in the Data acquisition view. This tab provides a user-friendly interface to generate synthetic data for your projects. You can create synthetic datasets using a variety of tools and models.

We have put together the following tutorials to help you get started with synthetic datasets generation:

Using integrated models directly available inside Edge Impulse Studio

Note that you will need an API Key/Access Token from the different providers to run the model used to generate the synthetic data.

Generate audio datasets using Eleven Labs

Generate audio data using the . This integration allows you to generate realistic sound effects for your projects, such as glass breaking, car engine revving, or other custom sounds. You can customize the sound prompts and generate high-quality audio samples for your datasets.

This integration allows you to expand your datasets with sounds that may be difficult or expensive to record naturally. This approach not only saves time and money but also enhances the accuracy and reliability of the models we deploy on edge devices.

Example: Glass Breaking Sound

In this tutorial, we focus on a practical application that can be used in a smart security system, or in a factory to detect incidents, such as detecting the sounds of glass breaking.

There is also a video version of this guide:

Getting Started

Only available with Edge Impulse Pro Plan and Enterprise Plan

Navigate to Data Acquisition: Once you're in your project, navigate to the Data Acquisition section, go to Synthetic data and select the ElevenLabs Synthetic Audio Generator data source.

Step 1: Get your ElevenLabs API Key

First, get your Eleven Labs API Key. Navigate to the Eleven Labs web interface to get your key and optionally test your prompt.

Step 2: Parameters

Here we will be trying to collect a glass-breaking sound or impact.

Prompt: "glass breaking"
Simple prompts are just that: they are simple, one-sided prompts where we try to get the AI to generate a single sound effect. This could be, for example, “person walking on grass” or “glass breaking.” These types of prompts will generate a single type of sound effect with a few variations either in the same generation or in subsequent generations. All in all, they are fairly simple.
There are a few ways to improve these prompts, however, and that is by adding a little bit more detail. Even if they are simple prompts, they can be made to give better output by improving the prompt itself. For example, something that sometimes works is adding details like “high-quality, professionally recorded footsteps on grass, sound effects foley.” It can require some experimentation to find a good balance between being descriptive and keeping it brief enough to have AI understand the prompt. e.g. high quality audio of window glass breaking
Label: The label of the generated audio sample.
Prompt influence: Between 0 and 1, this setting ranges from giving the AI more creativity in how it interprets the prompt to telling the AI to be more strict in following the exact prompt that you’ve given. 1 being more creative.
Number of samples: Number of samples to generate
Minimum length (seconds): Minimum length of generated audio samples. Audio samples will be padded with silence to minimum length. It also determines how long your generations should be. Depending on what you set this as, you can get quite different results. For example, if I write “kick drum” and set the length to 11 seconds, I might get a full drum loop with a kick drum in it, but that might not be what I want. On the other hand, if I set the length to 1 second, I might just get a one-shot with a single instance of a kick drum.
Frequency (Hz): Audio frequency, ElevenLabs generates data at 44100Hz, so any other value will be resampled.
Upload to category: Data will be uploaded to this category in your project.

Step 3: Generate samples

Once you've set up your prompt, and api key, run the pipeline to generate the sound samples. You can then view the output in the Data Acquisition section.

Benefits of Using Generative AI for Sound Generation

Enhance Data Quality: Generative AI can create high-quality sound samples that are difficult to record naturally.
Increase Dataset Diversity: Access a wide range of sounds to enrich your training dataset and improve model performance.
Save Time and Resources: Quickly generate the sound samples you need without the hassle of manual recording.
Improve Model Accuracy: High-quality, diverse sound samples can help fill gaps in your dataset and enhance model performance.

Conclusion

By leveraging generative AI for sound generation, you can enhance the quality and diversity of your training datasets, leading to more accurate and reliable edge AI models. This innovative approach saves time and resources while improving the performance of your models in real-world applications. Try out the Eleven Labs block in Edge Impulse today and start creating high-quality sound datasets for your projects.

Generate image datasets using Dall-E

This notebook explores how we can use generative AI to create datasets which don't exist yet. This can be a good starting point for your project if you have not collected or cannot collect the data required. It is important to note the limitations of generative AI still apply here, biases can be introduced through your prompts, results can include "hallucinations" and quality control is important.

This example uses the OpenAI API to call the Dall-E image generation tool, it explores both generation and variation but there are other tools such as editing which could also be useful for augmenting an existing dataset.

There is also a video version of this tutorial:

Local Software Requirements

Python 3
Pip package manager
Jupyter Notebook: https://jupyter.org/install
pip packages (install with pip installpackagename):
- openai https://pypi.org/project/openai/

! pip install openai

# Imports
import openai
import os
import requests

# Notebook Imports
from IPython.display import Image
from IPython.display import display
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

Set up OpenAI API

First off you will need to set up and Edge Impulse account and create your first project.

You will also need to create an API Key for OpenAI: https://platform.openai.com/docs/api-reference/authentication


# You can set your API key and org as environment variables in your system like this:
# os.environ['OPENAI_API_KEY'] = 'api string'

# Set up OpenAI API key and organization
openai.api_key = os.getenv("OPENAI_API_KEY")

Generate your first image

The API takes in a prompt, number of images and a size

image_prompt = "A webcam image of a human 1m from the camera sitting at a desk showing that they are wearing gloves with their hands up to the camera."
# image_prompt = "A webcam image of a person 1m from the camera sitting at a desk with their bare hands up to the camera."
# image_prompt = "A webcam image of a human 1m from the camera sitting at a desk showing that they are wearing wool gloves with their hands up to the camera."

response = openai.Image.create(
    prompt=image_prompt,
    n=1,
    size="256x256",
)
Image(url=response["data"][0]["url"])

Generate some variations of this image

The API also has a variations call which takes in an existing images and creates variations of it. This could also be used to modify existing images.

response2 = openai.Image.create_variation(
  image=requests.get(response['data'][0]['url']).content,
  n=3,
  size="256x256"
)
imgs = []
for img in response2['data']:
  imgs.append(Image(url=img['url']))

display(*imgs)

Generate a dataset:

Here we are iterate through a number of images and variations to generate a dataset based on the prompts/labels given.

labels = [{"prompt": "A webcam image of a human 1m from the camera sitting at a desk showing that they are wearing wool gloves with their hands up to the camera.",
          "label": "gloves"},
          {"prompt": "A webcam image of a person 1m from the camera sitting at a desk with their bare hands up to the camera.",
          "label": "no-gloves"}
        ]
output_folder = "output"
base_images_number = 10
variation_per_image = 3
# Check if output directory for noisey files exists and create it if it doesn't
if not os.path.exists(output_folder):
    os.makedirs(output_folder)

for option in labels:
    for i in range(base_images_number):
        response = openai.Image.create(
            prompt=option["prompt"],
            n=1,
            size="256x256",
        )
        try:
            img = response["data"][0]["url"]
            with open(f'{output_folder}/{option["label"]}.{img.split("/")[-1]}.png', 'wb+') as f:
                f.write(requests.get(img).content)
            response2 = openai.Image.create_variation(
                image=requests.get(img).content,
                n=variation_per_image,
                size="256x256"
            )
        except Exception as e:
            print(e)
        for img in response2['data']:
            try:
                with open(f'{output_folder}/{option["label"]}.{img["url"].split("/")[-1]}.png', 'wb') as f:
                    f.write(requests.get(img["url"]).content)
            except Exception as e:
                print(e)

Plot all the output images:

import os


# Define the folder containing the images
folder_path = './output'

# Get a list of all the image files in the folder
image_files = [f for f in os.listdir(folder_path) if os.path.isfile(os.path.join(folder_path, f)) and f.endswith('.png')]

# Set up the plot
fig, axs = plt.subplots(nrows=20, ncols=20, figsize=(10, 10))

# Loop through each image and plot it in a grid cell
for i in range(20):
    for j in range(20):
        img = mpimg.imread(os.path.join(folder_path, image_files[i*10+j]))
        axs[i,j].imshow(img)
        axs[i,j].axis('off')

# Make the plot look clean
fig.subplots_adjust(hspace=0, wspace=0)
plt.tight_layout()
plt.show()

These files can then be uploaded to a project with these commands (run in a separate terminal window):

! cd output
! edge-impulse-uploader .

(run edge-impulse-uploader --clean if you have used the CLI before to reset the target project)

What next?

Now you can use your images to create an image classification model on Edge Impulse.

Why not try some other OpenAI calls, 'edit' could be used to take an existing image and translate it into different environments or add different humans to increase the variety of your dataset. https://platform.openai.com/docs/guides/images/usage

Generate keyword spotting datasets using Google TTS

Local Software Requirements

Python 3
Pip package manager
Jupyter Notebook: https://jupyter.org/install
pip packages (install with pip installpackagename):
- pydub https://pypi.org/project/pydub/
- google-cloud-texttospeech https://cloud.google.com/python/docs/reference/texttospeech/latest
- requests https://pypi.org/project/requests/

# Imports
import os
import json
import time
import io
import random
import requests
from pydub import AudioSegment
from google.cloud import texttospeech

Set up Google TTS API

First off you will need to set up and Edge Impulse account and create your first project. You will also need a Google Cloud account with the Text to Speech API enabled: https://cloud.google.com/text-to-speech, the first million characters generated each month are free (WaveNet voices), this should be plenty for most cases as you'll only need to generate your dataset once. From google you will need to download a credentials JSON file and set it to the correct environment variable on your system to allow the python API to work: (https://developers.google.com/workspace/guides/create-credentials#service-account)


# Insert the path to your service account API key json file here for google cloud
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '../path-to-google-credentials-file.json'

Generate the desired samples

First off we need to set our desired keywords and labels:


# Keyword or short sentence and label (e.g. 'hello world')
keyword = [
    {'string':'edge','label':'edge'},
    {'string':'impulse','label':'impulse'},
           ]

Then we need to set up the parameters for our speech dataset, all possible combinations will be iterated through:

languages - Choose the text to speech voice languages to use (https://cloud.google.com/text-to-speech/docs/voices)
pitches - Which voice pitches to apply
genders - Which SSML genders to apply
speakingRates - Which speaking speeds to apply



# Languages, remove as appropriate
# languages = [
#     'ar-XA', 'bn-IN',  'en-GB',  'fr-CA',
#     'en-US', 'es-ES',  'fi-FI',  'gu-IN',
#     'ja-JP', 'kn-IN',  'ml-IN',  'sv-SE',
#     'ta-IN', 'tr-TR',  'cs-CZ',  'de-DE',
#     'en-AU', 'en-IN',  'fr-FR',  'hi-IN',
#     'id-ID', 'it-IT',  'ko-KR',  'ru-RU',
#     'uk-UA', 'cmn-CN', 'cmn-TW', 'da-DK',
#     'el-GR', 'fil-PH', 'hu-HU',  'nb-NO',
#     'nl-NL', 'pt-PT',  'sk-SK',  'vi-VN',
#     'pl-PL', 'pt-BR',  'ca-ES',  'yue-HK',
#     'af-ZA', 'bg-BG',  'lv-LV',  'ro-RO',
#     'sr-RS', 'th-TH',  'te-IN',  'is-IS'
# ]
languages = [
    'en-GB',
    'en-US',
]
# Pitches to generate (in semitones) range: [-20.0, 20.0]
pitches = [-2, 0, 2]
# Voice genders to use
genders = ["NEUTRAL", "FEMALE", "MALE"]
# Speaking rates to use range: [0.25, 4.0]
speakingRates = [0.9, 1, 1.1]

Then provide some other key parameters:

out_length - How long each output sample should be
count - Maximum number of samples to output (if all combinations of languages, pitches etc are higher then this restricts output)
voice-dir - Where to store the clean samples before noise is added
noise-url - Which noise file to download and apply to your samples
output-folder - The final output location of the noised samples
num-copies - How many different noisy versions of each sample to create
max-noise-level - in Db,

# Out length minimum (default: 1s)
out_length = 1
# Maximum number of keywords to generate
count = 30
# Raw sample output directory
voice_dir = 'out-wav'
# Creative commons background noise from freesound.org:https://freesound.org/people/Astounded/sounds/483561/
noise_url = 'https://cdn.freesound.org/previews/483/483561_10201334-lq.ogg'
output_folder = 'out-noisy'
num_copies = 2  # Number of noisy copies to create for each input sample
max_noise_level = -5  # Maximum noise level to add in dBFS (negative value)

Then we need to check all the output folders are ready


# Check if output directory for noisey files exists and create it if it doesn't
if not os.path.exists(output_folder):
    os.makedirs(output_folder)
# Check if output directory for raw voices exists and create it if it doesn't
if not os.path.exists(voice_dir):
    os.makedirs(voice_dir)

And download the background noise file


# Download background noise file
response = requests.get(noise_url)
response.raise_for_status()
noise_audio = AudioSegment.from_file(io.BytesIO(response.content), format='ogg')

Then we can generate a list of all possible parameter combinations based on the input earlier. If you have set num_copies to be smaller than the number of combinations then these options will be reduced:


# Generate all combinations of parameters
all_opts = []
for p in pitches:
    for g in genders:
        for l in languages:
            for s in speakingRates:
                for kw in keyword:
                    all_opts.append({
                            "pitch": p,
                            "gender": g,
                            "language": l,
                            "speakingRate": s,
                            "text": kw['string'],
                            "label": kw['label']
                        })
if len(all_opts)*num_copies > count:
    selectEvery = len(all_opts)*num_copies // count
    selectNext = 0
    all_opts = all_opts[::selectEvery]
print(f'Generating {len(all_opts)*num_copies} samples')

Finally we iterate though all the options generated, call the Google TTS API to generate the desired sample, and apply noise to it, saving locally with metadata:


# Instantiate list for file label information
downloaded_files = []

# Instantiates a client
client = texttospeech.TextToSpeechClient()

ix = 0
for o in all_opts:
    ix += 1
    # Set the text input to be synthesized
    synthesis_input = texttospeech.SynthesisInput(text=o['text'])
    # Build the voice request
    voice = texttospeech.VoiceSelectionParams(
        language_code=o['language'],
        ssml_gender=o['gender']
    )
    # Select the type of audio file you want returned
    audio_config = texttospeech.AudioConfig(
        audio_encoding=texttospeech.AudioEncoding.LINEAR16,
        pitch=o['pitch'],
        speaking_rate=o['speakingRate'],
        sample_rate_hertz=16000
    )
    # Perform the text-to-speech request on the text input with the selected
    # voice parameters and audio file type

    wav_file_name = f"{voice_dir}/{o['label']}.{o['language']}-{o['gender']}-{o['pitch']}-{o['speakingRate']}.tts.wav"

    if not os.path.exists(wav_file_name):
        print(f"[{ix}/{len(all_opts)}] Text-to-speeching...")
        response = client.synthesize_speech(
            input=synthesis_input, voice=voice, audio_config=audio_config
        )
        with open(wav_file_name, "wb") as f:
            f.write(response.audio_content)
        has_hit_api = True
    else:
        print(f'skipping {wav_file_name}')
        has_hit_api = False

    # Load voice sample
    voice_audio = AudioSegment.from_file(wav_file_name)
    # Add silence to match output length with random padding
    difference = (out_length * 1000) - len(voice_audio)
    if difference > 0:
        padding_before = random.randint(0, difference)
        padding_after = difference - padding_before
        voice_audio = AudioSegment.silent(duration=padding_before) +  voice_audio + AudioSegment.silent(duration=padding_after)

    for i in range(num_copies):
        # Save noisy sample to output folder
        output_filename = f"{o['label']}.{o['language']}-{o['gender']}-{o['pitch']}-{o['speakingRate']}_noisy_{i+1}.wav"
        output_path = os.path.join(output_folder, output_filename)
        if not os.path.exists(output_path):
            # Select random section of noise and random noise level
            start_time = random.randint(0, len(noise_audio) - len(voice_audio))
            end_time = start_time +len(voice_audio)
            noise_level = random.uniform(max_noise_level, 0)

            # Extract selected section of noise and adjust volume
            noise_segment = noise_audio[start_time:end_time]
            noise_segment = noise_segment - abs(noise_level)

            # Mix voice sample with noise segment
            mixed_audio = voice_audio.overlay(noise_segment)
            # Save mixed audio to file
            mixed_audio.export(output_path, format='wav')

            print(f'Saved mixed audio to {output_path}')
        else:
            print(f'skipping {output_path}')
        # Save metadata for file
        downloaded_files.append({
            "path": str(output_filename),
            "label": o['label'],
            "category": "split",
            "metadata": {
                "pitch": str(['pitch']),
                "gender": str(o['gender']),
                "language": str(o['language']),
                "speakingRate": str(o['speakingRate']),
                "text": o['text'],
                "imported_from": "Google Cloud TTS"
            }
        })

    if has_hit_api:
        time.sleep(0.5)

print("Done text-to-speeching")
print("")

input_file = os.path.join(output_folder, 'input.json')
info_file = {
    "version": 1,
    "files": downloaded_files
}
# Output the metadata file
with open(input_file, "w") as f:
    json.dump(info_file, f)

# Move to the out-noisy folder
! cd out-noisy
# Upload all files in the out-noisy folder with metadata attached in the input.json file
! edge-impulse-uploader --info-file input.json *

What next?

Now you can use your keywords to create a robust keyword detection model in Edge Impulse Studio!

Try out both classification models and the transfer learning keyword spotting model to see which works best for your case

Generate physics simulation datasets using PyBullet

This notebook takes you through a basic example of using the physics simulation tool PyBullet to generate an accelerometer dataset representing dropping the Nordic Thingy:53 devkit from different heights. This dataset can be used to train a regression model to predict drop height.

This idea could be used for a wide range of simulatable environments- for example generating accelerometer datasets for pose estimation or fall detection. The same concept could be applied in an FMEA application for generating strain datasets for structural monitoring.

There is also a video version of this tutorial:

Local Software Requirements

Python 3
Pip package manager

The dependencies can be installed with:

! pip install pybullet numpy

# Imports
import pybullet as p
import pybullet_data
import os
import shutil
import csv
import random
import numpy as np
import json

Create object to simulate

We need to load in a Universal Robotics Description Format file describing an object with the dimensions and weight of a Nordic Thingy:53. In this case, measuring our device it is 64x60x23.5mm and its weight 60g. The shape is given by a .obj 3D model file.

	<visual>
		<origin xyz="0.02977180615936878 -0.01182944632717566 0.03176079914341195" rpy="1.57079632679 0.0 0.0" />
		<geometry>
			<mesh filename="thingy53/thingy53 v2.obj" scale="1 1 1" />
		</geometry>
		<material name="texture">
			<color rgba="1.0 1.0 1.0 1.0" />
		</material>
	</visual>
	<collision>
		<origin xyz="0.02977180615936878 -0.01182944632717566 0.03176079914341195" rpy="1.57079632679 0.0 0.0" />
		<geometry>
			<mesh filename="thingy53/thingy53 v2.obj" scale="1 1 1" />
		</geometry>
	</collision>
	<inertial>
		<mass value="0.06" />
  		<inertia ixx="0.00002076125" ixy="0" ixz="0" iyy="0.00002324125" iyz="0" izz="0.00003848"/>
	</inertial>
</link>

Visualizing the problem

To generate the required data we will be running PyBullet in headless "DIRECT" mode so we can iterate quickly over the parameter field. If you run the python file below you can see how pybullet simulates the object dropping onto a plane

! python ../.assets/pybullet/single_simulation.py

Setting up the simulation environment

First off we need to set up a pybullet physics simulation environment. We load in our object file and a plane for it to drop onto. The plane's dynamics can be adjusted to better represent the real world (in this case we're dropping onto carpet)

# Set up PyBullet physics simulation (change from p.GUI to p.DIRECT for headless simulation)
physicsClient = p.connect(p.DIRECT)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setGravity(0, 0, -9.81)

# Load object URDF file
obj_file = "../.assets/pybullet/thingy53/thingy53.urdf"
obj_id = p.loadURDF(obj_file, flags=p.URDF_USE_INERTIA_FROM_FILE)

# Add a solid plane for the object to collide with
plane_id = p.loadURDF("plane.urdf")

# Set length of simulation and sampling frequency
sample_length = 2 # Seconds
sample_freq = 100 # Hz

We also need to define the output folder for our simulated accelerometer files

output_folder = 'output/'
# Check if output directory for noisey files exists and create it if it doesn't
if not os.path.exists(output_folder):
    os.makedirs(output_folder)
else:
    shutil.rmtree(output_folder)
    os.makedirs(output_folder)

And define the drop parameters

# Simulate dropping object from range of heights
heights = 100
sims_per_height = 20
min_height = 0.1 # Metres
max_height = 0.8 # Metres

We also need to define the characteristics of the IMU on the real device we are trying to simulate. In this case the Nordic Thingy:53 has a Bosch BMI270 IMU (https://www.bosch-sensortec.com/products/motion-sensors/imus/bmi270/) which is set to a range of +-2g with a resolution of 0.06g. These parameters will be used to restrict the raw acceleration output:


range_g = 2
range_acc = range_g * 9.81
resolution_mg = 0.06
resolution_acc = resolution_mg / 1000.0 * 9.81

Finally we are going to give the object and plane restitution properties to allow for some bounce. In this case I dropped the real Thingy:53 onto a hardwood table. You can use p.changeDynamics to introduce other factors such as damping and friction.

p.changeDynamics(obj_id, -1, restitution=0.3)
p.changeDynamics(plane_id, -1, restitution=0.4)

Drop simulation

Here we iterate over a range of heights, randomly changing its start orientation for i number of simulations per height. The acceleration is calculated relative to the orientation of the Thingy:53 object to represent its onboard accelerometer.

metadata = []
for height in np.linspace(max_height, min_height, num=heights):
    print(f"Simulating {sims_per_height} drops from {height}m")
    for i in range(sims_per_height):
        # Set initial position and orientation of object
        x = 0
        y = 0
        z = height
        orientation = p.getQuaternionFromEuler((random.uniform(0, 2 * np.pi), random.uniform(0, 2 * np.pi), random.uniform(0, 2 * np.pi)))
        p.resetBasePositionAndOrientation(obj_id, [x, y, z], orientation)
        
        prev_linear_vel = np.zeros(3)

        # Initialize the object position and velocity
        pos_prev, orn_prev = p.getBasePositionAndOrientation(obj_id)
        vel_prev, ang_vel_prev = p.getBaseVelocity(obj_id)
        timestamp=0
        dt=1/sample_freq
        p.setTimeStep(dt)
        filename=f"drop_{height}m_{i}.csv"
        with open(f"output/{filename}", mode="w") as csv_file:
                writer = csv.writer(csv_file)
                writer.writerow(['timestamp','accX','accY','accZ'])
        while timestamp < sample_length:
            p.stepSimulation()
            linear_vel, angular_vel = p.getBaseVelocity(obj_id)
            lin_acc = [(v - prev_v)/dt for v, prev_v in zip(linear_vel, prev_linear_vel)]
            prev_linear_vel = linear_vel
            timestamp += dt
            # Get the current position and orientation of the object
            pos, orn = p.getBasePositionAndOrientation(obj_id)

            # Get the linear and angular velocity of the object in world coordinates
            vel, ang_vel = p.getBaseVelocity(obj_id)

             # Calculate the change in position and velocity between steps
            pos_diff = np.array(pos) - np.array(pos_prev)
            vel_diff = np.array(vel) - np.array(vel_prev)

            # Convert the orientation quaternion to a rotation matrix
            rot_matrix = np.array(p.getMatrixFromQuaternion(orn)).reshape(3, 3)

            # Calculate the local linear acceleration of the object, subtracting gravity
            local_acc = np.dot(rot_matrix.T, vel_diff / dt) - np.array([0, 0, -9.81])
            # Restrict the acceleration to the range of the accelerometer
            imu_rel_lin_acc_scaled = np.clip(local_acc, -range_acc, range_acc)
            # Round the acceleration to the nearest resolution of the accelerometer
            imu_rel_lin_acc_rounded = np.round(imu_rel_lin_acc_scaled/resolution_acc) * resolution_acc
            # Update the previous position and velocity
            pos_prev, orn_prev = pos, orn
            vel_prev, ang_vel_prev = vel, ang_vel

            # Save acceleration data to CSV file
            with open(f"{output_folder}{filename}", mode="a") as csv_file:
                writer = csv.writer(csv_file)
                writer.writerow([timestamp*1000] + imu_rel_lin_acc_rounded.tolist())

        nearestheight = round(height, 2)
        metadata.append({
            "path": filename,
            "category": "training",
            "label": { "type": "label", "label": str(nearestheight)}
        })

Finally we save the metadata file to the output folder. This can be used to tell the edge-impulse-uploader CLI tool the floating point labels for each file.

jsonout = {"version": 1, "files": metadata}

with open(f"{output_folder}/files.json", "w") as f:
    json.dump(jsonout, f)

# Disconnect from PyBullet physics simulation
p.disconnect()

These files can then be uploaded to a project with these commands (run in a separate terminal window):

! cd output
! edge-impulse-uploader --info-file files.json

(run edge-impulse-uploader --clean if you have used the CLI before to reset the target project)

What next?

Now you can use your dataset a drop height detection regression model in Edge Impulse Studio!

See if you can edit this project to simulate throwing the object up in the air to predict the maximum height, or add in your own custom object. You could also try to better model the real environment you're dropping the object in- adding air resistance, friction, damping and material properties for your surface.

Labeling

Label audio data using your existing models

This example comes from the that has been slightly modify to upload the raw data back to Edge Impulse based on the inference results.

To run the example:

Clone this repository:

git clone git@github.com:edgeimpulse/example-active-learning-linux-python-sdk.git

Install the dependencies:

pip3 install -r requirements.txt

Grab your the API key of the project you want to upload the inferred results raw data:

Past the new key in the EI_API_KEY variable in the audio-classify-export.py file. Alternatively, load it from your environment variable:

export EI_API_KEY='ei_xxxx'

Download your modelfile.eim:

edge-impulse-linux-runner --download modelfile.eim

Run the script:

python3 audio-classify-export.py modelfile.eim yes,no 0.6 0.8

Here are the arguments you can set:

modelfile.eim, path the model.eim
yes,no, labels to upload, separated by comas, no space
0.6, low confidence threshold
0.8, high confidence threshold
<audio_device_ID, optional>

In a keyword spotting model, it can give the following results:

python3 audio-classify-export.py modelfile.eim yes,no 0.6 0.8
['modelfile.eim', 'yes,no', '0.6', '0.8']
{'model_parameters': {'axis_count': 1, 'frequency': 16000, 'has_anomaly': 0, 'image_channel_count': 0, 'image_input_frames': 0, 'image_input_height': 0, 'image_input_width': 0, 'input_features_count': 16000, 'interval_ms': 0.0625, 'label_count': 4, 'labels': ['no', 'noise', 'unknown', 'yes'], 'model_type': 'classification', 'sensor': 1, 'slice_size': 4000, 'use_continuous_mode': True}, 'project': {'deploy_version': 29, 'id': 10487, 'name': 'Keywords Detection', 'owner': 'Demo Team'}}
Loaded runner for "Demo Team / Keywords Detection"
0 --> MacBook Pro Microphone
2 --> Microsoft Teams Audio
3 --> Descript Loopback Recorder
4 --> ZoomAudioDevice
Type the id of the audio device you want to use: 
0
selected Audio device: 0

Result (0 ms.) no: 0.18 noise: 0.16     unknown: 0.20   yes: 0.46
Result (0 ms.) no: 0.13 noise: 0.58     unknown: 0.22   yes: 0.07
Result (0 ms.) no: 0.00 noise: 0.89     unknown: 0.10   yes: 0.01
Result (0 ms.) no: 0.00 noise: 0.01     unknown: 0.04   yes: 0.95
Result (0 ms.) no: 0.00 noise: 0.82     unknown: 0.10   yes: 0.07
Result (0 ms.) no: 0.02 noise: 0.77     unknown: 0.13   yes: 0.08
Result (0 ms.) no: 0.01 noise: 0.14     unknown: 0.26   yes: 0.59
Result (0 ms.) no: 0.07 noise: 0.76     unknown: 0.15   yes: 0.01
Result (0 ms.) no: 0.04 noise: 0.24     unknown: 0.11   yes: 0.61       Uploading sample to Edge Impulse...
Successfully uploaded audio to Edge Impulse.

Result (0 ms.) no: 0.02 noise: 0.93     unknown: 0.04   yes: 0.00
Result (0 ms.) no: 0.01 noise: 0.67     unknown: 0.32   yes: 0.01
Result (0 ms.) no: 0.02 noise: 0.18     unknown: 0.23   yes: 0.57
Result (0 ms.) no: 0.07 noise: 0.70     unknown: 0.22   yes: 0.01
Result (0 ms.) no: 0.03 noise: 0.83     unknown: 0.12   yes: 0.02
Result (0 ms.) no: 0.24 noise: 0.44     unknown: 0.21   yes: 0.11
Result (0 ms.) no: 0.23 noise: 0.25     unknown: 0.42   yes: 0.10
Result (0 ms.) no: 0.04 noise: 0.76     unknown: 0.18   yes: 0.02
Result (0 ms.) no: 0.16 noise: 0.67     unknown: 0.12   yes: 0.05
Result (0 ms.) no: 0.12 noise: 0.81     unknown: 0.06   yes: 0.01
Result (0 ms.) no: 0.54 noise: 0.24     unknown: 0.12   yes: 0.10
Result (0 ms.) no: 0.01 noise: 0.91     unknown: 0.05   yes: 0.03
Result (0 ms.) no: 0.65 Uploading sample to Edge Impulse...
Successfully uploaded audio to Edge Impulse.
noise: 0.08     unknown: 0.17   yes: 0.10
Result (0 ms.) no: 0.00 noise: 0.96     unknown: 0.03   yes: 0.00
Result (0 ms.) no: 0.04 noise: 0.80     unknown: 0.13   yes: 0.03
Result (0 ms.) no: 0.03 noise: 0.27     unknown: 0.16   yes: 0.54
Result (0 ms.) no: 0.05 noise: 0.66     unknown: 0.15   yes: 0.14
Result (0 ms.) no: 0.08 noise: 0.74     unknown: 0.14   yes: 0.04
Result (0 ms.) no: 0.01 noise: 0.87     unknown: 0.11   yes: 0.02
Result (0 ms.) no: 0.01 noise: 0.87     unknown: 0.06   yes: 0.06
...

Label image data using GPT-4o

In this tutorial, we will explore how to label image data using GPT-4o, a powerful language model developed by OpenAI. GPT-4o is capable of generating accurate and meaningful labels for images, making it a valuable tool for image classification tasks. By leveraging the capabilities of GPT-4o, we can automate the process of labeling image data, saving time and effort in data preprocessing.

We packaged in a "pre-built Transformation block" (available for all Enterprise plans), an innovative method to distill LLM knowledge.

This pre-built transformation block can be found under the Data sources tab in the Data acquisition view.

The block takes all your unlabeled image files and asks GPT-4o to label them based on your prompt - and we automatically add the reasoning as metadata to your items!

Your prompt should return a single label, e.g.

Is there a person in this picture? Answer with just 'yes' or 'no'.

How to use it

The GPT-4o model processes images and assigns labels based on the content, filtering out any images that do not meet the quality criteria.

Step 1: Data Collection

Navigate to the Data acquisition page and add images to your project's dataset. In the video tutorial above, we show how to collect a video recorded directly from a phone, upload it to Edge Impulse and split the video into individual frames.

Step 2: Add the labeling block

In the Data sources tab, add the "Label image data using GPT-4o" block:

Step 4: Configure the labeling block

OpenAI API key: Add your OpenAI API key. This value will be stored as a secret, and won't be shown again.
Prompt: Your prompt should return a single label. For example:

Is there a person in this picture? Respond only with "yes", "no" or "unsure" if you're not sure.

Disable samples w/ label: If a certain label is output, disable the data item - these are excluded from training. Multiple labels are accepted, separate them with a coma.
Max. no. of samples to label: Number of samples to label.
Concurrency: Number of samples to label in parallel.
Auto-convert videos: If set, all videos are automatically split into individual images before labeling.

Optional: Editing your labeling block

To edit your configuration, you need to update the json-like steps of your block:

Step 5: Execute

Then, run the block to automatically label the frames.

And here is an example of the returned logs:

Step 6: Train your model

Use the labeled data to train a machine learning model. See the end-to-end tutorial Image classification.

Step 7: Deployment

In the video tutorial, we deployed the trained model to an MCU-based edge device - the Arduino Nicla Vision.

Results

The small model we tested this on performed exceptionally well, identifying toys in various scenes quickly and accurately. By distilling knowledge from the large LLM, we created a specialized, efficient model suitable for edge deployment.

Conclusion

The latest multimodal LLMs are incredibly powerful but too large for many practical applications. At Edge Impulse, we enable the transfer of knowledge from these large models to smaller, specialized models that run efficiently on edge devices.

Our "Label image data using GPT-4o" block is available for enterprise customers, allowing you to experiment with this technology.

For further assistance, visit our forum.

Examples & Resources

Blog post: Label image data using GPT-4o blog post

Using the Edge Impulse Python SDK to upload and download data

You can find the API documentation for the functions found in this tutorial here.

WARNING: This notebook will add and delete data in your Edge Impulse project, so be careful! We recommend creating a throwaway project when testing this notebook.

Note that you might need to refresh the page with your Edge Impulse project to see the samples appear.

# If you have not done so already, install the following dependencies
!python -m pip install edgeimpulse

import edgeimpulse as ei

Note that you do not actually need to use the project in the Edge Impulse Studio. We just need the API Key.

Paste that API key string in the ei.API_KEY value in the following cell:

# Settings
ei.API_KEY = "ei_dae2..." # Change this to your Edge Impulse API key

Upload directory

The following file formats are allowed: .cbor, .json, .csv, .wav, .jpg, .png, .mp4, .avi.

from datetime import datetime

# Download image files to use as an example dataset
!mkdir -p dataset
!wget -P dataset -q \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/capacitor.01.png \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/capacitor.02.png

# Upload the entire directory
response = ei.experimental.data.upload_directory(
    directory="dataset",
    category="training",
    label=None, # Will use the prefix before the '.' on each filename for the label
    metadata={
        "date": datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
        "source": "camera",
    }
)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

# Review the sample IDs and get the associated server-side filename
# Note the lack of extension! Multiple samples on the server can have the same filename.
for id in ids:
    filename = ei.experimental.data.get_filename_by_id(id)
    print(f"Sample ID: {id}, filename: {filename}")

If you head to the Data acquisition page on your project, you should see images in your dataset.

Download files

# Get sample IDs for everything in the "training" category
infos = ei.experimental.data.get_sample_ids(category="training")

# The SampleInfo should match what we uploaded earlier
ids = []
for info in infos:
    print(info)
    ids.append(info.sample_id)

# Download samples
samples = ei.experimental.data.download_samples_by_ids(ids)

# Save the downloaded files
for sample in samples:
    with open(sample.filename, "wb") as file:
        file.write(sample.data.read())

# View sample information
for sample in samples:
    print(
        f"filename: {sample.filename}\r\n"
        f"  sample ID: {sample.sample_id}\r\n"
        f"  category: {sample.category}\r\n"
        f"  label: {sample.label}\r\n"
        f"  bounding boxes: {sample.bounding_boxes}\r\n"
        f"  metadata: {sample.metadata}"
    )

Take a look at the files in this directory. You should see the downloaded images. They should match the images in the dataset/ directory, which were the original images that we uploaded.

Delete files

If you know the ID of the sample you would like to delete, you can call the delete_sample_by_id() function. You can also delete all the samples in your project by calling delete_all_samples().

# Delete the samples from the Edge Impulse project that we uploaded earlier
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Take a look at the data in your project. The samples that we uploaded should be gone.

Upload folder for object detection

For object detection, you can put bounding box information (following the Edge Impulse JSON bounding box format) in a file named info.labels in that same directory.

Important! The annotations file must be named exactly info.labels

# Download images and bounding box annotations to use as an example dataset
!mkdir -p dataset
!rm dataset/capacitor.01.png dataset/capacitor.02.png
!wget -P dataset -q \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/dog-ball-toy.01.png \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/dog-ball-toy.02.png \
  https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/annotations/info.labels

# Upload the entire directory (including the info.labels file)
response = ei.experimental.data.upload_exported_dataset(
    directory="dataset",
)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

If you head to the Data acquisition page on your project, you should see images in your dataset along with the bounding box information.

# Delete the samples from the Edge Impulse project that we uploaded
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload individual CSV files

import csv
import io
import os

# Create example CSV data
sample_data = [
    [
        ["timestamp", "accX", "accY", "accZ"],
        [0, -9.81, 0.03, 0.21],
        [10, -9.83, 0.04, 0.27],
        [20, -9.12, 0.03, 0.23],
        [30, -9.14, 0.01, 0.25],
    ],
    [
        ["timestamp", "accX", "accY", "accZ"],
        [0, -9.56, 5.34, 1.21],
        [10, -9.43, 1.37, 1.27],
        [20, -9.22, -4.03, 1.23],
        [30, -9.50, -0.98, 1.25],
    ],
]

# Write to CSV files
filenames = [
    "001.csv",
    "002.csv"
]
for i, filename in enumerate(filenames):
    file_path = os.path.join("dataset", filename)
    with open(file_path, "w", newline="") as file:
        writer = csv.writer(file)
        writer.writerows(sample_data[i])

# Add metadata to the CSV data
my_samples = [
    {
        "filename": filenames[0],
        "data": open(os.path.join("dataset", filenames[0]), "rb"),
        "category": "training",
        "label": "idle",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
    {
        "filename": filenames[1],
        "data": open(os.path.join("dataset", filenames[1]), "rb"),
        "category": "training",
        "label": "wave",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
]

# Wrap the samples in instances of the Sample class
samples = [ei.experimental.data.Sample(**i) for i in my_samples]

# Upload samples to your project
response = ei.experimental.data.upload_samples(samples)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

If you head to the Data acquisition page on your project, you should see your time series data.

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload JSON data directly

The raw data must be encoded in an IO object. We convert the dictionary objects to a BytesIO object, but you can also read in data from .json files.

import io
import json

# Create two different example data samples
sample_data_1 = {
    "protected": {
        "ver": "v1",
        "alg": "none",
    },
    "signature": 0,
    "payload": {
        "device_name": "ac:87:a3:0a:2d:1b",
        "device_type": "DISCO-L475VG-IOT01A",
        "interval_ms": 10,
        "sensors": [
            { "name": "accX", "units": "m/s2" },
            { "name": "accY", "units": "m/s2" },
            { "name": "accZ", "units": "m/s2" }
        ],
        "values": [
            [ -9.81, 0.03, 0.21 ],
            [ -9.83, 0.04, 0.27 ],
            [ -9.12, 0.03, 0.23 ],
            [ -9.14, 0.01, 0.25 ]
        ]
    }
}
sample_data_2 = {
    "protected": {
        "ver": "v1",
        "alg": "none",
    },
    "signature": 0,
    "payload": {
        "device_name": "ac:87:a3:0a:2d:1b",
        "device_type": "DISCO-L475VG-IOT01A",
        "interval_ms": 10,
        "sensors": [
            { "name": "accX", "units": "m/s2" },
            { "name": "accY", "units": "m/s2" },
            { "name": "accZ", "units": "m/s2" }
        ],
        "values": [
            [ -9.56, 5.34, 1.21 ],
            [ -9.43, 1.37, 1.27 ],
            [ -9.22, -4.03, 1.23 ],
            [ -9.50, -0.98, 1.25 ]
        ]
    }
}

# Provide a filename, category, label, and optional metadata for each sample
my_samples = [
    {
        "filename": "001.json",
        "data": io.BytesIO(json.dumps(sample_data_1).encode('utf-8')),
        "category": "training",
        "label": "idle",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
    {
        "filename": "002.json",
        "data": io.BytesIO(json.dumps(sample_data_2).encode('utf-8')),
        "category": "training",
        "label": "wave",
        "metadata": {
            "source": "accelerometer",
            "collection site": "desk",
        },
    },
]

# Wrap the samples in instances of the Sample class
samples = [ei.data.sample_type.Sample(**i) for i in my_samples]

# Upload samples to your project
response = ei.experimental.data.upload_samples(samples)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

If you head to the Data acquisition page on your project, you should see your time series data.

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload NumPy arrays

Important! NumPy arrays must be in the shape (Number of samples, number of data points, number of sensors)

If you are working with image data in NumPy, we recommend saving those images as .png or .jpg files and using upload_directory().

import numpy as np

# Create example NumPy array with 2 time series samples
sample_data = np.array(
    [
        [ # Sample 1 ("idle")
            [-9.81, 0.03, 0.21],
            [-9.83, 0.04, 0.27],
            [-9.12, 0.03, 0.23],
            [-9.14, 0.01, 0.25],
        ],
        [ # Sample 2 ("wave")
            [-9.56, 5.34, 1.21],
            [-9.43, 1.37, 1.27],
            [-9.22, -4.03, 1.23],
            [-9.50, -0.98, 1.25],
        ],
    ]
)

# Labels for each sample
labels = ["idle", "wave"]

# Names of the sensors and units for the 3 axes
sensors = [
    { "name": "accX", "units": "m/s2" },
    { "name": "accY", "units": "m/s2" },
    { "name": "accZ", "units": "m/s2" },
]

# Optional metadata for all samples being uploaded
metadata = {
    "source": "accelerometer",
    "collection site": "desk",
}

# Upload samples to your project
response = ei.experimental.data.upload_numpy(
    data=sample_data,
    labels=labels,
    sensors=sensors,
    sample_rate_ms=10,
    metadata=metadata,
    category="training",
)

# Check to make sure there were no failures
assert len(response.fails) == 0, "Could not upload some files"

# Save the sample IDs, as we will need these to retrieve file information and delete samples
ids = []
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Upload pandas (and pandas-like) dataframes

pandas is popular Python library for performing data manipulation and analysis. The Edge Impulse library supports a number of ways to upload dataframes. We will go over each format.

Note that several other packages exist that work as drop-in replacements for pandas. You can use these replacements so long as you import that with the name pd. For example, one of:

import pandas as pd
import modin.pandas as pd
import dask.dataframe as pd
import polars as pd

import pandas as pd

The first option is to upload one dataframe for each sample (non-time series)

# Construct one dataframe for each sample (multidimensional, non-time series)
df_1 = pd.DataFrame([[-9.81, 0.03, 0.21]], columns=["accX", "accY", "accZ"])
df_2 = pd.DataFrame([[-9.56, 5.34, 1.21]], columns=["accX", "accY", "accZ"])

# Optional metadata for all samples being uploaded
metadata = {
    "source": "accelerometer",
    "collection site": "desk",
}

# Upload the first sample
ids = []
response = ei.experimental.data.upload_pandas_sample(
    df_1,
    label="One",
    filename="001",
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Upload the second sample
response = ei.experimental.data.upload_pandas_sample(
    df_2,
    label="Two",
    filename="002",
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

You can also upload one dataframe for each sample (time series). As with previous examples, we'll assume that the sample rate is 10 ms.

# Create samples (multidimensional, time series)
sample_data_1 = [ # Sample 1 ("idle")
    [-9.81, 0.03, 0.21],
    [-9.83, 0.04, 0.27],
    [-9.12, 0.03, 0.23],
    [-9.14, 0.01, 0.25],
]
sample_data_2 = [ # Sample 1 ("wave")
    [-9.56, 5.34, 1.21],
    [-9.43, 1.37, 1.27],
    [-9.22, -4.03, 1.23],
    [-9.50, -0.98, 1.25],
]

# Construct one dataframe for each sample
df_1 = pd.DataFrame(sample_data_1, columns=["accX", "accY", "accZ"])
df_2 = pd.DataFrame(sample_data_2, columns=["accX", "accY", "accZ"])

# Optional metadata for all samples being uploaded
metadata = {
    "source": "accelerometer",
    "collection site": "desk",
}

# Upload the first sample
ids = []
response = ei.experimental.data.upload_pandas_sample(
    df_1,
    label="Idle",
    filename="001",
    sample_rate_ms=10,
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Upload the second sample
response = ei.experimental.data.upload_pandas_sample(
    df_2,
    label="Wave",
    filename="002",
    sample_rate_ms=10,
    metadata=metadata,
    category="training",
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

You can upload non-time series data where each sample is a row in the dataframe. Note that you need to provide labels in the rows.

# Construct non-time series data, where each row is a different sample
data = [
    ["desk", "training", "One", -9.81, 0.03, 0.21],
    ["field", "training", "Two", -9.56, 5.34, 1.21],
]
columns = ["loc", "category", "label", "accX", "accY", "accZ"]

# Wrap the data in a DataFrame
df = pd.DataFrame(data, columns=columns)

# Upload non-time series DataFrame (with multiple samples) to the project
ids = []
response = ei.experimental.data.upload_pandas_dataframe(
    df,
    feature_cols=["accX", "accY", "accZ"],
    label_col="label",
    category_col="category",
    metadata_cols=["loc"],
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

A "wide" dataframe is one where each column represents a value in the time series data, and the rows become individual samples. Note that you need to provide labels in the rows.

# Construct time series data, where each row is a different sample
data = [
    ["desk", "training", "idle", 0.8, 0.7, 0.8, 0.9, 0.8, 0.8, 0.7, 0.8],
    ["field", "training", "motion", 0.3, 0.9, 0.4, 0.6, 0.8, 0.9, 0.5, 0.4],
]
columns = ["loc", "category", "label", "0", "1", "2", "3", "4", "5", "6", "7"]

# Wrap the data in a DataFrame
df = pd.DataFrame(data, columns=columns)

# Upload time series DataFrame (with multiple samples) to the project
ids = []
response = ei.experimental.data.upload_pandas_dataframe_wide(
    df,
    label_col="label",
    category_col="category",
    metadata_cols=["loc"],
    data_col_start=3,
    sample_rate_ms=100,
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

A DataFrame can also be divided into "groups" so you can upload multidimensional time series data.

# Create samples
sample_data = [
    ["desk", "sample 1", "training", "idle", 0, -9.81, 0.03, 0.21],
    ["desk", "sample 1", "training", "idle", 0.01, -9.83, 0.04, 0.27],
    ["desk", "sample 1", "training", "idle", 0.02, -9.12, 0.03, 0.23],
    ["desk", "sample 1", "training", "idle", 0.03, -9.14, 0.01, 0.25],
    ["field", "sample 2", "training", "wave", 0, -9.56, 5.34, 1.21],
    ["field", "sample 2", "training", "wave", 0.01, -9.43, 1.37, 1.27],
    ["field", "sample 2", "training", "wave", 0.02, -9.22, -4.03, 1.23],
    ["field", "sample 2", "training", "wave", 0.03, -9.50, -0.98, 1.25],
]
columns = ["loc", "sample_name", "category", "label", "timestamp", "accX", "accY", "accZ"]

# Wrap the data in a DataFrame
df = pd.DataFrame(sample_data, columns=columns)

# Upload time series DataFrame (with multiple samples and multiple dimensions) to the project
ids = []
response = ei.experimental.data.upload_pandas_dataframe_with_group(
    df,
    group_by="sample_name",
    timestamp_col="timestamp",
    feature_cols=["accX", "accY", "accZ"],
    label_col="label",
    category_col="category",
    metadata_cols=["loc"]
)
assert len(response.fails) == 0, "Could not upload some files"
for sample in response.successes:
    ids.append(sample.sample_id)

# Delete the samples from the Edge Impulse project
for id in ids:
    ei.experimental.data.delete_sample_by_id(id)

Data

Data ingestion

Collecting image data from the Studio

Collecting image data with your mobile phone

1. Collecting images

2. Alternative: upload data directly

Collecting image data with the OpenMV Cam H7 Plus

1. Setting up your environment

2. Collecting images

3. Sending the dataset to Edge Impulse

Using the Edge Impulse Python SDK to upload and download data

Upload directory

Download files

Delete files

Upload folder for object detection

Upload individual CSV files

Upload JSON data directly

Upload NumPy arrays

Upload pandas (and pandas-like) dataframes

Trigger connected board data sampling

1. Obtain an API key from your project

2. Connect your development kit to your project

3. Obtain your project's ID

4. Setup API Connection

5. Get the ID of the connected device

6. Trigger data sampling

Ingest multi-labeled data using the API

Getting Started

Prerequisites

Setup Your Environment

Prepare Your Data

Upload Data

Update Existing Samples

Synthetic data

Using integrated models directly available inside Edge Impulse Studio

Other tutorials

Generate audio datasets using Eleven Labs

Example: Glass Breaking Sound

Getting Started

Step 1: Get your ElevenLabs API Key

Step 2: Parameters

Step 3: Generate samples

Benefits of Using Generative AI for Sound Generation

Conclusion

Generate image datasets using Dall-E

Local Software Requirements

Set up OpenAI API

Generate your first image

Generate some variations of this image

Generate a dataset:

Plot all the output images:

What next?

Generate keyword spotting datasets using Google TTS

Local Software Requirements

Set up Google TTS API

Generate the desired samples

What next?

Generate physics simulation datasets using PyBullet

Local Software Requirements

Create object to simulate

Visualizing the problem

Setting up the simulation environment

Drop simulation

What next?

Labeling

Label audio data using your existing models

Label image data using GPT-4o

How to use it

Step 1: Data Collection

Step 2: Add the labeling block

Step 4: Configure the labeling block

Optional: Editing your labeling block

Step 5: Execute

Step 6: Train your model

Step 7: Deployment

Results

Conclusion

Examples & Resources

Collecting image data with your mobile phone

1. Collecting images