Introduction

Welcome to the tutorial series on OTA Model Updates with Edge Impulse Docker Deploy on Jetson Nano! In this series, we will explore how to update machine learning models over-the-air (OTA) using Edge Impulse and Docker on the Jetson Nano platform.

Prerequisites

Before getting started, make sure you have the following prerequisites:

• Jetson Nano Developer Kit

• Docker installed on Jetson Nano

• Edge Impulse account

• Be familiar with Edge Impulse and Docker deploy

Overview

In this tutorial, we will explore how to enable GPU usage and use a camera with the Jetson Nano. We will then deploy a machine learning model using Edge Impulse and Docker on the Jetson Nano. Finally, we will update the model over-the-air (OTA) using Edge Impulse.

Step 1: Enable GPU Usage on Jetson Nano

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

After installing the toolkit, restart the Docker service:

sudo systemctl restart docker

Now you can use the GPU for machine learning tasks in Docker containers.

Step 2: Use a Camera with Jetson Nano

To use a camera with Jetson Nano, you need to install the libgstreamer and libv4l libraries. Run the following commands to install the libraries:

sudo apt-get update
sudo apt-get install -y libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstrtspserver-1.0-0 libv4l-0 v4l-utils

After installing the libraries, you can use a camera with Jetson Nano.

Step 3: Deploy Machine Learning Model with Edge Impulse and Docker

To deploy a machine learning model with Edge Impulse and Docker, follow these steps:

Export your model from Edge Impulse as a Docker container. Copy the generated Docker command from the deployment section. Modify the Docker command to use the GPU and camera on Jetson Nano. Run the Docker command on Jetson Nano to deploy the model.

Step 4: Update Model Over-the-Air (OTA) with Edge Impulse

To update the model over-the-air (OTA) with Edge Impulse, follow these steps:

Train a new model in Edge Impulse. Export the new model as a Docker container. Copy the generated Docker command from the deployment section and build a new Docker image.

docker build -t my_video_inference_container .
Modify the Docker command to use the GPU and camera on Jetson Nano.

```Dockerfile

FROM nvcr.io/nvidia/l4t-base:r32.5.0

# Install necessary dependencies for video streaming
RUN apt-get update && apt-get install -y \
    ffmpeg \
    v4l-utils \
    && rm -rf /var/lib/apt/lists/*

# Set environment variables
ENV DISPLAY=:0
ENV QT_X11_NO_MITSHM=1

# Expose port for streaming
EXPOSE 80

# Mount USB camera device
RUN ln -s /dev/bus/usb/001/002 /dev/video0

# Command to run your model as a Docker container
CMD ["docker", "run", "--rm", "-it", \
    "-p", "80:80", \
    "public.ecr.aws/z9b3d4t5/inference-container:73d6ea64bf931f338de5183438915dc390120d5d", \
    "--api-key", "ei_07e1e4fad55f06b30839c062076a2ad4bbc174386330493011e75566405a5603", \
    "--run-http-server", "1337"]

1. Run the Docker command on Jetson Nano to deploy the new model.

docker run --rm -it --privileged --runtime nvidia -v /dev/bus/usb/001/002:/dev/video0 -p 80:80 public.ecr.aws/z9b3d4t5/inference-container:73d6ea64bf931f338de5183438915dc390120d5d --api-key ei_07e1e4fad55f06b30839c062076a2ad4bbc174386330493011e75566405a5603 --run-http-server 1337

1. Test the new model on Jetson Nano.

2. Monitor the model performance and update as needed.

Summary

In this tutorial series, we explored how to update machine learning models over-the-air (OTA) using Edge Impulse and Docker on the Jetson Nano platform. We enabled GPU usage, used a camera with Jetson Nano, deployed a machine learning model, and updated the model over-the-air.

Now you can easily update your machine learning models on Jetson Nano devices using Edge Impulse and Docker.

Introduction

This page is part of the Lifecycle Management with Edge Impulse tutorial series. If you haven't read the introduction yet, we recommend you to do so here.

Overview

OTA DFU with Notehub Blues Wireless have created a unique way to update firmware on your Notecard device is to perform an over-the-air DFU with Notehub.io. For instructions on how to update your Notecard firmware with Notehub.io, please visit the Manage Notecard Firmware guide.

Key Features of Blues Wireless OTA Updates:

• NOFU: Notehub Outbound Firmware Update allows you to update the firmware on the Swan from Notehub.

Closing the Loop:

• Blues Wireless has created their own Edge Impulse ingestion API integration which allows you to collect data from the notecard.

Prerequisites:

• Edge Impulse Account: If you haven't got one, sign up here.

• Trained Impulse: If you're new, follow one of our end-to-end tutorials

• Blues Wireless Account: If you haven't got one, sign up here.

• Blues Wireless Notecard: If you haven't got one, order here.

Webinar Recap: Optimized MLOps with Edge Impulse, Blues, and Zephyr:

Watch the full webinar here for insights into MLOps and its integration with Edge Impulse, Blues, and Zephyr.

Webinar Overview:

Implementing MLOps Workflow:

• Data collection using the Notecard.

• Sending data to Edge Impulse for model training.

• Deploying the trained model using Zephyr.

• Monitoring and continuous improvement.

Establishing a Robust MLOps Workflow for TinyML

Author: TJ VanToll link: https://dev.blues.io/blog/robust-ml-ops-workflow/

Introduction

In TinyML projects, the effectiveness of a machine learning model greatly depends on the quality of its training data. Gathering comprehensive training data is particularly challenging in TinyML due to the limited power and connectivity of tiny devices. At Blues, in collaboration with Edge Impulse and Zephyr, we have developed a workflow that facilitates MLOps (Machine Learning Operations) for tiny devices.

This tutorial outlines a method to implement a robust MLOps process, focusing on how to collect ML data from remote devices and update ML models on these devices in the field. The result is a workflow that seamlessly integrates data collection and model updating.

see the Establishing a Robust MLOps Workflow for TinyML Blues Guide for the in depth guide.

Collecting ML Data

Let's assume we're managing devices monitoring industrial equipment with accelerometers to detect abnormalities. For a successful MLOps workflow, these devices need to send accelerometer data to the cloud for training an updated model. Here, we use the Blues Notecard for this purpose.

The Blues Notecard

The Notecard is a system on module designed for easy connectivity in embedded projects. Its features include:

• 500MB of cellular connectivity

• Global cellular support over LTE-M, NB-IoT, or Cat-1

• Secure device-to-cloud communication

• Low-power hardware and firmware

• Easy embedding options

To set up the hardware, you'll need a Blues Starter Kit, which includes a development board (Notecarrier) and a Swan microcontroller. Additionally, an LIS3DH accelerometer connected to the Notecarrier via a Qwiic cable is required.

The Firmware

The project’s firmware gathers accelerometer readings and sends them to the cloud via the Notecard. We use Zephyr for firmware implementation, as it offers both low-level access and development conveniences. Zephyr firmware is compatible with STM32-based boards like the Swan, and the Notecard has a Zephyr SDK.

The firmware performs the following functions:

• Establishes a connection between the Notecard and Notehub

• Gathers data from the accelerometer

• Sends the data to Notehub

For example, the firmware takes accelerometer readings at intervals set by the Edge Impulse SDK and sends this data as a binary stream to Notehub.

Sending the Data

Notecard's new firmware (v5.3.1) introduces 'card.binary' APIs, enabling fast data transfer for large data packets. The data is then sent to Notehub using the 'web.post' method.

Getting the Data to Edge Impulse

The data from Notehub is forwarded to Edge Impulse through an HTTP server. This server, which can be created using any language or framework, listens for POST requests and parses the floating-point accelerometer values. The data is then sent to Edge Impulse's ingestion API, adding new entries to the model’s training set.

const emptySignature = Array(64).fill("0").join("");
const body = {
    protected: {
        ver: "v1",
        alg: "none",
        iat: Math.floor(Date.now() / 1000),
    },
    signature: emptySignature,
    payload: {
        device_name: "device-1",
        device_type: "LIS2HH12",
        interval_ms: 1,
        sensors: [
            { name: "accX", units: "m/s2" },
            { name: "accY", units: "m/s2" },
            { name: "accZ", units: "m/s2" },
        ],
        values: data,
    },
};

try {
    await fetch("https://ingestion.edgeimpulse.com/api/training/data", {
        method: "POST",
        headers: {
            "x-api-key": process.env.EDGE_IMPULSE_API_KEY,
            "x-file-name": "test",
            "x-label": "idle",
            "Content-Type": "application/json",
        },
        body: JSON.stringify(body),
    });
} catch (e) {
    console.log("Error publishing data", e);
}

see the section of the Blues Guide for more information.

Updating ML Models

The final aspect of the MLOps process is updating the models on the devices in the field. For this, we use Notecard Outboard Firmware Update. This feature allows OTA firmware updates without writing any code, offering flexibility and safety against bricking devices.

see the Updating ML Models section of the Blues Guide for more information.

Notecard Outboard Firmware Update

• Requires specific wiring (provided in the Blues Starter Kit)

• Activated via a 'card.dfu' request in the firmware

• Involves building a new firmware image file with the updated model

• The new firmware is uploaded to Notehub and applied to devices

see the What is notecard outboud update? section of the Blues Guide for more information.

Conclusion

The combination of data collection and model updating forms a robust MLOps process. This approach ensures continuous improvement of ML models based on real data and updates models remotely. The provided GitHub samples offer a reference implementation, adaptable to various project needs.

For more information, refer to the [Notecard Outboard Firmware Update guide](https://dev.blues.io/guides-and-tutorials/notecard-guides

The combination of data collection and model updating forms a robust MLOps process. This approach ensures continuous improvement of ML models based on real data and updates models remotely. The provided GitHub samples offer a reference implementation, adaptable to various project needs.

Learn More

• Build Your Edge ML Application: Follow the step-by-step Blues Swan tutorial.

Summary

This tutorial acts as a reference to the webinar and tutorial from Blues Wireless: Optimized MLOps with Edge Impulse, Blues, and Zephy view the full webinar.

Allxon provides essential remote device management solutions to simplify and optimize edge AI device operations. As an AI/IoT ecosystem enabler, connecting hardware (IHV), software (ISV), and service providers (SI/MSP), Allxon serves as the catalyst for fast, seamless connectivity across all systems.

Allxon Over-the-Air (OTA) deployment works perfectly with Edge Impulse OTA model update on Nvidia Jetson devices. This tutorial guides you through the steps to deploy a new impulse on multiple devices.

Introduction

This guide demonstrates how to deploy and manage Edge Impulse models on NVIDIA Jetson devices using Allxon's Over-the-Air (OTA) deployment capabilities. Allxon provides essential remote device management solutions to streamline and optimize edge AI device operations.

Prerequisites

Before you begin, ensure you have the following:

1. Updated impulse as a Docker container from Edge Impulse.

2. Get Allxon officially supported devices(https://www.allxon.com/).

3. Create an Allxon account.

Getting Started with Allxon

Allxon's services are compatible with a variety of hardware models. Follow these steps to complete the required preparations.

Add a Device to Allxon Portal

1. Install Allxon Agent: Use the command prompt to install the Allxon Agent on your device.

2. Pair Your Device: Follow the instructions to add your device to Allxon Portal.

Once added, your devices will appear in the Allxon Portal for management and monitoring.

Allxon OTA Deployment

To perform an OTA deployment, ensure you have your updated Impulse deployed as a Docker container from Edge Impulse.

Steps to Deploy

1. Generate OTA Artifact: Use the Allxon CLI to generate the OTA artifact.

2. Deploy OTA Artifact: Follow the Deploy OTA Artifact guide to complete the deployment.

Example Scripts

Below are example scripts to help you set up the OTA deployment.

ota_deploy.sh

#!/bin/bash
set -e
mkdir -p /opt/allxon/tmp/core/appies_ota/model_logs/
./install.sh > /opt/allxon/tmp/core/appies_ota/model_logs/log.txt 2>&1
echo "Model deployment has started. Please check /opt/allxon/tmp/core/appies_ota/model_logs/log.txt for progress."

install.sh

#!/bin/bash
docker run --rm --privileged --runtime nvidia \
 -v /dev/bus/usb/001/002:/dev/video0 \
 -p 80:80 \
 public.ecr.aws/z9b3d4t5/inferencecontainer:73d6ea64bf931f338de5183438915dc390120d5d \
 --api-key <replace with your project api key e.g. ei_07XXX > \
 --run-http-server 1337 &

Two minor modifications have been made to the standard Docker command from Edge Impulse docker deploy:

The -it option has been removed from the Docker command to avoid an error related to the lack of standard input during deployment. An & has been added to the end of the Docker command to send the process to the background.

Conclusion

By following these steps, you can efficiently deploy and manage Edge Impulse models on NVIDIA Jetson devices using Docker through Allxon. This setup leverages Allxon's remote management capabilities to streamline the process of updating and maintaining your edge AI devices.

We hope this section has helped you understand the process of Lifecycle Management and how to implement it in your own project. If you have any questions, please reach out to us on our forum.

Introduction

This page is part of the tutorial series. If you haven't read the introduction yet, we recommend you to do so .

In this tutorial, we'll guide you through deploying updated impulses over-the-air (OTA) to the ESP32 using Edge Impulse. We'll build on Espressif's end-to-end OTA firmware update workflow, incorporating Edge Impulse's API to check for updates and download the latest build.

We will modify the Edge Impulse C++ example for ESP32, and combine it with the OTA example from the ESP IDF repository. This will allow us to check for updates to your project on the server side of the IDF example and download the latest build. We will then modify the C++ example to incorporate the OTA functionality and update the device with the latest build. Finally we will add calls back to our ingestion service to monitor the performance of the updated impulse, and gather data. Closing the loop on our active learning cycle.

Let's get started!

Key Features of Espressif OTA Updates:

Prerequisites

• Edge Impulse Account: If you haven't got one, .

• Trained Impulse: If you're new, follow one of our

• Knowledge of the ESP IDF development framework and C++ for ESP32

• Installation of required software as detailed in the tutorial

Preparation

Begin by setting up your device for OTA updates following Espressif's OTA firmware update workflow. Use the built binary from the C++ example and modify it to incorporate OTA functionality.

Let's get started!

On your Espressif ESP-EYE (ESP32) development board

We created an example repository which contains a small application for Espressif ESP32, which takes the raw features as an argument, and prints out the final classification. Download the application as a .zip, or import this repository using Git:

We will need to add a few libraries to the project to facilitate the OTA update process. These will be taken from the ESP32 OTA example and are already included in the example project.

ESP IDF OTA

We are going to use the ESP IDF OTA example as a starting point. Running a python server to check for updates and download the latest build from edge impulse. This detemines if there is a new build available and downloads it to the device.You could add more advanced checking for model performance and versioning to this process. This example will only download the latest build if there is a new build available based on date to save lengthy code examples.

This example is available in the ESP IDF repository. You can find the example in:

Starting with the device side code, we will need to add a few libraries to the project to facilitate the OTA update process. These are taken from the ESP32 OTA example and are already included in the example project.

Step 1: Copy the IDF OTA example and clone a fresh version of the edge impulse example C++ inferencing repository

Now we will set up the Server Side components of the OTA update process

1. Python Server Side OTA

Modify the ESP OTA example python server to check for updates to your project, we will use the date to determine when a new update should be performed to keep things simple. You could add more advanced checking for model performance and versioning to this process. This example will only download the latest build if there is a new build available based on date to save lengthy code examples.

2.

Building the firmware to deploy to our device in the field. We will use the IDF build system to build the firmware. This would be automated but again to save time and complexity we will do this manually.

3. Updating the Device

Compare the model's timestamp or hash with the stored version. If it's different or newer, call the download_model() function.

4. Monitoring and Repeating the Process

Monitor the device to ensure the new impulse performs as expected and repeat the update process as needed.

Here we would make calls back to our ingestion service to monitor the performance of the updated impulse, and gather data. Closing the loop on our active learning cycle.

Device Side of the OTA Update Process

Now lets modify the C++ example to incorporate the device side of the OTA functionality and update our edge impulse project with the device side data. We will add the OTA functionality to the C++ example and update the device with the latest build.

Step 1: Include Necessary Headers

Include the necessary headers and components in your main ESP32 C++ based edge impulse project. There are a number of components that are required for the OTA update process:

Components

1. Non-Volatile Storage (NVS)

NVS is utilized to persistently store data like configuration settings, WiFi credentials, or firmware update times, ensuring retention across reboots.

2. HTTP Client

This library facilitates HTTP requests to the server for checking and retrieving new firmware updates.

3. OTA Operations

These headers aid in executing OTA operations, including writing new firmware to the flash and switching boot partitions.

4. FreeRTOS Task

FreeRTOS ensures OTA updates are conducted in a separate task, preventing blockage of other tasks and maintaining system operations during the update.

Lets start by adding the necessary headers to the main.cpp file in the edge impulse project.

Step 3: OTA Task Implement the OTA task that will handle the OTA update process. Connecting to the python server we created in the beginning of this tutorial. Which is checking for updates to your project

In the simple_ota_example_task function, you'll need to replace http://192.168.1.10/your-ota-firmware.bin with the actual URL where your firmware binary is hosted.

Step 4: App Main Here’s how the app_main function should look like:

Step 5: Set the target and configuration Make sure you have the partition table set up correctly to support OTA. In the IDF Menu Config, An OTA data partition (type data, subtype ota) must be included in the Partition Table of any project which uses the OTA functions.

Step 6: Compile and Flash You should compile the firmware and flash it to your ESP32 using the IDF build system commands.

Note: Before you run this code, make sure to replace http://192.168.1.10.com/your-ota-firmware.bin with the actual URL where your new firmware binary is hosted. Make sure that your ESP32 is connected to the internet to access this URL. Also, always test OTA functionality thoroughly before deploying it in a production environment.

Conclusion

This tutorial provides a basic guide for implementing OTA updates on Espressif ESP-EYE (ESP32) with Edge Impulse. It can be extended to include more advanced checking, utilizing more of espressif OTA functionality, and extending the python server to check for model performance and versioning. Happy coding!

Introduction

In this section we will explore how firmware updates and other scenarios are currently addressed, with traditional OTA. It should help you to get started planning your own updated impulse across a range of platforms.

Starting with platform-specific examples like Arduino Cloud, Nordic nRF Connect SDK / Zephyr and Golioth, Particle Workbench and Blues Wireless. Finally we will explore building an end-to-end example on the Espressif IDF.

By covering a cross section of platforms we hope to provide a good overview of the process and how it can be applied to your own project. With more generic examples like Arduino, Zephyr and C++ which can be applicable to all other vendors.

These tutorials will help you to get started with the following platforms:

Prerequisites

Overview

Edge Impulse recognises the need for OTA model updates, as the process is commonly referred to although we are going to be updating the impulse which includes more than just a model, a complete review of your infrastructure is required. Here is an example of the process:

Detect a change

The initiation of an update to your device can be as straightforward as a call to our API to verify the availability of a new deployment. This verification can be executed either by a server or a device with adequate capabilities. Changes can be dependent on a range of factors, including but not limited to the last modified date of the project, the performance of the model, or the release version of the project e.g. last modified date of the project endpoint:

Download the latest impulse

After we inquire about the last modification, and if an update is available, proceed to download the latest build through:

We could add further checking for impulse model performance, project release version tracking or other metrics to ensure the update is valid. However in this series we will try to keep it simple and focus on the core process. Here is an example of a more complete process:

• Identify components that influence change: Determine the components of your project that require updates. This could be based on performance metrics, data drift, or new data incorporation.

• Retrain: Focus on retraining based on the identified components of your project.

• Test and Validate: Before deploying the updated components, ensure thorough testing and validation to confirm their performance before sending the update.

• Deploy Updated Components: Utilize available OTA mechanisms to deploy the updated components to your devices. Ensure seamless integration with the existing deployment that remains unchanged.

• Monitor on device Performance: Post-deployment, continuously monitor the performance of the updated model to ensure it meets the desired objectives. See Lifecycle Management for more details.

The aim will be to make sure your device is always equipped with the most recent and efficient impulse, enhancing performance and accuracy.

Conclusion

Introduction

This tutorial is part of the series. If you haven't read the introduction yet, we recommend doing so .

We'll guide you through deploying updated machine learning models over-the-air (OTA) to the Nordic Thingy:53 using Edge Impulse. This process leverages the Nordic Thingy:53 app, allowing users to deploy firmware updates and facilitating on-device testing for Lifecycle Management.

Key Features of Nordic Thingy:53 OTA Updates:

• User-initiated firmware deployment via the Nordic Thingy:53 app.

• Remote data collection and on-device testing for machine learning models.

• Seamless integration with Edge Impulse for Lifecycle Management.

Prerequisites

• Edge Impulse Account: Sign up if you don't have one .

• Trained Impulse: If you're new, follow one of our

• Nordic Thingy:53: Have the device ready and charged.

• Nordic Thingy:53 App: Installed on your smartphone or tablet.

Preparation

Begin by connecting your Nordic Thingy:53 to the Edge Impulse platform and setting it up for data collection and model deployment.

Step-by-Step Guide

1. Setting Up Nordic Thingy:53 with Edge Impulse

• Connect your Nordic Thingy:53 to the Edge Impulse using the Nordic Thingy:53 app. This will be your interface for managing the device and deploying updates.

2. Collecting Data and Training the Model

• Use the Nordic Thingy:53 to collect relevant data for your machine learning application.

• Upload this data to Edge Impulse and train your model.

3. Deploying the Model via the Nordic Thingy:53 App

• Once your model is trained and ready, use the Nordic Thingy:53 app to deploy it to the device.

• The app allows you to initiate the OTA update, which downloads and installs the latest firmware containing the new model.

4. Remote On-Device Testing

• Conduct remote testing through the app to evaluate the model's performance in real-world scenarios.

• This step is crucial for validating the effectiveness of your machine learning model.

5. Continuous Improvement Cycle

• Continuously collect new data with the Nordic Thingy:53.

• Re-train your model on Edge Impulse with this new data.

• Deploy these updates to the Thingy:53 via the app, maintaining the cycle of Lifecycle Management.

Conclusion

This tutorial provides a straightforward approach to implementing OTA updates and Lifecycle Management on the Nordic Thingy:53 using Edge Impulse. The user-friendly Nordic Thingy:53 app facilitates easy deployment of firmware updates, making it ideal for rapid prototyping and iterative machine learning model development.

Additional Resources

This guide helps users leverage the capabilities of the Nordic Thingy:53 for advanced IoT applications, ensuring devices are always updated with the latest intelligence and improvements.

Introduction

This page is part of the Lifecycle management with Edge Impulse tutorial series. If you haven't read the introduction yet, we recommend you to do so . Balena can serve as the platform for deploying OTA updates to edge devices, including new models trained with Edge Impulse.

Prerequisites

• An active Edge Impulse account with a project and trained impulse. See the for more information on how to create a project and train an impulse.

• Follow the .

• - A sample balena project to deploy your impulse as on balena. It exposes an API which the "cam" service uses to get repeated inferences from the images captured by the webcam. See the

Overview

Balena offers a comprehensive platform for building, deploying and managing large fleets of IoT devices. It simplifies fleet management, enhances security, and streamlines the deployment of software and hostOS updates. This tutorial will guide you through using balena to deploy Edge Impulse model updates across your device fleet efficiently. This can be particularly useful for managing multiple devices in the field, ensuring they are always running the latest model. Devices like the NVIDIA Jetson Nano, NVIDIA Jetson Orin, Raspberry Pi, and other single-board computers are supported by balena and can be used to deploy Edge Impulse models.

For this example, we will deploy an Edge Impulse model as a Docker container on a Raspberry Pi using balenaOS. The model will run an HTTP inference server, allowing you to send data to the device for processing and receive predictions in real-time.

Prerequisites

• An active Edge Impulse account with a trained model.

• Follow the Edge Impulse Docker documentation.

Introduction to balena

Balena is a platform that provides tools for building, deploying, and managing large fleets of IoT devices. It simplifies the process of managing fleets of devices, offering a robust framework for deploying software and hostOS updates, monitoring device health, and ensuring security. Balena could serve as the fleet management and device management platform for deploying OTA updates, including new models trained with Edge Impulse.

Step 1: Exporting Your Model as a Docker Container

Go to your Edge Impulse project, navigate to the Deployment section and select Docker container as the deployment option. Follow the instructions to generate the Docker container command. It will look something like this:

Note the Container address and the api-key, as we will need to use them in a subsequent step below.

Step 2: Preparing Your Balena Fleet

Log in to your balenaCloud dashboard and create a new fleet.

BalenaOS - Add a new Device Select the appropriate device type that matches your hardware. Follow the instructions to add a device to your application and download the balenaOS image for it. Flash the downloaded OS image to your device's SD card using balenaEtcher or a similar tool. Power on your device with the SD card or USB key inserted; it should connect to your balena application automatically.

Step 3: Deploying Your Docker Container to Your balena Fleet

Clone the balena boilerplate Edge Impulse project from Github or start with a Dockerfile.template in a new directory on your local machine. Modify the Dockerfile.template to use your container address and api-key as mentioned earlier. Since balena uses Docker containers, you will simply use the container generated by the deployment screen of your Edge Impulse model.

Step 4: Build Your Application

Use the balena CLI to build, and scan for your local device and push your application to balenaCloud:

Wait for the application to build and deploy. You can monitor the progress in the balenaCloud dashboard.

Step 5: Accessing Your Inference Server

Once deployed, your device will start the Docker container and run the HTTP inference server.

You can access it using the device's IP address on your local network or through the Public Device URL feature provided by balenaCloud if enabled for your device. Remember that we decided to point this server to the port 80, but feel free to use another port.

Step 6: Monitoring and Managing Your Fleet

With your Edge Impulse inference server running on balena, now you can monitor and manage the other services running on your device fleet using balenaCloud's dashboard and tools. This includes monitoring device health, deploying updates, and rolling back changes if needed.

Furthermore, to deploy the latest ML model deployed from Edge Impulse after retraining, you will need to restart the edge-impulse service running in the connected devices. You can do it from the balenaCloud Fleet Devices page selecting them.

Conclusion

By following these steps, you should have a functional Edge Impulse inference server running on your balena devices, ready to process data and make predictions. This setup can be integrated into a robust OTA model update process, enabling Lifecycle management and improvement of your Edge AI enabled devices.

Additional Resources

Introduction

This page is part of the tutorial series. If you haven't read the introduction yet, we recommend you to do so .

In this tutorial, we'll guide you through deploying updated impulses over-the-air (OTA) using the Particle Workbench. We'll build on Particle's firmware update workflow, incorporating Edge Impulse's API to check for updates and download the latest build.

We will modify the Edge Impulse Photon 2 example to incorporate the OTA functionality and update the device with the latest build. Based on the particle workbench OTA example.

Prerequisites

• Edge Impulse Account: If you haven't got one, .

• Trained Impulse: If you're new, follow one of our

• Knowledge of the Particle Workbench

• Installation of required software as detailed in the

Preparation

Begin by setting up your device for OTA updates following the particle documentation. https://docs.particle.io/getting-started/cloud/ota-updates/

Let's get started!

Key Features of Particle Workbench OTA Updates:

• Development Devices: Specific devices can be marked for internal testing to validate firmware updates before a broader rollout.

• Firmware Upload: Developers upload the compiled binary to the console, ensuring the firmware version is included.

• Firmware Locking: Specific devices can be locked to run certain firmware versions for testing purposes.

• Release Targets: Firmware can be released to all devices or specific groups to control the rollout process.

• Intelligent Firmware Releases: Ensures timely delivery of updates without disrupting device operation, taking into account the device’s status and activity.

Particle OTA

Particle OTA is a fully-integrated over-the-air software update system that is built into the Particle IoT PaaS and Device OS. It allows customers to safely and reliably push software updates to single devices or entire fleets of devices directly from Particle’s device management console and developer tools, with no implementation work necessary.

Particle OTA allows you to update your entire IoT device (both the Particle device and any other components) by delivering three kinds of updates:

• Application OTA allows users to update the firmware application they are running on the Particle device in order to introduce new features, fix bugs, and generally improve the software application over time.

• Device OS OTA allows users to update Device OS to the latest version alongside an application update so that Device OS can be kept up to date with improvements and bug fixes while properly testing against the user-generated software application.

• Asset OTA allows users to include bundled assets in an OTA software update that can be delivered to other processors and components in the electronics system so that the Particle device can be responsible not just for updating itself but also the system that surrounds the device.

On your Edge Impulse project

Export your impulse to the Particle Workbench library:

Add the library through the Particle Workbench via:

1. Extract the .zip library.

2. Particle: Import Project and select project.properties. Examples can then be found in: yourprojectname/examples/

We created an example repository which contains a small application for your particle device.

Creating the Intial Impulse Export to Particle Library and Deployment firmware

Projects with external libraries

Compiling your project

When you're ready to compile your project, make sure you have the correct Particle device target selected and run particle compile <platform> in the CLI or click the Compile button in the Desktop IDE. The following files in your project folder will be sent to the compile service:

• Everything in the /src folder, including your .ino application file

• The project.properties file for your project

• Any libraries stored under lib/<libraryname>/src

You should now have a compiled binary file in your project folder named firmware.bin.

Next lets look at the OTA functionality and how we can incorporate it into our project.

Particle Workbench OTA Update Workflow and example

You will need to have a device deployed with an existing firmware containing an impulse before you can roll out a new version. Then you can use the OTA workflow to update the firmware on your device to a new version. While blocking all other devices from updating to the new version.

Step 1: Testing on Development Devices

Step 2: Locking Firmware for Testing

Lock Devices: Lock specific devices to the new firmware version and monitor their behavior. Unlock Devices: Unlock the devices once testing is complete and you are satisfied with the firmware's performance.

Step 3: Rolling Out the Firmware

Release Targets: Choose to release the firmware to specific groups or the entire fleet of devices. Intelligent Firmware Release: Opt for this to have the Particle Device Cloud intelligently manage the update rollout. Important Tips: Avoid Disruptions: Utilize Device OS APIs to control OTA availability, ensuring devices aren’t disrupted during critical operations. Managing OTA Updates: Use the console or REST API to force enable OTA updates if needed. Monitoring: Keep an eye on the devices' behaviors post-update to quickly identify and address any potential issues. With Particle's OTA capabilities, developers can ensure that their IoT devices are always running the latest, most secure, and most efficient version of their firmware, enhancing the reliability and functionality of their IoT ecosystems.

Creating a Webhook for the Edge Impulse ingestion API

This guide supplements the tutorial on OTA Model Updates with Edge Impulse on Particle Workbench, focusing on configuring a Particle webhook for sending data to the Edge Impulse ingestion API.

Steps for Webhook Configuration:

1. Access Particle Console:

   • Log in with your Particle account credentials.

2. Navigate to Integrations:

   • Click on the "Integrations" tab in the left-hand menu.

   • Select "Webhooks" from the available options.

3. Create a New Webhook:

   • Click "New Integration".

   • Choose "Webhook".

4. Webhook Configuration:

   • Name: Assign a descriptive name to your webhook.

   • Event Name: Specify the event name that triggers the webhook (e.g., "edge/ingest").

   • URL: Set this to the Edge Impulse ingestion API URL, typically something like https://ingestion.edgeimpulse.com/api/training/data.

   • Request Type: Choose "POST".

   • Request Format: Select "Custom".

5. Custom Request Body:

   • Input the JSON structure required by Edge Impulse. This will vary based on your project's data schema.

6. HTTP Headers:

   • Add necessary headers:

     • x-api-key: Your Edge Impulse API key.

     • Content-Type: "application/json".

     • x-file-name: Use a dynamic data field like {{PARTICLE_EVENT_NAME}}.

7. Advanced Settings:

   • Response Topic: Create a custom topic for webhook responses, e.g., {{PARTICLE_DEVICE_ID}}/hook-response/{{PARTICLE_EVENT_NAME}}.

   • Enforce SSL: Choose "Yes" for secure transmission.

8. Save the Webhook:

   • After entering all details, click "Save".

9. Test the Webhook:

   • Use example device firmware to trigger the webhook.

   • Observe the responses in the Particle Console.

10. Debugging:

• If errors occur, review the logs for detailed information.

• Ensure payload format aligns with Edge Impulse requirements.

• Verify the accuracy of your API key and other details.

Custom Template Example:

Copy and paste the following into the Custom Template section of the webhook:

Closing the Loop to send data to Edge Impulse for Lifecycle Management

Below we will do a basic example of sending data to Edge Impulse for Lifecycle Management. We could further extend this example to include more advanced checking, utilizing more of Particle OTA functionality, and add checks for model performance and versioning. The code gathers data from analog sensors, processes it, and sends it to Edge Impulse for Lifecycle Management. The webhook is triggered with the event name edge/ingest/sample.

Conclusion

This tutorial provides a basic guide for implementing OTA updates on your Particle Workbench connected device (Photon 2). It can be extended to include more advanced checking, utilizing more of espressif OTA functionality, and extending the python server to check for model performance and versioning. Happy coding!

Introduction

This page is part of the tutorial series. If you haven't read the introduction yet, we recommend you do so .

In this tutorial, we'll guide you through deploying updated impulses over-the-air (OTA) to Arduino using the Arduino IoT Cloud and Edge Impulse.

Let's get started!

Prerequisites:

• Edge Impulse Account: .

• Trained Impulse: If you're new, follow our and guides.

Key Features of Arduino OTA Updates:

• Remote Update: Update your Arduino board firmware remotely.

• Integration with Edge Impulse: Seamlessly incorporate machine learning models.

OTA Code

Here’s an example Arduino sketch for implementing OTA updates with Edge Impulse:

To expand the Arduino sketch to work with the Edge Impulse API, you'll need to add functionality to fetch the latest model updates from Edge Impulse and apply them to your device. Below is an extended version of the loop(), connectToWiFi(), and onOTAEvent() functions, integrating Edge Impulse API interactions:

loop() Function

• The loop() function regularly checks for updates from the Edge Impulse API.

connectToWiFi() Function

• The connectToWiFi() function includes a maximum number of attempts to connect to WiFi.

onOTAEvent() Function

• The onOTAEvent() function is triggered when a new update is available, downloading and applying the update.

This sketch sets up the Arduino to connect to the Arduino IoT Cloud and listens for OTA update notifications. When an OTA update is available, the onOTAEvent function is called, where you can implement the logic to download and update the firmware.

Preparing for OTA Updates

1. Setup Arduino IoT Cloud: Configure your device and variables in the Arduino IoT Cloud.

2. Connect your device: Ensure your Arduino board is connected to the internet.

3. Integrate with Edge Impulse: Export your trained impulse from Edge Impulse as an Arduino library.

Steps to Deploy Impulse to Arduino

1. Update Sketch: Incorporate the Edge Impulse library into your Arduino sketch.

2. Listen for OTA Updates: Use the Arduino IoT Cloud to push OTA updates to your device.

3. Test and Monitor: After deploying the update, monitor your device's performance and ensure the new impulse is functioning correctly.

Components

• ArduinoIoTCloud: Manages cloud connectivity and OTA updates.

• WiFiNINA: Handles WiFi connections for compatible boards.

Conclusion

This tutorial provides a basic guide for implementing OTA updates on Arduino boards using the Arduino IoT Cloud and integrating with Edge Impulse. Expand and develop this example to suit your project's needs. Remember, testing is crucial before deploying updates in a live environment. Happy coding!

Introduction

This page is part of the tutorial series. If you haven't read the introduction yet, we recommend you to do so .

In this tutorial, we'll guide you through deploying updated impulses over-the-air (OTA) to Arduino using Edge Impulse. We'll build on Arduino firmware update workflow, incorporating Edge Impulse's API to check for updates and download the latest build.

Let's get started!

Prerequisites:

• Edge Impulse Account: If you haven't got one, .

• Trained Impulse: If you're new, follow our and guides.

Key Features of Arduino OTA Updates:

Arduino OTA Update

OTA Code

Here’s the complete C code for implementing OTA updates with Edge Impulse on ESP-EYE (ESP32).

Prerequisites

• A trained impulse in Edge Impulse Studio

• Installation of required software as detailed in the tutorial

Preparation

Begin by setting up your device for OTA updates following Espressif's OTA firmware update workflow. Use the built binary from the C++ example and modify it to incorporate OTA functionality.

Steps to Deploy Impulse to ESP32

1. Copy the ESP OTA example and configure your wifi settings

Clone the example repository and adjust it according to your project and connectivity settings.

2. Server Side OTA

Modify the ESP OTA example server to check for updates to your project

2. Modify the ESP OTA example to check for updates to your project

Modify the Edge Impulse C++ example for ESP32 to check for updates to your project and download the latest build.

We will need to add a few libraries to the project to facilitate the OTA update process. These are taken from the ESP32 OTA example and are already included in the example project.

Components

1. Non-Volatile Storage (NVS)

NVS is utilized to persistently store data like configuration settings, WiFi credentials, or firmware update times, ensuring retention across reboots.

2. HTTP Client

This library facilitates HTTP requests to the server for checking and retrieving new firmware updates.

3. OTA Operations

These headers aid in executing OTA operations, including writing new firmware to the flash and switching boot partitions.

4. FreeRTOS Task

FreeRTOS ensures OTA updates are conducted in a separate task, preventing blockage of other tasks and maintaining system operations during the update.

3. Updating the Device

Compare the model's timestamp or hash with the stored version. If it's different or newer, call the download_model() function.

4. Monitoring and Repeating the Process

Monitor the device to ensure the new impulse performs as expected and repeat the update process as needed.

Conclusion

This tutorial provides a comprehensive guide for implementing OTA updates on Espressif ESP-EYE (ESP32) with Edge Impulse. Follow each step meticulously, ensuring all prerequisites and preparation steps are completed before proceeding to the deployment phase. Happy coding!

Note: Adjust the code snippets and steps to suit your specific requirements and always ensure to test thoroughly before deploying updates to live environments.

Introduction

This page is part of the Lifecycle Management with Edge Impulse tutorial series. If you haven't read the introduction yet, we recommend you to do so here.

In this tutorial, we'll guide you through deploying updated impulses over-the-air (OTA) using the Zephyr RTOS. We'll build on Zephyr's firmware update workflow, incorporating API calls to check for updates to your edge impulse project and download the latest build. As Zephyr does not have an OTA update service, and Golioth is used in their OTA guide. We'll use Golioth's OTA update service to deliver the updated firmware to the device. On devices running Zephyr RTOS, leveraging Golioth's cloud platform for the firmware update portion of the process.

Key Features of Golioth OTA Updates:

• Secure and encrypted firmware delivery

• Delta updates to minimize download size

• Integration with Zephyr RTOS

Prerequisites

• Edge Impulse Account: If you haven't got one, sign up here.

• Trained Impulse: If you're new, follow one of our end-to-end tutorials

• Knowledge of Zephyr RTOS and nRF Connect SDK

• A Golioth account and the Golioth SDK installed

Preparation

1. Clone the Golioth Zephyr SDK examples repository:

git clone https://github.com/golioth/zephyr-sdk.git

On Your nRF9160 Development Board:

1. Navigate to the Golioth firmware examples folder we are going to modify the device firmware update example to connect to the ingestion API to send data back from device.

cd zephyr-sdk/samples/dfu

Now copy the edge-impulse-sdk folder from the cloned repository into the dfu folder.

cp -r edge-impulse-sdk zephyr-sdk/samples/dfu

Include the header file the dfu example

#include "edge-impulse-sdk/classifier/ei_run_classifier.h"

Take the run inference example from your existing project

You will need to merge in the run inference function from your existing project. This will be the function that will be called to run inference on the device.

static void run_inference() {
    struct sensor_value accel[3];
    struct device *sensor = device_get_binding(SENSOR_DEV_NAME);
    if (sensor == NULL) {
        printk("Could not get %s device\n", SENSOR_DEV_NAME);
        return;
    }
    sensor_sample_fetch(sensor);
    sensor_channel_get(sensor, SENSOR_CHAN_ACCEL_XYZ, accel);

    float features[3];
    for (int i = 0; i < 3; i++) {
        features[i] = sensor_value_to_double(&accel[i]);
    }

Add the run inference function to the main loop

Now add the run inference function to the main loop of the dfu example. This will run inference every 2 seconds.

int main() {
    struct device *sensor = device_get_binding(SENSOR_DEV_NAME);
    if (sensor == NULL) {
        printk("Could not get %s device\n", SENSOR_DEV_NAME);
        return -1;
    }
    struct sensor_trigger trig;
    trig.type = SENSOR_TRIG_DATA_READY;
    trig.chan = SENSOR_CHAN_ACCEL_XYZ;
    if (sensor_trigger_set(sensor, &trig, sensor_trigger_handler) < 0) {
        printk("Could not set trigger for %s sensor\n", SENSOR_DEV_NAME);
        return -1;
    }

    while (1) {
        k_sleep(K_SECONDS(1));
    }

    return 0;
}

Add a call back to send sensor data to Edge Impulse with data captured

static void sensor_trigger_handler(struct device *dev, struct sensor_trigger *trigger) {
    run_inference();
    // Send data to Edge Impulse via ingestion API
    send_data_to_edge_impulse();

}

Add the send data to Edge Impulse function

static void send_data_to_edge_impulse() {
    // Set up HTTP client request
    struct http_client_request req;
    struct http_client_ctx ctx;
    char data_buf[512];
    snprintf(data_buf, sizeof(data_buf), "{\"label\":\"%s\",\"data\":\"%s\"}", "my_label", "my_data");
    http_client_request_init(&req);
    req.method = HTTP_POST;
    req.url = "http://ingestion.edgeimpulse.com/api/training/data";
    req.protocol = "HTTP/1.1";
    req.host = "ingestion.edgeimpulse.com";
    req.header_fields = "Content-Type: application/json\r\n"
                        "x-api-key: " EDGE_IMPULSE_API_KEY "\r\n";
    req.payload = data_buf;
    req.payload_len = strlen(data_buf);
    http_client_init(&ctx, &req);
    // Send HTTP request
    int ret = http_client_send(&ctx);
    if (ret < 0) {
        printf("HTTP request failed: %d\n", ret);
    } else {
        printf("HTTP request sent successfully\n");
    }
}

Add the sensor trigger to the main function

int main() {
    struct device *sensor = device_get_binding(SENSOR_DEV_NAME);
    if (sensor == NULL) {
        printk("Could not get %s device\n", SENSOR_DEV_NAME);
        return -1;
    }
    struct sensor_trigger trig;
    trig.type = SENSOR_TRIG_DATA_READY;
    trig.chan = SENSOR_CHAN_ACCEL_XYZ;
    if (sensor_trigger_set(sensor, &trig, sensor_trigger_handler) < 0) {
        printk("Could not set trigger for %s sensor\n", SENSOR_DEV_NAME);
        return -1;
    }

    while (1) {
        k_sleep(K_SECONDS(1));
    }

    return 0;
}

Building and Running

With the classification, ingestion code and Golioth implementation in place, proceed to build and run the application on the nRF9160 Development Board. Utilize the following commands:

west build -b nrf9160dk_nrf9160_ns -p
west flash

These commands will build and flash the firmware onto your development board.

Verifying The Operation

To verify that the Edge Impulse model is running and updating OTA with Golioth, observe the console output. It should display inference results at regular intervals. Also, you can monitor the device’s firmware version and its updates via the Golioth console.

Testing OTA Updates

Simulate an OTA update by changing the Edge Impulse model and repeating the build and flash process. Monitor the Golioth console for updates and check the device console to observe the changes in inference results.

Conclusion

Congratulations! You've successfully implemented OTA model updates with Edge Impulse on Zephyr RTOS and nRF Connect SDK, facilitated by Golioth. You can now continuously enhance and deploy machine learning models to your edge devices securely and efficiently.