Arduino Nicla Vision

The Nicla Vision is a ready-to-use, standalone camera for analyzing and processing images on the Edge. Thanks to its 2MP color camera, smart 6-axis motion sensor, integrated microphone, and distance sensor, it is suitable for asset tracking, object recognition, and predictive maintenance. Some of its key features include:

• Powerful microcontroller equipped with a 2MP color camera

• Tiny form factor of 22.86 x 22.86 mm

• Integrated microphone, distance sensor, and intelligent 6-axis motion sensor

• Onboard Wi-Fi and Bluetooth® Low Energy connectivity

• Standalone when battery-powered

• Expand existing projects with sensing capabilities

• Enable fast Machine Vision prototyping

• Compatible with Nicla, Portenta, and MKR products

Its exceptional capabilities are supported by a powerful STMicroelectronics STM32H747AII6 Dual ARM® Cortex® processor, combining an M7 core up to 480 Mhz and an M4 core up to 240 Mhz. Despite its industrial strength, it keeps energy consumption low for battery-powered standalone applications.

The Arduino Nicla Vision is available for around 95 EUR from the Arduino Store.

Installing dependencies

To set this device up in Edge Impulse, you will need to install the following software:

1. Edge Impulse CLI.

2. Arduino CLI.

   • Here's an instruction video for Windows.

   • The Arduino website has instructions for macOS and Linux.

3. On Linux:

   • GNU Screen: install for example via sudo apt install screen.

Connecting to Edge Impulse

There are two ways to connect the Nicla Vision to Edge Impulse:

• Using the official Edge Impulse firmware - it supports all onboard sensors, including camera.

• Using an ingestion script. This supports analog, IMU, proximity sensors and microphone (limited to 8 kHz), but not the camera. It is only recommended if you want to modify the ingestion flow for third-party sensors.

Edge Impulse firmware

1. Connect the development board to your computer

Use a micro-USB cable to connect the development board to your computer. Under normal circumstances, flash process should work without entering the bootloader manually. However if run into difficulties flashing the board, you can enter the bootloader by pressing RESET twice. The onboard LED should start pulsating to indicate this.

2. Update the firmware

The development board does not come with the right firmware yet. To update the firmware:

1. Download the latest Edge Impulse firmware, and unzip the file.

2. Open the flash script for your operating system (flash_windows.bat, flash_mac.command or flash_linux.sh) to flash the firmware.

3. Wait until flashing is complete, and press the RESET button once to launch the new firmware.

3. Setting keys

From a command prompt or terminal, run:

edge-impulse-daemon

This will start a wizard which will ask you to log in and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

Alternatively, recent versions of Google Chrome and Microsoft Edge can collect data directly from your development board, without the need for the Edge Impulse CLI. See this blog post for more information.

Edge Impulse ingestion scripts

1. Connect the development board to your computer

Use a micro-USB cable to connect the development board to your computer.

2. Update the firmware

The development board does not come with the right firmware yet. To update the firmware:

1. Download the latest Edge Impulse ingestion sketches and unzip the file.

2. Open the nicla_vision_ingestion.ino (for IMU/proximity sensor) or nicla_vision_ingestion_mic.ino(for microphone) sketch in a text editor or the Arduino IDE.

3. For IMU/proximity sensor data ingestion into your Edge Impulse project, at the top of the file, select 1 or multiple sensors by un-commenting the defines and select the desired sample frequency (in Hz). For example, for the accelerometer sensor:

/**
 * @brief   Sample & upload data to Edge Impulse Studio.
 * @details Select 1 or multiple sensors by un-commenting the defines and select
 * a desired sample frequency. When this sketch runs, you can see raw sample
 * values outputted over the serial line. Now connect to the studio using the
 * `edge-impulse-data-forwarder` and start capturing data
 */
#define SAMPLE_ACCELEROMETER
//#define SAMPLE_GYROSCOPE
//#define SAMPLE_PROXIMITY

/**
 * Configure the sample frequency. This is the frequency used to send the data
 * to the studio regardless of the frequency used to sample the data from the
 * sensor. This differs per sensors, and can be modified in the API of the sensor
 */
#define FREQUENCY_HZ        10

For microphone data ingestion, you do not need to change the default parameters in the nicla_vision_ingestion_mic.ino sketch.

1. Then, from your sketch's directory, run the Arduino CLI to compile:

   Copy

   arduino-cli compile --fqbn arduino:mbed_nicla:nicla_vision --output-dir .

2. Then flash to your Nicla Vision using the Arduino CLI:

   Copy

   arduino-cli upload --fqbn arduino:mbed_nicla:nicla_vision

1. Wait until flashing is complete, and press the RESET button once to launch the new firmware.

3a. Data forwarder (Fusion sensors)

From a command prompt or terminal, run:

edge-impulse-data-forwarder

This will start a wizard which will ask you to log in, and choose an Edge Impulse project. You will also name your sensor's axes (depending on which sensor you selected in your compiled nicla_vision_ingestion.ino sketch). If you want to switch projects/sensors run the command with --clean. Please refer to the table below for the names used for each axis corresponding to the type of sensor:

Note: These exact axis names are required for the Edge Impulse Arduino library deployment example applications for the Nicla Vision.

3b. Data forwarder (Microphone)

From a command prompt or terminal, run:

edge-impulse-data-forwarder --baud 1000000 --frequency 8000

This will start a wizard which will ask you to log in and choose an Edge Impulse project. You will also name your sensor axes - in the case of the microphone, you need to enter audio. If you want to switch projects/sensors run the command with --clean.

Verifying that the device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to your Edge Impulse project, and click Devices. The device will be listed here.

With everything set up you can now build your first machine learning model with these tutorials:

• Building a continuous motion recognition system.

• Recognizing sounds from audio.

• Responding to your voice

• Adding sight to your sensors

• Counting objects using FOMO

Looking to connect different sensors? Use the nicla_vision_ingestion.ino sketch and the Edge Impulse data forwarder to easily send data from any sensor on the Nicla Vision into your Edge Impulse project.

Deploying back to device

With the impulse designed, trained and verified you can deploy this model back to your Arduino Nicla Vision. This makes the model run without an internet connection, minimizes latency, and runs with minimum power consumption. Edge Impulse can package the complete impulse - including the signal processing code, neural network weights, and classification code - up into a single library that you can run on your development board.

Use the Running your impulse locally: On your Arduino tutorial and select one of the Nicla Vision examples.