NVIDIA Jetson

'NVIDIA Jetson Orin' refers to the following devices:

• Jetson AGX Orin Series, Jetson Orin NX Series, Jetson Orin Nano Series

'NVIDIA Jetson' refers to the following devices:

• Jetson AGX Xavier Series, Jetson Xavier NX Series, Jetson TX2 Series, Jetson TX1, Jetson Nano

'Jetson' refers to all NVIDIA Jetson devices.

The NVIDIA Jetson and NVIDIA Jetson Orin devices are embedded Linux devices featuring a GPU-accelerated processor (NVIDIA Tegra) targeted at edge AI applications. You can easily add a USB external microphone or camera - and it's fully supported by Edge Impulse. You'll be able to sample raw data, build models, and deploy trained machine learning models directly from the Edge Impulse Studio.

In addition to the NVIDIA Jetson and NVIDIA Jetson Orin devices we also recommend that you add a camera and/or a microphone. Most popular USB webcams work fine on the development board out of the box.

Installing dependencies

Follow NVIDIA's setup instructions found at NVIDIA Jetson Getting Started Guide depending on your hardware.

For example:

• NVIDIA Jetson Orin Nano Developer Kit

• NVIDIA Jetson AGX Orin Developer Kit

• NVIDIA Jetson Nano Developer Kit

JetPack

For NVIDIA Jetson Orin:

• use SD Card image with JetPack 5.1.2 or

• use SD Card image with JetPack 6.0

Note that you may need to update the UEFI firmware on the device when migrating to JetPack 6.0 from earlier JetPack versions. See NVIDIA's Initial Setup Guide for Jetson Nano Development Kit for instructions on how to get JetPack 6.0 GA on your device.

For NVIDIA Jetson devices use SD Card image with Jetpack 4.6.4. See also JetPack Archive or Jetson Download Center.

When finished, you should have a bash prompt via the USB serial port, or using an external monitor and keyboard attached to the Jetson. You will also need to connect your Jetson to the internet via the Ethernet port (there is no WiFi on the Jetson). (After setting up the Jetson the first time via keyboard or the USB serial port, you can SSH in.)

Make sure your ethernet is connected to the Internet

Issue the following command to check:

ping -c 3 www.google.com

The result should look similar to this:

3 packets transmitted, 3 received, 0% packet loss, time 2003ms

Running the setup script

To set this device up in Edge Impulse, run the following commands (from any folder). When prompted, enter the password you created for the user on your Jetson/Orin in step 1. The entire script takes a few minutes to run (using a fast microSD card).

For Jetson:

wget -q -O - https://cdn.edgeimpulse.com/firmware/linux/jetson.sh | bash

For Orin:

wget -q -O - https://cdn.edgeimpulse.com/firmware/linux/orin.sh | bash

Connecting to Edge Impulse

With all software set up, connect your camera or microphone to your Jetson (see 'Next steps' further on this page if you want to connect a different sensor), and run:

edge-impulse-linux

This will start a wizard which will ask you to log in, and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

Verifying that your device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to your Edge Impulse project, and click Devices. The device will be listed here.

Next steps: building a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

• Keyword spotting

• Sound recognition

• Image classification

• Object detection

• Object detection with centroids (FOMO)

Choose the deployment target according to your device and JetPack version. See table below.

For more information on Docker deployment see run inference using a Docker container.

Looking to connect different sensors? Our Linux SDK lets you easily send data from any sensor and any programming language (with examples in Node.js, Python, Go and C++) into Edge Impulse.

Deploying back to device

To run your impulse locally, just connect to your Jetson again, and run:

edge-impulse-linux-runner

This will automatically compile your model with full GPU and hardware acceleration, download the model to your Jetson, and then start classifying. Our Linux SDK has examples on how to integrate the model with your favourite programming language.

Image model?

If you have an image model then you can get a peek of what your device sees by being on the same network as your device, and finding the 'Want to see a feed of the camera and live classification in your browser' message in the console. Open the URL in a browser and both the camera feed and the classification are shown:

Troubleshooting

edge-impulse-linux reports "OOM killed!"

Using make -j without specifying job limits can overtax system resources, causing "OOM killed" errors, especially on resource-constrained devices this has been observed on many of our supported Linux based SBCs.

Avoid using make -j without limits. If you experience OOM errors, limit concurrent jobs. A safe practice is:

make -j`nproc`

This sets the number of jobs to your machine's available cores, balancing performance and system load.

edge-impulse-linux reports "[Error: Input buffer contains unsupported image format]"

This is probably caused by a missing dependency on libjpeg. If you run:

vips --vips-config

The end of the output should show support for file import/export with libjpeg, like so:

file import/export with libjpeg: yes (pkg-config)
image pyramid export: no
use libexif to load/save JPEG metadata: no
alex@jetson1:~$

If you don't see jpeg support as "yes", rerun the setup script and take note of any errors.

edge-impulse-linux reports "Failed to start device monitor!"

If you encounter this error, ensure that your entire home directory is owned by you (especially the .config folder):

sudo chown -R $(whoami) $HOME

Long warm-up time and under-performance

By default, the Jetson enables a number of aggressive power saving features to disable and slow down hardware that is detected to be not in use. Experience indicates that sometimes the GPU cannot power up fast enough, nor stay on long enough, to enjoy best performance. You can run a script to enable maximum performance on your Jetson.

ONLY DO THIS IF YOU ARE POWERING YOUR JETSON FROM A DEDICATED POWER SUPPLY. DO NOT RUN THIS SCRIPT WHILE POWERING YOUR JETSON THROUGH USB.

Your Jetson device device can operate in different power modes, a set of power budgets with several predefined configurations CPU and GPU frequencies and number of cores online. To enable maximum performance:

1. Switch to a mode with the maximum power budget and/or frequencies.

2. Then set the clocks to maximum.

To determine the maximum mode for your device visit the Supported Modes and Power Efficiency section in Jetson Linux Developer Guide for your L4T.

For example for R35.4.1:

• NVIDIA Jetson Orin, Jetson Orin NX and Jetson AGX Orin

• NVIDIA Jetson Xavier NX and Jetson AGX Xavier

For R32.7.4:

• Jetson Nano and Jetson TX1

• Jetson TX2

To enable maximum performance, switch to mode ID 0 and set the maximum frequencies of the clocks as follows.

sudo /usr/sbin/nvpmodel -m 0
sudo /usr/bin/jetson_clocks

For NVIDIA Jetson Xavier NX use mode ID 8

Additionally, due to Jetson GPU internal architecture, running small models on it is less efficient than running larger models. E.g. the continuous gesture recognition model runs faster on Jetson CPU than on GPU with TensorRT acceleration.

According to our benchmarks, running vision models and larger keyword spotting models on GPU will result in faster inference, while smaller keyword spotting models and gesture recognition models (that also includes simple fully connected NN, that can be used for analyzing other time-series data) will perform better on CPU.

Program fails to find shared library

If you see an error similar to this when running Linux C++ SDK examples with GPU acceleration,

jetson@localhost:~/example-standalone-inferencing-linux$ ./build/custom
./build/custom: error while loading shared libraries: libnvinfer.so.8: cannot open shared object file: No such file or directory

then please download and use the SD card image version for your target see JetPack. The error is likely caused by an incompatible version of NVidia's GPU libraries - or the absence of these libraries.