NVIDIA Omniverse

NVIDIA Omniverse™ is a scalable, multi-GPU real-time reference development platform for building and operating metaverse applications and based on Pixar's Universal Scene Description and NVIDIA RTX™ technology.

NVIDIA Omniverse stack

This tutorial describes how you can use the Edge Impulse extension within NVIDIA Omniverse to upload your synthetic datasets to your Edge Impulse project for computer vision tasks, validate your trained model locally, and view inferencing results directly in your Omniverse synthetic environment.

Getting started with NVIDIA Omniverse

Check out NVIDIA's documentation for information on getting started as a first-time user with the Omniverse platform.

Preliminary steps

• Create an NVIDIA Omniverse account
• Create an Edge Impulse account

Now continue with NVIDIA's Omniverse installation guide.

Installing the Edge Impulse Omniverse extension

Once you have installed NVIDIA Omniverse, you can now install the Edge Impulse extension into your Omniverse environment by following the README in the extension's GitHub repository.

Generating a synthetic dataset

Follow this tutorial by Edge Impulse expert George Igwegbe to create a synthetic dataset using NVIDIA Omniverse Replicator.

Note: Adding bounding boxes

In order to collect bounding box data from your scene, semantic information for the objects of interest must be specified. A comprehensive guide on how to do this can be found here.

To preview your bounding boxes, click on the icon to the right of the camera perspective button, select either tight or loose 2D bounding boxes and then "Show Window".

Select Window

You should see a popup like this:

Preview Bounding Boxes

Once you have confirmed your bounding boxes, you can check "bounding_box_2d_loose" and/or "bounding_box_2d_tight" under the "Parameters" tab in the Synthetic Data Recorder and the bounding boxes for your recorded data will appear in the output directory:

Data Recorder

Then, using the Edge Impulse extension you installed in the previous step, upload your dataset and download your trained model by following the steps below:

NVIDIA Omniverse Edge Impulse extension

1. Create a free Edge Impulse account: https://studio.edgeimpulse.com/

2. Connect to your Edge Impulse project by setting your API key (this key is obtained from your Edge Impulse project Dashboard > Keys > API Keys), then click Connect

3. Once your project is connected to your Edge Impulse Omniverse extension, select either the Data Upload or Classification drop-downs:

For synthetic data collection, select Data Upload, then specify your dataset's local path on your computer, select the dataset category to upload to (training, testing, or anomaly), then click Upload to Edge Impulse:

Data Upload

To add bounding box data, click the checkbox next to "Add Bounding Boxes" and then specify and RGB path with your RGB images and Bounding Box path with the raw bounding box data from Replicator, then click Upload to Edge Impulse:

Data Upload with Bounding Boxes

For classification tasks and to see the results of inferencing from your trained model in Omniverse, select Classification, then click Classify current scene frame to start inferencing locally on the edge within your Omniverse environment:

Classification

Next steps: building a machine learning model

With everything set up you can now build your machine learning model with these tutorials:

• Image classification
• Detect objects with bounding boxes
• Detect objects with centroids (FOMO)

Looking to connect different sensors? The Data forwarder lets you easily send data from any sensor into Edge Impulse.