> ## Documentation Index
> Fetch the complete documentation index at: https://docs.edgeimpulse.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run Edge Impulse vision on a JetBot with Rubik Pi

> Combine JetBot ROS 2 motor control on Qualcomm Rubik Pi with Edge Impulse Linux object detection.

This tutorial runs an Edge Impulse object detection model on a JetBot-class robot with a Qualcomm Rubik Pi, Ubuntu 24.04, and ROS 2 Jazzy. It combines the robot-specific [`jetbot_ros`](https://github.com/edgeimpulse/ei_jetbot_ros_rubikpi) package with [`edgeimpulse_ros`](https://github.com/edgeimpulse/edgeimpulse-ros).

The JetBot launch file starts a `v4l2_camera` driver that publishes camera frames. `edgeimpulse_ros` subscribes to them and publishes detections. `jetbot_ros` handles motor control, teleoperation, and navigation. Because perception is a separate node, you can reuse it on other robots.

![JetBot Gazebo simulation with a camera stream](https://raw.githubusercontent.com/edgeimpulse/ei_jetbot_ros_rubikpi/main/docs/images/jetbot_gazebo_sim_video.jpg)

JetBot Gazebo development environment and simulated camera stream

## What you'll build

Object detection on the robot's camera feed that publishes results to ROS 2, alongside motor control that a behavior node can drive.

```mermaid theme={"system"}
flowchart LR
    CAM["USB camera"] --> DRV["v4l2_camera"]
    DRV --> IMG["/jetbot/camera/image_raw"]
    IMG --> EI["edgeimpulse_ros"]
    MODEL["Linux .eim model"] --> EI
    EI --> DET["/edgeimpulse_detector/detections"]
    EI --> DIAG["/diagnostics"]
    DET --> BEHAVIOR["Robot behavior node"]
    BEHAVIOR --> CMD["/cmd_vel"]
    CMD --> MOTORS["JetBot motor controller"]
```

## Prerequisites

* A Rubik Pi running Ubuntu 24.04 and ROS 2 Jazzy
* A SparkFun JetBot AI Kit v2.0 or another JetBot-class chassis with a SparkFun or Waveshare motor controller
* A USB camera connected to the Rubik Pi
* A Linux `.eim` object detection model exported from Edge Impulse Studio
* Local copies of [ei\_jetbot\_ros\_rubikpi](https://github.com/edgeimpulse/ei_jetbot_ros_rubikpi) and [edgeimpulse-ros](https://github.com/edgeimpulse/edgeimpulse-ros)

Read [Run Edge Impulse object detection with ROS 2](https://docs.edgeimpulse.com/tutorials/topics/robotics/edgeimpulse-ros2-integration.md) first. It covers the model export, runtime, and topics that this tutorial uses.

## 1. Provision the Rubik Pi

On the Rubik Pi, run the repository's provisioner:

```bash theme={"system"}
cd ~/jetbot_ros_rubikpi
sudo bash scripts/rubikpi_provision.sh
```

The script installs the ROS 2 Jazzy base packages, V4L2 camera support, `vision_msgs`, `diagnostic_msgs`, OpenCV, NumPy, PortAudio (for the SDK's `pyaudio` dependency), and the JetBot hardware dependencies. If you use the SparkFun motor controller, check that the Qwiic driver is installed:

```bash theme={"system"}
python3 -c "import qwiic; print(qwiic.__file__)"
```

## 2. Build a shared ROS 2 workspace

The JetBot setup script links both packages into one workspace:

```bash theme={"system"}
cd ~/jetbot_ros_rubikpi
bash scripts/setup_edge_impulse_workspace.sh ~/jetbot_edgeimpulse_ws ~/jetbot_ros_rubikpi ~/edgeimpulse-ros

cd ~/jetbot_edgeimpulse_ws
source /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install
source install/setup.bash
```

Install the Edge Impulse Linux SDK and the JetBot runtime requirements:

```bash theme={"system"}
sudo apt install -y ros-jazzy-vision-msgs ros-jazzy-diagnostic-msgs \
                    ros-jazzy-v4l2-camera python3-opencv python3-numpy portaudio19-dev
sudo python3 -m pip install --break-system-packages \
  -r ~/jetbot_ros_rubikpi/requirements-edge-impulse.txt
```

> [!WARNING]
>
> These commands install the Edge Impulse runtime into the Ubuntu 24.04 system Python with `--break-system-packages`. Use this setup only on a dedicated robot image or a disposable development machine.

## 3. Start motor control and object detection

The integrated launch file starts the motor node, a `v4l2_camera` driver, and the `edgeimpulse_ros` detector. Give it the model path and the camera device:

```bash theme={"system"}
cd ~/jetbot_edgeimpulse_ws
source /opt/ros/jazzy/setup.bash
source install/setup.bash

ros2 launch jetbot_ros jetbot_edge_impulse.launch.py \
  motor_controller:=motors_sparkfun \
  model_path:=/home/$USER/models/jetbot-detector.eim \
  video_device:=/dev/video0
```

For a Waveshare motor controller, install its driver package, then launch with `motors_waveshare`:

```bash theme={"system"}
pip install Adafruit-MotorHAT

ros2 launch jetbot_ros jetbot_edge_impulse.launch.py \
  motor_controller:=motors_waveshare \
  model_path:=/home/$USER/models/jetbot-detector.eim \
  video_device:=/dev/video0
```

> [!WARNING]
>
> The launch file starts a `v4l2_camera` driver on `video_device`. Don't start a second camera driver on the same `/dev/video*` device. If MJPG gives an empty encoding or a `cv_bridge` exception, use YUYV.

## 4. Inspect inference output

View the detections and diagnostics:

```bash theme={"system"}
source ~/jetbot_edgeimpulse_ws/install/setup.bash
ros2 topic echo /edgeimpulse_detector/detections
ros2 run rqt_robot_monitor rqt_robot_monitor
```

| Topic | Purpose |
| - | - |
| `/edgeimpulse_detector/detections` | `vision_msgs/Detection2DArray` messages with the accepted bounding boxes. |
| `/edgeimpulse_detector/debug_image` | Annotated `sensor_msgs/Image` when `publish_debug_image:=true`. |
| `/diagnostics` | `diagnostic_msgs/DiagnosticArray` with FPS, latency, and model health. |

Don't let detections drive the robot until they're stable. When you add a behavior node, base each action on the confidence, label, and robot state rather than on every frame.

## Motor and camera checks

Before you connect detections to motion, test the motors on their own. Start hardware control:

```bash theme={"system"}
source ~/jetbot_edgeimpulse_ws/install/setup.bash
ros2 launch jetbot_ros jetbot_cpu.launch.py motor_controller:=motors_sparkfun
```

Then send a short forward command and a stop:

```bash theme={"system"}
ros2 topic pub -1 /cmd_vel geometry_msgs/msg/Twist "{linear: {x: 0.05}, angular: {z: 0.0}}"
ros2 topic pub -1 /cmd_vel geometry_msgs/msg/Twist "{linear: {x: 0.0}, angular: {z: 0.0}}"
```

## Simulation notes

The repository includes Gazebo Classic worlds for navigation and data collection. Run them on an Ubuntu or WSL development machine with Gazebo Classic installed:

```bash theme={"system"}
ros2 launch jetbot_ros gazebo_world.launch.py
```

You can point the detector's `image_topic` at the Gazebo camera stream, but the repository only tests the detector with a real V4L2 camera.

## Troubleshooting

| Symptom | Resolution |
| - | - |
| `package 'jetbot_ros' not found` | Source `/opt/ros/jazzy/setup.bash` and `~/jetbot_edgeimpulse_ws/install/setup.bash`. |
| Camera device is busy | Stop any other camera client using the same `/dev/video*` device before starting the launch. |
| Qwiic module cannot be imported | Re-run the provisioner, or install the SparkFun Qwiic packages into the system Python ROS 2 uses. |
| Waveshare motor import fails | Install `Adafruit-MotorHAT` in the Python environment used by the motor node. |
| No detections | Check the `.eim` path, `video_device`, model labels, and `confidence_threshold`. Check that the camera is publishing with `ros2 topic hz /jetbot/camera/image_raw`, and check `/diagnostics` for inference activity. |
| Robot moves but detections are unstable | Turn off autonomous behavior, collect images from the mounted camera, retrain the model, and raise `confidence_threshold` before you turn behavior back on. |

## Next steps

* Add a behavior node that subscribes to `/edgeimpulse_detector/detections` and publishes `/cmd_vel` commands with safety checks.
* Watch the FPS and latency on `/diagnostics` under load.
* Collect more training data from the camera's final position.
* Test autonomous behavior at low speed before you increase it.