> ## Documentation Index
> Fetch the complete documentation index at: https://docs.edgeimpulse.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run Edge Impulse object detection with ROS 2

> Use edgeimpulse_ros to run a Linux .eim model on a ROS 2 image topic and publish vision_msgs detection messages.

[`edgeimpulse_ros`](https://github.com/edgeimpulse/edgeimpulse-ros) is a ROS 2 package that runs an Edge Impulse Linux `.eim` model and publishes the results as standard ROS 2 messages. The JetBot and inspection tutorials in the [robotics section](https://docs.edgeimpulse.com/tutorials/topics/robotics.md) use it for perception. The UNO Q Braccio tutorial uses its own runner pipeline instead.

The node only runs inference. It subscribes to a `sensor_msgs/Image` topic from a separate camera driver and publishes `vision_msgs` messages. Robot control stays in robot-specific ROS 2 packages.

## What you'll build

A ROS 2 node that subscribes to a camera image topic, runs an Edge Impulse model locally, and publishes bounding boxes, latched label metadata, and diagnostics.

```mermaid theme={"system"}
flowchart LR
    CAM["Camera driver
(for example v4l2_camera)"] --> IMG["/image
sensor_msgs/Image"]
    IMG --> NODE["edgeimpulse_ros
edgeimpulse_detector"]
    MODEL["Linux .eim model"] --> NODE
    NODE --> DET["/edgeimpulse_detector/detections
Detection2DArray"]
    NODE --> DBG["/edgeimpulse_detector/debug_image
Image"]
    NODE --> DIAG["/diagnostics
DiagnosticArray"]
    DET --> ROBOT["Robot-specific ROS 2 nodes"]
```

![JetBot Gazebo simulation with a camera feed](https://raw.githubusercontent.com/edgeimpulse/ei_jetbot_ros_rubikpi/main/docs/images/jetbot_gazebo_sim.jpg)

A JetBot simulation environment used to develop robot-side ROS 2 workflows

## Prerequisites

* A ROS 2 Jazzy or Rolling workspace. Both ship `vision_msgs` 4.x. The robotics tutorials use ROS 2 Jazzy on Ubuntu 24.04. Humble isn't tested in CI yet.
* A camera driver that publishes a `sensor_msgs/Image` topic, such as `v4l2_camera` for a USB camera at `/dev/video*`.
* An Edge Impulse project with a trained image model: object detection, image classification, FOMO, or visual anomaly detection.

Install the system dependencies. If you don't use Jazzy, replace `jazzy` with your ROS distribution. The Ubuntu 24.04 system Python is [externally managed (PEP 668)](https://peps.python.org/pep-0668/), so install the SDK and `pyaudio` into your user site with `--break-system-packages`:

```bash theme={"system"}
sudo apt update
sudo apt install -y ros-jazzy-vision-msgs ros-jazzy-diagnostic-msgs \
                    ros-jazzy-v4l2-camera \
                    python3-opencv python3-numpy portaudio19-dev

pip install --user --break-system-packages edge_impulse_linux pyaudio
```

> [!NOTE]
>
> The Edge Impulse Linux SDK imports `pyaudio` even for image models, so you need `portaudio19-dev` to build it. Install the SDK into the Python environment that ROS 2 uses to start the node. Don't clone the SDK inside a colcon workspace, because colcon can treat it as a ROS package.

## 1. Export an Edge Impulse model

1. Train and test an object detection impulse in Edge Impulse Studio.
2. Go to **Deployment** and select the [Linux `.eim` binary](https://docs.edgeimpulse.com/studio/projects/deployment#deploy-as-a-linux-eim-binary) for your device's architecture.
3. Build and download the `.eim` file.
4. Copy the file to the inference host, for example to `~/models/object-detector.eim`.

Inference runs on the host, so camera frames and results don't leave the device.

## 2. Build the package

Clone the package into a ROS 2 workspace and build it:

```bash theme={"system"}
mkdir -p ~/edgeimpulse_ws/src
cd ~/edgeimpulse_ws/src
git clone https://github.com/edgeimpulse/edgeimpulse-ros.git edgeimpulse_ros

cd ~/edgeimpulse_ws
source /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install
source install/setup.bash
```

## 3. Run the detector

The node needs frames on an image topic. To try it, run the webcam demo, which also starts a `v4l2_camera` driver. The demo needs `ros-$ROS_DISTRO-v4l2-camera`:

```bash theme={"system"}
source ~/edgeimpulse_ws/install/setup.bash

ros2 launch edgeimpulse_ros edgeimpulse_with_camera.launch.py \
  model_path:=/home/$USER/models/object-detector.eim \
  publish_debug_image:=true
```

If you already run a camera driver, launch only the detector and point it at the driver's image topic:

```bash theme={"system"}
source ~/edgeimpulse_ws/install/setup.bash

ros2 launch edgeimpulse_ros edgeimpulse_detector.launch.py \
  model_path:=/home/$USER/models/object-detector.eim \
  image_topic:=/camera/image_raw \
  image_qos:=sensor_data
```

To override individual parameters from the command line, use `ros2 run`:

```bash theme={"system"}
ros2 run edgeimpulse_ros edgeimpulse_detector --ros-args \
  -p model_path:=/home/$USER/models/object-detector.eim \
  -p image_topic:=/camera/image_raw \
  -p image_qos:=sensor_data \
  -p confidence_threshold:=0.5
```

`confidence_threshold` drops boxes below the given score. The default, `-1.0`, uses the model's own threshold. Raise it, for example to `0.5`, when false positives are costly, or lower it to check recall in a controlled test.

## Outputs

Topics are relative to the node's namespace, `/edgeimpulse_detector` by default. The result topics depend on the model type.

| Topic | Type | Purpose |
| - | - | - |
| `~/detections` | `vision_msgs/Detection2DArray` | Bounding boxes and class hypotheses (object detection, FOMO, visual anomaly). |
| `~/classification` | `vision_msgs/Classification` | Class scores for image-classification models. |
| `~/anomaly` | `std_msgs/Float32` | Peak anomaly score for models with an anomaly output. |
| `~/debug_image` | `sensor_msgs/Image` | Annotated overlay when `publish_debug_image:=true`. |
| `~/vision_info` | `vision_msgs/VisionInfo` | Latched model metadata (always published). |
| `~/label_info` | `vision_msgs/LabelInfo` | Latched label map (always published). |
| `/diagnostics` | `diagnostic_msgs/DiagnosticArray` | FPS, latency, dropped frames, and model health when `publish_diagnostics:=true`. |

Inspect the output in another terminal:

```bash theme={"system"}
source ~/edgeimpulse_ws/install/setup.bash
ros2 topic echo /edgeimpulse_detector/detections
ros2 topic echo /edgeimpulse_detector/debug_image
ros2 run rqt_robot_monitor rqt_robot_monitor
```

Each `Detection2DArray` holds every accepted bounding box in one frame, in the original image's pixel coordinates. The `vision_msgs` pose and covariance fields are part of the message definition but carry no meaning for 2D detection.

## Useful parameters

| Parameter | Default | Description |
| - | -: | - |
| `model_path` | None | Required path to the `.eim` file. |
| `image_topic` | `image` | Input `sensor_msgs/Image` topic to subscribe to. |
| `image_transport` | `raw` | `raw` or `compressed` (uses `<image_topic>/compressed`). |
| `image_qos` | `sensor_data` | `sensor_data`, `reliable`, or `default`. Must match the publisher. |
| `resize_mode` | `auto` | `auto`, `squash`, `fit-shortest`, or `fit-longest`. `auto` uses the resize mode set in the studio. |
| `confidence_threshold` | `-1.0` | Minimum score to publish; `<0` uses the model's own threshold. |
| `publish_debug_image` | `false` | Publish an annotated debug image on `~/debug_image`. |
| `overlay_labels` | `true` | Draw class labels on detection boxes in the debug image. |
| `frame_id_override` | `""` | Override the source image `frame_id` when non-empty. |
| `publish_diagnostics` | `true` | Publish the `DiagnosticArray` on `/diagnostics`. |
| `warn_on_drop` | `false` | Log when the node drops a stale frame to process the latest one. |

The package includes a parameter file at `config/edgeimpulse_detector.yaml`.

## Troubleshooting

| Symptom | Resolution |
| - | - |
| `model_path` is required | Pass a readable `.eim` path with `-p model_path:=...`. |
| `dependency "pyaudio" is missing` | The SDK imports `pyaudio` even for image models: `sudo apt install portaudio19-dev && pip install --user --break-system-packages pyaudio`. |
| The Edge Impulse Linux SDK is required at runtime | Install the SDK into the interpreter ROS uses: `pip install --user --break-system-packages edge_impulse_linux`. |
| `Model file ... is not executable` | The `.eim` is a native binary; add the exec bit: `chmod +x /path/to/model.eim`. |
| No messages on `~/detections` | Confirm the camera driver is publishing with `ros2 topic hz <image_topic>`, and that `image_qos` matches the publisher (a `reliable` subscriber cannot receive from a `best_effort` publisher; `sensor_data` is safe). |
| Boxes look shifted or scaled | Set `resize_mode` to the resize mode of your impulse in the studio. |
| No detections | Confirm the model type, test it in the deployment environment, then adjust `confidence_threshold`. |

## Use it with a robot

When the detector publishes reliably, connect its output to your robot's logic:

* [Run Edge Impulse vision on a JetBot with Rubik Pi](https://docs.edgeimpulse.com/tutorials/topics/robotics/wheeled-robot-ros2-integration.md) combines detections with JetBot motor control.
* [Build a mobile inspection system with Edge Impulse and ROS 2](https://docs.edgeimpulse.com/tutorials/topics/robotics/wheeled-inspection-system.md) designs a rover inspection system with detection, anomaly scoring, and operator traceability.
* [Run Edge Impulse pick and place with UNO Q Braccio](https://docs.edgeimpulse.com/tutorials/topics/robotics/robotic-arm-ros2-integration.md) sorts objects with a Braccio arm, using its own runner pipeline.

For another robot, keep the same split: a camera driver, `edgeimpulse_ros` for perception, and a separate ROS 2 node that subscribes to `/edgeimpulse_detector/detections` and handles safety checks, state, and actuation.

## Next steps

* Test the model with the final camera, lighting, and object distance.
* Watch the FPS and latency on `/diagnostics` while the robot is under load.
* Add robot-specific checks before any autonomous motion.
* Keep your launch file and parameters in the robot's repository instead of changing the detector node.