> ## Documentation Index
> Fetch the complete documentation index at: https://docs.edgeimpulse.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run Edge Impulse pick and place with UNO Q Braccio

> Use Edge Impulse object detection, ROS 2, and an Arduino UNO Q to sort colored blocks with a TinkerKit Braccio arm.

This tutorial sorts colored blocks with a TinkerKit Braccio arm, using the pick-and-place workflow from the [UNO Q Braccio repository](https://github.com/edgeimpulse/unoq-braccio). An Ubuntu or WSL host runs ROS 2, the camera stream client, Edge Impulse inference, and the pick-and-place executor. The Arduino UNO Q only receives joint commands over a TCP bridge and moves the arm.

![UNO Q Braccio dashboard with named arm poses](https://raw.githubusercontent.com/edgeimpulse/unoq-braccio/main/docs/IMG_20260615_180657884_HDR~2.jpg)

UNO Q Braccio control dashboard and named arm poses

## What you'll build

A ROS 2 pipeline that streams camera frames, runs Edge Impulse inference in the repository's `edge_impulse_vision` node, publishes the detected label on `/edge_impulse/label`, and runs the matching pick-and-place sequence on the arm.

```mermaid theme={"system"}
flowchart LR
    STREAM["UNO Q MJPEG stream"] --> CAM["mjpeg_camera_node"]
    CAM --> IMG["/braccio/camera/image_raw"]
    IMG --> EI["edge_impulse_vision"]
    EI --> DET["/edge_impulse/detection
JSON string"]
    EI --> LABEL["/edge_impulse/label
String"]
    LABEL --> EXEC["pick_place_executor"]
    EXEC --> CMD["/braccio/joint_command"]
    CMD --> TCP["tcp_bridge via remote.launch.py"]
    TCP --> UNO["Arduino UNO Q + Braccio shield"]
```

## Prerequisites

* An Arduino UNO Q and a TinkerKit Braccio arm with an external 5 V servo power supply
* Ubuntu 24.04 with ROS 2 Jazzy, or WSL 2
* A local clone of [unoq-braccio](https://github.com/edgeimpulse/unoq-braccio)
* An MJPEG camera stream that the host can reach, usually from the UNO Q web or USB camera
* An Edge Impulse model runner command that takes an image path and prints one line of JSON
* Node.js and `npm`, if you use the repository script that installs the Edge Impulse Linux tools

By default, the workflow runs inference with a runner command. The repository can also use `edgeimpulse_ros`, as described in [Alternative: run inference with edgeimpulse\_ros](#alternative-run-inference-with-edgeimpulse_ros). For background on `edgeimpulse_ros`, see [Run Edge Impulse object detection with ROS 2](https://docs.edgeimpulse.com/tutorials/topics/robotics/edgeimpulse-ros2-integration.md).

> [!WARNING]
>
> This tutorial moves a physical arm. Check the clearance around it, keep people out of its reach, and keep the servo power switch within reach. Stock Braccio servos don't report position, torque, current, or temperature.

## 1. Prepare the UNO Q and arm

Flash and test the arm before you connect it to a model. To flash the firmware over USB:

```bash theme={"system"}
arduino-cli lib install Braccio
arduino-cli core install arduino:zephyr
arduino-cli compile --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware
arduino-cli upload -p /dev/ttyACM0 --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware
```

This tutorial controls the arm over the network, so run `app_lab/braccio_web_agent` or `app_lab/braccio_remote_agent` on the UNO Q. The host connects to the UNO Q TCP server on port `8765`.

Before you add vision, check that the arm moves safely through each named pose, using the repository's dashboard or manual test commands.

## 2. Build the workspace and install the Edge Impulse tooling

Build the Braccio workspace first:

```bash theme={"system"}
cd ~/unoq-braccio/ros2_ws
source /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install
source install/setup.bash
cd ..
```

To install the Edge Impulse Linux SDK and CLI on the host, run the repository's setup script:

```bash theme={"system"}
cd ~/unoq-braccio
bash scripts/setup_edge_impulse_linux.sh
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash
```

The script checks for `npm` and installs `edge_impulse_linux`, `opencv-python`, and the Edge Impulse Linux CLI. It doesn't add `edgeimpulse_ros` to the workspace.

> [!NOTE]
>
> If the Edge Impulse CLI or `.eim` tools need `EDGE_IMPULSE_API_KEY`, set it in your shell environment. Never commit it to scripts or repository files.

## 3. Match model labels to workflows

The default `edge_impulse/pick_place_workflows.yaml` supports these labels:

| Object detection label | Default drop location |
| - | - |
| `Red Block` or `red_block` | Left |
| `Blue Block` or `blue_block` | Right |
| `Yellow Block` or `yellow_block` | Center |

The YAML file defines the steps, joint poses, and waits for each item. Edit it before you turn on live inference. Each detector label must match a YAML item exactly.

`edge_impulse_vision` publishes a label only when a result is above its `min_confidence` threshold. `pick_place_executor` ignores new labels while a sequence runs, so repeated detections don't interrupt a motion.

## 4. Test a sequence without vision

To test the network connection and arm poses before the camera can trigger motion, publish a label by hand. Start the remote bridge:

```bash theme={"system"}
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765
```

In a second terminal, start the executor:

```bash theme={"system"}
source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 run unoq_braccio_driver pick_place_executor --ros-args -p workflow_file:=edge_impulse/pick_place_workflows.yaml
```

In a third terminal, publish the test label:

```bash theme={"system"}
source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic pub --once /edge_impulse/label std_msgs/msg/String "{data: 'Red Block'}"
```

Check that each pose is safe for your arm, table, and object position, and adjust the YAML if it isn't.

## 5. Launch the pick-and-place workflow

Start the UNO Q TCP bridge in one terminal:

```bash theme={"system"}
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765
```

In a second terminal, start the camera stream client, the Edge Impulse vision node, and the workflow executor:

```bash theme={"system"}
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_pick_place.launch.py \
  stream_url:=http://192.168.1.64:8080/stream \
  runner_command:="python3 edge_impulse/runner_template.py --image {image}" \
  workflow_file:=edge_impulse/pick_place_workflows.yaml
```

Replace `runner_template.py` with your Edge Impulse model runner. `app_lab/braccio_web_agent` serves both the bridge on port `8765` and the camera stream on port `8080`.

## Alternative: run inference with `edgeimpulse_ros`

Instead of the runner command, you can run inference with the [`edgeimpulse_ros`](https://github.com/edgeimpulse/edgeimpulse-ros) package through `edge_impulse_ros_pick_place.launch.py`. This launch file runs the `edgeimpulse_ros` detector on `/braccio/camera/image_raw`. A `detection_label_bridge` node converts the detector's `vision_msgs/Detection2DArray` output to the `/edge_impulse/label` string that the executor reads, so the pick-and-place workflow doesn't change.

Run this pipeline on a separate machine, or on the UNO Q in a Docker container.

**External host or VM**

Clone and build `edgeimpulse_ros` into `ros2_ws` first, then start the arm bridge in one terminal:

```bash theme={"system"}
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765
```

In a second terminal, start the camera stream, the `edgeimpulse_ros` detector, the label bridge, and the executor:

```bash theme={"system"}
source ~/unoq-braccio/ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_ros_pick_place.launch.py \
  stream_url:=http://192.168.1.64:8080/stream \
  model_path:=/absolute/path/to/model.eim \
  workflow_file:=edge_impulse/pick_place_workflows.yaml
```

**On the UNO Q (Docker)**

The 4 GB UNO Q can run the whole `edgeimpulse_ros` pipeline in a Docker container on its App Lab Linux OS. The container runs the ROS 2 graph, and the App Lab agents keep serving the camera MJPEG stream on `127.0.0.1:8080` and arm control on `127.0.0.1:8765`.

The repository's `Dockerfile` and `docker-compose.yml` use `network_mode: host`, so the container reaches the agents on `localhost` and DDS discovery works.

**1. Install Docker and free some space**

Install Docker Engine on the UNO Q's Debian-based App Lab OS. The arm64 `ros:jazzy` image runs natively on the Qualcomm Dragonwing Cortex-A53 cores.

```bash theme={"system"}
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER   # log out and back in for this to apply
```

The UNO Q has about 10 GB of eMMC storage. If it's nearly full, free some space before you build:

```bash theme={"system"}
docker system prune -af   # remove unused images and containers
df -h /                   # aim for at least 3 GB free
```

**2. Start the arm and camera agent**

In Arduino App Lab, run the `braccio_web_agent` app on the UNO Q. It serves arm control over TCP on port `8765` and the camera MJPEG stream on port `8080`. Check that it responds on localhost:

```bash theme={"system"}
curl -s http://127.0.0.1:8080/camera-status
curl -s http://127.0.0.1:8080/camera-devices   # confirm the right /dev/videoN
```

**3. Clone the repository**

```bash theme={"system"}
cd ~
git clone -b unoq-ros2-docker-braccio https://github.com/edgeimpulse/unoq-braccio.git
cd unoq-braccio
```

**4. Download your model**

Download the model on the UNO Q, so you get an aarch64 `.eim` file. An x86 build won't run on the UNO Q:

```bash theme={"system"}
sudo npm install -g edge-impulse-linux --force
mkdir -p models
edge-impulse-linux-runner --download models/model.eim   # log in when prompted
```

Check that the model's labels match the keys in `edge_impulse/pick_place_workflows.yaml`.

**5. Build and run on-device**

```bash theme={"system"}
docker compose up --build
```

The container runs `onboard_edge_impulse_pick_place.launch.py`, which starts `tcp_bridge` (`host:=127.0.0.1`), `mjpeg_camera_node` (`stream_url:=http://127.0.0.1:8080/stream`), the `edgeimpulse_ros` detector, `detection_label_bridge`, and `pick_place_executor` in one container.

**6. Verify the pipeline**

In a second shell, watch the labels and joint commands inside the container:

```bash theme={"system"}
docker exec -it unoq_braccio_edgeimpulse_ros bash -lc '
  source /braccio_ws/install/setup.bash
  ros2 topic echo /edge_impulse/label &
  ros2 topic echo /braccio/joint_command'
```

Place an object in front of the camera. When a detection scores above `min_confidence` (`0.65`), a label and a joint command appear and the arm moves.

> [!NOTE]
>
> The Braccio packages and `edgeimpulse_ros` are pure `ament_python`, so the `colcon build` in the container is small. It runs sequentially (`--executor sequential --parallel-workers 1`) to fit in the memory of the 2 GB and 4 GB UNO Q. If the build runs out of memory, add swap: `sudo fallocate -l 2G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile`.

`detection_label_bridge` reads detections from `detections_topic` (default `/edgeimpulse_detector/detections`) and publishes a label only when the top score is above `min_confidence` (default `0.65`). The executor, workflow YAML, and `/edge_impulse/label` topic are the same for both pipelines, so you can switch between them without changing the arm workflow.

## Monitor the decision path

To check the inference output, selected labels, and arm commands, run each of these in its own terminal:

```bash theme={"system"}
source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic echo /edge_impulse/detection
ros2 topic echo /edge_impulse/label
ros2 topic echo /braccio/joint_command
```

`/edge_impulse/detection` carries the raw detection result as a JSON string. `/edge_impulse/label` carries the selected label, which the executor maps to a workflow. If no label arrives, or the label doesn't match an item in `edge_impulse/pick_place_workflows.yaml`, the arm doesn't move.

## Test on macOS with a ROS 2 container

To test the build and the ROS 2 graph on macOS, run the workspace in a `ros:jazzy` Docker container. On Apple silicon, the arm64 image runs natively.

> [!WARNING]
>
> Docker Desktop on macOS can't pass USB serial devices (`/dev/tty*`) into a container, so the container can't reach an arm connected over USB. This tutorial controls the arm over the network (port `8765` for the arm, `8080` for the camera), so the container can reach real hardware on your LAN. To test without hardware, publish a synthetic detection as shown below.

Start a container with the Braccio repository and `edgeimpulse_ros` mounted in the same workspace. Replace the host paths with the locations of your clones:

```bash theme={"system"}
docker run -it --name braccio-test \
  -v ~/git/ros2-docs-repos/unoq-braccio:/root/unoq-braccio \
  -v ~/git/ros2-docs-repos/edgeimpulse-ros:/root/unoq-braccio/ros2_ws/src/edgeimpulse_ros \
  -w /root/unoq-braccio/ros2_ws \
  ros:jazzy bash
```

Inside the container, install the dependencies and build into `/tmp`, so your host checkout stays clean:

```bash theme={"system"}
apt update && apt install -y \
  ros-jazzy-vision-msgs ros-jazzy-diagnostic-msgs \
  python3-opencv python3-numpy portaudio19-dev python3-pip
pip install --break-system-packages edge_impulse_linux pyaudio requests

colcon build --symlink-install --build-base /tmp/build --install-base /tmp/install
source /tmp/install/setup.bash
```

Open more shells in the container with `docker exec -it braccio-test bash`, and run `source /tmp/install/setup.bash` in each. To test `detection_label_bridge` without hardware, run it in one shell:

```bash theme={"system"}
ros2 run unoq_braccio_driver detection_label_bridge
```

Publish a synthetic detection in a second shell:

```bash theme={"system"}
ros2 topic pub --once /edgeimpulse_detector/detections vision_msgs/msg/Detection2DArray \
  '{detections: [{results: [{hypothesis: {class_id: "red_block", score: 0.92}}], bbox: {center: {position: {x: 100.0, y: 80.0}}, size_x: 40.0, size_y: 40.0}}]}'
```

Check the output in a third shell:

```bash theme={"system"}
ros2 topic echo /edge_impulse/label      # data: "red_block"
ros2 topic echo /edge_impulse/detection  # {"label":"red_block","confidence":0.92,"bbox":{...}}
```

Publish again with `score: 0.4` to check that the bridge publishes nothing below `min_confidence`. To test the full pipeline, run `pick_place_executor` with a `workflow_file` and watch `/braccio/joint_command` while you publish labels.

## Troubleshooting

| Symptom | Resolution |
| - | - |
| The camera stream doesn't open | Check that the host can reach `stream_url` and that no other process is using the camera stream. |
| `setup_edgeimpulse_ros_wsl.sh` isn't found | The script was replaced. Use `scripts/setup_edge_impulse_linux.sh`. |
| The arm doesn't respond | Check that the UNO Q remote agent is running and reachable at the configured address on port `8765`. Test a manual pose first. |
| A detection doesn't move the arm | Check `/edge_impulse/detection` and `/edge_impulse/label`. The class name must match a workflow YAML entry exactly. |
| The same object triggers a move in every frame | Raise `min_confidence`, or add an operator enable switch to the robot logic. |

## Next steps

* Retrain with images from the final camera position, lighting, and object distances.
* Replace the fixed pickup poses with positions calibrated for your table.
* Add a physical or ROS-level enable switch before you leave the arm running unattended.
* Log inference timing and decisions to check that the system responds fast enough for the motion sequence.