Run Edge Impulse pick and place with UNO Q Braccio

This tutorial sorts colored blocks with a TinkerKit Braccio arm, using the pick-and-place workflow from the UNO Q Braccio repository. An Ubuntu or WSL host runs ROS 2, the camera stream client, Edge Impulse inference, and the pick-and-place executor. The Arduino UNO Q only receives joint commands over a TCP bridge and moves the arm.

UNO Q Braccio dashboard with named arm poses — UNO Q Braccio control dashboard and named arm poses

What you’ll build

A ROS 2 pipeline that streams camera frames, runs Edge Impulse inference in the repository’s edge_impulse_vision node, publishes the detected label on /edge_impulse/label, and runs the matching pick-and-place sequence on the arm.

Prerequisites

An Arduino UNO Q and a TinkerKit Braccio arm with an external 5 V servo power supply
Ubuntu 24.04 with ROS 2 Jazzy, or WSL 2
A local clone of unoq-braccio
An MJPEG camera stream that the host can reach, usually from the UNO Q web or USB camera
An Edge Impulse model runner command that takes an image path and prints one line of JSON
Node.js and npm, if you use the repository script that installs the Edge Impulse Linux tools

By default, the workflow runs inference with a runner command. The repository can also use edgeimpulse_ros, as described in Alternative: run inference with edgeimpulse_ros. For background on edgeimpulse_ros, see Run Edge Impulse object detection with ROS 2.

This tutorial moves a physical arm. Check the clearance around it, keep people out of its reach, and keep the servo power switch within reach. Stock Braccio servos don’t report position, torque, current, or temperature.

1. Prepare the UNO Q and arm

Flash and test the arm before you connect it to a model. To flash the firmware over USB:

arduino-cli lib install Braccio
arduino-cli core install arduino:zephyr
arduino-cli compile --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware
arduino-cli upload -p /dev/ttyACM0 --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware

This tutorial controls the arm over the network, so run app_lab/braccio_web_agent or app_lab/braccio_remote_agent on the UNO Q. The host connects to the UNO Q TCP server on port 8765. Before you add vision, check that the arm moves safely through each named pose, using the repository’s dashboard or manual test commands.

2. Build the workspace and install the Edge Impulse tooling

Build the Braccio workspace first:

cd ~/unoq-braccio/ros2_ws
source /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install
source install/setup.bash
cd ..

To install the Edge Impulse Linux SDK and CLI on the host, run the repository’s setup script:

cd ~/unoq-braccio
bash scripts/setup_edge_impulse_linux.sh
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

The script checks for npm and installs edge_impulse_linux, opencv-python, and the Edge Impulse Linux CLI. It doesn’t add edgeimpulse_ros to the workspace.

If the Edge Impulse CLI or .eim tools need EDGE_IMPULSE_API_KEY, set it in your shell environment. Never commit it to scripts or repository files.

3. Match model labels to workflows

The default edge_impulse/pick_place_workflows.yaml supports these labels:

Object detection label	Default drop location
`Red Block` or `red_block`	Left
`Blue Block` or `blue_block`	Right
`Yellow Block` or `yellow_block`	Center

The YAML file defines the steps, joint poses, and waits for each item. Edit it before you turn on live inference. Each detector label must match a YAML item exactly. edge_impulse_vision publishes a label only when a result is above its min_confidence threshold. pick_place_executor ignores new labels while a sequence runs, so repeated detections don’t interrupt a motion.

4. Test a sequence without vision

To test the network connection and arm poses before the camera can trigger motion, publish a label by hand. Start the remote bridge:

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765

In a second terminal, start the executor:

source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 run unoq_braccio_driver pick_place_executor --ros-args -p workflow_file:=edge_impulse/pick_place_workflows.yaml

In a third terminal, publish the test label:

source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic pub --once /edge_impulse/label std_msgs/msg/String "{data: 'Red Block'}"

Check that each pose is safe for your arm, table, and object position, and adjust the YAML if it isn’t.

5. Launch the pick-and-place workflow

Start the UNO Q TCP bridge in one terminal:

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765

In a second terminal, start the camera stream client, the Edge Impulse vision node, and the workflow executor:

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_pick_place.launch.py \
  stream_url:=http://192.168.1.64:8080/stream \
  runner_command:="python3 edge_impulse/runner_template.py --image {image}" \
  workflow_file:=edge_impulse/pick_place_workflows.yaml

Replace runner_template.py with your Edge Impulse model runner. app_lab/braccio_web_agent serves both the bridge on port 8765 and the camera stream on port 8080.

Alternative: run inference with `edgeimpulse_ros`

Instead of the runner command, you can run inference with the edgeimpulse_ros package through edge_impulse_ros_pick_place.launch.py. This launch file runs the edgeimpulse_ros detector on /braccio/camera/image_raw. A detection_label_bridge node converts the detector’s vision_msgs/Detection2DArray output to the /edge_impulse/label string that the executor reads, so the pick-and-place workflow doesn’t change. Run this pipeline on a separate machine, or on the UNO Q in a Docker container.

External host or VM
On the UNO Q (Docker)

Clone and build edgeimpulse_ros into ros2_ws first, then start the arm bridge in one terminal:

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765

In a second terminal, start the camera stream, the edgeimpulse_ros detector, the label bridge, and the executor:

source ~/unoq-braccio/ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_ros_pick_place.launch.py \
  stream_url:=http://192.168.1.64:8080/stream \
  model_path:=/absolute/path/to/model.eim \
  workflow_file:=edge_impulse/pick_place_workflows.yaml

The 4 GB UNO Q can run the whole edgeimpulse_ros pipeline in a Docker container on its App Lab Linux OS. The container runs the ROS 2 graph, and the App Lab agents keep serving the camera MJPEG stream on 127.0.0.1:8080 and arm control on 127.0.0.1:8765.The repository’s Dockerfile and docker-compose.yml use network_mode: host, so the container reaches the agents on localhost and DDS discovery works.

Install Docker and free some space

Install Docker Engine on the UNO Q’s Debian-based App Lab OS. The arm64 ros:jazzy image runs natively on the Qualcomm Dragonwing Cortex-A53 cores.

curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER   # log out and back in for this to apply

The UNO Q has about 10 GB of eMMC storage. If it’s nearly full, free some space before you build:

docker system prune -af   # remove unused images and containers
df -h /                   # aim for at least 3 GB free

Start the arm and camera agent

In Arduino App Lab, run the braccio_web_agent app on the UNO Q. It serves arm control over TCP on port 8765 and the camera MJPEG stream on port 8080. Check that it responds on localhost:

curl -s http://127.0.0.1:8080/camera-status
curl -s http://127.0.0.1:8080/camera-devices   # confirm the right /dev/videoN

Clone the repository

cd ~
git clone -b unoq-ros2-docker-braccio https://github.com/edgeimpulse/unoq-braccio.git
cd unoq-braccio

Download your model

Download the model on the UNO Q, so you get an aarch64 .eim file. An x86 build won’t run on the UNO Q:

sudo npm install -g edge-impulse-linux --force
mkdir -p models
edge-impulse-linux-runner --download models/model.eim   # log in when prompted

Check that the model’s labels match the keys in edge_impulse/pick_place_workflows.yaml.

Build and run on-device

docker compose up --build

The container runs onboard_edge_impulse_pick_place.launch.py, which starts tcp_bridge (host:=127.0.0.1), mjpeg_camera_node (stream_url:=http://127.0.0.1:8080/stream), the edgeimpulse_ros detector, detection_label_bridge, and pick_place_executor in one container.

Verify the pipeline

In a second shell, watch the labels and joint commands inside the container:

docker exec -it unoq_braccio_edgeimpulse_ros bash -lc '
  source /braccio_ws/install/setup.bash
  ros2 topic echo /edge_impulse/label &
  ros2 topic echo /braccio/joint_command'

Place an object in front of the camera. When a detection scores above min_confidence (0.65), a label and a joint command appear and the arm moves.

The Braccio packages and edgeimpulse_ros are pure ament_python, so the colcon build in the container is small. It runs sequentially (--executor sequential --parallel-workers 1) to fit in the memory of the 2 GB and 4 GB UNO Q. If the build runs out of memory, add swap: sudo fallocate -l 2G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile.

detection_label_bridge reads detections from detections_topic (default /edgeimpulse_detector/detections) and publishes a label only when the top score is above min_confidence (default 0.65). The executor, workflow YAML, and /edge_impulse/label topic are the same for both pipelines, so you can switch between them without changing the arm workflow.

Monitor the decision path

To check the inference output, selected labels, and arm commands, run each of these in its own terminal:

source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic echo /edge_impulse/detection
ros2 topic echo /edge_impulse/label
ros2 topic echo /braccio/joint_command

/edge_impulse/detection carries the raw detection result as a JSON string. /edge_impulse/label carries the selected label, which the executor maps to a workflow. If no label arrives, or the label doesn’t match an item in edge_impulse/pick_place_workflows.yaml, the arm doesn’t move.

Test on macOS with a ROS 2 container

To test the build and the ROS 2 graph on macOS, run the workspace in a ros:jazzy Docker container. On Apple silicon, the arm64 image runs natively.

Docker Desktop on macOS can’t pass USB serial devices (/dev/tty*) into a container, so the container can’t reach an arm connected over USB. This tutorial controls the arm over the network (port 8765 for the arm, 8080 for the camera), so the container can reach real hardware on your LAN. To test without hardware, publish a synthetic detection as shown below.

Start a container with the Braccio repository and edgeimpulse_ros mounted in the same workspace. Replace the host paths with the locations of your clones:

docker run -it --name braccio-test \
  -v ~/git/ros2-docs-repos/unoq-braccio:/root/unoq-braccio \
  -v ~/git/ros2-docs-repos/edgeimpulse-ros:/root/unoq-braccio/ros2_ws/src/edgeimpulse_ros \
  -w /root/unoq-braccio/ros2_ws \
  ros:jazzy bash

Inside the container, install the dependencies and build into /tmp, so your host checkout stays clean:

apt update && apt install -y \
  ros-jazzy-vision-msgs ros-jazzy-diagnostic-msgs \
  python3-opencv python3-numpy portaudio19-dev python3-pip
pip install --break-system-packages edge_impulse_linux pyaudio requests

colcon build --symlink-install --build-base /tmp/build --install-base /tmp/install
source /tmp/install/setup.bash

Open more shells in the container with docker exec -it braccio-test bash, and run source /tmp/install/setup.bash in each. To test detection_label_bridge without hardware, run it in one shell:

ros2 run unoq_braccio_driver detection_label_bridge

Publish a synthetic detection in a second shell:

ros2 topic pub --once /edgeimpulse_detector/detections vision_msgs/msg/Detection2DArray \
  '{detections: [{results: [{hypothesis: {class_id: "red_block", score: 0.92}}], bbox: {center: {position: {x: 100.0, y: 80.0}}, size_x: 40.0, size_y: 40.0}}]}'

Check the output in a third shell:

ros2 topic echo /edge_impulse/label      # data: "red_block"
ros2 topic echo /edge_impulse/detection  # {"label":"red_block","confidence":0.92,"bbox":{...}}

Publish again with score: 0.4 to check that the bridge publishes nothing below min_confidence. To test the full pipeline, run pick_place_executor with a workflow_file and watch /braccio/joint_command while you publish labels.

Troubleshooting

Symptom	Resolution
The camera stream doesn’t open	Check that the host can reach `stream_url` and that no other process is using the camera stream.
`setup_edgeimpulse_ros_wsl.sh` isn’t found	The script was replaced. Use `scripts/setup_edge_impulse_linux.sh`.
The arm doesn’t respond	Check that the UNO Q remote agent is running and reachable at the configured address on port `8765`. Test a manual pose first.
A detection doesn’t move the arm	Check `/edge_impulse/detection` and `/edge_impulse/label`. The class name must match a workflow YAML entry exactly.
The same object triggers a move in every frame	Raise `min_confidence`, or add an operator enable switch to the robot logic.

Next steps

Retrain with images from the final camera position, lighting, and object distances.
Replace the fixed pickup poses with positions calibrated for your table.
Add a physical or ROS-level enable switch before you leave the arm running unattended.
Log inference timing and decisions to check that the system responds fast enough for the motion sequence.

OVERVIEW

END-TO-END

TOPICS

TOOLS

INTEGRATIONS

Run Edge Impulse pick and place with UNO Q Braccio

What you’ll build

Prerequisites

1. Prepare the UNO Q and arm

2. Build the workspace and install the Edge Impulse tooling

3. Match model labels to workflows

4. Test a sequence without vision

5. Launch the pick-and-place workflow

Alternative: run inference with `edgeimpulse_ros`

Monitor the decision path

Test on macOS with a ROS 2 container

Troubleshooting

Next steps

​What you’ll build

​Prerequisites

​1. Prepare the UNO Q and arm

​2. Build the workspace and install the Edge Impulse tooling

​3. Match model labels to workflows

​4. Test a sequence without vision

​5. Launch the pick-and-place workflow

​Alternative: run inference with edgeimpulse_ros

​Monitor the decision path

​Test on macOS with a ROS 2 container

​Troubleshooting

​Next steps

What you’ll build

Prerequisites

1. Prepare the UNO Q and arm

2. Build the workspace and install the Edge Impulse tooling

3. Match model labels to workflows

4. Test a sequence without vision

5. Launch the pick-and-place workflow

Alternative: run inference with `edgeimpulse_ros`

Monitor the decision path

Test on macOS with a ROS 2 container

Troubleshooting

Next steps