Run Edge Impulse Pick and Place with UNO Q Braccio

This tutorial uses the current Edge Impulse pick-and-place workflow from the UNO Q Braccio repository. In the repo as it stands today, the Ubuntu or WSL host runs the camera stream client, the Edge Impulse runner command, and the pick-and-place executor. The Arduino UNO Q remains the arm endpoint that receives joint commands over the TCP bridge. The UNO Q is the arm endpoint. ROS 2, camera access, Edge Impulse inference, and workflow logic run on the Ubuntu or WSL host. This keeps the microcontroller firmware focused on receiving arm commands while the host handles perception and decision-making.

UNO Q Braccio dashboard with named arm poses — UNO Q Braccio control dashboard and named arm poses

What you will build

You will stream camera frames into ROS 2, run Edge Impulse inference through the repo’s edge_impulse_vision node, publish the selected label on /edge_impulse/label, and execute the matching pick-and-place sequence through the UNO Q TCP bridge.

Prerequisites

An Arduino UNO Q and a TinkerKit Braccio arm with an external 5 V servo power supply.
Ubuntu 24.04 with ROS 2 Jazzy, or WSL 2 with the Braccio ROS 2 workspace available.
A local clone of unoq-braccio.
A reachable camera stream for the current repo workflow, typically the UNO Q web or USB camera stream exposed as MJPEG.
An Edge Impulse model runner command that accepts an image path and prints one JSON result line.
Node.js and npm if you want to use the repository helper that installs the Edge Impulse Linux tooling.

Read Run Edge Impulse Object Detection with ROS 2 for general Edge Impulse and ROS 2 background. The default UNO Q Braccio pick-and-place path uses a repo-native runner command, but the repository also ships an optional edgeimpulse_ros backend (see Alternative: run inference with edgeimpulse_ros below).

This tutorial enables physical arm motion. Check mechanical clearance, keep people out of the arm’s range, and retain immediate access to servo power. Stock Braccio servos do not provide measured position, torque, current, or temperature feedback.

1. Prepare the UNO Q and arm

Flash and test the arm before using model output. For direct USB firmware flashing:

arduino-cli lib install Braccio
arduino-cli core install arduino:zephyr
arduino-cli compile --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware
arduino-cli upload -p /dev/ttyACM0   --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware

For the network workflow used below, run the app_lab/braccio_web_agent or app_lab/braccio_remote_agent on the UNO Q. The host connects to the UNO Q TCP server on port 8765. Before integrating vision, verify the arm can move safely through each named pose from the repository’s dashboard or manual test commands.

2. Build the workspace and install the Edge Impulse tooling

Build the Braccio workspace first:

cd ~/unoq-braccio/ros2_ws
source /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install
source install/setup.bash
cd ..

If you want the helper that installs the Edge Impulse Linux SDK and CLI on the host, run the repository script:

cd ~/unoq-braccio
bash scripts/setup_edge_impulse_linux.sh
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

That helper checks for npm, installs edge_impulse_linux and opencv-python, and installs the Edge Impulse Linux CLI. It does not symlink edgeimpulse_ros into the workspace.

If you use Edge Impulse CLI or Linux .eim helpers, keep EDGE_IMPULSE_API_KEY in your shell environment and never commit it into scripts or repository files.

3. Match model labels to workflows

The default edge_impulse/pick_place_workflows.yaml supports these labels:

Object-detection label	Default drop location
`Red Block` or `red_block`	Left
`Blue Block` or `blue_block`	Right
`Yellow Block` or `yellow_block`	Center

Edit that YAML file before enabling live inference. It defines the named steps, joint poses, and waits for every item. The detector label must match a supported YAML item exactly. The current flow publishes a label only when edge_impulse_vision sees a result above its min_confidence threshold. The pick_place_executor ignores new labels while a sequence is already running, so repeated detections do not interrupt an active motion sequence.

4. Test a sequence without vision

Start the remote bridge, then publish a label manually. This validates networking and arm poses before the camera can trigger motion.

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py   host:=192.168.1.64 port:=8765

In a second terminal, start the executor and send a test label:

source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 run unoq_braccio_driver pick_place_executor --ros-args   -p workflow_file:=edge_impulse/pick_place_workflows.yaml

In a third terminal, publish the test label:

source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic pub --once /edge_impulse/label std_msgs/msg/String   "{data: 'Red Block'}"

Confirm each pose is safe for the actual arm, table, and object position. Adjust the YAML before proceeding.

5. Launch the current pick-and-place workflow

Start the UNO Q TCP bridge in one terminal:

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py   host:=192.168.1.64 port:=8765

In a second terminal, start the camera stream client, the Edge Impulse vision node, and the workflow executor:

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_pick_place.launch.py   stream_url:=http://192.168.1.64:8080/stream   runner_command:="python3 edge_impulse/runner_template.py --image {image}"   workflow_file:=edge_impulse/pick_place_workflows.yaml

Replace runner_template.py with your actual Edge Impulse model runner when it is ready. If you use app_lab/braccio_web_agent, the same UNO Q endpoint can expose both the bridge on 8765 and the camera stream on 8080.

Alternative: run inference with `edgeimpulse_ros`

The runner-command path above is the repo default. If you prefer the maintained edgeimpulse_ros package, the repository also ships edge_impulse_ros_pick_place.launch.py. It runs the edgeimpulse_ros detector on /braccio/camera/image_raw and a detection_label_bridge node that maps the detector’s vision_msgs/Detection2DArray output to the same /edge_impulse/label string the executor already consumes, so the pick-and-place workflow is unchanged. You can run this backend from an external machine, or entirely on the UNO Q in a Docker container.

External host or VM
On the UNO Q (Docker)

Clone and build edgeimpulse_ros into ros2_ws first, then start the arm bridge in one terminal:

cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765

In a second terminal, start the camera stream, the edgeimpulse_ros detector, the label bridge, and the executor:

source ~/unoq-braccio/ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_ros_pick_place.launch.py \
  stream_url:=http://192.168.1.64:8080/stream \
  model_path:=/absolute/path/to/model.eim \
  workflow_file:=edge_impulse/pick_place_workflows.yaml

You don’t need a separate host or VM: the 4 GB UNO Q can run the whole edgeimpulse_ros pipeline in a Docker container on its own App Lab Linux OS. The container runs the ROS 2 graph, while the UNO Q’s App Lab agents keep serving the camera MJPEG stream (127.0.0.1:8080) and arm control (127.0.0.1:8765).The repo root ships a Dockerfile and docker-compose.yml that use network_mode: host, so localhost reaches those agents and DDS discovery works. (This is why on-device Docker is simpler than Docker Desktop on macOS, where host networking is unavailable.)

Install Docker and free some space

Install Docker Engine on the UNO Q’s Debian-based App Lab OS (arm64). The ros:jazzy image runs natively on the Qualcomm Dragonwing (Cortex-A53), so no emulation is needed.

curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER   # log out and back in for this to apply

The UNO Q’s eMMC is small (~10 GB). If it is nearly full, reclaim space before building:

docker system prune -af   # remove unused images and containers
df -h /                   # aim for at least ~3 GB free

Start the arm and camera agent

In Arduino App Lab, run the braccio_web_agent app on the UNO Q. It serves TCP arm control on :8765 and the camera MJPEG stream on :8080. Confirm it is reachable over localhost:

curl -s http://127.0.0.1:8080/camera-status
curl -s http://127.0.0.1:8080/camera-devices   # confirm the right /dev/videoN

Clone the repository

cd ~
git clone -b unoq-ros2-docker-braccio https://github.com/edgeimpulse/unoq-braccio.git
cd unoq-braccio

Download your model

Fetch the aarch64 .eim on-device so it matches the UNO Q’s Arm target — a model built for x86 will not run here:

sudo npm install -g edge-impulse-linux --force
mkdir -p models
edge-impulse-linux-runner --download models/model.eim   # log in when prompted

Check that your model’s labels match the keys in edge_impulse/pick_place_workflows.yaml.

Build and run on-device

docker compose up --build

The default command launches onboard_edge_impulse_pick_place.launch.py, which starts the tcp_bridge (host:=127.0.0.1), mjpeg_camera_node (stream_url:=http://127.0.0.1:8080/stream), the edgeimpulse_ros detector, detection_label_bridge, and pick_place_executor in one container.

Verify the pipeline

In a second shell, watch the decision path inside the container:

docker exec -it unoq_braccio_edgeimpulse_ros bash -lc '
  source /braccio_ws/install/setup.bash
  ros2 topic echo /edge_impulse/label &
  ros2 topic echo /braccio/joint_command'

Place an object in front of the camera. When a detection clears min_confidence (0.65) you will see a label, a joint command, and the Braccio move.

The Braccio packages and edgeimpulse_ros are pure ament_python, so the in-container colcon build is light. It is pinned to a sequential executor (--executor sequential --parallel-workers 1) to stay within RAM on the 2 GB / 4 GB UNO Q variants. If the build is killed for memory, add swap first: sudo fallocate -l 2G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile.

The bridge publishes a label only when the top detection score clears min_confidence (default 0.65), and reads detections from detections_topic (default /edgeimpulse_detector/detections). The pick_place_executor, workflow YAML, and /edge_impulse/label contract are identical to the runner-command path, so you can switch backends without touching the arm workflow.

Monitor the decision path

Use three terminals to verify the inference output, selected labels, and arm commands independently:

source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic echo /edge_impulse/detection
ros2 topic echo /edge_impulse/label
ros2 topic echo /braccio/joint_command

/edge_impulse/detection carries the raw detection result as a JSON string. /edge_impulse/label carries the selected label that the executor can map to a workflow. If the label is absent or does not match an item in edge_impulse/pick_place_workflows.yaml, no arm workflow is selected.

Test on macOS with a ROS 2 container

You do not need a Linux machine to validate the build and the ROS 2 graph. On macOS, run the workspace inside a ros:jazzy Docker container. On Apple Silicon the image runs natively (arm64), so no emulation is required.

Docker Desktop on macOS cannot pass USB serial devices (/dev/tty*) into a container, so a directly wired arm is not reachable from the container. The Braccio path in this tutorial is network-based (the UNO Q web endpoint on port 8765 and the camera stream on port 8080), so a container reaches real hardware over your LAN through those URLs. For pure wiring tests, inject a synthetic detection as shown below.

Start a container with the Braccio repository and edgeimpulse_ros mounted into the same workspace, building into /tmp so the host checkout stays clean:

docker run -it --name braccio-test \
  -v ~/git/ros2-docs-repos/unoq-braccio:/root/unoq-braccio \
  -v ~/git/ros2-docs-repos/edgeimpulse-ros:/root/unoq-braccio/ros2_ws/src/edgeimpulse_ros \
  -w /root/unoq-braccio/ros2_ws \
  ros:jazzy bash

Inside the container, install the dependencies and build:

apt update && apt install -y \
  ros-jazzy-vision-msgs ros-jazzy-diagnostic-msgs \
  python3-opencv python3-numpy portaudio19-dev python3-pip
pip install --break-system-packages edge_impulse_linux pyaudio requests

colcon build --symlink-install --build-base /tmp/build --install-base /tmp/install
source /tmp/install/setup.bash

Open more shells into the same container with docker exec -it braccio-test bash, then source /tmp/install/setup.bash in each. To exercise the detection_label_bridge without any hardware, run the bridge in one shell:

ros2 run unoq_braccio_driver detection_label_bridge

Publish a synthetic detection in a second shell:

ros2 topic pub --once /edgeimpulse_detector/detections vision_msgs/msg/Detection2DArray \
  '{detections: [{results: [{hypothesis: {class_id: "red_block", score: 0.92}}], bbox: {center: {position: {x: 100.0, y: 80.0}}, size_x: 40.0, size_y: 40.0}}]}'

Confirm the outputs in a third shell:

ros2 topic echo /edge_impulse/label      # data: "red_block"
ros2 topic echo /edge_impulse/detection  # {"label":"red_block","confidence":0.92,"bbox":{...}}

Re-publish with score: 0.4 to confirm the bridge stays silent below its min_confidence (default 0.65). To test the full chain, run pick_place_executor with a workflow_file and watch /braccio/joint_command while publishing labels.

Troubleshooting

Symptom	Resolution
The camera stream does not open	Confirm the `stream_url` is reachable from the host and that no other process owns the camera stream.
`setup_edgeimpulse_ros_wsl.sh` is not found	Use the current repository helper: `scripts/setup_edge_impulse_linux.sh`.
The arm does not respond	Verify the UNO Q remote agent is running and reachable at the configured address on port `8765`; test a manual pose first.
Detection does not cause a move	Inspect `/edge_impulse/detection` and `/edge_impulse/label`. The class name must match a workflow YAML entry exactly.
The same object appears in every frame	Keep the executor gated while a sequence is active, raise `min_confidence`, or add an explicit operator-enable gate in the robot logic.

Next steps

Retrain with images captured from the final camera position, lighting, and object distances.
Replace fixed pickup poses with calibrated positions for your table layout.
Add an explicit physical or ROS-level enable switch before operating unattended.
Record inference timing and decision logs to confirm the system is responsive enough for the motion sequence.

OVERVIEW

END-TO-END

TOPICS

TOOLS

INTEGRATIONS

Run Edge Impulse Pick and Place with UNO Q Braccio

Run Edge Impulse Pick and Place with UNO Q Braccio

What you will build

Prerequisites

1. Prepare the UNO Q and arm

2. Build the workspace and install the Edge Impulse tooling

3. Match model labels to workflows

4. Test a sequence without vision

5. Launch the current pick-and-place workflow

Alternative: run inference with `edgeimpulse_ros`

Monitor the decision path

Test on macOS with a ROS 2 container

Troubleshooting

Next steps

​Run Edge Impulse Pick and Place with UNO Q Braccio

​What you will build

​Prerequisites

​1. Prepare the UNO Q and arm

​2. Build the workspace and install the Edge Impulse tooling

​3. Match model labels to workflows

​4. Test a sequence without vision

​5. Launch the current pick-and-place workflow

​Alternative: run inference with edgeimpulse_ros

​Monitor the decision path

​Test on macOS with a ROS 2 container

​Troubleshooting

​Next steps

Run Edge Impulse Pick and Place with UNO Q Braccio

What you will build

Prerequisites

1. Prepare the UNO Q and arm

2. Build the workspace and install the Edge Impulse tooling

3. Match model labels to workflows

4. Test a sequence without vision

5. Launch the current pick-and-place workflow

Alternative: run inference with `edgeimpulse_ros`

Monitor the decision path

Test on macOS with a ROS 2 container

Troubleshooting

Next steps