Skip to main content

​
Run Edge Impulse Pick and Place with UNO Q Braccio

This tutorial uses the current Edge Impulse pick-and-place workflow from the UNO Q Braccio repository. In the repo as it stands today, the Ubuntu or WSL host runs the camera stream client, the Edge Impulse runner command, and the pick-and-place executor. The Arduino UNO Q remains the arm endpoint that receives joint commands over the TCP bridge. The UNO Q is the arm endpoint. ROS 2, camera access, Edge Impulse inference, and workflow logic run on the Ubuntu or WSL host. This keeps the microcontroller firmware focused on receiving arm commands while the host handles perception and decision-making.
UNO Q Braccio dashboard with named arm poses

​
What you will build

You will stream camera frames into ROS 2, run Edge Impulse inference through the repo’s edge_impulse_vision node, publish the selected label on /edge_impulse/label, and execute the matching pick-and-place sequence through the UNO Q TCP bridge.

​
Prerequisites

  • An Arduino UNO Q and a TinkerKit Braccio arm with an external 5 V servo power supply.
  • Ubuntu 24.04 with ROS 2 Jazzy, or WSL 2 with the Braccio ROS 2 workspace available.
  • A local clone of unoq-braccio.
  • A reachable camera stream for the current repo workflow, typically the UNO Q web or USB camera stream exposed as MJPEG.
  • An Edge Impulse model runner command that accepts an image path and prints one JSON result line.
  • Node.js and npm if you want to use the repository helper that installs the Edge Impulse Linux tooling.
Read Run Edge Impulse Object Detection with ROS 2 for general Edge Impulse and ROS 2 background. The default UNO Q Braccio pick-and-place path uses a repo-native runner command, but the repository also ships an optional edgeimpulse_ros backend (see Alternative: run inference with edgeimpulse_ros below).
This tutorial enables physical arm motion. Check mechanical clearance, keep people out of the arm’s range, and retain immediate access to servo power. Stock Braccio servos do not provide measured position, torque, current, or temperature feedback.

​
1. Prepare the UNO Q and arm

Flash and test the arm before using model output. For direct USB firmware flashing: 
arduino-cli lib install Braccio
arduino-cli core install arduino:zephyr
arduino-cli compile --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware
arduino-cli upload -p /dev/ttyACM0   --fqbn arduino:zephyr:unoq firmware/unoq_braccio_firmware
For the network workflow used below, run the app_lab/braccio_web_agent or app_lab/braccio_remote_agent on the UNO Q. The host connects to the UNO Q TCP server on port 8765. Before integrating vision, verify the arm can move safely through each named pose from the repository’s dashboard or manual test commands.

​
2. Build the workspace and install the Edge Impulse tooling

Build the Braccio workspace first: 
cd ~/unoq-braccio/ros2_ws
source /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install
source install/setup.bash
cd ..
If you want the helper that installs the Edge Impulse Linux SDK and CLI on the host, run the repository script: 
cd ~/unoq-braccio
bash scripts/setup_edge_impulse_linux.sh
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash
That helper checks for npm, installs edge_impulse_linux and opencv-python, and installs the Edge Impulse Linux CLI. It does not symlink edgeimpulse_ros into the workspace.
If you use Edge Impulse CLI or Linux .eim helpers, keep EDGE_IMPULSE_API_KEY in your shell environment and never commit it into scripts or repository files.

​
3. Match model labels to workflows

The default edge_impulse/pick_place_workflows.yaml supports these labels:
Object-detection labelDefault drop location
Red Block or red_blockLeft
Blue Block or blue_blockRight
Yellow Block or yellow_blockCenter
Edit that YAML file before enabling live inference. It defines the named steps, joint poses, and waits for every item. The detector label must match a supported YAML item exactly. The current flow publishes a label only when edge_impulse_vision sees a result above its min_confidence threshold. The pick_place_executor ignores new labels while a sequence is already running, so repeated detections do not interrupt an active motion sequence.

​
4. Test a sequence without vision

Start the remote bridge, then publish a label manually. This validates networking and arm poses before the camera can trigger motion. 
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py   host:=192.168.1.64 port:=8765
In a second terminal, start the executor and send a test label: 
source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 run unoq_braccio_driver pick_place_executor --ros-args   -p workflow_file:=edge_impulse/pick_place_workflows.yaml
In a third terminal, publish the test label: 
source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic pub --once /edge_impulse/label std_msgs/msg/String   "{data: 'Red Block'}"
Confirm each pose is safe for the actual arm, table, and object position. Adjust the YAML before proceeding.

​
5. Launch the current pick-and-place workflow

Start the UNO Q TCP bridge in one terminal: 
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py   host:=192.168.1.64 port:=8765
In a second terminal, start the camera stream client, the Edge Impulse vision node, and the workflow executor: 
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_pick_place.launch.py   stream_url:=http://192.168.1.64:8080/stream   runner_command:="python3 edge_impulse/runner_template.py --image {image}"   workflow_file:=edge_impulse/pick_place_workflows.yaml
Replace runner_template.py with your actual Edge Impulse model runner when it is ready. If you use app_lab/braccio_web_agent, the same UNO Q endpoint can expose both the bridge on 8765 and the camera stream on 8080.

​
Alternative: run inference with edgeimpulse_ros

The runner-command path above is the repo default. If you prefer the maintained edgeimpulse_ros package, the repository also ships edge_impulse_ros_pick_place.launch.py. It runs the edgeimpulse_ros detector on /braccio/camera/image_raw and a detection_label_bridge node that maps the detector’s vision_msgs/Detection2DArray output to the same /edge_impulse/label string the executor already consumes, so the pick-and-place workflow is unchanged. Clone and build edgeimpulse_ros into ros2_ws first, then start the arm bridge in one terminal: 
cd ~/unoq-braccio
source /opt/ros/jazzy/setup.bash
source ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup remote.launch.py host:=192.168.1.64 port:=8765
In a second terminal, start the camera stream, the edgeimpulse_ros detector, the label bridge, and the executor: 
source ~/unoq-braccio/ros2_ws/install/setup.bash

ros2 launch unoq_braccio_bringup edge_impulse_ros_pick_place.launch.py \
  stream_url:=http://192.168.1.64:8080/stream \
  model_path:=/absolute/path/to/model.eim \
  workflow_file:=edge_impulse/pick_place_workflows.yaml
The bridge publishes a label only when the top detection score clears min_confidence (default 0.65), and reads detections from detections_topic (default /edgeimpulse_detector/detections). The pick_place_executor, workflow YAML, and /edge_impulse/label contract are identical to the runner-command path, so you can switch backends without touching the arm workflow.

​
Monitor the decision path

Use three terminals to verify the inference output, selected labels, and arm commands independently: 
source ~/unoq-braccio/ros2_ws/install/setup.bash
ros2 topic echo /edge_impulse/detection
ros2 topic echo /edge_impulse/label
ros2 topic echo /braccio/joint_command
/edge_impulse/detection carries the raw detection result as a JSON string. /edge_impulse/label carries the selected label that the executor can map to a workflow. If the label is absent or does not match an item in edge_impulse/pick_place_workflows.yaml, no arm workflow is selected.

​
Test on macOS with a ROS 2 container

You do not need a Linux machine to validate the build and the ROS 2 graph. On macOS, run the workspace inside a ros:jazzy Docker container. On Apple Silicon the image runs natively (arm64), so no emulation is required.
Docker Desktop on macOS cannot pass USB serial devices (/dev/tty*) into a container, so a directly wired arm is not reachable from the container. The Braccio path in this tutorial is network-based (the UNO Q web endpoint on port 8765 and the camera stream on port 8080), so a container reaches real hardware over your LAN through those URLs. For pure wiring tests, inject a synthetic detection as shown below.
Start a container with the Braccio repository and edgeimpulse_ros mounted into the same workspace, building into /tmp so the host checkout stays clean: 
docker run -it --name braccio-test \
  -v ~/git/ros2-docs-repos/unoq-braccio:/root/unoq-braccio \
  -v ~/git/ros2-docs-repos/edgeimpulse-ros:/root/unoq-braccio/ros2_ws/src/edgeimpulse_ros \
  -w /root/unoq-braccio/ros2_ws \
  ros:jazzy bash
Inside the container, install the dependencies and build: 
apt update && apt install -y \
  ros-jazzy-vision-msgs ros-jazzy-diagnostic-msgs \
  python3-opencv python3-numpy portaudio19-dev python3-pip
pip install --break-system-packages edge_impulse_linux pyaudio requests

colcon build --symlink-install --build-base /tmp/build --install-base /tmp/install
source /tmp/install/setup.bash
Open more shells into the same container with docker exec -it braccio-test bash, then source /tmp/install/setup.bash in each. To exercise the detection_label_bridge without any hardware, run the bridge in one shell: 
ros2 run unoq_braccio_driver detection_label_bridge
Publish a synthetic detection in a second shell: 
ros2 topic pub --once /edgeimpulse_detector/detections vision_msgs/msg/Detection2DArray \
  '{detections: [{results: [{hypothesis: {class_id: "red_block", score: 0.92}}], bbox: {center: {position: {x: 100.0, y: 80.0}}, size_x: 40.0, size_y: 40.0}}]}'
Confirm the outputs in a third shell: 
ros2 topic echo /edge_impulse/label      # data: "red_block"
ros2 topic echo /edge_impulse/detection  # {"label":"red_block","confidence":0.92,"bbox":{...}}
Re-publish with score: 0.4 to confirm the bridge stays silent below its min_confidence (default 0.65). To test the full chain, run pick_place_executor with a workflow_file and watch /braccio/joint_command while publishing labels.

​
Troubleshooting

SymptomResolution
The camera stream does not openConfirm the stream_url is reachable from the host and that no other process owns the camera stream.
setup_edgeimpulse_ros_wsl.sh is not foundUse the current repository helper: scripts/setup_edge_impulse_linux.sh.
The arm does not respondVerify the UNO Q remote agent is running and reachable at the configured address on port 8765; test a manual pose first.
Detection does not cause a moveInspect /edge_impulse/detection and /edge_impulse/label. The class name must match a workflow YAML entry exactly.
The same object appears in every frameKeep the executor gated while a sequence is active, raise min_confidence, or add an explicit operator-enable gate in the robot logic.

​
Next steps

  • Retrain with images captured from the final camera position, lighting, and object distances.
  • Replace fixed pickup poses with calibrated positions for your table layout.
  • Add an explicit physical or ROS-level enable switch before operating unattended.
  • Record inference timing and decision logs to confirm the system is responsive enough for the motion sequence.