Object tracking

Object tracking is a post-processing layer that follows detected objects across video frames and gives each one a persistent ID. It turns per-frame bounding boxes into stable tracks, which reduces jitter and lets you count objects, measure dwell time, or trigger an event when something crosses a line. Object tracking works for models trained in Edge Impulse and models uploaded through Bring your own model (BYOM), and runs on-device in your deployment.

The Post-processing page with SORT tracking and its parameters on the left, and a drone video playing on the right where each parked car keeps its track ID as the camera moves and newly visible cars get new IDs. — Object tracking overview

How object tracking works

An object detection model looks at every frame in isolation. Object tracking adds continuity: each frame, the tracker matches new detections to the objects it is already following. A matched object keeps its ID and its track is updated. An unmatched detection starts a new track. A track that goes unmatched for too long is closed.

Choosing a tracking algorithm

Edge Impulse provides two tracking algorithms. Neither is better for every scene, so start from how your objects move:

Algorithm	Start with it when	How it matches detections
SORT tracking	Objects change speed or direction, or move less predictably, such as people walking through a store or forklifts turning in a warehouse.	Predicts each object’s next position with a Kalman filter, then assigns detections to predictions by bounding box overlap (IoU).
Linear tracking	Objects move mostly along predictable, linear paths, such as vehicles on a road or products on a conveyor belt.	Matches by bounding box overlap, or by centroid distance for centroid-based models like FOMO. Kalman filters smooth motion and bridge short missed detections.

If you’re not sure, upload a few videos from your deployment environment and render a preview with each algorithm. See tuning the tracker.

How SORT tracking worksSORT (Simple Online and Realtime Tracking) is a general-purpose tracking algorithm introduced by Bewley et al. in 2016. For each new frame, the tracker:

Predicts where each tracked object should be, using a Kalman filter that estimates its position, size, and velocity.
Matches new detections to those predictions using bounding box overlap (IoU) and an optimal assignment step. Detections only match tracks with the same label.
Updates matched tracks with the new detection, so each motion estimate keeps adapting as the object changes speed or direction.
Starts a track for every unmatched detection and closes tracks that go unmatched for longer than Max age frames.

A new track is only reported once it has been matched in Min hits consecutive frames, which filters out one-off false detections. A track is only reported in frames where it matches a detection. While it waits for a match, it keeps its ID but is not included in the output. SORT matches on position and motion alone and does not run a second neural network for appearance features, so its computational overhead stays low.

Configuring object tracking

Object tracking is available in any object detection project. You’ll find it under Post-processing in the left navigation bar.

The Studio left navigation bar with the Post-processing item highlighted under the impulse section. — Post-processing in the navigation bar

Uploading videos

To configure object tracking, upload one or more videos of representative scenes, for example a video of your production line if you want to track products on a conveyor belt. These videos do not need labels. Frames are squashed before running inference. For the most representative results, crop your video to the aspect ratio of your impulse input. For example, if your model input is 320x320, crop your video to a square before uploading. To upload videos, go to Data acquisition > Post-processing and click the upload icon. Most common video formats are supported, up to 100 MB per file.

The Data acquisition page with the Post-processing tab selected, showing three uploaded drone videos, the upload icon in the Dataset card header, and a preview of the selected video.

Tuning the tracker

Go to Post-processing in the left navigation bar. Under Object tracking, select an algorithm from the dropdown:

Simple Online and Realtime Tracking (SORT), marked as recommended in object detection projects.
Linear Tracking.
Disabled, to deploy without object tracking.

Then tune the parameters, click Save, and click Render preview to see the tracks on your post-processing videos. You can also show the raw bounding box predictions to compare the tracker’s output with your model’s. The first render takes a little longer, and the results are cached so later renders are faster.

The Post-processing page with Simple Online and Realtime Tracking (SORT) selected and its parameters on the left, and a rendered preview on the right where six parked cars each carry a track ID.

Set the parameters for the frame rate your deployed device will run at. If you tune on a 30 fps video, run your device at 30 fps too, because parameters such as Max age are counted in frames.

SORT tracking parameters

Parameter	Description
Max age	The number of frames a track is kept alive without a matching detection before the object is considered lost. Increase it to keep IDs through short occlusions or missed detections.
Min hits	The number of consecutive matched detections before a track is reported as an object. Increase it to filter out noisy, one-off detections.
IoU threshold	The minimum bounding box overlap (intersection over union) between a detection and a track’s predicted position for them to match. Lower it if fast-moving objects keep getting new IDs.
Limit objects by class	A comma-separated list of classes to track. Leave empty to track all classes.

Linear tracking parameters

Parameter	Description
Keep grace	The number of frames an object is kept after it stops being detected. This allows for brief occlusions or missed detections without ending the track.
Max. observations	The maximum number of observations used to match and stabilize a track. Higher values smooth out noise but can add latency.
Threshold	The matching threshold. For bounding box models (like MobileNetV2 SSD or YOLO-Pro), the minimum IoU between bounding boxes. For centroid-based models (like FOMO), the maximum Euclidean distance in pixels between centroids.
Limit objects by class	A comma-separated list of classes to track. Leave empty to track all classes.

For bounding box models, both IoU threshold and Threshold compare bounding box overlap the same way:

Pairs of overlapping bounding boxes at increasing IoU values, from barely touching to almost fully overlapping.

Configuring object tracking with the API

You can also select the algorithm and set its parameters with the Set post-processing block config endpoint. Set type to sort-object-tracking or object-tracking to switch algorithms. Parameters are passed as strings, for example:

{
  "enabled": true,
  "type": "sort-object-tracking",
  "parameters": {
    "maxAge": "1",
    "minHits": "3",
    "iouThreshold": "0.3",
    "limitClasses": ""
  }
}

Deploying your model

Once object tracking is configured, every deployment that contains an object detection block includes the tracker automatically, including the Linux EIM, mobile, C++ library, and WebAssembly targets.

Changing tracking parameters at runtime

You can change the tracking parameters of a deployed model without rebuilding it. The parameter names depend on the algorithm:

Algorithm	Runtime parameter names
SORT tracking	`max_age`, `min_hits`, `iou_threshold`
Linear tracking	`keep_grace`, `max_observations`, `threshold`

Linux CLI

Run edge-impulse-linux-runner --model-file <model.eim>. Click the gear icon in the web UI (http://localhost:PORT) to change the tracking parameters. To override the thresholds, start the runner with your parameters. e.g_._--thresholds 3.min_score=0.7,236.iou_threshold=0.1,236.max_age=5,236.min_hits=5

Mobile client

When you run your impulse in the Edge Impulse mobile client, click the gear icon to change the tracking parameters.

Node.js SDK

Call setLearnBlockThreshold on the runner with the ID of the post-processing block and the parameters to change. The runner’s model information lists each block’s ID and current values under thresholds.

// runner is an instance of LinuxImpulseRunner
await runner.setLearnBlockThreshold({
  id: 4, // ID of the object tracking block
  max_age: 3,
  min_hits: 3,
  iou_threshold: 0.3,
});

C++ library

The C++ library stores the tracking configuration in model-parameters/model_variables.h. With SORT tracking, it looks like this:

ei_object_tracking_sort_config_t ei_postprocessing_config_4 = {
    1,       /* implementation_version */
    1,       /* max_age */
    3,       /* min_hits */
    0.3000f, /* iou_threshold */
    true,    /* use_iou */
    nullptr, /* classes_to_track */
    0        /* classes_to_track_count */
};

With linear tracking, the struct is ei_object_tracking_config_t with keep_grace, max_observations, and threshold fields instead. To change a value at runtime, call set_threshold_postprocessing with the post-processing block, the parameter name, and the new value.

Comparing object tracking with standard object detection

To see the difference between raw and tracked bounding boxes, run a model with and without object tracking side by side:

Two side-by-side video feeds: raw detections flicker between frames on the left, while tracked boxes stay stable with persistent IDs on the right.

In one terminal, run the model with object tracking:

PORT=9200 edge-impulse-linux-runner --model-file ./model-with-object-tracking.eim

In a second terminal, run the model without object tracking:

PORT=9201 edge-impulse-linux-runner --model-file ./model-without-object-tracking.eim

Open http://localhost:9200 and http://localhost:9201 in two browser windows. The bounding boxes are smoother and more persistent with object tracking enabled.

Reading tracked objects from the inference output

C++ library

ei_impulse_result_t result;

// ... run inference ...

for (uint32_t i = 0; i < result.postprocessed_output.object_tracking_output.open_traces_count; i++) {
  ei_object_tracking_trace_t trace = result.postprocessed_output.object_tracking_output.open_traces[i];

  // Access tracked object data:
  // trace.id, trace.label, trace.x, trace.y, trace.width, trace.height
}

WebAssembly

let result = classifier.classify(/* frame or image data */);
console.log(result.object_tracking_results);

EIM files

When you read inference results from an EIM file, tracked objects are under the object_tracking field.

Advanced usage

For a more complex example, see the model cascade approach, which chains an object tracking model with an LLM:

example-llm-model-cascade-object-tracking

Troubleshooting

If you run into issues with object tracking, ask on the Edge Impulse forum or contact your solutions engineer.

OVERVIEW

PROJECTS

ORGANIZATIONS

How object tracking works

Choosing a tracking algorithm

Configuring object tracking

Uploading videos

Tuning the tracker

SORT tracking parameters

Linear tracking parameters

Configuring object tracking with the API

Deploying your model

Changing tracking parameters at runtime

Linux CLI

Mobile client

Node.js SDK

C++ library

Comparing object tracking with standard object detection

Reading tracked objects from the inference output

C++ library

WebAssembly

EIM files

Advanced usage

Troubleshooting