Model testing

When collecting data, we split the dataset into training and testing sets. The model was trained with only the training set, and the testing set is used to validate how well the model will perform on unseen data. This will ensure that the model has not learned to overfit the training data, which is a common occurrence.

Prerequisites

Make sure to have data samples on your test set, you can add data samples from the Data Acquisition page or the Live Classification page.

Test dataset

Model testing

To test your model, go to Model testing, and click Test all. The model will classify all of the test set samples and give you an overall accuracy of how your model performed.

Float32 vs int8 models

Currently, this page only uses the float32 version of your model. We are working on making this more flexible in the future.

classify all test images

This is also accompanied by a confusion matrix to show you how your model performs for each class and an interactive feature explorer that lets you click on a sample to easily visualize this dedicated result.

Model testing confusion matrix

The model testing data table has some quick actions available for each samples:

Model testing data table.

Limitation for anomaly detection

Make sure to label your samples exactly as anomaly or no anomaly in your test dataset so they can be used in the F1 score calculation for anomaly detection projects. We are working on making this more flexible.

Also note that the samples who does not match the known classes for the classifiers or anomaly for anomaly detection learning blocks are ignored from the accuracy or the F1 score calculation:

Ignored samples

Setting confidence threshold

Every learning block has a threshold. This can be the minimum confidence that a neural network needs to have, or the maximum anomaly score before a sample is tagged as an anomaly. You can configure these thresholds to tweak the sensitivity of these learning blocks. This affects both live classification and model testing.

Setting confidence threshold
Setting confidence threshold values

Evaluating individual samples

To see a classification in detail, go to the individual sample you are want to evaluate and click the three dots next to it, then just select show classification. This will open a new window that will display the expected outcome, and the predicted output of your model with its accuracy. This detailed view can also give you a hint on why an item has been misclassified.

Classification result. Showing the conclusions, the raw data and processed features in one overview.

Last updated