Classify sample

Classify a complete file against the current impulse. This will move the sliding window (dependent on the sliding window length and the sliding window increase parameters in the impulse) over the complete file, and classify for every window that is extracted. Depending on the size of your file, whether your sample is resampled, and whether the result is cached you'll get either the result or a job back. If you receive a job, then wait for the completion of the job, and then call this function again to receive the results. The unoptimized (float32) model is used by default, and classification with an optimized (int8) model can be slower.

POSThttps://studio.edgeimpulse.com/v1/api/{projectId}/classify/v2/{sampleId}

Path parameters

projectId*integer

Project ID

sampleId*integer

Sample ID

Query parameters

includeDebugInfoboolean

Whether to return the debug information from FOMO classification.

variantKerasModelVariantEnum (enum)

Keras model variant

int8float32akida

impulseIdinteger

Impulse ID. If this is unset then the default impulse is used.

Response

Body

any of

success*boolean

Whether the operation succeeded

errorstring

Optional error description (set if 'success' was false)

classifications*array of ClassifySampleResponseClassification (object)

learnBlock*ImpulseLearnBlock (object)

id*integer

Identifier for this block. Make sure to up this number when creating a new block, and don't re-use identifiers. If the block hasn't changed, keep the ID as-is. ID must be unique across the project and greather than zero (>0).

type*LearnBlockType (enum)

The type of learning block (anomaly, keras, keras-transfer-image, keras-transfer-kws, keras-object-detection, keras-regression). Each behaves differently.

anomalyanomaly-gmmkeraskeras-transfer-imagekeras-transfer-kwskeras-object-detectionkeras-regressionkeras-akidakeras-akida-transfer-imagekeras-akida-object-detectionkeras-visual-anomaly

name*string

Block name, will be used in menus. If a block has a baseBlockId, this field is ignored and the base block's name is used instead.

Example: "NN Classifier"

dsp*array of integer

DSP dependencies, identified by DSP block ID

itemsinteger

Example: 27

title*string

Block title, used in the impulse UI

Example: "Classification (Keras)"

descriptionstring

A short description of the block version, displayed in the block versioning UI

Example: "Reduced learning rate and more layers"

createdBystring

The system component that created the block version (createImpulse | clone | tuner). Cannot be set via API.

Example: "createImpulse"

createdAtstring (date-time)

The datetime that the block version was created. Cannot be set via API.

result*array of object

Classification result, one item per window.

Other propertiesnumber

anomalyResultarray of AnomalyResult (object)

Anomaly scores and computed metrics for visual anomaly detection, one item per window.

boxesarray of BoundingBoxWithScore (object)

For visual anomaly detection. An array of bounding box objects, (x, y, width, height, score, label), one per detection in the image. Filtered by the minimum confidence rating of the learn block.

label*string

x*number

y*number

width*number

height*number

score*number

scoresarray of array of number

2D array of shape (n, n) with raw anomaly scores for visual anomaly detection, where n can be calculated as ((1/8 of image input size)/2 - 1). The scores corresponds to each grid cell in the image's spatial matrix.

itemsnumber

meanScorenumber

Mean value of the scores.

maxScorenumber

Maximum value of the scores.

structuredResultarray of StructuredClassifyResult (object)

Results of inferencing that returns structured data, such as object detection

boxes*array of array of number

For object detection. An array of bounding box arrays, (x, y, width, height), one per detection in the image.

itemsnumber

labelsarray of string

For object detection. An array of labels, one per detection in the image.

itemsstring

scores*array of number

For object detection. An array of probability scores, one per detection in the image.

itemsnumber

mAP*number

For object detection. A score that indicates accuracy compared to the ground truth, if available.

f1*number

For FOMO. A score that combines the precision and recall of a classifier into a single metric, if available.

precision*number

A measure of how many of the positive predictions made are correct (true positives).

recall*number

A measure of how many of the positive cases the classifier correctly predicted, over all the positive cases.

debugInfoJsonstring

Debug info in JSON format

Example:

"{\n    \"y_trues\": [\n        {\"x\": 0.854, \"y\": 0.453125, \"label\": 1},\n        {\"x\": 0.197, \"y\": 0.53125, \"label\": 2}\n    ],\n    \"y_preds\": [\n        {\"x\": 0.916, \"y\": 0.875, \"label\": 1},\n        {\"x\": 0.25, \"y\": 0.541, \"label\": 2}\n    ],\n    \"assignments\": [\n        {\"yp\": 1, \"yt\": 1, \"label\": 2, \"distance\": 0.053}\n    ],\n    \"normalised_min_distance\": 0.2,\n    \"all_pairwise_distances\": [\n        [0, 0, 0.426],\n        [1, 1, 0.053]\n    ],\n    \"unassigned_y_true_idxs\": [0],\n    \"unassigned_y_pred_idxs\": [0]\n}\n"

minimumConfidenceRating*number

The minimum confidence rating for this block. For regression, this is the absolute error (which can be larger than 1).

detailsarray of ClassifySampleResponseClassificationDetails (object)

Structured outputs and computed metrics for some model types (e.g. object detection), one item per window.

boxesarray of array of number

Bounding boxes predicted by localization model

itemsnumber

labelsarray of number

Labels predicted by localization model

itemsnumber

scoresarray of number

Scores predicted by localization model

itemsnumber

mAPnumber

For object detection, the COCO mAP computed for the predictions on this image

f1number

For FOMO, the F1 score computed for the predictions on this image

objectDetectionLastLayerObjectDetectionLastLayer (enum)

mobilenet-ssdfomoyolov2-akidayolov5yolov5v5-drpaiyoloxyolov7tao-retinanettao-ssdtao-yolov3tao-yolov4

expectedLabels*array of StructuredLabel (object)

An array with an expected label per window.

startIndex*integer

Start index of the label (e.g. 0)

endIndex*integer

End index of the label (e.g. 3). This value is inclusive, so { startIndex: 0, endIndex: 3 } covers 0, 1, 2, 3.

label*string

The label for this section.

sample*RawSampleData (object)

sample*Sample (object)

id*integer

Example: 2

filename*string

Example: "idle01.d8Ae"

signatureValidate*boolean

Whether signature validation passed

Example: true

signatureMethodstring

Example: "HS256"

signatureKeystring

Either the shared key or the public key that was used to validate the sample

created*string (date-time)

Timestamp when the sample was created on device, or if no accurate time was known on device, the time that the file was processed by the ingestion service.

lastModified*string (date-time)

Timestamp when the sample was last modified.

category*string

Example: "training"

coldstorageFilename*string

label*string

Example: "healthy-machine"

intervalMs*number

Interval between two windows (1000 / frequency). If the data was resampled, then this lists the resampled interval.

Example: 16

frequency*number

Frequency of the sample. If the data was resampled, then this lists the resampled frequency.

Example: 62.5

originalIntervalMs*number

Interval between two windows (1000 / frequency) in the source data (before resampling).

Example: 16

originalFrequency*number

Frequency of the sample in the source data (before resampling).

Example: 62.5

deviceNamestring

deviceType*string

sensors*array of Sensor (object)

name*string

Name of the axis

Example: "accX"

units*string

Type of data on this axis. Needs to comply to SenML units (see https://www.iana.org/assignments/senml/senml.xhtml).

valuesCount*integer

Number of readings in this file

totalLengthMsnumber

Total length (in ms.) of this file

added*string (date-time)

Timestamp when the sample was added to the current acquisition bucket.

boundingBoxes*array of BoundingBox (object)

label*string

x*integer

y*integer

width*integer

height*integer

boundingBoxesType*enum

object_detectionconstrained_object_detection

chartType*enum

chartimagevideotable

thumbnailVideostring

thumbnailVideoFullstring

isDisabled*boolean

True if the current sample is excluded from use

isProcessing*boolean

True if the current sample is still processing (e.g. for video)

processingJobIdinteger

Set when sample is processing and a job has picked up the request

processingError*boolean

Set when processing this sample failed

processingErrorStringstring

Error (only set when processing this sample failed)

isCropped*boolean

Whether the sample is cropped from another sample (and has crop start / end info)

metadataobject

Sample free form associated metadata

Other propertiesstring

projectId*integer

Unique identifier of the project this sample belongs to

projectOwnerNamestring

Name of the owner of the project this sample belongs to

projectNamestring

Name of the project this sample belongs to

projectLabelingMethodenum

What labeling flow the project this sample belongs to uses

single_labelobject_detection

sha256Hash*string

Data sample SHA 256 hash (including CBOR envelope if applicable)

structuredLabelsarray of StructuredLabel (object)

startIndex*integer

Start index of the label (e.g. 0)

endIndex*integer

End index of the label (e.g. 3). This value is inclusive, so { startIndex: 0, endIndex: 3 } covers 0, 1, 2, 3.

label*string

The label for this section.

structuredLabelsListarray of string

itemsstring

createdBySyntheticDataJobIdinteger

If this sample was created by a synthetic data job, it's referenced here.

imageDimensionsobject

width*integer

height*integer

payload*RawSamplePayload (object)

Sensor readings and metadata

device_namestring

Unique identifier for this device. Only set this when the device has a globally unique identifier (e.g. MAC address).

Example: "ac:87:a3:0a:2d:1b"

device_type*string

Device type, for example the exact model of the device. Should be the same for all similar devices.

Example: "DISCO-L475VG-IOT01A"

sensors*array of Sensor (object)

Array with sensor axes

name*string

Name of the axis

Example: "accX"

units*string

Type of data on this axis. Needs to comply to SenML units (see https://www.iana.org/assignments/senml/senml.xhtml).

values*array of array of number

Array of sensor values. One array item per interval, and as many items in this array as there are sensor axes. This type is returned if there are multiple axes.

itemsnumber

cropStartinteger

New start index of the cropped sample

Example: 0

cropEndinteger

New end index of the cropped sample

Example: 128

totalPayloadLength*integer

Total number of payload values

windowSizeMs*integer

Size of the sliding window (as set by the impulse) in milliseconds.

Example: 2996

windowIncreaseMs*integer

Number of milliseconds that the sliding window increased with (as set by the impulse)

Example: 10

alreadyInDatabase*boolean

Whether this sample is already in the training database

warningstring

Request

Response

Classify sample

POSThttps://studio.edgeimpulse.com/v1/api/{projectId}/classify/v2/{sampleId}

Path parameters

projectId*integer

Project ID

sampleId*integer

Sample ID

Query parameters

includeDebugInfoboolean

Whether to return the debug information from FOMO classification.

variantKerasModelVariantEnum (enum)

Keras model variant

int8float32akida

impulseIdinteger

Impulse ID. If this is unset then the default impulse is used.

Response

Body

any of

success*boolean

Whether the operation succeeded

errorstring

Optional error description (set if 'success' was false)

classifications*array of ClassifySampleResponseClassification (object)

learnBlock*ImpulseLearnBlock (object)

id*integer

type*LearnBlockType (enum)

The type of learning block (anomaly, keras, keras-transfer-image, keras-transfer-kws, keras-object-detection, keras-regression). Each behaves differently.

anomalyanomaly-gmmkeraskeras-transfer-imagekeras-transfer-kwskeras-object-detectionkeras-regressionkeras-akidakeras-akida-transfer-imagekeras-akida-object-detectionkeras-visual-anomaly

name*string

Block name, will be used in menus. If a block has a baseBlockId, this field is ignored and the base block's name is used instead.

Example: "NN Classifier"

dsp*array of integer

DSP dependencies, identified by DSP block ID

itemsinteger

Example: 27

title*string

Block title, used in the impulse UI

Example: "Classification (Keras)"

descriptionstring

A short description of the block version, displayed in the block versioning UI

Example: "Reduced learning rate and more layers"

createdBystring

The system component that created the block version (createImpulse | clone | tuner). Cannot be set via API.

Example: "createImpulse"

createdAtstring (date-time)

The datetime that the block version was created. Cannot be set via API.

result*array of object

Classification result, one item per window.

Other propertiesnumber

anomalyResultarray of AnomalyResult (object)

Anomaly scores and computed metrics for visual anomaly detection, one item per window.

boxesarray of BoundingBoxWithScore (object)

For visual anomaly detection. An array of bounding box objects, (x, y, width, height, score, label), one per detection in the image. Filtered by the minimum confidence rating of the learn block.

label*string

x*number

y*number

width*number

height*number

score*number

scoresarray of array of number

itemsnumber

meanScorenumber

Mean value of the scores.

maxScorenumber

Maximum value of the scores.

structuredResultarray of StructuredClassifyResult (object)

Results of inferencing that returns structured data, such as object detection

boxes*array of array of number

For object detection. An array of bounding box arrays, (x, y, width, height), one per detection in the image.

itemsnumber

labelsarray of string

For object detection. An array of labels, one per detection in the image.

itemsstring

scores*array of number

For object detection. An array of probability scores, one per detection in the image.

itemsnumber

mAP*number

For object detection. A score that indicates accuracy compared to the ground truth, if available.

f1*number

For FOMO. A score that combines the precision and recall of a classifier into a single metric, if available.

precision*number

A measure of how many of the positive predictions made are correct (true positives).

recall*number

A measure of how many of the positive cases the classifier correctly predicted, over all the positive cases.

debugInfoJsonstring

Debug info in JSON format

Example:

"{\n    \"y_trues\": [\n        {\"x\": 0.854, \"y\": 0.453125, \"label\": 1},\n        {\"x\": 0.197, \"y\": 0.53125, \"label\": 2}\n    ],\n    \"y_preds\": [\n        {\"x\": 0.916, \"y\": 0.875, \"label\": 1},\n        {\"x\": 0.25, \"y\": 0.541, \"label\": 2}\n    ],\n    \"assignments\": [\n        {\"yp\": 1, \"yt\": 1, \"label\": 2, \"distance\": 0.053}\n    ],\n    \"normalised_min_distance\": 0.2,\n    \"all_pairwise_distances\": [\n        [0, 0, 0.426],\n        [1, 1, 0.053]\n    ],\n    \"unassigned_y_true_idxs\": [0],\n    \"unassigned_y_pred_idxs\": [0]\n}\n"

minimumConfidenceRating*number

The minimum confidence rating for this block. For regression, this is the absolute error (which can be larger than 1).

detailsarray of ClassifySampleResponseClassificationDetails (object)

Structured outputs and computed metrics for some model types (e.g. object detection), one item per window.

boxesarray of array of number

Bounding boxes predicted by localization model

itemsnumber

labelsarray of number

Labels predicted by localization model

itemsnumber

scoresarray of number

Scores predicted by localization model

itemsnumber

mAPnumber

For object detection, the COCO mAP computed for the predictions on this image

f1number

For FOMO, the F1 score computed for the predictions on this image

objectDetectionLastLayerObjectDetectionLastLayer (enum)

mobilenet-ssdfomoyolov2-akidayolov5yolov5v5-drpaiyoloxyolov7tao-retinanettao-ssdtao-yolov3tao-yolov4

expectedLabels*array of StructuredLabel (object)

An array with an expected label per window.

startIndex*integer

Start index of the label (e.g. 0)

endIndex*integer

End index of the label (e.g. 3). This value is inclusive, so { startIndex: 0, endIndex: 3 } covers 0, 1, 2, 3.

label*string

The label for this section.

sample*RawSampleData (object)

sample*Sample (object)

id*integer

Example: 2

filename*string

Example: "idle01.d8Ae"

signatureValidate*boolean

Whether signature validation passed

Example: true

signatureMethodstring

Example: "HS256"

signatureKeystring

Either the shared key or the public key that was used to validate the sample

created*string (date-time)

Timestamp when the sample was created on device, or if no accurate time was known on device, the time that the file was processed by the ingestion service.

lastModified*string (date-time)

Timestamp when the sample was last modified.

category*string

Example: "training"

coldstorageFilename*string

label*string

Example: "healthy-machine"

intervalMs*number

Interval between two windows (1000 / frequency). If the data was resampled, then this lists the resampled interval.

Example: 16

frequency*number

Frequency of the sample. If the data was resampled, then this lists the resampled frequency.

Example: 62.5

originalIntervalMs*number

Interval between two windows (1000 / frequency) in the source data (before resampling).

Example: 16

originalFrequency*number

Frequency of the sample in the source data (before resampling).

Example: 62.5

deviceNamestring

deviceType*string

sensors*array of Sensor (object)

name*string

Name of the axis

Example: "accX"

units*string

Type of data on this axis. Needs to comply to SenML units (see https://www.iana.org/assignments/senml/senml.xhtml).

valuesCount*integer

Number of readings in this file

totalLengthMsnumber

Total length (in ms.) of this file

added*string (date-time)

Timestamp when the sample was added to the current acquisition bucket.

boundingBoxes*array of BoundingBox (object)

label*string

x*integer

y*integer

width*integer

height*integer

boundingBoxesType*enum

object_detectionconstrained_object_detection

chartType*enum

chartimagevideotable

thumbnailVideostring

thumbnailVideoFullstring

isDisabled*boolean

True if the current sample is excluded from use

isProcessing*boolean

True if the current sample is still processing (e.g. for video)

processingJobIdinteger

Set when sample is processing and a job has picked up the request

processingError*boolean

Set when processing this sample failed

processingErrorStringstring

Error (only set when processing this sample failed)

isCropped*boolean

Whether the sample is cropped from another sample (and has crop start / end info)

metadataobject

Sample free form associated metadata

Other propertiesstring

projectId*integer

Unique identifier of the project this sample belongs to

projectOwnerNamestring

Name of the owner of the project this sample belongs to

projectNamestring

Name of the project this sample belongs to

projectLabelingMethodenum

What labeling flow the project this sample belongs to uses

single_labelobject_detection

sha256Hash*string

Data sample SHA 256 hash (including CBOR envelope if applicable)

structuredLabelsarray of StructuredLabel (object)

startIndex*integer

Start index of the label (e.g. 0)

endIndex*integer

End index of the label (e.g. 3). This value is inclusive, so { startIndex: 0, endIndex: 3 } covers 0, 1, 2, 3.

label*string

The label for this section.

structuredLabelsListarray of string

itemsstring

createdBySyntheticDataJobIdinteger

If this sample was created by a synthetic data job, it's referenced here.

imageDimensionsobject

width*integer

height*integer

payload*RawSamplePayload (object)

Sensor readings and metadata

device_namestring

Unique identifier for this device. Only set this when the device has a globally unique identifier (e.g. MAC address).

Example: "ac:87:a3:0a:2d:1b"

device_type*string

Device type, for example the exact model of the device. Should be the same for all similar devices.

Example: "DISCO-L475VG-IOT01A"

sensors*array of Sensor (object)

Array with sensor axes

name*string

Name of the axis

Example: "accX"

units*string

Type of data on this axis. Needs to comply to SenML units (see https://www.iana.org/assignments/senml/senml.xhtml).

values*array of array of number

Array of sensor values. One array item per interval, and as many items in this array as there are sensor axes. This type is returned if there are multiple axes.

itemsnumber

cropStartinteger

New start index of the cropped sample

Example: 0

cropEndinteger

New end index of the cropped sample

Example: 128

totalPayloadLength*integer

Total number of payload values

windowSizeMs*integer

Size of the sliding window (as set by the impulse) in milliseconds.

Example: 2996

windowIncreaseMs*integer

Number of milliseconds that the sliding window increased with (as set by the impulse)

Example: 10

alreadyInDatabase*boolean

Whether this sample is already in the training database

warningstring

Request

Response

{
  "success": false,
  "error": "text",
  "classifications": [
    {
      "learnBlock": {
        "type": "anomaly",
        "name": "NN Classifier",
        "dsp": [
          27
        ],
        "title": "Classification (Keras)",
        "description": "Reduced learning rate and more layers",
        "createdBy": "createImpulse",
        "createdAt": "2024-11-21T15:48:34.665Z"
      },
      "result": [
        {
          "idle": 0.0002,
          "wave": 0.9998,
          "anomaly": -0.42
        }
      ],
      "anomalyResult": [
        {
          "boxes": [
            {
              "label": "text",
              "x": 0,
              "y": 0,
              "width": 0,
              "height": 0,
              "score": 0
            }
          ],
          "scores": [
            [
              0
            ]
          ],
          "meanScore": 0,
          "maxScore": 0
        }
      ],
      "structuredResult": [
        {
          "boxes": [
            [
              0
            ]
          ],
          "labels": [
            "text"
          ],
          "scores": [
            0
          ],
          "mAP": 0,
          "f1": 0,
          "precision": 0,
          "recall": 0,
          "debugInfoJson": "{\n    \"y_trues\": [\n        {\"x\": 0.854, \"y\": 0.453125, \"label\": 1},\n        {\"x\": 0.197, \"y\": 0.53125, \"label\": 2}\n    ],\n    \"y_preds\": [\n        {\"x\": 0.916, \"y\": 0.875, \"label\": 1},\n        {\"x\": 0.25, \"y\": 0.541, \"label\": 2}\n    ],\n    \"assignments\": [\n        {\"yp\": 1, \"yt\": 1, \"label\": 2, \"distance\": 0.053}\n    ],\n    \"normalised_min_distance\": 0.2,\n    \"all_pairwise_distances\": [\n        [0, 0, 0.426],\n        [1, 1, 0.053]\n    ],\n    \"unassigned_y_true_idxs\": [0],\n    \"unassigned_y_pred_idxs\": [0]\n}\n"
        }
      ],
      "minimumConfidenceRating": 0,
      "details": [
        {
          "boxes": [
            [
              0
            ]
          ],
          "labels": [
            0
          ],
          "scores": [
            0
          ],
          "mAP": 0,
          "f1": 0
        }
      ],
      "objectDetectionLastLayer": "mobilenet-ssd",
      "expectedLabels": [
        {
          "label": "text"
        }
      ]
    }
  ],
  "sample": {
    "sample": {
      "id": 2,
      "filename": "idle01.d8Ae",
      "signatureValidate": true,
      "signatureMethod": "HS256",
      "signatureKey": "text",
      "created": "2024-11-21T15:48:34.665Z",
      "lastModified": "2024-11-21T15:48:34.665Z",
      "category": "training",
      "coldstorageFilename": "text",
      "label": "healthy-machine",
      "intervalMs": 16,
      "frequency": 62.5,
      "originalIntervalMs": 16,
      "originalFrequency": 62.5,
      "deviceName": "text",
      "deviceType": "text",
      "sensors": [
        {
          "name": "accX",
          "units": "text"
        }
      ],
      "totalLengthMs": 0,
      "added": "2024-11-21T15:48:34.665Z",
      "boundingBoxes": [
        {
          "label": "text"
        }
      ],
      "boundingBoxesType": "object_detection",
      "chartType": "chart",
      "thumbnailVideo": "text",
      "thumbnailVideoFull": "text",
      "isDisabled": false,
      "isProcessing": false,
      "processingError": false,
      "processingErrorString": "text",
      "isCropped": false,
      "projectOwnerName": "text",
      "projectName": "text",
      "projectLabelingMethod": "single_label",
      "sha256Hash": "text",
      "structuredLabels": [
        {
          "label": "text"
        }
      ],
      "structuredLabelsList": [
        "text"
      ],
      "imageDimensions": {}
    },
    "payload": {
      "device_name": "ac:87:a3:0a:2d:1b",
      "device_type": "DISCO-L475VG-IOT01A",
      "sensors": [
        {
          "name": "accX",
          "units": "text"
        }
      ],
      "values": [
        [
          0
        ]
      ],
      "cropStart": 0,
      "cropEnd": 128
    }
  },
  "windowSizeMs": 2996,
  "windowIncreaseMs": 10,
  "alreadyInDatabase": false,
  "warning": "text"
}