Bring your own model (BYOM)

​

Getting started

• TensorFlow SavedModel (saved_model.zip)
• ONNX model (.onnx)
• LiteRT (previously TensorFlow Lite) model (.tflite or .lite)
• XGBoost model (.json)
• LightGBM model (.txt)
• NGBoost model (.pkl)
• Generic scikit-learn-compatible model (.pkl)

​

Step 1: Upload your model

1. Upload your trained model.
2. Model performance: Do you want performance characteristics (latency, RAM and ROM) for a specific device? Select “No” to show the performance for a range of device types, or “Yes” to run performance profiling for any of our available officially supported Edge Impulse development platforms.
3. If your model is not already quantized, you can also upload a .npy file to Upload representative features (Optional), for example your validation set. This lets Edge Impulse automatically quantize the model for better on-device performance. See the quantization info callout below.

​

Step 2: Process your model

• Image
• Audio
• Other (for example, if your model takes accelerometer data as input)

• Classification (for example, if your model classifies images into different categories)
• Regression (for example, if your model predicts a continuous numerical value)
• Object detection (for example, if your model detects and localizes objects within an image)
• Freeform (for example, if your model produces outputs that do not fit into the other categories, such as image denoising, super-resolution, or pose estimation)
• Anomaly detection (for example, if your model predicts a numerical score indicating how anomalous the data is)
• Visual anomaly detection (for example, if your model predicts a 2D grid indicating how anomalous each cell of the image is)

​

Check model behavior

​

Preprocessing your data using Edge Impulse processing blocks

1. Use the code in the edgeimpulse/processing-blocks repo to preprocess your data before training your model. This contains the Python version of the processing blocks.
2. Then, on the embedded side, call extract_XXXX_features to preprocess your sensor data, and pass the resulting features into the ei_run_classifier function.

#include <stdio.h>
#include "edge-impulse-sdk/classifier/ei_run_classifier.h"

// Audio (int16)
static const int16_t features[] = {
    0, 0, 1, 0, 0, 0, ...rest
};

static const float frequency = 16000.0f;

// We need the audio as float, but don't want to allocate the whole features buffer as float in one go
// as that'd double the required RAM. So dynamically convert i16->f32 when the MFCC algorithm requires.
static int input_signal_get_data(size_t offset, size_t length, float *out_ptr) {
    return numpy::int16_to_float(features + offset, out_ptr, length);
}

int main(int argc, char **argv) {
    // construct input signal (input is i16)
    signal_t input_signal;
    input_signal.total_length = sizeof(features) / sizeof(features[0]);
    input_signal.get_data = &input_signal_get_data;

    // construct an MFCC config (should match the params used to preprocess before training)
    ei_dsp_config_mfcc_t config = {
        1, // uint32_t block_id;
        3, // uint16_t implementation_version;
        1, // int axes;
        13, // int num_cepstral;
        0.032, // float frame_length;
        0.024, // float frame_stride;
        32, // int num_filters;
        256, // int fft_length;
        101, // int win_size;
        300, // int low_frequency;
        0, // int high_frequency;
        0.98f, // float pre_cof;
        1, // int pre_shift;
    };

    // calculate the size of the output matrix
    matrix_size_t out_matrix_size =
        speechpy::feature::calculate_mfcc_buffer_size(
            input_signal.total_length, frequency, config.frame_length, config.frame_stride, config.num_cepstral, config.implementation_version);
    printf("out_matrix_size = %u x %u\n", out_matrix_size.rows, out_matrix_size.cols);

    // create a new output matrix
    matrix_t out_matrix(out_matrix_size.rows, out_matrix_size.cols);
    int mfcc_ret = extract_mfcc_features(&input_signal, &out_matrix, (void *)&config, frequency);
    if (mfcc_ret != EIDSP_OK) {
        printf("ERR: MFCC failed w/ %d", mfcc_ret);
        return 1;
    }

    // print the features so you can check adherence to your Python results
    printf("Features:\n");
    for (size_t ix = 0; ix < out_matrix.rows * out_matrix.cols; ix++) {
        printf("%f ", out_matrix.buffer[ix]);
    }
    printf("\n");

    // this you'll pass into the NN
    signal_t nn_signal;
    ei::numpy::signal_from_buffer(out_matrix.buffer, out_matrix_size.rows * out_matrix_size.cols, &nn_signal);

    // call ei_run_classifier as usual

}

​

Limitations

• The model must have 1 input tensor.
• You need to have a batch dimension. For example, if your input is an image of 320x320x3, your input shape should be (1,320,320,3), and the batch size must be equal to 1. For ONNX models, you can use a variable batch size. Edge Impulse sets it to 1.
• For classification and regression models, the model must have 1 output tensor, and the output tensor should have a batch dimension as well. For example, a classification model with 3 classes should have output shape (1,3).
• Regression models must have an output shape of (1,1).
• For freeform models, you can have multiple output tensors.
• Some operations are not supported on some devices, such as MCUs. If your model contains unsupported operations, you will see an error during the build step.

​

Troubleshooting

Job started
Converting SavedModel...
Scheduling job in cluster...
Job started
Application exited with code 1
INFO: No representative features passed in, won't quantize this model

Extracting saved model...
Extracting saved model OK

--saved-model /tmp/saved_model does not exist

Converting SavedModel failed, see above

Job failed (see above)

Could not profile: No uploaded model yet

This model won’t run on MCUs. Unsupported ops: Pow.