Bring your own model (BYOM)
Upload your own model directly into your Edge Impulse project (TensorFlow SavedModel, ONNX, or TensorFlow Lite)
Bring your own model or BYOM allows you to optimize and deploy your own pretrained model (TensorFlow SavedModel, ONNX, or TensorFlow Lite) to any edge device, directly from your Edge Impulse project.
Also make sure you have your own pretrained model available locally on your computer, in one of the following formats: TensorFlow SavedModel (
saved_model.zip), ONNX model (.onnx) or TensorFlow Lite model (.tflite)For this guide, we will be uploading a pretrained image classification TFLite model for plant disease classification, downloaded from the TensorFlow Dev Hub.
Then, from the Dashboard of your Edge Impulse project under "Getting started", select Upload your model:
Edge Impulse project dashboard.
- 1.Upload your trained model: Upload a TensorFlow SavedModel (
saved_model.zip), ONNX model (.onnx) or TensorFlow Lite model (.tflite) to get started. - 2.Model performance: Do you want performance characteristics (latency, RAM and ROM) for a specific device? Select "No" to show the performance for a range of device types, or "Yes" to run performance profiling for any of our available officially supported Edge Impulse development platforms.
Upload pretrained model - Step 1: Upload a model
After configuring the settings for uploading your model, select Upload your model and wait for your model to upload, you can check the upload status via the "Upload progress" section.
When selecting an ONNX model, you can also upload a
.npy file to Upload representative features (Optional). If you upload a set of representative features - for example, your validation set - as an .npy file we can automatically quantize this model for better on-device performance.
Uploading a pretrained .onnx model
Depending on the model you have uploaded in Step 1, the configuration settings available for Step 2 will change.
For this guide, we have selected the following configuration model settings for optimal processing for an image classification model with input shape
(300, 300, 3) in RGB format, Classification model output and 16 output labels: Tomato Healthy, Tomato Septoria Leaf Spot, Tomato Bacterial Spot, Tomato Blight, Cabbage Healthy, Tomato Spider Mite, Tomato Leaf Mold, Tomato_Yellow Leaf Curl Virus, Soy_Frogeye_Leaf_Spot, Soy_Downy_Mildew, Maize_Ravi_Corn_Rust, Maize_Healthy, Maize_Grey_Leaf_Spot, Maize_Lethal_Necrosis, Soy_Healthy, Cabbage Black RotAfter configuring your model settings, select Save model to view your model's on-device performance information for both MCUs and microprocessors (if applicable, depending on your model's arena size).
Step 2: Process your model
Optionally upload test data to ensure correct model settings and proper model processing:
Step 2: Check model behavior
Step 2: Check model behavior results
There are a couple of restrictions to converting models with our tooling:
- The model must have 1 input tensor.
- You need to have a batch dimension (e.g. if your input is an image of 320x320x3 then your input shape should be
(1,320,320,3)), and the batch size must be equal to 1. For ONNX models you can use a variable batch size (we'll set it to 1). - For classification and regression models: The model must have 1 output tensor, and the output tensor should have a batch dimension as well. E.g. a classification model with 3 classes should have output shape
(1,3).
--saved-model /tmp/saved_model does not exist:
If you encountered the following error:
Job started
Converting SavedModel...
Scheduling job in cluster...
Job started
Application exited with code 1
INFO: No representative features passed in, won't quantize this model
Extracting saved model...
Extracting saved model OK
--saved-model /tmp/saved_model does not exist
Converting SavedModel failed, see above
Job failed (see above)
Make sure to upload a
.zip archive containing at minimum a saved_model directory that contains your saved_model.pb.Could not profile: No uploaded model yet
If you encounter the following error:
Could not profile: No uploaded model yet
This often means that the model you are attempting to upload is unsupported. Only the following model formats are supported at this time:
- TensorFlow SavedModel (in .zip archive)
- ONNX (.onnx)
- TensorFlow Lite (.tflite or .lite)
This model won’t run on MCUs.
If you see a message along these lines:
This model won’t run on MCUs. Unsupported ops: Pow.
Then that means that part of your network is not supported by EON Compiler, and cannot run as-is on MCUs. A list of all supported ops can be found in our SDK; but there might be certain input/output types that are not supported even if the op is in this list (you'll get a proper error code if this is the case). If you're an enterprise customer please contact your solutions engineer, most of the times we can add the missing op within a few days.
If you want to use our built-in processing blocks to preprocess your data, for example to turn your audio files into spectrograms before training your network, then you should:
- 1.Use the code in the edgeimpulse/processing-blocks repo to preprocess your data before training your model. This contains the Python version of the processing blocks.
- 2.Then, on the embedded side, call
extract_XXXX_featuresto preprocess your sensor data, and pass the resulting features into theei_run_classifierfunction.
Here's an end-to-end example of the embedded code to preprocess using an MFCC block:
#include <stdio.h>
#include "edge-impulse-sdk/classifier/ei_run_classifier.h"
// Audio (int16)
static const int16_t features[] = {
0, 0, 1, 0, 0, 0, ...rest
};
static const float frequency = 16000.0f;
// We need the audio as float, but don't want to allocate the whole features buffer as float in one go
// as that'd double the required RAM. So dynamically convert i16->f32 when the MFCC algorithm requires.
static int input_signal_get_data(size_t offset, size_t length, float *out_ptr) {
return numpy::int16_to_float(features + offset, out_ptr, length);
}
int main(int argc, char **argv) {
// construct input signal (input is i16)
signal_t input_signal;
input_signal.total_length = sizeof(features) / sizeof(features[0]);
input_signal.get_data = &input_signal_get_data;
// construct an MFCC config (should match the params used to preprocess before training)
ei_dsp_config_mfcc_t config = {
1, // uint32_t block_id;
3, // uint16_t implementation_version;
1, // int axes;
13, // int num_cepstral;
0.032, // float frame_length;
0.024, // float frame_stride;
32, // int num_filters;
256, // int fft_length;
101, // int win_size;
300, // int low_frequency;
0, // int high_frequency;
0.98f, // float pre_cof;
1, // int pre_shift;
};
// calculate the size of the output matrix
matrix_size_t out_matrix_size =
speechpy::feature::calculate_mfcc_buffer_size(
input_signal.total_length, frequency, config.frame_length, config.frame_stride, config.num_cepstral, config.implementation_version);
printf("out_matrix_size = %u x %u\n", out_matrix_size.rows, out_matrix_size.cols);
// create a new output matrix
matrix_t out_matrix(out_matrix_size.rows, out_matrix_size.cols);
int mfcc_ret = extract_mfcc_features(&input_signal, &out_matrix, (void *)&config, frequency);
if (mfcc_ret != EIDSP_OK) {
printf("ERR: MFCC failed w/ %d", mfcc_ret);
return 1;
}
// print the features so you can check adherence to your Python results
printf("Features:\n");
for (size_t ix = 0; ix < out_matrix.rows * out_matrix.cols; ix++) {
printf("%f ", out_matrix.buffer[ix]);
}
printf("\n");
// this you'll pass into the NN
signal_t nn_signal;
ei::numpy::signal_from_buffer(out_matrix.buffer, out_matrix_size.rows * out_matrix_size.cols, &nn_signal);
// call ei_run_classifier as usual