Your impulse is not fully trained. Failed to calculate inferencing time: ERR: Could not open file /home/model.tflite

#1

Good morning,
I implemented a custom learning bloc and i pushed it to edge impulse.
i trained it and i get the following result with an error in the inference step. i cant also deploy it.
project id:532911 (impulse#13)

do you have any idea about that, please?

thanks an advance
Training model on 822 inputs…
Training model OK

Training model OK
Training model OK
Mean accuracy (testing set):
0.8053088803088801
Saving model…
0.8053088803088801
Saving model…
0.8053088803088801
Saving model…
Saving model OK
Saving model OK
Saving model…
Saving model OK
Saving model OK
Profiling model…
:heavy_check_mark: Job scheduled at 24 Nov 2024 20:16:29
:heavy_check_mark: Job started at 24 Nov 2024 20:16:29
:heavy_check_mark: Job scheduled at 24 Nov 2024 20:16:29
[spinner-done] Job started at 24 Nov 2024 20:16:29
Loading data for profiling…
Loading data for profiling OK
Loading data for profiling OK
Calculating performance metrics…
Calculating inferencing time…
Failed to calculate inferencing time: ERR: Could not open file /home/model.tflite

Model training complete
Model training complete

Job completed (success)

#2

Hi @aydi - based on the terminal output, it looks like the model is training and saving fine but that the model can’t be found for the profiling step.

I don’t have access to your custom code. If I did, here are some things that I would check to begin with:

  • Are you saving the output of your model as either the TFLite, SavedModel, or ONNX format?
  • Did you take the --output-directory argument as input?
  • Did you check that the output directory existed, and if not, create it?
  • Did you save the model to the default directory, e.g. save_saved_model(model, args.out_directory)
#3

hi@brianmcfadden -
My custom code use resnet50 to get features .the output of resnet50 is used to train oneclasssvm. the generated model is .pkl (because oneclasssvm is sckitlearn lib).

  • Did you save the model to the default directory, e.g. save_saved_model(model, args.out_directory) ? . yes i saved it as .pkl model (with open(os.path.join(args.out_directory, ‘modelocsvm.pkl’),‘wb’)). i followed the Custom scikit-learn ML block example for Edge Impulse example to generate the .pkl model using docker.
    do i have seperate the resnet50 and the oneclasssvm each one in each custom code?
    thanks an advance
#4

Thanks for the additional info, @aydi. I am checking on a couple of things with the team. Once I have some clarification, I’ll get back to you!

#5

OK, so there are a few things going on here, @aydi. Thanks for clarifying what you are trying to do and that you are saving a .pkl file.

First, there is some history with the example-custom-ml-block-scikit repo. This example was originally developed before we did automatic .pkl to .tflite conversion and it demonstrated how to do this by using a JAX inference function. The repo has since been updated to leverage the automatic .pkl to .tflite conversion, yet that conversion only supports specific types of scikit-learn models. Beyond that, the supported models have been incorporated directly into the Edge Impulse platform (for Pro and Enterprise plan customers). The list of models can be found in Supported classical ML algorithms (however, I can see that doc needs an update as we support more algos now!).

Next, even if you are using a supported scikit-learn model, I think you will run into issues trying to stack a ResNet-50 model and scikit-learn model into a single learning block. One way to handle this is to use a custom processing block that produces the features using the ResNet-50 model and passes that to the scikit-learn model in the learning block. We have a Sensor fusion using embeddings tutorial that describes this process.

I’ve identified a couple items that need updating:

  • The example repo to describe the limitations with automatic .pkl to .tflite conversion
  • The list of supported classical ML models in our docs