Python SDK with ONNX model

Help
#1

Question/Issue:

Project ID:

Context/Use case:
I am trying to use the Python SDK to make a profiling of an Onnx model. When I use ei.model.profile() function, I got this error:

Could not profile: Was unable to load_model of type <class ‘onnx.onnx_ml_pb2.ModelProto’> with exception Unexpected model type

1 Like
#2

Hi @Bouhmid,

ModelProto is a container that is not supported in the Python SDK–it only supports loading .onnx files. You need to save your ONNX model as a file first (e.g. my_model.onnx) before using it with the Python SDK. For example, if my_model is a ModelProto object in ONNX, you need to save it first with:

onnx.save(my_model, "my_model.onnx")

Then you can call:

profile = ei.model.profile(model=my_model.onnx, device='cortex-m4f-80mhz')

Hope that helps!

1 Like
#3

I saved my model in an ONNX file and then pass it to the profile function:

    profile = ei.model.profile(model= './my_model.onnx',
                               device='cortex-m4f-80mhz')

I got this error:

Could not profile: No uploaded model yet
#4

Hi @Bouhmid,

Is there any way you could share your .onnx file so we can try to replicate the error?

Best,

Shawn

#5

Feel free to also share your project ID so we can check the logs if needed.

Best,

Louis

#6

Hello @Bouhmid, did you solve the problem?

#7

Hello @HasOut,
No, this problem is still unresolved. I forwarded my model to the Edge Impulse team and I am waiting for their response.

#8

I’m facing the same problem. Please if you have any updates let me know. Thank you.

#9

Hello @shawn_edgeimpulse @louis,

Are there any novelties?
I have tried with a simple pre-trained Alexnet model using Pytorch and I am facing the same issue!

#10

Hello @Bouhmid,

I am trying to replicate your issue with AlexNet to Pytorch to ONNX to Edge Impulse.
I am using this Colab: Google Colab

I can upload the model to EI.

Screen Shot 2023-04-25 at 17.19.42
Screen Shot 2023-04-25 at 17.19.423064×1272 231 KB

However, I now have an issue with the input format (nchw vs nhwc).
I’m checking internally

Best,

Louis

#11

I tried the code in Google Colab and it worked for me also. But when I tried it locally (.ipynb file), it didn’t work (it kept loading for maybe an hour). Should I always work on Google Colabs?

Thank you,
Ahmed

#12

Hello @Bouhmid,

No it should work the same way.
Which part is loading for an hour?

For the input format, we should convert it to the right format when receiving the ONNX file.
I am checking with the team where the issue comes from.

Best,

Louis

#13

Uploading pre-trained model!
The project is not even created in jobs.
No problem, I will just upload the model to your site next time ( But I think you should check the problem with implementing the profile function locally )

Besides, are LSTM models not supported for the moment?

#14

For the local profiling, it is solved!

I just want to make sure that LSTM models are not yet supported!

Thank you,

#15

Hello @Bouhmid,

I replied in the other thread: Are LSTM models deployable using Python SDK? - #3 by louis

Best,

Louis

1 Like
#16

Hey @louis ,

I wanted to follow up on your suggestion. I think it may work in cases of torch.hub.load function, but what if I need to load a custom model? I’ve written a function called load_pretrained that may be useful:

def load_pretrained(model, pretrained):
    pretrained_dict = torch.load(pretrained, map_location=device)
    if 'state_dict' in pretrained_dict:
        pretrained_dict = pretrained_dict['state_dict']
    model_dict = model.state_dict()
    pretrained_dict = {k[6:]: v for k, v in pretrained_dict.items() if (k[6:] in model_dict and v.shape == model_dict[k[6:]].shape)}
    model_dict.update(pretrained_dict)
    model.load_state_dict(model_dict, strict=False)

    return model

This function loads a pretrained model from a given path and merges the weights with the model that’s passed in as an argument.

Let me know what you think!