Struggling to Convert Pytorch → ONNX to Edge impulse
216047(Not very Interesting)
Thesis for University - Trying to Convert Model
I am a student at the University of Amsterdam who is currently writing his thesis about SWIN Transformers for Motion Amplification.
I have spoken to a few people from the Edge Impulse team and I am seeking the vibrant community for some help. (Thanks in advance and thanks again Team!)
What’s the Issue
My Code can be found here
The checkpoints (Which should be put in a ckpt/ directory) can be found here.
- Currently in Possession of a PyTorch Model (Trained in PyTorch 1.11 but also a version with 10 Epochs in PyTorch 2.0.1) converted to ONNX with set magnification factors but: The input of this model is 3 input tensors (Two tensors that are images and a magnification tensor). Model from Ricard Lato.
- Unable to Convert this model into ONNX (unsure if ONNX is successful)
Possible Solutions / Suggestions from Me
(Open to any other ideas)
- Convert from PyTorch —> Tensorflow using HuggingFace such as done in there Transformer examples (Unable to link as do not have permissions to add more than 2)
- Convert from PyTorch —> ONNX —> Tensorflow light
- Convert Model from Pytorch to TensorFlow by rewriting the model from the original model reference above and retraining
- As Edge Impulse only supports one input tensor convert the two frames into one new tensor each time and set a standard magnification on export to ONNX.
- Is this even possible in regards to converting SWIN Transformers from PyTorch —> ONNX —> TFlite
What occurred when uploading any of the ONNX files to Edge Impulse
Traceback (most recent call last): File "/app/convert-via-tf-onnx.py", line 68, in <module> ei_tensorflow.onnx.conversion.onnx_to_tflite(args.onnx_file, file_float32, file_int8, File "/app/./resources/libraries/ei_tensorflow/onnx/conversion.py", line 69, in onnx_to_tflite raise Exception('More than 1 input tensor, not supported') Exception: More than 1 input tensor, not supported Conversion using onnx-tf also failed, cannot use this ONNX file - contact your solutions engineer, or post the logs on the forum. As I mentioned in the call there are a couple of restrictions to converting models with our tooling: - Must have 1 input tensor - batch == 1 - You are restricted to operations that Tensorflow Lite supports (https://www.tensorflow.org/lite/guide/ops_compatibility) If you can solve these for your model then things should work. It is possible you may have more luck converting a tensorflow model rather than oonx. I'm not an expert on these kinds of models but I'm sure we can find you some expertise on the forum!