[BYOM] LSTM onnx model convert to int8 vela tflite run fail

,
#1

I create a pytorch LSTM model and convert it to onnx. Then, upload it to BYOM and let it convert to int8 vela tflite model to run on Seeed_Grove_Vision_AI_Module_V2/EPII_CM55M_APP_S/app/scenario_app/ei_standalone_inferencing at main · HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2 · GitHub .
Facing the issue about AllocateTensors() failed.
Screenshot from 2024-11-29 10-40-52

I also see the model which converted by BYOM. The LSTM module will be unrolled to many operators which could not be supported by ethosu vela. Because I saw at vela supported OPs which can support UNIDIRECTIONAL_SEQUENCE_LSTM, is it possible the BYOM can convert LSTM model to UNIDIRECTIONAL_SEQUENCE_LSTM operater?

#2

Hi @kris_himax
We have a bug in the calculating tensor arena size. We are working on fix, but temporarily workaround is:
Depending on your deployment type (EON Compiled or TFLM):

  • for the EON Compiler deployments, look for kTensorArenaSize in the tflite-model/tflite_learn_*_compiled.cpp and increase the size (depending on the memory available, you can start with doubling it)
  • For the TensorFlow deployments, look for tflite_learn_*_arena_size constant in the tflite-model/tflite_learn_*.h and increase this value.

BR,
Matt