I have build a small example (project ID 74857) and the estimated runtime is less than 100 ms. However, when I build it with the C++ lib and the sdk then the runtime is 5.3 seconds. I am not sure what I am doing wrong or where the bottleneck is.
@Lukas We go with the inferencing engine that makes the most sense on the hardware. For embedded systems that’s EON Compiler w/ TensorFlow Lite Micro kernels, on Linux (without an accelerator) that’s TensorFlow Lite w/ XNNPACK, on Jetson Nano it’s TensorRT, etc. So not needed on Linux systems, TFLite+XNNPACK already gives very good performance, and the overhead of the interpreter is not so bad as on embedded systems.