Hi, is there a way to run or integrate my external delegate with the Linux C++ SDK?
I currently have a working .so dynamic library that uses the TensorFlow Lite external delegate mechanism.
Hi @kolejnik,
Yes you can. For example you can place your <your-lib>.so in one of the subfolders in tflite/ for the target your building for. Then in Makefile you adapt the build flags for the target you’re buidling, for example to link against you’re library add LDFLAGS+= -l<your-lib>.
// Raul
Hi @rjames,
If I understand correctly, first I need to compile my external delegate as a static library. Then, I should link it in the Makefile as you described, and it should select my external delegate without any changes in the C++ code. Is that correct?
I’m asking because I don’t think you can link a dynamic library (.so) as a static library (please correct me if I’m wrong).
Thanks for your help.
Hi @kolejnik,
No need to build your library as a static library. You can link against your dynamic library. We do this already for example with our flex delegates (see download_flex_delegates.sh and Makefile).
Note that because it’s linking dynamically you’ll have to copy the flex delegate library also to the target (e.g. in /usr/local/lib) before execution.
Hi @rjames,
I’ve linked my dynamic library, but when I run CUSTOM APP, I still see the log message INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Are we sure there aren’t any additional changes needed in the C++ code? I found the function TfLiteDelegate* TfLiteExternalDelegateCreate(const TfLiteExternalDelegateOptions* options); in the TensorFlow repository, and I’m wondering if we need to call this function in the Edge Impulse SDK to activate the external delegate.
Hi @kolejnik,
Yes, I think you’ll hack it into the SDK, at least for an MVP.
Take a look at edge-impulse-sdk/classifier/inferencing_engines/tflite_tidl.h around line 94 where we dynamically load delegate and use it. Maybe it sets you in the right direction.
If you have a public repository, please share.
To see what libraries your custom app depends on run:
readelf -d build/custom | grep NEEDED
See if you’re lib is in the list.
1 Like
Hi @rjames,
I successfully added the external delegate, and everything is working perfectly now.
Thank you so much for your guidance and support, I really appreciate it!
1 Like