Hi, is there a way to run or integrate my external delegate with the Linux C++ SDK?
I currently have a working .so
dynamic library that uses the TensorFlow Lite external delegate mechanism.
Hi, is there a way to run or integrate my external delegate with the Linux C++ SDK?
I currently have a working .so
dynamic library that uses the TensorFlow Lite external delegate mechanism.
Hi @kolejnik,
Yes you can. For example you can place your <your-lib>.so
in one of the subfolders in tflite/
for the target your building for. Then in Makefile
you adapt the build flags for the target you’re buidling, for example to link against you’re library add LDFLAGS+= -l<your-lib>
.
// Raul
Hi @rjames,
If I understand correctly, first I need to compile my external delegate as a static library. Then, I should link it in the Makefile
as you described, and it should select my external delegate without any changes in the C++ code. Is that correct?
I’m asking because I don’t think you can link a dynamic library (.so) as a static library (please correct me if I’m wrong).
Thanks for your help.
Hi @kolejnik,
No need to build your library as a static library. You can link against your dynamic library. We do this already for example with our flex delegates (see download_flex_delegates.sh and Makefile).
Note that because it’s linking dynamically you’ll have to copy the flex delegate library also to the target (e.g. in /usr/local/lib
) before execution.
Hi @rjames,
I’ve linked my dynamic library, but when I run CUSTOM APP
, I still see the log message INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Are we sure there aren’t any additional changes needed in the C++ code? I found the function TfLiteDelegate* TfLiteExternalDelegateCreate(const TfLiteExternalDelegateOptions* options);
in the TensorFlow repository, and I’m wondering if we need to call this function in the Edge Impulse SDK to activate the external delegate.
Hi @kolejnik,
Yes, I think you’ll hack it into the SDK, at least for an MVP.
Take a look at edge-impulse-sdk/classifier/inferencing_engines/tflite_tidl.h
around line 94 where we dynamically load delegate and use it. Maybe it sets you in the right direction.
If you have a public repository, please share.
To see what libraries your custom app depends on run:
readelf -d build/custom | grep NEEDED
See if you’re lib is in the list.