We’ve released the ability for DSP blocks to host pretrained neural networks. This enables you to use any NN as a feature extractor. For example, PoseNet to first get pose vectors before training a small classifier. This flow is 100% compatible with our deployment pipeline, so you can compile this impulse to any hardware (that supports the ops of the model). Here’s an example using PoseNet: GitHub - edgeimpulse/pose-estimation-processing-block
We’ll be updating the docs in the coming week (and show some more demo’s, e.g. around complex sensor fusion that this also enables), but wanted to share this w/ the community immediately
The README file in this GitHub repo says “Due to the size of the model and some unsupported ops it won’t work on MCU in its current form. If you decide to train a smaller custom model, you’ll need to replace model.tflite .”
However, I currently have no idea how to replace the model with a smaller one to make the pose estimation preprocessing block work on a MCU. Could you provide some guidelines? Thank you.
Model Size:
Have you tried running the EON tuner to reduce the size? I can see if that was attempted on the two projects named pose estimation / advanced.
Supported Ops
I’ll follow up with our embedded team to verify the supported operations for the your specific device.
Additionally, lets loop @shawn_edgeimpulse from the university program to see if he can provide any specific advice or guidance here for you.
For this to work on MCUs, you’ll need to train your lightweight pose estimation model outside of Edge Impulse. We currently don’t support pose estimation output format in Edge Impulse.
We support using it as a custom DSP block, thus use it as a feature extractor before training your model.
If you want to replace the model.tflite, start with the Github repository you shared as a template. You will find in the root folder the model.tflite. Start by trying to replace it with your own version and adapt the rest of the code if needed.
The projects named “pose estimation” didn’t use PoseNet, they are actually just experimental.
I’ll try the EON tuner to see how it works on the custom DSP block then.
