Now that the trained model (.lite file) is created, I tried to take this model as a replacement to an existing ARM KWS model. I modified some specific parts of their code to match with my EdgeImpulse tflite model that I could successfully build the project with this replacement. However, I see different behavior. I was wondering how much the preprocessing step is different that the model is trained based on that and does not work well in other projects. I mean whether the MFCC calculation is different than others? or even other parts of preprocessing, such as shifting, normalization, and etc.
I would appreciate any feedback.
Yes, the preprocessing step plays a big part in determining which features are sent to the neural network (i.e. prior to calling inference with the .lite file). To make sure that your custom preprocessing code is similar to what’s in Edge Impulse, I recommend using a known-good sample from your Edge Impulse project. If you go to the “MFCC” page, you can click on individual samples to see both the raw input and the raw MFCCs. Copy the raw features (time-series data that makes up the audio data) into your custom DSP code. Run it, and compare the output to the “processed features” that I’ve highlighted in red in the image. Try this for a few samples from different classes to make sure that the MFCCs are close.
Thank you for your comment and I tried to capture MFCC outputs in the ARM project and compare those with the one I see in Edge Impulse with the same input vector (taken from raw features). They are not the same as can be seen in the following shot:
I take that raw features and feed them into an ARM internal KWS model to see if it could spot the keyword correctly and seemingly it did correctly.
So, my question is why the MFCC is different from the one in other platforms? could someone help me realize the way Edge Impulse calculates MFCC?
That being said, sometimes features don’t need to be exact (considering there are a different techniques used to compute the various parts of the MFCCs, like different FFT libraries). You just need to be “close enough” for KWS to work. Keep in mind that “close enough” is relative–the more you deviate from how EI computed MFCCs, the lower your accuracy will likely be.
