Question/Issue:
When I am using spectral anlysis block my accuracy is above 90% or almost 100% where as when i am using custom processing block where i am only removing Rms, Skew and Kurtosis values from spectral analysis block my accuracy is almost half around 47%, why is there such a big difference ?
does edge impulse works better with it’s own builtin blocks or does those values (Rms, Skew and Kurtosis) has such a big impact on results ?
Project ID:
169460
Context/Use case:
Hello @Omer1222,
That actually depends on what you are trying to classify (thus your dataset).
Feel free to try the “calculate feature importance” function in the DSP block → Generate features.
I will tell you which features has the most impact on the pre-processing.
Best,
Louis
Thank you so much for your response, how does edge impulse calculates feature of importance ?
Builtin Block
Hello @Omer1222,
There are actually two things to consider:
- The number of features your dataset will learn on. The more features usually the better but this also has an impact on your on-device performances
- The feature quality (or importance). This is where you can choose carefully which features will have more impact and thus, by removing less important features you can optimize your on-device performances.
In your case, you remove features and theses features had a significant impact. That is probably why you saw a decrease in accuracy.
In your custom block, you can try removing the features that have less impact to see if you can keep a good accuracy while reducing the resources needed to run your DSP:
Best,
Louis
2 Likes
Hi @Omer1222,
We use Recursive Feature Elimination (RFE) algorithm to calculate the feature importance of the blocks.
More info is in here:
Best,
Omar
2 Likes
A RandomForestClassifier is trained on the data and the feature_importances_ are extracted from the trained classifier using RFECV.
Best,
Louis
1 Like
Thank you so much for your response, I want to ask that if i calculate these feature from DSP block just by running this code on my pc and then give them as input to my own model on google colab will it produce the same results as edgeimplse or is there something else applied while creating feature of importance which makes results better when we run the model on edge impulse?
I am asking that because my accuracy is coming 100% almost every time.
I have another question why my “loss” is coming as 1.09 and not 53.3% in first picture i uploaded?
Yes, it should produce the same results, we are not doing anything else on the DSP side.
To answer your loss, and accuracy question, I need to understand what is your dataset and what are you trying to do.
Best,
Omar