Overfitting Issue

Help
#1

Hello, this is my first time using Edge Impulse so please have patience with me!
I’m trying to make a project that identifies baby sounds using the Arduino Nano 33 BLE Sense Rev2.
While the training accuracy is quite good at 91.7%, when I run model testing, I get an accuracy of 84.72%.

I’ve looked through the forum and have adjusted my project as needed but I seem to be having the same issue.
I’ve tried to add more data but it only seems to make the problem worse.
I suspect that this is a problem of overfitting.

Project ID: 211985

#2

Hi @babybot,

It looks like you did all the right things when training your model: lots of data, balanced dataset, good features, etc. I think the issue here is that your ‘bah’ and ‘dah’ sounds are all very similar. Even human ears can struggle discerning the ‘b’ and 'd’sounds sometimes (think about the times you’ve had to spell something to someone over the phone). The reason that home assistant/smart speakers use uncommon 3+ syllable words/phrases as wake words (e.g. “Alexa,” “OK Google,” “Hey Siri”) is to avoid making them sound like anything else that might be spoken in the nearby vicinity (i.e. avoid false positives).

Indeed, if you look at the confusion matrix of your test set, you can see that’s where most of the confusion occurs. I might recommend a couple of approaches

  • Think about your project requirements: do you absolutely need to discern between ‘bah’ and ‘dah’? Or can you just identify that the baby said something along those lines?
  • If you need to discern between similar-sounding words, you will need to create better features. In this case, I recommend increasing the values of the MFCC’s number of coefficients, frame length, and filter number. You can also decrease the frame stride to ideally get better resolution (time-wise) among the sounds.

If you’d like to run some experiments, you can use the EON Tuner to try different models for you to help point you in the right direction. In the EON Tuner, click Configure target, go to the Space tab, and add more hyperparameter settings to the "dspBlocks" dictionary. Run the EON Tuner to see what kind of DSP hyperparameters help you the most.

#3

On another note, how did you link Rev2 with EI (trying to do this for the IMU) ?

#4

Hi @ggiakas,

Arduino Nano 33 BLE Sense Rev2 support in Edge Impulse is currently in the works and should be released soon.

1 Like