I am trying to train a model on Edge Impulse to run on an AudioMoth acoustic sensor. Initially I want to train a binary model, which will classify presence or absence of rare gunshot events, and then record the gunshot and not record the non gunshot sounds. As these events are rare, I want to maximise recall at the cost of increased false positives, as the aim is to then analyse the data more thoroughly after it has been recorded. When training models in Python, I have achieved this by selecting the model which has the best recall for gunshots, and then choosing a decision threshold for the presence score that will maximise the classification of gunshot events (I chose the threshold with the maximum F2 score).
I cannot see a way to do this in Edge Impulse. I can see that when I test the model there is an option to set the decision threshold, but when I do this, I end up with a lot of classifications that have both categories, and are consequently not classified correctly. What I want is to focus only on the ‘presence’ score, and anything above the threshold will be classified as present, as opposed to getting a lot of ‘unknowns’ or lots of 1 gunshot + 1 background scenarios. In reality, every sound has background in it, so I don’t care about this at all - I just want to focus on if a gunshot appears.
Thank you for any advice you can offer on this!