FOMO Bad accuracy in similar samples

Hi,
I am currently working on a project which will help me avoid stray cats from intruding into my house while still allowing mine to enter. For this I am trying to train a model to differentiate any lambda cats from mine. The idea is to play a scary sound at the cat flap when a stray cat is detected and play nothing when my cat is.

My dataset for the training process consists of 128 photos of stray cats (labelled as “cat”) from this website and 62 photos of my cat (labelled as “minette”) which I took with my phone.

The greatest F1 score I achieved was 62.7%. 91.4% of the stray cats were correctly labelled as “cat” but 100% of Minette’s photos (my cat) were labelled… as background:

I think my dataset isn’t consistent enough to help differentiate cats from mine because they have a lot of features in common and too few distinctive ones.
The feature explorer seems to confirm my hypothesis:

The main problem is that my project is already too close to the training time limit of twenty minutes, so adding new data samples of my cat will likely compromise the thing.

May someone have any idea/workaround regarding this issue ?

Here are the best training settings I used:
Number of training cycles: 120
Learning rate: 0.15

My project ID is 436092

Many thanks in advance !

Hi Nirijam,
i don’t know how to look at your project, so i can only guess.
I have had a similar task ( distinguish different types of birds ) and also have had low recognition rates first.
After increasing the framesize to 124x124 and change to “color” the results getting better, - about 90 %.
It is still a problem if the colors / drawing of the birds are very close together, a lot of missmatches are at the results.
Again increasing the framesize seams to make this better, but results at low speed at little devices like a ESP32cam, - i bought some Risk-V boards to check if it works fast enough for this kind of tasks.

Some thoughts to your projct at all:
Are you realy in mind playing a scary sound is a good idea ? Did you think about your neighbors and / or if the alarm starts at the middle of the night ? :wink:

2 Likes

Hi herbk,
Thanks for your advices! I was using a framesize of 96x96 which may explain the low accuracy too.
As you said, this increases a lot the process time (worse I got was like 11s haha) so i’ll try to find the right balance.

I’ll let you know if i have better results overall, I’m using the Tiny ML Kit from Arduino (Nano BLE Sense board + OV7675 cam).

Concerning the scary sound (that might awake my neighbors), well they’re pretty far away from us so I believe they won’t hear anything. Anyway, it was the landlord idea so not my responsability haha. In case of complaints, I have a backup solution which consists of spraying water right onto the cat :slight_smile:

Thanks again for your reply,
Nirijam