FOMO get lower than 80% Accuracy

Help
#1

Hi All

I try to training model to detect driver drowsiness with 4 object detection/classification using FOMO, but alwayas get accuracy lower than 80% and also on F1 Scores.

here is the labeling will be:

  1. mata terpejam = closed eye
  2. mata terbuka = open eye
  3. menguap = yawn
  4. tidak menguap = not yawn

Due to limitations time training, i try to use BYOM and increase di epochs up to 50-100 but get still similiar results.

Could you guys help to debug, what happen actually? and any advice to get better accuracy?

Thank you

#2

Hello @nugro,

I can see that for your classes open eye and closed eye, you selected the entire image as the object.

FOMO works best when object takes about 1/8th of the image.
See FOMO: Object detection for constrained devices | Documentation for more details.

I’d consider changing your dataset where it contains those two classes.
Also your yawn and not yawn classes contains eyes, which will be considered as background during the training.

Best,

Louis

#3

Hi @louis

So, do you mean, it will be better if give label closed_eye/open eye and yawn/not_yawan in same image instead of what i do right now?

I’d consider changing your dataset where it contains those two classes.

image 1.

image
image1172×942 55.9 KB

image 1 will detect mouth as background if i labeled like that picture? which is mouth also part of yawn not yawn class.

image 2.

image
image1056×942 57.7 KB

is the second image is what you mean?

also, where i can get fully confusion matrix like json file if train using EI platform when train FOMO locally?

Thank you

#4

Hello @nugro,

Yes, like in the second image!

For the confusion matrix, we apply some custom scoring to FOMO. If I remember correctly, you can find it in the expert mode with a name like CentroidScoring. But I am not sure where you can find the source code.

Also, I did a webinar on FOMO some time ago, feel free to watch it to understand the underlying concepts.

Best,

Louis

#5

Hi @louis

If I remember correctly, you can find it in the expert mode with a name like CentroidScoring

  1. I think it just return precision, recall and F1 scores, not as comprehensive data like json on dashboard?

  2. Since, I need like mAP and IoU data for research paper regarding implementing FOMO algorithm. So, my question is that possible?

Thank you