FOMO Missing Objects

What could cause FOMO to miss an object?

Background Statement
Previously I ran another experiment similar to this one and nothing changed except maybe the screw location but the position was very similar to the previous experiment. In the previous experiment a second erroneous Bounding Box (BB) was detected. In post 8 of the previous experiment user matkelcey gave advice on how to solve the erroneous BB issue.

These experiments take repeated inferences of a screw, so one would think that:

  • All of the FOMO prediction scores would be similar, and
  • A FOMO detection would occur at-each-and-every inference (within the bounds of the Accuracy of the model)

Problem Statement
In this experiment:

  • Good News: At each FOMOed detection only a single Bounding Box (BB) was found, aka no extraneous BB existed.
  • Bad News: 40% of the time no FOMO detections occurred even tho a trained FOMO object was present. Based on the EI Studio reported Accuracy I would expect misses 10% of the time. (Actually I would expect less than 10% of the time a miss would occur since the object did not move, nor did the lighting change.)

The following run sequence chart (RSC) shows how the prediction scores varied over time. Until someone can prove me wrong, I will I assume that the prediction scores are a Gaussian distribution about the F1 score exhibited in the Edge Impulse (EI) Studio, Impulse Design, Object Detection page. The (RSC) starts at such an F1 score but then starts to drop until the EI_CLASSIFIER_OBJECT_DETECTION_THRESHOLD is reached at which point FOMO says no objects are present. Then oddly enuf the Model recovers only to decline back into a No Object Found state.

It seems oddly weird that the prediction scores are correlated.

Figure 1. Run Sequence chart showing correlated prediction scores as well as many images where FOMO failed to detect an object.

Figure 2. A zoomed in view showing how the FOMO “misses” become more frequent and then less frequent over time.


Q1: What could cause FOMO to miss an object?

Q2: What could cause the coorelation in the prediction scores?

The model is trained with cross entropy so maximises P(object | image) in expectation over the entire training set. So it’ll do well with images like training, and poorly for images unlike training. You can add either more data and/or model capacity to improve.

The score getting worse/better over time is illustrating examples of the image on/off the manifold that the model has learnt about screws. We always expect images captured closer in time should have closer predictions; that’s the nature of the smoothness of linear classifiers ( neural network or otherwise )

Like a lot of things it’s likely you can fit a normal distribution to these scores, but I reckon that’s just thanks to the “central limit thereom”. I wouldn’t read anything into it.

What are the two variables you are saying are correlated? time ~ P(object) ? That’s 100% expected and desired ( see comment re: smoothness above )