FOMO Returning 'null' for Class Name

Help
#1

Project ID: 173213, a FOMO model that counts screws in an image. There is only 1 class in the Model called screw.

The quantized (int8) Model memory usage seems typical (nice and small) for a FOMO Model:

  • RAM USAGE = 244.0K
  • FLASH USAGE = 74.6K

Upon running the static_buffer.ino with an output of:

Edge Impulse standalone inferencing (Arduino)
run_classifier returned: 0
Predictions (DSP: 10 ms., Classification: 1239 ms., Anomaly: 0 ms.):
[0.00000]
(null): 0.00000

  • the classification time of 1239 ms seems very long for running on a Sony Spresense
  • not to mention the model returns null and not the class name of screw

When running the float Model:

  • RAM USAGE = 898.8K
  • FLASH USAGE = 96.6K

the output is:

Edge Impulse Inferencing Demo
Edge Impulse standalone inferencing (Arduino)
run_classifier returned: 0
Predictions (DSP: 11 ms., Classification: 3136 ms., Anomaly: 0 ms.):
[0.00000]
(null): 0.00000

  • the memory requirements estimation for the FOMO model alone seems excessive
  • the classification time of over 3 seconds seems so very long for running on a Sony Spresense
  • again the model returns null and not the class name of screw

Questions

  • Why are the Models returning null for the Class type with a prediction value of 0.00000000000000000000000000000000000000000000000000000000000?
#2

Hello @MMarcial,

You’re working on the Sony Spresense with the Arduino Sketch correct?
So the Cortex-M4F is indeed pretty slow even with FOMO but it should not give you (null): 0.00000.

What are the results when you use the pre-compiled firmware?

Best,

Louis

#3

The short answer is the Edge Impulse version of static_buffer.ino that I used is in no way capable of handling a FOMO model. I am not sure what version I used because there seems to be no versioning on the EI Example Arduino files.

I created a version called static_buffer_fomo_too.ino that handles Image Classification models as well as FOMO (Image Segmentation) models. The code is here.

The long answer…

@louis The pre-compiled firmware worked since the issue is in the Arduino file.

The line
ei_printf(" %s: %.5f\n", result.classification[ix].label, result.classification[ix].value);
prints
(null): 0.00000

…as is expected when one is running a FOMO model since
result.classification[ix].label does not exist.

What one wants are the properties off of
result.bounding_boxes
and not properties off of
result.classification

#4

@MMarcial,

Thanks, I’ll check with the Embedded Team if they can modify the static_buffer.ino default example to support FOMO.

We should have something like what we have in the camera sketches:

    // Run the classifier
    ei_impulse_result_t result = { 0 };

    EI_IMPULSE_ERROR err = run_classifier(&signal, &result, debug_nn);
    if (err != EI_IMPULSE_OK) {
        ei_printf("ERR: Failed to run classifier (%d)\n", err);
        return;
    }

    // print the predictions
    ei_printf("Predictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
                result.timing.dsp, result.timing.classification, result.timing.anomaly);
#if EI_CLASSIFIER_OBJECT_DETECTION == 1
    bool bb_found = result.bounding_boxes[0].value > 0;
    for (size_t ix = 0; ix < result.bounding_boxes_count; ix++) {
        auto bb = result.bounding_boxes[ix];
        if (bb.value == 0) {
            continue;
        }

        ei_printf("    %s (", bb.label);
        ei_printf_float(bb.value);
        ei_printf(") [ x: %u, y: %u, width: %u, height: %u ]\n", bb.x, bb.y, bb.width, bb.height);
    }

    if (!bb_found) {
        ei_printf("    No objects found\n");
    }
#else
    for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
        ei_printf("    %s: ", result.classification[ix].label);
        ei_printf_float(result.classification[ix].value);
        ei_printf("\n");
    }
#if EI_CLASSIFIER_HAS_ANOMALY == 1
    ei_printf("    anomaly score: ");
    ei_printf_float(result.anomaly);
    ei_printf("\n");
#endif
#endif

Thanks for figuring this out!

Best,

Louis

#5

Hello @MMarcial,

We have a PR in progress, the static_buffer example will be updated soon so it can “support” FOMO without any modifications.

Best,

Louis