Not getting the correct prediction

Question/Issue:
I am not getting the correct prediction with Pico microphone MAX9814. I can get the correct result with “Live Classification” from the Edge Impulse platform, but not on the device. Can you please help?

Project ID:
175000
Context/Use case:
This is my code,

#include <stdio.h>
#include "edge-impulse-sdk/classifier/ei_run_classifier.h"
#include "pico/stdlib.h"
#include "analog_microphone.h"
#include "analog_microphone.c"
#include "tusb.h"

// configuration
#define INSIZE 16
const struct analog_microphone_config config = {
    // GPIO to use for input, must be ADC compatible (GPIO 26 - 28)
    .gpio = 26,

    // bias voltage of microphone in volts
    .bias_voltage = 1.25,

    // sample rate in Hz
    .sample_rate = 16000,

    // number of samples to buffer
    .sample_buffer_size = INSIZE,
};

// variables
int16_t sample_buffer[INSIZE];
volatile int samples_read = 0;

float features[2700];

void on_analog_samples_ready() {
    // callback from library when all the samples in the library
    // internal sample buffer are ready for reading 
    samples_read = analog_microphone_read(sample_buffer, INSIZE);
}

int raw_feature_get_data(size_t offset, size_t length, float * out_ptr) {
    memcpy(out_ptr, features + offset, length * sizeof(float));
    return 0;
}


int main(void) {
    // initialize stdio and wait for USB CDC connect
    stdio_init_all();


    printf("connected\n");

    ei_impulse_result_t result = {
        nullptr
    };

    ei_printf("Edge Impulse standalone inferencing (Raspberry Pi Pico)\n");

    // initialize the analog microphone
    if (analog_microphone_init( & config) < 0) {
        ei_printf("analog microphone initialization failed!\n");
        while (1) {
            tight_loop_contents();
        }
    }

    // set callback that is called when all the samples in the library
    // internal sample buffer are ready for reading
    analog_microphone_set_samples_ready_handler(on_analog_samples_ready);

    // start capturing data from the analog microphone
    if (analog_microphone_start() < 0) {
        ei_printf("PDM microphone start failed!\n");
        while (1) {
            tight_loop_contents();
        }
    }


    //if (sizeof(features) / sizeof(float) != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE)
    //{
    //  ei_printf("The size of your 'features' array is not correct. Expected %d items, but had %u\n",
    //            EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, sizeof(features) / sizeof(float));
    //  return 1;
    //}
    
    while (1) {

        ei_printf("\nStarting inferencing in 2 seconds...\n");
        //sleep_ms(3000);
        ei_printf("Sampling...\n");

        // wait for new samples
        while (samples_read == 0) {
            tight_loop_contents();
        }

        // store and clear the samples read from the callback
        int sample_count = samples_read;
        samples_read = 0;
        ei_printf("sample_count returned: %d\n", sample_count);

        // loop through any new collected samples
        for (int i = 0; i < sample_count; i++) {
            float val = sample_buffer[i];
            features[i] = val;
        }

        ei_printf("size of features/floats: %d\n", sizeof(features) / sizeof(float));
        ei_printf("EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE: %.3f\n", EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE);
        ei_printf("EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME: %.3f\n", EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME);
        ei_printf("size of features returned: %d\n", sizeof(features));
        ei_printf("size of features returned: %d\n", sizeof(features) / sizeof(features[0]));

        //sample_count returned: 16
        //size of features/floats: 2700
        //EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE: 0.000
        //EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME: 0.000
        //size of features returned: 10800
        //size of features returned: 2700
        //run_classifier returned: 0


        // the features are stored into flash, and we don't want to load everything into RAM
        signal_t features_signal;
        features_signal.total_length = sizeof(features) / sizeof(features[0]);
        features_signal.get_data = &raw_feature_get_data;


        // invoke the impulse
        EI_IMPULSE_ERROR res = run_classifier(&features_signal, &result, false);

        ei_printf("run_classifier returned: %d\n", res);

        if (res != 0)
            return 1;

        ei_printf("Predictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.): \n",
            result.timing.dsp, result.timing.classification, result.timing.anomaly);

        // print the predictions
        ei_printf("[");
        for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
            ei_printf("%.5f", result.classification[ix].value);
            #if EI_CLASSIFIER_HAS_ANOMALY == 1
            ei_printf(", ");
            #else
            if (ix != EI_CLASSIFIER_LABEL_COUNT - 1) {
                ei_printf(", ");
            }
            #endif
        }
        #if EI_CLASSIFIER_HAS_ANOMALY == 1
        printf("%.3f", result.anomaly);
        #endif
        printf("]\n");
        sleep_ms(3000);

    }

    return 0;
}

Hello @fd18,

Are the data samples used for the training the same as the ones used for the inference?
Or you trained your model with another microphone/device and then you uploaded the model on your Pico?
If so, the microphone output might produce a different signal than the one used for the training and it could explain the difference. You can also tweak the microphone output signal (modify the gain for example, to have something closer to your training data).
Normalizing the signal in your DSP block can also help to reduce the difference.

I hope that helps,

Best,

Louis

Thank you. how can I normalize the signal ?

Which DSP block are you using?
For Spectrogram and MFE blocks, this can be done through the “Noise floor (dB)” parameter, for the Spectral Analysis block through the “scale axis”.

Also, the live classification uses float32 models, if you downloaded the quantized version of your models, here are some tips that can help you improve the accuracy: Increasing model performance - Edge Impulse Documentation

Best,

Louis

I am using Spectogram and uses the same Pico board to generate the data and prediction. I copied the raw data from the Testing Model and I was able to get the correct prediction result. However when using the standalone live data the prediction is inaccurate.

@fd18

//EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE: 0.000
//EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME: 0.000

These values are integers and should not be 0 can you confirm in model-parameters/model_metadata.h?

Secondly, it appears you’re only updating up to IN_SIZE (16) values into features rather than EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (2700) before running inference.
This is not correct.

For audio, we have 2 APIs to run inference.

  1. run_classifier()
  2. run_classifier_continuous()

(1) expects features_signal to fully be update with all EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE BEFORE calling. To see an example, see nano_ble33_sense_microphone.ino in our Arduino Library export.
(2) allows you to send slices of audio until EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE values have been obtained. To see an example, see nano_ble33_sense_microphone_continous.ino in our Arduino Library export.

Note that the above examples also use double buffer, so that no data is lost. For more details see our Continous audio sampling doc.

I’d first try getting all 2700 sample values updated into your features array and then go from there.

// Raul

@rjames Thanks for detailed explanation.
From model_metadata.h, I got the following:

#define EI_CLASSIFIER_RAW_SAMPLE_COUNT           2700
#define EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME      1
#define EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE       (EI_CLASSIFIER_RAW_SAMPLE_COUNT * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME)

And now the result looks good after getting all 2700 sample values.

1 Like