Deploying audio model on Portenta H7

Help
#1

Hi!

I saw somewhere on here that it is possible to use the Edge Impulse binaries for the Nano 33 BLE Sense for the Portenta H7 in the case of audio impulses. I’d love to get this running - and I have a well-trained model ready to go, but how is this accomplished?

To my understanding, OpenMV doesn not support these kinds of models, so what are my options for testing out the TFLite model on my Portenta H7?

Thanks in advance!

#2

Hello @avielbr,

I don’t think the Portenta H7 has a built-in microphone but you can get the portenta vision shield lora on the Arduino Store: https://store.arduino.cc/portenta-vision-shield-lora

I have been able to compile when choosing the m7 core but I do not have the vision shield to test it.

Screenshot 2021-05-24 at 11.26.06
Screenshot 2021-05-24 at 11.26.063572×2090 944 KB

Maybe @aurel you can try with yours?

Regards,

Louis

#3

Hi, thanks for the reply.

I should have clarified - I have the vision shield!

#4

So, when I try to compile the .ino file on my Portenta, I get the error sound_inference.h: No such file or directory. I see that the header file exists in the folder exported from Edge Impulse, but how would I go about getting the the IDE to find it during compilation?

EDIT: I added the .zip folder as outputted by EI to the Arduino IDE libraries, and it seems to have found the .h file… Waiting to see if it passes compilation!

1 Like
#5

Hi @avielbr Yeah that should do the trick. We’re currently working on official support for the Portenta H7 with examples on running vision models as Arduino sketches.

1 Like
#6

Hi @janjongboom,

Indeed it successfully compiled, however I’m not seeing any output on the serial monitor. Is there any further setup/modifications to the code required to test out my audio impulse on the Arduino IDE?

Here is the full library :slight_smile:

#7

@avielbr

With:

  • Portenta H7 + LoRa Vision shield
  • Arduino Mbed OS Portenta Boards v2.1.0 Arduino core.

It builds and runs the nano_ble33_sense_microphone example on the Portenta H7, but didn’t capture output from the microphone until I changed the microphone_inference_start function to:

static bool microphone_inference_start(uint32_t n_samples)
{
    inference.buffer = (int16_t *)malloc(n_samples * sizeof(int16_t));

    if(inference.buffer == NULL) {
        return false;
    }

    inference.buf_count  = 0;
    inference.n_samples  = n_samples;
    inference.buf_ready  = 0;

    // initialize PDM with:
    // - one channel (mono mode)
    // - a 16 kHz sample rate
    if (!PDM.begin(1, EI_CLASSIFIER_FREQUENCY)) {
        ei_printf("Failed to start PDM!");
        microphone_inference_end();

        return false;
    }

    // configure the data receive callback
    PDM.onReceive(&pdm_data_ready_inference_callback);

    // optionally set the gain, defaults to 20
//    PDM.setGain(80);
    PDM.setBufferSize(4096);

    return true;
}

So weirdly setGain on Portenta does not seem to make the microphone work anymore.

edit: I do see the board hard faulting after a few inferences (SOS pattern on the LED next to USB port). I’ve asked our embedded folks to take a look.

2 Likes
#8

Can confirm this works! Now to get my model’s accuracy up to scratch.

Thanks Jan – I don’t know how you guys manage to be so productive while also being so present and helpful on these forums! Much appreciated.

2 Likes
#9

Hi Jan,

Any news on this front? I’d love to use my Portenta microphones in a project but running into this same issue.

#10

Nothing yet except for what I showed above… We’re coming out with a full Portenta port (with all sensors enabled directly from Edge Impulse, and deploy from Studio) once all fixes have landed in the Portenta Arduino core that we need, but hard to put an exact time on that…