Arduino audio inference at non-16kHz sample rate

#1

I was working on an audio classification project but recording audio at 44.1kHz. I set up my Impulse appropriately and trained the model and deployed it as an Arduino library and started working with the Nano 33 BLE Sense example code.
It was failing to start the PDM and I couldn’t figure out why until I peeked at the source of the nRF52x PDM implementation: ArduinoCore-nRF528x-mbedos/PDM.cpp at master · arduino/ArduinoCore-nRF528x-mbedos · GitHub

// configure the sample rate and channels
  switch (sampleRate) {
    case 16000:
      NRF_PDM->RATIO = ((PDM_RATIO_RATIO_Ratio80 << PDM_RATIO_RATIO_Pos) & PDM_RATIO_RATIO_Msk);
      nrf_pdm_clock_set(NRF_PDM_FREQ_1280K);
      break;
    case 41667:
      nrf_pdm_clock_set(NRF_PDM_FREQ_2667K);
      break;
    default:
      return 0; // unsupported
  }

So it seems like only 16000Hz and 41667Hz are supported. Not sure of the reasoning behind this but I was able to get through this just by manually setting the PDM.begin() function to sample at 41667Hz, while leaving the RAW_SAMPLE_COUNT at 44100. It’s not ideal but I think it’s close enough to work fine.

Also, this isn’t anything major but I noticed the Arduino example code for audio inferencing seems to assume 16kHz sampling regardless of EI_CLASSIFIER_FREQUENCY. This causes the Sample length display to be wrong. The actual implementation is still faithful to the set frequency from the Impulse Design phase so it’s not really a problem though.

// summary of inferencing settings (from model_metadata.h)
ei_printf("Inferencing settings:\n");
ei_printf("\tInterval: %.2f ms.\n", (float)EI_CLASSIFIER_INTERVAL_MS);
ei_printf("\tFrame size: %d\n", EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE);
ei_printf("\tSample length: %d ms.\n", EI_CLASSIFIER_RAW_SAMPLE_COUNT / 16);

Just sharing the experience in case anyone else encounters a similar situation or if anyone knows more about the PDM implementation logic.

#2

Hi @athommandram,

That’s correct. You need to do a few modifications to the Arduino sketch to support additional frequencies, see this post:

Aurelien

1 Like
#3

Ahh I missed that post, sorry! I just began down the rabbit hole of calculating it myself (for the Adafruit Feather nRF52840 Sense) :sweat_smile: - thanks so much for already figuring it out and sharing the implementation

1 Like
#4

So, can I just ask, is it workable, using the tweaks suggested above to do inferencing on the nRF52840 using a model trained with 44.1kHz samples? Or is the chip not capable? All of the samples I’ve gathered are also 44.1 and I think the sounds I’m trying to identify have a lot of energy in the higher frequencies, so before I go and resample everything by hand I’m curious if the OPs efforts were successful, using the linked sketch modifications. Thanks!

#5

Hey braddo, yes I was able to record and run inference at higher frequencies. It seemed to keep up fine with the 41667 Hz option, just the memory would get filled up much faster so I had to limit the window of recording. I didn’t try to get it closer to 44.1kHz but I think technically you could set up the clock control bits to hit 45.5kHz.

Like the built in definitions in the mbedos repo skip from 41667 to 50000 Hz, but I think maybe you could set the bits to make the divider 11, to get 45.5kHz?

The documentation for the nRF module was saying things like they don’t guarantee the PDM will work well at higher frequencies, and the reference table for the PDM clock control only goes up to 20.8kHz
https://infocenter.nordicsemi.com/index.jsp?topic=%2Fps_nrf52840%2Fpdm.html&anchor=register.PDMCLKCTRL

But seemed to work ok in my limited testing.