Edge Impulse inference much slower than evaluated on Arduino Nano 33 BLE

Report bugs
,
#1

Question/Issue:
Significantly increased latency on Arduino Nano 33 BLE when running audio classification.

Project ID:
717452

Context/Use case:
I am currently working on a real-time audio classification model on the Arduino Nano 33 BLE. The model uses 16 kHz audio input and a 1-second window. Inference is performed using the Arduino library deployment. On the device, the DSP block runtime (result.timing.dsp) is approximately 3479 ms, and classification takes about 324 ms (result.timing.classification).
However, Edge Impulse Studio estimates DSP latency at around 488 ms and classification latency at 225 ms (using Arduino Nano 33 BLE Sense as the target).
In previous projects, the actual runtime closely matched these estimates, with total inference time under 1 second.

Summary:
Unexpectedly high DSP latency (~3.5 seconds) and increased classification time on the Arduino Nano 33 BLE, compared to the estimated values for a similar board (Arduino Nano 33 BLE Sense). Since models in earlier projects did not show this behavior, the issue may be related to a newer version of the Edge Impulse SDK.

Steps to Reproduce:

  1. In Arduino IDE, include the Arduino library as a .ZIP file (Sketch > Include Library > Add .ZIP library).
  2. Create a sketch from the static buffer example (File > Examples > Project name - Edge Impulse > static_buffer > static_buffer).
  3. In the sketch, paste raw features (any 16000 float values) into the static const float features[] definition.
  4. Select the connected board in Arduino IDE (Tools > Port > Arduino board).
  5. Upload the sketch.
  6. Open the Serial Monitor in Arduino IDE to view the output.

Expected Results:
Total inference time of about 713 ms (including approximately 488 ms for DSP processing and 225 ms for classification).

Actual Results:
Total inference time of about 3802 ms (including approximately 3478 ms for DSP processing and 324 ms for classification).

Reproducibility:

  • Always

Environment:

  • Platform: Arduino Nano 33 BLE (nRF52840)
  • Build Environment Details: Arduino IDE 2.3.6
  • OS Version: Windows 10
  • Edge Impulse Version (Firmware): 1.72.14
  • Project Version: 1.0.6
  • Custom Blocks / Impulse Configuration:
    • MFE block
      • Frame length: 0.02 s
      • Frame stride: 0.01 s
      • Filter number: 40
      • FFT length: 512
      • Low frequency: 125 Hz
      • High frequency: 7500 Hz
      • Noise floor: -80 dB
    • Neural network classifier:
      • Keras model
      • Quantized (int8)

Logs/Attachments:

Edge Impulse standalone inferencing (Arduino)
run_classifier returned: 0
Timing: DSP 3478 ms, inference 324 ms, anomaly 0 ms
#2

Just gotta check this… what optimization setting are you compiling with?

#3

No optimizations are used. Just standard TensorFlow Lite deployment.