Arduino Nicla Voice Frustrations with Cough-detection model: 90% recall in Studio, 0% detections on-device

Report bugs
, , ,
#1

Question/Issue:
We trained an 8-class audio classifier for cough detection on the Arduino Nicla Voice (Syntiant NDP120). On Edge Impulse’s held-out testing set it achieves 90.2% cough recall (571 held-out coughs) and 82.4% binary (cough vs. not-cough) accuracy. After deploying to the physical board of the Arduino Nicla Voice, the on-device classifier has never once reported cough across dozens of live trials— it consistently reports ambient_noise instead. We ran a controlled test, with the same cough sounds played back with the device simply sending the sounds via its the microphone to EI studio, compared to when the model is deployed to the Nicla and running live classification. This tells us that the sound / stimulus or the microphone isn’t the problem; the problem is that the model, when deployed to the Nicla, goes from very useful to garbage.

Project ID: 820917

Context/Use case:
Academic project building a wearable cough detector for respiratory-disease surveillance. Model: 3× Dense(256) + Dropout(0.2), 8 classes (cough, conversation, ambient_noise, laugh, sigh, sneeze, sniff, throat_clear), trained on 5,372 samples from published corpora (cough-speech-sneeze/AudioSet-derived, VocalSound, Google Speech Commands, ESC-50, DEMAND). DSP block: Audio (Syntiant), log-bin (NDP120/200) feature extractor — confirmed matching Arduino’s documented requirement for this board. Standard Classification learning block (not Transfer Learning).

Summary:
An audio classification model performs very well on Edge Impulse studio, the same model performs awful when deployed to the Nicla Voice.

Steps to Reproduce:

  1. Confirmed 90.2% cough recall on the held-out testing set via Edge Impulse studio
  2. Ran “Find posterior parameters” (no calibration) → cough threshold 0.7, window 5, backoff 20.
  3. Built and flashed the “Arduino Nicla Voice” deployment target. Build succeeded cleanly: 543,496 params, 0.8ms/inference (17,322 cycles @ 21.5MHz), ~4.1µJ/inference — comfortably within budget.
  4. Live-tested on-device via the official edge-impulse-run-impulse CLI
  5. Ruled out: mic hardware failure (see controlled test below), boot/firmware failure (clean boot log, no init errors, all three .synpkg files present and loaded), wrong DSP block (confirmed Audio (Syntiant) + log-bin).
  6. Controlled comparison: played the same recorded cough (from a phone, fixed distance/volume) through two paths back-to-back:
  • On-device (NDP120 quantized “bitmatch” model): no match fired; classifier held at ambient_noise throughout.
  • “Use Nicla as microphone” → Studio float32 model (Live Classification’s “Classify new data”): the same cough audio, captured through the same physical mic, was correctly identified — 95% confidence on one window, 83% on the adjacent window, both landing exactly on the cough’s waveform burst.

Expected Results:
Comparable performance between Studio evaluation and on-device inference, or at least some non-zero cough detection rate on-device given 90%+ recall in Studio.

Actual Results:
0% on-device cough detections across all live trials, vs. confident correct classification of the identical audio when routed through the float32 model instead of the on-chip quantized one.

Reproducibility:
Always (0% on-device detection rate, consistent across multiple sessions and dozens of trials).

Environment:

  • Platform: Arduino Nicla Voice
  • Deployment target: “Arduino Nicla Voice” (full binary), Syntiant NDP120 library TDK v13.1
  • Edge Impulse CLI: 1.36.0
  • OS: macOS

To find out Edge Impulse Version:

  • if you have pre-compiled firmware: run edge-impulse-run-impulse --raw and type AT+INFO. Look for Edge Impulse version in the output.
  • if you have a library deployment: inside the unarchived deployment, open model-parameters/model_metadata.h and look for EI_STUDIO_VERSION_MAJOR, EI_STUDIO_VERSION_MINOR, EI_STUDIO_VERSION_PATCH
  • Edge Impulse CLI Version: [e.g., 1.5.0]
  • Project Version: [e.g., 1.0.0]
  • Custom Blocks / Impulse Configuration: [Describe custom blocks used or impulse configuration]
    Logs/Attachments:
    [Include any logs or screenshots that may help in diagnosing the issue]

Related reports:
Two other threads here describe a similar Studio-vs-device mismatch specifically with the Syntiant DSP block:

Has anyone found a fix beyond switching to Transfer Learning (Keyword Spotting)? We’d like to understand what’s actually happening in the Syntiant bitmatch conversion, or narrow down whether it’s the quantization or the posterior-window duration. Happy to share the project or run further diagnostics if useful.