Hi All, I exported the default arduino (nano BLE) code from an audio inferencing project - the code works. Hooray!
However, I can’t really use it unless I can add to it the ability to detect a button press. I suspect that the code cannot see my button press because the inferencing code is blocking. See below, it uses an innocuous “delay(1)” statement, that I think is chewing all available cycles on the powerful little NRF device. I have attempted to adjust the default code so that it is not blocking. You can see below the “my code” section immediately before the commented out “original code” section.
Something like this would only work if, on each iteration of the main loop this routine is called, so the microphone is recording its samples one at a time and this code signals “hey I’m done now” when the buffer is full and ready to inference. But I’m not clear, maybe this code is called only once (per full load of the buffer), and that is why the delay() is in there - if the explicit intention is that it does not leave this routine until it’s done maybe I have to do more serious edits elsewhere - but I can’t even find the code that calls this routine (it is not referenced in the main ino file), I guess it’s buried in one of the includes? (can’t find those either, curses Arduino).
Can someone who knows tell me if I am on the right track here? The below does not work
* @return True when finished
*/
static bool microphone_inference_record(void)
{
bool ret = true;
if (inference.buf_ready == 1) {
if(Serial) {
ei_printf(
"Error sample buffer overrun. Decrease the number of slices per model window "
"(EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW)\n");
}
ret = false;
}
/** original code
while (inference.buf_ready == 0) {
delay(1);
}
inference.buf_ready = 0;
return ret;
end original code **/
/// my code
if (inference.buf_ready == 0) {
__asm__ __volatile__ ("nop\n\t"); // don't do anything for a very short while
inference.buf_ready = 1;
ret = false;
return ret;
}
else {
inference.buf_ready = 0;
return ret;
}
/// end my code
}
By the way, an alternative to the above approach would be to replace
while (inference.buf_ready == 0) {
delay(1);
}
with
while (inference.buf_ready == 0 and !buttonpressed) {
delay(1);
}
but I think the same question would apply - does this routine get called more than once per buffer load to incrementally fill, or is it called only once per full buffer? If the latter it doesn’t seem like it will be enough to make this code non-blocking because the code around it is expecting to wait/block.
If that turns out to be the case, I would love a tip on how to look upstream and fix this issue.
I was excited to find this other post on Arduino forum with exactly the same problem, but conflicting with I2C. However, unfortunately no solution posted. Is there any place where the autogenerated code from Edge Impulse is explained a bit so that someone might troubleshoot it? This may just be PDM code, will look further, but it’s super frustrating to get a model trained and working but not be able to use it.
The inference code is indeed blocking. With the Arduino Nano 33 BLE, the while (inference.buf_ready == 0)... is not going to be a big deal. That only exists to ensure that the audio buffer is filled prior to calling run_classifier_continuous(). You can definitely run other code in that while loop if you wish.
The bigger issue is that run_classifier_continuous() is going to block for ~300 ms on the Nano while MFCC + inference is performed. If you wish to identify a button press during that time, your best bet will be to use a hardware interrupt (e.g. attachInterrupt()) and set a global flag. Then during or before that “wait while buffer fills” loop, do something with the button press. For example:
volatile uint8_t btn_flag = 0;
void btn_isr() {
btn_flag = 1;
}
void setup() {
pinMode(button_pin, INPUT_PULLUP);
attachInterrupt(digitalPinToInterrupt(button_pin), btn_isr, FALLING);
...
}
void loop() {
...
if (btn_flag > 0) {
btn_flag = 0;
// Do something because the button has been pressed
}
while (inference.buf_ready == 0) {
delay(1);
}
...
}
Note that there will be up to a 300 ms delay between the button press and the action, as the CPU is hogged by run_classifier_continuous() for large chunks at a time.
Thanks for chiming in again Shawn, really appreciate the input.
I will say that I’ve made some progress since this post and so far it seems that, apart from that while loop, the code (mostly?) doesn’t seem to be blocking… (with the while loop it is 100% non-functional!) In other words, it will run just fine and record, inference and print the scores for my keysounds continuously with button checks interleaved. I am not here to tell you how your code works, but somehow this method is functioning, and it seems like the FAQ description your continuous inferencing seems amenable to this kind of implementation. It may be that this 32MHz little micro is just that fast and your code is just that efficient.
I was able to adapt my state machine to only call the code that was in your main loop() when a button is detected, and while it is running, a button press will stop it instantly without a formal interrupt and ISR. When my button handler is done, I can go back and just run the EI loop() code again and it works. I would try the interrupt routine but unfortunately, I can’t wait 300ms to detect a button. Some of the buttons are endstops for a motor that will break the device (this is not a life or injury threatening machine but I don’t want to break it!)
It is possible or even likely that some frames in the record/inference buffer are somehow injured and the scoring is not as good as it would be if left undisturbed. Gererally I am able to detect noise and my keysounds as expected, however, every 15 minutes or so I will get a buffer overrun. Which I might be able to avoid by more formally stopping/starting PDM or other routine as you had previously suggested.
Besides the occasional overrun, I still have one issue to work out but 99% done. Too bad that issue has already taken 2 weeks
I will post what I think to be simplified working code later because I really do want to understand what is the impact to the detection quality of doing it this way and if there are any more detailed ideas for how to address the overruns etc. that you and your engineering team might suggest.
