Running inference with Syntiant TinyML board

Hello, I’m trying to run my own EI model using the Syntiant firmware and example provided here: https://docs.edgeimpulse.com/docs/tutorials/hardware-specific-tutorials/responding-to-your-voice-syntiant-rc-commands-go-stop.

It looks like the model is good, and I flashed the firmware, adding the model to the firmware from github. However, looking at the Serial output, it doesn’t look like it is performing inference (or looping):

Starting Syntiant TinyML
Initializing SD card...initialization done.
Enabling battery charging mode
Running in Bridge Modesetup for audio done
Loaded configuration
Inferencing settings:
	Interval: 0.0625 ms.
	Frame size: 15488
	Sample length: 968 ms.
	No. of classes: 3
Starting inferencing, press 'b' to break

Has anyone seen this before? Any suggestions on how to get inference working on the Syntiant TinyML board? I believe I also saw this behavior with the default stop/go firmware.

Thanks!
Justin

OK, making progress! It turns out the upload script only uploads the sketch bin file and not the NN bin file (ei_model.bin) so that’s why it wasn’t performing inference. Once I borrowed the firmware loader bat file (just replacing the firmware and model files with my own) I have the board running inference.

What I’m trying to do is play a wav file once a keyword is spotted. I have the code working correctly as a standalone sketch, but when I try to put it into the on_classification_changed method, the code hangs. I’m wondering if there is a timing/mutex thing that is preventing the wav file from playing. Any suggestions? For example, here is what i have in my code:

void on_classification_changed(const char *event, float confidence, float anomaly_score) {

    if (strcmp(event, "breathe") == 0) {
        // Toggle LED
        digitalWrite(LED_RED, HIGH);
        Serial.println("Playing breathe!");
        myFile1 = SD.open("breathe2.wav", FILE_READ);
        AudioZero.begin(22050);
        AudioZero.play(myFile1);
        AudioZero.end();
        myFile1.close();
    }

Any suggestions would be appreciated!
Thanks!

Hi @jlutzwpi,

That’s a good question. The Syntiant library has indeed a periodic timer to control LEDs. You could try to disable it before ei_classification_output is called in the syntiant.cpp file:

timer4.enableInterrupt(false);
ei_classification_output(match -1);
timer4.enableInterrupt(true);

Aurelien

Hi Aurelien, thanks for the suggestion. It looks like I’m getting the same behavior. I did some more debugging and it looks like the hang up is in the audio code timer interrupt registers:

void AudioZeroClass::tcConfigure(uint32_t sampleRate)
{
	// Enable GCLK for TCC2 and TC5 (timer counter input clock)
	GCLK->CLKCTRL.reg = (uint16_t) (GCLK_CLKCTRL_CLKEN | GCLK_CLKCTRL_GEN_GCLK0 | GCLK_CLKCTRL_ID(GCM_TC4_TC5)) ;
	while (GCLK->STATUS.bit.SYNCBUSY);

	tcReset();

	// Set Timer counter Mode to 16 bits
	TC5->COUNT16.CTRLA.reg |= TC_CTRLA_MODE_COUNT16;

	// Set TC5 mode as match frequency
	TC5->COUNT16.CTRLA.reg |= TC_CTRLA_WAVEGEN_MFRQ;

	TC5->COUNT16.CTRLA.reg |= TC_CTRLA_PRESCALER_DIV1 | TC_CTRLA_ENABLE;

	TC5->COUNT16.CC[0].reg = (uint16_t) (SystemCoreClock / sampleRate - 1);
	while (tcIsSyncing());
	
	// Configure interrupt request
	NVIC_DisableIRQ(TC5_IRQn);
	NVIC_ClearPendingIRQ(TC5_IRQn);
	NVIC_SetPriority(TC5_IRQn, 0);
	NVIC_EnableIRQ(TC5_IRQn);

	// Enable the TC5 interrupt request

        //*** code is hanging here ***
	TC5->COUNT16.INTENSET.bit.MC0 = 1;
	while (tcIsSyncing());
}	

The strange thing is that this code runs standalone. When I try to integrate it with the Edge Impulse firmware, it hangs up. Guessing it’s an issue between the EI firmware timer and the audio timer? Any SAMD21 register experts that you are aware of? Thanks!

So it looks like the timer is still running, only the interrupt’s callback is disabled. I’ve checked a bit more the SAMDtimer library used by Syntiant and using timer4.enable(false) instead may do the trick.

If you are in touch with Syntiant, their support may be able to help on this as well.

Aurelien

Aurelien, thanks! Yes, I tried the timer4.enable(false) as well and same behavior. I am tagging up with Syntiant later today so I’m hoping they may have an idea.

Thanks!