Different Inference Time on Edge Impulse Studio vs STM32 Discovery Kit

Report bugs
#1

Question/Issue:
[I am here to report an issue i am having with my model when i deploy it on my target board - stm32l4r9 discovery kit.]

Project ID:
[614273]

Context/Use case:
[Respiratory rate estimation]

Summary:
[First of, the inference time recorded on the edge impulse studio is about 133ms post training. however when i deploy on the device inference time is about 6200ms - which i am able to infer by using a timer on my board. I have used a timer - which starts just before the classifier runs and stops when the classifier ends - because whenever the results are printed on screen the default code shows 0 ms… which is not true when you visualize it over a plot

I have tried to fix issue on my own but to no avail. Hence why I am coming here
Please find screenshot attached]

Expected Results:
[Inference time of 133 ms]

Actual Results:
[Inference time of 6200ms]

Reproducibility:

  • [*] Always
  • [ ] Sometimes
  • [ ] Rarely

Environment:

  • Platform: [STM32 Discovery Kit]
  • Build Environment Details: [STM32CubeIDE]
  • OS Version: [Windows 10]
  • Edge Impulse Version (Firmware): [Not sure]

image
image725×578 63.1 KB

#2

Hi @kenadis - my initial suggestion is to switch off any prints that you have and only print the time result

I am assuming you are measuring the time before an after calling “run_inference” - however if you call it with “debug = true” all your features and other things are printed, which eats a lot of time.

So, set “debug = false” and make sure nothing else is printed between your measurements

#3

@ivan Thanks for your suggestion.

In my current implementation i only have print function being executed after run_classifier().

After implementing your suggestion of setting debug to false. The Inference time reduces from 6200ms to 5800 ms now. But overall the inference time is still high compared to what was estimated during training and testing on the Studio.

Also I am curious to know why the run_classifier() function does not by default print the inference time.

Find attached screenshot of my code and print out results

image
image659×670 41.5 KB

image

#4

@kenadis apologies for longer replies - it is hard to say why this is happening - does it actually take this time? (like does it feel like 6 seconds)

The other thing I can think of is some interrupts perhaps?

I see that this is all a separate method - how does the overall firmware work?

#5

Thanks for your reply. I am still having same issues after 1 month.

My code Uses a timer interrupt to collected adc values every 100ms (10hz) to populate a feature vector. Once feature vector is equal to expected feature size, then it calls the process_m_inference function above.

As you can see in the code snippet shared I have deliberately put the another timer to count the time before and after the run_classifier() is called, to not include effect of other code. I would not need to do this if the run_classifier was actually tracking the inference time as expected.

And yes the wait time does feel like 6 seconds. You can see the gap in the plot below:

Graph_1
Graph_11875×1039 112 KB