I have 40-50% unused CPU while seeing 60-100 ms inference speed in docker on both a RPi4 and Odroid N2+ which makes me believe I’m IO bound on the camera.
I’m using a See3Cam_CU30 from Econ systems and we leverage the onboard MJPEG codec when running on a Jetson Nano.
I’ve tried modifying the gst-launch-1.0 params but get an “unsupported format” error which, if I’m understanding the architecture of the edge-impulse-linux-runner is from another process which is picking up the .jpg files in /dev/shm/* and performing inference on them.
A few questions:
Any recommendations on exploring why I’m not using more CPU?
Is there docu on the edge-impulse-linux-runner ( if not, am I understanding the arch correctly ? )
Do the .jpgs in /dev/shm hold on to any memory? I would guess not, but perhaps use up inodes?
Should those get cleaned up after stopping the process?
just a datapoint but I hacked gstreamer.js to GST_DEBUG=4 and use this pipeline that works from terminal but fails from edge-impulse-linux-runner with gstreamer reporting a syntax error. I console.log the same as the spawn command and paste it into the terminal to test.
gst-launch-1.0 v4l2src device=/dev/video0
!
image/jpeg,width=640,height=480
!
jpegdec
!
tee name=t
t. ! queue ! videoconvert ! ximagesink
t. ! queue ! jpegenc ! multifilesink location=test%05d.jpg
The node profiler is suggesting that the resizeImage ( which calls sharp.resize ) is consuming the most cycles but it’s not clear to me why CPU usage is so low. I nice’d node and it didn’t make a difference. I’m still investigating.
Couple of questions :
Can I safely ignore the use of the word “classification” in documentation and code when working with fomo? My guess is that the docu hasn’t caught up yet?
I was looking for a standalone example with fomo but not finding anything specific to fomo…do I need to adapt existing examples? For example, using opencv python or c/c++ API to pull frames via v4l?
In the c++ standalone examples there is step to copy features into main.cpp … this is an enormous array of features when using fomo … am I barking up the wrong tree?
I feel like my best course of action is to the layers of abstraction so that I can determine what is preventing full use of the CPU.
Yes resizing is greedy indeed, especially when the input image is very large. If your camera support it, you can also try to grab the image at the desired size directly from the sensor. Or at least collect them with a low resolution size.
The image classification part is available in the code to be compatible with image classification projects too. Can you point me which part of the documentation is not clear to you so I can see how to improve it please?
thanks @louis
I see that sharp uses vips and is fast, but I would expect it to be even MORE greedy. I’ll post some summary information from node profiler for context.
Lowest res on this camera is 640x480.
I think, for example, it was mainly that the fomo detection was being performed in a classifier file : /usr/lib/node_modules/edge-impulse-linux/build/library/classifier/image-classifier.js vs image-object-detector.js… it’s not a big deal, I just wanted to be sure I was doing the right thing with fomo ( which seems like I am )