I’ve been scratching my head on this over the past days, and can’t seem to figure it out. Any help would be greatly appreciated.
My setup currently runs a Raspberry Pi 4 model B, with a Raspberry Pi camera model 2. The OS I installed is Raspberry Pi 4 Legacy, 64 bit Full (Bullseye). I did not have a screen available, so had to set up using SSH and VNC, for reference in case this effects anything.
Since this is an M2 camera with RP4, I had to change the config.txt to include dtoverlay=imx219 in order to confirm the camera working using the command libcamera-hello. I was then able to use libcamera-still -o image.jpg to save the image taken onto home directory. However, I was not able to use raspistill and discovered libcamera stack replaced raspistill, and another stack named picamera2 was introduced as well. I eventually built my model and deployed it as “Linux (AARCH64)” since my intent is to have python scrips detect the results and respond accordingly (flash LED, record encounter, etc.).
I eventually installed the edge impulse SDK onto the RP, and was able to successfully run the model using edge-impulse-linux-runner. This is where my problems begin. Since I want the model to be ran within my own application in python, I referenced to tool provided within this website and eventually came across an example that grabs data from a webcam and classifies it in real-time.
The code can be found here: linux-sdk-python/examples/image/classify.py at master · edgeimpulse/linux-sdk-python · GitHub
When I try running this code, it seems like the example script is unable to find a the camera connected to the RP, even though the camera was being used correctly for use with the Edge Impulse model via command above. I tried to increase port range to 10, and used ls /dev/video* to list all video devices. The terminal showed the multiple ones ranging from 0 to 31. I then modified get_webcams() to the following:
def get_webcams():
port_ids = []
for port in range(32): # Increased range up to 31
print("Looking for a camera in port %s:" % port)
camera = cv2.VideoCapture(port)
if camera.isOpened():
ret, frame = camera.read()
if ret:
backendName = camera.getBackendName()
w = camera.get(3)
h = camera.get(4)
print("Camera %s (%s x %s) found in port %s " % (backendName, w, h, port))
port_ids.append(port)
else:
print("Camera in port %s could not capture frames." % port)
camera.release()
else:
print(f"Failed to open camera at port {port}.")
return port_ids
This resulted in: ModuleNotFoundError: No module named 'pyaudio'
I installed PortAudio Libraries and PyAudio and decided to test again to eventually receive: Exception: Cannot find any webcams
I wasn’t sure what exactly was going on, but it seems like the script was unable to successfully access or initialize any of the camera devices on the RP. I checked through the RP config and double checked all updates again. I then attempted to see if I could directly access the camera with the following:
import cv2
from picamera2 import Picamera2
from edge_impulse_linux.image import ImageImpulseRunner
def test_camera(device_path):
camera = cv2.VideoCapture(device_path)
if camera.isOpened():
ret, frame = camera.read()
if ret:
print(f"Successfully accessed {device_path}")
cv2.imshow("Test Frame", frame)
cv2.waitKey(0)
cv2.destroyAllWindows()
else:
print(f"Failed to read from {device_path}")
camera.release()
else:
print(f"Failed to open {device_path}")
test_camera('/base/soc/i2c0mux/i2c@1/imx219@10')
To which I obtained the device path from edge-impulse-linux-runner. The camera once again failed to open. I decided to switch things up and use the picamera2 library just to test if I could use the library’s configuration directly. I typed up the following code which was able to work:
from picamera2 import Picamera2
def capture_image():
picam2 = Picamera2()
picam2.configure(picam2.create_still_configuration())
picam2.start()
picam2.capture_file("test.jpg")
print("Image captured as test.jpg")
picam2.stop()
if __name__ == "__main__":
capture_image()
Seeing how with works, I was wondering if I could use OpenCV to show a preview, but received this error even after installing the required packages:
cv2.error: OpenCV(4.9.0) /io/opencv/modules/highgui/src/window.cpp:1255: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvNamedWindow'
I even tried to rebuild OpenCV with GUI Support manually with no success. I then moved on assuming I could capture the image using “picamera2”, but received an error: TypeError: Object of type ndarray is not JSON serializable.
After running into this type of problem before, even trying many things such as converting the image data to Base64, I knew it was time to ask for help. I don’t quite understand how edge-impulse-linux-runner configures the model and accesses the camera when examples to classify real-time data do not. Classifying and image itself works using the example provided under tools. Yet, I can’t seem to get this camera to work under any circumstance for real-time classification.
My questions would be, what type of data does the runner exactly receive or need to successfully start classifying real-time images/video? If this requires the configuration of M2 camera, then what exactly is being used to detect this camera? Is there an example I could learn from?
Thank you for your patience!