Hi everyone
I have built a model that classified snapshots from a camera.
It takes in input jpg files from an external camera
My model is built, efficient (from the web site) and downloaded back to my RPI4
But what I see is that the RPI4 tries to classify an image from the camera, and I do not want to use the camera : I already have my jpg file to classify !
How can i do that ?
Any idea ?
Thank you in advance for your help / tips 
wisdoom
Hi @Wisdoom,
We have an example using our Python SDK: GitHub - edgeimpulse/linux-sdk-python: Use Edge Impulse for Linux models from Python
Check the Still Image example, it takes a jpg as an argument.
Aurelien
Hello @aurel
I did that : I have used the python example as a basis, and I have made some progresses
Indeed : The model is correctly downloaded to the RPI4, the example script runs with my image.
But now, I am facing some other issues
The symptom is simple : my image is not classified correctly by the RPI
I use the same image that I use to test my model in the studio.
The model can return âOpenedâ or âClosedâ , depending on my awning (Store banne in French) is closed or open.
Using studio, I have a very good accuracy, and the test image returns an âOpenedâ state. But using the RPI, it always returns a Closed State of 100%
So I checked and have seen an issue with the image resizing
In studio, I use the âLetter Boxâ resize option, to keep the full image even if it has a 16/9 ratio
But this is not implemented in the python SDK
So I have added a python method to resize it the same way
def resize2SquareKeepingAspectRation(img, size, interpolation):
h, w = img.shape[:2]
c = None if len(img.shape) < 3 else img.shape[2]
if h == w: return cv2.resize(img, (size, size), interpolation)
if h > w: dif = h
else: dif = w
x_pos = int((dif - w)/2.)
y_pos = int((dif - h)/2.)
if c is None:
mask = np.zeros((dif, dif), dtype=img.dtype)
mask[y_pos:y_pos+h, x_pos:x_pos+w] = img[:h, :w]
else:
mask = np.zeros((dif, dif, c), dtype=img.dtype)
mask[y_pos:y_pos+h, x_pos:x_pos+w, :] = img[:h, :w, :]
return cv2.resize(mask, (size, size), interpolation)
the method is called in the python script juste before calling
features, cropped = runner.get_features_from_image(img)
Instead, I use
# imread returns images in BGR format, so we need to convert to RGB
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
resized = resize2SquareKeepingAspectRation(img, 160, cv2.INTER_AREA)
# get_features_from_image also takes a crop direction arguments in case you don't have square images
features, cropped = runner.get_features_from_image(resized)
But the result is still the same : Closed at 100%
Do you know what could be the cause here ?
Thanks again for your help
Sylvain
Hello @Wisdoom ,
Could you save a version of your image resized with your method on your computer and upload it to your testing dataset and then run the inference using the live inference page from the studio please?
You can do something like that:
resized = resize2SquareKeepingAspectRation(img, 160, cv2.INTER_AREA)
cv2.imwrite("IMAGE_NAME.png", resized)
I just want to be sure itâs classified correctly in the studio and that your resized method does not do anything else too before investigating further.
Let me know
Regards,
Louis
Hello Louis
I did that, but using : cv2.imwrite(âresized.pngâ,cv2.cvtColor(resized, cv2.COLOR_RGB2BGR))
because the image has already been modified by BGR2RVB
The result is âslightlyâ better in studio, as I get a 50% Opened 50% Closed result
It seems to me that the produced image is less antialiased that the one produced by Studio, or not antialiased at all
I assumed that you used the float32 version of your model in your python script correct?
Because the quantized version gives much worse results from what I see in your project.
I can also see that your classes are strongly imbalanced, maybe try to rebalance your weights or try to train again with this parameter enabled.
Hopefully youâll have a more robust model and the results will be better locally.
Regards,
Louis
Seems that I am using the Quantized from what I see hereâŠ
Is there any way to switch that at RPI level ou should I build another version with the Studio ?
In parallel I will check this ârebalanceâ thing
But what scares me is the difference of behavior between local run results and Studio results. I was assuming it was really this image resizing thing that was in cause
The Model Testing on the studio always uses the float32.
You can also download your model using the Edge Impulse Linux CLI, this will download by default the float32 model and will select the right hardware optimizations based on your device.
edge-impulse-linux-runner --download mymodel.eim`
Regards,
Louis
Well, I did have build the float32 model, and have downloaded the new version to the RPI.
The result is still the same
python script.py ./mymodel.eim open.jpg
MODEL: /home/pi/./mymodel.eim
Loaded runner for "Wisdoom / StoreRecognition"
Result (65 ms.) Closed: 1.00 Opened: 0.00
And another test using the exact same image (downloaded from my test set)
python script.py ./mymodel.eim O_yourcamera_20220418-120000.jpg
MODEL: /home/pi/./mymodel.eim
Loaded runner for "Wisdoom / StoreRecognition"
Result (58 ms.) Closed: 1.00 Opened: 0.00
And in Studio (same image)
This 100% in the RPI run is really different
hi, I was using a nodeJS SDK for the same task - classifying the image file.
I was using sharp npm package to process image and edge-impulse-linux npm module.
So here is a code which I used. Create a package.json file:
{
"dependencies": {
"edge-impulse-linux": "^1.3.1",
"sharp": "^0.30.3"
}
}
then create a simple js file (I named it classify_image.js on my raspi):
const { LinuxImpulseRunner } = require("edge-impulse-linux");
const sharp = require('sharp');
// This script expects two arguments:
// 1. The model file
// 2. An image file.
// tslint:disable-next-line: no-floating-promises
async function main () {
try {
if (!process.argv[2]) {
console.log('Missing first argument (model file)');
process.exit(1);
}
if (!process.argv[3]) {
console.log('Missing second argument (image file)');
process.exit(1);
}
// Load the model
let runner = new LinuxImpulseRunner(process.argv[2]);
let imagefile = process.argv[3];
let model = await runner.init();
console.log(model.modelParameters.image_input_frames)
console.log('Starting the custom classifier for',
model.project.owner + ' / ' + model.project.name, '(v' + model.project.deploy_version + ')');
console.log('Parameters', 'freq', model.modelParameters.frequency + 'Hz',
'window length', ((model.modelParameters.input_features_count / model.modelParameters.frequency) * 1000) + 'ms.',
'classes', model.modelParameters.labels);
let resized = resizeImage(model, imagefile).then(resized => {
runner.classify(resized.features).then(classifyRes =>
console.log(classifyRes.result)
)
});
await new Promise(resolve => setTimeout(resolve, 1000));
} catch (ex) {
console.error(ex);
process.exit(1);
} finally {
process.exit(0);
}
}
async function resizeImage(model, data) {
// resize image and add to frameQueue
let img;
let features = [];
if (model.modelParameters.image_channel_count === 3) {
img = sharp(data).resize({
height: model.modelParameters.image_input_height,
width: model.modelParameters.image_input_width,
});
let buffer = await img.raw().toBuffer();
for (let ix = 0; ix < buffer.length; ix += 3) {
let r = buffer[ix + 0];
let g = buffer[ix + 1];
let b = buffer[ix + 2];
// tslint:disable-next-line: no-bitwise
features.push((r << 16) + (g << 8) + b);
}
}
else {
img = sharp(data).resize({
height: model.modelParameters.image_input_height,
width: model.modelParameters.image_input_width
}).toColourspace('b-w');
let buffer = await img.raw().toBuffer();
for (let p of buffer) {
// tslint:disable-next-line: no-bitwise
features.push((p << 16) + (p << 8) + p);
}
}
return {
img: img,
features: features
};
}
main().catch(err => {
console.error(err)
})
then on command line first install nodejs:
sudo apt install nodejs
and then install/run your classify_image.js:
npm install
node classify_image.js path_to_eim_file path_to_image
and here is output of execution on my mddel file and picture:
node classify_image.js car.eim test.1650462255.jpg â INT 18s îĄ 3.0.0 15:45:06
1
Starting the custom classifier for Ramil Israfilov / Car Parking Occupancy Detection - FOMO (v5)
Parameters freq 0Hz window length Infinityms. classes [ 'car' ]
{
bounding_boxes: [
{
height: 16,
label: 'car',
value: 0.9869808554649353,
width: 8,
x: 0,
y: 0
}
]
}
Thanks for that !
I indeed am using python SDK, but it does not implement the letterbox resizing as in Studio
I assume that I could change to node , but I would rather have it working in python, that could help to improve the python SDK by integrating the letterbox resizing (and furthermore, it could be linked to studio config, because the resizing method should be exactly the same as in Studio, and chosen automatically)
On which platform do you run that ?
I am trying your script, but I get an error on the ToBuffer() method of sharpâŠ
Donât know why⊠are there come dependencies here ? I use a raspbian latest 64bits
node: symbol lookup error: /home/pi/Node2/node_modules/sharp/build/Release/sharp-linux-arm64v8.node: undefined symbol: vips_fail_on_get_type
OK !
I have something that I prefer !
I have added some dependencies and itâs better :
node classify_image.js ../NodeScript/modelfile.eim ../O_yourcamera_20220418-120000.jpg
1
Starting the custom classifier for Wisdoom / StoreRecognition (v10)
Parameters freq 0Hz window length Infinityms. classes [ 'Closed', 'Opened' ]
Before Resize
{
classification: { Closed: 0.9999607801437378, Opened: 0.00003926035788026638 }
}
The classification is still incorrect but at least I do not have a 100% on Closed state
I run it on RaspberrPi 4 with raspbian OS lite 32-bit (armv7):
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.10.103-v7l+ #1529 SMP Tue Mar 8 12:24:00 GMT 2022 armv7l GNU/Linux