Raspberry integration test

robhazes · May 16, 2021, 9:08am

I am trying to run this integration example om my Pi4.

When I run step 8: “npm run build” I get errors. This is what the log shows:

0 info it worked if it ends with ok
1 verbose cli [ ‘/usr/local/bin/node’, ‘/usr/local/bin/npm’, ‘run’, ‘build’ ]
2 info using npm@6.14.13
3 info using node@v14.17.0
4 verbose config Skipping project config: /home/pi/.npmrc. (matches userconfig)
5 verbose run-script [ ‘prebuild’, ‘build’, ‘postbuild’ ]
6 info lifecycle edge-impulse-example-linux-with-twilio@1.0.0~prebuild: edge-impulse-example-linux-with-twilio@1.0.0
7 info lifecycle edge-impulse-example-linux-with-twilio@1.0.0~build: edge-impulse-example-linux-with-twilio@1.0.0
8 verbose lifecycle edge-impulse-example-linux-with-twilio@1.0.0~build: unsafe-perm in lifecycle true
9 verbose lifecycle edge-impulse-example-linux-with-twilio@1.0.0~build: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/pi/node_modules/.bin:/home/pi/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
10 verbose lifecycle edge-impulse-example-linux-with-twilio@1.0.0~build: CWD: /home/pi
11 silly lifecycle edge-impulse-example-linux-with-twilio@1.0.0~build: Args: [ ‘-c’, ‘tsc -p .’ ]
12 silly lifecycle edge-impulse-example-linux-with-twilio@1.0.0~build: Returned: code: 1 signal: null
13 info lifecycle edge-impulse-example-linux-with-twilio@1.0.0~build: Failed to exec build script
14 verbose stack Error: edge-impulse-example-linux-with-twilio@1.0.0 build: tsc -p .
14 verbose stack Exit status 1
14 verbose stack at EventEmitter. (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
14 verbose stack at EventEmitter.emit (events.js:376:20)
14 verbose stack at ChildProcess. (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
14 verbose stack at ChildProcess.emit (events.js:376:20)
14 verbose stack at maybeClose (internal/child_process.js:1055:16)
14 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
15 verbose pkgid edge-impulse-example-linux-with-twilio@1.0.0
16 verbose cwd /home/pi
17 verbose Linux 4.19.97-v7l+
18 verbose argv “/usr/local/bin/node” “/usr/local/bin/npm” “run” “build”
19 verbose node v14.17.0
20 verbose npm v6.14.13
21 error code ELIFECYCLE
22 error errno 1
23 error edge-impulse-example-linux-with-twilio@1.0.0 build: tsc -p .
23 error Exit status 1
24 error Failed at the edge-impulse-example-linux-with-twilio@1.0.0 build script.
24 error This is probably not a problem with npm. There is likely additional logging output above.
25 verbose exit [ 1, true ]

Any suggestions?

robhazes · May 16, 2021, 9:17pm

A new version of npm was available tonight and installing that by this command “npm install -g npm” fixed this error.

robhazes · May 16, 2021, 10:13pm

Almost there but get these errors when compiling:

pi@raspberrypi:~ $ npm run build

edge-impulse-example-linux-with-twilio@1.0.0 build
./node_modules/.bin/tsc -p .

example-linux-with-twilio/webserver-twilio.ts:70:13 - error TS2345: Argument of type ‘{ device: string | { name: string; id: string; }; intervalMs: number; }’ is not assignable to parameter of type ‘{ deviceId: string; intervalMs: number; }’.
Object literal may only specify known properties, but ‘device’ does not exist in type ‘{ deviceId: string; intervalMs: number; }’. Did you mean to write ‘deviceId’?

70 device: device,
~~~~~~~~~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:34 - error TS2345: Argument of type ‘(result: any, timeMs: any, imgAsJpg: any) => Promise’ is not assignable to parameter of type ‘(result: RunnerClassifyResponseSuccess, timeMs: number) => void’.

131 imageClassifier.on(‘result’, async (result, timeMs, imgAsJpg) => {
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:41 - error TS7006: Parameter ‘result’ implicitly has an ‘any’ type.

131 imageClassifier.on(‘result’, async (result, timeMs, imgAsJpg) => {
~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:49 - error TS7006: Parameter ‘timeMs’ implicitly has an ‘any’ type.

131 imageClassifier.on(‘result’, async (result, timeMs, imgAsJpg) => {
~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:57 - error TS7006: Parameter ‘imgAsJpg’ implicitly has an ‘any’ type.

131 imageClassifier.on(‘result’, async (result, timeMs, imgAsJpg) => {
~~~~~~~~

example-linux-with-twilio/webserver-twilio.ts:136:51 - error TS7006: Parameter ‘x’ implicitly has an ‘any’ type.

136 for (let b of r.bounding_boxes.filter(x => x.value >= 0.5)) {
~

node_modules/@types/socket.io/index.d.ts:20:68 - error TS7016: Could not find a declaration file for module ‘socket.io-parser’. ‘/home/pi/node_modules/socket.io-parser/index.js’ implicitly has an ‘any’ type.
Try npm install @types/socket.io-parser if it exists or add a new declaration (.d.ts) file containing declare module 'socket.io-parser';

20 import { Encoder as ParserEncoder, Decoder as ParserDecoder } from ‘socket.io-parser’;
~~~~~~~~~~~~~~~~~~

Found 7 errors.

Any suggestions?

aurel · May 18, 2021, 5:03pm

Hi @robhazes,

The gstreamer library used to capture camera feed has been updated in the latest edge-impulse-linux release, the camera.start() format has slightly changed.

Can you update the edge-impulse-linux and retry?

npm update -g edge-impulse-linux

Aurelien

robhazes · May 18, 2021, 10:07pm

Hi and thanks a lot for looking at this

I tried npm update -g edge-impulse-linux but got a lot of permission errors, so I ran:
sudo npm update -g edge-impulse-linux

Re-compiled and got more errors, here they are, hope it is useful:

pi@raspberrypi:~ $ npm run build

> edge-impulse-example-linux-with-twilio@1.0.0 build /home/pi
> tsc -p .

error TS5033: Could not write file '/home/pi/build/example-linux-with-twilio_new/webserver-twilio.d.ts': EACCES: permission denied, open '/home/pi/build/example-linux-with-twilio_new/webserver-twilio.d.ts'.

error TS5033: Could not write file '/home/pi/build/example-linux-with-twilio_new/webserver-twilio.js': EACCES: permission denied, open '/home/pi/build/example-linux-with-twilio_new/webserver-twilio.js'.

error TS5033: Could not write file '/home/pi/build/example-linux-with-twilio_new/webserver-twilio.js.map': EACCES: permission denied, open '/home/pi/build/example-linux-with-twilio_new/webserver-twilio.js.map'.

example-linux-with-twilio/webserver-twilio.ts:70:13 - error TS2345: Argument of type '{ device: string | { name: string; id: string; }; intervalMs: number; }' is not assignable to parameter of type '{ deviceId: string; intervalMs: number; }'.
  Object literal may only specify known properties, but 'device' does not exist in type '{ deviceId: string; intervalMs: number; }'. Did you mean to write 'deviceId'?

70             device: device,
               ~~~~~~~~~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:34 - error TS2345: Argument of type '(result: any, timeMs: any, imgAsJpg: any) => Promise<void>' is not assignable to parameter of type '(result: RunnerClassifyResponseSuccess, timeMs: number) => void'.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:41 - error TS7006: Parameter 'result' implicitly has an 'any' type.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                            ~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:49 - error TS7006: Parameter 'timeMs' implicitly has an 'any' type.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                                    ~~~~~~

example-linux-with-twilio/webserver-twilio.ts:131:57 - error TS7006: Parameter 'imgAsJpg' implicitly has an 'any' type.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                                            ~~~~~~~~

example-linux-with-twilio/webserver-twilio.ts:136:51 - error TS7006: Parameter 'x' implicitly has an 'any' type.

136             for (let b of r.bounding_boxes.filter(x => x.value >= 0.5)) {
                                                      ~

example-linux-with-twilio_new/webserver-twilio.ts:70:13 - error TS2345: Argument of type '{ device: string | { name: string; id: string; }; intervalMs: number; }' is not assignable to parameter of type '{ deviceId: string; intervalMs: number; }'.
  Object literal may only specify known properties, but 'device' does not exist in type '{ deviceId: string; intervalMs: number; }'. Did you mean to write 'deviceId'?

70             device: device,
               ~~~~~~~~~~~~~~

example-linux-with-twilio_new/webserver-twilio.ts:131:34 - error TS2345: Argument of type '(result: any, timeMs: any, imgAsJpg: any) => Promise<void>' is not assignable to parameter of type '(result: RunnerClassifyResponseSuccess, timeMs: number) => void'.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

example-linux-with-twilio_new/webserver-twilio.ts:131:41 - error TS7006: Parameter 'result' implicitly has an 'any' type.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                            ~~~~~~

example-linux-with-twilio_new/webserver-twilio.ts:131:49 - error TS7006: Parameter 'timeMs' implicitly has an 'any' type.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                                    ~~~~~~

example-linux-with-twilio_new/webserver-twilio.ts:131:57 - error TS7006: Parameter 'imgAsJpg' implicitly has an 'any' type.

131     imageClassifier.on('result', async (result, timeMs, imgAsJpg) => {
                                                            ~~~~~~~~

example-linux-with-twilio_new/webserver-twilio.ts:136:51 - error TS7006: Parameter 'x' implicitly has an 'any' type.

136             for (let b of r.bounding_boxes.filter(x => x.value >= 0.5)) {
                                                      ~

node_modules/@types/socket.io/index.d.ts:20:68 - error TS7016: Could not find a declaration file for module 'socket.io-parser'. '/home/pi/node_modules/socket.io-parser/index.js' implicitly has an 'any' type.
  Try `npm install @types/socket.io-parser` if it exists or add a new declaration (.d.ts) file containing `declare module 'socket.io-parser';`

20 import { Encoder as ParserEncoder, Decoder as ParserDecoder } from 'socket.io-parser';
                                                                      ~~~~~~~~~~~~~~~~~~


Found 16 errors.

npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! edge-impulse-example-linux-with-twilio@1.0.0 build: `tsc -p .`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the edge-impulse-example-linux-with-twilio@1.0.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/pi/.npm/_logs/2021-05-18T22_03_18_679Z-debug.log
pi@raspberrypi:~ $

aurel · May 19, 2021, 1:29pm

I just tried it on my RPi 4 and I had no issue.
What I would suggest to do is:

npm uninstall -g edge-impulse-linux --force
npm config set user root && sudo npm install edge-impulse-linux -g --unsafe-perm

Delete your existing twilio folder example and download it again from github, and run the npm install / npm run build commands.

Also, you can refer to our RPi4 installation guide if you want to start from scratch:

Aurelien

janjongboom · May 19, 2021, 5:31pm

I’m a bit puzzled on why you’d need to run npm install -g edge-impulse-linux. The example has its own version of the Node.js SDK for Linux, so nothing should change there. npm install in the example folder and then running npm run build should be enough…

robhazes · May 20, 2021, 12:23pm

Thanks a lot! The cause was that I was not running the commands from the proj.dir but from the folder above, sorry about that… Almost running now, just a camera problem. I use the Pi-cam and get this error message, see below. I am not sure what to change where…

pi@raspberrypi:~/example-linux-with-twilio $ node build/webserver-twilio.js twilio-example.eim
Starting the image classifier for Rob / test2_public_clone v2 (v10)
Parameters image size 320x320 px (3 channels) classes [ '40', '50' ]
Error: Multiple cameras found ("bcm2835-isp", "bcm2835-isp", "mmal service 16.1"), add the camera to use to this script (node build/webserver-twilio.js model.eim cameraname)
    at /home/pi/example-linux-with-twilio/build/webserver-twilio.js:59:19
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

robhazes · May 20, 2021, 1:12pm

Found the solution at the end of the demo video. Camera name hast to be added by parameter. Now it works and I can start playing

Should the cam resolution also be passed by parameters or has this error to be fixed in another way?

pi@raspberrypi:~/example-linux-with-twilio $ node build/webserver-twilio.js twilio-example.eim "bcm2835-isp"
Starting the image classifier for Rob / test2_public_clone v2 (v10)
Parameters image size 320x320 px (3 channels) classes [ '40', '50' ]
Using camera bcm2835-isp starting...
Error: Could not find resolution info for this device
    at GStreamer.start (/home/pi/example-linux-with-twilio/node_modules/edge-impulse-linux/build/library/sensors/gstreamer.js:61:19)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async /home/pi/example-linux-with-twilio/build/webserver-twilio.js:64:9

robhazes · May 22, 2021, 4:16pm

Anyone got this example working with the Pi-cam? I have no idea what to do about the resolution error, see post above.

janjongboom · May 24, 2021, 10:27am

@robhazes We automatically determine the camera size. There have been a few patches related to the Pi camera since the launch:

$ npm update edge-impulse-linux

From the example-linux-with-twilio folder will fix this. Will also update the repository!

robhazes · May 24, 2021, 11:40am

Thanks Jan! That was the trick to make it work!

And the correct parameter for the Pi cam is in my case “mmal service 16.1”

So the run command then looks like this:
pi@raspberrypi:~/example-linux-with-twilio $ node build/webserver-twilio.js twilio-example.eim "mmal service 16.1"

robhazes · May 27, 2021, 3:20pm

Is there an easy way to feed the deployed model with a pre-recorded video, instead of feeding it with the camera?

I searched through the /build/webserver-twilio.js file for places to add some code for this but was not able to find out.

Any suggestions?

robhazes · May 28, 2021, 8:16pm

When deployed to the Raspberry, is there an averaging filter active which can be adjusted / switched off, like in this example?

aurel · June 2, 2021, 2:36pm

Hi @robhazes,

This should be do-able to use a pre-recorded video, as long as you can convert frame to to jpeg. You will need to replace the camera.on('snapshot', async (data) function with your own code to read frames from your video.

Regarding the averaging filter, you could add that in the imageClassifier.on() function (https://github.com/edgeimpulse/example-linux-with-twilio/blob/master/webserver-twilio.ts#L131), keeping the history of the result for a few cycles and only emit the results if you have 3 similar detections in a row for instance.

Aurelien

janjongboom · June 3, 2021, 9:44am

@robhazes You can see exactly what we do with the classifier here: https://github.com/edgeimpulse/edge-impulse-linux-cli/blob/6000345ddc00ae81d2e078aff048e923c8f33e2a/library/classifier/image-classifier.ts#L50

You can replace the data here with a buffer from a video or a still image. In essence: 1) resize / crop image to right size, 2) grab raw pixels, 3) classify. Done.