Computer vision on the portenta using arduico C/C++

Hello,

Is it possible to write Arduino C/C++ code to run the computer vision model? If so, any good examples? Or is the recommended method for doing this to use openMV and python (there seams to be more examples of this) ?

The reason for the request is that i like the Arduino C/C++ LORA library…thinking about it, can you use both? i.e. use lora module to send the LORA messages on one core and python and the model on the other core?

Hi @kelter101757,

@janjongboom wrote a project using the Portenta and LoRaWAN: https://github.com/edgeimpulse/example-portenta-lorawan

It uses our C++ library so you can write your own Arduino code.
Let me know if that helps,

Aurelien

2 Likes

Salut @aurel,

We attented the TTN conference last time when the lorawan example was demonstrated. Once I arrived at home after the show, I succeeded in building properly the firmware for the lora portenta vision shield .
Unfortunately, with the actual arduino-build script on example-portenta-lorawan, I do not arrive at compiling for our new training model on Edge Impulse.
Our use case in the city of Leuven is where we want to use TTNv3 for trash detection at 50 places. I had hoped the script still worked as it performed well the first time in September. Maybe some update has been done on your side that messed things up.

Here are my arduino-cli outcomes:

> $ arduino-cli version
> arduino-cli.exe alpha Version: 0.19.3 Commit: 12f1afc2 Date: 2021-10-12T10:15:19Z
$ arduino-cli core list
ID                       Installed Latest Name
arduino:mbed_edge        2.5.2     2.5.2  Arduino Mbed OS Edge Boards
arduino:mbed_nano        2.5.2     2.5.2  Arduino Mbed OS Nano Boards
arduino:mbed_nicla       2.5.2     2.5.2  Arduino Mbed OS Nicla Boards
arduino:mbed_portenta    2.5.2     2.5.2  Arduino Mbed OS Portenta Boards
arduino:mbed_rp2040      2.5.2     2.5.2  Arduino Mbed OS RP2040 Boards
arduino:samd_beta        1.6.25    1.6.25 Arduino SAMD (32-bits ARM Cortex-M0+) Beta Boards
arduino:samd             1.8.11    1.8.11 Arduino SAMD Boards (32-bits ARM Cortex-M0+)
sensebox:samd            1.3.3     1.3.3  senseBox SAMD Boards
SODAQ:avr                1.1.1     1.1.1  SODAQ AVR Boards
SODAQ:samd               1.6.11    1.6.11 SODAQ SAMD (32-bits ARM Cortex-M0+) Boards
STMicroelectronics:stm32 2.1.0     2.1.0  STM32 boards groups (Board to be selected from Tools submenu 'Board part number')
arduino:mbed             2.5.2     2.5.2  [DEPRECATED] [DEPRECATED - Please install standalone packages] Arduino Mbed OS Boards
$ arduino-cli board list
Port  Protocol Type              Board Name                        FQBN                               Core
COM3  serial   Unknown
COM40 serial   Serial Port (USB) Arduino Portenta H7 (M7 core)     arduino:mbed:envie_m7              arduino:mbed
               Serial Port (USB) Arduino Portenta H7 (M7 core)     arduino:mbed_portenta:envie_m7     arduino:mbed_portenta
               Serial Port (USB) Arduino Portenta H7 (ThreadDebug) arduino:mbed:envie_m7_thread_debug arduino:mbed
$ arduino-cli lib list
Name                           Installed Available Location              Description
Adafruit_BusIO                 1.7.2     -         LIBRARY_LOCATION_USER -
Adafruit_FONA_Library          1.3.8     -         LIBRARY_LOCATION_USER -
Adafruit_GFX_Library           1.10.7    -         LIBRARY_LOCATION_USER -
Adafruit_MQTT_Library          2.1.0     -         LIBRARY_LOCATION_USER -
Adafruit_SCD30                 1.0.5     -         LIBRARY_LOCATION_USER -
Adafruit_SleepyDog_Library     1.4.0     -         LIBRARY_LOCATION_USER -
Adafruit_SSD1306               2.4.3     -         LIBRARY_LOCATION_USER -
Adafruit_Unified_Sensor        1.1.4     -         LIBRARY_LOCATION_USER -
Arduino_Pro_Tutorials          1.0.4     -         LIBRARY_LOCATION_USER -
BME680                         1.0.10    -         LIBRARY_LOCATION_USER -
BSEC_Software_Library          1.6.1480  -         LIBRARY_LOCATION_USER -
FlashStorage                   1.0.0     -         LIBRARY_LOCATION_USER -
MKRWAN                         1.0.13    1.1.0     LIBRARY_LOCATION_USER Support library for MKR WAN 1300/1310
MKRWAN_v2                      1.3.2     -         LIBRARY_LOCATION_USER -
PCF8523                        1.0.0     -         LIBRARY_LOCATION_USER -
RTClib                         1.14.1    -         LIBRARY_LOCATION_USER -
SenseBoxMCU-Lib-master         1.4.1     -         LIBRARY_LOCATION_USER -
SparkFun_SCD30_Arduino_Library 1.0.12    -         LIBRARY_LOCATION_USER -
ThingSpeak                     2.0.1     -         LIBRARY_LOCATION_USER -
TinyGPSPLUS                    1.0.2     -         LIBRARY_LOCATION_USER -
TinyWireM                      1.1.0     -         LIBRARY_LOCATION_USER -
USBHost                        1.0.5     -         LIBRARY_LOCATION_USER -
WiFi101                        0.16.1    -         LIBRARY_LOCATION_USER -

At the beginning of the running script and after 10’ of ‘Still building’ the result is:

$ sh arduino-build.sh --build
arduino-build.sh: line 20: rev: command not found
arduino-build.sh: line 24: ((: exe alpha Version: 0: syntax error in expression (error token is “alpha Version: 0”)
arduino-build.sh: line 30: ((: exe alpha Version: 0: syntax error in expression (error token is “alpha Version: 0”)
arduino-build.sh: line 57: ((: exe alpha Version: 0: syntax error in expression (error token is “alpha Version: 0”)
Building firmware-arduino-portenta
Still building…
Flag --build-properties has been deprecated, please use --build-property instead.
Still building…
Still building…
Still building…

C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:225:7: error: ‘DigitalOut’ in namespace ‘mbed’ does not name a type
mbed::DigitalOut led_red(LED_RED);
^~~~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:226:7: error: ‘DigitalOut’ in namespace ‘mbed’ does not name a type
mbed::DigitalOut led_green(LED_GREEN);
^~~~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:227:7: error: ‘DigitalOut’ in namespace ‘mbed’ does not name a type
mbed::DigitalOut led_blue(LED_BLUE);
^~~~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp: In member function ‘void EiDevicePortenta::set_state(tEiState)’:
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:236:9: error: ‘led_red’ was not declared in this scope
led_red.write(1);
^~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:237:9: error: ‘led_green’ was not declared in this scope
led_green.write(1);
^~~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:237:9: note: suggested alternative: ‘degrees’
led_green.write(1);
^~~~~~~~~
degrees
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:238:9: error: ‘led_blue’ was not declared in this scope
led_blue.write(1);
^~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:238:9: note: suggested alternative: ‘_glue’
led_blue.write(1);
^~~~~~~~
_glue
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:242:9: error: ‘led_red’ was not declared in this scope
led_red.write(1);
^~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:243:9: error: ‘led_green’ was not declared in this scope
led_green.write(0);
^~~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:243:9: note: suggested alternative: ‘degrees’
led_green.write(0);
^~~~~~~~~
degrees
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:244:9: error: ‘led_blue’ was not declared in this scope
led_blue.write(1);
^~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:244:9: note: suggested alternative: ‘_glue’
led_blue.write(1);
^~~~~~~~
_glue
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:249:13: error: ‘led_red’ was not declared in this scope
led_red.write(0);
^~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:250:13: error: ‘led_green’ was not declared in this scope
led_green.write(1);
^~~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:250:13: note: suggested alternative: ‘degrees’
led_green.write(1);
^~~~~~~~~
degrees
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:251:13: error: ‘led_blue’ was not declared in this scope
led_blue.write(1);
^~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:251:13: note: suggested alternative: ‘_glue’
led_blue.write(1);
^~~~~~~~
_glue
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:254:13: error: ‘led_red’ was not declared in this scope
led_red.write(1);
^~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:255:13: error: ‘led_green’ was not declared in this scope
led_green.write(1);
^~~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:255:13: note: suggested alternative: ‘degrees’
led_green.write(1);
^~~~~~~~~
degrees
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:256:13: error: ‘led_blue’ was not declared in this scope
led_blue.write(1);
^~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:256:13: note: suggested alternative: ‘_glue’
led_blue.write(1);
^~~~~~~~
_glue
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp: In function ‘void ei_printf(const char*, …)’:
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:323:5: error: ‘va_start’ was not declared in this scope
va_start(args, format);
^~~~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:323:5: note: suggested alternative: ‘stat’
va_start(args, format);
^~~~~~~~
stat
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:325:5: error: ‘va_end’ was not declared in this scope
va_end(args);
^~~~~~
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:325:5: note: suggested alternative: ‘rand’
va_end(args);
^~~~~~
rand
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp: At global scope:
C:\Users\wim\Desktop\example-portenta-lorawan-master\src\ingestion-sdk-platform\portenta\ei_device_portenta.cpp:357:7: error: ‘Stream’ in namespace ‘mbed’ does not name a type
mbed::Stream* ei_get_serial() {
^~~~~~
Still building…
Still building…
Still building…
Still building…

Error during build: exit status 1

Any clue was the cause is of the build error when compiling the *.cpp files files? Very awkward because the first time all went smoothly, but now ?

I would really appreciate if you could deal with this one as we really want to work out this EI and lorawan use case.

Bàv, Wim

Hi @wdebbaut,

This firmware was written for mbed@1.3.1 you’re currently on 2.5.2. Can you try downgrading.

arduino-cli core install arduino:mbed@1.3.1

Then rebuilding again.

Splendid James,

after downgrading to arduino:mbed@1.3.1, see the result:

I always thought one should have the latest firmware :wink:
I can keep on working now with our research project: trash detection with the use of lorawan.

Have a nice sunday.

2 Likes

I’m glad you got it working.

Yes you’re right, the firmware @aurel shared was an demo example created 9 months ago. Which should get you set up [for now]. The good news is that we have Portenta H7 edge impulse firmware coming soon with support for all the sensors for the latest mbed. Stay Tuned!

Indeed good news @rjames !

That the portenta H7 will appear as a firmware deployment in the studio.
In any case, as for now, I am still continuing the Arduino deployment with mbed1.3.1 on our lorawan network.
As soon as the Portenta H7 firmware appears, I will try to get the lora vision shield working as well like in the portenta-lorawan-example previously. Awesome use cases with the LPWAN capabilities.

Hear from you later.

3 Likes

Awesome @wdebbaut that you are using the Portenta with LoRa vision shield, I am also working in this area lately but with Helium not TTN. With Version 2.6.1 of the Arduino MBED core I am hoping things move a bit more smooth for the Portenta.

Hello @wdebbaut,

FYI, we just officially released the Arduino Portenta H7 with full support: https://www.edgeimpulse.com/blog/announcing-official-support-for-the-arduino-portenta-h7
I thought you might be interested. Feel free to ping me for any question.

Regards,

Louis

Thanks Louis,
we will definitely try out this official way of supporting the Portenta H7 and lorawan shield.

One more question: in your lorawan example on Github, the lorawan uplink is triggered by a change in image classification.
Is there any code that you can provide me with a constant upload (say every 3 minutes) of the inference?
In the snippet of ei_main.cpp code there is a function:

void event_changed(const char* event) {
ei_printf("\tevent changed: %s\n", event);
memcpy(last_event, event, 31);
is_event_changed = true;
}

Do you have an example with not an event triggered but a constant print out of the inference after the OTAA join ?

Have a nice day?