Feature Request: Arduino Portenta Vision Shield Coding Simplification

#1

Since Edge is starting to work on Vision programming with the Arduino PortentaH7 and PortentaH7Lite, can we consider the audience.

This is a Pro Auduino but it is essentially a board for advanced Makers and Firms that don’t want to hire: Mechatronic, Electrical or Computer Science Engineers. The above Engineers could probably do the same as the $104 USD Portenta with $47-$63 USD Vision Shield(s) at a fraction of the cost using the $10 ESP32, PCB board design like https://jlcpcb.com/ and https://easyeda.com/ and lots of inexpensive components.

For the rest of us there is the Portenta.

@rjames

Can Edge Impulse put some planning into how to make the Vision code as simple as possible? First let’s get it working, but then can we look at other ways to make it easier.

Here is my version of the Tensorflow “Hello Sine Wave” example that has been made easier by putting most of the confusing code in the #include "model.h" and #include "rocksetta.h" repository folder here. Note: all my Portenta examples can be installed with the Arduino IDE searchable library type “Portenta” and look for portenta-pro-community-solutions

/*
 * b01_makerML_hello_world.ino
 * simplifying TensorflowLite Micro Machine Learning for all makers
 * 
 * 
 * 
 * By Jeremy Ellis 
 * Twitter @rocksetta
 * Website https://rocksetta.com
 * created Aug 20th, 2020 
 * Github    https://github.com/hpssjellis/my-examples-for-the-arduino-portentaH7/tree/master/m09-Tensoflow
 *  
 *  
 *  Hashtags
 *  #TensorflowLiteMicro
 *  #TensorflowLiteForArduino
 *  #TinyMY
 *  #MakerML
 */





#include "Arduino.h"
#include <TensorFlowLite.h>
#include "model.h"
#include "rocksetta.h"


/*================= Start Maker Area ======================================*/


int myCounter = 0;
int myLed = LED_BUILTIN;   //or 7 or 5 or LED_BUILTIN or LEDB

void setup() {
  
  Serial.begin(9600);
  pinMode(myLed, OUTPUT);
  modelSetup(model_tflite);  // name of the model in the tab model.h
  
}


void loop() {
   myCounter +=1;
    if (myCounter >= 360){
        myCounter = 0;
    }
    
    float x_val = myCounter * 3.14/180;  // degrees to radians
    float y_val = sin(x_val);

    // This uses your model to make a prediction
    float predicted = modelPredict(x_val);

    Serial.println("sin(" +String(x_val)+ ") = " + String(y_val) + "\t predicted: " + String(predicted) );


    // y=1 LED is fully on. The LED's brightness can range from 0-255.
    int brightness = (int)(127.5f * (predicted+1));

     #if  defined (CORE_CM7)  ||  defined (CORE_CM4)

         if (brightness <= 128){
              digitalWrite(myLed, HIGH);  // means off
         } else {
               digitalWrite(myLed, LOW);  // means on             
         }
     #else
         analogWrite(myLed, brightness);  // not on Porttenta
     #endif 



    delay(3); // slows the process down a bit to see the sine wave

  
}


/*================= End Maker Area ======================================*/

The above example is a much more Arduino friendly, Machine Learning Model version of the TensorflowLite example hello_world.ino that you can find here here

Vision Machine Learning Models are a fair bit more complex than using Tensorflow to make a sine wave, but the point of this request is that we try to make the Edge Impulse Portenta Vision examples as user friendly as possible.

I am willing to help test, debug and make suggestions etc if needed.

Good luck.

#2

Hi, so we have a long standing ticket that I hope to get to with the embedded team this year, with an API that abstracts away the built-in drivers and give you something like this:

// need to make it clear that this runs on the same thread as the run_forever
void on_classification_changed(const char *event, float confidence, float anomaly_score) {
    if (strcmp(event, "unknown") == 0) {
        // turn LED on
    }
}

int main() {
    // optionally set up some smooth parameters or thresholds when you call this function
    ei_classifier_run_forever(&on_classification_changed);
}

So three lines to integrate on any fully supported dev board, abstracting away the nitty gritty setup, and just let you focus on the application. I think it’s along the lines of what you’re thinking.

#3

Thanks so much @janjongboom, that is so exciting. :100:

Simplification is important, simplification with flexibility and still some power is where it gets difficult. Good luck to the Edge Impulse team.

As always I am willing to test run draft code especially if it is for the Arduino Portenta

#4

@rjames Not sure if you heard but the Arduino MBED core 1.6.1 just got released

#5

@Rocksetta Yep, we have Portenta PR open now, hope to have it soon in Studio !

1 Like
#6

Any chance one of the examples could use a grayscale OLED? I find it a must for testing / debugging.