Data Forwarder Auto Label?

@janjongboom I have almost figured things out. Thanks for your help Jan. Hopefully I can make a video soon. My present playlist about the Arduino PortentaH7 is here.

Latest issue, I am using the data forwarder which works fine, but I have just clued in that I can’t auto send the label with the data. Edge Impulse is expecting a batch of data that all works for the same label. Then change the label and upload the next batch. I am used to sending an xDataSet with a matching yDataSet of labels. Can Edge Impulse, using the data forwarder, accept a label with the data? Any suggestions for my use case?

Hi @Rocksetta, if you have the data and the label already then wrapping it in the JSON according to the acquisition format is by far the easiest. If you have some example I’m happy to put a script together to do so.

1 Like

That would be really useful. Can the script ignore the encryption keys just to test if dummy data is working?

I am just trying to do the absolute microcontroller basic program of having 8 generic digital inputs and 4 digital output labels. I can generate data from my Arduino like the following. Say the 8 inputs are temperature sensors triggered at a set temperature and the 4 outputs are turning on or off room heaters.

Training Data --> Training Target (labels)
[0,0,0,1,0,1,0,0],[1,0,1,0]
[0,1,0,1,0,1,0,0],[1,1,1,0]
[1,0,0,0,0,1,0,0],[1,0,0,0]

The data could also be formatted as input first then output, although the above would be easier to generate.

[0,0,0,1,0,1,0,0]
[0,1,0,1,0,1,0,0]
[1,0,0,0,0,1,0,0]

[1,0,1,0]
[1,1,1,0]
[1,0,0,0]

I have a longer explanation here.

This could probably be programmed much easier using traditional programming and if statements, but it is a good ML starting point for my Robotics students.

Note: These labels (target data) are digital but in practice the output from the NN would be floats.

Thanks for the help @janjongboom. It makes much more sense now that I understand that each label gets it’s own JSON file and you can easily upload multiple JSON files.

Each of my JSON files looks like this

{"protected":{"ver":"v1","alg":"none","iat":1603914126},"signature":"0","payload":{"device_type":"TEST","interval_ms":10,"sensors":[{"name":"sensor1","units":"N/A"}],"values":[0,0,0,1,0,0,1,0]}}

which in expanded format looks like and can be explained at https://docs.edgeimpulse.com/reference#data-acquisition-format

{
   "protected":{
      "ver":"v1",
      "alg":"none",
      "iat":1603914126
   },
   "signature":"0",
   "payload":{
      "device_type":"TEST",
      "interval_ms":10,
      "sensors":[
         {
            "name":"sensor1",
            "units":"N/A"
         }
      ],
      "values":[
         0,
         0,
         0,
         1,
         0,
         0,
         1,
         0
      ]
   }
}

What is really nice here, is that the encryption and device name is not relevant just for testing since I am uploading using the dashboard data acquisition part of the website.

Next step, looking at using the C++ library or the Arduino Library on an Arduino, on windows and Ubuntu. Windows will need the long file name issue solved.

See forum here and here

1 Like

Ok, the “platform.local.txt” file issue to fix long file names in windows using the Arduino regular IDE was not a big deal. I was thinking I was going to have to edit the file for all my includes, but it really is just taking this file and putting it in the main arduino folder for in my case mbed 1.3.0 right beside the file called platform.txt.

The only really confusing part is that AppData is a hidden directory and the file for me was in:

C:\Users\MYUSERNAME\AppData\Local\Arduino15\packages\arduino\hardware\mbed\1.3.0

So on windows just go to “users” find yourself and the type “/AppData” without the qoutes that will get you to the Local folder and you can find your Arduino board after that.

The file you need to save is here

By @aurel new platform.local.txt

Just check that in windows it actually was saved with the .txt ending, at first mine didn’t.

Next issue. When using the Arduino Library, which example program to use. I think all the programs are going to run the newly generated model, just not all the programs are going to make sense running your new model if you are using custom data inputs and outputs. I will have to look into it.

Next issue. When using the Arduino Library, which example program to use. I think all the programs are going to run the newly generated model, just not all the programs are going to make sense running your new model if you are using custom data inputs and outputs. I will have to look into it.

Use the static buffer example for custom models.

@janjongboom, I want to start making some videos about my awesome experience with EdgeImpulse soon. Is there any chance of bundling the platform.local.txt file in its own folder with the arduino library download. Makes most sense there. Perhaps in a folder called “for windows users” with a readme.txt file that explains where to put it. I never know if changes like this are easy or ridiculously difficult to do.

@Rocksetta the underlying issue is almost fixed in Arduino - so I hope to see this live in the IDE and CLI tools very soon. See https://github.com/arduino/arduino-cli/pull/961

1 Like