EdgeImpulse "Version Control" for software changes

Feature requests
#1

Edgeimpulse Feature Request, Version Control

The microcontroller industry as a whole faces the challenge of version control for machine learning (ML) and IoT applications, and EdgeImpulse is no exception. While we may not have control over the entire industry, we can explore ways to address this issue within our own platform. Specifically, I suggest implementing a type of ML version control similar to what is used in web development.

For instance, when I create an ML webpage in JavaScript, I always include a specific NPM (node package manager) version to ensure compatibility, instead of relying on the most modern version.

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"> </script> 

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.0.0"> </script>

This technique allows webpages written years ago to still run without issues. Maintainers can change the version to the newest version to see if the page still runs. It is a bit of a catch-22, since the old page will not have any of the new security updates, but the new version might break the page.

Over the past three years, EdgeImpulse has only deprecated my work once, when Arduino made a change to the Portenta Camera library. EdgeImpulse promptly responded with a change to the outputted Arduino export, which was commendable. However, as IoT and ML devices become more popular, the risk of devices becoming unsupported due to OTA (over the air) upgrades to newer versions increases.

It’s surprising that the industry hasn’t addressed this issue earlier, probably because their are so many players involved with production of one product. I believe that PlatformIO has a method of stating the library version to use for each build, which EdgeImpulse could explore. Unfortunately the Arduino IDE has a manual library upgrade system which I don’t think they will be changing anytime soon. By adopting a version control system for ML models, EdgeImpulse could protect itself from similar complaints in the future.

EdgeImpulse has done a good job in maintaining compatibility with updates so far, but it’s important to address the issue of version control as the industry evolves. By implementing a version control system for ML models, we can ensure that devices using EdgeImpulse technology continue to function as intended and avoid becoming obsolete due to updates.

Does anyone have any opinions or is this something that has already been solved by some other method?

#2

Hello @Rocksetta

This is all about MLOps. However, MLOps for tinyML is still a ‘work in progress’. I suggest reading Edge Impulse’s First Academic Paper take a look at MLOps for Scaling TinyML and Weights & Biases

Inside a project, I use Edge Impulse together with Weights & Biases (wandb) where I use Weights & Biases for experiment tracking, dataset versioning, model versioning etc. Some of the issues you mentioned can be solved by combining wandb with EI (using the API).

1 Like
#3

@Joeri your suggestions definitely help.

I just looked at a few of my Edgeimpulse exported Arduino libraries and they do have a library.properties file which does list a version number, but the version seems based on my number of exports not on the Edgeimpulse build.

version=1.0.9
version=1.0.6
version=1.0.1
version=1.0.8
version=1.0.2

Yes my version is relevant information but I think the Edgeimpulse build is also relevant. As I said, I know the Portenta build was changed slightly to match an Arduino camera change, just wondering if Edgeimpulse can identify their build version so that developers can list a last working build to reduce deprecation issues.

Unlike most Arduino libraries where you can manually load an old version of the build, I think with Edgeimpulse you would need to save an old version of your build. Can @dansitu or @janjongboom weigh in on this?

The basic question is, can or does Edgeimpulse give a build version to their exported libraries?

#4

Hello,

I think I miss understood the question. You like to track the changes in the used libraries (the dependencies), not the model. Correct?

#5

I would like to be able to track any changes in the edgeImpulse exported libraries. As far as I know their has only been one change in the arduino Portenta exported libraries. Is there a way to know what version of the exported library we are presently at?

Thank goodness Edgeimpulse has not made many changes as each change deprecates my teaching code.

1 Like