I apologize if this is largely a result of my ignorance w.r.t. how pre-trained models work, but I’ve been playing around with the BYO model feature. Just as a test, I exported the data for an audio classification block and then trained it locally. I then uploaded the trained ML block and attempted to run the same training and validation data through a training session. I was expected the loss to be extremely low right off the bat, since in theory the model should be already optimized for that data, but that wasn’t the case. The accuracy was roughly 50/50, as though the model had never been trained (by comparison, the uploaded model had a loss of around 0.06). What am I missing?
@janjongboom any chance you could look at this?
Can you share the project ID please? We can take a look.
Project is 64474. Thanks!
Anything more on this?
I’ve had to move on, so that project may no longer be in a useful state. If someone is able to look at this, please let me know and I can try to put things back the way they were.
Recently I have started to work more with BYOM and when there is a mistake made like skipping this step then you can get strange results:
“Make sure you run the training pipeline once locally - as we download some things on demand. This is not possible in Edge Impulse as your block runs without a connection to the outside world.”
If you can try to set the project up again we can have a look, but hopefully its just a case of a missed step in the readme.
I’m not sure I’m totally following. I did train the data locally and then upload the model. At that point I retrained the model online, expecting to see it quickly converge. Instead, the online training seemed to almost start from scratch, despite having the benefit of the pretrained model.
@jefffhaynes We’ve now split this up in two, hopefully that resolves your confusion:
- BYOM - upload a locally trained model and deploy via Edge Impulse. Upload your model via the UI (Dashboard > Upload your model ) and then deploy as usual, or do it all w/o the UI via the Python SDK. ← this seems to match with what you try to achieve.
- BYOA (Bring your own architecture) - the flow where you push your training pipeline, and we can retrain in the Studio (previously known as BYOM ).
Already started using it, thanks!