How to use multiple sets of test data on a trained impulse

Hello
I’m quite new to Edge Impulse so apologies if this has been answered already.

  1. On a trained impulse, I’d like to explore different sets of Test datasets to check model performance with different sources (e.g collected via different sensors than the original, or under different conditions). It wall also help me understand better model generalisability and robustness. At the moment I can upload the Test data files at the ‘Data Acquisition’ stage, along with Train(+validation) data files. My problem is that after training the model, and looking at how well it has worked against the Test data, I can’t find a way of bulk deleting the original Test dataset and uploading a new one. Individual Test data files can be deleted or disabled one-by-one, but can’t see a way of doing it en-masse. I don’t want to delete all Train/Validation data and have to retrain the model.

  2. After running the Test data through the model and obtaining the confusion matrix, is there also a simple way of downloading the Test prediction results for each input test file, e.g. as a single CSV file. Apologies but I can’t locate the menu option.

Many thanks in advance
Roy

Hi @rasanderson,

You can select and delete all samples in just the test set by going to your Data acquisition page, clicking the checkbox icon, and clicking the checkbox next to “Sample name.”

I do not believe there is a way to get the test results directly from Studio. This is a good idea, so I’ve filed an issue with the engineering team. If you wanted to script such testing, you could get sample IDs from your test set and then feed those into this API call (Classify sample) to run inference on the test set to get the accuracy programmatically.

Hi @shawn_edgeimpulse
Super - thank you very much for quick response. That’s really helpful and just what I was looking for.
Roy

1 Like