Error log at training time

Help
#1

Hi all. I get the following errors at training time (embeddings). Any clue for the culprit?

Converting TensorFlow Lite float32 model…
Converting TensorFlow Lite int8 quantized model…
Loading data for profiling…
Loading data for profiling OK

Creating embeddings…
[ 0/3747] Creating embeddings…
[3400/3747] Creating embeddings…
[3747/3747] Creating embeddings…
/app/keras/.venv/lib/python3.10/site-packages/sklearn/decomposition/pca.py:789: RuntimeWarning: invalid value encountered in divide
self.explained_variance_ratio = self.explained_variance_ / total_var
/app/keras/.venv/lib/python3.10/site-packages/sklearn/manifold/_t_sne.py:1032: RuntimeWarning: invalid value encountered in divide
X_embedded = X_embedded / np.std(X_embedded[:, 0]) * 1e-4

#2

Hi @Cramredan - I’ll take a look at this!

#3

OK, it seems like that error is from trying to create the data explorer visualization on the learning block page after training. A divide by zero may mean that there is no variance in the embeddings. Without more information, like the project ID so I can take a look, it’s tough to say what is going on.

Here are a few things you can check:

  1. Check the data explorer on the learning block page: is the projection blank, a single point, or otherwise malformed?
  2. Check the model’s training/validation results: are accuracy and loss reasonable, and does the confusion matrix show predictions across multiple classes rather than one class only?
  3. Check the input data and DSP features: compare a few samples from different labels to confirm they contain distinct signals and produce different feature values.