NN CLASSIFIER Training Error occurs

I have gone through the process of creating of

  1. Creating a project
  2. Connecting a Nano 33 BLE Sense Board
  3. Acquiring data ( Training 51s and Test 51s ) :- Audio Data 16kHz
  4. Impulse Design
    Creating an Impulse MFCC and Neural Network (Keras)
    Feature Generation for MFCC
    NN Classifier :- at this point I start the training and the below error occurs

Perhaps it is a data issue. Probably not enough training data for an application, but the aim was just to quickly go through the process. I tried Raw data and the MFCC which both produced this error.

Please note there is an exception within the exception, hence the first exception should be the one of most interest.

Creating job… OK (ID: 197924)

Copying features from processing blocks…
Copying features from processing blocks OK

Job started
Splitting data into training and validation sets…
Splitting data into training and validation sets OK

Training model
Training on 32 inputs, validating on 8 inputs
Epoch 1/100
Traceback (most recent call last):
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/data_adapter.py”, line 1136, in catch_stop_iteration
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py”, line 1081, in evaluate
tmp_logs = test_function(iterator)
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py”, line 580, in call
result = self._call(*args, **kwds)
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py”, line 650, in _call
return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/function.py”, line 1665, in _filtered_call
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/function.py”, line 1746, in _call_flat
ctx, args, cancellation_manager=cancellation_manager))
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/function.py”, line 598, in call
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/eager/execute.py”, line 60, in quick_execute
inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence
[[node IteratorGetNext (defined at home/train.py:57) ]] [Op:__inference_test_function_1022]

Function call stack:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/train.py”, line 415, in
File “/home/train.py”, line 360, in main_function
X_train, X_test, Y_train, Y_test, len(X_train), classes)
File “/home/train.py”, line 57, in train_model
model.fit(train_dataset, epochs=100, validation_data=validation_dataset, verbose=2, callbacks=callbacks)
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py”, line 66, in _method_wrapper
return method(self, *args, **kwargs)
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py”, line 872, in fit
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py”, line 66, in _method_wrapper
return method(self, *args, **kwargs)
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py”, line 1088, in evaluate
callbacks.on_test_batch_end(step, logs)
File “/usr/local/lib/python3.7/contextlib.py”, line 130, in exit
self.gen.throw(type, value, traceback)
File “/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/data_adapter.py”, line 1153, in catch_stop_iteration
TypeError: unsupported operand type(s) for *: ‘int’ and ‘NoneType’

Application exited with code 1 (Error)

Job failed (see above)

Hi @PaulCreaser, I think this is because you don’t have enough data. The batch size is then larger than the amount of data you have and this error is thrown (the error is not very descriptive, @dansitu could you add something better here?).

Quick fix is too either 1) add more data, or 2) lower the window increase in the ‘Create impulse’ screen.

That is indeed a hopeless error message! I can actually tweak our code so that we don’t run into this issue at all.

This has now been fixed and will be deployed later today.

1 Like

Thanks. I can confirm the problem has been fixed.

Great to hear, thank you for the report!