UnicodeDecodeError when "Splitting data into training and validation sets"

Report bugs
#1

When first attempting to train model, Training output shows:

Creating job... OK (ID: 1016858)

Job started
Splitting data into training and validation sets...
Traceback (most recent call last):
  File "/home/train.py", line 333, in <module>
    main_function()
  File "/home/train.py", line 187, in main_function
    mode, RANDOM_SEED, dir_path, 0.2)
  File "./resources/libraries/ei_tensorflow/training.py", line 22, in split_and_shuffle_data
    Y = ei_tensorflow.utils.load_y_structured(dir_path, 'y_train.npy', len(X))
  File "./resources/libraries/ei_tensorflow/utils.py", line 7, in load_y_structured
    Y_structured_file = json.loads(file.read())
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position 0: invalid start byte

Application exited with code 1 (Error)

Job failed (see above)

Project details:
box detection
250 pics in 320x320px
1 type of label

#2

@idohalamit,

Could you try to change the Processing Block (under the Create Impulse view in the studio) from Raw Data to Image please. Let me know if it solves your issue.

Regards

#3

I changed that, then I generated features again (I didn’t realize before, but the features were originally wrong) and “Training output” now shows:

Creating job... OK (ID: 1018537)

Copying features from processing blocks...
Copying features from DSP block...
Copying features from DSP block OK
Copying features from processing blocks OK

Job started
Splitting data into training and validation sets...
Splitting data into training and validation sets OK
Traceback (most recent call last):
  File "/home/train.py", line 333, in <module>
    main_function()
  File "/home/train.py", line 195, in main_function
    SPECIFIC_INPUT_SHAPE)
  File "./resources/libraries/ei_tensorflow/training.py", line 185, in get_datasets
    train_dataset = get_dataset_standard(X_train, Y_train)
  File "./resources/libraries/ei_tensorflow/training.py", line 118, in get_dataset_standard
    output_shapes=(tf.TensorShape(X_values[0].shape), tf.TensorShape(Y_values[0].shape)))
AttributeError: 'dict' object has no attribute 'shape'

Application exited with code 1 (Error)

Job failed (see above)
#4

Hey,

You would need to select the Object Detection (Images) in your training block from the Create Impulse view instead of Transfer Learning (Images). :smiley:

Also note that at the moment, only RGB is supported by the Object Detection learning block.

Regards,

Louis

#5

Thanks.
Is there any way to get around this? maybe upsampling to RGB?

#6

I am not 100% sure but if in the Image page you select RGB instead of greyscale, I think it will generate RGB features (even if your image is greyscale). It is a bit of loss in the performance but the MobileNetV2 SSD FPN-Lite 320x320 model that we are using needs to use RGB parameters.

I think @dansituwas considering adding other pre-trained model in the future but I have no ETA at the moment. If you are interested in knowing more about the Object Detection models, you can have a look at this github repository where you can see some of the existing pre-trained model: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md

Regards,

Louis

#7

Thank you so much! The training seems to be working, even though I’m hitting incredibly low scores at the moment.

1 Like
#9

image

1 Like
#10

I’m not sure whether to open a new topic, but now when I try to deploy the model, the OpenMV option does not appear.

image
image866Ă—547 43.4 KB

I tried downloading the .lite file, but it raises issues in the code when loading or classifying.

#11

@idohalamit OpenMV is only available for image models. What does your impulse look like?

#12

Bounding box detection.

image
image1490Ă—652 119 KB

Is it not supported?

#13

Hi @idohalamit, no object detection is currently not supported on any Cortex-M device (including the OpenMV Cam).

#14

Just to clarify because I’m not entirely familiar with the terminology, models that consist of finding bounding boxes of a single object are not supported?

#15

@idohalamit,

they are supported by Edge Impulse, but as you can see on the Object Detection page near On-device performance, this model takes about 3.6MB of RAM.
This is why you cannot run the Object Detection models on Cortex-M devices.

However, it does work on Raspberry Pis or on Jetson Nanos

I hope this clarifies your understanding.

Regards,

Louis

#16

Edit: I solved the memory problem, but it’s still unsupported.

#17

Correct, anything with bounding boxes is not supported on the OpenMV (at the moment).