FOMO Training with GPU

Help
,
#1

Question/Issue:
Hi, I am using a Nicla Vision for Object Detection. I have 5000 images from the Coco dataset (for testing, will be 75000) which encompasses 5 classes and I am trying to train the model with a GPU but it returns this error:

Node: ‘model/block_2_expand/Conv2D’
OOM when allocating tensor with shape[128,48,160,160] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node model/block_2_expand/Conv2D}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.
[Op:__inference_train_function_502751]
Attached to job 19331449…
Application exited with code 1
Job failed (see above)

The model was generated successfully on CPU but took a very long time. When training with GPU, is the training occurring locally or via Edge Impulse’s servers?

Either way, any advice on training the models with GPU would be greatly appreciated.

Project ID:

Context/Use case:
Object Detection. The objective is to compare a model trained on large datasets (Coco) vs trained on smaller, custom datasets.

#2

Hi @Fun_username

You will need to do that in an enterprise trial, and we will need a company email to approve that. Please create an account with a real email (not a temporary one)

Please see our privacy policy, we do not allow temporary email accounts to use our services as that can put us at risk of attack:

  • To send periodic emails — the email address you provide may be used to send you information, notifications that you request about changes to topics or in response to your username, respond to inquiries and/or other requests or questions, or to alert you of updates to protect your system from new types of online attacks and to send periodic emails containing information relevant to your account and the products you use.

@Fun_username please provide me with a correct email via pm so we can update your account.

Anyone else reading this with a temporary mail please review our policy and contact - Contact Sales

Best

Eoin