Large Accuracy Differences between MobileNet V1 and V2?

I’m working on training a 2-class image classification model. The process I’m following is exactly similar to the Plants vs Lamps tutorial, except using two image classes instead of three. I follow the steps in the tutorial to load images and process them into 96x96 square images.

I’ve been testing out the accuracy of the multiple base models available during the Transfer Learning step. With each test I select a new base model from the list (e.g. MobileNetV1 96x96 0.25, MobileNetV2 96x96 0.35, etc.) and click ‘Start Training’.

The accuracy results of the models are as follows:

MobileNetV1 96x96 0.25 - 87%
MobileNetV1 96x96 0.2 - 57%
MobileNetV1 96x96 0.1 - 62%

MobileNetV2 96x96 0.35 - 98.3%
MobileNetV2 96x96 0.1 - 97%
MobileNetV2 96x96 0.05 - 97%

There is a notable performance difference between the V1 and V2 MobileNet models. I’m happy with the V2 performance but curious why it would beat out V1 so much. It’d help to get some clarity on what is happening so I can make sure its not somethign I’m doing on my end.

Is this difference typically expected?

Hey rhammell,

Generally I think this is expected. The different pre-trained mobilenets we offer make tradeoffs in terms of size and inferencing time against achievable accuracy.

The results you recorded look to track pretty closely with this, where the smallest models (e.g. MobileNetV1 96x96 0.1) having the lowest accuracy. The improvement from v1 to v2 is likely both due to improvements in the v2 model architecture, but also the generally larger sizes of our v2 based pre-trained models.

For reference I found a good summary of the improvements in MobileNetV2 here:

1 Like