When training models I often want help determining the optimum learning rate. Perhaps a pull down list of suggestions and/or past attempts listed with some feedback such as the final loss or the average rate of change of loss. Presently I just seem to be wasting resources trying lots of different learning rates and forgetting what I have already tried.
Jer
Hello @Rocksetta ,
This is something we’ve been been thinking of. I cannot tell you which shape it will take but in the meantime, for supported models, you can get a first estimation using the EON tuner and then fine tune the parameters by hand.
Regards,
Louis
Thanks @louis I haven’t used the EON tuner much as I thought it was more for when everything was working fine you just wanted things to get better. I typically hang around proof of concept and making quick models to see what works.
Speaking of which I don’t think FOMO is active yet with the EON tuner.
What I am starting to do is save an image of my confusion matrix, training cycles, learning rate, loss first and loss last. Hopefully after a few training runs I can make a better estimate of a learning rate.
FOMO needs more learning cycles , any idea if regular users can do a few more cycles than before?
