Need help with pytorch....cannot resume training

[vishalrao]

Does this stuff require you to be a maths/statistics genius?

I almost failed my maths and related subjects and barely scraped through back when I was studying lol so I probably can only dabble in this area as an end user that consumes these toolkits like nanoGPT etc.

 

[lockhrt999]

Does this stuff require you to be a maths/statistics genius?

I almost failed my maths and related subjects and barely scraped through back when I was studying lol so I probably can only dabble in this area as an end user that consumes these toolkits like nanoGPT etc.

Nope. In India, importance given to Mathematics within IT education is super overblown.

Read more > https://course.fast.ai/

 

[vishalrao]

@lockhrt999 Thanks! That website looks very encouraging and promising. I'll try to spend time over the coming weekends to go through the content. I'm mostly interested in the NLP area.

 

[lockhrt999]

I'm mostly interested in the NLP area.

Coursera has a full specialization on NLP https://www.coursera.org/specializations/natural-language-processing. You can do it for free if you don't care for a certificate.

 

[bhp91]

I am not really sure how you get lower training time on float32.

That's because GPU's don't have native support for bfloat16. Bfloat16 was developed by Google specifically for their TPU's.

You can still get these much smaller model sizes and faster inference by applying post training quantization on a already trained model.