Need help with pytorch....cannot resume training

vishalrao · Jun 9, 2023

Does this stuff require you to be a maths/statistics genius?

I almost failed my maths and related subjects and barely scraped through back when I was studying lol so I probably can only dabble in this area as an end user that consumes these toolkits like nanoGPT etc.

lockhrt999 · Jun 10, 2023

vishalrao said:
Does this stuff require you to be a maths/statistics genius?

I almost failed my maths and related subjects and barely scraped through back when I was studying lol so I probably can only dabble in this area as an end user that consumes these toolkits like nanoGPT etc.

Nope. In India, importance given to Mathematics within IT education is super overblown.

Read more > https://course.fast.ai/

vishalrao · Jun 10, 2023

@lockhrt999 Thanks! That website looks very encouraging and promising. I'll try to spend time over the coming weekends to go through the content. I'm mostly interested in the NLP area.

lockhrt999 · Jun 10, 2023

vishalrao said:
I'm mostly interested in the NLP area.

Coursera has a full specialization on NLP https://www.coursera.org/specializations/natural-language-processing. You can do it for free if you don't care for a certificate.

bhp91 · Jun 13, 2023

Emrebel said:
I am not really sure how you get lower training time on float32.

That's because GPU's don't have native support for bfloat16. Bfloat16 was developed by Google specifically for their TPU's.

You can still get these much smaller model sizes and faster inference by applying post training quantization on a already trained model.

Need help with pytorch....cannot resume training

vishalrao

Global Moral Police

lockhrt999

Not a Fan.

vishalrao

Global Moral Police

lockhrt999

Not a Fan.

bhp91