Assuming the training software could be run on the hardware and that we could distribute the load as if it was 2023, would it be possible to train a modern LLM on hardware from 1985?
Assuming the training software could be run on the hardware and that we could distribute the load as if it was 2023, would it be possible to train a modern LLM on hardware from 1985?
It wouldn’t. Training the neural nets for LLMs are all about brute force, and it’s only been possible in the last few years to train these models without spending billions. Even going back to 2010 I think it’d be largely infeasible.
The good news is if we fast forward even just a few years, training will become relatively cheap compared to today.
They didn’t ask if it it could be done without spending billions, or whether it would be feasible, i.e., practical, just whether it would be possible.
So my collection of about 20 Commodore 64 isn’t enough? Do I need another 20? /s
I think you’d need roughly a billion of them. Minus the 20 you have, of course.
Estimate of up to 30 million C64 were produced so I guess not enough if I managed to get every single one in the world
You would need at least 44 more