"I can train an LLM on my laptop, with open source data and models" => do you have any idea how false this is? You obviously have not tried to do this. Yes, you can train an LM, but not a LLM, or anything remotely useful. "open source models" yees these exist e.g. LLaMa, but are also massive and trained to the detriment of the environment etc - frankly you have no chance of finetuning even these on a laptop. The weights alone of LLaMa 3 are 16GB, and training will require even more VRAM.
A product without a market? How come OpenAI is earning *billions* in revenue each year then? Yeah they're making a loss, but the product is absolutely useful - nowhere near as much as advertised, I agree, but it absolutely helps with writing code amongst other stuff.
Unfortunately, it IS necessary to have a ton of data to produce LLMs (yes this is a fundamental issue with deep learning), but even if you don't see the point, many others do. If you don't allow companies to steal everyones IP to train on, China *is going to do it*, and good luck taking DeepSeek to court to complain when they're not based in the UK.
IMO people care way too much about their stuff being trained on. If you are an indie game dev and your art is added to a training dataset, how does this actually negatively impact you in any way? Your art is such a small drop in the ocean of training data, it's not like everyone will suddenly be reproducing your style. The vast majority of training data is stock images etc that nobody cares about, artists complain way too much about this.
I enjoy your newsletter, but please stick to talking about topics you have a clue about.
For anyone who wants to add to the UK government's consultation on copyright 'protection' (i.e. no protection at all) against AI companies' exploitation, the ALCS has a brilliant guide to help you unpick the questions:
Thanks for the balanced criticism of the government's position on copyright and AI. I spent about six hours answering all the questions earlier this week and was left-facing with the same impression: their proposals will drive inequality by syphoning value from creators' works in the bank account of tech titans.
I was reminded of those 90s anti piracy adverts "you wouldn't steal a car", except with "you wouldn't steal a library"... except that's exactly what these companies are having done, yet rather than seeking a prosecution and protection of rights, the government is trying to backfill the law to make theft okay so long as it's done on a grand scale. It's literally criminal.
Some extremely naive takes in this article.
"I can train an LLM on my laptop, with open source data and models" => do you have any idea how false this is? You obviously have not tried to do this. Yes, you can train an LM, but not a LLM, or anything remotely useful. "open source models" yees these exist e.g. LLaMa, but are also massive and trained to the detriment of the environment etc - frankly you have no chance of finetuning even these on a laptop. The weights alone of LLaMa 3 are 16GB, and training will require even more VRAM.
A product without a market? How come OpenAI is earning *billions* in revenue each year then? Yeah they're making a loss, but the product is absolutely useful - nowhere near as much as advertised, I agree, but it absolutely helps with writing code amongst other stuff.
Unfortunately, it IS necessary to have a ton of data to produce LLMs (yes this is a fundamental issue with deep learning), but even if you don't see the point, many others do. If you don't allow companies to steal everyones IP to train on, China *is going to do it*, and good luck taking DeepSeek to court to complain when they're not based in the UK.
IMO people care way too much about their stuff being trained on. If you are an indie game dev and your art is added to a training dataset, how does this actually negatively impact you in any way? Your art is such a small drop in the ocean of training data, it's not like everyone will suddenly be reproducing your style. The vast majority of training data is stock images etc that nobody cares about, artists complain way too much about this.
I enjoy your newsletter, but please stick to talking about topics you have a clue about.
For anyone who wants to add to the UK government's consultation on copyright 'protection' (i.e. no protection at all) against AI companies' exploitation, the ALCS has a brilliant guide to help you unpick the questions:
https://www.alcs.co.uk/news/your-rights-are-under-threat-from-ai-its-time-to-have-your-say/
British citizens can respond to the survey here:
https://ipoconsultations.citizenspace.com/ipo/consultation-on-copyright-and-ai/consultation/
Thanks for the balanced criticism of the government's position on copyright and AI. I spent about six hours answering all the questions earlier this week and was left-facing with the same impression: their proposals will drive inequality by syphoning value from creators' works in the bank account of tech titans.
I was reminded of those 90s anti piracy adverts "you wouldn't steal a car", except with "you wouldn't steal a library"... except that's exactly what these companies are having done, yet rather than seeking a prosecution and protection of rights, the government is trying to backfill the law to make theft okay so long as it's done on a grand scale. It's literally criminal.