One mind-boggling aspect of the current marketing campaign around AI is that it is based on utterly insane economic assumptions, which most people just silently accept. Like, for example, the assumption that AI will not only solve all kinds of problems, but will also be either extremely cheap or free to use.
Let's talk about capital investments behind these systems. Our society spends tons of resources on very niche AI research, then it spends resources on making specialized AI hardware, then it spends resources on training models and finally it spends resources on constructing actual AI systems. All of this is very expensive, and is currently done at unprecedented scale.
After a generative AI system is brought online, it requires at least as much IT maintenance as any other set of servers. (More if you are forced to do something weird because of extreme processing density.) It also requires all kinds of maintenance that is unique to generative AI, like safety tuning, retraining on newer data and so on. Finally, the system consumes vast amounts of electrical power for its basic operation.
What does this add up to? Something rather bizarre. Almost all generative AI services today seem to operate at a massive loss. Plus, they needed huge amounts of upfront investment to bring them online. Plus, most of them are pretty bad at what they do, so it is universally assumed that they will be upgraded via even more capital investment.
In short, there is no way this technology can remain cheap or free without massive subsidies from somewhere. Some people claim that it will eventually break even by increasing our productivity. Aside from being awfully vague, this assertion fails to account for something everyone in tech should be well-familiar with: opportunity costs. The usual choice is not, as many hype peddlers claim, between using generative AI and using nothing. That's a laughable false dichotomy. The choice is between investing resources into generative AI or investing those resources into something else.
Let me drive this home with a simple example. Which of the following scenarios do you think leads to higher overall productivity of all people involved?
1. Someone trains an LLM model on StackOverflow posts. A thousand developers then ask it how to do X with technology Y. The model answers, running trillions of operations for each token, giving different answers to different developers (e.g. based on their phrasing).
2. Someone writes a manual on how to do X with technology Y. A thousand developers then find that exact manual via typical search.
Which option is more computationally efficient? Which option can be quickly changed to accommodate a change in Y? Which option is likely to result in negative side-effects of various sorts?
What's utterly insane about the tech industry right now is that it is eager to subsidize option #1 to the order of hundreds of billions, while refusing to subsidize option #2 even to the order of several millions. (Many crucial pieces of the Internet infrastructure are maintained by unpaid volunteers. Many extremely popular pieces of software have basically no good documentation and undergo zero UI/UX studies.)
I am pretty sure there will be people who miss the point I'm making here. They will be tempted to say something along the lines of "oh, but LLMs are universal, while documentation for some technology is only useful for a handful of people". That doesn't change any considerations involving resource allocation and productivity. If you look at it from the perspective of everyone involved, the example with searchable manual requires miniscule investment of resources with a quick pay-off. That pay-off can then be invested into improving something else, while the searchable manual continues to provide value at very little expense. On the other hand, the massive investment into LLM doesn't have an obvious break-even point at all, since the system will continue to require significant resources for operation and maintenance. (Not to mention the externalities like people breaking things or wasting extra time because of hallucinated answers.)