Sam Altman:
A lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we’re going to be able to run them on our computers, and so we’re not going to need all the stuff in the data center.
But from a global allocation perspective, actually it’s quite terrible to have everybody, like, have a bunch of the world’s limited supply of DRAM sitting in their own computer and being used five percent of the time.
Like you want maximum utilisation in the cloud if you’re trying to deal with this very supply-constrained environment,
And somehow many people’s intuition is the opposite of that
TL;DR Guy selling product: “Here’s why owning your own product instead of renting it from me is a terrible idea”
McDonald’s can probably make hamburgers more efficiently than everyone can at home, but I’m not eating that shit any more than I’m giving Sam Altman my money or data.
McDonald’s does but its still more expensive than making hamburgers at home, why?
Because profit motive is an inefficiency.
Profit motive is also a misallocation of limited resources. Regular soup kitchens can feed hundreds of people at the cost of only a few dollars per portion. The only advantage they have is buying wholesale. Imagine spawning out of thin air a nonprofit with the vertical integration and logistics of McDonalds. How many people could it feed for next to nothing?
Oh damn! Now there’s an analogy!