Beware software brain

April 23, 2026

Nilay Patel put out an excellent Decoder episode today. In it, he dove into the idea of “software brain”, which is something he’s been workshopping during the past several Vergecast episodes. It’s his attempt to explain why so many Silicon Valley folks are head over heels about AI, at the same time that most normal people hate it. The idea really resonates with me.

It’s worth checking out the post. There’s a 15 minute YouTube video, a podcast version, and a written piece on the page. But here’s the main thrust. Software folks are used to thinking of things as software problems. To them, automation is always a win, and LLMs are the best automation tools to come out in decades. But most people don’t think in terms of automation. Instead they see bullshit features, terrible art, people losing their jobs, and massive amounts of money and energy going into data centres.

This isn’t guesswork, either. Recent polls from the Pew Research Center and Quinnipiac University show that most people surveyed think AI will do more harm than good. Plus, this concern isn’t just among luddites. More people than ever are using AI, and those people don’t like what’s being put in front of them.

Some people afflicted with software brain think this is a marketing problem. Sam Altman bought a livestream that no one watches as a way to convince people of the value of AI. But I think Patel put it best when he said: “The iPhone was a consumer product so great that enterprises were forced to adopt it! That’s the bar, not the other way around.”

This is apt because LLMs are mostly enterprise products at the moment. Software developers and large organizations are leveraging them for automation, and that’s working well. But most people don’t care about those uses, and the downsides aren’t small.

As I wrote yesterday, I think LLMs can be very useful, and that we’ll trend over time toward local LLMs which don’t require massive server farms. I also think that AI is better when it’s boring — when it’s a useful feature of something else, and not the in-your-face focus. No one was all that upset with content-aware fill or DLSS (at least previous versions). Both of these features used AI and were trained on large data sets, but they happened on-device and provided a clear benefit to people.

If AI companies were focused on these sorts of features, or even focused primarily on the enterprise, I don’t think there’d be as much of an issue. Instead, they’re making insane promises about the power of their products, and telling ordinary people they’ll be left behind if they don’t get on board. Of course, they’ve also painted themselves into a corner. They can’t really say anything less if they want to keep the bubble intact.