As author Ted Chiang has noticed, “Most fears about A.I. are greatest understood as fears about capitalism.” After we think about rogue AI methods optimizing the world to loss of life, we’re truly describing what many firms already do. If a corporation’s ethos is misaligned, any know-how platform it creates will amplify that misalignment.
Take a second to mirror on the group you maintain most expensive: Does it have an ethos, or is it drifting with out one? Does it battle for human flourishing or towards it? Is it creating worth or extracting it? Are you shaping its ethos, or is it shaping yours? Regardless of your formal title, you’ve affect over this group. But when you can’t reply these questions with confidence, you aren’t in management. You’re a passenger in a automobile you assume you’re driving.
But when no person is consciously driving, the place is the group going? If the habits of superorganisms had been random, we’d see as many evolving towards flourishing as towards corruption. As a substitute, the overwhelming majority wind up in the identical place, with indistinguishable values.
I just lately had a dialog with the CEO of a multibillion‑greenback firm who’d come to see me for assist with a baffling downside. He had made a daring new guess on the promise of AI to revolutionize his enterprise. He’d personally overseen the event of a brand new AI product and examined it with clients. They cherished it. Happy that he had created one thing superb, he turned it over to the remainder of his workforce to commercialize and scale.
Six months later, this new product was going nowhere. The gross sales workforce wasn’t promoting it. Builders had been loath to work on it. Marketing wouldn’t market it. When confronted, the executives would insist that the product was a part of their quarterly priorities. Every of them was telling their groups to deal with it. But, by some means, line workers and center managers had been constantly avoiding the AI product. It was a sample all through the group.
When the CEO personally intervened, he might simply get particular person clients to undertake the brand new product. However he couldn’t get his workforce to do the identical, even after investing extensively in coaching, efficiency bonuses, and prime‑down directives. He couldn’t establish anyone particular person or group actively blocking the AI initiative. He even changed a number of key executives with new ones explicitly chosen for his or her enthusiasm for this new course. And but, inside months, the uneasy establishment prevailed.
He shook his head. “You realize what’s driving me loopy? I based this firm fifteen years in the past. After I first proposed the core concept, everybody thought I used to be nuts. Traders, potential workers, even my household.
“Now that we’re profitable, once I recommend we have to pivot, lots of the identical individuals, together with my very own management workforce, resist as a result of ‘we will’t threat what we’ve constructed.’ ”
He threw up his fingers. “They’re defending one thing that exists solely as a result of I took the ‘loopy’ threat all of them thought was inconceivable. Now they assume I’m reckless and might’t appear to understand what number of occasions they’ve benefited from my pushing by way of the inconceivable to get us to new success.”
He might really feel the stress his insistence was placing on his workers, he instructed me, as in the event that they had been being torn between two authorities. However who, precisely, was this different authority? Like so many founders, he had assumed that being in cost meant the group would mirror his values. How might or not it’s that he had all of the management on paper however so little in each day follow? And if he wasn’t in management, who was?
John Steinbeck noticed this predicament clearly in The Grapes of Wrath. Tenant farmers are being evicted from their land throughout the Mud Bowl, however after they confront the lads finishing up the evictions, they will’t get a transparent reply as to who’s making the choice to devastate them. The lads despatched to do the soiled work reply to the financial institution supervisor, who solutions to distant homeowners, and people homeowners reply to shareholders. Every of them has a job within the group, however by some means none of them is in cost.
This results in the farmers’ bewildered query: Who’s the financial institution?
Steinbeck portrays it as a faceless system by which no particular person may be held accountable, regardless that actual individuals are shedding their properties and livelihoods. He calls the financial institution “the monster. Males made it, however they will’t management it.” My good friend the CEO, grappling with resistance he couldn’t find, skilled the identical bewilderment. If the particular person on the prime isn’t in cost, who’s?
The CEO I used to be counseling wasn’t preventing his personal individuals on the AI initiative. He was wrestling with a distinct form of dwelling being—one which had developed its personal character, its personal preferences, and its personal will. His private ethos of innovation was at conflict with the group’s emergent character, which craved security, certainty, and predictability. The truth that he had created it mattered little or no.
Life is outlined not by flesh and blood however by particular properties. Residing issues keep boundaries between themselves and their setting. They metabolize sources into power. They develop, adapt, and reproduce. They exhibit behaviors that emerge from their elements however can’t be predicted by learning these elements in isolation. Most significantly, they show a will to outlive that shapes each motion they take.
Organizations exhibit each one in all these properties. They keep authorized and cultural boundaries that outline who’s inside and who’s out. They metabolize capital, expertise, and uncooked supplies into services. They develop by way of hiring, adapt to market pressures, and spawn subsidiaries or merge with others. An organization’s “tradition” emerges from hundreds of each day interactions however can’t be present in any worker handbook or org chart. And when threatened, organizations exhibit a tenacity that may override the preferences of any particular person inside them—even their nominal chief.
Each is a superorganism unto itself, working in accordance with its personal guidelines of existence.
Now we will reply Steinbeck’s query, Who’s the financial institution? An emergent intelligence, each bit as alive as you or me. A superorganism.
That is who the financial institution is.
After we inform horror tales about AI—methods that optimize ruthlessly towards a slim objective, blind to human prices—we’re, in some ways, describing the financial institution. Males made it, however they will’t management it. Our fears about AI are, as Chiang suggests, usually fears that we’ll pour unprecedented technical energy into establishments whose ethos is already misaligned with human flourishing.
If we would like totally different outcomes, we can’t simply ask what our machines are optimizing for. We’ve to ask the identical query of our organizations.
Excerpted from Incorruptible by Eric Ries, revealed by Authors Fairness. Copyright © 2026 by Eric Ries. Reprinted by permission of Authors Fairness.
