Who Gets It First
Who gets an AI tool first matters about as much as what the tool is allowed to do. That's what came back when I tested the advice I gave in an interview that just came out.
There are two common ways to stage a rollout. One stages what the tool is allowed to do...everyone gets it, and the human checking is heavy at first and lighter at each stage. The other stages who gets it: a small group of your strongest people gets it first, figures it out, and everyone else learns from them. I argued for the second.
I ran both on the model from my last research note, the weighted voter model of how honey bees pick a nest site (Valentini et al. 2014, Soma et al. 2024), with an org chart added: 200 people in 25 teams, leads who are heard more, an executive, deference to the boss, and people who differ in how well they read results.
On a close call, giving the tool to everyone at once with nobody checking left the organization on a worse way of working 69% of the time. The staged rollout brought that to 3%. The ten sharpest people first brought it to 2%, with about a sixth of the checking, because only ten people did the checking. Both plans work, and for the same reason, which is that somebody checks before the organization copies.
They cost different things and fail in different ways. The staged rollout has nobody waiting and pays in checking, and with half of its checks skipped it was wrong 30% of the time. Once its last stage took the checks away it couldn't follow a change at all, because nobody in it was trying anything new, and a check only tests what somebody proposes. The ten who kept checking and trying things brought the organization along in 38% of runs, and a staged rollout that kept one check and had everyone try something new 1% of the time followed in 69.5%. What follows a change is the checked trying, not the plan. People first pays in waiting, and it depends on who goes first...ten random people with the same checks left 13%, because a check is only as good as the person doing it.
Then I let people build new ways of working instead of choosing among existing ones. With the same amount of building, the ten sharpest improved the organization about 1.7 times as much as everyone building carefully, and with one person's share of the building each they barely moved it, so the few do more only when they get the capacity. Giving their work an audience did more than ordering everyone to adopt it, and an order people could check first did better than one they couldn't.
A couple of things did not go my way. On a harder problem, one with a lot of good-enough answers to get stuck on, who built barely mattered and the size of the change mattered more. And most of the numbers predicted before the first runs were wrong, all in the same direction: once anybody checked, the organization settled faster and more accurately than expected.
A lot of this has precedents in research on herding, social learning and how new practices spread, and the paper says which parts. These are simulations, not a study of people.