Why Your Best Employees Quit Using AI After 3 Weeks (And the 6 Skills That Would Have Saved Them)

Toward the very end of 2025, a study happened that we didn't pay attention to. Microsoft tracked 300,000 employees using AI C-Pilot. Excitement peaked for the first three weeks.

Then there was a crater of disappointment. For those of you in the audience, that wasn't just because of Copilot, and we'll get into that. And then most people just quietly stopped using AI.

The survivors figured out one thing. AI isn't a tool skill. It's a management skill.

That is not a co-pilot specific lesson. That's a larger take and that changes everything about how you and I need to be training. Let me get into what I mean here.

6 months ago, your company probably rolled out AI tools to everyone. Maybe it was 9 months ago, who knows? Everyone got access to chat GPT or they got access to Copilot or Claude, whatever it is.

Somebody ran a training session. Here's how to write prompts. Here's what the tool can do.

Go be productive. It probably took six hours. Usage dashboards today.

What do they show of the people that got trained in your org? Most orgs, it's the 8020 rule, the way you don't like to see it. 20%ish monthly active users, 80%ish of the seats are dormant.

So, what happened to the 80% that give up? That's what the Microsoft study looked at. Well, they tried it, right?

They typed, "Help me with this report. " They got something generic. They tried again.

They got something confident and wrong. They tried a third time. they decided it was faster to do the work themselves.

I have heard that story over and over again. This is not just the Microsoft study. Simon Willis was talking about this just this week when he argued that the context you get implicitly from spending time with AI models gives you the ability to get the most out of them and gives you a tremendous leg up on people who are entering the AI space cold.

Well, there's a lot of people that got to enter the AI space cold. There's a tremendous need for catchup and yet most organizations lose most of their people in this trough. What did the survivors figure out?

What can we learn from it? And how can we start to scale those lessons learned? That's what this video is all about.

The 101 level is fine in most corporate training. I don't want to go after the basics. Tool tours, prompting fundamentals, generic use cases.

Those are not the issue, guys. If you want to know here's what chat GB GPT can do it's fine at the 400 level I also don't think there are issues with our training we have technical implementation we have API integrations we have rag architectures we have fine-tuning if you are a technical builder if you are a developer you are in good shape but the training market has bifurcated into just those two poles the 101 basics and the 401 level technical implementation it has skipped the middle entirely and the middle is where most of the productivity gains for most people actually live. The 2011 level is where the question shifts from how do I use this tool to where does this tool fit in my workflow and how do I know when the output is trustworthy.

This is applied judgment. It's not about writing better prompts per se. It's about knowing which parts of your work AI ought to do, which parts you ought to do, and how to verify the relationship between them.

when you get something back, is it good or not? And the strategic insight that most organizations miss is that this is not a technology adoption problem. It gets dressed as one, but it's really an organizational capability problem.

We've been categorizing AI wrong from the start. I think Ethan Mik puts this really well when he says the best users of AI are good managers. They're good teachers.

The skills that make you good at AI are not prompting skills. They're people skills. That's a good reframe, right?

The 2011 skill isn't really a technical skill per se. That's for 401. It's it's a task decomposition skill set.

You get into quality assessment. You get into iterative refinement as a skill. Those are new skills for workers.

And knowing when to trust is a new skill. We don't teach those as management skills right now. We teach those, if we teach them at all, which most of the time we don't, as tool skills.

Think about the implications. The skills that predict AI success aren't new skills at all. They're the same skills that have always made people effective leaders.

Which means your AI training problem might be a management development problem in disguise. And it means your AI champions probably shouldn't be your most technical people. They should be your best managers.

I think there's a reason why token consumption leaderboards at major organizations are often dominated by senior execs and people who might be at like the very distinguished engineering level. It's not because they were in the code before. It is because they have great management skills and they have great domain knowledge.

And when you put those two together, you get a compelling package for uptake on AI. Think about it this way. Would you hand a 100page RFP to a brand new intern and just say, "Handle this.

" Well, of course you wouldn't. You'd break the work into pieces. You'd tell them which parts to tackle first.

You'd manage them, right? You'd explain what good looks like. You'd review the work and you'd give them constructive feedback.

That's really the mindset we should all be using at the 2011 level, working with AI. And the people who have figured this out, who treat AI like a capable but inexperienced collaborator who needs management, they're the ones who made it through that 3-week trough in the Microsoft study. The people who expected management, including senior leaders, or who expected nothing at all, both of those classes of people gave up because it's not magic.

And here's where it gets complicated. AI is jagged. AI has different capabilities at different tasks that make it hard for 2011 level AI participants and learners to figure out what to use AI for.

A BCG and Harvard study showed that consultants use AI on very very different types of tasks. Well, that's not a surprise, right? And they found that inside AI's capability frontier, things that it actually handles well, the consultants were able to speed up, right?

they could finish 12% more tasks, 25% faster, etc. But on tasks outside that capability frontier for for where you know, and I know that frontier moves on tasks that looked like AI should handle them, but it couldn't, consultants were 19 percentage points less likely to get to correctness than those working without AI. In other words, I just want to be really clear about this so you don't miss it.

The BCG and Harvard study essentially showed that 2011 level usage of AI is hard because people tend to have a single mental model for their AI usage and they just assume the AI is probably good at reports or the AI is probably good at spreadsheets or whatever it is and they don't have yet the nuance that allows them to figure out where AI is actually going to be useful or not. And because they don't have that, they get gains where AI is good and they miss losses. They're basically more likely to be incorrect on parts of their work they previously did well.

You see quality degradation because they don't realize that AI is not a universal level up. So that paradox needs to shape our AI training strategies. The experts typically know where those jagged boundaries are and they typically have the judgment to avoid the traps.

At a 401 level, this is not an issue. You can understand where you would use Python to solve a problem set, what the edges of your context window look like. So what's the play for 2011?

What's the play for the missing middle? You need the expertise to know where the boundaries are. But the biggest gains tend to acrue to non-experts working within those boundaries.

So this implies a model that most organizations haven't considered when it comes to training. Experts should be mapping the frontier of their domains. They should be creating guard rails and verification protocols and they should be actively enabling non-experts to work safely within those boundaries.

In other words, if your organization, which most do, has a small cadre of really sharp AI adopters, people comfortable in that 401 level, they're the frontier mappers, and you should be encouraging them actively to not just push on their own capability sets, but to actively set up an environment where the rest of the missing middle can actually be productive. Now that same study identified two work patterns that I think are worth calling out here. One of them is called the centaurs right people who are able to clearly divide work between themselves and the AI.

So named because you know half human half horse cleanly divided right the human does the strategy framing the AI generates the option set whatever it is you have distinct responsibilities. Another group also roughly at the 2011 level acted as cyborgs. They completely integrated their workflow with AI and they continuously interacted with the technology.

So the boundary between human and AI work became fluid. Look, both patterns work. Both patterns led to productivity gains in the study.

But here's what matters strategically. They're suited to very different contexts. Centaur mode works really well for high stakes work where you need very clear accountability, very clear verification checkpoints, high human judgment.

Legal work comes to mind, medical work comes to mind. Cyborg mode works best for creative and iterative work, building work where continuous refinement tends to improve the output. The mistake would be thinking you have to pick one pattern and apply it to everybody.

The 2011 skill is knowing which pattern fits which task and ideally being able to switch your mode between them. From the employees perspective, closing a 2011 gap sounds a lot like I realize the skill isn't writing better prompts. It's actually just breaking my work into pieces and knowing which bits the AI is good at.

From the manager's perspective, it sounds different, but still transformational. We went from some people use it sometimes to this is just how we do RFP responses now. We we got a playbook and the new hires just learn it.

So what are these 2011 skills? I keep talking about them. Let me get specific.

I think there are six of them and zero of them are prompting techniques for those of you keeping count. First context assembly. Knowing what information to provide from which sources and why.

The 101 user either dumps entire documents into AI or provides almost no context. Both produce kind of mediocre results. The 2011 user understands that AI is sensitive to context quality and takes the time to provide the right background, the right constraints, the right examples to improve output.

The second skill is quality judgment. Knowing when to trust AI output and when to verify it. This operates against two dimensions.

knowing which task types require what level of verification. So like high stakes legal work has to get scrutinized. Low stakes drafts, yeah, you could edit them lightly.

And then knowing within a given output which parts are likely reliable and which parts are likely to be problematic. So AI can confidently state accurate information and will sometimes hallucinate in the exact same paragraph. The 2011 skill is learning to detect that and understand how quality works at the level of the document and within the document.

The third skill is task decomposition. Breaking work into AI appropriate chunks rather than throwing an entire task at the tool or avoiding it. And this is where the management framing really helps.

You're identifying which subtasks you want to delegate to you versus to the AI just like you would with a team member. The fourth skill is iterative refinement. moving from 70 to 95% through very structured passes.

So the 101 user will accept that first output and its AI slop or they'll just abandon the whole effort. The 2011 user treats the first draft as a starting point and wouldn't accept the intern's first draft and knows how to iterate and refine. Same principle, but they take it further.

Fifth skill, workflow integration. Betting AI into how work actually gets done rather than treating it like a side tool. And the difference shows up on whether AI is a separate activity like I'll try the AI thing later or is it an integrated capability like this is just how we do RFPs.

Now the sixth skill is frontier recognition. Knowing when you're operating outside the AI's capability boundary. This is the skill that prevents that nasty whatever it was 19 percentage point performance drop.

It requires building explicit knowledge of where AI excels versus fails for your particular work. and then sharing failure cases so the team learns the boundaries. Yes, we need to share our failures.

Notice what is not on this list. Prompt engineering is not on this list. Tool specific features are not on this list.

Technical implementation is not on this list. Those do matter, but they're not what separates success from failure at the 2011 level. In my experience, the skills that matter are very much manager skills.

their judgment skills and they transfer across tools and they survive model upgrades. So, this sounds great. What's actually blocking adoption?

Here's what the research says is actually stopping people from using AI effectively. Fear of doing it wrong. People don't know if they're allowed to use it, what's safe to paste in, whether they'll get in trouble if the AI makes a mistake.

without really really clear organizational guidance that leans to yes talented people see AI as a risk and avoid it. I cannot tell you how many times my first AI conversation is are we allowed to do that? If that's your first question because it is worried about security, you have already failed the adoption loop because the first thing your team thinks when they think of AI is they think of a giant red stop sign.

It's not going to work. The 2011 gap is not just a skill gap. It is a permission gap.

Your most conscientious employees are the ones who care the most about doing good work, and they're the ones most likely to opt out. The people you most want to do 2011 work will opt out if you're not able to say yes. Well, ironically, IT departments end up trying to put guard rails in place that restrain their productive employee base from being productive and that do not disincentivize the reckless employees who would be taking inappropriate risks with AI anyway.

It tends to focus on infrastructure and security and they forget the capability gap that business users actually face. And this is a mental model mismatch, right? It and CISOs think in terms of systems, inputs, outputs, deterministic processes.

AI does not work that way. AI works like a person. And so when you give the AI problem to it, it's like giving your people to IT instead of HR.

You end up getting infrastructure when you need capability building. And I don't say that to make a larger philosophical statement about AI and personhood. I'm just saying behaviorally AI is inconsistent at times.

It's context dependent. It requires management. It's not well suited to the way IT departments think.

The next thing I'm going to call out as a failure mode is that generic tools tend to stall at enterprise scale. Chad GPT and Claude and Copilot are remarkably flexible. That flexibility is their strength for individual use and it is their weakness for enterprise deployment.

Most Gen AI systems don't retain feedback. They don't adapt to context. They don't learn.

Every interaction starts from zero. The productivity gains that individual power users achieve in your org do not automatically transfer to their teammates unless you put work in. This is a knowledge management problem that masquerades as a tech problem.

Individual learning will not scale in AI terms without deliberate effort on the part of the org. I will also add there's a time bomb ticking that most organizations aren't paying enough attention to because the apprentice model is collapsing. Junior employees used to develop judgment by doing the routine work that's now often delegated to AI research tasks, first draft writing, etc.

The unglamorous work that taught people how the domain actually functions. If organizations do not rebuild that pathway, they are going to face a judgment deficit that compounds over time because seniors who can map the frontier won't be around forever. The juniors who never built that judgment are going to end up being promoted one way or another.

and the organization will lose the expertise that makes AI effective. This isn't just a problem for next year's planning cycle. It is a structural issue that every company needs to think about.

So what are the organizational moves that enable you to unlock the 2011 gap? First, make sure that you create AI labs with power users, not just with the 401 technology nerds. These need to be lightweight and fastmoving teams that actually experiment with workflows and they must include employees with no technical background.

You have to be able to show how AI adds value without having to know what an API is. Second, conduct systematic discovery across functions. Trek bicycle is great here.

They interviewed every single department about how AI might improve their work. And from those interviews, the team was able to get to 40ome concrete use cases. Your org probably has similar hidden knowledge, but you must put the work in to surface it.

I will add here that vocalized use cases are only partially correct, and you usually find the real use cases when you dig under the surface and start building. So look at that initial presentation as the first cut, not the final cut. Third, make success visible.

Run low stakes competitions. What's a workflow you've meaningfully improved using AI? becomes a question you can ask on a Friday.

Surface practical applications. Create organizational learning. Create social proof.

People adopt what they see other people winning with. Take advantage of that. Next, invest in hours, not just access.

Employees who receive more than 5 hours of formal AI training are doubledigit percentage points more likely to become regular users. Part of the gap between tool rollout and adoption goes back to Simon Willis talking this week about spending time with AI in his domain, which is code, but same idea. You have to be willing to let people spend time on this.

Define your guard rails explicitly. What data is allowed? How can you disclose AI assistance?

What does good look like? So few people who are building AI policy bother to ask what positive AI usage looks like. It's always about the negative.

That makes 2011 adoption really hard. Last but not least, share the failure cases systematically. When someone discovers a task that AI handles poorly for you today, that knowledge needs to spread and create mechanisms for sharing what doesn't work, not just what does work.

Then take the failure cases out to the frontier, the 401 users, and see how quickly you can solve for them. Because those guys will be the first ones to figure it out. And AI keeps evolving and they're likely to crack it soon.

No matter what, your employees are already using AI. The shadow IT problem is massive. The value is real.

Workers see it, but organizations aren't structured to capture it. And without that work, there's a massive coordination and management failure. That means most people are stuck at the 101 level.

And we don't have organizational support needed to get to 2011. And that unlock is huge. Getting the 80% of your org to 2011 level is what distinguishes companies that are humming on AI and moving very quickly and realizing real gains from companies that like tried it and like now they have this like population problem because a few of them are at 401 level and going like crazy and most of them couldn't care less.

So ask yourself some questions. Ask honestly, can my people identify the subtasks AI should do versus what they should do? Do we have a way to iterate and not just accept first outputs?

Do we have that culture in place? Has AI been integrated into our workflows or is it just a side activity? Do we know for our work where AI fails?

If you can't answer those questions, your people are probably stuck at 101. They're in the trough and most of them are not going to figure it out on their own because even though they're capable, the organizational context doesn't support their learning. Ultimately, the difference between AI activity and AI fluency isn't about the tools you deploy.

It's about whether you've invested in the judgment layer that makes those tools reliable. That's the 2011 challenge. And it is solvable if your organization is willing to invest in the middle layer that most training programs skip.

Don't lose your 2011 people. They're incredibly valuable.