The 2019 Org Chart Strikes Back: A Copilot Hallucination Story
An anonymised account of what happens when Copilot grounds in a stale tenant — and what the post-mortem revealed about the cleanup we should have done first.
This story is real. Names changed; the bug is exactly as it happened.
A regional sales lead at a mid-market software company asks Microsoft 365 Copilot a routine question: “Who’s the new VP of Customer Success?” The VP started a month ago, replaced someone who’d been in the role for four years.
The internal announcement went out, the new VP was added to the executive team page on the intranet, and the company-wide all-hands had been recorded with introductions.
Copilot answers with the name of the previous VP.
Confidently. Cites a 2019 org chart PDF that lived in the HR site.
The sales lead, trusting Copilot, references the previous VP by name in an email to a major customer about an escalation. The customer — who reads tech news — replies: “I thought [previous VP] left to take a role at [competitor]?” Embarrassing but contained.
Two weeks later, a procurement coordinator asks Copilot which approver they need to route a vendor request to. Copilot tells them.
The named approver hasn’t worked at the company for 18 months.
The procurement coordinator emails a stranger asking for approval; the email bounces; the procurement coordinator wastes two days finding the actual approver; the vendor’s invoice gets paid late; the relationship sours.
Two more weeks: a new hire’s manager asks Copilot for the company expense policy.
Copilot returns the 2020 policy, which capped meal reimbursements at $25. The new hire submits a $33 dinner and gets denied.
They escalate. The escalation reaches HR.
HR confirms the policy is now $40. HR also discovers — to their visible surprise — that Copilot is still surfacing the 2020 policy.
The CIO calls a meeting.
What they found wasn’t a bug. It was a tenant that had been allowed to drift for six years.
Six issues.
Zero Copilot bugs. All of them content hygiene problems that predated Copilot by years.
Within a week of the meeting, they ran a comprehensive storage and content scan.
Output: 1,847 recommendations across stale sites, duplicate documents, files with obsolete references, and orphaned content. They didn’t action all of them — the team did the top 200, which covered the categories most likely to be cited by Copilot.
Three months later: no new Copilot incidents. Storage bill down 19% as a side effect.
The customer success VP incident, the procurement incident, the expense policy incident — the company had been running for years without Copilot and these issues never surfaced.
People who needed an answer asked another person. They got the right answer.
The wrong answer was sitting in SharePoint the whole time, but nobody read it because nobody had a reason to.
Copilot is the reason. Every question is now an opportunity for the tenant to lie to an employee who has no way to know it’s lying.
This isn’t an argument against Copilot. It’s an argument for treating Copilot rollout as a content-quality milestone, not just a license-purchasing decision.
The tenant has to be inventoried, the obvious offenders archived, the unowned content claimed or deleted.
If that work doesn’t happen before Copilot is enabled, the work happens after, in incidents.
The order doesn’t change the work. It changes whether you do the work calmly or in a meeting called by your CIO.
An anonymised account of what happens when Copilot grounds in a stale tenant — and what the post-mortem revealed about the cleanup we should have done first.
Microsoft 365 Copilot grounds every answer in your tenant’s own content. When that content is stale, contradictory, or duplicated, Copilot doesn’t shrug — it answers with confidence. Here’s why that’s a problem and what to do about it.
Most Copilot governance programs focus on access controls, sensitivity labels, and prompt filters. They overlook the most consequential variable: the quality of the underlying content. Here’s why content hygiene is the foundation, and what to do this quarter.
opilot grounds its answers in your tenant’s content — and the content is full of contradictions, duplicates, and zombie data. Here are the five patterns we see in every scan that turn Copilot into a confident misinformer.
That extra TB Microsoft is about to sell you is probably already in your tenant. You just can’t see it.
A 143-user enterprise tenant cut $18,000 off its annual Microsoft bill in 90 days — without touching live content. Here’s the playbook.