UK Government: a 20,000-license Microsoft 365 Copilot experiment

The Government Digital Service (GDS) set out to test whether an AI assistant built into everyday office tools would change how civil servants work: how satisfied they are, how much effort routine tasks take, the quality of their output and the time they spend on administrative work.

Rather than rely on vendor claims, the government ran its own time-limited experiment across several departments and published the method, the results and the limits of the evidence.¹

From 30 September to 31 December 2024, 20,000 Microsoft 365 Copilot licenses were spread across 12 organizations, each committing at least 1,000. Participants included the Department for Work and Pensions, HM Revenue and Customs, the Home Office, the Ministry of Justice, the Welsh Government, the Office for National Statistics and Companies House, among others.

The evaluation combined 7,115 survey responses with anonymized usage data for 14,500 users. Copilot agents arrived partway through the trial and were left out of the analysis. Support and training were run by each department, and the report stresses that people had to review the output at all times.

The report lists its own limits: time savings are self-reported, it could not see how saved time was used, the holiday period shortened the final month, some licenses arrived late, and the tool did less well on nuanced, context-heavy or data-heavy work and on sensitive HR topics.¹²

In our view, this is one of the most useful public evaluations of Microsoft 365 Copilot because the buyer, not the vendor, designed and published it, including its weaknesses. The headline time savings are self-reported and rest on range midpoints, so other organizations should treat 26 minutes as an upper-bound perception rather than a measured gain. The reusable parts are the design (a fixed trial window, a minimum license commitment per department, survey plus usage telemetry) and the finding that benefits were smaller for nuanced, data-heavy and sensitive work. A public body planning its own trial can copy the method and add a control group or task timing to get harder evidence.