Skip to content
AI Primer
breaking

Gergely Orosz reports Microsoft email failures lasting at least 12 hours

Gergely Orosz reports widespread Outlook.com and Microsoft 365 email failures lasting at least 12 hours, with servers failing again during recovery. CPU exhaustion was reported, but the cause remains unconfirmed.

3 min read
Gergely Orosz reports Microsoft email failures lasting at least 12 hours
Gergely Orosz reports Microsoft email failures lasting at least 12 hours

TL;DR

Microsoft told the University of Queensland that some machines were too unhealthy to accept a network-change sync. Its public status page showed Microsoft 365 and consumer-product degradation while reporting no Azure issues.

Email and Copilot disruption

Gergely Orosz, author of The Pragmatic Engineer, said in his initial report that Outlook.com users and Microsoft's own employees could not send or receive email.

The Microsoft advisory he shared named these potentially affected services:

  • Exchange Online
  • Microsoft Teams
  • SharePoint Online
  • Microsoft 365 Copilot
  • Microsoft Defender XDR
  • Microsoft Planner
  • Microsoft To Do
  • Microsoft Intune
  • Microsoft Defender for Office 365
  • Microsoft Purview, including authorization, auto labelling, data loss prevention, communication compliance and data classification
  • Power BI, specifically intermittent failures exporting reports with MIP sensitivity labels to PDF

Orosz later said he'd heard that Teams was working, despite its inclusion in the advisory.

Network gateways and recovery traffic

Twelve hours into the incident, Orosz described a brutal recovery loop: servers were dying as Microsoft brought them back online. He interpreted Microsoft's updates as suggesting a possible thundering herd, with another issue still unidentified.

The University of Queensland's incident log relayed Microsoft's recovery work on November 13. Times below are AEST, ten hours ahead of UTC:

  • 09:37: Some returned servers were remaining stable. Microsoft had shifted traffic and connections to alternative pathways and observed limited recovery in some regions. Protective traffic-management measures were being introduced in one location to reduce load.
  • 11:10: Active workloads had been failed over to other network gateways. Queued email was expected to arrive with delays. Microsoft provided no restoration ETA.
  • 15:35: Microsoft was scaling up virtual machines to increase capacity. Engineers were also attempting to force-sync a recent network change onto affected machines, but machines stuck in a bad state were preventing the changes from syncing.
  • 16:33: Engineers were evaluating traffic throttling, protocol adjustments and failover routing. Internal testing was expanding into a contained environment to improve traffic monitoring and issue detection.

At that last update, equivalent to 06:33 UTC, the university's incident page still marked the disruption unresolved.

CPU reports and AI speculation

Microsoft told UQ that the issue was confined to a Microsoft 365 network gateway. Its updates reported no indication of malicious activity and promised a post-incident report covering the root cause and repair work after mitigation, according to the university's record.

Orosz said he'd heard reports of CPU spikes or exhaustion. In the same exchange about AI-generated code, he said Microsoft changes have human owners who push them, and that a postmortem was needed before drawing definite conclusions.

Suspected LLM bot replies

Orosz also blocked an account in the incident discussion that he described as an LLM bot.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 2 threads
TL;DR1 post
Email and Copilot disruption2 posts
Share on X