GitHub had degraded availability across about 28 services between 08:43 and 10:44 UTC on Sunday 13 September, according to its own incident report [1].
At the worst point, about 96% of attempts to create issues through the website failed, and more than 90% of sign-ups. About 4% of Actions workflows were affected by failures to issue tokens [1].
The cause was an internal data-cleanup job that began writing to a shared database cluster at 07:33 UTC. That cluster holds permission data read on nearly every authenticated request [1].
What it covers
Services on GitHub.com, including Issues, Pull Requests, Actions, Codespaces, Pages, Notifications, Code Scanning, Git LFS and account sign-up [1].
GitHub says the safeguard pacing the job watched only how far database replicas were lagging, which stayed low, while the primary database ran out of connections. A retry loop around token creation then kept it saturated [1].
It says it will rate-limit background jobs on shared databases, add request timeouts and bound retries, and move authorisation data out of the shared cluster within two weeks [1].
What it does not tell you
How many users or organisations were affected. The report gives failure rates for some operations only [1].
It is GitHub's own account; no outside party has measured the outage [1].
Comments