Certificates
Expired certificates: Equifax, Ericsson and a date nobody watched
An expired certificate left Equifax blind for 19 months and took mobile networks down in two countries. Why a date fixed in advance still causes outages.
A certificate is a small file with a date written inside it. Software uses it to prove something, such as who a server is, and when the date passes the proof stops. What happens next depends on the design. Sometimes the service stops. Sometimes something worse happens: a tool that was watching quietly stops watching.
Equifax, 2017: the way in
On 7 March 2017 a critical flaw in Apache Struts, a web framework, was made public. The US Department of Homeland Security alerted Equifax the next day. On 9 March an internal alert went to more than 400 people with an instruction to patch within 48 hours.
The copy of Struts running Equifax’s consumer dispute portal was still unpatched on 13 May, when attackers got in through that flaw, and it stayed unpatched until the breach was discovered in late July. The US House Oversight Committee’s majority staff report, published in December 2018, says the attack ran for 76 days and calls the breach “entirely preventable”.
Equifax, 2017: the blindness
Equifax had a device meant to inspect the traffic going in and out of that portal. To read encrypted traffic, the device needed a valid certificate. Its certificate had expired on 31 January 2016. For 19 months Equifax could not see what was leaving.
On 29 July 2017 the company uploaded new certificates, 67 of them that evening. The report says it immediately noticed suspicious traffic.
The committee counted at least 324 expired certificates across the company when the breach happened, and seventy-nine of them were on devices watching highly business-critical domains. One Equifax employee had already said the company needed to decide who owned the “care and feeding” of certificates and to create a tested renewal process. The committee’s conclusion was that with that in place, the monitor on the portal would have been working when the intrusion began. That is the committee’s account, not a court finding.
Ericsson, 2018
On 6 December 2018 the O2 4G network in Great Britain and SoftBank’s mobile services in Japan went down. SoftBank said its LTE service and several others were unavailable for over four hours.
Ericsson supplied equipment to both, and its first analysis pointed to an expired certificate in the software versions installed with the affected customers, two versions of a core-network component called the SGSN-MME. Its chief executive, Börje Ekholm, apologised and said the faulty software was being decommissioned. Press reports put the number of countries at eleven. Ericsson wouldn’t confirm that, and the customer counts differ from one report to the next, so none are given here.
The dates are getting shorter
The House report notes that web certificates issued before 1 March 2018 could last up to 39 months, and those issued after it, 27. The industry’s Baseline Requirements have since gone much further. The maximum falls from 398 days to 47 in three steps: 200 days from 15 March 2026, 100 days from 15 March 2027 and 47 days from 15 March 2029. The ballot behind the change passed in April 2025, with 25 of 30 certificate issuers and all four certificate consumers voting yes.
At 47 days, a calendar reminder isn’t a plan. Somebody will miss one.
Where The Ghost Patch comes in
The blurb for The Ghost Patch describes a process, hidden in a machine’s swap space, that has been quietly fixing a system since January. It renews the expired certificates. It patches the holes. In the Equifax report, neither job got done. The cure for that is duller than a novel: give every certificate an owner, put its expiry date where somebody will see it, and automate the renewal.
The Prologue and Chapter 1 are free to read on this site.
Sources
- US House of Representatives, Committee on Oversight and Government Reform, “The Equifax Data Breach”, majority staff report, 115th Congress, December 2018
- TelecomTV, “Ericsson apologises for software fault that brought down cellular networks”, 7 December 2018
- TechCrunch, “Here’s what caused yesterday’s O2 and SoftBank outages”, 7 December 2018
- CA/Browser Forum, ballot SC-081v3, voting closed 11 April 2025
- CA/Browser Forum, Baseline Requirements, section 1.2.2 (relevant dates)