Let’s Encrypt revokes 3 million certificates due to a bug

The developers of Let’s Encrypt Certification Authority warned that today, on March 4, 2020, they will be forced to revoke 3,048,289 certificates.

It’s due to the error in the Boulder management software, which is used to check users and their domains before issuing certificates.

“Due to the 2020.02.29 CAA Rechecking Bug 6.9k, we unfortunately need to revoke many Let’s Encrypt TLS/SSL certificates. We’re e-mailing affected subscribers for whom we have contact information”, — wrote Let’s Encrypt representatives.

For comprehension of the problem, you need to understand what the CAA (Certificate Authority Authorization) standard is. It was approved back in 2017 and allows domain owners to prohibit certification authorities for issuing certificates for their domains.

In fact, domain owners can add a special CAA field (CAA field) in the DNS of their domain, and only the certification authority specified in this field can issue a certificate to the domain. All certification authorities (including Let’s Encrypt) are required to follow the CAA, in accordance with the law, otherwise they will face serious fines.

On February 29, 2020, it was reported that the error in the Boulder code related to the implementation of CAA appeared in the summer of 2019 and sometimes forced him to ignore CAA verification.

“If the certificate request contained N domain names that required CAA re-verification, Boulder selected one domain name and checked it N times. In fact, if the subscriber validated the domain for time X, the CAA record for this domain allowed us issuing a Let’s Encrypt certificate during X. But the subscriber was able to obtain a certificate containing this domain name for a period of X + 30 days, even if later on this domain the CAA record was set for the name, prohibiting the issuance of Let’s Encrypt certificates”, – explains Let’s Encrypt developers.

Last weekend, the bug was fixed, and now Boulder checks the CAA field correctly. Let’s Encrypt engineers write that, according to their data, no one had time to abuse the problem. However, the certification authority is forced to revoke all certificates that were issued in violation of CAA audits in accordance with industry rules.

In fact, of all 116,000,000 currently active certificates, only 2.6% were affected by this problem. As mentioned above, these are 3,048,289 certificates. About a million of them are duplicates for the same domains and subdomains, that is, the actual number of certificates affected by the problem is approximately 2,000,000.

Let’s Encrypt already notifies all victims by email. In addition, you can verify your certificates using a special service or relying on the serial numbers of problem certificates (details are published on this special page).

It turns out to be early to feel satisfaction that, according to Google, more than 80% of sites and applications encrypt their traffic if similar problems occur. It’s good that the problem is only in a bug that nobody used, but in fact attackers are quite prone to manipulating certificates, for example, we wrote that attackers pretend to be legal managers of the enterprise to buy security certificates on the Internet and then sell them on underground forums.