Beginning Saturday, January 15, 2022, Enom experienced a series of complications with a planned data center migration that caused significant disruptions for a subset of our customers.
We sincerely apologize to all of those impacted. We pride ourselves on being a reliable domain registration platform, and this weekend we fell short. We are committed to regaining your trust and to serving you better.
A full internal audit is underway and an incident report is forthcoming. This will include a summary of events and scope, learnings, and policy and process changes to mitigate future issues.
In the meantime, we have consolidated an overview and timeline of events to transparently share with our valued community.
EVP, Tucows Domain Services
Enom had a large-scale data center migration planned for Saturday, January 15, 2022, from 09:00 EST to 21:00 EST.
We first notified Enom resellers of this migration via email on December 7, 2021, with reminder emails on January 7 and January 12, 2022. In these communications, resellers were told that they would be unable to register and manage domains, email, and SSL certificates through the Enom Control Panel and API during the 12-hour maintenance period, and that DNS services were expected to resolve normally throughout the entire duration of the migration.
As the data center migration was underway, we experienced delays in transferring our database, which extended the duration of the work period and the length of time registration and management services were offline.
Alongside this issue, a bug in the new DNS provisioning system resulted in incomplete DNS records for up to 350,000 domains.
A handful of customers began reporting issues on Saturday, January 15, 2022 at 14:00 EST, which continued overnight. Enom engineers were actively resolving the reported DNS issues throughout Sunday, January 16, 2022, while a parallel effort was underway to determine the magnitude of the problem. All DNS resolution issues—including MX records—were resolved by January 17, 2022 at 13:45 EST.
At the same time, the following isolated issues arose:
|Inability to view, access, and search on domains under management via Enom Control Panel||Jan. 17, 2022 @ 13:00 EST|
|Credit card payment processing issues||Jan. 17, 2022 @ 21:00 EST|
|Enom Control Panel login issue||Jan. 18, 2022 @ 10:00 EST|
|Delays in system-generated email||Jan. 18, 2022 @ 12:00 EST|
A cadence of incident updates was posted via Enom Status page, beginning Saturday, January 15, 2022, at 21:00 EST. We marked this outage as resolved on Tuesday, January 18, 2022, at 15:45 EST.
More information will be publicized in our incident report. We’ll distribute a summary of findings once complete.
On January 22, 2022, at 15:31 EST the Incident Report has been published to the Enom Statuspage.