This email went out to all Blogs About Hosting clients earlier:
It appears as though the power outage problem at our datacenter has been resolved. We are currently testing the servers – – domain by domain – – to double check for any residual issues or problems. It does not appear, at this time, that there have been any – but we want to be doubly sure.
I have included the update emails that we sent out earlier at the bottom of this notice, just in case any of you did not receive the updates during the downtime.
We are experiencing power problems in our Virginia Datacenter. There is a power outage affecting the entire building.
We are running on backup power, but the generators have not taken the power load from the UPS. Therefore, we have taken the rather extreme action of shutting down servers before we experience irreparable hardware failure.
We are in constant communication with the engineers and expect to have the situation rectified ASAP.
We apologize for this inconvenience and will update you as soon as possible.
Here is the latest word from our Datacenter on the power outage.
The building engineer notified us that the commercial power feed is “flickering”, which indicates that the utility company is working on the issue outside of the building. We are bringing in all available employees to have as many people on hand when power is restored to make sure all servers are powered on as quickly as possible. __________________
It appears the entire state of Virginia is experiencing power outages in various places. A map of Virginia shows the extent of outages: http://www.dom.com/news/northva.jsp (That little blue dot below Hwy 267 west of Fairfax is right where our datacenter is located).
Word has it that power should be restored soon.
We will continue to keep you updated, and are very sorry for this inconvenience , and are currently looking into options available to provide an account credit to everyone for the unexpected downtime today.
We just received word from our datacenter team that power has been restored to the network room and some customer servers. Not all rows have power, and we are working on getting internal systems and network gear turned on first. Power will be restored in phases until all servers are powered on.
Thanks for hanging in there with us! We still have a lot of work to do, but things are looking up now.
Power has been restored to our edge routers, but we are still working on powering up the distribution routers. This means that even though servers are powered on, network connectivity has not yet been restored. The situation is being handled by our very capable network engineers and the distribution routers should be online shortly.
Network connectivity has been restored to some areas of the datacenter. We do not yet have an accurate count of exactly how many servers have network connectivity and how many may still be down for other reasons. We are going to start checking servers (ping sweeps and manual check) to make sure everything comes back online properly.
Please know we are currently working diligently to ensure complete and total connectivity and web service function at this time.
Thank you for your patience.
Blogs About Hosting