The fallout from the data center fire at OVH in Strasboug, France continues. OVH, the third largest cloud provider in the world, lost one of 4 data centers in their complex in Strasboug last week. One other data center in the complex was seriously damaged and two more were shut down due to water being sprayed on everything. The shutdown has affected more than 2 million web sites.
It appears that one of their UNINTERRUPTABLE power supplies was, in fact, very interruptable and caught fire, burning down the data center.
The CEO of OVH told customers to fire up their disaster recovery/business continuity plans.
What, you don’t have one?
Well, I guess you can just take the backup that your cloud provider makes and upload it to a web server at another provider.
Wait; OVH has some bad news for you. That backup – it is not recoverable.
So, bottom line, your web site went dark, and all of your data is gone.
I do have some good news.
Your cloud provider probably won’t charge you for the time that your server is dead.
Beyond that, you are pretty much on your own.
If you lost years of customer data, it is gone. IF you have the right kind of cyber insurance it might pay you something, but probably not much. Insurance may pay for the cost of recovering your data, but if the data is gone there is nothing to recover. No payout. If you lose business and you have the right coverage, you will get compensated up to a point, but if it is unrecoverable and the damage to the business is unrecoverable, you probably will not get compensated at a level that you want.
What is important to understand is that your cloud provider likely has no liability for your systems going down or your data going away forever. Check your contract. You might be able to get them to change their contract for you. Oh, wait, probably not. Other than they probably won’t charge you while your servers aren’t working. Your contract may remain in force, so if you do move to another data center provider you may get to pay twice.
OVH is going to start powering up some servers later this week, but that just begins the process. OVH says it will take 6-8 days just to power on the servers in one of the 4 data centers in the cluster. All of the support equipment has to be tested and some probably has to be replaced. Likely the fiber going into the site has to be replaced. Think about the effort to figure out where all those tiny strands of glass are connected to.
They hope to build 15,000 new servers over the next few weeks. That probably is only the beginning of what they need to do. Will this new infrastructure slapped together by very tired, stressed out engineers work reliably?
Let’s assume they get things in one of the data centers working in a couple of weeks. Are your customers going to still be your customers?
What if all of their history with you is gone? Will you even know how to price deals to those customers?
Cloud providers generally operate under what they call a “shared responsibility” model. This is code for “you take it as is and we are not responsible for very much”.
Generally speaking, fires burning down data centers are infrequent and losing all of your data is also infrequent, but infrequent does not mean never and does not mean they are responsible for fixing it or paying you.
If you have not already thought this problem through, our recommendation is that you consult with an expert, because the problems are often subtle, and you may not have what you think you have. We are happy to assist, of course.
I am sure there will be lawsuits – I have no idea what the laws for that, in France, are like, but if the laws are similar to the laws in the U.S., the lawsuits are likely to fail. In the meantime, you are out of business, literally.