There is never a good time for the information systems of one of the world’s largest hosted services providers to go down, but in this case, at least it wasn’t April 6th.
One of the Dutch firm Wolters Kluwer’s big services is CCH, the online tax software that many accounting firms, large and small, use.
CCH does have a version of their software that you can install on your own servers, but like many software providers, they want the monthly recurring revenue (MRR) that comes from a hosted version, so they push very hard to get users to use the online versions of their software.
So when, on May 6th, the firm, which provides services to clinicians, nurses, accountants, lawyers, audit, risk, compliance and regulatory servers found out that they were having “network and service issues”, that was not something that executive management wanted to hear. The media says that the firm started seeing “technical anomalies” and an investigation discovered malware. Brian Krebs, a former WaPo reporter and now security blogger started hearing reports several days before, on May 3rd and he reported what he heard to Wolters Kluwer at the time. Some reports said the infection was MegaCortex, a enterprise class ransomware attack.
It is also something that their clients, who often work against deadlines, did not want to hear.
Worse yet, because they had to shut down many of their services, they didn’t have a good way to tell their clients what was happening.
Customers resorted to posting messages on the company’s Facebook page. Some said that the outage was even affecting the locally installed Taxprep T1 and Taxprep T3 software. This is not completely unexpected as that software probably gets information – forms and rules perhaps, from an online repository.
Customers also said on Facebook that they were not terribly happy with Wolters Kluwer’s level of communications. They said “WK needs more professional ways to communicate with corporate clients than through Facebook posts. We’re running businesses not planning reunions. I only found out about this thread from a google search. Facebook isn’t exactly my go-to for reliable information pertaining to business”.
WK said that they restored some services on the 7th but did not have any ETA for when everything was going to be working again.
In a tone-deaf response to the situation, they said “Our process and protocols assure a high degree of confidence in the security of our applications and platforms before they are brought back online. We have seen no evidence that customer data and systems were compromised or that there was a breach of confidentiality of that data.”
Maybe they should start with “we are sorry”.
When one media outlet reached out for information, WK did not even respond.
Enough about the details of this incident. Lets look at what they did wrong and what YOU would do in the same circumstances.
NUMBER ONE – It seems like (I have no inside information) that their cybersecurity incident response program (CSIRP) is inadequate. While they are not the first (or last) large company (over 4 billion Euros a year) to fail to have an adequate CSIRP, that doesn’t give them any forgiveness. Remember the totally botched incident response from Equifax after their breach?
While Wolters Kluwer will survive this – including the inevitable breach of contract and breach of fiduciary responsibility lawsuits that will happen – small companies just literally close their doors. Between the reputation damages, the breach response mitigation costs and the distraction, your average company is going to be very stressed over an incident like this.
If you use online services or provide online services, is your CSIRP up to the task? While the needs of service providers and customers are different, both need FREQUENTLY TESTED CSIRPs.
It appears clear that they did not think about how to communicate with their customers (what a crisis communications expert brings to the party) in the event of a major outage.
While we don’t know yet whether they had cyber breach insurance, do you have it? Not only does it help pay for the cost of responding to an incident, but it gets you a lot of expertise.
If you use online service providers, do you have a business continuity program that allows you to continue conducting business if one of your service providers goes offline like Wolters Kluwer did? I might add – WITH NO WARNING! In this case, it looks like at least parts of the operation were offline for several days. How would you continue to do business?
Notice the communication that came from the lawyers in their statement, “We have seen no evidence that customer data or systems compromised“. This is dramatically different than “no customer data or systems were compromised”. If you were completely clueless, you would have no evidence. That doesn’t mean that your data wasn’t taken.
If you are using an online service provider, what does your contract say about service level agreements or damages. Most of the time the damages are limited to the amount of money you paid during the time they were down. Let’s say that the service costs you $10,000 a month and it was down for 2 days. That means you might get as much as $10,000/30×2=$667. Will $667 cover your losses? Just checking. That assumes that the fine print doesn’t let them completely off the hook.
It doesn’t even look like they were doing a good job of responding to Facebook posts. Their Facebook site currently says:
Please visit our website for an update on our progress with restoring our applications and platforms. We have already brought online several of our systems, including CCH SureTax and CCH Axcess. We are fully committed to restoring remaining services as quickly as possible for our customers. Our teams are working hard around the clock to completely restore access, and appreciate your continued patience. https://bit.ly/302ekMF
The post on their web site really doesn’t have a lot of information on it other than they are working hard on fixing things, which I am sure that they are.
Next lets talk about logging. VERY comprehensive and detailed logs are needed if you are going to be able to figure out if anything was stolen. That is why in many cases, company are forced to assume data was stolen, even though they really don’t have a clue.
And don’t forget about backups. Is everything backed up? Are you sure? Time and again we hear about companies that thought everything was backed up only to find out that it wasn’t And while it is okay if it takes a long time to backup your data, if it is going to take a week to restore it, that could be a problem.
Lastly, lets talk about disaster recovery. Sometimes when you get hit by a ransomware attack, your backups get hit too. While it is hard to protect your DR site (you DO have a DR site, right?), it is possible. The challenge is how long it will take you recover. And that is directly related to cost. The shorter the recovery window, the more money it is going to cost. But, you need to think that through and have realistic expectations. Don’t stick your head in the digital sand and hope. That is not a great strategy.
Bottom line – pretty much every company will have a bad day – or week at some point in time. The real question is how you would respond to it.
There is good news, assuming that you are neither Wolters Kluwer or one of their customers.
You have free training on how NOT to handle a breach. All you need to do is watch what they are doing and ask if that would be how you would like to be treated if you were their customer.
If you are one of their customers – and there are tens of thousands of you out there – I’m sorry that they are not better prepared.
Information for this post came from SC Magazine and Security Week.