Tag Archives: Cloudflare

Cloudflare Exposes Customer Secrets

Cloudflare, the company that helps web sites perform when under stress, including when under denial of service attacks, was the victim of a self induced cyber breach.  For those who are not familiar with Cloudflare, it acts as a front end to a customer company’s web servers. With Cloudflare in front of a company’s servers, the servers can stand up to incredible loads and massive denial of service attacks.

What is more amazing is how they handled it.

First a little bit of the story.

The bug likely exposed data between September of last year and this month.

Cloudflare modifies web pages that pass through its servers as part of the process that it uses.  To do that, they created some software that parses web pages and makes the needed changes.

Tavis Ormandy, a security researcher that works for Google, discovered a bug that caused the Cloudflare servers to send unintended data out with the modified web page.  Among the data that was exposed included authentication tokens, cookies, encryption keys and text of the whole packet.  To make matters worse, the data that was exposed might be from any of it’s customers, not just the web site that the user was visiting.

In addition to that, some of the data was cached by  Google.  While they didn’t say, it is likely that Google web page crawlers were probably among the “users” that visited Cloudflare cached web sites.

Now the good part.

Once Tavis figured out what was going on, it was a Friday night and he knew that he needed to act fast.  An email to the help desk wasn’t going to cut it.

So it put out an emergency plea on his Twitter page.  Given who Tavis is, a LOT of people follow his Twitter feed.   The plea said that he needed to talk to someone on Cloudflare’s security team NOW!.

Again, given who Tavis is, Twitter did it’s Twitter thing and Cloudflare security reached out to Tavis quickly.  He explained the problem to them and within 47 minutes they had deployed a fix that mitigated the problem, but did not completely fix it.

Because of Cloudflare’s size, they were able to quickly create a cross functional team in San Francisco and another in London to work on the problem.  Working 12 hour shifts, they handed off the work internationally 24 hours a day until they were convinced they had all of the leaked data under control.

Within 7 hours they had a complete fix in place but it took several days to work with Google to delete all of the cached data off Google’s servers.  Working 24×7 with Google they now feel that all of the leaked data has been purged, so they were able to notify customers of the situation.

I already received one email from a web site hosted behind Cloudflare telling me that I should change my password.  They said that we should expect many more notices given that Cloudflare protects millions of web sites.

Obviously, this was a pretty subtle bug but what was amazing was that within 47 minutes they were able to deploy the initial mitigating changes and within 7 hours they were able to deploy a complete set of fixes.  Right now, by comparison, the same team that Tavis works for, Google’s Project Zero, just disclosed a Microsoft bug because Microsoft was not able to even release a fix, never mind get it deployed, in 90 days.  7 hours vs 90 days+ is the power of the cloud.  One platform; total control over the environment.  That is an amazing benefit of cloud based services.

While there is nothing for you to do regarding this breach, watch out for notices that tell you to change your password.  Unless you want to suffer the same fate that the DNC did last during the election cycle last year, DO NOT click on any link in those emails – Go to the appropriate website yourself, log in and navigate to the password change page to change your password yourself.

Pretty amazing story.

Information for this post came from Ars Technica and the Cloudflare Blog.

Facebooktwitterredditlinkedinmailby feather

Denial of Service Attack Meets Ransomware

Cloudflare, the denial of service prevention vendor, is reporting hearing of gangs who threaten denial of service attacks unless the victim pays a ransom in bitcoins.  Even though they have heard from over 100 customers, none have been attacked, whether they pay or not.

Here is the scam.  You use the name of a known DDoS group – in this case, the Armada Collective – and threaten people with being attacked.  The attacker may – or may not – have any relation to that group.

You set the payment level low for avoiding the attack – in this case, 10 bitcoins or about $4,000.

You threaten people that if they don’t pay they will be attacked and the fee to stop the attack will go up to 20 bitcoins and go up by 10 bitcoins a day.

You also tell people that you have a magic attack that bypasses anti-DDoS vendors like Cloudflare.

And then, you sit around and wait until some people pay up.

This is a whole lot simpler than actually having a way to launch a DDoS attack or having a way to bypass Cloudflare’s protections.

To date, according to a company that reviews the bitcoin blockchain, these attackers have received at least $100,000.  While that is not much, there may be other bitcoin accounts that they have not examined and  the attackers only cost is sending out a few emails.

While there certainly is no way to know if the attacker can launch an attack, at least so far, they do not seem to have either the ability or desire to do so.

The folks at Cloudflare have talked to other anti-DDoS vendors and they also have customers who have received the emails.

It is certainly possible that these attackers COULD have the capability to launch an attack – we just do not know.

One reason to doubt it is that they seem to be reusing bitcoin accounts between different targets.  Given bitcoin is anonymous, if they did, in fact, plan to attack someone, they would not have an easy way to figure out who has paid and who has not paid.

At the moment, Cloudflare seems to think this is an empty threat, but things do change.  Now that they have been outed on Cloudflare’s blog, they could decide to escalate.  OR, they could decide to fold for a while, wait for people to forget and try it again.

No one knows.

Information for this post came from Cloudflare.

Facebooktwitterredditlinkedinmailby feather