Tag Archives: Electric grid

A Cyber Event Interrupted US Power Grid Operations

Stories – and only stories – abound about whether the Ruskies have infiltrated the US power grid – years ago.  The government is not going to tell the truth for fear of scaring the crap out of people.

On March 5th a “cyber event” interrupted the power grid in parts of the western United States.  While in this case it did not cause blackouts ….

California,  Utah. Wyoming.

A cyber event, according to the DOE, involves unauthorized access  to hardware, software or data.  Who?  Not clear.  What?  Not clear.  Why? Not clear.

But …. not a good sign.

The incident lasted from 7 AM to 9 PM that day.  That is a long time.

The DOE did not respond to a request for information.

The Western Electricity Coordinating Council declined to comment.

For security reasons we cannot disclose any further information was the only comment.

So, while this time we averted disaster, that doesn’t mean we will next time.

Was this a test?  To see how the grid responded?  To test a capability?  Kind of like pulling the fire alarm to see how long it takes for the fire department to arrive.

I suspect that, whatever happened, the feds will *TRY* to fix the problem.   But the feds do not have a great track record.

Now might be a good time to buy a generator.

Consider your own cybersecurity  program.

And your disaster recovery/business continuity program.

If you are not familiar with this song in the movie Hoodwinked, it is entertaining and relevent.  Source: Environment & Energy News.

Electric Grid Still Vulnerable To Hackers

The WSJ Blog had a guest post from Deloitte talking about why the U.S. electric grid is still vulnerable to attack. The short answer is that the grid is being used and managed in a way that it was never designed to operate and the utilities and manufacturers have not adjusted to that fact (see article).

This is going to get a little complicated, so I apologize.  Think about when was the last time you saw an electric generating plant being built in your town.  Probably never.  That is because it likely didn’t happen in your life time.  Many generating plants are 20, 30 and even 40 years old or more.

In most cases, that would be before the Internet was popular.

The controls in those plants were designed to be hard wired from the control to the control room in that plant.  If there was a remote substation and it had a problem, a technician drove over there, unlocked the door and typed on the terminal in the substation to figure out what was happening.

Fast forward that to today.

Substations and control rooms are connected to each other over a variety of networks, some wireless, some not.  Many are on the Internet.  In some cases there is supposed to be an “air gap” between the Internet and the control room, but, mostly for convenience and cost – and sometimes human error – that is mostly a concept.

For a variety of reasons, utilities are really bad about patching their software – in part because the manufacturers don’t have any liability if the systems are broken in to and do have liability if you patch it, break things and shut down the plant.  Even if the manufacturer does not have liability, the utility doesn’t want a generating plant to go off line, so they are very leary of patches.  Unfortunately, this is very similar to hospitals with all their computer based diagnostic and management software.

Back to the electric utilities.  Utilities manage their generating and delivery networks using an HMI or Human Machine Interface.  That is a fancy term for software based control panel.  Most HMIs run on a version of Windows.  Many of them run on Windows XP and some run on even older versions.  Why is that?  Because when the manufacturer wrote the software 10, 20 or more years ago, that was the current operating system.  The manufacturer has moved on to a newer, better software product.  The plant is still the same plant it was 40 years ago.

But, unlike you, who likely replaces your cell phone every two years and your computer every 3-5 years, for the electric utility, things are not so simple.

That HMI connects to thousands, tens of thousands or even hundreds of thousands of knobs, dials and valves in a plant.  When you want to change that HMI, you need to make sure that most of those things still work, otherwise, the plant could go BOOM!

When Microsoft or the vendors come out with a patch – which they do all the time – the utility needs to test them to make sure that they don’t break anything in their installation – not that Microsoft has ever issued a patch that broke anything – silly rabbit!  All you need is a test version of each and every plant (since no two are alike) to test it on.  NOT. GONNA.  HAPPEN.  EVER!

In addition to that, the manufacturer might say that the version of software that the utility is running is no longer supported and they need to upgrade – for a “slight” fee.  And oh, yeah, some of those knobs, dials and valves are not supported in the new version, so they need to be replaced too.  Just figuring out whether there is a problem is complicated and time consuming.

So, the utilities just leave well enough alone.

That was bad enough before the Internet.  Now with the Internet, things are just too tempting to not connect things to the Internet somehow.  Think Iran and Stuxnet.  And that is in a relatively undeveloped country.  In the U.S., where tech is cool, utilities have figured out that they can save time and money by connecting things, sometimes through convoluted channels, some not so convoluted, to the Internet.

Which is why the electric grid is still vulnerable and not likely to change any time soon.

I hate to be pessimistic, but after Bhopal (the Union Carbide plant in India that produced pesticides and had a leak in the 1980s where 500,000+ people were injured and thousands died), the chemical industry got religion about safety and made changes.  The U.S. utility industry understands safety.  They have not yet wrapped their hands around security.  Let’s hope that it doesn’t take a Bhopal-like event for the industry to wrap their hands around security.