Blackouts: The Power Grid Is Too Sensitive for Its Own Good
August 10, 2004
By MATTHEW L. WALD
Washington Post Staff Writers and Washington Post Staff Writers
or a year, engineers have been trying to figure out how to prevent the kind of chain-reaction collapse that brought down the electric power grid from Michigan to New York last Aug. 14, causing the largest blackout in American history. But they are confronting a vexing problem: The disturbance spread so quickly that day largely because hundreds of components acted exactly as they had been programmed to do.
The components were relays -- devices about the size of a microwave oven, installed at critical nodes along the power grid, that are supposed to protect its key components by disconnecting them in the event of a change in current, voltage or frequency that could cause damage.
Relays did not start the blackout; the cause has been traced to much simpler problems, like utilities' failing to trim trees under power lines in Ohio and failing to notice when the power lines sagged into the trees and went out of service. But those failures forced electricity to take other routes, causing power surges. In places that were not directly threatened by the surges, the relays sensed them anyway and proceeded to shut down crucial components of the grid.
As the United States and Canadian governments put it in November in their report on the blackout, these relays helped spread a ''cascade'' of failures, reducing ''the potential time in which grid operators might have identified the growing problem and acted constructively to contain it.''
In a sense, the relays did their job, because they protected the power lines, generators and substations that make up the electric system; there was hardly any damage to the network of electrical equipment, and most of the system was back within 24 hours. But one transmission expert, Douglas A. Voda, a senior vice president at the electric equipment manufacturer ABB, said, ''There's a conflict between protection of an asset and protection of a system.''
The report identified ''Zone 3'' relays as a critical part of the problem on Aug. 14. Zone 3's are the most sensitive relays, designed to detect the smallest disturbances to the system and to assure that a circuit is broken even if the problem is far away and the nearest relay has failed to react.
Those relays thus shut down components that were not threatened by the power surges. Alison Silverstein, a senior policy analyst for the Federal Energy Regulatory Commission who was an author of the blackout report, said some of the relays that were set to operate as Zone 3's were not even recognized by their utility owners as being set that way.
The relays, said Paul Proudfoot, an investigator at the Michigan Public Service Commission, were set to deal with normal operating conditions, and events on Aug. 14 created power flows that engineers never foresaw.
For example, as the problems in Ohio grew, the Detroit area was called on to send 2,000 megawatts from a regional utility company, American Electric Power, toward Toledo. Had it done so, the blackout might have been stopped, Mr. Proudfoot said. But that might have required blacking out part of Detroit, or setting the relays in advance to allow such a huge power transfer.
Nobody planned for that, he said, adding, ''If you and I had trudged into the office two days before the blackout and said we've got to supply 2,000 megawatts to Toledo from American Electric Power, they'd have shooed us out like morons.''
Another problem is the potential mismatch of generation and load -- that is, between supply and demand. Some relays are set to interrupt circuits if electric frequency drops too far below the standard 60 cycles per second, or if the voltage drops too low; both are signs of generation deficiency. But if transmission lines fail and the electric grid begins to break into isolated islands, some of those islands will have too much power and some will have too little.
Robert Blohm, an electricity consultant based in New Jersey, said one problem with the current system was that it could sometimes reduce demand -- by shutting off a factory or a neighborhood, say -- but was not set up to deal with destabilizing surpluses, by dropping generation, partly because the companies that make power do not want to lose the income.
To the extent that automatic actions were taken on Aug. 14, the relays were like sailors looking through telescopes, each with a narrow view of a complex panorama. They each saw a power flow on a single line, and determined independently to pull the plug on a part of the grid. It was far too fast for human intervention. In a few seconds, 50 million people were in the dark.
Experts say new attention to the relays could reduce the tendency of the grid to unravel after an initial failure. One approach is to build a computer simulation of what happened in the blackout, and then tinker with the settings, to determine what would have happened if various parts had been set to act later, or, perhaps, earlier. Had some lines not tripped, the disturbance could have weakened and eventually petered out, like a ripple in a pond that disappears as it travels farther from its source, the government report said.
So the Northeast Power Coordinating Council, an industry group covering much of the Northeast, and others are conducting a millisecond-by-millisecond reconstruction of the cascade -- an event so complicated that the engineers, who once hoped to have the reconstruction finished by the first anniversary, are now aiming for November.
Adding to the complication, disturbances on the electrical grid develop on a time scale measured in cycles, each cycle being one-sixtieth of a second, the frequency of the alternating current. Thus effective action requires almost instantaneous communication and computerized decision making.
To Roger W. Gale, a former Energy Department official who is now an electricity consultant, the term ''cascade'' suggests the possibility of a remedy: linking the relays by fiber optics or microwaves, and integrating their data. ''Instead of cascade of shutting down,'' he said, ''we would have a cascade of information flow.'' A central computer would ''read the real logic of what's happening.''
That would be integration on a wider scale than the grid has ever had. ''Most of the methodologies and concepts of operation for power systems were developed back in the 60's and 70's, before having access to a lot of computational and communication and control tools and devices we now have,'' said Massoud Amin, an electrical engineering professor at the University of Minnesota.
Solving the relay problem is one of three approaches to building a grid that tolerates upsets, experts said. A second is changes at generating stations, which can be fussy about fluctuations in the voltage or frequency of power coming in from the outside. According to the report, some were set to trip much sooner than protecting the generator required.
The third is the possibility of reducing the grid's complexity by breaking it up into smaller areas, each using a conventional alternating current system, and linking them with direct current, which transfers energy but not disturbances.
But what last summer's blackout illustrates is that until changes are made, an error at one point in the grid will still have the potential to bring it all down.
George C. Loehr, a former executive director of the Northeast Power Coordinating Council, is a management consultant and member of the group Power Engineers Supporting Truth, which has made suggestions on preventing future blackouts. ''As long as you have a big interconnection and everybody's part of it,'' he said, ''we're vulnerable for whatever happens in one part of the interconnection to happen everywhere else.''