AS WE HARDEN PHYSICAL AND CYBER SECURITY IN LIGHT OF SEPT. 11, HACKERS AND A STRING OF NATURAL DISASTERS, we must also consider sustaining operations and security during emergency conditions.
At CenterPoint Energy, we have spent a great deal of time and effort throughout the last eight years hardening our control center against unwanted cyber and physical intrusions. Hurricane Rita provided us with an enormous operational challenge, and we chose to rely on innovative cyber and physical security solutions, rather than accept a lower level of security. Our advanced emergency planning enables us to provide good security while maintaining our operational capabilities.
As the local regulated utility in the fourth largest city in the United States, CenterPoint Energy has a large, hardened control center in Houston, Texas. During emergencies, many additional people arrive at the control center to support emergency operations, and they all bring laptop computers with them. For many of these people, this is their first exposure to the heightened cyber and physical security controls that CenterPoint Energy has implemented.
During the annual review of our Emergency Operating Plan (EOP) prior to the 2005 hurricane season, control center personnel realized the carefully constructed cyber and physical security controls that had been engineered for a peace-time context would be inadequate during emergency operations. We also knew that our EOP “guests” would need unescorted physical access as well as electronic access to the corporate business network from within the control center secure networks. CenterPoint Energy recognized that whatever security infrastructure and technology we put in place to address EOP conditions would be ineffective if we bungled the “people and operations” part of the process. We wanted the security of our control system to be “business as usual” under the most unusual of conditions. We felt that a lower level of security during emergency conditions was unacceptable, because it is during these periods of emergency that the security and reliability of our control system are most critical.
CenterPoint Energy employs a fortified security architecture designed to strictly control traffic between its business network and secured control system networks. In preparing for the 2005 hurricane season, we created an isolated network segment, called the Red Zone, similar in function to a demilitarized zone, in several offices and conference rooms in close proximity to our War Room. Since many of these offices are inhabited during normal operation, these Red Zone rooms can be switched from their secured — or Green Zone — network to the Red Zone network as necessary. Large reversible signs that indicate each room's current (Red or Green Zone) state were mounted outside all of these rooms for rapid incident response if necessary. We planned to do an audit of all guest laptops before they could be allowed in the Red Zones, and have a record of each audit attached to the laptops.
By the late summer of 2005, all of these plans had been put in to place at CenterPoint Energy's control center. Then, on Sept. 21, the third most powerful hurricane on record was headed for a direct hit on the CenterPoint Energy service area and our EOP was activated. On Thursday, Sept. 22, the first wave of guests landed at the control center. Just as predicted, they carried system maps under one arm and a laptop tucked under the other. The personnel stationed at the control center for the duration of Hurricane Rita actually exceeded our normal weekday head count due to the addition of transmission, substation and distribution operations personnel.
While our EOP guests adapted quickly to the heightened security, we underestimated the laptop audit process needs. In normal conditions, we audit control center PCs every 90 days at a minimum. This was not the case for many of the laptops that showed up at the control center. Some laptops required the activation of virus protection, others required virus protection updates, and one or two required operating system updates before security protection could be activated. Due to this experience, CenterPoint Energy is considering having several laptops prestaged and ready for use as well as a separate Red Zone that could be used to audit several laptops simultaneously. In addition, CenterPoint Energy will, as part of our EOP, issue a communiqué to all personnel on the cyber and physical security they can expect at the control center, and request that security protection on all incoming laptops be updated before they arrive.
Due to CenterPoint Energy's careful planning, both Red and Green Zone networks functioned in a business-as-usual manner during the EOP period. Throughout the Rita emergency, CenterPoint Energy achieved our goal of sustaining control system security posture, while delivering the performance and stability our operating personnel expect.
Thomas Flowers, P.E., manages the control systems division for CenterPoint Energy and is the critical infrastructure protection coordinator for CenterPoint Energy's transmission and substation business unit. [email protected]