Published: March 16, 2016
by Alex Woodie
Cyber intruders who gained access to an AS/400 at a water district were able to manipulate the flow of chemicals into the public water supply, Verizon says in its latest Data Breach Digest. While customers served by the water district were not harmed, the episode shows the potential consequences of failure to properly secure critical systems in an increasingly connected world.
Verizon dedicated five pages to laying out the disturbing breach of a water district that it referred to as Kemuri Water Company (KWC), which is not a real name. The water district had first contacted Verizon's RISK Team to conduct a proactive assessment of its security system. KWC insisted it had never been compromised. However, after just a little probing, the RISK Team found evidence of an actual breach by a "hacktivist" group with ties to Syria.
According to details of the breach, the hacktivists first infiltrated KWC's systems by exploiting known security vulnerabilities in a Web-based payment server application that KWC had set up to allow customers to pay their bills and view water usage information. Unfortunately, that system was directly linked by cable to its backend "AS400" system. Making matters worse, the water district stored login credentials for the AS/400 on that front-end Web server, and the AS/400 was directly connected to the Internet.
KWC's aging AS/400 system (it was more than 10 years old, according to Verizon) served many purposes, as it does for most organizations that run the platform, which has gone through several name changes (iSeries, System i) and is now officially called IBM i for Power Systems by IBM. Among the applications are core financials, billing, and database containing personally identifiable information (PII) about customers.
SCADA Plot
The water district also used the AS/400 as a supervisory control and data acquisition (SCADA) system to directly control hundreds of programmable logic controllers (PLCs) that opened and closed valves that govern the flow of water and chemicals used to treat the water. Verizon's RISK Team found evidence that the hacktivists logged into this operational technology (OT) system and manipulated the valves controlling the flow of chemicals.
"It became clear that KWC management was aware of potential unauthorized access into the OT systems of the water district," Verizon says in its report. "More specifically, an unexplained pattern of valve and duct movements had occurred over the previous 60 days. These movements consisted of manipulating the PLCs that managed the amount of chemicals used to treat the water to make it safe to drink, as well as affecting the water flow rate, causing disruptions with water distribution."
The hackers also stole more than 2.5 million files that contained PII data, according to the report. There was no evidence that the data breach led to any fraudulent activity, Verizon says. That's not surprising, considering the hackers worked out of IP addresses that were used in previous hacktivist activities, the telco and IT giant says. "The typical semantic footprint of a hacktivist attack shows greater interest in denying and disrupting the victim's ability to conduct business than stealing information for financial gain," Verizon says in its report. "That was definitely the case here."
The bad news, of course, is that cyber criminals operating in the Middle East were able to release potentially dangerous chemicals into the public drinking water supply serving several counties in the United States. , KWC had systems in place to detect the chemical release and took immediate steps to fix the problem after being alerted to the problem.
"KWC's breach was serious and could have easily been more critical," Verizon says in its report. "If the threat actors had a little more time, and with a little more knowledge of the ICS/SCADA system, KWC and the local community could have suffered serious consequences."
Lessons Learned
From an IT and IBM i point of view, there are several lessons to be learned from the KWC breach. Some of the lessons are obvious, while others less so.
Among the basic lessons at play here are the need to apply patches and remediate known security vulnerabilities that affect Web applications. It's also not a good idea to store user names and passwords for critical systems like AS/400s in plain text on front-end Windows and Linux servers, or to expose backend servers like the AS/400 to the public Internet. This is the low-hanging fruit of IT security, but all too often, organizations continue to violate these basic tenets of security and rack up the "duh" moments by the dozen.
Having SCADA systems directly connected to front-end billing systems (as KWC had) is not a best practice, but is undoubtedly fairly common. Verizon also took KWC to task for employing a single administrator for the AS/400 system. While having duplicate hardware, software, and network connectivity is standard practice for many shops, having redundancy in personnel is also something worth considering.
But some of the other lessons from the KWC hack are not so obvious.
Not too long ago, OT systems such as SCADA were housed separate from IT systems, such as corporate networks and payment servers. That "air gap" served as a barrier to cyber snoopers and criminals. But as technology matured and data centers grew, organizations recognized there were benefits to grabbing more "real time" data from operational systems, and hence, that air gap disappeared. The problem is compounded by having IT administrators remotely manage OT systems over the Internet.
"This new technology can provide a false sense of security, as operating budgets do not take into account the time to support, maintain and operate the new technology--thus it becomes ineffective," Verizon concludes. "Threat actors have the upper hand when technology is not maintained and they develop ways to circumvent how it works. Continuous operational and security training, coupled with additional staff, are required to stay on the same level playing field as threat actors."
You can download a copy of the Verizon Breach Digest at www.verizonenterprise.com/verizon-insights/data-breach-digest/2016/.
Leave a Reply