NERC Case Notes: Reliability Standard CIP-007-6

Unidentified Registered Entity 2 (SERC_URE2), FERC Docket No. NP18-25-000 (August 30, 2018)

NERC Violation ID: SERC2017017669

Reliability Standard: CIP-007-6

Requirement: R2, P2.2

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: SERC Reliability Corporation (SERC)

Issue: SERC_URE2 failed to assess three security patches for applicability within 35 days of availability. SERC_URE2 submitted a Self-Report stating that the violation stemmed from not assessing the security patches until two days past the 35-day window. The patch was sourced from SERC_URE2’s parent company, and subsequently the parent company staff discovered the violation during a business operations staff meeting and found that it had not conducted the patch assessments due to an oversight of responsibilities. The three security patches were applicable to production AD domain controllers classified as Electronic Access Control and/or Monitoring Systems (EACMSs) associated with eight High Impact BES Cyber Systems. SERC_URE2 identified the root causes of the violation as a lack of controls and human performance issues, specifically referring to the parent company failing to integrate alerts or reminders in its monitoring processes. SERC_URE2 also cited a year-end workload compounded by unplanned employee absences as an explanation for the oversight.

Finding: SERC found the violation constituted a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). By its failure to assess security patches, SERC_URE2 could have permitted known vulnerabilities to remain available for exploit, giving bad actors additional time to exploit and potentially degrade local operations or impact the BPS. However, the EACMSs at issue resided within secured Physical Security Perimeters with access controls and real-time monitoring and alerting for anomalous activity. This violation involved only three security patches affecting a small number of SERC_URE2 specific EACMs. Although the three security patches were assessed two days late (at 37 days instead of 35 days), the patches were applied within 44 days of release. The duration of the violation started when SERC_URE2 exceeded the 35-day window to assess released security patches and ended when it assessed the missed patches. SERC considered SERC_URE2’s internal compliance program as a mitigating factor but found SERC_URE2 and its affiliate’s compliance history to be an aggravating factor in the penalty determination. To mitigate the violation, SERC_URE2, among other steps, evaluated missed security patches for applicability, addressed human performance, and implemented additional controls and training to support the assessment processes.

Penalty: $220,000

FERC Order: Issued August 30, 2018 (no further review)

Unidentified Registered Entity 1 (NPCC_URE1), FERC Docket No. NP20-3-000 (October 31, 2019)

NERC Violation ID: NPCC2018019847

Reliability Standard: CIP-007-6

Requirement: R2

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Northeast Power Coordinating Council, Inc. (NPCC)

Issue: An on-site compliance audit revealed that an unidentified entity failed to include three (3) Medium Impact Bulk Electric System (BES) Cyber Systems in its patch management process. These unmanaged switches were not being tracked or evaluated. The root cause of this violation was a misunderstanding of the applicability of requirements, which resulted in the switches being excluded from patch evaluations.

Finding: NPCC found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system reliability. Although not evaluating applicable systems for cyber security patches subjects the devices to exploitation and unauthorized access, no harm resulted from this violation. The duration of the violation began on July 1, 2016, when the entity failed to include the BES Cyber Systems and ended on July 19, 2018 when the entity added the BES Cyber Systems to its patch tracking spreadsheet and reviewed software updates for applicability. NPCC deemed the entity’s internal compliance program to be a neutral factor in the penalty determination, and after reviewing the entity’s compliance history, found that there were no relevant instances of noncompliance. To mitigate the violation, the entity updated its patch checklist to include a check for firmware, reviewed firmware and the NERC’s standard and its process documentation.

Penalty: $84,000

FERC Order: October 31, 2019

Unidentified Registered Entity 1 (NPCC_URE1), FERC Docket No. NP20-3-000 (October 31, 2019)

NERC Violation ID: NPCC2018019846

Reliability Standard: CIP-007-6

Requirement: R5

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Northeast Power Coordinating Council, Inc. (NPCC)

Issue: A compliance audit determined that an unidentified entity failed to change known default passwords on forty-five (45) Medium Impact Cyber Assets. The root cause of this violation was a failure to implement Critical Infrastructure Procedures Standard Requirements.

Specifically, the entity chose not to change passwords on the applicable systems because the substations did not have External Routable Connectivity.

Finding: NPCC found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system reliability. Although unchanged default passwords can provide attackers with unauthorized access to applicable Cyber Assets, no damage occurred as a result of the violation. The duration of the violation began on July 1, 2016, when the entity failed to change the known default passwords and ended on September 28, 2018 when the passwords were changed. NPCC deemed the entity’s internal compliance program to be a neutral factor in the penalty determination, and after reviewing the entity’s compliance history, found that there were no relevant instances of noncompliance. To mitigate the violation, the entity changed the passwords and updated its training program.

Penalty: $84,000

FERC Order: October 31, 2019

Unidentified Registered Entity 1 (MRO_URE1), FERC Docket No. NP19-17-000 (August 29, 2019)

NERC Violation ID: MRO2017018152

Reliability Standard: CIP-007-6

Requirement: R5.7

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Midwest Reliability Organization (MRO)

Issue: During a compliance audit, MRO determined that multiple Cyber Assets were not configured to either limit the number of unsuccessful authentication attempts or did not generate alerts after a threshold of unsuccessful authentication attempts. The root cause of this violation was an unidentified entity’s failure to fully understand the reliability standard and requirement. The entity believed that it was not required to file a Technical Feasibility Exception (TFE) if the device could not reach its requirements. Additionally, the entity only considered whether a device had the capability to limit the number of unsuccessful authentication attempts and failed to consider a device’s event forwarding capability in conjunction with a collection system(s) that can generate an alert.
Finding: MRO found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system (BPS) reliability. At the time of the compliance audit, the majority of the devices were receiving some level of protection, while others were granted a TFE that resolved the noncompliance or had a low inherent risk to the BPS. No harm is known to have occurred. The violation began on July 1, 2016 when the reliability standard became mandatory and enforceable and ended on October 31, 2018 when all applicable Cyber Assets were configured to either lockout or send a real-time alert. MRO considered the scope of the noncompliance and the discovery method to be an aggravating factor in the disposition. Although noncompliance that impacts a high population of applicable devices should be self-detected through internal controls, the noncompliance minimally impacted the BPS. Thus, although MRO determined that the noncompliance should not be eligible for the compliance exception treatment, it decided that a financial penalty was not warranted. To mitigate the violation the entity submitted a TFE, conducted an extent of condition review, configured all applicable devices to either lockout or send a real-time alert, augmented the account implementation form to add additional steps and permit the elevation of concerns for peer or supervisory review, validated updated processes, and provided training.

Penalty: $0

FERC Order: August 29, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-15-000 (July 31, 2019)

NERC Violation ID: WECC2017016928

Reliability Standard: CIP-007-6

Requirement: R2, P2.1, 2.2, 2.3

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Western Electricity Coordinating Council (WECC)

Issue: On February 3, 2017, an unidentified entity submitted a Self-Report when it determined that it was not in compliance with the Reliability Standard. Specifically, the entity’s patch management process utilized a configuration management application to maintain a comprehensive software whitelist that was intended to track all software and the associated security patch sources installed on all High Impact and Medium Impact BES Cyber System (BCS) BES Cyber Assets (BCAs), and the associated Electronic Access Control and Monitoring System (EACMS), Physical Access Control Systems (PACS), and Protected Cyber Assets (PCAs). During the entity’s efforts to true-up its software whitelist to the actual installed software on its BCS BCAs and the associated EACMS and PACS, the entity discovered that several software applications were not originally captured in the software whitelist during a Critical Infrastructure Procedure (CIP) Version 5 implementation effort. Furthermore, on December 13, 2016 and February 2, 2017, the entity discovered that patch sources were missing from the software whitelist. Since none of this software was being tracked for cyber security patches, there were no patches being evaluated, applied, or for which mitigation plans were created. WECC determined that the entity failed to identify a source or sources to track the release of cyber security patches for applicable Cyber Assets and failed to evaluate security patches for applicability for the software stored on these devices. Furthermore, the entity did not create a dated mitigation plan or revise an existing mitigation plan. The root cause of the violation was ill-defined, misunderstood, or unenforced management policy guidance and expectations. Specifically, the entity had no project plans in place to address the requirements, did not know the scopes of the tasks, and had constrained resources. Furthermore, there was a misalignment of the operations team’s skill sets and resource assignment.

Finding: WECC found the violation constituted a moderate risk and did not pose a serious or substantial risk to bulk power system reliability. The entity had implemented strong controls which included none of the affected Cyber Assets being internet-facing and employing multiple monitoring systems and methods to log, detect, and alert on the overall health of the affected Cyber Assets. The violation began on July 1, 2016 when the Reliability Standard and Requirement became enforceable and ended on December 19, 2018 when the entity completed its mitigation plan. WECC considered the entity’s internal compliance program to be a mitigating factor and its compliance history to be an aggravating factor in the penalty determination. The entity received mitigating credit for admitting to the violation, and WECC applied mitigating credit for improvements that the entity was making on its system. These improvements include a System-Wide Transmission Protection Standardization and Upgrade Project, which was not undertaken as a result of a mitigation plan, but rather was the result of the entity’s systematic post-event root cause analysis and corrective action planning program. To mitigate the violation, the entity, among other things, inventoried all installed software applications utilizing its Security Information and Event Management (SIEM) tool and added any missing installed software applications to asset management tool software, used a whitelist to ensure that all installed software applications are added to and being trained in the vulnerability management service, developed and documented a process for the evaluating software and firmware entries in the software whitelist, and held training.

Penalty: $74,000

FERC Order: July 31, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-15-000 (July 31, 2019)

NERC Violation ID: WECC2017016939

Reliability Standard: CIP-007-6

Requirement: R3; P3.1

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Western Electricity Coordinating Council (WECC)

Issue: On February 6, 2017, an unidentified entity, submitted a Self-Report after it realized that its physical port locking method for deterring, detecting, or preventing malicious code on CIP applicable Cyber Assets had not been locked on Medium Impact BES Cyber System (MIBCS) BCAs without External Routable Connectivity (ERC) as of July 1, 2016. The entity identified this violation on January 19, 2017 and found that the employee responsible for the task mistakenly applied the CIP-007-6 R1, Part 1.1 methodology of leaving the physical ports open, instead of the logical ports. After identifying the missing port locks, the entity began the process of physically port locking ports on BCAs, which was completed on February 10, 2017. On some BCAs, the entity did not physically port lock one port because they were in the process of decommissioning the devices, which was completed on December 13, 2016. Furthermore, antivirus had not been installed. The root cause of the violation was a lack of understanding the documented process.

Finding: WECC found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system reliability. While the entity failed to deploy methods to deter, detect, or prevent malicious code on MICBS without ERC, the entity implemented an extensive Security Information and Event Management (SIEM) architecture that monitors changes on HIBCS and MIBCS Cyber Assets, alerts the operations group of unauthorized changes, and monitors network switch configuration. The violation began on July 1, 2016 when the Reliability Standard and Requirement became enforceable and ended on May 19, 2017 when the entity physically port locked the remaining BCAs in scope and added antivirus to the PCA. WECC considered the entity’s internal compliance program to be a mitigating factor and determined that the entity’s compliance history should not serve as a basis for aggravating the penalty because it was distinct, separate, and not relevant to this violation. The entity received mitigating credit for admitting to the violation, and WECC applied mitigating credit for improvements that the entity was making on its system. These improvements include a System-Wide Transmission Protection Standardization and Upgrade Project, which was not undertaken as a result of a mitigation plan, but rather was the result of the entity’s systematic post-event root cause analysis and corrective action planning program. To mitigate the violation, the entity, among other things, placed tamper tape on open ports on the BCAs in scope, implemented a mandatory escort checklist, documented a process to capture cyber security controls for all new cyber assets and/or new device types at transmission facilities, installed antivirus, and communicated to applicable personnel new process changes.

Penalty: $74,000

FERC Order: July 31, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-15-000 (July 31, 2019)

NERC Violation ID: WECC2017016938

Reliability Standard: CIP-007-6

Requirement: R4; P4.2.2

Violation Risk Factor: Medium

Violation Severity Level: High

Region: Western Electricity Coordinating Council (WECC)

Issue: During a December 7, 2016 log review, an unidentified entity identified a potential logging issue with its Security Information and Event Management (SIEM), the event logging and alerting tool utilized to perform CIP-007-6 R4 for its High Impact BES Cyber Systems (HIBICS) and Medium Impact BES Cyber System (MIBCS) and the associated Electronic Access Control and Monitor System (EACMS), access Protected Cyber Assets (PCAs), and Physical Access Control Systems (PACS), as applicable, for technically capable devices. As a result, the entity worked with the SIEM vendor to determine that the SIEM database had been corrupted since November 8, 2016. Subsequently, the entity rebuilt the indexes in the database and brought the SIEM back to a normal operating state by December 26, 2016. While during the 48-day timeframe in which the SIEM database was not operating correctly, the identified Cyber Assets were still logging locally. Additionally, during the 48-day timeframe, Cyber Assets were not able to send logs to the SIEM in order for the SIEM to generate alerts for a detected failure, but were cached on local devices; thus, when the SIEM became operational, all logs were forwarded on, normalized, and correlated. Therefore, once the SIEM database was repaired, all data was restored and captured for the 48-day timeframe. Furthermore, the antivirus continued to function as expected during this timeframe and could send its logs to the antivirus policy administrator console. Finally, the entity reported that as a result of the issue with the SIEM, the Cyber Assets associated with its HIBCS were not included in the 15-calendar day log review during the 48 days in which the SIEM database was not operating correctly. The unidentified entity submitted a Self-Report on February 6, 2017. The root cause of the violation was an equipment malfunction.

Finding: WECC found the violation constituted a moderate risk and did not pose a serious or substantial risk to bulk power system reliability. Although the entity failed to generate alerts for a defected failure of Part 4.1 event logging, as required by the Reliability Standard, the entity implemented strong controls including antivirus, Physical Security Perimeters, and task reminders to remind employees to review logs. The violation began on November 8, 2016 when the SIEM stopped functioning correctly and ended on December 26, 2016 when the SIEM began logging and alerting for events. WECC considered the entity’s internal compliance program to be a mitigating factor and its compliance history to be an aggravating factor in the penalty determination. The entity received mitigating credit for admitting to the violation, and WECC applied mitigating credit for improvements that the entity was making on its system. These improvements include a System-Wide Transmission Protection Standardization and Upgrade Project, which was not undertaken as a result of a mitigation plan, but rather was the result of the entity’s systematic post-event root cause analysis and corrective action planning program. To mitigate the violation, the entity, among other things, corrected the SIEM database corruption verified that the SIEM database was operational, conducted a summary review of logs, and provided training.

Penalty: $74,000

FERC Order: July 31, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-15-000 (July 31, 2019)

NERC Violation ID: WECC2017016940

Reliability Standard: CIP-007-6

Requirement: R5; P5.5.1, P5.5.2

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Western Electricity Coordinating Council (WECC)

Issue: On February 6, 2017, an unidentified entity submitted a Self-Report after it violated the Reliability Standard. While the entity’s engineers were executing its change management process to install new Medium Impact Bulk Electric System (BES) Cyber System (MIBCS) BES Cyber Asset (BCA) BCAs at a switching station on December 9, 2016, the entity’s Operations subject matter experts (SMEs) provided the BCAs temporary passwords so that they could be functionally tested prior to their deployment into the Electronic Security Perimeter (ESP) where the BCA password length and complexity would be automatically enforced via a substation remote access system. After the Operations SMEs provided the temporary passwords, the SMEs identified that both the temporary passwords and the enforcement of password length and the complexity in the substation remote access system for these particular BCAs did not meet minimum password parameters. Even though the substation remote access system and the BCAs could support such parameters, upon discovery of the violation, it was determined that the Operations SMEs would enforce password length and complexity procedurally until the scope of the potential issue could be determined and corrected in the substation remote access system. Furthermore, the entity determined that BCAs and Electronic Access Control and Monitoring Systems (EACMS) Cyber Assets associated with the MIBCSs at switching stations did not have the appropriate password parameters in place. However, as of January 25, 2017, all passwords for the Cyber Assets had been updated to meet length and complexity requirement, and all password setting within the substation remote access system had been corrected. The root cause of the violation was a lack of internal controls.

Finding: WECC found the violation constituted a moderate risk and did not pose a serious or substantial risk to bulk power system reliability. Although the entity failed to implement a technical or procedure process for password-only authentication for interactive user access and to enforce password parameters, it had strong controls. Therefore, while the password length and complexity did not meet the requirements between July 1, 2016 and January 25, 2017, password enforcement was still set to a minimum length of five characters or more (depending on the device type) and a minimum complexity of two different character types during the violation duration. The violation began on July 1, 2016 when the Standard and Requirement became mandatory and enforceable to the entity, and ended on January 25, 2017 when password parameters were set for the accounts to the devices in scope. WECC considered the entity’s internal compliance program to be a mitigating factor and found that its compliance history should not serve as a basis for aggravating the penalty because it was distinct, separate, and not relevant to this violation. The entity received mitigating credit for admitting to the violation, and WECC applied mitigating credit for improvements that the entity was making on its system. These improvements included a System-Wide Transmission Protection Standardization and Upgrade Project, which was not undertaken as a result of a mitigation plan, but rather was the result of the entity’s systematic post-event root cause analysis and corrective action planning program. To mitigate the violation, the entity, among other things, updated the passwords to meet length and complexity requirements, updated the Security Information and Event Management policy test, created a tool to assist in identifying CIP requirements, and held a mitigation closure meeting.

Penalty: $74,000

FERC Order: July 31, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-12-000 (June 27, 2019)

NERC Violation ID: WECC2017018752

Reliability Standard: CIP-007-6

Requirement: R5; P5.5

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Western Electricity Coordinating Council (WECC)

Issue: On December 5, 2017, an unidentified entity submitted a Self-Report after it violated the Reliability Standard. While an employee was changing passwords for non-Critical Infrastructure Procedures (CIP) devices, an employee also changed the passwords of two BES Cyber Assets (BCAs) that were associated with a Medium Impact Bulk Electric System (BES) Cyber System (MIBCS) at the primary and back-up Control Centers and used the same password requirements of the non-CIP devices. The entity discovered the noncompliance on December 9, 2016 during its quarterly access review and determined that the entity failed to implement its documented process for password-only authentication for interactive user access when it did not enforce password parameters for length and complexity. The root cause of the violation was incorrect performance due to lack of process controls around password changes. Specifically, an employee, who had authorization to change passwords for both CIP and non-CIP devices and who was tasked with changing passwords for non-CIP devices while performing routine tasks on the non-CIP devices, also changed the passwords on two BCAs.

Finding: WECC found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system reliability. Although the entity failed to implement its documented process for password-only authentication for interactive user access when it did not enforce password parameters for length and complexity, no harm is known to have occurred. The violation began on November 2, 2016 when password length and complexity were not enforced on the two BCAs and ended on December 14, 2016 when the entity enforced the password and complexity on the two BCAs. WECC considered the entity’s internal compliance program to be a mitigating factor and found that its compliance history was an aggravating factor in determining the disposition track. The entity did not receive mitigating credit for self-reporting. To mitigate the violation, the entity changed the password length and held a meeting with members of the team to discuss CIP asset password policy and employee responsibilities and reconfigured the BCAs in scope to no longer be CIP assets.

Penalty: $0

FERC Order: June 27, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-12-000 (June 27, 2019)

NERC Violation ID: WECC2018019340

Reliability Standard: CIP-007-6

Requirement: R2; P2

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Western Electricity Coordinating Council (WECC)

Issue: On March 1, 2018, an unidentified entity submitted a Self-Certification stating that it was in violation of the Reliability Standard. During its Self-Certification review on January 16, 2018, the CIP Lead discovered that commercial software had not been evaluated for security patch applicability that was installed on two Electronic Access Control and Monitoring Systems (EACMS) Cyber Assets associated with a Medium Impact Bulk Electric System (BES) Cyber System (MIBCS) at its primary and backup Control Centers. The software had been removed from a list on an unnamed spreadsheet. The version of the software residing on the EACMS Cyber Assets was listed incorrectly. Earlier in the year, the responsible engineer removed the PACS Cyber Assets from its association to a BES Cyber System. As that was the only Cyber Asset listed on the spreadsheet as containing the software, the Cybersecurity Supervisor assumed that all instances of the software had been removed from all MIBCS and associated Cyber Assets. The employee therefore annotated the entry on the spreadsheet as no longer requiring assessment, when in fact, a version of the software was still residing on the two EACMS Cyber Assets. The root cause of the violation was an inadequate security patch management tracking process. Specifically, the task of when and how to remove a source from the security patch tracking list was not covered in the documented process.

Finding: WECC found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system reliability. Although the entity failed to appropriately implement its patch management process to track, evaluate, and install cyber security patches for applicable Cyber Assets, which should include the identification of a source or sources for the release of cyber security patches for applicable Cyber Assets that are updateable and for which a patching source exists, the entity implemented good compensating controls and no harm is known to have occurred. The violation began on September 17, 2017 when cyber security patches for the two EACMS should have been tracked and ended on February 20, 2018 when the entity tracked, evaluated, and applied applicable updates. WECC considered the entity’s internal compliance program to be a mitigating factor and found that its compliance history was an aggravating factor in determining the disposition track. The entity did not receive mitigating credit for self-reporting. To mitigate the violation, the entity evaluated commercial software updates, applicable security patches to the EACMS Cyber Assets in scope, updated its Security Patch Management Program, and provided training to stakeholders on the updates to the Security Patch Management Program.

Penalty: $0

FERC Order: June 27, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-12-000 (June 27, 2019)

NERC Violation ID: WECC2017018732

Reliability Standard: CIP-007-6

Requirement: R5

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Western Electricity Coordinating Council (WECC)

Issue: On December 4, 2017, an unidentified entity submitted a Self-Report stating that it was in violation of the Reliability Standard. The entity reported that on July 17, 2017, it discovered that three Cyber Assets, which on January 26, 2017 it categorized as Protected Cyber Assets (PCAs) associated with Medium Impact BES Cyber Systems (MIBCS) without External Routable Connectivity (ERC) at three separate substations, did not have passwords. Thus these Cyber Assets did not have methods to enforce authentication of interactive user access. The PCAs contained software and applications written in-house by the entity and an administrator account where the password functionality had not been enabled. When new Reliability Standards went into effect, these Cyber Assets were not updated to enforce authentication of interactive user access because of potential operational and safety matters impacts, as well as a lack of clarity over the interpretation of the Requirement. The root cause of the violation was an insufficient number of trained of experienced employees assigned to a task. Specifically, in its transition to a different Reliability Standard, the entity did not ensure that the persons responsible for identifying and implementing security controls for PCAs had adequate training and/or experience to appropriately protect them.

Finding: WECC found the violation constituted a moderate risk and did not pose a serious or substantial risk to bulk power system reliability. Although the entity failed to have methods to enforce authentication of interactive user access, change known default passwords, and enforce password parameters, the entity had compensating controls in place that lessened the risk and no harm is known to have occurred. The violation began on July 1, 2016 and ended on February 13, 2018. WECC considered the entity’s internal compliance program to be a mitigating factor and found that its compliance history was an aggravating factor in determining the penalty disposition. To mitigate the violation, the entity adjusted the operability of the applicable PCAs to allow for password functionality, enable the functionality on the three PCAs to implement authentication of user access, change the default password, and met with group responsible for the PCAs to review and discuss procedures.

Penalty: $87,000

FERC Order: June 27, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-6-000 (March 28, 2019)

NERC Violation ID: WECC2017017207

Reliability Standard: CIP-007-6

Requirement: R1; P1.1

Violation Risk Factor: Medium

Violation Severity Level: High

Region: Western Electricity Coordinating Council (WECC)

Issue: When an unidentified entity was preparing its baseline on a workstation classified as a Bulk Electric System (BES) Cyber Asset (BCA) associated with its Medium Impact BES Cyber System (MIBCS), it evaluated all ports, and those that were considered unneeded were slated for removal. During a Compliance Audit, the audit team was provided an unidentified item that was not reflected in the device’s baseline. Upon further review, it was determined that the baseline was correct and that the unnecessary ports had been overlooked during the removal process. The BCA is an engineering workstation in the primary Control Center’s separate, but associated data center, and is not actively used to monitor or control the supervisory control and data acquisition (SCADA) network. WECC concluded that there was a failure to ensure that only those logical network accessible ports that were determined to be needed on a BCA within the MIBCS were enabled. The root causes of the violation were an oversight by the employee responsible for disabling the ports who did not follow the documented procedure for disabling unneeded ports that were not part of the baseline configuration and the lack of an internal control to ensure employees followed the procedures.

Finding: WECC found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system reliability. Although the failure to enable only logical network accessible ports were determined to be needed could result in a malicious actor gaining access to the BCA to cause harm to the SCADA system, the entity had implemented access control at the Electronic Security Perimeter to only allow approved traffic into the protected network. Based on these controls, WECC determined that the likelihood of the potential harm occurring was low. The violation began on July 1, 2016 when the reliability standard became mandatory and enforceable and ended on February 28, 2017 when the ports that were not needed were disabled. WECC noted that had it not been for the Compliance Audit, the violation duration would have been longer due to the lack of defective controls. Based on this, WECC applied an aggravating factor and escalated the disposition treatment to an expedited settlement. WECC considered the entity’s internal compliance program to be a neutral factor and found that there were no relevant instances of noncompliance after it reviewed the entity’s compliance history. To mitigate the violation, the entity disabled logical network ports, updated documentation, documented a process to periodically review baseline configurations against a report of open ports to ensure only necessary logical ports are open and that the baselines are accurate, trained personnel, and added the reliability standard as a regular agenda item for the monthly CIP Compliance meetings.

Penalty: $0

FERC Order: March 28, 2019 (no further review)

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP19-6-000 (March 28, 2019)

NERC Violation ID: WECC2017016991

Reliability Standard: CIP-007-6

Requirement: R2; P2.1, 2.2, 2.3

Violation Risk Factor: Medium

Violation Severity Level: High

Region: Western Electricity Coordinating Council (WECC)

Issue: An unidentified entity submitted a Self-Report stating that for three Cyber Assets classified as Bulk Electric System Cyber Assets (BCAs), it did not assess security patches after the initial review of security patches on July 1, 2016 was conducted. The devices and software support the primary and backup Control Centers containing a Medium Impact Bulk Electric Cyber System (MIBCS). WECC determined a scope increase from the original Self-Report and identified three additional devices classified as Protected Cyber Assets (PCA) where the entity failed to maintain documentation that it had performed a patch evaluation at least once every thirty-five days. Furthermore, the entity did not document a patch source for one Electronic Access or Monitoring System and seven Physical Access Control Systems, and although the entity created a mitigation plan for security patches assessed and not applied, it did not include specific implantation timeframes. The root cause of the violation was inadequate security patch management program for CIP compliance. The lack of knowledge and understanding of CIP Standards resulted in the implementation of inadequate security patch management program.

Finding: WECC found the violation constituted a minimal risk and did not pose a serious or substantial risk to bulk power system reliability. Because the entity failed to evaluate security patches within thirty-five calendar days of the last evaluation, to document a patch source for applicable assets, to maintain documentation that it performed patch evaluations once every thirty-five calendar days for the MIBCS and associated PCAs, EACMS and PACs, the entity potentially exposed itself to a malicious actor using known attack methods to gain access of a BES Cyber System. However, the entity reduced the likelihood of the risk by preventive controls such as permitting only allowed traffic into and out of the ESP as well as implementing Intrusion Detection System devices to each network to detect malicious code. Thus, WECC determined that the likelihood of the potential harm occurring was low. The violation began on July 1, 2016 when the reliability standard became mandatory and enforceable and ended on February 23, 2017 for Part 2.1 when the entity included patching sources in its patch management process and September 21, 2017 for Parts 2.2 and 2.3 when the entity evaluated security patches and updated its mitigation plan. WECC noted that the entity did not have defective controls in place that could have helped identify the issuer sooner and to lessen the violation duration. Had it not been for the entity’s Compliance audit, the violation duration would have been longer due to the lack of defective controls. Based on this, WECC applied an aggravating factor and escalated the disposition treatment to an expedited settlement. WECC considered the entity’s internal compliance program to be a neutral factor and found that there were no relevant instances of noncompliance after it reviewed the entity’s compliance history. To mitigate the violation, the entity, among other things, updated the patch tracking workbook to include and maintain a list of all applicable devices and software, installed applicable patches or mitigation plans, and reviewed supporting documents to determine if additional updates were needed.

Penalty: $0

FERC Order: March 28, 2019 (no further review)

Unidentified Registered Entity 1 (FRCC_URE1), FERC Docket No. NP19-5-000 (February 28, 2019)

NERC Violation ID: FRCC2018019002

Reliability Standard: CIP-007-6

Requirement: R2; P2.2

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Florida Reliability Coordinating Council, Inc. (FRCC)

Issue: During a Spot Check conducted from January 15, 2018 through January 19, 2018, FRCC determined that an unidentified entity was not in compliance with the Reliability Standard. The entity failed to evaluate security patches for four Energy Management System (EMS) servers, five (5) operator workstations within the EMS network, one Physical Access Control Systems server, and two Programmable Local Access Control Panels. Although every patch was not critical, there were critical patches that missed the thirty-five day installation window. These missed patches could have prolonged the presence at software vulnerabilities, which, if exploited, could grant access to unauthorized personnel or misuse of Cyber Assets. The root causes of the violation were multiple vendors responsible for patching on different segments (Supervisory Control and Data Acquisition (SCADA) and non-SCADA) of the Cyber Assets and a lack of oversight.

Finding: FRCC found the violation constituted a moderate risk and did not pose a serious or substantial risk to bulk power system reliability (BPS). Because the entity failed to execute its patch management process, it could have prolonged the presence of software vulnerabilities, which if exploited, could grant access to unauthorized personnel or misuse of Cyber Assets, impacting the reliability of the BPS. However, the risk was reduced because all of the devices were protected by a Physical Security Perimeter and all Cyber Assets were within the Electronic Security Perimeter. Furthermore, while the patches did not meet the thirty-five day requirement, they were being installed on a quarterly basis. The entity did perform a vulnerability review and determined that there were no know instances of unauthorized access or breaches to the entity’s Cyber Systems. Furthermore, the Cyber Assets were being monitored by three external vendors. The violation began on March 23, 2017 when the entity failed to evaluate its security patches for applicability at least once every thirty-five calendar days on twelve out of twenty-nine cyber assets and ended on March 5, 2018 when patches were evaluated and completed. FRCC considered the entity’s internal compliance program and positive cooperation as mitigating factors when determining the penalty. FRCC reviewed the entity’s compliance history and determined there was a relevant instance of noncompliance, which it considered to be aggravating. FRCC resolved the noncompliance in an SNOP as aggravation for the previous noncompliance. To mitigate the violation, the entity, among other things, evaluated and applied all security patches, designated a single vendor to monitor for all newly released security patches, developed situational awareness internal control, and provided training.

Penalty: $0

FERC Order: February 28, 2019 (no further review)

Unidentified Registered Entity 1 (FRCC_URE1), FERC Docket No. NP19-5-000 (February 28, 2019)

NERC Violation ID: FRCC2018019016

Reliability Standard: CIP-007-6

Requirement: R5; P5.6; 5.7

Violation Risk Factor: Medium

Violation Severity Level: Severe

Region: Florida Reliability Coordinating Council, Inc. (FRCC)

Issue: During a Spot Check conducted from January 15, 2018 through January 19, 2018, FRCC determined that an unidentified entity was not in compliance with the Reliability Standard. For Part 5.6, the entity failed to enforce password changes or failed to change the password at least once every 15 calendar months for all eight shared accounts. For Part 5.7, the entity failed to implement controls to limit the number of unsuccessful authentication attempts to generate alerts after a threshold of unsuccessful authentication attempts or failed to generate alerts after a threshold of unsuccessful authentication attempts on the three firewalls and four switches. The root cause of violation was an absence of internal controls related to password changes on shared accounts.

Finding: FRCC found the violation constituted a moderate risk and did not pose a serious or substantial risk to bulk power system reliability (BPS). Because the entity failed to change the passwords by the required timeframe, it could have exposed the passwords to malicious individuals, thus allowing unauthorized access to Cyber Assets. This risk increased because some of the Cyber Assets at issue were designed to provide perimeter protection to other Cyber Assets. Additionally, the entity’s failure to configure an account lockout policy or alert a certain number of failed authentication attempts could have caused reliability concerns for the entity. However, from July 1, 2016 to July 1, 2018, there was no known unauthorized access or breaches to any of the entity’s Cyber Assets and no harm is known to have occurred. The violation began on July 1, 2016 when the entity failed to enforce password changes and limit unsuccessful attempts or generate alerts and ended on January 24, 2018 when the entity updated their processes to require the changing of the passwords and limited unsuccessful authentication attempts or generate alerts after a threshold of unsuccessful authentication attempts as well as established required alerting. FRCC considered the entity’s internal compliance program and positive cooperation as mitigating factors when determining the penalty. FRCC reviewed the entity’s compliance history and determined there were no relevant instances of noncompliance. To mitigate the violation for Part 5.6, the entity, among other things, scheduled the process of changing the passwords for shared accounts to take place each year during the first quarter to ensure they are changed within the required timeframe, reviewed all shared accounts to ensure that all accounts are justified and still needed, and changed all shared account passwords. To mitigate the violation for Part 5.7, the entity, among other things, updated Security Information and Event Management to analyze the logs from the firewalls and switches, tested and verified loges for all applicable Cyber Assets, and provided training.

Penalty: $0

FERC Order: February 28, 2019 (no further review)

Registered Entity (Name Redacted), FERC Docket No. NP20-2-000

Region: Western Electricity Coordinating Council (WECC)

Issue: The Entity violated the CIP Reliability Standards when it failed to develop baseline configurations for its Bulk Electric System Cyber Assets and to test changes to those configurations before deploying them in the production environment. The Entity's CIP compliance program lacked strong internal controls related to configuration management. The lack of sufficient internal controls around configuration management and change management resulted in less than adequate procedures, insufficient oversight, and a lack of written documentation and communication about how to maintain security in accordance with the NERC Reliability Standards.

Finding:

CIP-007-6 R5 - WECC determined that the entity failed to submit Technical Feasibility Exception (TFE) requests for cyber assets that weren't capable of limiting the number of unsuccessful authentication attempts or generating alerts after a threshold of unsuccessful authentication attempts. The cyber assets in scope included BES Cyber Assets, Protected Cyber Assets and Physical Access Control Systems associated with its High Impact BES Cyber Systems (HIBCS) located at its primary and backup control centers. The root cause of this violation was that the Entity lacked a documented process to perform routine oversight of the TFEs that had been or needed to be filed with WECC. WECC determined that the violation posed a minimal risk to the reliability of the bulk power system (BPS).

CIP-010-2 R1 - WECC found two violations of CIP-010-2 R1. As to the first violation, WECC determined that the entity failed to (a) develop a baseline configuration that included applied security patches on Electronic Access Points and Electronic Access Control and Monitoring Systems (EACMS); (2) update the baseline configuration within 30 calendar days of completing a change that deviated from the existing baseline configuration on PCAs; and (3) test changes in a test or production environment that models the baseline configuration to ensure that the required cyber controls of CIP-005-5 R1 were not adversely affected. Additionally, the Entity failed to document testing results, and if a test environment was used, the differences between the test and the production environment. The root cause of this violation was that the Entity lacked, or had less than adequate, change control process documentation. WECC determined that this violation posed a serious and substantial risk to the reliability of the BPS.

As to the second violation, WECC determined that the Entity failed to develop baseline configurations for HIBCS BCAs and EACMS in the Demilitarized Zone associated with the HIBCS, which included the operating system or firmware where no independent operation system existed, as required by CIP-010-2 R1 Part 1.1 sub-part 1.1.1. The root cause of the violation was that the Entity did not have proper procedures for personnel to follow, which resulted in employees making incorrect decisions regarding documenting baselines for the affected cyber assets. WECC determined that this violation posed a moderate risk to the reliability of the BPS.

CIP-010-2 R2 – WECC found two separate violations of CIP-010-2 R2.

In the first case, WECC determined that the Entity failed twice to monitor changes to the baseline configurations at least once every 35 calendar days, and document and investigate detected unauthorized changes, as required by CIP-010-2 R2 Part 2.1. In the first instance, the Entity missed one Polycom system classified as a PCA associated with the HIBCS during manual monitoring, 22 days past the requirement of once every 35 calendar days. In the second instance, the Entity discovered virtual hosts associated with HIBCS that had not been monitored for baseline configuration changes 301 days past the requirement of once every 35 calendar days. The root cause in the first instance was software failure and in the second was that the entity lacked written communication. WECC determined that this violation posed a moderate and substantial risk to the reliability of the BPS.

In the second case, WECC determined that the Entity had not been monitoring "active" BCAs for changes to logically accessible ports and services, due to an omission of the Cyber Assets from the manually populated vulnerability scanner asset list. The root cause of the violation was that the Entity lacked written documentation or communication for responsible personnel to know that BCAs required nightly scans. WECC determined that the violation posed a moderate risk to the reliability of the BPS.

The following were considered in WECC's assessment of penalty: (a) the Entity's compliance history with CIP-007-6 R5 as an aggravating factor; (b) the Entity's failure to promptly provide information upon repeated requests which WECC's impeded the review/resolution of violations; (c) the Entity agreed to settle the violations and the monetary penalty; (d) the Entity admitted to and accepted responsibility of the violations; (e) one of the violations was self-reported timely after discovery (WECC2017018485); and (f) the Entity was cooperative during the WECC audit which resulted in a timely resolution of one of the violations (WECC2018019376).

Penalty: $378,000

FERC Order: Issued November 29, 2019 (no further review)

Registered Entity (Name Redacted), FERC Docket No. NP20-15-000

Please search for this docket no. here ››

Registered Entity (Name Redacted), FERC Docket No. NP20-21-000

Please search for this docket no. here ››

Unidentified Registered Entity 2 (SERC_URE2) and Unidentified Registered Entity 3 (SERC_URE3), FERC Docket No. NP18-25-000

Please search for this docket no. here ››

Unidentified Registered Entity 1 (WECC_URE1), FERC Docket No. NP18-21-000

Region: Western Electricity Coordinating Council (WECC)

Issue: WECC_URE1 reported the violations of the standards above as per follows:

(a) CIP-007-6 – a violation of (CIP-007-6) R2; P2.1, 2.2, 2.3 and 2.4 had occurred as WECC_URE1 reported that it was instructed by the designer, manufacturer, and warranty provider of the high voltage direct current protection and control system (HVDC PCS) at a substation, that its patches and updates for software and firmware provided during the warranty period would be functional patches rather than security patches and only for the designer's proprietary devices. Due to a setting in a single device being incorrect, the substation Station Control and Monitoring network suffered a failure during the trial operation phase. The designer's position was not to apply security patches to the system devices because those security patches could pose a risk to system reliability without providing a tangible security benefit. As a term of purchase, the designer had approving authority for implementation of patches for third-party manufactured devices that were utilized by the designer in the system design. The HVDC PCS at the substation was an engineered high speed, real time, closed loop, protection and control system. This engineered system consists of multiple devices running multiple software packages that are closely integrated by the manufacturer, to meet the protection and control requirements. The manufacturer and designer sold the engineered system to WECC_URE1 and retained knowledge of how the software and hardware integrations work. As required by CIP-007-6 R2 Part 2.1, WECC_URE1 had identified the manufacturer as the patch source for the HVDC PCS at the substation.

However, since the manufacturer didn't have a website with patching information for the HVDC PCS, it was required to notify WECC_URE1 when new security patches were available.  The evaluation of security patches for the BES Cyber Assets (BCAs) and Protected Cyber Assets (PCAs) at the substation and the creation of mitigation plan(s), had not taken place since the requirement became effective on July 1, 2016 as required by CIP-007-6 R2 Parts 2.1, 2.2, 2.3, and 2.4. Additionally, WECC_URE1 was performing its CIP-007-6 R2 security patch management program for the other CIP devices by different manufacturers within this substation.

(b) CIP-005-1 – a violation of CIP-005-1 (R2; R2.4) occurred when WECC_URE1 discovered one Cyber Asset at a substation with External Routable Connectivity (ERC) containing Medium Impact BES Cyber Systems (MIBCS) that allowed access to a relay without any authentication procedure.

(c) CIP-007-3a – a violation of CIP-007-3a (R3.1) occurred when WECC_URE1 was unable to confirm that it documented the assessment of security patches and security upgrades for applicability within 30 calendars days of their availability for 7 Critical Cyber Assets (CCAs).  During an internal non-compliance investigation, WECC_URE1 determined that after experiencing network connection issues, one of its technicians saved evidence of security patch assessment to his computer's hard drive, which was subsequently wiped clean without saving a copy when the technician's employment ended; resulting in a gap in security patch management documentation.  During scheduled maintenance, WECC_URE1 performed a full investigation which identified 58 additional RAS system devices, and 6 RAS servers.  WECC_URE1 failed to document the assessment of security patches and security upgrades for applicability within 30 calendar days of their availability for 71 CCAs.

(d) CIP-006-6 – a violation of CIP-006-6 (R2; P2.2) occurred when WECC_URE1 occurred when WECC_URE1, during an internal spot check of visitor logs, discovered that an entry made on a specific date did not include a last exit time for one visitor as required by CIP-006-6 R2 Part 2.2.

Finding:

(a) CIP-007-6 – WECC determined that WECC_URE1 failed to implement a program or process for tracking (2.1), evaluating (2.2), and installing cyber security patches or creating a mitigation plan to address the vulnerabilities addressed by each security patch that was not installed (2.3) and implementing the mitigation plan (2.4) for its designer devices at its substation, designated as a MIBCS, consisting of BCAs and PCAs. The root cause of this violation was WECC_URE1's not integrating the substation HVDC PCS into the field patch management process. WECC_URE1 underestimated the resources and effort needed to operate a compliant security patch management program at its substation. WECC determined that this noncompliance posed a moderate risk to the Bulk Power System's (BPS's) reliability.  In this instance, WECC_URE1 failed to implement or document a security patch management program for its manufacturer's devices within its MIBCS consisting of BCAs and PCAs, located at the substation, as required by CIP-007-6 R2. Such failure could potentially result in unauthorized electronic access to the vulnerable systems within WECC_URE1's substations. Unauthorized electronic access, due to unpatched software, could result in complete control of the unpatched devices due to malware infection or other successful intrusion into the network locations of the unpatched systems. The result could be complete control of the affected system and an anchor point for reconnaissance and spreading through the environment, which could result in significant negative effects to the BPS, including neighboring entities via interties.  The substation monitors and controls a WECC major path. However, the substation has no ERC to the substation; therefore, any compromise would have to be performed at the physical site itself. WECC_URE1 implemented weak controls to detect this violation.

(b) CIP-005-1 – WECC determined that WECC_URE1 failed to implement strong procedural or technical access controls at the access point of an Electronic Security Perimeter (ESP) to ensure authenticity of the accessing party for one device that allowed access to a relay. The root cause of this violation was that the accuracy or effectiveness of changes were not verified or validated.  This violation posed a minimal risk to the BPS's reliability.  In this instance, WECC_URE1 failed to ensure authentication at the device access point for all user access into WECC_URE1's substation containing MIBCS as required.  Such failure could result in inappropriate access via a compromised or misused remote access account resulting in significant negative effects on the reliability and stability of the Bulk Electric System, including but not limited to disruption, manipulation, or compromise of the MIBCS for reconnaissance. WECC_URE1 implemented weak detective controls because the issue was identified as a result of an audit 8 years after the device should have been compliant with the CIP-005-1 R2.4.

(c) CIP-007-3a - The root cause of the first device issue was the improper identification and classification of a device in the PACS systems.  The root cause of the network equipment issue was the poor transition of duties between personnel. The root cause of the RAS system issue was the lack of written communication.  This violation posed a moderate risk to the BPS's reliability.  A failure to document the assessment of security patches within 30 days may lead to security exposures that could result in serious adverse effects on WECC_URE1's operations, including the installation of potentially harmful software on the CCAs.  Installation of malware or other harmful software could lead to unintended access and eventually to control of systems and software. Given that this violation involved several different locations containing the CCAs, a widespread coordinated attack would be conceivable. As such, the failure to adequately protect devices and resources could affect the reliability of the BPS.

(d) CIP-006-6 - The root cause of this violation was due to a mental lapse that lead to an incorrect performance. Specifically, the visitor escort was trained appropriately but did not follow the process. This violation posed a minimal risk to the BPS's reliability.  Such failure could lead to the visitor being allowed back in without knowledge; without justifiable cause; and potentially without a visitor escort. In addition, ensuring that visitors are logged out provides closure of the visitor management process and clarification that the individual is still not somewhere in the facility, which during an emergency could be an issue for security personnel attempting to locate the individual for safety concerns. 

As part of its penalty assessment, WECC considered WECC_URE1's internal compliance program (ICP) and compliance history as factors in the penalty determination. Additionally, SERC noted several specific circumstances in each violation that limited its impact, scope or duration.

Penalty: No penalty

FERC Order: No further review (August 30, 2018)

Registered Entity (Name Redacted), FERC Docket No. NP19-9-000

Please search for this docket no. here ››

Registered Entity (Name Redacted), FERC Docket No. NP19-10-000

Please search for this docket no. here ››

Unidentified Registered Entity 1 (RFC_URE1), Unidentified Registered Entity 2 (RFC_URE2), Unidentified Registered Entity 3 (RFC_URE3), and Unidentified Registered Entity 4 (RFC_URE4), FERC Docket No. NP18-16-000

Please search for this docket no. here ››

NP19-4-000: Unidentified Registered Entity

Please search for this docket no. here ››

NP18-14-000: Unidentified Registered Entity

Please search for this docket no. here ››

NP20-12-000: Unidentified Registered Entity

Please search for this docket no. here ››

NP19-11-000: Unidentified Registered Entity

Please search for this docket no. here ››

NP18-6-000: Unidentified Registered Entity

Please search for this docket no. here ››

NP18-4-000: Unidentified Registered Entity (URE)

Please search for this docket no. here ››

NP20-6-000: Unidentified Registered Entity 3 (URE-3)

Please search for this docket no. here ››

NP20-20-000: Unidentified Registered Entity

Please search for this docket no. here ››