1 of 21

From Power Lines �to Packets

Network Troubleshooting in the Power Grid

Daniel Lopez, WCNA, �Lead Integration and Automation Engineer�daniel_lopez@selinc.com

#sf25us

2 of 21

Agenda

  • What is SEL?
  • Power grid components overview
  • Devices communicating in the grid
  • What these networks look like
  • Power systems protocols
  • Real customer issue

#sf25us

3 of 21

Schweitzer Engineering Laboratories

  • Dr. Edmund O. Schweitzer (1982)
  • First microprocessor-based relay

#sf25us

4 of 21

Schweitzer Engineering Laboratories

  • Dr. Edmund O. Schweitzer (1982)
  • First microprocessor-based relay

#sf25us

5 of 21

Schweitzer Engineering Laboratories

  • Dr. Edmund O. Schweitzer (1982)
  • First microprocessor-based relay
  • Mission – making electric power safer, more reliable, and more economical

#sf25us

6 of 21

Schweitzer Engineering Laboratories

  • Dr. Edmund O. Schweitzer (1982)
  • First microprocessor-based relay
  • Mission – making electric power safer, more reliable, and more economical
  • R&D – protection systems forensics

#sf25us

7 of 21

Schweitzer Engineering Laboratories

  • Dr. Edmund O. Schweitzer (1982)
  • First microprocessor-based relay
  • Mission – making electric power safer, more reliable, and more economical
  • R&D – protection systems forensics

#sf25us

8 of 21

Power grid overview

Residential customer

Distribution

Control room

Transmission

Step-up �transmission substation

Step-down �distribution substation

Generation

#sf25us

9 of 21

Single site example

Emergency generator 1

Emergency generator 2

Emergency power source

Normal �power source

Utility transformer

Life safety

Critical

Equipment

ATS

ATS

ATS

#sf25us

10 of 21

Protective relays

  • Traveling wave fault location
  • Communications
  • Protection
  • Broken conductor detection

#sf25us

11 of 21

Why communicate?

  • Local and area applications
    • Differential Protection
  • Interoperability
  • Control and monitoring
    • SCADA
    • HMI

#sf25us

12 of 21

Network example

Typical substation

  • Merging Units
  • Relays
  • Network switches
  • Data concentrators
  • HMI/SCADA

#sf25us

13 of 21

Power system protocols

Modbus

​

#sf25us

14 of 21

Power system protocols

Synchrophasors (C37.118)

#sf25us

15 of 21

Power system protocols

IEC 61850 GOOSE�(Generic Object Oriented Substation Event)

#sf25us

16 of 21

Customer issue – background

  • Issue in an oil field
    • Occurrence is frequent
    • Originated months ago
  • Impact
    • Backup generation offline
    • Manual intervention required

Source 1

Source 2

Backup generators

Oil field operations

#sf25us

17 of 21

Customer issue – details

  • Monitoring quality of �IEC 61850 GOOSE message
  • Sender and receiver are �on the same switch
  • Other communications �are not affected

Network switch

Backup generators

IED

IED

IED

PLC

#sf25us

18 of 21

Open Wireshark

#sf25us

19 of 21

Root cause

  • Duplex mismatch
  • Packet loss

Full duplex

Half duplex

Common auto-negotiation failure scenario

#sf25us

20 of 21

Root cause

  • Duplex mismatch
  • Packet loss
  • Reconfigured switch settings

Source 1

Source 2

Backup generators

Generation is back online!

Oil field operations

#sf25us

21 of 21

Feedback

#sf25us