Ethernet Communications Loss - Plantwide Shutdown


I’m new to Opto22, so this might be a basic question, but…

Currently I have a plant network composed of multiple PR1s. The ‘Main’ PR1 is running a control engine sending commands to the other PR1s in normal operation.

Every device (PLCs etc.) in the plant is routed through an ethernet switch. My question is, how would you handle a plant shutdown if the switch lost power, so there was no communications between any of the PLCs? Is there a way to run a control engine on each individual controller (to handle Ethernet loss shutdowns), while simultaneously accepting the “Main” PR1’s control engine (which runs normal operations)?

Thanks for taking the time to read this post.

Each PR-1 has a “Scratchpad” memory area that can be read/written.
This is a handy way to pass information back and forth between controllers.
Are you using PAC Control as your control language?
If so check out the Users Guide under the Help menu for more information.

That is a pretty common challenge that a lot of controls folks face.
I had to figure out a way to do it with SNAP controllers 25 years or more ago. The hospital I worked at had around 35 controllers scatted around the facility and needed a few central ‘master’ controllers to keep everything working smoothy. Different parts of the network would go out from time to time. I had to set things up to ensure everything kept working during the outage.

I’m sure others will chime in here with some other suggestions… But a few thoughts from my experience.

  1. Use the dual network ports on the PR1.
    At the hospital the SNAP PAC controllers also had dual ports and so I had dual network switches plugged into different power sources and different IP addressing setup for each. You could do something similar with the PR1s pretty easily. (Assuming you can bear the one time burden and cost of running the network cables for the second network).
  2. I set things up such that the main central controller had only the lightest of touches to each of the other controllers. In other words, each controller ran a strategy that ensured that if they lost contact with the mother ship/HQ the process they were controlling would continue to function safely. The central controller only touched or controlled the bear minimum needed to coordinate.

I just want to add to @Beno comments which are great suggestions on how you should approach larger and/or more complicated systems.

Having (cost effective) intelligence at the IO level has been one of the features that makes the Opto22 system a powerful platform (bang for your buck) for process and plant wide applications.

  • Setup each IO rack with a logic to handle local IO
  • Utilize peer to peer communications (ie. scratchpad) to pass pertinent data between controllers.
  • Setup “loss of communication” protocols within the logic to ensure either safe shutdown or stand alone operation at the local level.

On larger systems utilize a central controller to pass data between the various controllers reduces network congestion and is easier to manage, however ad hoc peer to peer communication will also work (but may increase network congestion).

By distributing the processing power across multiple controllers / IO racks you effectively create a truly Distributed Control System that is more reliable and flexible than a Redundant Controller with Remote IO. In the even of a IO rack failure you lose the IO on both approaches but when isolated due to networking issues (or the loss of another controller) the local processes continue to work (which would not be the case with remote IO).

We have utilize this approach with great success on a wide array of plant wide control applications. Not only is the setup more robust, but this approach makes it easier to piecemeal sections of the plant into a comprehensive control system.