Here is the simplified situation:
Upon redundant controller failure, Modbus communication is tied up and can not be established with the new primary controller, until the strategy is restarted. When strategy restarts, set points revert to 0.00.
In depth situation:
We have around 10 environmental chambers on a redundant controller.
All the chambers are controlled by WATLOW F4’s. The F4’s communicate via Modbus. I have a SCM 232 module in position zero on a rack of one of the 3 Brains. Because we have one SCM module, only one F4 can have communication open at any time. The control chart cycles through, waits for the individual chamber charts to finish before moving to the next chamber. This all works well until…
We have a redundant controllers. When we simulate a controller failure, we can not regain communication with the F4 (probably due to the communication not getting closed properly?) until we restart the strategy. Upon strategy restart, all set points are lost and revert to 0.00. Not exactly optimum :[
I have redundant variables and sync blocks on the main and individual chamber charts and have a clear understanding of transactional carts. But toss Modbus in there (new to modbus) and all of a sudden 40 hours of my time has vanished along with my set points :(.
Help me Obi-Wan Opto-Awesomes, your my only hope…
Here is a list of hardware and software versions I am working with from memory:
PacProject Pro 9.6 (Control, Display and redundant manager)
Redundant Controllers (S2’s running the most recent firmware as of 03/21/19)
Arbiter (most recent Firmware as of 03/21/19)
Modbus Integration Kit
SCM Module position zero (version and brains i dont remember exactly)
Comunication Handle - ModbusMasterTCP with RTU protocol
Also have 2 OPC servers (main scanner and back up) but will integrated them after the communication is perfect unless I am advised otherwise.
Willing to bundle up the Strategy and Display files if requested.
Do you know what value is in the nStatus parameter of the modbus functions when communication fails?
I had to modify my modbus Open Communcation blocks to look like this:
if (GetNumCharsWaiting(chCommHandle) < 0 or not IsCommunicationOpen(chCommHandle)) then
nStatus = OpenOutgoingCommunication(chCommHandle);
else
ClearCommunicationReceiveBuffer(chCommHandle);
nStatus = 0;
endif
I don’t remember the specific problem I was having, but that method was more robust at setting things straight.
-52 is telling you the comm handle isn’t open, but the modbus subs attempt to open it when that happens (at least when first entered). Maybe step through the chart and see exactly where it is kicking out will give some clues.
So your redundant/persistent variables are being set to zero on strategy restart? Not sure on that - only time I’ve seen that is when doing a restart from power up and the internal battery was dead.
The redundancy kit is outside my wheel house, so not sure if I can help much more on that aspect. I’m wondering if the comm handle is set to be redundant/persistent though and whether it needs to be or not.
Thank you, the com not being open is a good start.
the set points are probably being set to zero upon strategy restart due to clearing all buffers on the power up chart.
One of the issues I am having is the persistent variables and sync blocks (Transactional carts) on the communications page. In order to interrogate on an action taking place, you have to send the transactional cart to initiate the actions and then interrogate. This complicates the communications and results in sync blocks being placed after the communication setup parameters. Upon chart continue after a controller failure, it attempts to start at the closest block previous to the failure. If that block falls after the com parameters then it is trying to use a com handle that has not been set up on the current controller. sigh
Upon reading the “best practices” “building strategies” (AWESOME STUFF THANK YOU) it says:
“Some Experienced PAC Control programmers recommend placing a ‘default values’ block a the beginning of the Powerup chart to reset values for important variables in case of a power cycle. Instead of making each variable persistent, you can write their values periodically to a persistent table. The default values block checks to see if a value is zero; if it is, it’s replaced by the value from the table”
Then I can remove the sync blocks from the communications page, and when / if there is a controller failure the entire chart will run from the beginning instead of trying to start somewhere in the middle.
Thank you so much for your help, I will post a result when I finish the needed changes.
If there is anything else I am missing or a short cut I am all ears! I can not be the only person who has this problem…and i am interested in how others have resolved similar issues.
