I have a strategy with several RIOs. Just recently all the IO from the RIO was frozen in the HMI. In Debug, when I click on one of the RIO IO units it has an error:
I/O Unit not enabled. Previous communication failure may have disabled the unit automatically.
I have an IO_Enabler chart on this strategy and the IsIOUnitCommEnabled function is returning true even with that error. Is this supposed to work like this? How do I re-enable a RIO through the strategy after a communications failure?
It sounds like this:
https://www.opto22.com/support/resources-tools/knowledgebase/kb89056
But cant be… you would be running better than 2.0?
Will dig into it a bit more tomorrow.
These (RIOs) are 2.0.2-b.199, so must be something different.
The PR1 that the strategy is running in is using 3.0.0-b.62
Have you heard anything on this? We are still seeing this issue.
Do your remote RIO’s use host names in the PAC Control strategy?
https://www.opto22.com/support/resources-tools/knowledgebase/kb89421
No, they are using IP addresses. I need to setup something on my test bench to reproduce and troubleshoot this, just haven’t had the time yet.
I have been unable to reproduce this on my bench where the “I/O Unit not enabled” message appears and Enable comm is yes. I’m at a loss to how this is happening, any insight or help is appreciated.
No problem. I always appreciate your help Philip.
I put a RIO on an EPIC with PAC Control 10.3x (the release version - for once) and am kicking at it.
So far, very quick test before I got pulled away late this afternoon, it was working as expected, but I am not going to move on just yet.
Anyway, said all that to ask, is that the same sort of set up you are using? RIO as IO from EPIC?
Yes, the strategy is running on a PR1 with RIO and EB1s for IO units. The RIOs and EB1s are connected through various point to point wireless networks and occasionally lose connection (I assume). The IO enabler chart starts the EB1s back up, but not the RIOs.
Do you know if there is a way to check in the strategy whatever PAC Control is checking to generate the “I/O Unit is not enabled.” error message? If I could check that, then I could have the strategy disable and re-enable communications to automatically bring it back on line as a work around.
The issue with the IO enabler chart is it only looks at IsIOUnitCommEnabled which is returning true in this case (even though it probably shouldn’t).
Philip, were you able to replicate the issue? very curious about.
I have not been able to reproduce it after a few hours testing and to the best of my knowledge @philip has not been able to reproduce it on the work bench but it’s still happening in the field.
No I haven’t been able to reproduce this in a controlled environment. It seems to happen after a period of time of having a bad connection between the control engine and the IO. In the strategy I disable and re-enable communication every five minutes now which I think will bring the IO back on line when it gets in this state and doesn’t seem to hurt it when all is working okay. I’m testing that out now.
Well, it sounds like your network do not like your RIO’s, perhaps gateway and dns misconfigs? I guess that was verified already. I will ping by chart and count timestamp comms drops but it is confusing comms seems to be open but is not.
Is there any MAC filtering in the network IT side?
This happens after a period of comm failures. When the communications come back, the strategy sees the IO unit as not being enabled and only looks at the IVAL, yet IsIOUnitCommEnabled returns true so there isn’t a way to detect when this is happening so it gets stuck that way.
Running:
DisableCommunicationToIoUnit(RIO);
EnableCommunicationToIoUnit(RIO);
periodically is what I am testing as a work around for now.
Not sure if my issue is related to your issue. But I have had some funny remote comms issues with my EPIC and remote brains.
Here in SA we have a lot of loadshedding and power drops. I have an EPIC connected to remote Ethernet-S64 brains. Power would drop and come back during the day and the customer would complain about the remote IO not working. When I dial in and check IO comms via PAC Control all IO is connected and communicating. I can ping the remote S64 brains without an issue. What I cannot do is control any of the IO on the S64’s until I clear the RAM from the EPIC and download the strategy again. Enabling and disabling comms does not help. I have tried the Enable/Disable commands without success.
It seems like the EPIC thinks the re-connection is successful after power failure but it actually isn’t. Perhaps some stale comms handles?
Have you tried stopping the epic strategy wait a couple of seconds and re run the strategy?
Yes, stopping and starting the strategy will reconnect to the IO units, but this needs fixed without manual intervention. So far the Disable and Enable occasionally seems to be an okay work around. I’ve been unable to find a root cause (likely some bug in the firmware). Unfortunately this is are far as I will likely take this as we have a work around and no one is going to want to pay for the time to work on it further and/or put customer equipment and product at risk. It would be one thing if I could reproduce this on my own network, but I have not been able to.
Hi, something similar happened to me, epic and a bunch of eb2’s in a common panel, another panel far away with a single eb2 lost power due some electric issue and the ups run out, when power came back comms were down, yet the startegy was constantly checking for open comms and will re enable if down…nothing happened… I did stop and run the strategy and that solved the problem and have never repeted again but power haven’t failed again either. I will try to kill power to it and see if happens again, stay tuned…
Stopping and stating the strat did not have any effect on the comms. I had to clear RAM and do a download for it to connect again