About 2 weeks ago we successfully began transmitting our RS232 data from a Rice Lake scale to the CSERI-4 module of our EPIC system. After much troubleshooting, this became possible only when we downgraded the firmware to v.1.6f.
Suddenly yesterday, our PAC Control program stopped receiving the scale data. Looking at the program in the Autostep mode shows it is not able to open the communication handle (gives error 87, which is “open error”.
In the above loop, it always fails and then just keeps retrying every 2 seconds.
In all the years we have run this program on 5 or 6 different EPIC units, we have never had this type of error. The CSERI-4 module is only 2 weeks old. Just for kicks, I installed firmware 1.6g, and then went back to 1.6f just to see if that made a difference (it did not).
Any other troubleshooting steps I can take to try to resolve this?
Also, if anyone has a scale with serial communication that works well with the EPIC, please share the make & model. These Rice Lake units are nice, but I think the RS232 comm is not the greatest (I am planning to migrate to RS485 Modbus eventually, but maybe there is a better way).
In case it helps, this is what my Communication Handle looks like (all settings shown match the Rice Lake).
Hi Grant, If I remeber correctly one of the things that changed in the firmware was the termination on the module. If the scale does not have a proper and robust RS232 driver it could be more of a hardware issue than a PAC Control issue. Do you have any other RS232 devices you could test with? Or perhaps swap the module out to see if by chance it is a module issue. When I integrate scales I look for ones with a 4-20ma option as it is much easier and reliable to implement.
Hi Grant,
I have a customer slowly replacing their Rice Lake scales with Cardinal Model 201’s. I have others that are switching the other way. One thing about the Cardinal is it has an Ethernet Modbus TCP feature, which can be very nice. It also has the serial port, which would be less change for you.
I agree with Norm that your issue could be a node in the system other than the CSERI module, EPIC, or Control Engine, but some digging may be required for a complete diagnosis.
I looked diligently for relevant documentation to find how many serial comm handles can be opened. The PAC Project Pro User Guide documents the limits of number of TCP comm handles but not serial comm handles. Obviously the resource limits of TCP handles are more well known, and I’d hypothesize the ceiling for serial comm handles is very high so not documented. You could try to show the affirmative or show the contrary by rebooting the EPIC when it’s safe to do so, run your strategy, and see whether comms come back. Of course, this is not a definitive test.
In general resources are managed by the kernel. An ideal kernel reclaims resources after a program exits, but in reality even kernels have limitations. Stopping a strategy or restarting the Control Engine isn’t going to free up the TCP resource pool in the kernel because that takes time due to hardware limitations. As for serial resources, I don’t know what happens under the hood, but I wouldn’t be surprised to learn that there could be clean up work left for the kernel.
You posted a screenshot at the top of this thread. I noticed that the flow opens comm handles in a loop. If a failed attempt at opening a comm handle slows down the resource freeing more than the delay/time to the next attempt, then the kernel can’t keep up. In my experience with TCP on linux common practice is to exponentially back off attempts to allocate the resource. Unfortunately > 2s is a looong time in automation. Another possible explanation is that there are unclosed comm handles == a hypothetical limit for open serial handles, but is this even an issue for serial on opto linux?
Of course, I’m hypothesizing here. I hope that when you do reach a definitive solution that you post back so the eyes on the issue can be reallocated or, perhaps, increased.
Have you tried any modbus slave software on a laptop with a RS-232 adapter connected directly to the module? It’s the simplest way to verify if its the module or the scale.
Yep, I pretty much arrived at the same conclusion. Unfortunately I am at another location today so will not be able to try it until Monday. Will post back when I have an answer
PS: RS232 does not use Modbus, but I get your point!
I think my CSERI-4 module might be defective, as I moved it to an identical working EPIC + scale system (connected via RS-232) and it showed the same behavior, i.e. would not open the communication handle. I will email Opto22 support team to follow up.
