DOTNET with new firmware

I have a custom application using dotnet with both EPIC and RIO devices. After upgrading firmware and testing, dotnet gives warnings about needing a version number for libstdc++. The library is installed and I am assuming that the new firmware may have later versions of the library in the linux distribution. The application appears to run OK except it is tripping up on opening a TCP connection for listening on a port not otherwised used. It appeared to be working fine on the previous firmware.

Is anyone else seeing issues with new firmware and dotnet? I should mention that the optmmp library I have is working fine, all of the device I/O functions work. I am having difficult with the TCP connection which is used to cumminicate to an external system.

(I am using dotnet 3.1 LTS)

See the following messages.

$ ./dotnet --info
./dotnet: /usr/lib/libstdc++.so.6: no version information available (required by ./dotnet)
./dotnet: /usr/lib/libstdc++.so.6: no version information available (required by ./dotnet)
./dotnet: /usr/lib/libstdc++.so.6: no version information available (required by ./dotnet)
./dotnet: /usr/lib/libstdc++.so.6: no version information available (required by ./dotnet)
.NET Core SDK (reflecting any global.json):
Version: 3.1.421
Commit: 42eb11123f

Runtime Environment:
OS Name: Linux
OS Version:
OS Platform: Linux
RID: linux-arm
Base Path: /home/admin1/dotnet/sdk/3.1.421/

Host (useful for support):
Version: 3.1.27
Commit: 2d4bb962fd

.NET Core SDKs installed:
3.1.421 [/home/admin1/dotnet/sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.App 3.1.27 [/home/admin1/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.27 [/home/admin1/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
.NET Downloads (Linux, macOS, and Windows)

dotnet stuff is a bit tricky for us to test here as its not something we use very often.

I did a little bit digging with your extra info in this last post… Best as I can see from the engineering docs we did not change the version of the libstdc++.so.6 with this firmware update.

I also did some Google foo on the ‘no version…’ and it seems to be a notice of a version miss match and not a critical issue.
One page I read suggested that you rename /usr/lib/libstdc++.so.6 to be /usr/lib/libstdc++.so.6.old As I said, I cant test it here, but feel its worth a mention.

Regarding the TCP connection that used to work, but now does not, I am not sure. There were no changes to the firewall or networking in this update. Just the addition of the dataservices for the most part.

Sorry we cant help much more without getting acquainted with your dotnet app.

Thanks Beno, I realize support is limited for this.

It is difficult to determine if the issue I am having is a dotnet issue, problem in my code, or a firmware issue One of the things I am doing is setting up a simple TcpListener and a separate sender and see if it will talk. It acts like the ports are blocked even though I have a firewall rule setup and it worked before OK

Of course I will post here as I go.

Thanks

Sanity check (mine - not yours!)

You say the firewall rule is setup… Can you share a screenshot or just the exact rule?
What EPIC Eth port are you coming in on, 0 or 1?
UDP or TCP transmission?
Are you using a port above port 1024?

I think the connection issue is on my side. Still testing. I have the firewall set at a port number in the 50000 range for the non vpn port, TCP/UDP. Using TCP. I am pretty sure the “./dotnet: /usr/lib/libstdc++.so.6: no version information available (required by ./dotnet)” message is harmless as all else works. I am away from the remote location at the moment so will post back here when I confirm my hunch.

Beno, I am curious, have you tried opening the optommp interface using the loopback IP, 127.0.0.1?. I am teleworking today, so don’t have eyes on it here today but this idea popped into my mind of possibly setting up the ETH0 on an IP address needed externally and do the IO on the loopback, aka localhost.

I will also try to use the lun port and do a port redirect. Otherwise, still stuck in a conflict.

Update: Using the loopback interface (127.0.0.1) for the optommp server works great.

After some tedious debugging I learned some useful things. The legacy application we have ported to Epic/RIO uses raw TCP sockets. It originally ran on windows based industrial PCs.

The whole basis of using the RIO/EPIC is to migrate this off of windows, which in my humble opinion is unsuitable for this application. The EPIC/RIO allows us to simplify the hardware design by eliminating a windows based component, saving power and space.

What we discovered is that by default dotnet sets socket defaults to blocking I/O whereas Linux based systems use non-blocking. I wrote a small test program using the dotnet TCPListener class and it apparently does the right thing and it works as far as listening/sending TCP messages on the EPIC/RIO.

The lesson here is that in spite of what people may think, dotnet is not 100% portable across platforms. In this case a rather subtle bug where a background thread was happily waiting and doing nothing, blocked inside itself and hung itself. To give some credit where due. The dotnet world is working flawlessly for all other aspects of this application without changing thousands of lines of code.

I still have some work to convert this to a working implementation and will update further when I get there.

1 Like

Thanks a heap for reporting back.
Very interesting and I guess it makes a bit of sense. The Windows TCP stack is nothing like the Linux TCP stack.
That said, you would think dotnet would know (roughly) what stack its running on and make some internal adjustments, but it would seem not.

Very valuable debugging and again, I really appreciate you dropping back and letting everyone know. Going to save someone a whole bunch of time at some point in the future!

New information. Due to other priorities we got sidetracked on this issue. I was not able to resolve it on the immdeiate need so we inserted a mini pc to use with the RIO Less than ideal.

This week we are back at it and did an experiment. We took the exact code which runs natively on the RIO and copied it to a mini PC running linux. It runs like a charm there. So in the RIO, even though we have created the appropriate firewall rule, it will not accept an incoming connection. In a sense the application is a tiny server listening for the remote host to connect. It is refusing to accept incoming connections and sits there waiting. ie our software is not seeing anything come through.

I can see that by default the RIO system denies incoming by default. So trying to figure out how to enable incoming on this. I see system uses ufw and it may be I just need the correct rule definition for ufw.

Any suggestions?

You have shell installed?
If not, please get a free shell license and get it setup, we will need it to trouble shoot this.
Once you have shell installed, what is the result of this:

sudo ufw show raw | grep 50002

Of course, change the 50002 for the port your .Net is using and the port you opened via groov Manage.

Wait… You have not opened that port via groov Manage Firewall menu?

Yes we did open via the groov manage firewall.

The ufw show raw gives about 9 lines of things for the the ip4.

0 0 ACCEPT TCP – * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:50002

I changed the port number shown here

That does not seem to be a complete result.

Can you grab more of it in text or image please…

(Specifically, looking for the interfaces, eht0 etc)

I am googling here for what are the first two columns? Those numbers on the first line are increasing. So maybe that’s the remote machine trying to connect.

Correct, the first is packets and the second is bytes.

So we are back to something about .Net… The data is getting in, but the .Net is not hearing it or responding correctly?

Yes, now I just did another test with just using the TcpListener class. It is getting a connection and receiving data on the same rio device. Something very odd about how we use sockets going on here. The other linux machine I tested that was fine is a 64 bit linux, more recent kernel version. I am going to explore this some more and post back. From the ufw show, it is trying to do something.

PS When we get this puzzle solved. I will post a known to work example of both sides

1 Like

Well we solved it:

Our legacy code had this chunk of code:

            try
            {
                listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                IPEndPoint ipPEndPoint = new IPEndPoint(IPAddress.Any, 12345);

                listener.Bind(ipPEndPoint);
                listener.Listen(0); // backlog of 0 connections
            }

On all other platforms including 32 bit and 64 bit windows and faster Linux machines it works fine.

Change the allowed backlog to 2 rather than zero fixed it. As follows:

                listener.Listen(2); // backlog of 2 connections

We have a continuous heartbeat going on this connection. I suspect that with the real time linux running other things at higher priority such as the optmmp process, the backlog exceeds zero. Buried in the dotnet documentation, under this backlog condition the socket will silently just wait and do nothing. That matches the behavior we could see. In fact with the right code you could catch it accepting the first message after the connect and then it just sits there silently.

3 Likes

Far out. Nice work.
Thanks so much for sticking with this and closing the loop all the way to the end.
This is going to save someone some time and is a great asset to the forums!

1 Like