Node Red HTTP IN Node Post Endpoint

[Robertson]

I have a node red flow that uses an HTTP In node to create an endpoint to accept data from other devices. It parses the data and put it into the Epic’s MariaDB.

I am testing it using Postman (https://www.postman.com/).

When I am inside my network I use the local IP address of the Epic (note no port specified). It works.

Here is the post command I use in Postman:

https://10.1.5.101/node-red/test?signalin=7.54&eventdatetime=2021-04-17 13:52:58&unitid=21477&channel=1

When I am outside my network I am port forwarding all traffic coming to my gateway router xxx.xx.xx.xxx:18801 to the Epic (10.1.5.101). It will not work. I have tried specifying ports 443 and 1880. It will not work.

Here is the post commands I used:

https://xxx.xx.xx.xxx:18801/node-red/test?signalin=7.54&eventdatetime=2021-04-17 13:52:58&unitid=21477&channel=1

I can monitor the router traffic and see that it being allowed through the firewall.

Router log entry when forwarding to port 1880

2021-04-17 16:33:36 Allow xxx.xx.xxx.xxx xxx.xx.xxx.xxx 1880/tcp 56384 18801 0-External 0-LAN Bridge Temp Allowed 52 125 (Node Red HTTP Endpoint 10.1.5.101-00) proc_id=“firewall” rc=“100” msg_id=“3000-0148” dst_ip_nat=“10.1.5.101” dst_port_nat=“1880” tcp_info=“offset 8 S 4253526887 win 64240” geo_src=“USA” geo_dst=“USA”

Router log entry when forwarding to port 443

2021-04-17 16:54:07 Allow xxx.xx.xxx.xxx xxx.xx.xxx.xxx 443/tcp 56919 18801 0-External 0-LAN Bridge Temp Allowed 52 125 (Node Red HTTP Endpoint 10.1.5.101-00) proc_id=“firewall” rc=“100” msg_id=“3000-0148” dst_ip_nat=“10.1.5.101” dst_port_nat=“443” tcp_info=“offset 8 S 2826133454 win 64240” geo_src=“USA” geo_dst=“USA”

Here is the Postman error I get when I try forwarding to port 1880

Here is the Postman error I get when I try forwarding to port 443

What port does the HTTP In node listen on?

Any ideas on how to fix or troubleshoot further?

Thanks,
Craig

[Beno]

It listens on 1880.
https://cookbook.nodered.org/http/create-an-http-endpoint
That’s going to be tricky on the EPIC/RIO because we put Node-RED behind our authentication and encryption service to make Node-RED secure.
Node-RED does not respond to 1880 (as you have discovered) as we do an internal forward to 443.

Do you have to use the TCP node?
I always use the UDP node for this sort of thing since you can specify the port.
Open the firewall on that port number and I’m up and running in a minute or two.

[Robertson]

Beno, you asked about a TCP node, just to be clear, and don’t know if it matters but I am currently unsuccessfully using an HTTP IN node with an HTTP RESPONSE node.

The customer specified an HTTP Post endpoint. I will check to see if UDP can be used.

I looked for how to structure a UDP Post request on Postman and came up empty.

This is new for me…so I’m not even sure that is possible.

Also, in your explanation, you mention …“Node-RED being behind our authentication and encryption service.” Why does it work when I am inside my network? I specify the api key in the header.

Also, I have successfully used this network approach to access the MariaDB on the Epic from outside our network…redirecting the traffic at the gateway router on port xxxxx to the Epic’s port 3306.

Thanks for your insights.

[Beno]

We don’t redirect port 3306, that’s all yours to do what you like with. (In your case, use on a database).
Port 1880 is a little special.

You have a good point about it working on the local network. Perhaps the fact that it works locally means this is not as big of an issue as I thought. If it works locally, then it works on the EPIC and the problem you are finding has nothing to do with the EPIC.
I will need to test it some I guess and see what I can find.

[philip]

The only port that Node-red will be available on outside of the EPIC is 443. Edit: At least when using the HTTP In node - you may be able to find a module that allows you to HTTP listen on your own independent port like node-red-contrib-http-instance (node) - Node-RED - I know nothing about that node, just searched for http server on the node-red site.

Your second postman error is showing that it is seeing a self-signed certificate - you will need to add the self-signed certificate to the store of the machine that is running postman. Though I am confused how you got it working locally.

Using Self-Signed SSL Certificates with Postman | Postman Blog

[Beno]

As usual @philip comes in with much clarity and insight. Thanks.

If I can try and sum up some ideas now from the past few posts…

EPIC forwards /node-red URL requests from 443 to 1880. But in doing so, you must authenticate (with a groov Manager user account) and encrypt using the certificate on the EPIC.
As Philip points out, you might try another http node, one that allows you to chose any port (above 1024) you like. Then you just need to open the EPIC firewall and sort out the SSL side of things.

@philip

Though I am confused how you got it working locally.

This is what threw me off as well.
But, I have a thought… I wonder if locally you are using Postman in a browser that has accepted the EPIC self-signed certificate? This would allow the https post to work, but on the remote machine, the SSL cert has not been installed so it is failing.
A helpful resource for working with certificates can be found here:
http://developer.opto22.com/epicdev/browserssl/

[Robertson]

Success!

Thank you Beno and Philip. I took your recommendations and turned off a Postman setting called “Enable SSL certificate verification”.

My gateway router settings are forwarding TCP data coming in on port xxxxx to the Epic’s port 443.

The Epic’s firewall already had an entry for port 443.

Appreciate the help.

[Robertson]

The test with Postman worked well after I turned off the setting “Enable SSL certificate verification”.

When we went to integrate it with the customer they objected to turning off this feature. They said it was a security risk they were not willing to take.

They suggested obtaining a Server Certificate from LetsEncrypt…one of the options listed in the link philip suggested above.

In LetsEncrypt Getting Started information they recommend using the Certbot ACME client with shell access.

Looking at Certbot… to get instructions on installation they ask what software the website is running on and what OS. Both questions I do not know the answer to.

Certbot Instructions

Am I headed in the right direction? Is this the best way to implement a Server Certificate for the Epic?

[Beno]

It does not matter.
LetsEncrypt have a drop down that includes ‘other’ for both of those.
The cert is in no way linked to the OS or the webserver.

That said. EPIC is using Nginx and Linux.

Keep in mind that this will only ‘work’ if the customer software and the EPIC both have access to the Internet.
The customer software will need to check the EPIC cert with LetsEncrypt.
Also you will need to add the cert to both EPIC and the customer computers trust store.

[Robertson]

Thanks Beno.

The drop-down for system has lots of Linux options. It looks like depending on the one you select it creates the SSH commands. Given the Epic is a custom Linux build, which one should I choose?

How does the process above relate to the certificate management system in groovManage?

[Beno]

Since you are not going to use SSH at all, the drop down options do not matter in slightest.

When you get the LetsEncrypt cert, follow step 7 to install it on the EPIC:
http://developer.opto22.com/epicdev/browserssl/server/#vii-upload-the-server-certificate

Then follow step 3 to install it on the computer trust store:
http://developer.opto22.com/epicdev/browserssl/client/#Windows

[philip]

Can you add the self signed certificate to the remote computers trust store? Then you wouldn’t need to disable the certificate verification.

If you want to go the public certificate route:

LetsEncrypt certificates are only valid for 90 days. They are typically setup to use automated renewal which will not work on the EPIC. If you want to use LetsEncrypt, then it would be best for your customer to setup a separate device/server with nginx on the local network to reverse proxy to the PR1 and have the certbot scripts installed for automated renewal. The reverse proxy would be exposed to the internet and the EPIC would not. The reverse proxy will need to trust the self-signed cert of the PR1.

You should not need to install any certificates in the remote computers trust store since LetsEncrypts CA should already be trusted.

If unable to do the automated route, then the other option is to purchase a certificate from another CA which are valid for 1 year, which would lower the labor costs of renewing the certificate every 90 days.

[Beno]

@philip

Can you add the self signed certificate to the remote computers trust store? Then you wouldn’t need to disable the certificate verification.

This. ^

There are, as @philip points out, so many pain points with the LetsEncrypt route.
Far better to spin up a new cert on the EPIC and install that in the computers trust store.
If you want to go this route, I again refer you to the link I posted before: Getting a Trusted Connection Between a Web Browser and groov EPIC | Opto 22 Developer

[Robertson]

Ok…thanks for being patient with me. I was thinking I could use SSH.

If the customer pushes us to the public certificate route, I’m finding that you need a domain name of the server hosting the site. Since the EPIC is behind a gateway router and has a non-routable IP address and is not necessarily hosting a webpage what do you use?

[Robertson]

When you say add the “self signed certificate” to the remote computers trust store… do you mean the one that is created in groovManage here:

[philip]

You can download the currently used self-signed certificate by clicking the Download Public Certificate. This will be in PEM format, you may need to convert this to another format depending on the remote systems needs (there are web converters you can use - it is safe, as it is a public certificate). If the system is windows, then you want p7b format. Then you can open the certmgr and add the certificate to the Trusted Root Certification Authorities.

The built-in self signed certificates common name will be something like opto-00-00-00, since the remote system will likely not be able to resolve that, you may need to add that to the remote systems host file or add the entry to the local DNS server. That will prevent invalid common name errors with the certificate. If you must connect via IP address, then you will need to create a certificate with the IP address that the remote system uses to access the PR1 as the common name and add that as the server certificate for the PR1 (or you can create the certificate with the opto-00-00-00 as well as the IP in the Subject Alternative Name). To create that cert you would probably want to use something like openssl to create it.

Also note the expiration date of the self-signed certificate so you can do this all over again in a couple years. Fun!

[EngAppDev]

With the help of someone who is versed in this, we were able to successfully setup a public certificate using ZeroSSL. He also configured a domain name to point at our gateway router.

Using the combination of the domain name and port forward rule I configured on the router I can get to the EPIC from the outside using HTTPS.

I can also use Postman configured with “Enable SSL certificate verification” set ON and send POST request successfully.

The only thing that is not working now is my node-red-contrib-pac read and write nodes will not connect to the EPIC controller. The error is “Unable to get issuer cert locally”

[Beno]

Sounds like outside access is Ok since you can now check the cert, but internally the certificate can not be checked, so fails.
Did you try removing the ‘cert.pem’, ie leave the field blank?
You are using an API key value from a user that has PAC Control REST permission? (Set in groov Manage).

EDIT. If you are running Node-RED on the same PR1 as the PAC Control strategy, you must use ‘localhost’ not the IP address as shown in your screenshot.

[EngAppDev]

Thanks Beno. The localhost change worked.