I am learning how to use the REST API to obtain data from an EPIC PR1. I am aiming to obtain both I/O data and calculated variables from a PAC Control strategy. I followed torchard’s scripts here to create a very basic script. Right now I am just doing single calls to understand the data structure that is obtained. An unexpected surprise for me was the significant delay in obtaining data both from an I/O module and from PAC Control. The script is below:
import sys # to handle argument values
import requests # to make get/post requests
import time
# ignore insecure https requests warning:
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
# head is a JSON object that holds API key information
head = { 'apiKey' : 'SecretKey',
'Content-Type' : 'application/json' }
host = 'opto-host-name'
mod = "1" # 1st argument = module number
chn = "2" # 2nd argument = channel number
# construct request URL for the 'local' built-in rack of I/O *status*
# url = 'https://'+host+'/manage/api/v1/io/local/modules/'+mod+'/channels/'+chn+'/digital/status' #digital module call
# url = 'https://'+host+'/manage/api/v1/io/local/modules/'+mod+'/analog/values' #gets all values from analog IO
url = 'https://'+host+'/pac/device/strategy/tables/floats/fDataLogtable' #test PAC Control call
# make the RESTful get request and save the response
start = time.time()
response = requests.get(url, headers=head, verify=False)
end = time.time()
req_time = end-start #in seconds
When I execute this request for both my PAC control strategy (accessing a float table) and for an analog I/O (accessing all values of the module) both take nearly the same time:
PAC Control call: req_time = 21.80 s
Analog I/O call: req_time = 23.57 s
This is surprising to me as I’d expect the amount of data in both to be fairly small. (my table has 100 rows and the I/O has 12 channels). I am not an expert in this space, but I would not think that parsing that data would take 20 seconds, and makes this whole approach unusable for a monitoring effort. I understand from other forum posts that the REST api is the slowest way to obtain data, but I have seen no indication in documentation that it is several orders of magnitude slower than other approaches (if there is any, please let me know). I have 7 I/O modules and if we assume it takes 20s per module, then we have just over a 2-minute time resolution for our data logging. This ignores the additional PAC control variable calls, which would take around another minute if you call all the int, float, string variables and multiple tables that cannot be called in bulk.
This suggests to me that the bottleneck of the REST api call must be in some other aspect such as the authentication or ‘opening’ the connection. As such, I am hoping I could get feedback on how to accelerate these steps. Currently, these REST calls are being done from a computer connected via ethernet to the EPIC module, with no other connection to other systems. The EPIC system has no data service active, but is running groovView.
The first thing I would consider is the bandwidth of the local network, if your traffic goes through routers / switches with low throughput or additional security, that would easily add to the response time.
Since you are already using Python, have you considered using the OptoMMP library instead? Like you said, REST API is slower and creates a connection for each individual request, where MMP is fast and can maintain an open socket to the controller for as many read/writes as you need before closing it, including the scratch pad areas.
Regarding documentation on speeds, that’s a great point — I’ll run some tests to get concrete data and put together a separate post to help decide when REST is the right approach.
Philip beat me to it. I was just setting up to run your script here…
If it was ‘normal’ to take 15+ seconds for such a simple RESTful call, two things, 1. We would not have released it in that state. 2. If we had, it would be very clear in the language about what sort of poll response time to expect.
When the subject comes up about ‘REST is slower than MMP’ we are talking in the 5msec times. Not 15+ second time frames.
As Philip said, I wondered about some sort of name lookup timeout or some issue with the cert. ie, something else is taking the time to do your script, not the RESTful call from the PC to the EPIC.
You could also try running the script on the EPIC itself to test the RESTful call response time.
We have 100’s of customers using the REST interface for many different control scenarios. This is the first forum post with these sorts of numbers, so its just a matter of finding out why and then figuring out how to resolve it.
Thank you for your responses. Following phillip’s suggestion to use the numerical IP address rather than the host name made a world of difference. All the api calls I test with the IP address now respond in around 0.3-0.5s, which is a lot closer to my expectations! I might have missed language in the documentation on the use of IP address versus host name, but if not then that would be a good hint to provide somewhere.
Regarding the bandwidth of the ethernet connection between the PR1 and the PC. Do you have thoughts on how to quantify this to determine if there may also be an opportunity to lower responses times further? I also wonder if the PR1 has to schedule my api request within its various PAC strategy scheduling ordering and that may add some latency as well? Does the PR1 handle all api requests and PAC strategy within the same thread? All this is certainly minor improvements compared to the speedup of using the IP address, but I am curious as my response is still about double what philip noted in his own test (though there will likely be call-to-call variation).
Host name lookups should about the same speed as IP address. There might be a 100msec or so delay the very first time that your PC does the hostname to IP lookup, but that after the speeds will be the same.
There is some miss-configuration on your network or PC that is causing the delay for every REST call. That’s why using IP vs hostname is not mentioned in the documentation.
The API server to totally different from PAC Control. The EPIC is a quad core CPU and PAC Control is multithreaded, so there is no impact on either.
Its hard for us to start digging into your network setup. Do you have an IT department that can assist you in that regard?
I literally just have an ethernet cable from the computer to the PR1, no switch, or router in between. As such, my guess is that the PC’s Windows Defender setup might be getting in the way. Is there documentation I could use when discussing with IT about what ports should be allowed to connect between the PR1 and the computer? Any PR1 configuration that might be worth investigating? Pinging the PR1 from the PC command line I get a response time <1ms when using the IP address. Repeating with the PR1 host name it noticeably takes around 20s (timed 3x with my phone, nothing rigorous) before the actual ping commands are sent, which return at the same speed as the IP address pings. This does point to the hostname-IP translation being the culprit. Not sure what configuration of the connection between the PR1 and the PC would need to be addressed to prevent this. In the meantime I will just proceed with the IP address directly.
