A customer who is brand new to all things Opto 22, automation, and programming called and needed some help with saving a simple data log to his controller. After helping him get things going, I thought this chart might be useful for someone else who is new to these things and can get them started. Attached is the chart to accomplish this.
Warning: If you set a short save interval or leave this running for an extended period of time, you can fill up all of the space on your controller rather quickly. Mary has a good example of a Change Log that has some more elaborate error checking to prevent this problem.
For the sake of simplicity, I left the error checking at a minimum and just cut to the chase to create, open, and write to a file.
FYI, the trick you mention (in the comments, on the chart) of looking at the file created on the controller using your browser, works best on Firefox (Edge seems okay too), but not Chrome and IE for some reason. Also see this “comm handle 101” for other gory details and recommended tools/troubleshooting.
On the topic of super simple file read examples, here is my code (heavily stolen from OptoMary) for reading a CSV file.
The kicker here is that I am using this code on SoftPac… Because it runs on the same PC as my Dropbox, it can see the C: drive data as its local file storage.
In this example I am using another service (http://ifttt.com) to write the csv text file, this code just reads it and puts it into a string table.
// check the dropbox for any files that may have come in from IFTTT.
// define 'file_dropbox' as the following in your strategy tree..... Or do it inline.... Your call...
// file:r,C:\dropbox\IFTTT\SMS\sms.txt
setCommunicationHandleValue("file:r,C:\Dropbox\Ben\IFTTT\Data\csv.txt", file_dropbox) ;
ifttt_status = OpenOutgoingCommunication(file_dropbox);
if (ifttt_status == 0) then
//pull the contents of the file into a table.
SetEndOfMessageTerminator(file_dropbox, ',');
ifttt_status = ReceiveStrTable(4, 0, csv_data_table, file_dropbox);
// Ok, we got the file contents, delete the file so IFTTT can make the same file next time.
ifttt_status = SendCommunicationHandleCommand(file_dropbox, "delete");
ifttt_status = CloseCommunication(file_dropbox);
endif
I have this code in a chart that runs every 15 minutes.
I found that if you don’t delete the file, a new file will be written with a different file number, so to keep things clean, I delete the file after I get the data.
Note that I have told the IFTTT recipe to call the file it creates ‘csv.txt’, that’s where that name comes from.
I also told IFTTT to put a ‘,’ between the data.
I have decided that this year is going to be the year of the quantified self, so I bought a fitness tracker to wear… A Fitbit Charge HR.
Since I am in groov all day long, I thought it would be nice to have some of my health data in there as well… a sort of health dashboard (Hey, I have a PAC Controller status dashboard there, why not one for the master controller - me!)
Thus I use IFTTT to get my Fitbit sleep and activity data and make a csv file(s) in my drop box.
Since we also own a Withings body scale, I also use IFTTT to get the data from the scale every time I weigh myself and create a CSV file.
Lastly, I also have a Withings blood pressure cuff, so yeah, IFTTT to CSV every time I take my blood pressure.
The four files are sucked up by SoftPAC and I can see my data in groov and SoftPAC also squirts it in a MySQL database running at my house.
We have a few other customers using the combination of IFTTT, DropBox and Opto for other non-mission critical data handshakes.
This is where the code came from in the first place, Mary and I helped them out with the file reading.
Take a look at the channels they have, there are some really interesting things you can connect to…
Let me know if you have any questions or if you want to share any ideas you put into action using it.
Sounds cool, thanks for sharing. Have you looked into logging any of the data/metrics to a time-series database?
I’ve been experimenting with sending data from a PAC-S (among other things) to Graphite or InfluxDB and then I use Grafana to display them. A TSDB will perform a lot better over time than a relational database, being fit for the purpose. Being able to graph multiple years of data in milliseconds is a pretty cool ability.
Imagine a decade long trend of your blood pressure that you can take to your MD!
No, have not given it much thought outside of MySQL. Mostly because I only get Fitbit csv files every 24 hours, so we are not talking about a lot of data, even if I keep this up for 10 years. (3650 records).
I am working the Fitbit API to see if can get my heart rate from throughout each day, that might make sense to time-series… will look at that when I get the data flowing.
Thanks for the keyword Grafana. Had not seen that one before. No question it looks really slick.
One of my un-movable criteria is to be able to generate the graph as a jpg or png for use in the groov video gadget.
Do you know if Grafana can generate a static image when called? (Say with a cron-job).
Yes, long term data for the MD would be sweet. I have started a bit late I guess, at the moment we barely have a few days data… (Also, I don’t have an MD in this country yet… Been here almost 8 years and have never been to the doctors!).
Grafana can render a png, but you may have to deal with authentication. If you have the installation setup for no authentication, then you can get a direct link to a graph as a png and it can show the latest data each time it is refreshed.
Just tested this, the URL to get a png looks like this:
This looks great but I can’t see the attached strategy and Mary’s link is broken.
If I can’t get to this one, does anyone have a simple example on how to read and write to a file? I can’t seem to accomplish either one despite having read several forum topics on the subject.
Thanks! You’re right, lots of good stuff there. I did finally find a post from Mary describing what I was looking for. The key for me was to close the connection. Once I did that everything started working.
Hi Guys,
Sorry about broken links (unfortunately, those didn’t translate well in this new platform so I’ve been fixing them by hand as I come across them, please do keep letting me know when you find them so I can fix 'em!)
Thanks,
Mary
I don’t send data at ms rates. I was speaking about the rendering of a chart with large amounts of data, not the actual data interval.
Where these databases are used in the IT world, data is sent to it that fast and from multiple sources, so it can be done - provided the controller can send the data out at that rate - Soft PAC and possibly EPIC might be able to get into the several ms range when pushing the data from a strategy. A PAC would have a hard time with that without some extra effort (UDP streaming of OptoMMP data from the IO unit to a system that then processes the data into the TSDB, maybe??).
Also, latency to the data storage wouldn’t matter as much if you use UDP.
