I have a groov EPIC PR1 running version 3.6.0-b.32. The system was working fine; however, after installing some nodes in Node-RED (node-red-contrib-modbustcp and node-red-contrib-tcp-ping), the disk filled up completely.
This is the second time this has happened. The first time, I had to perform a factory reset because the system stopped working after a reboot.
We have a lot of customers using those popular nodes, I also use them a fair bit, they don’t do anything to fill up the drive, so I think we can discount that as being the root cause (and as you mentioned, its happen before).
You say you cant access SSH? Is that because you don’t have a shell license installed, or you do have it installed, but cant log in for some reason?
Having shell installed would be the best way to find out what’s taking up the space.
Without it, you have to factory reset and just start things up slowly and keep an eye on the drive space.
Shell has some Linux commands for finding large files, so that’s why I mentioned it would be the quickest way to determine the cause.
Ignition logs can eat up the drive space, more so if you have a network issue and are logging lots of fail/retries.
groov View trends, if you have a LOT, can fill up the drive. It may not start because the drive is full, just depends on the other apps you have running as to when each starts (or cant start because of the zero space).
Just to re-enforce the behavior you are seeing, I have had a log file run wild and fill up the Epic PR1 drive. The log file was being written from within my strategy. I could not login using SSH, I could not login in to groov Manage, it did not respond to API. There was no way that I could find to free up any space. My only option was to factory reset, and to fix my logging code.
I’ve come across a lot of disk full systems over the years and never not been able to even log in via shell. Sure, once logged in, very limited functions are possible till I make some space, but never been locked out of shell, even on both EPIC and RIO (perhaps they were not really quite at zero?). Your experience of not being able to shell in is very interesting.
Yes, I have seen quite a few PAC Control strategies fill all space. Also rouge TCP comm handles use up all the sockets and lock out different commands.
Thanks for your response, Beno. Regarding SSH, I do have the license installed, but I can’t create the account despite entering the admin password. I assume it’s because the disk is full.
As for Node-RED and groov View, the groov EPIC is reading data from a groov RIO connected to two pressure sensors. The groov EPIC communicates with the groov RIO using a radio link, and due to various factors, the signal is constantly interrupted.
I am using groov View to display the process results, and for the connection with the groov RIO, I use Modbus Device because the connection is more stable (initially, I managed to connect using Opto 22 I/O Unit, but after the first interruption, it didn’t reconnect). However, in recent days, the interruptions have increased, so Node-RED was considered as an alternative because, in my experience with groov EPIC, the Modbus connection using Node-RED is more robust than using groov View.
I assume that the constant connection error messages generated by Node-RED are what filled up the disk. Regarding the trend graphs, I only have two, each with one variable. I was monitoring the disk status daily, and it only filled up after applying those nodes in Node-RED.
If you have already installed the license, then you don’t need to create the account in groov Manage a second time (or each time you use it).
Just log in using PuTTY or such.
The Node-RED log is capped at the last 1000 lines (unless you changed the settings.js at some point), so it will not fill up a drive no matter how many communications errors you have.
Something interesting going on here.
Reach out to our support group and see if they can help out before you do the reset.