Problems with disappearing files on microSD card

Hi,
I have a R1 running PacProjectPro 10.4 (f/w 10.4c) which stores files on the microSD card (8Gb)
I have a strange problem with files disappearing off the microSD card after several hours of operation. If the microSD card is removed from the controller and checked on my PC, the files are present. Also, if the controller is power cycled, the files return. It seems to be a problem with the controller file system getting corrupted over time. There are only 10 files, they are small (<1Kb) and are only read once every 8 hours.
I have tried STOPPING the strategy and found the files still disappear overnight, so it doesn’t seem to be software related.
I tried using a different microSD card.

Anyone have any ideas?

Thanks,
Corey

HI Corey

Strange problem indeed. Have you perhaps done a litmus test by simply trying another SD card? IF you get the same results you at least eliminate the SD card from being one of the problems and can then start looking into the controller/software.

Has it been working for some time and the problem only recently surfaced? Has anything changed before the problem popped up (firmware or software updates, power failures) ?

Hi Corey,

I have never used the SD card on a controller, but seem to recall that you can FTP to the card?
That wold be a good check, from a PC running something like FileZilla or from Windows File Explorer, path out to the card and see whats going on there.
The fact that they go away when the strategy is not running is a bit of a clue that something else is accessing the files. Just need to track down what/who.

I often access my log files on the SD card fitted in the R1 using Filezilla. Always found it reliable.

There have been problems in the past with using a ‘lower spec’ SD card where files were corrupted or completely missing, especially those on fleabay. Now I only use better quality SD cards such as Samsung (others are available!).

That’s a really great point you raise there.
I have moved all my Raspberry Pi micro-SD cards to ‘High Endurance’ type and have a zillion less issues with them over the past few years than when I first started out. (I have many Raspberry Pis installed around the world as ADSB aircraft monitoring stations).

@C.Hughes Not sure if you are in USA or not, but if so, here an example for what you can get from the likes of Amazon: https://www.amazon.com/s?k=high+endurance+microsd&ref=nb_sb_noss_1

I have tried using a different SD card of the same brand. I will try getting some ‘high endurance’ cards. Today I found that the card left overnight in the controller was totally corrupted and had to be reformatted in order to write or read and files on it.
I set up a separate test controller running the same code and it also caused the files to disappear/corrupted.
I am testing the separate test controller to see if it will corrupt the files when the strategy is NOT running.
To answer a question above, this strategy is new. I have other strategies that do similar functions storing on microsd cards and never had a problem.
I have a lock flag to prevent different charts from accessing files on the sd card at the same time.
There is a loop that checks the microsd card every minute to see if a certain file exists (by trying to open the file for reading). If the file exists it gets transferred to a server and deleted.

Thanks for the ideas. I will post back when I learn more.

Thanks for the extra details.
Very interesting the three tests… Same strategy in two controllers, same problem.
Different strategies in other controllers, no problem.

I have not used the lock flag a lot, but I do know that having more than one chart accessing the card is an issue.
I wonder how much work it would be to re-craft your strategy to have an SD card master control chart.
It and only it can read/write from the card. All other charts ask that one chart for what they need when they need.

1 Like

Update - It seems the high endurance microsd card fixed the issue.
Thanks for the help.
Corey

1 Like

Thanks so much for following up and letting us know.
Its so helpful to have a thread with a happy ending!