Developing a system in a closed network environment - No access to Internet. Concern for maintenance includes:
I currently have that scenario at some of our processes and have been regularly updated and running fine for over 15 years. These systems are updated to the latest 10.4 Pro and all the Epic (Node Red, PAC, groov View, Ignition) and Snap is also up to date. Also MS WIN10 Pro along with defender is up to date for they both can be installed/updated without requiring being on-line. I have read that either WIN 11 or the upcoming WIN12 will require being on-line to install and activate thus this may become a hurdle.
The first three of your bullet points get wrapped up together.
Opto22 has a monolithic firmware update package that includes the OS, system updates and application software package.
This single file is cryptographicly signed by Optos key and is checked by the controller and rejected if its is not signed correctly.
This file can be copied onto a USB stick and thus walked across your air-gaped network.
The update is done from a computer on that isolated network.
Bullet point 4 is also wrapped up in that monolithic update. If you need to install new Node-RED nodes, they are installed on an EPIC with Internet access and the project is backed up and restored on the isolated EPIC.
EPIC has been designed from the ground up to be as secure as possible. Cybersecurity is a very important topic at Opto and we have a lot of resources to help guide you in this area.
Thanks PilotMan and Beno!
I sent this subject out to see if I had missed anything and to see if anyone else has had experience on this subject.
15 years in this environment gives me lots of confidence, PilotMan!
In my research I have come to the same conclusion summed up in your reply, Beno, and thanks for the great Cybersecurity link.
Jim
When isolating a network from the internet you should consider how to prevent wireless communication from defeating the air gap. The most common over-the-air communication technologies include bluetooth, wifi, cell, sound and microphones, but there are others. In addition to wireless attack vectors USB sticks are well known for jumping the air gap.
Bluetooth worms are not given much attention, but they do exist. An infected bluetooth device can spread a bluetooth worm to other devices in range. Bluetooth devices can share an internet connection, so an infected bluetooth device can infect a node in an air-gapped network and then share its internet connection with the infected device. There have been several famous bluetooth vulnerabilities in recent years that don’t require two bluetooth devices to be paired for the exploit to succeed.
Bluetooth hardware is very common in devices today. For example, I wouldn’t know where to find a laptop without bluetooth hardware. In years past, a bluetooth antenna could be flicked off with a soldering iron, but now bluetooth hardware is epoxied onto the silicon and can’t be desoldered. Depending on the device and operating system, bluetooth hardware or the bus its on can be powered down, but this usually involves writing a specialized program (I’m not aware of open source software available to do this). Thus in the closed network, either use devices that do not have bluetooth hardware (e.g. an old laptop) or power down the bluetooth bus with a specialized program you write. Disabling bluetooth in the Windows or Mac OS GUI doesn’t stop the hardware from doing radio work or communicating with the kernel.
Stuxnet propagated through USB devices and then autonomously disrupted its target. The attack vector is an inherent flaw in USB technology and a fix isn’t feasible. For storage devices, I much prefer to use optical media or SD cards. Hopefully that’s an option for at least some of the nodes in your closed network. Some BIOSes have a feature to disable USB hardware, and the BIOS can be password protected. Like bluetooth hardware, depending on the operating system and device, a specialized program you write may be able to power down USB ports. Physical security is critical for reasons greater than USB malware (and including), but please be mindful of what USB devices you use yourself. Using none in any of the nodes in the closed network is the safest option.
Even in a closed network, you should consider securing the devices as best you can, so that if there is an path to jump the air gap, there are other layers of security that may prevent malware.
As I wrote in my previous post, even an air-gapped network should have additional security measures in place in case there is a path to jump the air gap. DIY security is daunting. Unless you’re prepared for hours upon hours of work, the rest of this post may not be feasible.
I’m a proponent of security on each device in a network whether in an air-gapped network, behind a LAN, or a laptop/firewall/cell trio. This usually requires root access to the device. GRV-EPIC has shell access available, so EPICs are candidates for additional security. However, there are not many packages in the EPIC repository, so building security tools from source is the way to go.
Before securing your closed network, test in a non-production environment.
You said the network won’t have access to the internet. Therefore you can effectively block routing for all traffic except private addresses for redundant security. This can be done through groov Manage.
You may also consider firewalling outgoing traffic via the shell. What you pass/filter is domain dependent, so I cannot provide specific guidance.
You can prevent rogue devices from spoofing MAC address by manually mapping MAC/IPs for the devices in the air-gapped network and then filtering incoming and outgoing ARP traffic.
You can harden the linux kernel with sysctl, but this depends on your firmware version as the linux kernel has been updated in EPIC.
Groov Manage has an option to disable USB. This may prevent stuxnet-like malware.
Building from source and using a network intrusion detection system such as snort or suricata can detect some network-based exploits by inspecting packets for malicious or suspicious characteristics. However, they can also block legitimate traffic. Therefore run network intrusion detection tools on alert mode rather than block mode.
In the past I used a tool called tripwire to detect tampering of files. Tripwire is no longer maintained, but there is no replacement with similar functionality. Essentially tripwire creates hashes for all configured files from a base system and regularly monitors the files for changes. If a file changes tripwire can alert you on another device in the air-gapped network.
If the operating system supports TPMs, then they can provide protection against firmware malware and hardware tampering. I’m not aware of support for TPMs on the EPIC, but if you have PCs in your closed network may already include TPMs.
I could go on. If you have questions on a particular topic, feel free to ask.
Thank you for these great tips.
Jim