pfSense as an ICS / SCADA environment internal segmentation firewall

ka3ax

Hello pfSense experts. We would like to protect our industrial control computers and equipment with ICS / SCADA internal segmentation firewall. We use pfSense as our router / perimeter firewall and we would like to try it as ICS / SCADA environment firewall.

Are there manuals or guidelines for this scenario? Can people share their success stories?

JKnott

@ka3ax

Perhaps you can start with the requirements. Sayings ICS /Scada doesn't say much. What ports are required etc.

ka3ax

Hi JKnott and thank you for your reply.

JKnott> Sayings ICS /Scada doesn't say much.

Indeed, my description is too short. I will try to provide more details below. We invited a 3rd party company to perform network security audit. We have several outdated WinXP computers in our production area. The auditor recommends replace those old computers with fresh Win 10 machines. Some of them are controlling industrial equipment – not easy (or not possible) to upgrade. Auditor told that we can put those old computers in specially protected network segment when we have no other option. We need internal segmentation firewall in this scenario.

JKnott> What ports are required etc.

All our computers (including those outdated ones) are connected using standard Cat 5 / RJ45 / gigabit LAN. I think even two port pfSense box is good for start.

Auditor mentioned Stormshield SNi40 as an example of ICS firewall, and we also got an offer mentioning Fortinet Fortigate 30D from another company.

All the offers insist that classic perimeter firewall will require additional settings to protect ICS (Industrial Control Systems) area. I am in contact with the auditor trying to get the list of most important points / settings for internal segmentation firewall. We like our pfSense router and I wonder if can use it also as an ICS firewall.

JKnott

@ka3ax

Well, pfSense is certainly a capable firewall, but you still have to determine requirements. For example, you mentioned a protected network segment. Do those devices need any access beyond that network? If not, then don't connect it to anything else. Is there custom software involved that might use non-standard ports? Until you can answer that sort of question, we can't give an answer.

BTW, I recently saw Windows 98 being used in an office, but I don't think it was connected to the Internet.

johnpoz

If all you want is a downstream firewall then sure pfsense can do that without any issues.. But curious what this specifically means

that classic perimeter firewall will require additional settings to protect ICS

Did they just mean if your ICS network is hanging off the edge router/firewall that you would need to make sure it can not talk to other network segments on the same router/firewall - or that your other segments can only create specific unsolicited traffic inbound into the ics network, etc.

While pfsense can run IPS as well, maybe they are looking for something specific?

bmeeks

The nature of the ICS/SCADA network is of some importance here. For example, if simply a factory that is making widgets using automated machinery, that represents one level of security and something as simple as having all of those devices on a separate IP subnet isolated behind a firewall that prohibits all or nearly all unsolicited inbound connections is OK.

On the other hand, if that ICS/SCADA network is say a power generation station or chemical processing plant, then you would want much stricter isolation even up to using data diodes which absolutely prohibit inbound communications of any kind (via hardware). Here is a link to one company's data diode products: https://owlcyberdefense.com/. There are several other vendors out there as well. I just happen to be familiar with this vendor's products. Google "data diode" for more information about the technology in general. Data diodes are outstanding devices for totally isolating a network from inbound traffic while still allowing devices behind the data diode to send out status messages. So you could stream out data for monitoring the processes behind the data diode, but you absolutely cannot send anything back (including even protocol handshaking). Data diodes allow you to have your cake and eat it, too, because you have the isolation of a true air-gap (when considering inbound traffic), but yet you can still send out data from the isolated networks.

ICS/SCADA networks can be particularly vulnerable to modern cyber exploits because they are almost never designed with security in mind. There is usually no concept of SSL. Even SSH is not common and Telnet is still widely available (yep ... ). To protect these inherently trusting networks and devices you need robust cyber practices. It is very important to severely limit communications into and out of such networks. That's where data diodes can help. For ICS/SCADA networks, a firewall is good, but not really all you need. Consider how severely such a trusting network could be instantly compromised if some WinXP machine back there made an outbound connection to an external web site and downloaded some wormable exploit. Data diodes make this physically impossible.

It is helpful when implementing cyber protections for ICS/SCADA systems to break your network up into functional layers. Typical layers would be Control, Monitoring, and Business. The Control Network layer is where all devices that have actual process control functions reside. This network layer gets maximum protection and for sure would be isolated behind a data diode. The Monitoring Network layer contains systems that only provide information about processes but can't control anything. This network layer may or may not be isolated from the Business Layer (but it is a good practice to at least provide a firewall segment between the two). The Business Layer is typically the company's internal business LAN.

ka3ax

johnpoz, bmeeks, thanks for your inputs.

I still do not have full list of important ICS firewall differences (from regular firewall). I remember few points discussed:

no access to the Internet or to email should be allowed from ICS networks
ICS networks should be defended from other plant networks, especially those with Internet access
limited or on-demand (normally blocked) remote access
Those requirements are conflicting with our typical access and maintenance scenarios...

JKnott

@bmeeks said in pfSense as an ICS / SCADA environment internal segmentation firewall:

There is usually no concept of SSL. Even SSH is not common and Telnet is still widely available (yep ... ).

Of course the reason for that is SCADA predates the public Internet and was originally run over dedicated networks. About 11 years ago, I was in this place updating some telecommunications equipment. They were still using dial up modems to connect the various sites to their control centre. Yep, the Ontario power network was controlled with dial up modems. When I was in there, about 3 years ago, the equipment I put in was gone and they'd moved on to a more modern system over fibre.

JKnott

@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:

johnpoz, bmeeks, thanks for your inputs.

I still do not have full list of important ICS firewall differences (from regular firewall). I remember few points discussed:

no access to the Internet or to email should be allowed from ICS networks

ICS networks should be defended from other plant networks, especially those with Internet access

limited or on-demand (normally blocked) remote access
Those requirements are conflicting with our typical access and maintenance scenarios...

The no internet access is easy enough. You just don't connect it to anything. Critical networks are often completely separate from those used for other purposes. What triggers the "on demand" access?

bmeeks

@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:

johnpoz, bmeeks, thanks for your inputs.

I still do not have full list of important ICS firewall differences (from regular firewall). I remember few points discussed:

no access to the Internet or to email should be allowed from ICS networks

This is to protect those vulnerable components from malware infestation. The primary exploit vectors these days are through browser or browser plugin flaws (Adobe Flash being one example) and email attachments or hyperlinks. If vulnerable components and networks can't access the Internet or email, those attack vectors are mitigated.

ICS networks should be defended from other plant networks, especially those with Internet access

This is another layer of protection from the threats I described above. Trusted networks can act as a "carrier" of a virus that then infects your sensitive networks. You want to severely limit "trust" in ICS networks.

limited or on-demand (normally blocked) remote access

Remote access equals bad cyber practice! In the industry I retired from recently, remote access to control systems was absolutely prohibited by government regulations. You got into serious trouble for violating that one! We used those data diodes to enforce that rule. If I needed to make any change to a control network, my butt went into the physical plant, through all the physical security and stood in front of the actual device (or least a terminal within a physical security boundary that was attached to the control network). And speaking of physical security, you need to have absolute physical access control to all parts of your Control Layer network. That means locked rooms where Control Layer components are located. It also means some type of alarm system for locked cabinets that might have to be located outside your typical physical security boundary (think a device that might have to be outside the fence).

Those requirements are conflicting with our typical access and maintenance scenarios...

I understand it can be a pain in the butt, but remote access to critical Control Layer components is just never a good idea. It is very hard to secure at a very high confidence level. You just can never eliminate human error.

One thing I did not see you mention, but that is a HUGE concern, is what are called portable media and mobile devices. This area includes USB memory sticks, flash memory cards, CD/DVD discs and even cell phones being charged via USB cables. You need very tight controls on these devices around Control and even Monitoring Layer networks. It would be so easy to compromise a system via a USB stick or even some employee plugging in their cell phone's USB charger cable into a vulnerable PC. In fact, USB media is exactly how the infamous Stuxnet virus infiltrated the Iranian nuclear processing facility and then went on to damage multiple centrifuge machines there. A worker was used to unknowingly carry the virus into the secure facility on a USB stick. When he plugged the USB stick into a computer on the secure and isolated network, that allowed it to spread across the sensitive Control Layer network and do its work.

bmeeks

I'll say one last thing to maybe more directly answer one of your questions about the differences between a typical firewall and an ICS firewall.

Functionally, there is no real difference. One firewall is fundamentally the same as another when speaking of today's stateful inspection technology. It's mostly a difference of features, support and of course cost. Cost runs the gamut from free to thousands and thousands of dollars for the hardware and software support contracts.

I love pfSense, and it is a very secure firewall out of the box. But it has one shortcoming when used with ICS networks. It wants to be able to talk to the Internet. The "home" screen, immediately upon opening, attempts to query the pfSense/Netgate servers to check for new firmware updates. The package and update system of pfSense is also heavily dependent on Internet connectivity. You can, with much trouble, create a sort of offline package server, but it takes quite a bit of skill to pull this off.

For ICS networks, especially those most sensitive Control Layer networks, you want a firewall that is happy being 100% isolated from the Internet. One that allows you to update it with a single file by uploading the file to the firewall from a local management network connection. So when looking for your ICS/SCADA firewall, ask the vendor about their support for 100% isolated setups where the firewall has absolutely no Internet access. Can the firewall work in that environment without issue, and does the vendor provide an offline update process that is workable for you? There are firewalls out there that fit this bill, but unfortunately none of the ones I'm familiar with are free.

ka3ax

bmeeks, your post inspired me to read more about data diodes and unidirectional networks. Also found few articles explaining "why firewalls are not recommended for securing SCADA systems"...

Luckily our site is not a power plant network and remote access to control systems is not prohibited by government regulations in our region... :)

It is just a production plant with few assembly machines. If disaster strikes we can restore everything quite quickly, our estimation is 8-48 hrs. And management is happy with this estimation. So I hope that reasonable / descent security level will help us to improve our security score without going to very expensive solutions and paranoid security mode. :)

bmeeks

@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:

bmeeks, your post inspired me to read more about data diodes and unidirectional networks. Also found few articles explaining "why firewalls are not recommended for securing SCADA systems"...

Luckily our site is not a power plant network and remote access to control systems is not prohibited by government regulations in our region... :)

It is just a production plant with few assembly machines. If disaster strikes we can restore everything quite quickly, our estimation is 8-48 hrs. And management is happy with this estimation. So I hope that reasonable / descent security level will help us to improve our security score without going to very expensive solutions and paranoid security mode. :)

That's why my first post mentioned first determining the "value" of the network you are protecting. It's one thing if the loss is just monetary, but quite another if loss or sabotage of the network could result in injury or death to people. For a typical factory scenario, I would also say a data diode and all of the other physical security expenses I mentioned are overkill. However, if you are protecting a nuclear power station and its reactor, or a chemical processing facility or say the national electric grid, then more robust controls are definitely called for.

However, even with just a factory network, you will benefit from tight portable media and mobile device control. It would be a bad thing for the company to lose thousands of dollars of production because Joe brought in some pictures of his grandkids on a compromised USB stick and plugged it into a PC in the plant control room to show his co-workers.

johnpoz

@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:

We have several outdated WinXP computers in our production area

If this was the case in any of the scenarios you mentioned, National Elec grid, Nuc planet - I would seriously be like WTF People!!!

bmeeks

@johnpoz said in pfSense as an ICS / SCADA environment internal segmentation firewall:

@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:

We have several outdated WinXP computers in our production area

If this was the case in any of the scenarios you mentioned, National Elec grid, Nuc planet - I would seriously be like WTF People!!!

WTF is an appropriate response, but that old nemesis money comes into play. You have vendors that provide highly specialized components (sometime custom designed for just one site) and they want many millions of dollars to "upgrade" their component for you and to certify their old software on new operating systems. It becomes, unfortunately, a money decision made on a risk vs cost to mitigate scale. Sometimes totally isolating the network and associated devices is the most cost effective approach for a short-term solution. Longer term you plan on replacing that component or system. However, in the case of power plants, consider the impact of spending millions and millions of dollars to upgrade systems on consumers' power bills. They will yell and scream and the utility rate regulators are stingy with granting rate increases.

NogBadTheBad

@johnpoz said in pfSense as an ICS / SCADA environment internal segmentation firewall:

If this was the case in any of the scenarios you mentioned, National Elec grid, Nuc planet - I would seriously be like WTF People!!!

https://www.theregister.co.uk/2018/11/09/royal_navy_old_os_at_sea/

johnpoz

Yeah that really is a WTF... but if that is true
"It’s all standalone, not connected to the outside world,”

That is something

JKnott

@johnpoz said in pfSense as an ICS / SCADA environment internal segmentation firewall:

I would seriously be like WTF People!!!

Unfortunately, there is a lot of gear around where the manufacturer can't or won't update it. In some cases the hardware doesn't have the needed resources to support newer software. I have used some EXFO test equipment that ran XP and even with it, it was painfully slow. In that case, the only recourse is to buy newer equipment.

JKnott

@NogBadTheBad said in pfSense as an ICS / SCADA environment internal segmentation firewall:

https://www.theregister.co.uk/2018/11/09/royal_navy_old_os_at_sea/

Ah yes, Windows Moron Edition.

bmeeks

I had to deal with even a Windows NT 4.0 system, but at least it was a completely standalone single machine located in a high security area.

You can make these machines secure by locating them behind high security physical barriers, locking down USB access with key-lock physical blocking devices inserted into the ports, and by putting them on data-diode isolated networks so two-way communcations with the "outside" is impossible. You would only use the data-diode option if it was necessary for the older machine to send its data to some other network. If everything was local, then most times no network is even connected and the RJ45 port is also physically blocked closed with a key-lock plug.