pfSense as an ICS / SCADA environment internal segmentation firewall
-
Hello pfSense experts. We would like to protect our industrial control computers and equipment with ICS / SCADA internal segmentation firewall. We use pfSense as our router / perimeter firewall and we would like to try it as ICS / SCADA environment firewall.
Are there manuals or guidelines for this scenario? Can people share their success stories?
-
Perhaps you can start with the requirements. Sayings ICS /Scada doesn't say much. What ports are required etc.
-
Hi JKnott and thank you for your reply.
JKnott> Sayings ICS /Scada doesn't say much.
Indeed, my description is too short. I will try to provide more details below. We invited a 3rd party company to perform network security audit. We have several outdated WinXP computers in our production area. The auditor recommends replace those old computers with fresh Win 10 machines. Some of them are controlling industrial equipment – not easy (or not possible) to upgrade. Auditor told that we can put those old computers in specially protected network segment when we have no other option. We need internal segmentation firewall in this scenario.
JKnott> What ports are required etc.
All our computers (including those outdated ones) are connected using standard Cat 5 / RJ45 / gigabit LAN. I think even two port pfSense box is good for start.
Auditor mentioned Stormshield SNi40 as an example of ICS firewall, and we also got an offer mentioning Fortinet Fortigate 30D from another company.
All the offers insist that classic perimeter firewall will require additional settings to protect ICS (Industrial Control Systems) area. I am in contact with the auditor trying to get the list of most important points / settings for internal segmentation firewall. We like our pfSense router and I wonder if can use it also as an ICS firewall.
-
Well, pfSense is certainly a capable firewall, but you still have to determine requirements. For example, you mentioned a protected network segment. Do those devices need any access beyond that network? If not, then don't connect it to anything else. Is there custom software involved that might use non-standard ports? Until you can answer that sort of question, we can't give an answer.
BTW, I recently saw Windows 98 being used in an office, but I don't think it was connected to the Internet.
-
If all you want is a downstream firewall then sure pfsense can do that without any issues.. But curious what this specifically means
that classic perimeter firewall will require additional settings to protect ICS
Did they just mean if your ICS network is hanging off the edge router/firewall that you would need to make sure it can not talk to other network segments on the same router/firewall - or that your other segments can only create specific unsolicited traffic inbound into the ics network, etc.
While pfsense can run IPS as well, maybe they are looking for something specific?
-
The nature of the ICS/SCADA network is of some importance here. For example, if simply a factory that is making widgets using automated machinery, that represents one level of security and something as simple as having all of those devices on a separate IP subnet isolated behind a firewall that prohibits all or nearly all unsolicited inbound connections is OK.
On the other hand, if that ICS/SCADA network is say a power generation station or chemical processing plant, then you would want much stricter isolation even up to using data diodes which absolutely prohibit inbound communications of any kind (via hardware). Here is a link to one company's data diode products: https://owlcyberdefense.com/. There are several other vendors out there as well. I just happen to be familiar with this vendor's products. Google "data diode" for more information about the technology in general. Data diodes are outstanding devices for totally isolating a network from inbound traffic while still allowing devices behind the data diode to send out status messages. So you could stream out data for monitoring the processes behind the data diode, but you absolutely cannot send anything back (including even protocol handshaking). Data diodes allow you to have your cake and eat it, too, because you have the isolation of a true air-gap (when considering inbound traffic), but yet you can still send out data from the isolated networks.
ICS/SCADA networks can be particularly vulnerable to modern cyber exploits because they are almost never designed with security in mind. There is usually no concept of SSL. Even SSH is not common and Telnet is still widely available (yep ... ). To protect these inherently trusting networks and devices you need robust cyber practices. It is very important to severely limit communications into and out of such networks. That's where data diodes can help. For ICS/SCADA networks, a firewall is good, but not really all you need. Consider how severely such a trusting network could be instantly compromised if some WinXP machine back there made an outbound connection to an external web site and downloaded some wormable exploit. Data diodes make this physically impossible.
It is helpful when implementing cyber protections for ICS/SCADA systems to break your network up into functional layers. Typical layers would be Control, Monitoring, and Business. The Control Network layer is where all devices that have actual process control functions reside. This network layer gets maximum protection and for sure would be isolated behind a data diode. The Monitoring Network layer contains systems that only provide information about processes but can't control anything. This network layer may or may not be isolated from the Business Layer (but it is a good practice to at least provide a firewall segment between the two). The Business Layer is typically the company's internal business LAN.
-
johnpoz, bmeeks, thanks for your inputs.
I still do not have full list of important ICS firewall differences (from regular firewall). I remember few points discussed:
- no access to the Internet or to email should be allowed from ICS networks
- ICS networks should be defended from other plant networks, especially those with Internet access
- limited or on-demand (normally blocked) remote access
Those requirements are conflicting with our typical access and maintenance scenarios...
-
@bmeeks said in pfSense as an ICS / SCADA environment internal segmentation firewall:
There is usually no concept of SSL. Even SSH is not common and Telnet is still widely available (yep ... ).
Of course the reason for that is SCADA predates the public Internet and was originally run over dedicated networks. About 11 years ago, I was in this place updating some telecommunications equipment. They were still using dial up modems to connect the various sites to their control centre. Yep, the Ontario power network was controlled with dial up modems. When I was in there, about 3 years ago, the equipment I put in was gone and they'd moved on to a more modern system over fibre.
-
@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:
johnpoz, bmeeks, thanks for your inputs.
I still do not have full list of important ICS firewall differences (from regular firewall). I remember few points discussed:
- no access to the Internet or to email should be allowed from ICS networks
- ICS networks should be defended from other plant networks, especially those with Internet access
- limited or on-demand (normally blocked) remote access
Those requirements are conflicting with our typical access and maintenance scenarios...
The no internet access is easy enough. You just don't connect it to anything. Critical networks are often completely separate from those used for other purposes. What triggers the "on demand" access?
-
@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:
johnpoz, bmeeks, thanks for your inputs.
I still do not have full list of important ICS firewall differences (from regular firewall). I remember few points discussed:
- no access to the Internet or to email should be allowed from ICS networks
This is to protect those vulnerable components from malware infestation. The primary exploit vectors these days are through browser or browser plugin flaws (Adobe Flash being one example) and email attachments or hyperlinks. If vulnerable components and networks can't access the Internet or email, those attack vectors are mitigated.
- ICS networks should be defended from other plant networks, especially those with Internet access
This is another layer of protection from the threats I described above. Trusted networks can act as a "carrier" of a virus that then infects your sensitive networks. You want to severely limit "trust" in ICS networks.
- limited or on-demand (normally blocked) remote access
Remote access equals bad cyber practice! In the industry I retired from recently, remote access to control systems was absolutely prohibited by government regulations. You got into serious trouble for violating that one! We used those data diodes to enforce that rule. If I needed to make any change to a control network, my butt went into the physical plant, through all the physical security and stood in front of the actual device (or least a terminal within a physical security boundary that was attached to the control network). And speaking of physical security, you need to have absolute physical access control to all parts of your Control Layer network. That means locked rooms where Control Layer components are located. It also means some type of alarm system for locked cabinets that might have to be located outside your typical physical security boundary (think a device that might have to be outside the fence).
Those requirements are conflicting with our typical access and maintenance scenarios...
I understand it can be a pain in the butt, but remote access to critical Control Layer components is just never a good idea. It is very hard to secure at a very high confidence level. You just can never eliminate human error.
One thing I did not see you mention, but that is a HUGE concern, is what are called portable media and mobile devices. This area includes USB memory sticks, flash memory cards, CD/DVD discs and even cell phones being charged via USB cables. You need very tight controls on these devices around Control and even Monitoring Layer networks. It would be so easy to compromise a system via a USB stick or even some employee plugging in their cell phone's USB charger cable into a vulnerable PC. In fact, USB media is exactly how the infamous Stuxnet virus infiltrated the Iranian nuclear processing facility and then went on to damage multiple centrifuge machines there. A worker was used to unknowingly carry the virus into the secure facility on a USB stick. When he plugged the USB stick into a computer on the secure and isolated network, that allowed it to spread across the sensitive Control Layer network and do its work.
-
I'll say one last thing to maybe more directly answer one of your questions about the differences between a typical firewall and an ICS firewall.
Functionally, there is no real difference. One firewall is fundamentally the same as another when speaking of today's stateful inspection technology. It's mostly a difference of features, support and of course cost. Cost runs the gamut from free to thousands and thousands of dollars for the hardware and software support contracts.
I love pfSense, and it is a very secure firewall out of the box. But it has one shortcoming when used with ICS networks. It wants to be able to talk to the Internet. The "home" screen, immediately upon opening, attempts to query the pfSense/Netgate servers to check for new firmware updates. The package and update system of pfSense is also heavily dependent on Internet connectivity. You can, with much trouble, create a sort of offline package server, but it takes quite a bit of skill to pull this off.
For ICS networks, especially those most sensitive Control Layer networks, you want a firewall that is happy being 100% isolated from the Internet. One that allows you to update it with a single file by uploading the file to the firewall from a local management network connection. So when looking for your ICS/SCADA firewall, ask the vendor about their support for 100% isolated setups where the firewall has absolutely no Internet access. Can the firewall work in that environment without issue, and does the vendor provide an offline update process that is workable for you? There are firewalls out there that fit this bill, but unfortunately none of the ones I'm familiar with are free.
-
bmeeks, your post inspired me to read more about data diodes and unidirectional networks. Also found few articles explaining "why firewalls are not recommended for securing SCADA systems"...
Luckily our site is not a power plant network and remote access to control systems is not prohibited by government regulations in our region... :)
It is just a production plant with few assembly machines. If disaster strikes we can restore everything quite quickly, our estimation is 8-48 hrs. And management is happy with this estimation. So I hope that reasonable / descent security level will help us to improve our security score without going to very expensive solutions and paranoid security mode. :)
-
@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:
bmeeks, your post inspired me to read more about data diodes and unidirectional networks. Also found few articles explaining "why firewalls are not recommended for securing SCADA systems"...
Luckily our site is not a power plant network and remote access to control systems is not prohibited by government regulations in our region... :)
It is just a production plant with few assembly machines. If disaster strikes we can restore everything quite quickly, our estimation is 8-48 hrs. And management is happy with this estimation. So I hope that reasonable / descent security level will help us to improve our security score without going to very expensive solutions and paranoid security mode. :)
That's why my first post mentioned first determining the "value" of the network you are protecting. It's one thing if the loss is just monetary, but quite another if loss or sabotage of the network could result in injury or death to people. For a typical factory scenario, I would also say a data diode and all of the other physical security expenses I mentioned are overkill. However, if you are protecting a nuclear power station and its reactor, or a chemical processing facility or say the national electric grid, then more robust controls are definitely called for.
However, even with just a factory network, you will benefit from tight portable media and mobile device control. It would be a bad thing for the company to lose thousands of dollars of production because Joe brought in some pictures of his grandkids on a compromised USB stick and plugged it into a PC in the plant control room to show his co-workers.
-
@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:
We have several outdated WinXP computers in our production area
If this was the case in any of the scenarios you mentioned, National Elec grid, Nuc planet - I would seriously be like WTF People!!!
-
@johnpoz said in pfSense as an ICS / SCADA environment internal segmentation firewall:
@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:
We have several outdated WinXP computers in our production area
If this was the case in any of the scenarios you mentioned, National Elec grid, Nuc planet - I would seriously be like WTF People!!!
WTF is an appropriate response, but that old nemesis money comes into play. You have vendors that provide highly specialized components (sometime custom designed for just one site) and they want many millions of dollars to "upgrade" their component for you and to certify their old software on new operating systems. It becomes, unfortunately, a money decision made on a risk vs cost to mitigate scale. Sometimes totally isolating the network and associated devices is the most cost effective approach for a short-term solution. Longer term you plan on replacing that component or system. However, in the case of power plants, consider the impact of spending millions and millions of dollars to upgrade systems on consumers' power bills. They will yell and scream and the utility rate regulators are stingy with granting rate increases.
-
@johnpoz said in pfSense as an ICS / SCADA environment internal segmentation firewall:
If this was the case in any of the scenarios you mentioned, National Elec grid, Nuc planet - I would seriously be like WTF People!!!
https://www.theregister.co.uk/2018/11/09/royal_navy_old_os_at_sea/
-
Yeah that really is a WTF... but if that is true
"It’s all standalone, not connected to the outside world,”That is something
-
@johnpoz said in pfSense as an ICS / SCADA environment internal segmentation firewall:
I would seriously be like WTF People!!!
Unfortunately, there is a lot of gear around where the manufacturer can't or won't update it. In some cases the hardware doesn't have the needed resources to support newer software. I have used some EXFO test equipment that ran XP and even with it, it was painfully slow. In that case, the only recourse is to buy newer equipment.
-
@NogBadTheBad said in pfSense as an ICS / SCADA environment internal segmentation firewall:
https://www.theregister.co.uk/2018/11/09/royal_navy_old_os_at_sea/
Ah yes, Windows Moron Edition.
-
I had to deal with even a Windows NT 4.0 system, but at least it was a completely standalone single machine located in a high security area.
You can make these machines secure by locating them behind high security physical barriers, locking down USB access with key-lock physical blocking devices inserted into the ports, and by putting them on data-diode isolated networks so two-way communcations with the "outside" is impossible. You would only use the data-diode option if it was necessary for the older machine to send its data to some other network. If everything was local, then most times no network is even connected and the RJ45 port is also physically blocked closed with a key-lock plug.
-
@bmeeks said in pfSense as an ICS / SCADA environment internal segmentation firewall:
I had to deal with even a Windows NT 4.0 system, but at least it was a completely standalone single machine located in a high security area.
IIRC, Windows NT only met Orange Book C2 security by having no connections to anything and floppies/USB sticks blocked.
-
@JKnott said in pfSense as an ICS / SCADA environment internal segmentation firewall:
@bmeeks said in pfSense as an ICS / SCADA environment internal segmentation firewall:
I had to deal with even a Windows NT 4.0 system, but at least it was a completely standalone single machine located in a high security area.
IIRC, Windows NT only met Orange Book C2 security by having no connections to anything and floppies/USB sticks blocked.
Luckily for me that NT machine was in a display-only role that simply received data from a remote sensor over a proprietary analog interface and displayed the output for trending. It was for local use only and had no external connection.
-
@johnpoz said in pfSense as an ICS / SCADA environment internal segmentation firewall:
"It’s all standalone, not connected to the outside world,”
That is somethingSomehow with the whole twist this thread has been taking (and the horror scenarios...) while reading "isolated from the network" I had to remember something I read years before about "tempest shielding" like some sort of "bunker" isolation. Reminds me of that.
-
Thank you all guys for all the info and motivating scary stories. :)
Back to my original question: Are there manuals or guidelines for turning pfSense into an ICS / SCADA firewall?
-
@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:
ICS / SCADA firewall?
You have yet to call out the specifics of this.. I don't even think its a thing to be honest.. There are firewalls, and then there are firewalls.. There isn't X firewall, and then B firewall for what devices you put behind them.
You can put you ISC network behind a firewall.. You can then allow or block whatever traffic you want or don't want based on all kinds criteria. You can run IPS on the traffic you allow if you so desire. What else do you need?
If I put my plex behind a firewall, I could call it my plex firewall if I wanted ;)
-
@johnpoz is correct. There is no difference between an ICS/SCADA firewall and any other typical firewall in terms of security or performance. Of course there are differences in appearance, perhaps differences in features and certainly differences in cost. An ICS/SCADA firewall is not more secure than any other firewall assuming both are configured with the proper firewall rules.
Where there can be differences are in these two areas:
-
Hardware touted as an ICS/SCADA firewall is more likely to be rated for use in harsher environmental conditions (higher temps, maybe higher humidity, etc.).
-
Going back to one of my earlier posts, a true ICS/SCADA firewall is more likely to be able to operate isolated from the Internet. That means it does not need to constantly check a home web site for firmware updates and it is more likely to support an offline update process (meaning you can update the firewall's software without needing an Internet connection directly from the firewall).
In terms of feature set differences, a vendor selling you so-called dedicated ICS/SCADA firewalls may well offer some pre-packaged firewall rules as features. Say something like having some canned Modbus rules ready for you, or perhaps rules for other standard industrial control protocols. Of course when it comes down to it, ports are ports and UDP and TCP are the same everywhere, so you could replicate any of these pre-packaged rules on your own. But you might have to do more research to find the proper ports and protocols, for example.
A vendor selling you what they term an ICS/SCADA firewall is likely to be more attuned to industrial control terminology and technology, so their after-the-sale tech support may be superior to what you might get from say a more generic perimeter firewall vendor should you have questions related to industrial control technologies.
So to sum up what I'm saying, in terms of security there is no inherent advantage of one firewall over another. It's the features one brand provides over another that is most likely to matter to you. And of course don't discount the operating environment. Typical industrial locations are much more harsh than a nice climate-controlled server room in a data center.
In terms of making pfSense an ICS/SCADA firewall -- sure you can do that. But there may be some headaches. How big the headaches are depends on whether or not pfSense will have Internet access. If the firewall has unrestricted Internet access, then it can be a fine ICS/SCADA firewall. Just pay attention to any physical environment issues and buy hardware appropriate for the environment. If you need to put pfSense into a situation where it will not have direct Internet access, then you start to have the headaches. When the home page opens after login, it will attempt to contact the Netgate servers to check for updates. The GUI will be very slow during this time if there is no Internet access. Also, lack of direct Internet access will be a real burden if you want to load optional packages such as Snort or Suricata. Those all expect Internet connectivity in order to function.
-
-
@bmeeks said in pfSense as an ICS / SCADA environment internal segmentation firewall:
Hardware touted as an ICS/SCADA firewall is more likely to be rated for use in harsher environmental conditions (higher temps, maybe higher humidity, etc.).
Going back to one of my earlier posts, a true ICS/SCADA firewall is more likely to be able to operate isolated from the Internet. That means it does not need to constantly check a home web site for firmware updates and it is more likely to support an offline update process (meaning you can update the firewall's software without needing an Internet connection directly from the firewall).
In terms of feature set differences, a vendor selling you so-called dedicated ICS/SCADA firewalls may well offer some pre-packaged firewall rules as features. Say something like having some canned Modbus rules ready for you, or perhaps rules for other standard industrial control protocols. Of course when it comes down to it, ports are ports and UDP and TCP are the same everywhere, so you could replicate any of these pre-packaged rules on your own. But you might have to do more research to find the proper ports and protocols, for example.
A vendor selling you what they term an ICS/SCADA firewall is likely to be more attuned to industrial control terminology and technology, so their after-the-sale tech support may be superior to what you might get from say a more generic perimeter firewall vendor should you have questions related to industrial control technologies.
So to sum up what I'm saying, in terms of security there is no inherent advantage of one firewall over another. It's the features one brand provides over another that is most likely to matter to you. And of course don't discount the operating environment. Typical industrial locations are much more harsh than a nice climate-controlled server room in a data center.If your running Snort it may be an idea to enable the SCADA Preprocessors, they're disabled by default.
-
@ka3ax said in pfSense as an ICS / SCADA environment internal segmentation firewall:
Thank you all guys for all the info and motivating scary stories. :)
Back to my original question: Are there manuals or guidelines for turning pfSense into an ICS / SCADA firewall?
Since I did not see a reply on your question or that this topic is closed, allow me to pitch in from my perspective.
Let me elaborate first: I have some novice experience with PfSense but not in an ICS environment. However I do know a thing or two about ICS/SCADA environment. Sometimes referred to as Operational Technology (OT).
The most important question we need to answer is if PfSense is able to handle industrial protocols like Modbus/TCP (port 502), Profinet (ports 34962 - 34964), etc.
For those who are not aware of these protocols: these are used to communicate between i.e. a Human Machine Interface (HMI) and a Programmable Logic Controller (PLC) and are widely used, from water treatment plants, chemical factories to opening and closing a bridge. PLCs are also used for changing the traffic lights from red to orange/green. Tunnels (for car traffic) are most of the time packed with OT devices to control things like air flow, alarms for fire, vehicles that are to high, etc. So we come across these systems more then we realize in our daily life. Every manufactured good in your house, from plastic storage boxes, electrical wires to processed foods like coffee beans and bags of chips, have been made and gone through quality checks using OT environments.
As was stated earlier, these systems can sometimes be very old (think 1970's, 1980's) and originally communicated using serial connections (COM ports like RS-323 and RS-485). When TCP/IP based networking was introduced into manufacturing (OT) environments these protocols have been adjusted (encapsulated with TCP/IP markers around the original traffic) to allow them over rout-able networks.
However, the issue is that these protocols, or at least most of then, where not designed to be handled by IT network based routing. So I expect that when using PfSense you might run into interesting issues. Especially when you add deep packet inspecting in the mix (although I do not think PfSense is capable of doing that)
Now let's become pragmatic. If you have the ability to create a test environment to see if PfSense is a viable option, I would do that. You can set up a (virtual) test environment, run some tests and check the logs for specific issues. As you can see the rout-able protocols have port numbers that you can allow, block, drop, etc. So, in general, I think that yes, the firewalling function would support industrial protocols as it is also port based. You should be able to specify which PLC is allowed to connect to which PLC and/or HMI based on IP address or MAC address as you would in an IT environment. You can also set access to HMI and PLC for i.e. vendor support but keep in mind that using a stepping stone server in a DMZ would always be preferred over a regular connection using a VPN into the business network.
Also invest time to look into latency that the PfSense will bring. Especially when real time information in the SCADA systems is important. So discuss with the business how much latency between the levels would be acceptable for the manufacturing process if you haven't done so already of course.
On more pitfall would be a dual homed system. A system that circumvents the firewall by having two network connections, one going to Level 2 or 3 networks and one going down to the Level 1 network. (for more on ICS networking look up the purdue model)
I know it has almost been three years but I hope I have been able to help you out.
-
I still think this is an amazing hack (Now that STUXnet isn't public)
https://blog.checkpoint.com/2018/08/12/faxploit-hp-printer-fax-exploit/The attack vector is "sooo elegant"..... (Scary)
Who would think a "Fax" could do a Network exploit ....
Would even circumvent the "Diode" ... If it was the "Closed net printer".Took them a loong time , but where there's will (fame) , power or $$$ ...
/Bingo