Two firewalls on the same network
-
This post is more a curiosity than the problem. Please don't waste my post with comments like 'why did you do this' or 'don't you know', etc.
There's no value what so ever with those kinds of comments. Don't respond if that's all you can offer.I gladly point out this is above my knowledge level at this time which is why I'm posting to try and better understand what I'm seeing.
I have a network where there are two firewalls on the same subnet.
The main firewall is on 10.0.0.1/24. It has the DHCP server and is the gateway for all devices on the LAN. No problems with this.A colleague added a second firewall at 10.0.0.2/24 because it was the only way he could get additional bandwidth into the same LAN as the first firewall is limited to two interfaces and its Internet connection is maxed out.
On the LAN, some devices had their gateway changed to the 10.0.0.2 firewall and their network routes changed to reflect .2 being their gw.
Those devices can still communicate with other devices on the same LAN and vise versa which is what was needed in the moment.
Another firewall will be added with more interfaces to take care of the problem on the long term basis.
The idea worked perfectly just to get something up and running in minutes using a separate Internet connection to the same LAN.
Since then, I started looking at this, curious about the setup but found something I'm not understanding.
On this second firewall, he's got rules that allow incoming connections to some devices. One of those is a port 80 device he's using to do some testing.
Being at another location over the Internet, I checked and can reach that web server. it responds back to the correct gw .2, so is fully interactive.
I decided to run nmap to the same connection but I get 'host seems down'. From the same remote server, I ran curl to the same connection and it responds normally.
That makes no sense to me. The device is able to communicate perfectly over port 80 but running an nmap to the same IP/port shows no response.
# curl http://a.b.c.d/ <!DOCTYPE html> <html> <head> ***snip*** # nmap -p 80 a.b.c.d Starting Nmap 7.70 ( https://nmap.org ) at 2023-04-09 06:25 MST Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn Nmap done: 1 IP address (0 hosts up) scanned in 3.14 secondsEven if it's not the right way to set up this second firewall, it worked in a pinch for what was needed. However, I'm not understanding why nmap would give this result when curl can reach the web server.
-
@lewis
Some programs start with a "ping/icmp" - If unsuccesfull they skip the rest pf the tests against that "host"But you usually can tell them to drop the initial "ping"
-
Yup that^. nmap does check any ports unless the host responds to ping unless you tell it not to do that. So like:
nmap -Pn -p80 192.168.1.1 -
Oh cool, learned something new. I thought this might be something on the firewall.
The firewall has the ports allowed of course but the server didn't have ICMP in it's iptables.
BTW, anyone have any thoughts on this setup? Could they keep it as it is? Seems to work fine. Just never heard of a setup like this.
-
Two routers/gateways in one subnet creates a high possibility of asymmetric routes. If you do see issues that will almost certainly be the cause.
-
@stephenw10 said in Two firewalls on the same network:
Two routers/gateways in one subnet creates a high possibility of asymmetric routes.
Since you mentioned it, I read up a little about it and yes, that could be a problem.
Asymmetric routing happens when traffic takes different paths in opposite directions, which can cause issues such as packet loss, delays, and inconsistent network behavior. By separating the LAN network into two subnets and configuring routing and firewall rules to direct traffic correctly, one can ensure that traffic flows symmetrically between the two subnets and through the appropriate firewall.
So... that's kind of what the plan was.
Configure each firewall to handle a different subnet of the same network, and then configure routing rules to ensure that traffic is routed correctly between the subnets.
For example, assign the 10.0.0.0/24 subnet to one pfSense firewall and the 10.0.1.0/24 subnet to the other pfSense firewall. Both subnets are still part of the same network (10.0.0.0/16), so all LAN devices can still communicate with each other.
To ensure that traffic is routed correctly between the subnets, I could configure static routes on each pfSense firewall to direct traffic to the other firewall for the appropriate subnet.
On the firewall handling the 10.0.0.0/24 subnet, I could configure a static route for the 10.0.1.0/24 subnet to be directed to the other pfSense firewall. On the other firewall handling the 10.0.1.0/24 subnet, have a static route for the 10.0.0.0/24 subnet to be directed to the other pfSense firewall.
Then configure firewall rules on each firewall to control access between the subnets and to the Internet as usual.
For example, rules to allow traffic from the 10.0.0.0/24 subnet to the Internet through one firewall, and traffic from the 10.0.1.0/24 subnet to the Internet through the other firewall.
This setup can allow you to utilize the bandwidth of both pfSense firewalls while still maintaining communication between all LAN devices.
I wasn't thinking about adding rules to each firewall to take care of traffic for the other so maybe that's how one could avoid the problem.
-
@lewis said in Two firewalls on the same network:
For example, assign the 10.0.0.0/24 subnet to one pfSense firewall and the 10.0.1.0/24 subnet to the other pfSense firewall. Both subnets are still part of the same network (10.0.0.0/16), so all LAN devices can still communicate with each other.
Doing that is more likely to introduce problems IMO. If anything in that network segment was set to a /16 subnet mask which would enable them to send traffic directly that would cause asymmetry for anything that isn't using /16. You need to have the same subnet mask set for everything in the segment.
If you need to have two routers for the two WAN connections you should have them on different network segments with a transport subnet between them. That way all traffic is routed and there cannot be asymmetry.
-
@lewis said in Two firewalls on the same network:
only way he could get additional bandwidth into the same LAN as the first firewall is limited to two interfaces and its Internet connection is maxed out.
Alternatively, upgrade the firewall hardware? What is the bandwidth available, what is it maxing out at, and why?
-
I imagine it's 2x 1G drops at a DC.
Yeah the best solution here would be to upgrade one of the firewalls to have enough interfaces to route all traffic through it.
-
@stephenw10 said in Two firewalls on the same network:
2x 1G drops
I realized that at some point after posting. :) Or maybe two ISP connections. Either way pfSense can load balance.
-
Doing that is more likely to introduce problems IMO. If anything in that network segment was set to a /16 subnet mask which would enable them to send traffic directly that would cause asymmetry for anything that isn't using /16. You need to have the same subnet mask set for everything in the segment.
In this scenario, there is only one single LAN, no additional interfaces to add to a firewall, only another machine could be installed which has also two interfaces. Also, would like to avoid using VLAN as that would separate the devices which is not the goal.
If I understand correctly, using different subnet masks in the same network segment is what could cause asymmetric routing.
If devices on the same network segment have different subnet masks, some devices may be able to send traffic directly to other devices without going through the firewall, leading to inconsistent routing and potential security issues.
I would have thought the other way, that with their own masks, they could not communicate with any other device with another mask.
Then to make this work, all devices on the same network segment should have the same subnet mask, to ensure that traffic is routed symmetrically through the firewalls.
As I said, if I understand the comments, the next thing that's not clear to me in how the firewalls would be configured.
The /16 on the clients would allow them to communicate together and with their respective firewall for their GW and Internet access.
Would the firewalls be using what I mentioned above for their own networks 10.0.0.0/24 and 10.0.1.0/24 but still with a /24 or a /16?
-
@lewis said in Two firewalls on the same network:
with their own masks, they could not communicate with any other device with another mask
They can't but someone could enter the wrong mask. If you set them up with two /24 masks then in theory you should be fine. If someone uses a /16 on one, traffic from that device will not go through the router to get to the other subnet, because the device thinks it is already in that subnet.
Also DHCP can only be set up on one router. IPv6 is another layer of complexity.
-
Yes, only one would have DHCP. The devices that would need access via the second firewall would be manually configured.
What then would the LAN masks be on the firewalls?
Say one is 10.0.0.0 and the new one is 10.0.1.0. -
@lewis Ok let me back up and reread....if you have two routers it will work for Internet access if they are both in 10.0.0.0/24, you just need to set some PCs with a different gateway.
If you put them in separate subnets 10.0.0.0/24 and 10.0.1.0/24 then you'll need to make sure PCs have the correct /24 mask.
Let's say at some point in the future you add another interface to both routers. PC10 may connect to 10.0.0.1 to get to PC300 on the third network, while the reply from PC300 comes back through 10.0.0.2, and that's the asymmetric issue.
-
This is the same installation we looked at some time ago yes? There's quite a bit of history there!
You can put two subnets on the same layer 2 segment and it can work fine as long as you're careful. The reasons to not do that are that it's all too easy to get it wrong and that there is no separation between the subnets. The latter is usually a security issue but I'm guessing it isn't for you.
You say you don't want to use a VLAN but doing so would make things a lot clearer and safer. And without a VLAN but devices in separate subnets that traffic still has to go through both routers to reach hosts in the other subnet.
Steve
-
in some rarely special cases it might be making sense to
go this way, but only in some;- Testing out something Hardware/Software/Service based
- Testing throughput, VPN and other things perhaps
- Lab network for try out and config out things
But let being fair and say it not common what you where setting up! And how much more you set up things in your
own manner, thinking, "love to do it your way",......You will be ending earlier or later in a more or less massively problems glued situation or plain said
the workaround are growing more and more
based on that not common setup.If you have the ability to set up two vlans and give each another subnet like 192..... and 172..... you may be better sorted.
If it is then even your wish, that the devices must be able
"to talk" each to another you may be better going with;- setting up a WiFi bridge between the both networks
- setting up a network cable from port to port from
the pfSense units - setup from both units a port to a switch that they will be able to share data through that communication.
But all in all you may be not running then later, into this or that behaviour or problem and even and even again you set up something like a workaround again and again,
and later no one knows if a problems is occurring,
from what it comes and how to solve it. -
@steveits said in Two firewalls on the same network:
if you have two routers it will work for Internet access if they are both in 10.0.0.0/24, you just need to set some PCs with a different gateway.Right. Just not clear on where the /16 is then because we were talking about using /16 across all LAN devices.
stephenw10
This is the same installation we looked at some time ago yes? There's quite a bit of history there!Great memory but no, this is different. The other was two separate networks needing to communicate together. It was accomplished using routing between the firewalls.
In this case, it's two firewalls on the same network because of some limitations that can't be changed. Have servers but only two interfaces.You say you don't want to use a VLAN but doing so would make things a lot clearer and safer. And without a VLAN but devices in separate subnets that traffic still has to go through both routers to reach hosts in the other subnet.
Being honest, VLAN's make me nervous. A lot of the stuff I've been doing and asking for help on are new things to me. It's easy to set up simple networks and firewalls but I seem to come across a bit more complex (to me) stuff regularly.
Dobby_
You will be ending earlier or later in a more or less massively problems glued situation or plain said the workaround are growing more and more
based on that not common setup.Yes, agreed, patchy things lead to more patches and eventually things break.
Really, I decided to post here looking for thoughts on what I was seeing but it's turned into a lot of useful information that could become a longer term solution than the kludge it is now, even if it's working.
There is no option to add WiFi. Mentioning VLANs again... maybe the second firewall could be just that, the VLAN firewall instead of a full out firewall. However, doesn't that add up to the same thing? No matter if it's a VLAN or a new subnet, isn't that about the same?
-
Really the simplest thing here is probably what you have already. Just set some hosts to use a different gateway within the same subnet. That will work fine as long as you don't accidentally forward traffic through one router to a host that is using the other router as it's gateway. Or otherwise create asymmetry in the route.
-
So keep the 10.0.0.2/24 on the second firewall and just point those servers that need that route to it?
So long as the first firewall never references any of the devices using the second firewall as their GW, then that should prevent problems.
It would be nice to keep it as he set it up, it seems to work ok.
-
If it was misconfigured somewhere I'd expect the issues to be pretty obvious!
Just be aware of the potential asymmetry issues with two routers in one subnet and avoid them if you're making changes there.
The problems we see with networks like that are usually when admins change or are forced to do things they wouldn't usually attempt. If it's not documented it's easy to introduce asymmetry and if you're not expecting that it can be difficult to diagnose.
