Using pfsense with multiple WANs
-
In the meantime, I can ping between the two pf01 and pf02 but I've never done a rule where something internal can get to something else internal from one firewall to another.
Meaning, there's no NAT'ing involved here, it's just DCLAN/LAN pf01 to LAN 02. However, there's a default rule that prevents private IP's from coming into the firewall too but I think that's on the WAN side only.
I also notice that pfsense automatically created an anti lockout rule on the DCLAN but not the LAN of pf01.
What kind of crazy rules do I need to set up to allow this to work? So far, nothing I've tried has allowed me to access anything on the pf02 LAN from a windows machine on the LAN side of pf01.
-
Hmm, well I just typed out this big reply about how that won't work but decided to test it to be sure and found something I've never tried before. If you add a gateway you can apply it in the firewall rule and it does change the reply-to tagging which should work for you here. You just have to add gateways for each upstream pfSense instance in pf01 and make sure the rules passing the port forwards are set with that.
You can use separate subnets for each instance but you don't have to. You can just put all the pfSense instances in 10.100.0.0/24 for example.
This is a very unusual config but it does look like it will work.
-
@stephenw10 said in Multi LAN networks to one pfsense:
Hmm, well I just typed out this big reply about how that won't work but decided to test it to be sure and found something I've never tried before. If you add a gateway you can apply it in the firewall rule and it does change the reply-to tagging which should work for you here. You just have to add gateways for each upstream pfSense instance in pf01 and make sure the rules passing the port forwards are set with that.
You can use separate subnets for each instance but you don't have to. You can just put all the pfSense instances in 10.100.0.0/24 for example.
This is a very unusual config but it does look like it will work.
Can you give me an example. This is really getting over my head at this point, probably because I'm just not able to visualize it since I've never done anything like this.
Eventually, once all the servers are moved over to the new network, maybe then it can be simpler. I was wondering if the pf01 could just handle a network and be the gateway for the other pfsense firewalls. Then the other firewalls would simply be assigned an IP in the same network.
For example, pf01 would handle 10.100.0.1/24 and pf02 would have 10.100.0.2, pf03 would have 10.100.0.3.
-
Urgh, nope I spoke too soon. My first instinct was right. You can't apply reply-to tagging like that, the policy routing breaks the port forward.
So back to where we were, traffic coming into the DCLAN interface will only go back via the gateway defined on it.
So to do this you need separate interfaces that can have different gateways.
I think we already discussed VLANs and decided they could not be used?In which case to make this work you will probably need to use tunneling of some sort. So for example a GRE tunnel between each each pfX and pf01 you can forward the traffic across.
Steve
-
There is one free interface on the firewall, another 10gb NIC but there's only one Ethernet cable from the DC to connect to the LAN.
I seem to recall that VLANs cannot exist inside of another VLAN right?
What ever way this could work, it's got to be within the constraints of the DC provided VLAN.
-
You can use QinQ to move VLANs within a VLAN but you would normally expect to have the switches carrying that configured for it. The DC would likely have to be involved.
Otherwise I would suggest using GRE tunnels:
https://docs.netgate.com/pfsense/en/latest/interfaces/gre.htmlSteve
-
I'm looking at the GRE page.
It mentions routing packets between two locations that are not directly connected, which do not require encryption.Since it's my own LAN, I don't need encryption so that's good but the traffic itself, a lot of it would be encrypted such as https. I assume this just means that it would pass that along as is.
I'll give this a try shortly.
-
There are a lot of articles out there but I can't find one that is simply pf to pf. Most are pf to openwrt and others and most are with IPSEC too. I can't find enough information to know if I need to set this up on the pf01 or pf02.
Based on that page, you create a GRE tunnel on one end.
This is what I've done so far.I have a VIP of 10.100.0.3 on pf02.
I have DCLAN set to 10.100.0.1/24 on pf01.
I created the tunnel as per the images and enabled the interface.
From the CLI of each pf, I cannot ping the other.
pf02: ping 10.100.0.1 or 2, nothing.
pf01: ping 10.100.0.3, nothing.I assume this is because I have to set up rules now.
-
The remote address is the other end of the tunnel outside the tunnel so it that is on pf01 it should be the pf02 VIP, 10.100.0.3.
The local and remote tunnel addresses are the IPs inside the GRE tunnel. So that should be a new subnet for example 10.102.0.1 and 10.102.0.2 using /30 subnet.Create the tunnel on both pfSense instances and assign the generated interfaces. You can then forward or route traffic across it.
Steve
-
pf02 is the old firewall with 10.0.0.1/24 on it.
pf01 is to become the main firewall with DCLAN and 10.0.0.1/24 LAN.Like this on pf01?
I don't need GRE set up on pf02?
-
Yes, you need to setup GRE at both ends so pf02 also.
I would try to use a numbering scheme that is logical because it much easier to9 avoid mistakes that way.
So to start I would probably change the IP pf02 is using in the DCLAN to 10.100.0.2. Use 10.100.0.3 for pf03 etc.
Otherwise, yes that looks good. Youy cna use 10.103.0.0/30 for the tunnel to pf03 etc.
Steve
-
You have no idea how frustrated I am having to keep asking questions and how much I appreciate that you've helped me all this time. I really should have opened a new question for the GRE.
Something about this is either confusing or I've already got it right.
First, we want to create a tunnel between two firewalls using GRE. The GRE tunnel needs to communicate between the two firewalls so IPs pf01 10.100.0.1/24 (24 since I'll have other pf on the same tunnel/network in different subnets) and pf02 10.0.0.1/24.
The tunnel then has its own internal IPs so pf01 10.102.0.1/30 and pf02 10.102.0.2/30 to create one net.
Adding additional pfxx will be simpler since there are no hosts behind that network, we're just using pf as a means to get IP traffic from one network firewall to another.
So if I add pf03, then pf03 can have a LAN 10.3.0.1/24 connecting to DCLAN 10.100.0.1/24. Then a GRE tunnel, pf03 10.103.0.2/30 and pf02 will have 10.103.0.1/30.
IF I have this, then the following which is how I have it all set up should be ok.
IF this is ok, then the only thing I'm missing is one or more rules.
IF this is correct, I just need to start pointing hosts behind pf02 from 10.0.0.x/24 to 10.102.0.1/30.This is a bit simple and mind boggling at the same time because it's all text. Seeing one done would really simplify everything for me and anyone else wanting to understand this. I suspect I'm still screwing something up because from pf02 cli, I cannot ping 10.102.0.1 and from pf01 cli, I cannot ping 10.102.0.2 but it could be because I don't have any rule/s yet. Truly not sure what kind of rule I'll need anyhow.
-
The parent interface for the GRE tunnel on pf02 must be the VIP
10.100.0.2. The LAN interface there as you have it set there is not in the same subnet.On pf03 the LAN (is that's what's on the DCLAN VLAN) should be 10.100.0.3. Or you can add that as a VIP on top of some other subnet like pf02.
The GRE tunnel there will then be between pf03 and pf01.Otherwise that looks right. Can you ping across the tunnels?
Steve
-
Sorry, I only brought up pf3 in terms of your comment about trying to use a numbering scheme. Right now, there are only pf01 and pf02 at the moment to keep things simpler.
On pf02, I have a VIP of 10.100.0.2/30 on the LAN.
Do I need a VIP on pf01 also? I thought the DCLAN. I only have the DCLAN static set which is 10.100.0.1/24.I'm probably not getting this yet but it seems that the DCLAN interface just gets the one static IP.
However, GRE tunnels get their own end to end IPs and then another set for communications inside the tunnel.
Could I just hire you for an hour to get this done over a phone call?
It would save us 100 messages :).I was hoping it could be something useful for anyone else that is wanting to understand how to do this. There's a lot of information out there but it's pretty vague when you've never done it before.
-
I forgot to add...
I can ping from pf01 to 10.100.0.2
I can ping from pf02 to 10.100.0.1 -
We have paid support if you need it. That pretty much rules out paying me here.
You're correct, you don't need a VIP on pf01 because it's DCLAN interface is already in the subnet directly.
If you've assigned the GRE interfaces at each end they should show as gateways and will be trying to ping each other. If you have added firewall rules on those pings should succeed and the gateways will show as on-line.
If that's the case you can try forwarding something from pf02 to the GRE tunnel IP at pf01
10.102.0.1. And from there to a server behind it.Steve
-
If there's an hourly option available, I'd do that. I'm sure this could be done in 20 minutes so don't really need an ongoing plan. Besides, I send all kinds of business this way since I cannot speak highly enough about pfsense to all network folks I talk with.
In terms of what I have set up, it's as the images show. I posted another comment about being able to ping, apparently between tunnel interfaces now.
I didn't need a rule to allow the pings however so I assume it's automatically allowed because of the GRE at both ends.
Based on what you said, I needed a rule for pings to work so that must mean I've got something messed up.
-
You need to be able to ping 10.102.0.2 from pf01 and the other way, 10.102.0.1 from pf02. So pinging inside the tunnel. You need rules on the GRE interfaces to allow that.
Steve
-
Sounds like I have everything done right but must still be missing something small.
Cannot ping from one to the other.
-
How are you testing?
Check the routing table at each end (Diag > Routes). Make sure a route to the other side exists.
