Using pfsense with multiple WANs
-
There are a lot of articles out there but I can't find one that is simply pf to pf. Most are pf to openwrt and others and most are with IPSEC too. I can't find enough information to know if I need to set this up on the pf01 or pf02.
Based on that page, you create a GRE tunnel on one end.
This is what I've done so far.I have a VIP of 10.100.0.3 on pf02.
I have DCLAN set to 10.100.0.1/24 on pf01.
I created the tunnel as per the images and enabled the interface.
From the CLI of each pf, I cannot ping the other.
pf02: ping 10.100.0.1 or 2, nothing.
pf01: ping 10.100.0.3, nothing.I assume this is because I have to set up rules now.
-
The remote address is the other end of the tunnel outside the tunnel so it that is on pf01 it should be the pf02 VIP, 10.100.0.3.
The local and remote tunnel addresses are the IPs inside the GRE tunnel. So that should be a new subnet for example 10.102.0.1 and 10.102.0.2 using /30 subnet.Create the tunnel on both pfSense instances and assign the generated interfaces. You can then forward or route traffic across it.
Steve
-
pf02 is the old firewall with 10.0.0.1/24 on it.
pf01 is to become the main firewall with DCLAN and 10.0.0.1/24 LAN.Like this on pf01?
I don't need GRE set up on pf02?
-
Yes, you need to setup GRE at both ends so pf02 also.
I would try to use a numbering scheme that is logical because it much easier to9 avoid mistakes that way.
So to start I would probably change the IP pf02 is using in the DCLAN to 10.100.0.2. Use 10.100.0.3 for pf03 etc.
Otherwise, yes that looks good. Youy cna use 10.103.0.0/30 for the tunnel to pf03 etc.
Steve
-
You have no idea how frustrated I am having to keep asking questions and how much I appreciate that you've helped me all this time. I really should have opened a new question for the GRE.
Something about this is either confusing or I've already got it right.
First, we want to create a tunnel between two firewalls using GRE. The GRE tunnel needs to communicate between the two firewalls so IPs pf01 10.100.0.1/24 (24 since I'll have other pf on the same tunnel/network in different subnets) and pf02 10.0.0.1/24.
The tunnel then has its own internal IPs so pf01 10.102.0.1/30 and pf02 10.102.0.2/30 to create one net.
Adding additional pfxx will be simpler since there are no hosts behind that network, we're just using pf as a means to get IP traffic from one network firewall to another.
So if I add pf03, then pf03 can have a LAN 10.3.0.1/24 connecting to DCLAN 10.100.0.1/24. Then a GRE tunnel, pf03 10.103.0.2/30 and pf02 will have 10.103.0.1/30.
IF I have this, then the following which is how I have it all set up should be ok.
IF this is ok, then the only thing I'm missing is one or more rules.
IF this is correct, I just need to start pointing hosts behind pf02 from 10.0.0.x/24 to 10.102.0.1/30.This is a bit simple and mind boggling at the same time because it's all text. Seeing one done would really simplify everything for me and anyone else wanting to understand this. I suspect I'm still screwing something up because from pf02 cli, I cannot ping 10.102.0.1 and from pf01 cli, I cannot ping 10.102.0.2 but it could be because I don't have any rule/s yet. Truly not sure what kind of rule I'll need anyhow.
-
The parent interface for the GRE tunnel on pf02 must be the VIP
10.100.0.2. The LAN interface there as you have it set there is not in the same subnet.On pf03 the LAN (is that's what's on the DCLAN VLAN) should be 10.100.0.3. Or you can add that as a VIP on top of some other subnet like pf02.
The GRE tunnel there will then be between pf03 and pf01.Otherwise that looks right. Can you ping across the tunnels?
Steve
-
Sorry, I only brought up pf3 in terms of your comment about trying to use a numbering scheme. Right now, there are only pf01 and pf02 at the moment to keep things simpler.
On pf02, I have a VIP of 10.100.0.2/30 on the LAN.
Do I need a VIP on pf01 also? I thought the DCLAN. I only have the DCLAN static set which is 10.100.0.1/24.I'm probably not getting this yet but it seems that the DCLAN interface just gets the one static IP.
However, GRE tunnels get their own end to end IPs and then another set for communications inside the tunnel.
Could I just hire you for an hour to get this done over a phone call?
It would save us 100 messages :).I was hoping it could be something useful for anyone else that is wanting to understand how to do this. There's a lot of information out there but it's pretty vague when you've never done it before.
-
I forgot to add...
I can ping from pf01 to 10.100.0.2
I can ping from pf02 to 10.100.0.1 -
We have paid support if you need it. That pretty much rules out paying me here.
You're correct, you don't need a VIP on pf01 because it's DCLAN interface is already in the subnet directly.
If you've assigned the GRE interfaces at each end they should show as gateways and will be trying to ping each other. If you have added firewall rules on those pings should succeed and the gateways will show as on-line.
If that's the case you can try forwarding something from pf02 to the GRE tunnel IP at pf01
10.102.0.1. And from there to a server behind it.Steve
-
If there's an hourly option available, I'd do that. I'm sure this could be done in 20 minutes so don't really need an ongoing plan. Besides, I send all kinds of business this way since I cannot speak highly enough about pfsense to all network folks I talk with.
In terms of what I have set up, it's as the images show. I posted another comment about being able to ping, apparently between tunnel interfaces now.
I didn't need a rule to allow the pings however so I assume it's automatically allowed because of the GRE at both ends.
Based on what you said, I needed a rule for pings to work so that must mean I've got something messed up.
-
You need to be able to ping 10.102.0.2 from pf01 and the other way, 10.102.0.1 from pf02. So pinging inside the tunnel. You need rules on the GRE interfaces to allow that.
Steve
-
Sounds like I have everything done right but must still be missing something small.
Cannot ping from one to the other.
-
How are you testing?
Check the routing table at each end (Diag > Routes). Make sure a route to the other side exists.
-
Hmm, I'm not sure where that /32 is coming from with pf01 10.102.0.2/32.
I see the mask weirdness but not sure how to handle that since I want pf01 to be reachable by more than one pfxx later.
-
The 10.100.0.2 VIP should be /24 on pf02. The subnet should be the same everywhere.
That would not stop it reaching pfo1 though since it's inside the /30Do you see any packets on the GRE interfaces in Status > Interfaces at either end?
How are you testing link?
-
I changed that to /24.
I've been testing by pinging only.
pf02 GRE Interface (opt1, gre0) Status up IPv4 Address 10.102.0.2 Subnet mask IPv4 255.255.255.252 Gateway IPv4 10.102.0.1 IPv6 Link Local fe80::225:90ff:fe0e:a370%gre0 MTU 1476 In/out packets 0/91731 (0 B/6.42 MiB) In/out packets (pass) 0/91731 (0 B/6.42 MiB) In/out packets (block) 0/33 (0 B/3 KiB) In/out errors 0/0 Collisions 0 pf01 GRE Interface (opt3, gre0) Status up IPv4 Address 10.102.0.1 Subnet mask IPv4 255.255.255.252 Gateway IPv4 10.102.0.2 IPv6 Link Local fe80::ae16:2dff:feb8:8400%gre0 MTU 1476 In/out packets 0/77642 (0 B/6.22 MiB) In/out packets (pass) 0/77642 (0 B/6.22 MiB) In/out packets (block) 0/75 (0 B/6 KiB) In/out errors 0/0 Collisions 0 -
Ok so both sides are sending packets and not seeing anything incoming.
Do you see blocked traffic on either firewall?
GRE uses it's own protocol (gre) so if you have rules that allow only icmp/udp/tcp on the DCLAN connected interfaces that will block it.
If so adds rules to allow gre. Or just allow all IPv4 as a test.Ste ve
-
Looking at the screenshots you posted above the pf02 LAN rules are only allowing traffic from LANnet. And the gre traffic will be coming from the VIP subnet 10.100.0.0/24 so you will need another rule there if you haven't added one since then.
-
I just don't know enough about this to make it work. I'm going to end up breaking pf02 and really messing things up.
-
Try filtering by
gre. You should see the states with traffic both ways:
And if it's passing correctly you should see both those gateway monitoring pings replying:
It looks like there is no incoming state there so either the firewall rules are blocking it, check the firewall logs, or one side has no route for some reason.
Steve
