pfSense HA LAN Interfaces Only
-
@viragomann Is it normal for the DHCP server to show the actual interface IP of the pf node or should it show the CARP LAN VIP IP (192.168.2.1)?
2.80 is master and 2.81 is slave (currently CARP is on Master) -
@viragomann sorry.. Was thinking how I can provide internet access to my backup node. Just for the purpose to keep the apps up to date as the master. Since there is no active wan when it's in slave mode.
-
@iptvcld said in pfSense HA LAN Interfaces Only:
Is it normal for the DHCP server to show the actual interface IP of the pf node
Yes, the DHCP server is the real interface IP.
You only have to enter others node IP at "Failover peer IP" in the DHCP server settings. This ensures that the server is exclusively running on the present master. -
@iptvcld said in pfSense HA LAN Interfaces Only:
Was thinking how I can provide internet access to my backup node. Just for the purpose to keep the apps up to date as the master. Since there is no active wan when it's in slave mode.
There is a way to go over the masters LAN to the internet:
Add the masters LAN IP as gateway on the secondary an set up a gateway group with the WAN DHCP gw as Tier 1 and the masters LAN as Tier 2. Set the gateway group as default gw.
But first of all disable the sync of routing settings on the masters System > HA page.Now the secondary goes out over the masters LAN interface to the internet, when the WAN gw isn't available.
-
@viragomann fantastic.. I will give this a shot today. So when I swing the wan over from primary to secondary node as a failover, this will restore internet to flow over the backup only?
And does this solution affect the primary wan internet in any negative way? Or will this just provide internet to the secondary node as a client for the purpose to update apps on the router.
Do I need to add additional CARP VIP for this? If master goes down my carp for the Lan interfaces will go master on the backup node. And I assume the secondary will loose access to the Internet right since it was routing from there (which is fine and makes since) at that point I would swing wan over.
-
@iptvcld said in pfSense HA LAN Interfaces Only:
I will give this a shot today. So when I swing the wan over from primary to secondary node as a failover, this will restore internet to flow over the backup only?
Yes, the gateway group cares that the the secondary use primarys LAN only if the WAN DHCP isn't available.
It's only to provide internet to the secondary and doesn't affect any other connections.
I guess, it's a good idea to set a public IP for monitoring (e.g. 8.8.8.8) in the LAN gateway settings. This way the secondary detects the gateway as offline in case you activate the CARP maintenance mode on the master and will switch over to the WAN gw.
-
@viragomann
I guess, it's a good idea to set a public IP for monitoring (e.g. 8.8.8.8) in the LAN gateway settings. = would I do this on the primary node or secondary node?This way the secondary detects the gateway as offline in case you activate the CARP maintenance mode on the master and will switch over to the WAN gw. = if I enable carp mant mode on the master node and I don't swing the wan fiber over to secondary node, it would simply mean no internet until that connection is made? What would happen if don't don't enable carp maint mode on master and just swing over wan to secondary? I guess my carp for Lan will still be master on my primary as Lan is still up.
I'm sorry I am just trying to get a full picture and you have been really great help.
-
@iptvcld said in pfSense HA LAN Interfaces Only:
I guess, it's a good idea to set a public IP for monitoring (e.g. 8.8.8.8) in the LAN gateway settings. = would I do this on the primary node or secondary node?
On the secondary. This gateway should only exist on the secondary at all.
f I enable carp mant mode on the master node and I don't swing the wan fiber over to secondary node, it would simply mean no internet until that connection is made?
Of course you have to connect the WAN cable to the secondary manually in this case to get internet.
What would happen if don't don't enable carp maint mode on master and just swing over wan to secondary?
Nothing. Failover only happens when a CARP interface failing is detected on the primary.
Pulling the WAN cable from the master and connect to the secondary will simple result in loosing internet. The primary will keep staying in master mode. -
@viragomann I hope I did this correctly - I also have a OpenVPN address in the group and not sure what Tier to assign to that. I leave it default Never (not sure if that should be something else as it left it out when i pressed save)
-
Master pf LAN Interface IP: 192.168.2.80
Backup pf LAN Interface IP: 192.168.2.81
LAN CARP = 192.168.2.1Cant get the internet to flow to backup
-
@viragomann hmm. Can't seem to get to working. I have turned off HA static route sync from the master node and then on the backup node I have created a new gateway using LAN pointing to the LAN interface IP of the master node and then created a Gw Group with pppoe wan as tier 1 and my new gw as tier 2 and set this group as default but still no internet. I have also included pics of what I did. Any suggestions?
-
@iptvcld
The gateway and gateway group are looking correctly, but you have to troubleshoot the gateway monitoring problem.
Ensure that pinging 8.8.8.8 from 192.168.2.81 is allowed on the master and that there is an outbound NAT rule present on WAN for that source. -
@viragomann I did a test from my master to ping out to 8.8.8.8 and that works ok.
Also on master; this is what my outbound nat section looks like. I can see under source there is y LAN network 192.168.2.0/24
I am not too sure what else i should be looking at; would really appreciate your help. thank you!
-
@iptvcld
I think, I've forgotten something:
On the secondary you will need to add an outbound NAT rule to LAN for 127.0.0.0/8 and switch into hybrid mode. -
@viragomann Thanks; so on my backup PF i did the below:
This did not work still then i tried this
But this did not work as well.
-
@iptvcld
Keep the first one.Tried to restart the machine?
If it still doesn't take a packet capture on the LAN interface. You should see periodic pings to 8.8.8.8. As well you should see them on the masters LAN and also on the masters WAN if it works as expected.
-
@viragomann I also just noticed since i made this NAT mode change from auto to hybrid and also adding the mapping on my backup PF; after the reboot, it reverted back to auto and got rid of the mapping as my master sync'd over. Should i be making this change on my master then?
-
@iptvcld
Yes, you can add this rule to the master as well. It won't be used there anyway. -
@viragomann i did this on the master now and i saw it synced over to the backup. I rebooted the backup again as well.
I ran a packet capture on the LAN (backup pf) and i see a few of these lines. IP: 76.64.x.x is my wan
10:54:16.598225 IP 76.64.x.x > 8.8.8.8: ICMP echo request, id 13920, seq 321, length 9but GW still shows offline
When i run a ping to 8.8.8.8 using backup, i get this
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=115 time=2.599 ms--- 8.8.8.8 ping statistics ---
3 packets transmitted, 1 packets received, 66.7% packet loss
round-trip min/avg/max/stddev = 2.599/2.599/2.599/0.000 ms -
@iptvcld said in pfSense HA LAN Interfaces Only:
I ran a packet capture on the LAN (backup pf) and i see a few of these lines. IP: 76.64.x.x is my wan
10:54:16.598225 IP 76.64.x.x > 8.8.8.8: ICMP echo request, id 13920, seq 321, length 9Even if the WAN cable is disconnected??
I was assuming that the WAN is offline, so not clear, why it use the WAN IP. -
@viragomann That is correct Sir; i do not have my WAN connected to my backup pf right now
-
@iptvcld
Not clear, what's wrong here.
As first step for troubleshooting you can remove the monitoring IP from the LAN gateway on the backup. Hence it should monitor the masters LAN and the gateway state should get online.
But not sure if the failover will work after. -
@viragomann I ran a packet cap again and i am seeing those monitoring requests come in as well:
11:14:31.073869 IP 8.8.8.8.853 > 192.168.2.81.17018: tcp 0
11:14:31.073887 IP 76.64.x.x.11762 > 8.8.8.8.853: tcp 0 -
@viragomann When i remove the monitoring ip from the GW; yes it puts the master LAN IP in there and yes it shows Green and Online but no internet.
Ah i really hoped this would have worked out; was such a nice idea to have.
-
@iptvcld said in pfSense HA LAN Interfaces Only:
I ran a packet cap again and i am seeing those monitoring requests come in as well:
11:14:31.073869 IP 8.8.8.8.853 > 192.168.2.81.17018: tcp 0
11:14:31.073887 IP 76.64.x.x.11762 > 8.8.8.8.853: tcp 0So you're using 8.8.8.8 for DNS resolution as well.
Possibly that's an issue when you've stated the WAN gateway in System > General Setup.
However, when using this IP for gateway monitoring, pfSense will automatically add a static route and point it to the respective gateway. So there might be a conflict.As mentioned above, you can use any other public IP for gateway monitoring which is responding to pings. It might be a good idea, to use another IP.
Also you should have a check at System > Advanced > Miscellaneous > State Killing on Gateway Failure.
For further testing you can set the WAN gateway as down manually on the backup.
-
@viragomann Correct; i am using 8.8.8.8 and 8.8.4.4 as DNS Res via system > gateway.
I have now changed my GW monitoring IP to 1.1.1.1 on the secondary pf. Also on the secondary pf i have placed a CHECK under System > Advanced > Miscellaneous > State Killing on Gateway Failure.
When i try to ping 8.8.8.8 from the secondary; i get this timeout
PING 8.8.8.8 (8.8.8.8): 56 data bytes 92 bytes from 127.0.0.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 263f 0 0000 01 01 0000 127.0.0.1 8.8.8.8 92 bytes from 127.0.0.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 b74f 0 0000 01 01 0000 127.0.0.1 8.8.8.8 92 bytes from 127.0.0.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 66ba 0 0000 01 01 0000 127.0.0.1 8.8.8.8 --- 8.8.8.8 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss
I have marked my PPPOE WAN as down on the 2nd pf as well.
This is so bizarre..
I also have just an open rule for LAN
-
@iptvcld said in pfSense HA LAN Interfaces Only:
When i try to ping 8.8.8.8 from the secondary; i get this timeout
You cannot use 8.8.8.8 for troubleshooting, since you direct it to WAN gateway.
However, for getting the DNS resolution work on the secondary when the WAN cable is connected to the primary, you should set the gateway to "none" for all DNS servers in System > General Setup.I have now changed my GW monitoring IP to 1.1.1.1 on the secondary pf.
So try to ping 1.1.1.1 while taking a capture on the LAN interface.
-
@viragomann My master pf DNS settings like like this
and secondary pf is the same
When i ping 1.1.1.1 in seconday the packet capture just gave this
12:57:15.477198 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 797, length 9
-
@iptvcld said in pfSense HA LAN Interfaces Only:
When i ping 1.1.1.1 in seconday the packet capture just gave this
12:57:15.477198 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 797, length 9You should see the same on the masters LAN. Check that out, please.
-
@viragomann thats correct; i just ran a packet cap on master LAN and then sent a 1.1.1.1 ping from the secondary pf
13:05:44.735976 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 1785, length 9
-
@iptvcld
Well, so 1.1.1.1 is routed to the masters LAN address.
Now, if you take a capture on the masters WAN you should also see the ICMP packets to 1.1.1.1, but coming from the WAN IP.If that's not the case, either the firewall rule or the outbound NAT on the master might failing anyhow.
-
@viragomann Ran a packet cap on master and i see this request 3 times
13:10:40.370864 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 2360, length 9
seems to be coming from the secondary pf LAN IP still and not the WAN IP
Outbound NAT on master is this:
And LAN Rules
-
@viragomann Ran it again and i see this
13:17:05.527815 IP 76.64.x.x > 1.1.1.1: ICMP echo request, id 55322, seq 0, length 64 13:17:05.542715 IP 1.1.1.1 > 76.64.x.x: ICMP echo reply, id 55322, seq 0, length 64
then
13:17:05.562166 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 34782, seq 78, length 9
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=55 time=15.187 ms--- 1.1.1.1 ping statistics ---
3 packets transmitted, 1 packets received, 66.7% packet loss
round-trip min/avg/max/stddev = 15.187/15.187/15.187/0.000 ms -
@viragomann Does this need to be unchecked under my WAN interface?
Block private networks and loopback addresses -
@iptvcld said in pfSense HA LAN Interfaces Only:
Ran it again and i see this
13:17:05.527815 IP 76.64.x.x > 1.1.1.1: ICMP echo request, id 55322, seq 0, length 64
13:17:05.542715 IP 1.1.1.1 > 76.64.x.x: ICMP echo reply, id 55322, seq 0, length 64then
13:17:05.562166 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 34782, seq 78, length 9PING 1.1.1.1 (1.1.1.1): 56 data bytes
Strange!
Check the masters state table and filter for 1.1.1.1 after pinging from backup.
-
@iptvcld said in pfSense HA LAN Interfaces Only:
@viragomann Does this need to be unchecked under my WAN interface?
Block private networks and loopback addressesNo, this is only for incoming packets. There is no need to allow private addresses on WAN.
-
This post is deleted! -
@viragomann This is the state table after pinging from 2nd
-
@viragomann When try to ping 1.0.0.1 from 2nd; i get all fails
PING 1.0.0.1 (1.0.0.1): 56 data bytes 92 bytes from 127.0.0.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 ee77 0 0000 01 01 0000 127.0.0.1 1.0.0.1 92 bytes from 127.0.0.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 4453 0 0000 01 01 0000 127.0.0.1 1.0.0.1 92 bytes from 127.0.0.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 071e 0 0000 01 01 0000 127.0.0.1 1.0.0.1 --- 1.0.0.1 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss
And when i run this ping while doing packet cap on master LAN; i dont see the entries come over
-
@iptvcld said in pfSense HA LAN Interfaces Only:
This is the state table after pinging from 2nd
The last two lines look well for only one ping packet. One state entry for LAN and one for WAN.
But the other lines are showing faults at all. The packets column shows the packets for each direction (request, respond). So it shows many requests, but few responds.Not clear, what's going wrong here, now.
Maybe there are some hints in the system log?