dual wan setup with openvpn client, not failing over
-
- i have a cable based primary ISP. i have been having problem with them of late so i have been looking into getting a cellular backup. i found the netgear lb1120 device and i picked up a prepaid sim card. i am using a white box pfsense machine that i built; it has a dual NIC intel board. because of the limit on the onboard NICs i am using VLAN for the WAN and LAN connections in pfsense.
- i have the ISP cable modem plug into my main switch port #1 in access mode, VLAN10, and the lb1120 plug into switch port #2 in access mode, VLAN20.
- switch port #3 is in trunk mode with VLAN10 and VLAN20
- i have create the corresponding VLANs in pfsense with the correct NIC and then create the pfsense interface for each WAN connection. WAN for the cable ISP and WANC for cellular
- this is a home setup so both WAN connections provide DHCP IPs for the public address
- wan_dhcp has 8.8.8.8 configured for its monitor IP
- wanc_dhcp has 8.8.4.4 configured for its monitor IP
- pfsense default gateway is set to wan_dhcp
- i created a gateway group called wan and put wan_dhcp and wanc_dhcp. wan_dhcp is tier 1 and wanc_dhcp is tier 2. trigger level is set to member down
- i have two openvpn client configuration in pfsense, the interface on both is set to GW Group wan. the two different openvpn connections are using the same port to connect on.
- not sure that it matters but for completeness, the lb1120 is running in router mode. so the pfsense wanc interface get a RFC1918 address. the subnet that the lb1120 is configured for DHCP leases is not in my current network architecture so there is no routing problems because of it. i have tried to run in bridge mode, but the lb1120 only stays online for about 5 minutes at a time before it drops its LTE connection and reboots. in bridge mode i do not have that problem. in pfsense on the wanc interface, i have unchecked the two check boxes for reserved networks because of this
- when my ISP connection drops i check status -> gateways -> gateway groups in pfsense. i see that wan_dhcp offline and wanc_dhcp online. so i should be up and running the on cellular connection now. also when the ISP connection is down, the wan_dhcp address gets set to 192.168.100.10. not a subnet on my network so should not be a routing problem
- i then check openvpn client. status -> openvpn -> client instance statistics section. both openvpn clients configured in pfsense are down. i try to restart them and same outcome
things that i have tried:
#1:
- i stopped the second openvpn client from reconnecting.
- created a floating firewall rule on wan and wanc interfaces to route the traffic using the advance rule setting for the gateway group wan. i turned on logging for the rule.
- restart the openvpn client, failed to connect
- i see the rule matching and getting logged in the firewall log. interface is wan (has a triangle beside it) source is 192.168.100.10 and destination is the correct openvpen server ip for this connection
--- confused why the interface says wan and not wanc and why 192.168.100.10 was listed as source address
- there is no outbound NAT for that subnet, since it is not a subnet that i use. i created a outbound NAT rule for that on both WAN interfaces.
- restart openvpn client, still did not connect#2 - tried to change the openvpn client interface to other choices in the drop down. - wan was not an option. wanc was - any - there was no difference between interface set to GW Group wan and this option - localhost - openvpn client still did not connect but in the firewall log, i see the source as 127.0.0.1 instead of 192.168.100.10 - wanc - openvpn connected and traffic was routing over the tunnel correctly
` #3
- openvpn client interface set to GW Group wan
- started a packet capture in pfsense for the wan interface
- restart openvpn client, failed
- stop capture, downloaded, and opened in Wireshark
- none of the openvpn traffic was in capture` #4
- openvpn client interface set to GW Group wan
- started a packet capture in pfsense for the wanc interface
- restart openvpn client, failed
- stop capture, downloaded, and opened in Wireshark
- openvpn traffic was in capture
- source IP is 192.168.100.10, destination is the correct IP for the openvpn server
- i had packet capture running on the edge device on the openvpn server network. the openvpn client traffic never reaches the server networkwhat am i missing or doing incorrectly?