dual wan setup with openvpn client, not failing over



    • i have a cable based primary ISP. i have been having problem with them of late so i have been looking into getting a cellular backup. i found the netgear lb1120 device and i picked up a prepaid sim card. i am using a white box pfsense machine that i built; it has a dual NIC intel board. because of the limit on the onboard NICs i am using VLAN for the WAN and LAN connections in pfsense.
    • i have the ISP cable modem plug into my main switch port #1 in access mode, VLAN10, and the lb1120 plug into switch port #2 in access mode, VLAN20.
    • switch port #3 is in trunk mode with VLAN10 and VLAN20
    • i have create the corresponding VLANs in pfsense with the correct NIC and then create the pfsense interface for each WAN connection. WAN for the cable ISP and WANC for cellular
    • this is a home setup so both WAN connections provide DHCP IPs for the public address
    • wan_dhcp has 8.8.8.8 configured for its monitor IP
    • wanc_dhcp has 8.8.4.4 configured for its monitor IP
    • pfsense default gateway is set to wan_dhcp
    • i created a gateway group called wan and put wan_dhcp and wanc_dhcp. wan_dhcp is tier 1 and wanc_dhcp is tier 2. trigger level is set to member down
    • i have two openvpn client configuration in pfsense, the interface on both is set to GW Group wan. the two different openvpn connections are using the same port to connect on.
    • not sure that it matters but for completeness, the lb1120 is running in router mode. so the pfsense wanc interface get a RFC1918 address. the subnet that the lb1120 is configured for DHCP leases is not in my current network architecture so there is no routing problems because of it. i have tried to run in bridge mode, but the lb1120 only stays online for about 5 minutes at a time before it drops its LTE connection and reboots. in bridge mode i do not have that problem. in pfsense on the wanc interface, i have unchecked the two check boxes for reserved networks because of this
    • when my ISP connection drops i check status -> gateways -> gateway groups in pfsense. i see that wan_dhcp offline and wanc_dhcp online. so i should be up and running the on cellular connection now. also when the ISP connection is down, the wan_dhcp address gets set to 192.168.100.10. not a subnet on my network so should not be a routing problem
    • i then check openvpn client. status -> openvpn -> client instance statistics section. both openvpn clients configured in pfsense are down. i try to restart them and same outcome

    things that i have tried:
    #1:
    - i stopped the second openvpn client from reconnecting.
    - created a floating firewall rule on wan and wanc interfaces to route the traffic using the advance rule setting for the gateway group wan. i turned on logging for the rule.
    - restart the openvpn client, failed to connect
    - i see the rule matching and getting logged in the firewall log. interface is wan (has a triangle beside it) source is 192.168.100.10 and destination is the correct openvpen server ip for this connection
    --- confused why the interface says wan and not wanc and why 192.168.100.10 was listed as source address
    - there is no outbound NAT for that subnet, since it is not a subnet that i use. i created a outbound NAT rule for that on both WAN interfaces.
    - restart openvpn client, still did not connect

    #2
    - tried to change the openvpn client interface to other choices in the drop down.  
    	- wan was not an option.  wanc was
    	- any - there was no difference between interface set to GW Group wan and this option
    	- localhost - openvpn client still did not connect but in the firewall log, i see the source as 127.0.0.1 instead of 192.168.100.10
    	- wanc - openvpn connected and traffic was routing over the tunnel correctly
    

    ` #3
    - openvpn client interface set to GW Group wan
    - started a packet capture in pfsense for the wan interface
    - restart openvpn client, failed
    - stop capture, downloaded, and opened in Wireshark
    - none of the openvpn traffic was in capture

    ` #4
    - openvpn client interface set to GW Group wan
    - started a packet capture in pfsense for the wanc interface
    - restart openvpn client, failed
    - stop capture, downloaded, and opened in Wireshark
    - openvpn traffic was in capture
    - source IP is 192.168.100.10, destination is the correct IP for the openvpn server
    - i had packet capture running on the edge device on the openvpn server network. the openvpn client traffic never reaches the server network

    what am i missing or doing incorrectly?


Log in to reply