[solved] BUG: pfsense 2.4.4 update_breaks http/https from LAN - workaround
-
Steve, I still have issues with linuxmint and debian clients accessing the package updates via the vpn and google mail struggles. these all previously worked. One thing i notice under the Diag - Interfaces.
DNS servers all show below the WAN interface.
127.0.0.1
1.1.1.1
9.9.9.9
1.0.0.1
9.9.9.10
81.3.27.54
46.182.19.48
In actual fact, they are assigned via the general settings per
1.1.1.1, 9.9.9.9 = vpn1
1.0.0.1, 9.9.9.10 = vpn2
81.3.27.54, 46.182.19.48 = wan -
I'm pretty sure that's just a display artifact. Check the routing table. DNS servers set to a specific WAN should show a static route via that gateway.
Steve
-
Steve, i got to the bottom of the problem i am having. Sorted and confirmed DNS redirecting to the box, and then out over 853. Confirmed with android devices that ignore dhcp and try and connect to google.
I have a IP4/6 blocking rule for everything on all interfaces and only allow selected ports within the LAN and separately selected ports via the WAN
I have 2 ExpressVPNs as Tier 1 and 2. Routing correctly shows 1x Quad9 + 1xCloudfare DNS per connection.
When both connections are up browsing is working but things like android updates, linux updates, etc don't work. This is true, whether i use the pool gateway or the individual gateways.
When i force one of the VPNs down, then everything is working.
So, it seems i miss something..from a route blocking / etc? Do you have some idea? -
Hmm, if the VPNs are in a gateway group just one of them going down would not normally do anything.
The fact that changes anything implies perhaps you are allowing the VPN connection to add new routes to the system still, or a new default route even.
Or potentially you have rules that not applied when the gateway is down. Check in System > Advanced > Misc. Do you have 'Skip rules when gateway is down' enabled?
Steve
-
I had it checked, and now unchecked. Doesn't make any difference. As soon as i enable the 2nd VPN the android clients are unable to update (as practical example). In the gateway group VPN1 = Tier 1 and VPN2 = Tier 2 based on member down. So in theory, VPN2 shouldn't even be touched, but seems like some kind of routing problem. although strange browsing still works
-
Hmm. Probably need to dig into the ruleset and states here to see what rules are passing the traffic and where.
-
something much more fundamental Steve.
I turned all rules off. and only open DNS and HTTP/HTTPS from LAN to Gateway Pool.
With 2x OpenVPN Client active; i can browse the net, but android updates time out.
Shutdown one of the OpenVPN clients, and android updates takes off. -
@gwaitsi said in [solved] BUG: pfsense 2.4.4 update_breaks http/https from LAN - workaround:
With 2x OpenVPN Client active; i can browse the net, but android updates time out.
Shutdown one of the OpenVPN clients, and android updates takes off.Well maybe the android update servers block that VPN provider/IP.
-
@grimson Nope. Both VPN1 and 2 are ExpressVPN access points in different countries. If either one is disabled, it works. it is only when both are enable at the same time that it doesn't work. Android was just quick/easy example. Linux Mint updates break. only the standard standards servers are found at a much lower bandwidth, no local servers. When only 1 VPN enabled, linux standard servers increase bandwidth dramatically and the local servers can be found.
-
Appears like it's load-balancing between those gateways but they should be failover right?
Check how they show in /tmp/rules.debug.
Steve
-
yep, trigger level is member down.
note sure what i am looking for m8, but here is what i seeGateways
GWRED_DHCP = " route-to ( em0 192.168.0.1 ) "
GWLUXVPN_VPNV4 = " route-to ( ovpnc1 10.149.0.73 ) "
GWVPNNLD_VPNV4 = " route-to ( ovpnc3 10.156.0.129 ) "
GWExpressVPN = " route-to { ( ovpnc1 10.149.0.73 ) } "Load balancing anchor
rdr-anchor "relayd/*"
UPnPd rdr anchor
rdr-anchor "miniupnpd"
anchor "relayd/"
anchor "openvpn/"
anchor "ipsec/*"VPN Rules
anchor "tftp-proxy/*"
-
@stephenw10 I am guessing the issue relates to the pushing of the DNS?
Options error: option 'route' cannot be used in this context ([PUSH-OPTIONS])
Options error: option 'dhcp-option' cannot be used in this context ([PUSH-OPTIONS])
Options error: option 'redirect-gateway' cannot be used in this context ([PUSH-OPTIONS])
PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 10.154.0.1,comp-lzo no,route 10.154.0.1,topology net30,ping 10,ping-restart 60,ifconfig 10.154.1.18 10.154.1.17,peer-id 23' -
Hmm, well those options look broken. I assume the servers are sending them to you.
You might try the options in the OpenVPN client 'do not pull routes' and or 'do not add routes'. Those may prevent whatever routing changes are happening.
You definitely don't want it redirecting the gateway. Luckily that's a broken option!
Your file does not have a load-balance group shown. The behaviour is very much like that but can't be for that reason.
Steve
-
it is definitely to with the openvpn client config. using "force gateway" down doesn't help.
Only disabling one of either of the clients works. -
Ok so what actually changes when you disabled one? Or presumably enable it again breaking stuff?
Different routes? Different DNS?
Steve
-
Below is the working route/gateway table. Firewall rules used to push traffic to VPN
Internet: Destination Gateway Flags Netif Expire default 192.168.0.1 UGS em0 1.1.1.1 10.156.0.29 UGHS ovpnc3 9.9.9.9 10.156.0.29 UGHS ovpnc3 10.156.0.29 link#9 UH ovpnc3 10.156.0.30 link#9 UHS lo0 46.182.19.48 192.168.0.1 UGHS em0 81.3.27.54 192.168.0.1 UGHS em0 91.xx.xx.xx 10.156.0.29 UGHS ovpnc3 127.0.0.1 link#6 UH lo0 192.168.0.0/24 link#1 U em0 192.168.0.234 link#1 UHS lo0 192.168.20.0/24 link#2 U em1 192.168.20.5 link#2 UHS lo0 192.168.21.0/24 link#4 U em3 192.168.21.5 link#4 UHS lo0Table from where is does not work.
Internet: Destination Gateway Flags Netif Expire default 192.168.0.1 UGS em0 1.0.0.1 10.149.0.13 UGHS ovpnc1 1.1.1.1 10.156.0.29 UGHS ovpnc3 9.9.9.9 10.156.0.29 UGHS ovpnc3 9.9.9.10 10.149.0.13 UGHS ovpnc1 10.149.0.13 link#10 UH ovpnc1 10.149.0.14 link#10 UHS lo0 10.156.0.29 link#9 UH ovpnc3 10.156.0.30 link#9 UHS lo0 46.182.19.48 192.168.0.1 UGHS em0 81.3.27.54 192.168.0.1 UGHS em0 85.xx.xx.xx 10.149.0.13 UGHS ovpnc1 91.xx.xx.xx 10.156.0.29 UGHS ovpnc3 127.0.0.1 link#6 UH lo0 192.168.0.0/24 link#1 U em0 192.168.0.234 link#1 UHS lo0 192.168.20.0/24 link#2 U em1 192.168.20.5 link#2 UHS lo0 192.168.21.0/24 link#4 U em3 192.168.21.5 link#4 UHS lo0This is the traceroute (which works) from when both VPNs are up.
traceroute to mintlinux.mirror.wearetriple.com (93.187.10.106), 30 hops max, 60 byte packets 1 10.156.0.1 (10.156.0.1) 30.850 ms * 30.785 ms 2 * v741.ce01.ams-01.nl.leaseweb.net (37.48.118.60) 30.738 ms * 3 * ae-5.cr01.ams-01.nl.leaseweb.net (81.17.33.128) 30.667 ms * 4 be-111.bb03.ams-01.leaseweb.net (31.31.38.200) 30.622 ms * be-112.bb03.ams-01.leaseweb.net (31.31.38.204) 30.578 ms 5 * triple-it.telecity2.nl-ix.net (193.239.116.57) 37.565 ms * 6 mirror.wearetriple.com (93.187.10.106) 37.501 ms 25.516 ms *however, this is the error from the linux package manager and as i say, on the whole http browsing works.
Failed to fetch http://mintlinux.mirror.wearetriple.com/packages/dists/tessa/InRelease Cannot initiate the connection to mintlinux.mirror.wearetriple.com:80 (2a00:1f00:dc06:10::106). - connect (101: Network is unreachable) Could not connect to mintlinux.mirror.wearetriple.com:80 (93.187.10.106), connection timedyou can see it is resolving in the application error message, so i don't think it is a dns issue. If i shutdown either of the VPN clients, this will work.
