One gateway down after upgrade to 2.3, issue doesn't seem to be dpinger related
After upgrading pfSense from 2.0 to 2.3 I've encountered an issue with one interface/gateway. I've followed the advice on https://forum.pfsense.org/index.php?topic=110043.0 without fixing the issue so I thought I'd make a dedicated thread.
I currently run pfSense with 8 network interfaces.
- WAN 1 - PPPoE DSL ISP A - 212.159.X.A - GW: 62.24.X.X
- WAN 2 - PPPoE DSL ISP A - 92.27.X.A - GW: 62.24.X.X
- WAN 3 - PPPoE DSL ISP A - 212.159.X.B - GW: 62.24.X.X
- WAN 4 - PPPoE DSL ISP A - 212.159.X.C - GW: 62.24.X.X
- WAN 5 - PPPoE DSL ISP B - 81.149.X.A - GW: 81.139.X.X
- LAN 1 - 10.0.1.1/24
- LAN 2 - 10.0.2.1/24
- LAN 3 - 10.0.3.1/24
WAN 1 - 4 are fibre DSL connections from the same IP. All IP addresses are static and assigned when establishing the PPPoE connection. WAN2's static IP was changed at some point by the ISP to a range different from the other interfaces from the same ISP.
WAN 5 is also a fibre DSL connection from a different ISP with again a static IP address but assigned by the ISP.
All interfaces are up and the DSL looks like it is connected. The WANs are in a gateway group. All connections use the same BT openreach modems. However after upgrading from 2.0 –-> 2.3, WAN 2 gateway's status always shows as down.
I've sanity checked the gateway monitoring configuration and added a data payload as suggested in the above linked thread. It's had no effect and I don't think its related to dpinger, as I cannot manually ping from WAN2 using the web gui:
From WAN 2:
PING 126.96.36.199 (188.8.131.52) from 92.27.X.A: 56 data bytes --- 184.108.40.206 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss
PING 220.127.116.11 (18.104.22.168) from 212.159.X.A: 56 data bytes 64 bytes from 22.214.171.124: icmp_seq=0 ttl=57 time=10.271 ms 64 bytes from 126.96.36.199: icmp_seq=1 ttl=57 time=10.764 ms 64 bytes from 188.8.131.52: icmp_seq=2 ttl=57 time=10.399 ms --- 184.108.40.206 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 10.271/10.478/10.764/0.209 ms
Also interesting is tracerouting. If I run a traceroute via WAN2 I get the following:
1 81.139.X.X 10.935 ms 11.389 ms 10.708 ms 2 * * * 3 * * *
1 62.24.X.X 5.102 ms 4.897 ms 4.539 ms 2 220.127.116.11 5.407 ms 5.432 ms 5.204 ms 3 18.104.22.168 5.431 ms 5.668 ms 5.668 ms
The first hop looks very strange. in fact…
1 81.139.X.X 14.068 ms 14.197 ms 13.859 ms 2 22.214.171.124 14.056 ms 14.311 ms 13.643 ms 3 126.96.36.199 14.972 ms 14.960 ms 14.556 ms
.. it looks like when running a traceroute from WAN 2 the first hop is the same as a traceroute from WAN 5 which uses a different ISP and different gateway!
Out of interest, incoming connections to WAN2 are functioning and I've tried deleting the interface and gateway and setting up again. I've power cycled the router and the WAN2 vDSL modem. This setup has been running for 3 years using an older version of pfSense and has been a stable working setup for our office.
Any advice would be greatly appreciated and please let me know if I can provide any extra information.