Problems when a route is lost
-
Currently I'm running 2010 Feb 3 snapshot.
Here is my configuration:WAN – static link to corporate data center. GW_WAN is the default gateway.
OPT1, OPT2 -- PPPoE links to the same gateway (gw0) of ISP A. But they monitor other IP addresses (gw1, gw2) outside of gw0.
OPT3 -- PPPoE link to another gateway of ISP B.
OPT1 and OPT2 are set to load balance outbound connections. Furthermore there are TCP and UDP servers behind OPT1, OPT2 and OPT3.
The IPv4 routing table seems to be correct, with a minor problem that both the routes to gw1, gw2 are filled with Gateway = gw0 (which is correct) and Netif = pppoe1 (which is not correct: should be pppoe1 for gw1 and pppoe2 for gw2).
(The IPv6 routing table seems to be correct. But I dont use IPv6.)
With all links up, inbound connections to all OPTx work, and OPT1-OPT2 load balance fine.
When the WAN link is down, however, the default route is lost and no connection through OPTx can be made. (Outbound connections via OPTx still work and load balancing is still fine.)
When either OPT1 link or OPT2 link is down, the routes to both gw1 and gw2 are lost, and no inbound UDP connections through OPT1 and OPT2 can be made. When the link is up again, inbound connectivity are resumed but the previously lost routes are not restored and OPT1-OPT2 do not load balance anymore.
-
When the WAN link is down, however, the default route is lost and no connection through OPTx can be made. (Outbound connections via OPTx still work and load balancing is still fine.)You mean internally from the pfSense box itself?!
When either OPT1 link or OPT2 link is down, the routes to *both* gw1 and gw2 are lost, and no inbound UDP connections through OPT1 and OPT2 can be made. When the link is up again, inbound connectivity are resumed but the previously lost routes are not restored and OPT1-OPT2 do not load balance anymore.When the pppoe links come up they do not restore the routes?!
Or you say that you have configured static routes which do not get re-entered when the links come up? -
@ermal:
When the WAN link is down, however, the default route is lost and no connection through OPTx can be made. (Outbound connections via OPTx still work and load balancing is still fine.)You mean internally from the pfSense box itself?!
By 'outbound' I mean from the LAN to Internet. By 'inbound' I mean from Internet to the LAN. When the default route is lost no inbound connections through OPTx can be made, while outbound connections via OPTx are still possible and load balanced.
@ermal:
When either OPT1 link or OPT2 link is down, the routes to *both* gw1 and gw2 are lost, and no inbound UDP connections through OPT1 and OPT2 can be made. When the link is up again, inbound connectivity are resumed but the previously lost routes are not restored and OPT1-OPT2 do not load balance anymore.When the pppoe links come up they do not restore the routes?!
Or you say that you have configured static routes which do not get re-entered when the links come up?From the routing table, the routes gw1 and gw2 are marked as UGHS, indicating that they are static routes, but I didn't add them manually. They are automatically created and added to the routing table when I declare gw1 and gw2 as 'Monitor IPs' in the System:Gateways GW_OPT1 and GW_OPT2 config pages. When OPT1 is down, gw1 and gw2 are both lost. When OPT1 comes up again gw1 and gw2 are not restored. I don't know if gw1 / gw2 (which are not gateways, just Monitor IPs) have some thing to do with load balancing. It seems that without them, load balancing cannot work, though.
-
Can i have a look at your /tmp/rules.debug when you loose the default route!
A route table dump and /var/db and /var/etc contents will be useful.If you want to keep am private send them to my email address.
-
Sorry for this late reply but I'm getting in serious troubles… Now I'm running Mon Feb 8 02:15:18 EST 2010 snapshot and, no matter what I do, policy-based routing definitively stopped working. So did load balancing. Everything outbound now go via WAN -- the default route.
And what's more, I can not change default gateway by WebUI anymore.
So I am completely lost, don't know what to look at first. Do you still care to see my /tmp/rules.debug? I cannot find out your e-mail address...
-
eri at pfsense.org
Please send your logs, config.xml, rules.debug, ifconfig, netstat -rn.