Problem with Combination of Static Routes and Failover Group (solved)
-
I'm running version 2.0-BETA4 (i386) built on Sat Nov 27 04:12:08 EST 2010. I'm starting to like pf a lot more than iptables!
I have a pfSense router at 172.16.1.1, attached to the 172.16.1.0/24 subnet. I also have another router at 172.16.1.5, which has a second interface that is attached to the 172.16.2.0/24 subnet. I believe the pfSense book calls this is an asymmetric routing situation. Here is a diagram:
PC1 (172.16.1.69) –----- pfsense (LAN: 172.16.1.1, WAN:24.x.x.x) ------ other router (LAN1: 172.16.1.5, LAN2: 172.16.2.5) ----- PC2 (172.16.2.1)
On pfSense, I have a static route defined as: Network 172.16.2.0/24, Gateway 172.16.1.5, Interface: LAN.
On the System: Advanced: Firewall and NAT page, I have CHECKED the checkbox "Bypass firewall rules for traffic on the same interface", as recommened by the pfSense book and several forum posts about static routes.
I have grepped /tmp/rules.debug, to confirm that checking the checkbox does indeed cause these rules to be automatically created:
pass quick on $LAN proto tcp from 172.16.1.0/24 to 172.16.2.0/24 flags any keep state(sloppy) label "pass traffic between statically routed subnets"
pass quick on $LAN from 172.16.1.0/24 to 172.16.2.0/24 keep state(sloppy) label "pass traffic between statically routed subnets"
pass quick on $LAN proto tcp from 172.16.2.0/24 to 172.16.1.0/24 flags any keep state(sloppy) label "pass traffic between statically routed subnets"
pass quick on $LAN from 172.16.2.0/24 to 172.16.1.0/24 keep state(sloppy) label "pass traffic between statically routed subnets"In the firewall, I initially changed the Gateway for the default LAN rule to use a failover group instead of "default", which would have used the system routing table.
From pfSense (172.16.1.1), I can ping PC2 (172.16.2.1), via the other router (172.16.1.5). However, from PC1 (172.16.1.69), I CANNOT ping PC2.
When I looked at the States page, in the "Source -> Router -> Destination" column, I could see the problem:
172.16.1.69 -> 24.x.x.x -> 172.16.2.1In other words, because of the failover group, packets coming from the LAN to the 172.16.2.0/24 subnet were being routed out the WAN interface instead of going back out the LAN interface.
As a workaround, I changed the Gateway for the default LAN rule back to "default", to use the system routing table. With that default setting, now I CAN ping PC2 from PC1. Now on the States page, I can see: 172.16.1.69 -> 172.16.2.1, which is good.
Question: How can I get this to work with a failover group AND still be able to ping PC2 from PC1? I can possibly think of a few ways but I'm wondering what the best practice would be in this case. I suppose I could leave the default LAN rules with the failover group and then create a rule above it for the destination of 172.16.2.0/24. But what state type would I use because this is an asymmetric routing situation? Would I use "none" or "sloppy"?
-
Just create a rule before the failover group with the state option to sloppy and no gateway defined for that specific rule.
Possibly duplicate one of the autocreated rules you pasted. -
@ermal:
Just create a rule before the failover group with the state option to sloppy and no gateway defined for that specific rule.
Possibly duplicate one of the autocreated rules you pasted.Thanks Ermal! I made a copy of the default LAN rule that had the failover group. I put the copy above the default LAN rule. In the copy, I made the following changes:
-
Changed gateway from failover to default
-
Changed destination from * to 172.16.2.0/24
-
Changed state type from keep state to sloppy state
Now failover works AND I can ping PC2 from PC1.
Okay, here's a follow up question. When I used our old Linux router in place of the pfSense router, the Linux router generates ICMP redirect packets when PC1 pings PC2, which automatically leads to a route being added on PC1 which goes directly to 172.16.1.5 instead of going to 172.16.1.1 first. That is, PC1 automatically gets the route 172.16.2.1 via 172.16.1.5.
For some reason, pfSense does not seem to be sending these ICMP redirects to PC1 because the redirecting route does not get added to PC1. What could be the reason? As far as I can tell, I haven't created any firewall rules on pfSense that would block these ICMP redirects from being sent to PC1.
-
-
That is a tunable you can set in pfSense.
net.inet.ip.redirect which should be 1 if you want it to be sent. -
Okay, here's a follow up question. When I used our old Linux router in place of the pfSense router, the Linux router generates ICMP redirect packets when PC1 pings PC2, which automatically leads to a route being added on PC1 which goes directly to 172.16.1.5 instead of going to 172.16.1.1 first. That is, PC1 automatically gets the route 172.16.2.1 via 172.16.1.5.
For some reason, pfSense does not seem to be sending these ICMP redirects to PC1 because the redirecting route does not get added to PC1. What could be the reason? As far as I can tell, I haven't created any firewall rules on pfSense that would block these ICMP redirects from being sent to PC1.
Solved! My bad! PC1's firewall was blocking ICMP redirects. PC1 is running Windows XP Home (I know, I know). I guess the Windows Firewall blocks ICMP redirects by default. To fix it, you go to Control Panel | Windows Firewall | Advanced | ICMP Settings. Then you check the checkbox "Allow Redirect" and click OK, OK, etc.
After that, pinging PC2 from PC1 causes PC1 to automatically get the route 172.16.2.1 via 172.16.1.5 in its routing table. To test that it works, I starting pinging PC2 from PC1 and then unplugged the LAN cable from pfSense. As expected, PC2 still kept responding to PC1's pings. Woot!
So happy with pfSense 2 so far. Can't wait for official release! :)
By the way, not shilling here but…if you don't have the pfSense Book yet, you should ALL buy it. Not only is it a great resource (even for new pfSense 2) but you'll also be supporting the pfSense project. C'mon you guys, don't be cheap. Buy the book!
-
@ermal:
That is a tunable you can set in pfSense.
net.inet.ip.redirect which should be 1 if you want it to be sent.Thanks again, Ermal. I did not have to change the value of that tunable as it appears the default is 1.
> sysctl net.inet.ip.redirect net.inet.ip.redirect: 1Yet another follow up question: I've been searching these forums and googling but I cannot find the definition of the sloppy state type. What does it mean, exactly?
-
Its just a way to tell pf(4) to not do fancy tracking of tcp sequences but just keep state of packets.
It just matters for tcp. -
@ermal:
Its just a way to tell pf(4) to not do fancy tracking of tcp sequences but just keep state of packets.
It just matters for tcp.In case anyone is wondering, from the man (5) page for pf.conf, I found this description of sloppy state tracking:
sloppy Uses a sloppy TCP connection tracker that does not check sequence numbers at all, which makes insertion and ICMP teardown attacks way easier. This is intended to be used in situations where one does not see all packets of a connection, e.g. in asymmetric routing situations. It cannot be used with modulate or synproxy state.Note the word asymmetric in the description.