[workaround] policy based routing in 2010-01-20
-
i don't know if it was this build specifically or what but i could not get PBR to work. i have a requirement that a voip adapter only goes through one of the two ISPs. when i was testing 2010-01-20, it would go through the ISP i didn't want it to.
i even swapped WAN<->OPT1 but it didn't help. here's what the rule looks like:
Proto Source Port Destination Port Gateway
* 192.168.1.102 * * * OPT1clearing the state table several times didn't help. i'm back on 1.2.3 now because of this problem. anyone ideas?
EDIT/workaround:
I have some more details on this problem. it's not actually fixed. if you run tcpdump, opt1 works fine. otherwise, it doesn't work. my workaround is to enable permanent promiscuous mode (ifconfig <interface>promisc). this interface does have a spoofed mac address.
after disabling the spoofed mac and rebooting, i don't seem to need promisc mode anymore. i'll monitor it for a while to confirm this.
i had the spoofed mac in both a physical system and a vmware machine. they were different physical boxes with different nics - one was intel 100mbit, it's now broadcom gigabit. this issue didn't exist in 1.2.3-RC1 which i ran with the spoofed mac on box boxes.</interface>
-
I confirm this problem. It was appear at end of december.
-
Same problem here. but…
when it did not work as expected I turned to the pfsense packet sniffer to try to see what was happening.
Didn't get much out of the sniffer, nothing was logged.
To my surprise the PBR worked, as long as I had the sniffer turned on...
So I guess the sniffer changes the mode of the interface, promiscuous mode? -
What driver is this?
Or better provide an 'ifconfig' command output.
A 'netstat -rn' one.
Plus a /tmp/rules.debug content to reproduce this. -
In my case it's an all LAB envirorment.
So the pfSense is installed under vmware server 2.0.Output of:
$ netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.100.1 UGS 3 12702 le0 127.0.0.1 link#5 UH 0 0 lo0 127.0.0.2 127.0.0.1 UHS 0 0 lo0 172.16.1.0/24 link#3 U 0 1831 le2 192.168.80.0/24 link#2 U 0 371959 le1 192.168.80.250/32 192.168.80.2 US 1 349181 le1 192.168.90.0/24 192.168.80.2 US 0 41186 le1 192.168.100.0/24 link#1 U 2 252439 le0 192.168.100.160 link#1 UHS 0 0 lo0 Internet6: Destination Gateway Flags Netif Expire ::1 ::1 UH lo0 fe80::%le0/64 link#1 U le0 fe80::20c:29ff:fe22:79b8%le0 link#1 UHS lo0 fe80::%le1/64 link#2 U le1 fe80::20c:29ff:fe22:79c2%le1 link#2 UHS lo0 fe80::%le2/64 link#3 U le2 fe80::20c:29ff:fe22:79cc%le2 link#3 UHS lo0 fe80::%lo0/64 link#5 U lo0 fe80::1%lo0 link#5 UHS lo0 ff01:1::/32 fe80::20c:29ff:fe22:79b8%le0 U le0 ff01:2::/32 fe80::20c:29ff:fe22:79c2%le1 U le1 ff01:3::/32 fe80::20c:29ff:fe22:79cc%le2 U le2 ff01:5::/32 ::1 U lo0 ff02::%le0/32 fe80::20c:29ff:fe22:79b8%le0 U le0 ff02::%le1/32 fe80::20c:29ff:fe22:79c2%le1 U le1 ff02::%le2/32 fe80::20c:29ff:fe22:79cc%le2 U le2 ff02::%lo0/32 ::1 U lo0and
$ cat /tmp/rules.debug #System aliases loopback = "{ lo0 }" WAN = "{ le0 }" LAN = "{ le1 }" DMZ = "{ le2 }" IPsec = "{ enc0 }" #SSH Lockout Table table <sshlockout>persist #Snort2C table table <snort2c>table <virusprot># User Aliases set loginterface le0 set loginterface le1 set loginterface le2 set optimization normal set limit states 23000 set skip on pfsync0 scrub in on $WAN all fragment reassemble scrub in on $LAN all fragment reassemble scrub in on $DMZ all fragment reassemble nat-anchor "natearly/*" nat-anchor "natrules/*" # Outbound NAT rules # Subnets to NAT tonatsubnets = "{ 192.168.80.0/24 172.16.1.0/24 }" nat on $WAN from $tonatsubnets port 500 to any port 500 -> 192.168.100.160/32 port 500 nat on $WAN from $tonatsubnets port 5060 to any port 5060 -> 192.168.100.160/32 port 5060 nat on $WAN from $tonatsubnets to any -> 192.168.100.160/32 port 1024:65535 # Load balancing anchor rdr-anchor "relayd/*" # TFTP proxy rdr-anchor "tftp-proxy/*" table <vpns>{ 192.168.90.0/24 } table <direct_networks>{ 192.168.100.0/24 192.168.80.0/24 172.16.1.0/24 } # UPnPd rdr anchor rdr-anchor "miniupnpd" anchor "relayd/*" anchor "firewallrules" #--------------------------------------------------------------------------- # default deny rules #--------------------------------------------------------------------------- block in log all label "Default deny rule" block out log all label "Default deny rule" # We use the mighty pf, we cannot be fooled. block quick proto { tcp, udp } from any port = 0 to any block quick proto { tcp, udp } from any to any port = 0 # Block all IPv6 block in quick inet6 all block out quick inet6 all # snort2c block quick from <snort2c>to any label "Block snort2c hosts" block quick from any to <snort2c>label "Block snort2c hosts" # package manager early specific hook anchor "packageearly" # carp anchor "carp" # SSH lockout block in log quick proto tcp from <sshlockout>to any port 22 label "sshlockout" block in quick from <virusprot>to any label "virusprot overload table" antispoof for le0 antispoof for le1 # allow access to DHCP server on LAN anchor "dhcpserverLAN" pass in on $LAN proto udp from any port = 68 to 255.255.255.255 port = 67 label "allow access to DHCP server" pass in on $LAN proto udp from any port = 68 to 192.168.80.2 port = 67 label "allow access to DHCP server" pass out on $LAN proto udp from 192.168.80.2 port = 67 to any port = 68 label "allow access to DHCP server" antispoof for le2 anchor "spoofing" # loopback anchor "loopback" pass in on $loopback all label "pass loopback" pass out on $loopback all label "pass loopback" anchor "firewallout" # let out anything from the firewall host itself and decrypted IPsec traffic pass out all keep state allow-opts label "let out anything from firewall host itself" pass out on $IPsec all keep state label "IPsec internal host to host" # make sure the user cannot lock himself out of the webConfigurator or SSH anchor "anti-lockout" pass in quick on le1 from any to (le1) keep state label "anti-lockout rule" # NAT Reflection rules # User-defined rules follow pass in quick on $WAN reply-to ( le0 192.168.100.1 ) from any to any keep state label "USER_RULE" pass in quick on $DMZ from 192.168.80.0/24 to 172.16.1.2/24 keep state label "USER_RULE" pass in log quick on $LAN route-to ( le1 192.168.80.250 ) proto { tcp udp } from ! 192.168.80.250 to 192.168.90.0/24 keep state label "USER_RULE: reroute replify" pass in quick on $LAN from any to any keep state label "USER_RULE: Default allow LAN to any rule" pass in quick on $IPsec from any to 192.168.80.0/24 keep state label "USER_RULE" # VPN Rules pass out on $WAN route-to ( le0 192.168.100.1 ) proto udp from any to 192.168.100.165 port = 500 keep state label \"IPsec: 192.168.100.165 - outbound isakmp\" pass in on $WAN reply-to ( le0 192.168.100.1 ) proto udp from 192.168.100.165 to any port = 500 keep state label \"IPsec: 192.168.100.165 - inbound isakmp\" pass out on $WAN route-to ( le0 192.168.100.1 ) proto udp from any to 192.168.100.165 port = 4500 keep state label \"IPsec: 192.168.100.165 - outbound nat-t\" pass in on $WAN reply-to ( le0 192.168.100.1 ) proto udp from 192.168.100.165 to any port = 4500 keep state label \"IPsec: 192.168.100.165 - inbound nat-t\" pass out on $WAN route-to ( le0 192.168.100.1 ) proto esp from any to 192.168.100.165 keep state label \"IPsec: 192.168.100.165 - outbound esp proto\" pass in on $WAN reply-to ( le0 192.168.100.1 ) proto esp from 192.168.100.165 to any keep state label \"IPsec: 192.168.100.165 - inbound esp proto\" # package manager late specific hook anchor "packagelate" anchor "limitingesr" # uPnPd anchor "miniupnpd"</virusprot></sshlockout></snort2c></snort2c></direct_networks></vpns></virusprot></snort2c></sshlockout> -
Which PBR is not functioning for you?
-
pass in log quick on $LAN route-to ( le1 192.168.80.250 ) proto { tcp udp } from ! 192.168.80.250 to 192.168.90.0/24 keep state label "USER_RULE: reroute replify"
Trying to redirect trafic that is going over the VPN to a wan accelerator from replify.
Replify has an option to use WCCP instead of PBR, that what we realy would like to do.
The GRE wccp is another thing? Not very familiar with wccp ;(
-
Hrrrmm that is something that i should look because it might be not working because of another fix.
WCCP even pfSense supports. Check out under Interface->assign->GRE
-
Ok thanx for your efforts, I'll just sniff my PBR for the time being ;)
WCCp was the main reason for us to try the 2.0.
Thing is I just do not understand WCCP :(
I rely would like to do it using WCCP2 instead of the PBR if possible. -
i didn't leave a tcpdump open and test it as ITR. The NIC drivers involved were 2x fxp and 2x em
I am back to 1.2.3-RC1 right now. i've also moved from real hardware to vmware so the ifconfig won't help much. i'll look at testing 2.0 again maybe this weekend and provide the rules.debug, ifconfig and routing table if it's still having issues. -
i can say as of the build from yesterday, this problem is no longer occurring. i am able to setup a pbr and clear the state table and it takes effect. here's an example of my one requiered pbr after clearing the state table.
00:25:21.711783 IP <voip box="">.5060 > <voip provider="">.5060: SIP, length: 362
00:25:21.771167 IP <voip provider="">.5060 > <voip box="">.5060: SIP, length: 433kudos guys!</voip></voip></voip></voip>
