Update pfsense 2.0.1 stable to 2.1 problem with routes
-
I can also confirm that upgrading from 2.0.3 to 2.1 release has a routing issue. The LAN interface appears to route traffic fine but other interfaces only route to the default gateway. I've confirmed that the router ignores its static routes and locally connected subnets (for interfaces other than LAN). It just sends everything to the default gateway. All the interfaces in this setup are vlan interfaces (not sure if that matters). I will reiterate that all of this worked prior to upgrading to 2.1 release.
-
I had the same problem while upgrading from 2.0.1 to 2.1.0. After some digging on the subject I've realized that the problem is probably associated with <negate_networks>table. Quick comparison of /etc/inc/filter.inc shows that version 2.0.1 inserts directly connected networks into <negate_networks>table:
/* add a Negate_networks table */ $natrules .= "table <negate_networks>{"; if($direct_networks_list) $natrules .= " $direct_networks_list "; if($vpns_list) $natrules .= " $vpns_list "; $natrules .= "}\n";</negate_networks>while in 2.1.0 this piece of code is missing:
/* add a Negate_networks table */ $aliases .= "table <negate_networks>"; if($vpns_list) $aliases .= "{ $vpns_list }"; $aliases .= "\n";</negate_networks>I suspect that this is the cause of problem. The change isn't mentioned at "New Feature and Changes" (https://doc.pfsense.org/index.php/2.1_New_Features_and_Changes#NAT.2FFirewall_Rules.2FAlias) but one can see that it was made by user ermal on Nov 16, 2012 (https://github.com/pfsense/pfsense/commit/b4227df690fb7a989ead9b3928ebaaaa34b495eb
Edit: To clarify the role of <negate_networks>: pfSense is using this table in auto created rules that are connected to user rules with policy routing.
Regards,
John</negate_networks></negate_networks></negate_networks> -
Great job John!
I think we have just to wait and see… and hope some developer take care of it...
I had the same problem while upgrading from 2.0.1 to 2.1.0. ….
why "had"?
How did you solve it? Maybe replacing code on filter.inc? -
We do not know the reason of modifying <negate_networks>so it's probably better to add rule using GUI. For example if you have custom rule:
pass in quick on $LAN $GWFailOverGW from $AllowedLocalNets to any keep state label "USER_RULE: allow ... "add the following one just BEFORE above rule:
pass in quick on $LAN from $AllowedLocalNets to $DirectlyConnectedNets keep state label "My negate policy routing for destination"where
$AllowedLocalNets and $DirectlyConnectedNets are custom aliases.Regards,
John</negate_networks> -
why "had"?
How did you solve it? Maybe replacing code on filter.inc?I've simply reverted to redundant live copy of 2.0.1 release. English isn't my native language, so forgive me for any inaccuracy.
Regards,
john -
How did you solve it? Maybe replacing code on filter.inc?
ok - I've booted 2.1 release again, but I have plenty of rules with policy routing - so I ended with this quick (but very ugly) fix involving /etc/inc/filter.inc modification:
# diff /etc/inc/filter.inc /etc/inc/filter.inc_org 591,595d590 < /* xxxxxx 20131102 */ < $my_direct_nets_list = "192.168.1.0/24 172.16.1.0/24"; < if($my_direct_nets_list) < $aliases .= "table <my_direct_nets>{ $my_direct_nets_list }\n"; < 600,602d594 < /* xxxxxx 20131102 add direct nets */ < if($my_direct_nets_list) < $aliases .= "{ $my_direct_nets_list }";</my_direct_nets>Regards
John -
Great job John!
I think we have just to wait and see… and hope some developer take care of it...
....Indeed I was thinking….
As I stated here http://forum.pfsense.org/index.php/topic,68031.msg375663.html#msg375663
my problem is for sure due to multiwan gw...I "diffed" /etc/inc/filter.inc 2.1-RC1 (working) and 2.1-Release... looking for differences affecting vpns and/or gw... And I found out this:
2.1-RC1/* exception(s) to a user rules can go here. */ /* rules with a gateway or pool should create another rule for routing to vpns */ if((($aline['route'] <> "") && (trim($aline['type']) == "pass") && strstr($dst, "any")) && (!isset($config['system']['disablenegate']))) { /* negate VPN/PPTP/PPPoE/Static Route networks for load balancer/gateway rules */ $negate_networks = " to <negate_networks>";</negate_networks>2.1-Release
/* exception(s) to a user rules can go here. */ /* rules with a gateway or pool should create another rule for routing to vpns */ if((($aline['route'] <> "") && (trim($aline['type']) == "pass") && strstr($dst, "any")) && (!isset($config['system']['disablenegate']))) { /* negate VPN/PPTP/PPPoE/Static Route networks for load balancer/gateway rules */ $negate_networks = " to <negate_networks>" . filter_generate_port($rule, "destination");</negate_networks>But I do not think that's the issue…
Maybe I can try to modify filter.inc and see...
-
$negate_networks = " to <negate_networks> " . filter_generate_port($rule, "destination");</negate_networks>From memory, the filter_generate_port code was added because before that, a rule that fed traffic for a particular port(s) into a gateway group would result in a wider pass rule to Negate_Networks that passed traffic on any port. So this is actually a good fix to tighten up the behaviour.
-
Here is my take on the situation, using an example config like:
LAN1: 10.42.1.0/24
LAN2: 10.42.2.0/24
WAN1: some public IP
WAN2: some other public IP
OpenVPN tunnel 10.99.1.0/24 to a remote LAN 10.99.2.0/24
Gateway group: LoadBalance - WAN1 and WAN2 on tier1.User rule on LAN1: Pass source LAN1 subnet destination any gateway LoadBalance
The "negate_networks" thing in 2.0.n is saying:
"Oh no, a rule like that cannot possibly be what you want, because it just won't work to stuff traffic for LAN2, the OpenVPN tunnel and remote LAN into the LoadBalance gateway group. You (the sysadmin) have said you are happy to pass to destination any, so I (pfSense code) will help you by putting an extra rule in place that passes traffic for destination LAN2, OpenVPN tunnel and remote LAN without putting it into LoadBalance gateway group."Generally this is a good thing - traffic to destination any actually gets to its destination. Some across a VPN, some delivered locally direct to LAN2 and some fed into LoadBalance gateway group.
In 2.1 this "helping" is no longer done for locally connected networks (LAN2), so now it says:
"Oh no, a rule like that cannot possibly be what you want, because it just won't work to stuff traffic for the OpenVPN tunnel and remote LAN into the LoadBalance gateway group. You (the sysadmin) have said you are happy to pass to destination any, so I (pfSense code) will help you by putting an extra rule in place that passes traffic for destination OpenVPN tunnel and remote LAN without putting it into LoadBalance gateway group."So, in 2.1, the user rules need to actually say what traffic they want to pass between local LANs - nothing comes "for free" behind the scenes. Before a rule that feeds "destination any" traffic into a particular gateway or gateway group, you need to put any rules to pass traffic to other local LANs. It seems to me that this is a change in behaviour from 2.0.n to 2.1 and people with multi-LAN installs need to be aware of it and add user rules to their LAN(s) appropriately.
It is probably a good thing, as now in 2.1 you can use "destination any" into a gateway (group) and not also open up traffic from LAN1 to LAN2… But I wonder why it does not go further and stop doing the negate_networks for VPN? Maybe that would create too many issues for users who don't understand their rules? Pass rules on LANn to destination "VPN-networks" would have to be added before "destination any" rules into a gateway (group), people would feed "destination any" into a gateway (group) and then wonder why their VPN traffic died.
-
Really sorry but… I do not understand....
In 2.1 this "helping" is no longer done for locally connected networks (LAN2), so now it says:
"Oh no, a rule like that cannot possibly be what you want, because it just won't work to stuff traffic for the OpenVPN tunnel and remote LAN into the LoadBalance gateway group. You (the sysadmin) have said you are happy to pass to destination any, so I (pfSense code) will help you by putting an extra rule in place that passes traffic for destination OpenVPN tunnel and remote LAN without putting it into LoadBalance gateway group."If I do not get wrong With this "pfSense sw help" I should be able to get traffic pass between VPN LANs but not from LAN1 to LAN2 (LAN to DMZ in my case: indeed I do not remember but it seems to me like it was ok…)
Instead I can not pass traffic through mainsite LAN and VPN2 remote LAN (VPN1 is ok) .
But I guess, There's something simiilar in my case… because as I stated here http://forum.pfsense.org/index.php/topic,68031.15.html it seems pfsense tries "routing/loadbalancing" traffic directed to VPN2 remote LAN using default gw.So, in 2.1, the user rules need to actually say what traffic they want to pass between local LANs - nothing comes "for free" behind the scenes. Before a rule that feeds "destination any" traffic into a particular gateway or gateway group, you need to put any rules to pass traffic to other local LANs. It seems to me that this is a change in behaviour from 2.0.n to 2.1 and people with multi-LAN installs need to be aware of it and add user rules to their LAN(s) appropriately.
Correct me if i get wrong, but I doubt this is 2.1 feature… as Everything works flawless on 2.1-RCX otherwise it should not work also on BETA, RC0, RC1 etc.
Thanks
-
I should be able to get traffic pass between VPN LANs but not from LAN1 to LAN2
Yes, that is the way the code appears to work. So maybe there is something else also going on. The way to track it down is to look in /tmp/rules.debug and see:
a) What networks get included in <negate_networks>.
b) What rules direct traffic into a gateway or gateway group.
c) Which of those rules also have a pass rule above to allow traffic straight through to <negate_networks>.If you want to post your /tmp/rules.debug and an overview of which subnets are LAN1, LAN2, VPN tunnels, at the other end of a VPN etc, then it should be possible to analyse what is going on, and then find the code that is generating an unexpected/unwanted ruleset.</negate_networks></negate_networks>
-
If you want to post your /tmp/rules.debug and an overview of which subnets are LAN1, LAN2, VPN tunnels, at the other end of a VPN etc, then it should be possible to analyse what is going on, and then find the code that is generating an unexpected/unwanted ruleset.
Thank you… you have opened my eyes!!!
Those are the differences on /tmp/rules.debug
2.1-RC1 (online: production fw)
.... table <vpn_networks>{ 10.106.100.0/24 192.168.11.0/24 10.106.100.0/24 192.168.12.0/24 10.116.100.0/24 192.168.12.0/24 192.168.21.0/24 10.106.100.0/24 192.168.12.0/24 10.116.100.0/24 192.168.12.0/24 } table <negate_networks>{ 10.106.100.0/24 192.168.11.0/24 10.106.100.0/24 192.168.12.0/24 10.116.100.0/24 192.168.12.0/24 192.168.21.0/24 10.106.100.0/24 192.168.12.0/24 10.116.100.0/24 192.168.12.0/24 } ....</negate_networks></vpn_networks>2.1-RELEASE (offline: test machine with just lan netif up)
... table <vpn_networks>{ 10.116.100.0/24 192.168.12.0/24 192.168.21.0/24} table <negate_networks>{ 10.116.100.0/24 192.168.12.0/24 192.168.21.0/24} ....</negate_networks></vpn_networks>maybe (maybe!! ;) ) RC1 is a lil'bit redundant… but it works...
Hence This is the final proof: That's a bug.
Tomorrow I'll try to be "verbose"... -
Hmm, interesting stuff. As I think I said earlier I have 'negate networks' unchecked and added my own rules. Probably why I've not seen any issues.
Are you sure this is an issue with what ends up in the <negate_networks>table rather than what rules are generated from it?
Steve</negate_networks>
-
Hmm, interesting stuff. As I think I said earlier I have 'negate networks' unchecked and added my own rules. Probably why I've not seen any issues.
Are you sure this is an issue with what ends up in the <negate_networks>table rather than what rules are generated from it?
Steve</negate_networks>
As you can see above, probably, the issue is due to the <vpn_networks>because It does not list all vpn networks (e.g. there isn't 10.106.100.0) and (I think) <negate_networks>is made up of <vpn_networks>+other stuff…</vpn_networks></negate_networks></vpn_networks>
-
From my reading of the code, <negate_networks>now only contains the same as <vpn_networks>- other stuff is no longer put in it.
What is special about the VPN networks that are not listed in the 2.1-RELEASE rules.debug?
Are they not listed in "IPv4 Remote Network/s" box in the server/client config?The Remote Network/s box can now have a comma-separated list of networks. I just discovered that if I have more than 1 network there, then none of them appear in vpn_networks. If I cut it down to just 1 network, then it appears in vpn_networks. That is a bug. Got to go right now, back can look at it in a day or so, or maybe someone else can look at the code before then…</vpn_networks></negate_networks>
-
In my conf, I specify 2nd remote networks in OpenVPN server "advanced configuration" field as "in the previous (old) way", and hence this is because I can get just the first VPN to work.
Fake care of the following:( in my conf) disabling multiwan get both vpn to work!Meantime I submitted a bug on https://redmine.pfsense.org/issues/3309
-
The Remote Network/s box can now have a comma-separated list of networks. I just discovered that if I have more than 1 network there, then none of them appear in vpn_networks. If I cut it down to just 1 network, then it appears in vpn_networks. That is a bug. Got to go right now, back can look at it in a day or so, or maybe someone else can look at the code before then…
https://github.com/pfsense/pfsense/pull/850 fixes the problem when there is a comma-separated list in "Remote Network(s)".
In my conf, I specify 2nd remote networks in OpenVPN server "advanced configuration" field as "in the previous (old) way"
Hmmm - perhaps there used to be code that found the entries in the advanced configuration also? But I don't think so. Anyway, with the fix above you will be able to put lists of Remote Network(s) and have it come through correctly to vpn_networks and negate_networks.
-
The Remote Network/s box can now have a comma-separated list of networks. I just discovered that if I have more than 1 network there, then none of them appear in vpn_networks. If I cut it down to just 1 network, then it appears in vpn_networks. That is a bug. Got to go right now, back can look at it in a day or so, or maybe someone else can look at the code before then…
https://github.com/pfsense/pfsense/pull/850 fixes the problem when there is a comma-separated list in "Remote Network(s)".
In my conf, I specify 2nd remote networks in OpenVPN server "advanced configuration" field as "in the previous (old) way"
Hmmm - perhaps there used to be code that found the entries in the advanced configuration also? But I don't think so. Anyway, with the fix above you will be able to put lists of Remote Network(s) and have it come through correctly to vpn_networks and negate_networks.
Thanks Phil.
I fixed the problem on filter.inc on pfSense test server and now vpn_networks and negates_networks seem ok (using comma separated remote network on vpn_server config).
Now I should to test it on production…. I hope to be able to do it on next monday....
I'll let you know... -
After upgrading to 2.1 I encountered routes problem.
My configuration is:
2 WAN:
- WAN
- WAN_VOIP
1 DMZ with:
- 1 web server host forced to use WAN gw (WAN gw specified in DMZ firewall rule)
- 1 asterisk host forced to use WAN VOIP gw (WAN_VOIP gw specified in DMZ firewall rule)2 LAN:
- LAN forced to to use WAN VOIP (WAN GW specified in LAN FW rule)
- LAN_VOIP forced to use WAN (WAN VOIP GW specified in LAN_VOIP FW rule)
After upgrade to 2.1: no connection from internet to DMZ asterisk, DMZ webserver
At this time I downgraded to 2.03
Thanks
–
Francesco -
I solved this problem. In the version pfsense 2.1 stable, also add static routes, you also have to add in paragraph rules to allow traffic to that route.
This well because security have gotten more but if you do not know the routes stop working when updating from 2.0.1 to 2.1 pfsense
Thanks for all.
