Firewall blocks returning TCP traffic



  • Hello everyone!
    I have established LAN2LAN IPsec VPN in tunnel mode between two routers.On my side its pfSense 2.0.1, on the other - Cisco PIX (not sure about model).GRE over IPsec also established.All neccessary host in both LAN's can ping each other without any problems.But when it comes to TCP connections firewall is blocking returning TCP traffic on pfsense side.I turned on some rule logging and found asymmetric routing.I have read some explainations about this isue,but haven't found any solution yet.Maybe anyone have some ideas?



  • CAn you provide more details? Like what asymmetric routing issue you have found. What rules do you have in your IPSEC rule table?



  • About asymmetric routing http://forum.pfsense.org/index.php?topic=35400.0
    There's only one IPsec rule I have-allow everything from everyone.
    Also in my lan there is second pfsense box (192.168.0.16). So the network 10.0.0.0/8 is located behind it.On cisco side its network 192.168.230.33/28.
    Routing needed between them.
    IPsec on pfsense works in tunnel mode:local network 192.168.255.1/32, remote network 10.0.145.7/32.
    GRE tunnel:parent interface GRESOURCE (fwe0,192.168.255.1), remote address 10.0.145.7, tunnel local address 172.31.2.9,remote tunnel address 172.31.2.10, mobile tunnel.
    GRE gateway 172.31.2.10 on interface GRE(gre0)
    LAN gateway 192.168.0.16 on interface LAN(sk0,192.168.0.1)
    Routing to cisco network: to 192.168.230.33/28 through GRE gateway
    Routing to pfsense network:to 10.0.0.0/8 through LAN gateway.

    For example I made some logging rules in firewall and found that TCP initial connection from 192.168.230.45(cisco side) to 10.0.4.2:25(pfsense side) is going from LAN interface (192.168.0.1).But return traffic according to routing rules goes through GRE interface.THis is when the firewall blocks it.(Default rule). I suppose it can be possible to allow asymmetric routing in firewall config file…



  • Problem is solved!
    I managed to find firewall config file. Its located in /etc/inc/filter.inc
    It's not really config file,but it contains "Default deny rule" we need to find:
    block out $log on all label "Default deny rule"

    After few hours of experiments I decided to modify "Default deny rule" instead of adding "pass" rule above it.
    So this is the result: block out $log on ! gre2 label "Default deny rule without gre2"
    This modified rule affects all interfaces except gre2.
    So I tried to "telnet 192.168.0.17 25" and fortunately saw "ESMTP Ready" reply.
    Thanks everyone!



  • If you changed filter.inc, then on any update, the file will change back. It would be better to put a rule above it that allows traffic to gre2, or fix the asymmetric routing by adding the appropriate routes in the second pfsense machine and in the cisco at the remote location.
    I might be missing something but the local network and remote network should not be /32 unless you are only allowing one machine to access only one machine on the remote network. Which pfsense machine is establishing the tunnel?



  • Next update will overwrite filter.inc. I already notice it when changing some other files.I have disabled autoupdate in this case.
    What about pass rules…I tried to insert it above default deny rules,but haven't succeed. Traffic was blocked anyway. The rule was just like pass out $log on gre2 I'd like to set it properly,but not sure about syntax.Could you help me with it?

    I don't have any idea how to fix asymmetric routing.In order to route traffic from 10.0.0.0/8 to cisco side LAN it is neccessary to use GRE interface on the first pfsense as a gateway.And the traffic coming from cisco side to 10.0.0.0/8 have no other way to reach it but through LAN interface (192.168.0.1).I also have two other IPsec VPN connections in transport mode-the same problem with asymmetric traffic remains.I'm surprised I haven't found any posts with similar problems on the forum.Maybe I'm doing something wrong?

    About /32 networks. 10.0.145.7/32 in phase 2 settings is set as an address,not network.It is an address of remote tunnel end,given to me by network administrator on the cisco side.And 192.168.255.1 is an adddress of fwe0 interface,used as aparent interface for GRE.

    IPsec connection and GRE tunnel both established from pfsense1 which is connected directly to ISP. Second pfsense is used as internal router with transparent proxy and don't affect situation at all.For testing purposes I was connecting to 192.168.0.17:25 - SMTP server,connected to LAN interface of pfsense1 via switch. pfsense2 have IP 192.168.0.16.



  • Okay .. let me see if I can illustrate this.

    cisco net <–---> cisco <-----> (internet) <------> pfsense1 <------> main lan <------> pfsense2 <------> lan2

    You are trying to get from Lan2 to cisco net right? so on the cisco, the route needs to be setup to point lan2 subnet to pfsense2 WAN address ( the one in main lan ). pfsense1 should go ahead and handle the traffic to cisco net ... that is unless pfsense2 is not natting. in that case allow rules needs to be setup in pfsense 1 to allow that subnet access and a route should be setup in pfsense1 the same as in cisco. the WAN rule should have no effect on the traffic as it should be going through the tunnel and not out the WAN. Are you using OpenVPN or the ipsec tab?



  • I'm using IPsec tab.
    The thing is when I'm trying to reach cisco net from lan2 its working fine.Initial TCP connection starts not on GRE interface and firewall have no intentions to block it beacause default deny rule handles "out" traffic.But if connection starts on cisco side-pfsense fw blocks returning TCP packets from lan2 to cisco net on pfsense GRE interface.Adding allow rules to GRE interface isn't helping at all.



  • is there a route on the cisco to tell lan2 traffic go to pfsense2's main lan ipaddress? if not it will try to go out the internet. Which pfsense machine 1 or 2 is blocking? It should not have even made it from the cisco if there is not a route to push lan2 traffic through the vpn …



  • The only route set on cisco is:to lan2 via pfsense1 GRE(172.31.2.9).And next pfsense1 has static routing rule to forward traffic to lan2 via pfsense2 WAN ipaddress.Blocking happens on pfsense1 because IPsec and GRE tunnels ends on it.There's a smart network administrator on a cisco side.He is smart enough to make proper routing through VPN.The trouble appears on my pfsense side,becaus I'm not good in VPN building  8)



  • System>Advanced, Firewall/NAT, check " Bypass firewall rules for traffic on the same interface ". That'll work around it for the vast majority of cases, not enough of a diagram type picture of the network to tell 100% for sure if it will here but it probably will.



  • @cmb:

    System>Advanced, Firewall/NAT, check " Bypass firewall rules for traffic on the same interface ". That'll work around it for the vast majority of cases, not enough of a diagram type picture of the network to tell 100% for sure if it will here but it probably will.

    I'm not sure this will help.Log shows that traffic goes in through one interface and leaves through another.And the description of this option tells about traffic entering and leaving on the SAME interface.Anyway, thanks for advice.I will try it as soon as possible.



  • seeing the specific routes and such would be nice.
    I used to use ipsec for a site to site VPN. I switched to openvpn mainly because the ipsec kept dropping and was a bit slower than openvpn. We have been using openvpn for about 1.5 years and have been really happy with it.I have a similar setup, but I don't use Cisco. It is 3 pfsense FWs. One at the DC, 2 at the office. 1 of those is protecting a lab (rather protecting us from the lab). routing to the lab didn't work until I added a static route on the DC FW to point to the WAN ip of LAB FW (basic firewall rules to allow LAN and DC subnets with no NAT to those from the LAN or DC subnets). I don't know why you would have any blocks going on at all. If you are getting blocks, then there might be something wrong with the rule in ipsec because this is the only place to block. If it gets there it should be past the WAN at that point.



  • @podilarius:

    seeing the specific routes and such would be nice.
    I used to use ipsec for a site to site VPN. I switched to openvpn mainly because the ipsec kept dropping and was a bit slower than openvpn. We have been using openvpn for about 1.5 years and have been really happy with it.I have a similar setup, but I don't use Cisco. It is 3 pfsense FWs. One at the DC, 2 at the office. 1 of those is protecting a lab (rather protecting us from the lab). routing to the lab didn't work until I added a static route on the DC FW to point to the WAN ip of LAB FW (basic firewall rules to allow LAN and DC subnets with no NAT to those from the LAN or DC subnets). I don't know why you would have any blocks going on at all. If you are getting blocks, then there might be something wrong with the rule in ipsec because this is the only place to block. If it gets there it should be past the WAN at that point.

    Are you sure that your traffic from DC to the lab is really encrypted?I think when you made a static route to the lab WAN ipaddress your DC pfsense use Internet connection instead of OpenVPN tunnel.To encapsulate traffic to the lab in OpenVPN tunnel you have to create gateway on DC pfsense with ipaddress of OpenVPN interface of lab pfsense. And then create a static route to lab subnet through this gateway.Am I wrong? I also using OpenVPN on pfsense2 with routing settings just like I wrote above.And its working quite fine.

    In my case there was no other choice between OpenVPN or IPSec.The organization with cisco is a big GSM provider.It's network administrator didn't even try to discuss any other VPN techs but GRE-over-IPsec.I was building VPN with two large GSM providers and both were insisting on IPsec.

    Back to the question of asymmetric routing.I really don't know how it happens.In firewall rules IPsec tab,Lan tab,GRE tab-on all neccessary interfaces I set rule "allow everything from everyone on every protocol" and traffic was blocked anyway.After a few hours of googling I found similar problem on one forum (don't remeber where exactly,maybe even here).At that topic the guy had the same GRE-over-IPsec and TCP blocking.This is when I started to dig about asymmetric routing.I'm glad it ends.I was fighting it since last october :)



  • I am very sure … I am using private IPs and is not internet routable. They have no choice but to go over the VPN.
    So here is what I have and it works.

    DB NEtwork 10.a.b.0/24
    DC route -> 10.b.c.0/24 GW 10.a.a.15 (for DC traffic)

    Office Net 10.a.a.0/24
    Office Route -> 10.b.c.0/24 GW 10.a.a.15 (for LAN traffic)

    I can get all the way from either direction.



  • @podilarius:

    I am very sure … I am using private IPs and is not internet routable. They have no choice but to go over the VPN.
    So here is what I have and it works.

    DB NEtwork 10.a.b.0/24
    DC route -> 10.b.c.0/24 GW 10.a.a.15 (for DC traffic)

    Office Net 10.a.a.0/24
    Office Route -> 10.b.c.0/24 GW 10.a.a.15 (for LAN traffic)

    I can get all the way from either direction.

    Alright I understand. Thanks for your help!



  • Hi

    I've been having this problem as well, the only solution I found was to create a floating firewall rule for the gre interface to pass out any traffic, make sure the quick option is ticked and also set keep-state to none on the advanced options. To be honest I don't know if this is the best solution or really why it works, but it has on my setup. I've downloaded some capture files on the interfaces to go through see if I better understand it. But hope this helps.



  • @tonyw:

    Hi

    I've been having this problem as well, the only solution I found was to create a floating firewall rule for the gre interface to pass out any traffic, make sure the quick option is ticked and also set keep-state to none on the advanced options. To be honest I don't know if this is the best solution or really why it works, but it has on my setup. I've downloaded some capture files on the interfaces to go through see if I better understand it. But hope this helps.

    I'm surprized you managed to solve this problem using pfsense GUI.I also tried floating rules,but it didn't help in my case.



  • Just an FYI. I just realized that the DC firewall is not 2.0.x. It is still 1.2.3 which allows you to put in a route through a gateway that is not on the interface.
    This saddens me. It is a valid configuration, but, I am using OpenVPN, so I can just push the route through the client. :)





  • @dhatz:

    So to summarize, is GRE-over-IPsec between Cisco and pfsense 2.0.1 configurable from webGUI ?

    Yes.


Log in to reply