Problem changing gateway through rules



  • hi,

    i have 2 locations with 2 PFsense router. Between these two site, there is a openvpn connection with routet subnets.

    Sitewan setup.png

    what i want.. Traffic coming in through wan1 on the upper pfsense routet through vpn and than enter dmz and back.

    I created all needed routes. upper pf can ping dmz net.
    I created a NAT rule for the needed port an upper PF with destination the ip in the dmz net.
    I created a additional gateway on the lower pf with ip address of the upper pf openvpn interfaceaddress.
    I created a rule on the lower pf with source dmz net and destination any. under advanced i choose the created gateway.

    now my problem:

    when i ping or traceroute from inside the dmz to the internet, everthing is fine. Traffic is routed through vpn and than to the internet and back.
    But when i access from the internet. traffic is routed through upper pf, through vpn to the dmz server, the traffic in the back direction is routed through the wan interface on the lower pfsense and so it isnt working.

    when i do a outsite nat on the upper pf for traffic to the dmz and nat the source interface, it is working, but than i cant see the puplic ip of the request.

    someone has a idea what is going wrong ?

    regards...

    DukyNuky



  • @dukynuky said in Problem changing gateway through rules:

    I created a additional gateway on the lower pf with ip address of the upper pf openvpn interfaceaddress.

    You should countermand that and assign interfaces to the OpenVPN instances at both sites instead. Just select the OpenVPN at "available network ports", hit "add", enable the new interface and enter a friendly name if you want.

    @dukynuky said in Problem changing gateway through rules:

    I created a rule on the lower pf with source dmz net and destination any. under advanced i choose the created gateway.

    That rule direct all traffic from the DMZ to the other side of the VPN. I'm in doubt if you really want that.
    For your intention that's not necessary.
    You need just a rule on the VPN interface to allow access to the DMZ.



  • hi,

    thanks.. i already created the interfaces...
    and yes.. i want all traffic from and to the dmz to be routed through the vpn tunnel.

    the main problem is that the lower pf routes the traffic releated to the routing table.. and default gateway for lower pf is the wan interface.. so the traffic entering the network from wan1 on upper pf is leaving through wan on lower pf and so the connection from extern cant be Establisht..



  • That's not the default behaviour if it is set up correctly.
    pfSense tags packets which are coming in on an interface with an assigned gateway with the "reply-to" flag. With the help of this it can direct response packets back to the correct gateway.

    Requirements:
    There must be a gateway defined on the incoming interface.
    The particular incoming traffic must not be allowed by an floating rule.

    So if your interfaces are set up properly check which rule allows the traffic.



  • yes thats correct...

    accept for one thing.. default gateway is the interfaceip of dmz net. default gateway for lower pf is wan1.

    it seems.. the reply to feature isnt working .. i did packet capture on wan interface on upper pf and saw the incoming traffic.
    same for the vpn interface.

    on lower pf the dmz interface.. i see the traffic from internet and the reply from the dmz server.. but the reply i only see it on the wan interface of the lower pf...

    the reply to feature isnt disabled in the advanced settings.. is there a way to check that ?



  • I've just tested that on my site-to-site OpenVPN, it works with no issues.

    Have you checked the firewall log? Can you see the log entry there for that connection? So again, check which rule allows it.



  • check rules.. simple allow any any rules on the interfaces. i removed gateway change on dmz firewall rule...
    now traffic from dmz just passes out wan interface on lower pfsense..

    when i connect to wan on upper pfsense.. i see traffic passing wan interface on upper pf.. then traffic passes openvpn interface on upper pf.. then i see traffic coming in on lower pf openvpn interface.. and leaving dmz interface..

    then i see back traffic incoming on dmz interface.. and then leaving through wan interface on lower pf..

    seems reply to isnt working.. how can i check for that ? is there a way ?



  • ok.. found out...

    i checked /tmp/rules.debug:
    anchor "userrules/*"
    pass in log quick on $OpenVPN inet from any to any tracker 1560717223 keep state label "USER_RULE"
    pass in log quick on $OpenVPN inet6 from any to any tracker 1560717223 keep state label "USER_RULE"

    changed:
    pass in log quick on $OpenVPN reply-to ( ovpnc1 10.10.10.1 ) inet from any to any tracker 1560717223 keep state label "USER_RULE"

    now it is working.. seems pfsense isnt creating the reply to rules.. :(

    edit: checked further..

    i have in my firewall rules one tap for the created openvpn interface and a tap called openvpn. All traffic is matcht against openvpn.. for the interface .. pfsense created a reply to rule.. but traffic passes openvpn tap ..

    can someone tell me why and why is there 2 taps for openvpn...

    edit2: i disabled the fw rule under openvpn.. now traffic matches the interface rules and it seems to be working...

    can someone tell me why there is this "global" openvpn tab ?

    thanks for everyone who tryed to help me :)

    regards

    dukynuky



  • Have you really done my very first recommendation?
    You have to assign interfaces to the particular OpenVPN instance (ovpnc?, ovpns?) and define the rules on these interface tabs.
    reply-to doesn't work on interface groups and the OpenVPN tab stands for the interface group including of all OpenVPN interfaces, even if you have only one!



  • First.. As i saed.. Problem solved...

    To your question... Yes i did it before as i saed... I had already created Interfaces for the ovpn Tunnel... But i didnt know that this openvpn tab in the Fw rules stands for a interface Group and that pfsense Do NOT create Reply to rules for that Group.. When you have a matching rule in the interface Group... The traffic will Pass like in my scenario and back traffic will not take the right way because of missing Reply to rule...

    When i add the rule by Hand for the interface Group.. Everything works.. Or Do it the way pfsense wants US to Do it and delete the rules in the interface Group and create rules per interface, so Reply to rules are created and it is working...

    Maybe it would be Nice to enable Reply to rule for an interface Group in the advanced settings of a rule there.. Or is there something special scenario why you cant handle it that way..

    In my scenario i Do not need special rules per openvpn connection and if i create a Reply to rule by Hand.. It is working too...

    Edit: very confusing is the Point "disable Reply to rules" in the interface Group rules... This is useless because pfsense will never create Reply to rules, no matter if you check or uncheck this Point



  • Yes, I didn't mention clearly enough, that the rule on the lower pfSense allowing the forwarded traffic from the other site has to be defined on the manually added interface.

    @dukynuky said in Problem changing gateway through rules:

    When i add the rule by Hand for the interface Group.. Everything works.. Or Do it the way pfsense wants US to Do it and delete the rules in the interface Group and create rules per interface, so Reply to rules are created and it is working...

    That may given by the Rule Processing Order: https://docs.netgate.com/pfsense/en/latest/firewall/firewall-rule-processing-order.html

    @dukynuky said in Problem changing gateway through rules:

    Maybe it would be Nice to enable Reply to rule for an interface Group in the advanced settings of a rule there.. Or is there something special scenario why you cant handle it that way..

    As mentioned above, the reply-to function depends on the gateway defined on an interface. Maybe it's cause an interface group my include multiple interfaces and so also may include multiple gateways, which cannot be handled by reply-to.


  • LAYER 8 Netgate

    @dukynuky said in Problem changing gateway through rules:

    pass in log quick on $OpenVPN reply-to ( ovpnc1 10.10.10.1 ) inet from any to any tracker 1560717223 keep state label "USER_RULE"
    now it is working.. seems pfsense isnt creating the reply to rules.. :(

    That will not survive filter rule rewrites.

    The traffic coming into the lower pfsense MUST NOT MATCH rules on the OpenVPN tab or the state will not get reply-to.

    The traffic must match the rules on the Assigned interface tab. If it matches a rule on the OpenVPN tab, those are processed first so the assigned interface rules will never be reached, and therefore no reply-to.

    All of this works. It just has to be configured correctly.

    I would just remove all rules from the OpenVPN tab and put the necessary rules on the appropriate assigned interface tab and never worry about it again. Some topologies support this method, some don't.


Log in to reply