Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Cannot NAT trough OPT1 interface on multiwan

    Scheduled Pinned Locked Moved 2.2 Snapshot Feedback and Problems - RETIRED
    49 Posts 18 Posters 16.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • V
      villavai
      last edited by

      i cannot give the debug because I've already revert back to 2.1.5, and since pf is my only fw, and is being use in production site, so i hope someone kindly test it in 2.2 latest snapshot to give the answer is this problem still exist or not.

      thank you.

      1 Reply Last reply Reply Quote 0
      • N
        naras
        last edited by

        @villavai:

        i cannot give the debug because I've already revert back to 2.1.5, and since pf is my only fw, and is being use in production site, so i hope someone kindly test it in 2.2 latest snapshot to give the answer is this problem still exist or not.

        thank you.

        Unfortunately, it's still there.

        1 Reply Last reply Reply Quote 0
        • E
          eSpezi
          last edited by

          Hi everybody,

          this is not only a NAT problem.
          It seems that the routing in general does not work properly with multiwan. (in my case WAN_KD=cable router, WAN_TO=PPPoE modem)
          I'm running the Oct 03 10:14:50 CDT 2014 BETA, but also had the same problem with a previous version (Oct 01).

          I can't even access the pfS Webgui from the WAN_TO side, although in the firewall log the request is shown as PASS.

          My config:
          set optimization normal
          set timeout { adaptive.start 0, adaptive.end 0 }
          set limit states 198000
          set limit src-nodes 198000

          #System aliases

          loopback = "{ lo0 }"
          WAN_KD = "{ re0 }"
          LAN = "{ re2 }"
          WAN_TO = "{ pppoe0 }"
          OpenVPN = "{ openvpn }"

          #SSH Lockout Table
          table <sshlockout>persist
          table <webconfiguratorlockout>persist
          #Snort tables
          table <snort2c>table <virusprot>table <bogons>persist file "/etc/bogons"
          table <vpn_networks>{ 192.168.254.0/24 }
          table <negate_networks>{ 192.168.254.0/24 }

          User Aliases

          admin_ports = "{  456  4562 }"
          ovpn_ports = "{  1194  1199 }"
          table <ovpn_remote_networks>{  192.168.254.0/24  192.168.222.0/24 }
          OVPN_Remote_Networks = "<ovpn_remote_networks>"
          table <rfc1918_private_networks>{  10.0.0.0/8  172.16.0.0/12  192.168.0.0/16 }
          RFC1918_Private_Networks = "<rfc1918_private_networks>"
          table <server>{  172.16.100.100 }
          Server = "<server>"

          Gateways

          GWWAN_KD_GW = " route-to ( re0 192.168.0.1 ) "
          GWWAN_TO_PPPOE = " route-to ( pppoe0 XXX.WAN.TO.GW ) "
          GWGWG_LAN = "  route-to { ( pppoe0 XXX.WAN.TO.GW )  }  "

          set loginterface re2

          set skip on pfsync0

          scrub on $WAN_KD all    fragment reassemble
          scrub on $LAN all    fragment reassemble
          scrub on $WAN_TO all    fragment reassemble

          no nat proto carp
          no rdr proto carp
          nat-anchor "natearly/"
          nat-anchor "natrules/          "

          Outbound NAT rules (automatic)

          Subnets to NAT

          tonatsubnets = "{ 127.0.0.0/8 172.16.0.0/16 192.168.254.0/24 }"
          nat on $WAN_KD  from $tonatsubnets to any port 500 -> 192.168.0.2/32  static-port
          nat on $WAN_KD  from $tonatsubnets to any -> 192.168.0.2/32 port 1024:65535 
          nat on $WAN_TO  from $tonatsubnets to any port 500 -> XXX.WAN.TO.IP/32  static-port
          nat on $WAN_TO  from $tonatsubnets to any -> XXX.WAN.TO.IP/32 port 1024:65535

          Load balancing anchor

          rdr-anchor "relayd/*"

          TFTP proxy

          rdr-anchor "tftp-proxy/*"

          NAT Inbound Redirects

          rdr on re0 proto tcp from any to 192.168.0.2 port 443 -> $Server
          rdr on re0 proto tcp from any to 192.168.0.2 port 8182 -> $Server port 5900

          UPnPd rdr anchor

          rdr-anchor "miniupnpd"

          anchor "relayd/"
          anchor "openvpn/          "
          anchor "ipsec/*"

          Block all IPv6

          block in log quick inet6 all tracker 1000000001 label "Block all IPv6"
          block out log quick inet6 all tracker 1000000002 label "Block all IPv6"
          #–-------------------------------------------------------------------------

          default deny rules

          #---------------------------------------------------------------------------
          block in log inet all tracker 1000000101 label "Default deny rule IPv4"
          block out log inet all tracker 1000000102 label "Default deny rule IPv4"
          block in log inet6 all tracker 1000000103 label "Default deny rule IPv6"
          block out log inet6 all tracker 1000000104 label "Default deny rule IPv6"

          IPv6 ICMP is not auxilary, it is required for operation

          See man icmp6(4)

          1    unreach        Destination unreachable

          2    toobig          Packet too big

          128  echoreq        Echo service request

          129  echorep        Echo service reply

          133  routersol      Router solicitation

          134  routeradv      Router advertisement

          135  neighbrsol      Neighbor solicitation

          136  neighbradv      Neighbor advertisement

          pass  quick inet6 proto ipv6-icmp from any to any icmp6-type {1,2,135,136} tracker 1000000105 keep state

          Allow only bare essential icmpv6 packets (NS, NA, and RA, echoreq, echorep)

          pass out  quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {129,133,134,135,136} tracker 1000000106 keep state
          pass out  quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {129,133,134,135,136} tracker 1000000107 keep state
          pass in  quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000108 keep state
          pass in  quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000109 keep state
          pass in  quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {128,133,134,135,136} tracker 1000000110 keep state

          We use the mighty pf, we cannot be fooled.

          block log quick inet proto { tcp, udp } from any port = 0 to any tracker 1000000111
          block log quick inet proto { tcp, udp } from any to any port = 0 tracker 1000000112
          block log quick inet6 proto { tcp, udp } from any port = 0 to any tracker 1000000113
          block log quick inet6 proto { tcp, udp } from any to any port = 0 tracker 1000000114

          Snort package

          block log quick from <snort2c>to any tracker 1000000115 label "Block snort2c hosts"
          block log quick from any to <snort2c>tracker 1000000116 label "Block snort2c hosts"

          SSH lockout

          block in log quick proto tcp from <sshlockout>to (self) port 4562 tracker 1000000301 label "sshlockout"

          webConfigurator lockout

          block in log quick proto tcp from <webconfiguratorlockout>to (self) port 456 tracker 1000000351 label "webConfiguratorlockout"
          block in log quick from <virusprot>to any tracker 1000000400 label "virusprot overload table"

          block bogon networks (IPv4)

          http://www.cymru.com/Documents/bogon-bn-nonagg.txt

          block in log quick on $WAN_KD from <bogons>to any tracker 1000001551 label "block bogon IPv4 networks from WAN_KD"
          antispoof log for $WAN_KD tracker 1000001570
          antispoof log for $LAN tracker 1000002620
          antispoof log for $WAN_TO tracker 1000003670

          loopback

          pass in  on $loopback inet all tracker 1000004761 label "pass IPv4 loopback"
          pass out  on $loopback inet all tracker 1000004762 label "pass IPv4 loopback"
          pass in  on $loopback inet6 all tracker 1000004763 label "pass IPv6 loopback"
          pass out  on $loopback inet6 all tracker 1000004764 label "pass IPv6 loopback"

          let out anything from the firewall host itself and decrypted IPsec traffic

          pass out  inet all keep state allow-opts tracker 1000004765 label "let out anything IPv4 from firewall host itself"
          pass out  inet6 all keep state allow-opts tracker 1000004766 label "let out anything IPv6 from firewall host itself"
          pass out  route-to ( re0 192.168.0.1 ) from 192.168.0.2 to !192.168.0.0/24 tracker 1000004861 keep state allow-opts label "let out anything from firewall host itself"
          pass out  route-to ( pppoe0 XXX.WAN.TO.GW ) from XXX.WAN.TO.IP to !XXX.WAN.TO.IP/32 tracker 1000004862 keep state allow-opts label "let out anything from firewall host itself"

          User-defined rules follow

          anchor "userrules/*"
          pass  in  quick  on $OpenVPN inet from $OVPN_Remote_Networks to 172.16.0.0/16 tracker 1412417430 keep state  label "USER_RULE"
          pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 ) inet proto tcp  from any to 192.168.0.2 port $admin_ports tracker 1412198572 flags S/SA keep state  label "USER_RULE"
          pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 ) inet proto udp  from any to 192.168.0.2 port $ovpn_ports tracker 1412415560 keep state  label "USER_RULE"
          pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 )  proto tcp  from any to $Server port 443 flags S/SA keep state  label "USER_RULE: NAT "
          pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 )  proto tcp  from any to $Server port 5900 flags S/SA keep state  label "USER_RULE: NAT "
          pass  in  quick  on $LAN inet proto tcp  from any to 172.16.100.1 port $admin_ports tracker 1412207357 flags S/SA keep state  label "USER_RULE"
          pass  in  quick  on $LAN inet from 172.16.0.0/16  to <negate_networks>tracker 0100000101 keep state  label "NEGATE_ROUTE: Negate policy routing for destination"
          pass  in  quick  on $LAN  $GWGWG_LAN inet from 172.16.0.0/16 to any tracker 0100000101 keep state  label "USER_RULE: Default allow LAN to any rule"
          pass  in  quick  on $WAN_TO reply-to ( pppoe0 XXX.WAN.TO.GW ) inet proto tcp  from any to XXX.WAN.TO.IP port $admin_ports tracker 1412448206 flags S/SA keep state  label "USER_RULE"

          VPN Rules

          anchor "tftp-proxy/*"

          Any news on the issue?

          Thanks,
          Harry</negate_networks></bogons></virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c></server></server></rfc1918_private_networks></rfc1918_private_networks></ovpn_remote_networks></ovpn_remote_networks></negate_networks></vpn_networks></bogons></virusprot></snort2c></webconfiguratorlockout></sshlockout>

          1 Reply Last reply Reply Quote 0
          • M
            mikeisfly
            last edited by

            I can confirm, I am having the same issue using the Oct 4th 64bit build. I can provide configurations if needed but I know I have configured everything correctly and the firewall blocks show no blocks. Downgrading back to 2.1.5 fixes the issues with the exact same configuration file. Just to give an over view of what I did.

            created a rule under lan to have my plex machine use the second WAN.
            Created a NAT (port Forward for TCP 32400) to plex machine
            Rule was automatically generated.

            Some fixes that I have found:

            Changing default gateways to second WAN (Puts everything on second WAN), Not the desired behavior
            Downgrading to 2.1.5 (everything works as expected)

            I have this running on my home router so I can flash firmware again if Needed.

            1 Reply Last reply Reply Quote 0
            • E
              eri--
              last edited by

              Actually seems that the one that have posted here have some issues in their configuration.

              They have set a generic policy routing rule on their lan which overrides the reply-to(NAT of their OPTx) tha would make things work.

              1 Reply Last reply Reply Quote 0
              • E
                eSpezi
                last edited by

                Hi Ermal,

                on my Dual WAN (WAN1 = Cable Router, WAN2 = PPPoE Modem) installation I successfully can ping each WAN IF.
                But I only can reach pfSenses WebGui or anything else on the current Default Gateway.
                By current I mean, that if the default Default Gateway (WAN1) is marked as down (unfortunately still way too often false alarm by apinger) then I am able to reach the WebGui via WAN2.

                I followed your suggestions, switched to manual outbound and added the propsed rule on top of everything, but that doesn't change anything.

                Has anybody had success with this workaround?
                Any other suggestions to make it work?

                Thanks,
                Harry

                Edit:
                Ermal in my case, you're right. I've policy routing on lan to a failover gateway group. That same scenario used to work fine with 2.15

                1 Reply Last reply Reply Quote 0
                • M
                  mattbunce
                  last edited by

                  Excuse my ignorance, but I can't quite figure out if there is a resolution to this yet? I see from Gloom this should be working if you set manual NAT rules on the OPT1 interface (not sure exactly how to, or what this means?).

                  Also, ermal suggests that the problem is that we're using a "generic policy routing rule" - and this is overriding the reply-to. Again, I'm not exactly sure what is meant by "generic policy routing rule" and if there's anything we can do to work around this?

                  I have run Wireshark on the NAT target machine, and whilst I don't understand what I'm looking at, you can clearly see that the http request I am making to the NAT forwarded port (from an external network) is hitting the target machine, so it does seem to be the reply that is getting lost somewhere.

                  My setup

                  WAN = DHCP client
                  OPT1 = PIA VPN

                  I have set-up forwarding via Firewall > NAT and manually created a floating rule for TCP/UDP traffic on OPT1 coming in the my target IP.

                  I have two LAN rules, the first is for target machine and sets the default gateway as OPT1. The second is for all devices and has the default gateway as the router default (which would be WAN).

                  The floating rule seems to make no difference, I see the traffic within Wireshark whether or not the floating rule is enabled.

                  1 Reply Last reply Reply Quote 0
                  • C
                    cmb
                    last edited by

                    It's not working, none of the workarounds are worth trying (short of for experimentation purposes if you want). This is the bug:
                    https://redmine.pfsense.org/issues/3760

                    1 Reply Last reply Reply Quote 0
                    • M
                      mattbunce
                      last edited by

                      Thanks cmb - That saves me spending any more time on this for now!

                      1 Reply Last reply Reply Quote 0
                      • E
                        eri--
                        last edited by

                        Please try a next coming snapshot it should be fixed.

                        1 Reply Last reply Reply Quote 0
                        • E
                          eSpezi
                          last edited by

                          Hi Ermal,

                          thanks a lot for addressing the issue!
                          Is the fix already in the Wed Oct 15 14:37:51 CDT 2014 Build?

                          Cheers,
                          Harry

                          1 Reply Last reply Reply Quote 0
                          • E
                            eri--
                            last edited by

                            Yeah it should be on that build or newer.

                            1 Reply Last reply Reply Quote 0
                            • M
                              mattbunce
                              last edited by

                              Unfortunately, this still doesn't seem to be working for me. I've tried used the automatically generated linked rule and manually creating a floating rule, but nomatter what I try it is still not working for me on the October 16th 09:56:37 version.

                              Is there anything else I should be looking at?

                              1 Reply Last reply Reply Quote 0
                              • P
                                phil.davis
                                last edited by

                                @mattbunce:

                                Unfortunately, this still doesn't seem to be working for me. I've tried used the automatically generated linked rule and manually creating a floating rule, but nomatter what I try it is still not working for me on the October 16th 09:56:37 version.

                                Is there anything else I should be looking at?

                                Follow the bug report also: https://redmine.pfsense.org/issues/3760
                                On that, cmb indicates he has done some testing and it is still not fixed.

                                As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                1 Reply Last reply Reply Quote 0
                                • M
                                  mattbunce
                                  last edited by

                                  Cheers Phil

                                  I have already commented on the bug report too, I just didn't want to get too 'chatty' on there, because it seems the folks on there have done some pretty advanced diagnostics and I didn't want to distract from the good work they were doing :)

                                  1 Reply Last reply Reply Quote 0
                                  • E
                                    eri--
                                    last edited by

                                    Another round of fixes.
                                    Next snapshot should have the missing piece for fixing this.

                                    1 Reply Last reply Reply Quote 0
                                    • M
                                      mattbunce
                                      last edited by

                                      I still can't seem to get this working on the 17th October, 11:27:45 version.

                                      Have you had a chance to try it out ermal? Perhaps I am not configuring the Nat and Rules pages correctly?

                                      M

                                      1 Reply Last reply Reply Quote 0
                                      • M
                                        mikeisfly
                                        last edited by

                                        This is not working for me and I'm using 2.2-BETA (amd64) built on Fri Oct 17 20:02:23 CDT 2014 FreeBSD 10.1-RC2

                                        I have attached my LAN, NAT (Port Forwrd), Manual Outbound NAT Rules.

                                        I have also captured packets on both my WAN, WAN2 and LAN interfaces. On my WAN2 and LAN I can see the Syn, and the Syn Ack packets however I don't see the ack packet. Also there are some retransmissions. When I capture packets on my WAN and filter for port 32400 the capture is blank which tells me that the packet is not being sent out the default gateway. I have attached the .pcap files here in the form of a .txt file if you would like to look at them in wireshark the extension just needs to be changed back to .pcap . Hope this helps

                                        Thanks,

                                        P.S. Not sure why I was trying to embed these images from my onedrive folder but it didn't seem to work so I had to add attachments.

                                        ![LAN Rule.png](/public/imported_attachments/1/LAN Rule.png)
                                        ![LAN Rule.png_thumb](/public/imported_attachments/1/LAN Rule.png_thumb)
                                        ![Nat Port forward.png](/public/imported_attachments/1/Nat Port forward.png)
                                        ![Nat Port forward.png_thumb](/public/imported_attachments/1/Nat Port forward.png_thumb)
                                        ![outbound Nat.png](/public/imported_attachments/1/outbound Nat.png)
                                        ![outbound Nat.png_thumb](/public/imported_attachments/1/outbound Nat.png_thumb)
                                        [capture on LAN.txt](/public/imported_attachments/1/capture on LAN.txt)
                                        [capture on wan2.txt](/public/imported_attachments/1/capture on wan2.txt)

                                        1 Reply Last reply Reply Quote 0
                                        • W
                                          Wolf666
                                          last edited by

                                          I think my problem is the same.
                                          If someone has some spare time to read: https://forum.pfsense.org/index.php?topic=82944.0

                                          I am running:

                                          Version 2.2-BETA (amd64)
                                          built on Fri Oct 17 20:02:23 CDT 2014
                                          FreeBSD 10.1-RC2

                                          I am available to test any solution and report back.

                                          Modem Draytek Vigor 130
                                          pfSense 2.4 Supermicro A1SRi-2558 - 8GB ECC RAM - Intel S3500 SSD 80GB - M350 Case
                                          Switch Cisco SG350-10
                                          AP Netgear R7000 (Stock FW)
                                          HTPC Intel NUC5i3RYH
                                          NAS Synology DS1515+
                                          NAS Synology DS213+

                                          1 Reply Last reply Reply Quote 0
                                          • M
                                            mikeisfly
                                            last edited by

                                            It's a known bug and the developers are working on it. Until then if this is a deal breaker you can try going back to 2.1.5 until the issue is resolved. That is what I'm doing.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.