Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    2 WAN, 1:1 NAT, Outgoing not working (Solved)

    Scheduled Pinned Locked Moved
    NAT
    2
    14
    6.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DerelictD
      Derelict LAYER 8 Netgate
      last edited by

      Sounds like you're screwing something up.

      Where is CARP in the mix? Is this an HA cluster? What are the pertinent interface and CARP VIP addresses?

      1:1 NAT does not route traffic. If you want traffic to egress out a specific interface you have to policy route it. While traffic is going outbound out a specific interface as determined by policy routing then the routing table, 1:1 NAT will be consulted and the source address translated accordingly.

      Chattanooga, Tennessee, USA
      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
      Do Not Chat For Help! NO_WAN_EGRESS(TM)

      1 Reply Last reply Reply Quote 0
      • P
        preid
        last edited by

        Hi, Derelict,

        CARP is looking after a dual node cluster. I have 5 physical interfaces - WAN1, WAN2, LAN1, LAN2, and the CARP replication. All have CARP VIPs.

        I set up my 1:1 NAT using an IP Alias VIP in the WAN2 subnet. Maybe it should be another CARP VIP, instead? Maybe that's where I'm screwed up?

        I'll look into the policy route to get my traffic for my WAN2 1:1 NAT to go out the WAN2 interface if it originates from the internal machine. I know that on a SonicWALL you do this via the NAT rules (SonicWALL also wants a loopback rule, so internal machines can reach the 1:1 NAT machine via the WAN IP. Do I need the same sort of thing with pfSense, as well?)

        Thanks for the help, Derelict.

        1 Reply Last reply Reply Quote 0
        • P
          preid
          last edited by

          Ok. My source route totally solved my outgoing traffic for the 1:1 NAT setup, which I'm super happy about.

          Now if I could just get my backup CARP node to stop ARPing for my 1:1 NAT WAN IP, I'd be set. Every time it does that, communications breaks down because traffic goes back to to the backup node, which never gets to my internal PC.

          I'm getting closer, though! Maybe I'll try switching the IP Alias VIP to a CARP VIP, and that'll stop the backup from ARPing for things it can't move traffic for? Or should I be doing something else with CARP in the mix?

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            IP Alias VIPs in a CARP setup should be tied to the CARP VIP. They will swing over to the backup when the CARP VIP does.

            Delete the VIP on the secondary and edit the VIP on the primary changing the interface to the CARP VIP.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • P
              preid
              last edited by

              Ok. I deleted the IP Aliases from my backup node, then switched my IP Aliases on the master node from type IP Alias to type CARP, fed it a password and VHID Group, and saved. I think that is what you were telling me to do. :)

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                Not really. You can keep the IP Alias VIP but you only want to define it on the primary and change the interface on the VIP to a CARP VIP on that interface.

                The more CARP you have on an interface the more heartbeat traffic there is and the more likely heartbeats will be missed on the secondary. As long as there are not a lot it should be fine but the IP Alias VIPs tacked onto CARP VIPs is a pretty elegant solution.

                ![Screen Shot 2016-10-03 at 4.05.12 PM.png](/public/imported_attachments/1/Screen Shot 2016-10-03 at 4.05.12 PM.png)
                ![Screen Shot 2016-10-03 at 4.05.12 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2016-10-03 at 4.05.12 PM.png_thumb)

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • P
                  preid
                  last edited by

                  Ah ha!

                  The image you attached really made it clear what you meant. I made the change, and things seem to be working pretty well now.

                  The only issue I seem to have left is that my machine on WAN2 that has been 1:1 NAT'd, policy routed out WAN2, and outbound NAT'd to it's 1:1 NAT IP can't talk to anything through my port forwards on WAN1. It can ping stuff, but it can't connect with anything TCP.

                  I'm thinking I need some kind of loopback route rule somewhere?

                  1 Reply Last reply Reply Quote 0
                  • DerelictD
                    Derelict LAYER 8 Netgate
                    last edited by

                    Hmm. That should work OK.

                    https://doc.pfsense.org/index.php/Port_Forward_Troubleshooting

                    Chattanooga, Tennessee, USA
                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                    1 Reply Last reply Reply Quote 0
                    • P
                      preid
                      last edited by

                      I would have thought so, too, but it doesn't seem to.

                      This is what I have:

                      WAN1: 10.10.100.241/28
                      WAN2: 10.10.200.161/28
                      LAN PC 1: 192.168.230.190/24
                      LAN PC 2: 192.168.230.8/24

                      1:1 NAT:

                      • Interface: WAN1

                      • External IP: 10.10.100.244

                      • Internal IP: 192.168.230.190

                      • Interface: WAN2

                      • External IP: 10.10.200.172

                      • Internal IP: Single host, 192.168.230.8

                      Outbound NAT (in Hybrid Outbound NAT mode):

                      • Interface: WAN1

                      • Source: Any

                      • Destination: Any

                      • NAT Address: 10.10.100.244

                      • Interface: WAN2

                      • Source: Any

                      • Destination: Any

                      • NAT Address: 10.10.200.172

                      VIPs:

                      • Type: IP Alias

                      • Interface: 10.10.100.241 (WAN1 IP)

                      • Address: 10.10.100.244/32

                      • Type: IP Alias

                      • Interface: 10.10.200.161 (WAN2 IP)

                      • Address: 10.10.200.172/32

                      Firewall (Policy route only for WAN2 - WAN1 didn't need it, since it's the default gateway):

                      • Interface: LAN

                      • Protocol: Any

                      • Source: Single host, 192.168.230.8

                      • Gateway: OPT3GW - 10.10.200.174 - WAN2 Gateway

                      • Interface: WAN1

                      • Protocol: HTTP

                      • Source: Any

                      • Destination: Single host, 192.168.230.190

                      • Interface: WAN1

                      • Protocol: ICMP

                      • Source: Any

                      • Destination: Single host, 192.168.230.190

                      • Interface: WAN2

                      • Protocol: HTTP

                      • Source: Any

                      • Destination: Single host, 192.168.230.8

                      • Interface: WAN2

                      • Protocol: ICMP

                      • Source: Any

                      • Destination: Single host, 192.168.230.8

                      Results:

                      • 192.168.230.8 can ping 10.10.100.244
                      • 192.168.230.190 can ping 10.10.200.172 (as can a couple of other internal hosts, but many internal hosts get ping timeouts, even though they all use the same LAN gateway - 192.168.230.1, which is the pfSense LAN CARP VIP)
                      • Neither LAN host can connect via HTTP to the other via their IP Aliases
                      • Both hosts can ping and connect to the pfSense management interface on the opposite WAN (both the VIP and the real IPs of the pfSense boxes)
                      • Both LAN hosts can be reached via ICMP and HTTP from the WAN by external hosts

                      Looking at the state tables, it appears that traffic between WAN interfaces should be handled internally - the traffic doesn't appear to be heading out to the external ISP gateways at all, which is what I would expect. When pinging from 192.168.230.8 to 10.10.100.244, for example, I can see a state of:

                      LAN - ICMP - 192.168.230.8:4 -> 192.168.230.190:4 (10.10.100.244:4) - 0:0 - 4 / 0

                      But if I try to connect via HTTP, I see a state of:

                      LAN - TCP - 192.168.230.8:17123 -> 192.168.230.190:80 (10.10.100.244:80) - CLOSED:SYN_SENT - 3 / 0

                      Yet I have no problem connecting via HTTP directly using the private IPs (192.168.230.8 -> 192.168.230.190).

                      192.168.230.8 can't connect through any port forwards, in fact - not just the ones with 1:1 NAT and stuff configured. Even connecting to one my basic port forwards in to another internal host through the WAN1 CARP VIP fails, though they work fine from external hosts.

                      Somehow, my traffic is dropped on it's way from the LAN through WAN1 and back into the LAN. This happens on other routers I've worked with, too, and required a "loopback rule" to get working to translate the LAN traffic to appear to be traffic coming into the WAN2 interface, where all the normal firewall rules then apply. It seems like pfSense needs something similar somewhere, perhaps?

                      1 Reply Last reply Reply Quote 0
                      • DerelictD
                        Derelict LAYER 8 Netgate
                        last edited by

                        Source any on outbound NAT is almost never right. You will be NATting things you don't want to NAT, like traffic sourced from the firewall. Don't be lazy here. Define rules for the inside networks you want to NAT for.

                        If you are testing all of this from inside expecting to ping pong from the inside to the WAN address and back inside you need to look at NAT reflection.

                        Ugly hack (it is on the other firewalls too) but you seem to be insistent on it. Split DNS is another (better) option.

                        https://doc.pfsense.org/index.php/Why_can%27t_I_access_forwarded_ports_on_my_WAN_IP_from_my_LAN/OPTx_networks

                        Chattanooga, Tennessee, USA
                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                        1 Reply Last reply Reply Quote 0
                        • P
                          preid
                          last edited by

                          That was my mistake when transcribing my setup. I did, indeed, have a source entered for my outbound NAT rules (the LAN IP of the related host/32).

                          I wouldn't use such a hack, except I know some of the machines that will be coming into the internal network are from a data center where they were behind a transparent filtering bridge and communicated with each other using their WAN IPs, since they didn't have any LAN IPs at the time. Over time I can clean this up and kill the hack, but until that happens, the external IPs need to work from the internal hosts to avoid a mess.

                          Turning on the ugly hack does allow me to reach my 192.168.230.8 (10.10.200.172) machine from my other LAN machines using the 10.10.200.172 IP, but oddly it doesn't seem to allow my 192.168.230.8 machine to reach my 10.10.100.244 machine on WAN1. Even ping stopped working. WAN1 -> WAN2 is all good. WAN2 -> WAN1, not so much. Weird.

                          I see in the state table:

                          LAN - ICMP - 192.168.230.8:1 -> 192.168.230.190:1 (10.10.100.244:1) - 0:0 - 3 / 0
                          WAN2 - ICMP - 10.10.200.172:1 (192.168.230.8:1) -> 192.168.230.190:1 - 0:0 - 3 / 0

                          By the way, thank you so much for all your help. I know the set up has a lot of moving parts, and I appreciate your assistance in nailing down the issues I've run into.

                          1 Reply Last reply Reply Quote 0
                          • P
                            preid
                            last edited by

                            I figured it out. My 1:1 NAT rule for 192.168.230.8 had NAT Reflection set to Enabled, whereas the 1:1 NAT rule for 192.168.230.190 has NAT Reflect set to System Default. As soon as I switched it to Enabled, things started working. :)

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.