Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Slow NAT

    Scheduled Pinned Locked Moved
    NAT
    4
    12
    654
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      albgen
      last edited by

      hi,

      I have pfsense running on proxmox as a virtual machine. Basically is working everything normally. Except that i noticed that an API which is NAT-ed, has a high latency when replying.
      The API is running on the LAN side of the pfSense, and initially we were blaming the API for the slowness.
      Than, i tested locally from another machine on the LAN and is super fast. 40ms vs 3500ms

      TCP offloading is disabled.
      Any idea what could be the issue?

      thanks

      S 1 Reply Last reply Reply Quote 0
      • S
        SteveITS Rebel Alliance @albgen
        last edited by

        @albgen Can you be more specific about the NAT setup and where the PC using the API is? If you're also seeing packet loss or disconnects one possibility is an asymmetric routing issue, which is usually a config problem somewhere. (devices using different gateways, etc.)
        https://docs.netgate.com/pfsense/en/latest/troubleshooting/asymmetric-routing.html

        Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
        When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
        Upvote 👍 helpful posts!

        A 1 Reply Last reply Reply Quote 0
        • A
          albgen @SteveITS
          last edited by albgen

          @SteveITS said in Slow NAT:

          https://docs.netgate.com/pfsense/en/latest/troubleshooting/asymmetric-routing.html

          Regarding the configs, find it here config-pfSenseEdgeRouter.localdomain-20230623180532.xml

          LAN side has several machines, Windows and Linux servers.
          The API is running on machine 1. There is a NAT(port forward) from wan to lan for the port of the API(7068).

          Tests are done using postman.
          From the same machine 1, if i issue a request for the api, it is super fast(40ms). From machine 2 which is on the same network as machine 1, i issue the same request, result is super fast.

          From any laptop/computer which requests comes from the wan, more than 3 seconds of reply.

          I see on the states of the firewall 2 connections from the machine outside wan. They are not closed and they are opened
          e6d3e4cf-7f7d-4321-99ee-a132de471602-image.png

          The response is 0.5MBytes. So you see 2.1MB of traffic because i tried 4 times, to call the same endpoint.

          S 1 Reply Last reply Reply Quote 0
          • S
            SteveITS Rebel Alliance @albgen
            last edited by

            @albgen
            Machine1's gateway is the pfSense LAN IP?

            Is Machine2 connecting to Machine1 through pfSense's WAN IP via NAT reflection? Or direct to Machine1 across the network?

            If it's using reflection and is fast, yet connecting from WAN is slow, then I am not sure what would be going on, I don't recall running into that.

            Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
            When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
            Upvote 👍 helpful posts!

            A 1 Reply Last reply Reply Quote 0
            • A
              albgen @SteveITS
              last edited by

              @SteveITS Correct, Machine 1 gateway is the pfSense LAN IP.
              Machine 2 also has gateway the pfSense LAN IP and is on the same interface, so LAN1

              V 1 Reply Last reply Reply Quote 0
              • V
                viragomann @albgen
                last edited by

                @albgen
                Ensure that you've "Log packets matched from the default block rules in the ruleset" enabled and check if there are blocks in the log from machine 1.

                Also there is obviously a router in front of your pfSense. Connect a computer directly to the WAN interface of pfSense and test the access via NAT from it.

                A 1 Reply Last reply Reply Quote 0
                • A
                  albgen @viragomann
                  last edited by

                  @viragomann I think it is not even possible to enable the "Log packets on the default block rules in the ruleset" .
                  There is no router in front. It is just a masquerade of the nic: The topology is
                  nic with public ip->dnat(iptables -t nat -A PREROUTING --dst 65.XXX.XX.XX -j DNAT --to-destination 10.21.21.120)->wan pfsense.

                  V 1 Reply Last reply Reply Quote 0
                  • V
                    viragomann @albgen
                    last edited by

                    @albgen said in Slow NAT:

                    @viragomann I think it is not even possible to enable the "Log packets on the default block rules in the ruleset" .

                    Status > System Logs > Settings > Log packets matched from the default block rules in the ruleset

                    There is no router in front. It is just a masquerade of the nic: The topology is
                    nic with public ip->dnat(iptables -t nat -A PREROUTING --dst 65.XXX.XX.XX -j DNAT --to-destination 10.21.21.120)->wan pfsense.

                    Yeah, this is exactly what a NAT router does.

                    What is the sense of forwarding the whole traffic instead of having the public IP directly on pfSense?

                    Presumably your pfSense is virtualized. Which hypervisor? Is it installed accordingly?
                    Do you have other connections through pfSense, which have a low latency?

                    A 1 Reply Last reply Reply Quote 0
                    • A
                      albgen @viragomann
                      last edited by

                      @viragomann

                      those logs are already enabled.
                      Of course it is virtualised. Check the first post

                      1 Reply Last reply Reply Quote 0
                      • A
                        albgen
                        last edited by

                        I just added a NAT on WAN2 which has a Public IP setup and yet the same slowness.

                        johnpozJ 1 Reply Last reply Reply Quote 0
                        • johnpozJ
                          johnpoz LAYER 8 Global Moderator @albgen
                          last edited by johnpoz

                          @albgen so you think pfsense natting the traffic is adding like 3.5 seconds to your response time?

                          Well easy enough to check if pfsense is doing that.. Do a sniff on your wan and on your lan at same time and send some traffic what is the delay caused by pfsense sending it on..

                          So example

                          mathnat.jpg

                          Here is me coming from can you see me to port 23040 which is forwarded to my plex on 32400 on 192.168.9.10

                          So sniffing on my wan and my lan at same time with tcpdump I see when it hits my wan and when it gets sent on, see the Syn and the Syn,Acks

                          Lets do the math..

                          So syn hits my wan at 48.108772, and that is sent on to 192.168.9.100 at 48.108886 so a delay of .000114 seconds, or 0.114 ms

                          Now the syn,ack hits my lan at 48.109230, and sent out my wan at 48.109270 for a delay caused by pfsense of .000040 seconds..

                          So lets see the delay your pfsense creating - 3.5 seconds seems highly unlikely to me..

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.03 | Lab VMs 2.7.2, 24.03

                          A 1 Reply Last reply Reply Quote 0
                          • A
                            albgen @johnpoz
                            last edited by albgen

                            @johnpoz thanks for the tip and i did the same test.
                            Window on top is WAN and on the bottom is LAN. I just captured 10 packets from each interface and seems it is pretty fast so the culprit is not the NAT.

                            a243489b-bc55-49e5-87b2-747bd73a304f-image.png

                            Found though two solutions but still not why it is happening.

                            • Remove Accept-Encoding header from the http request - result is very fast.

                            • Using a reverse proxy with https is still fast with and without the Accept-Encoding header

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post