Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Throughput from Lan to Wan

    Scheduled Pinned Locked Moved General pfSense Questions
    26 Posts 4 Posters 5.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • O
      Orkopaede
      last edited by

      Hello,

      I have a strange problem with my PF. When I use iperf to test the speed from my PC to the PF I get around 900 Mbit. From the PF to the Internet 200 Mbit. But from the PC to the Internet only 16Mbit. Checksum offload and further I have already deactivated. Does somebody has any idea?

      S 1 Reply Last reply Reply Quote 0
      • O
        Orkopaede
        last edited by

        In addition, the system data here:
        Protectli FW6D.
        47b7cf19-3c51-4b0d-b397-3939650d81c0-grafik.png

        The CPU utilization with top -HaSP under load also looked good to me.

        382d5b6b-b1a0-41a4-9661-4287d0adc754-grafik.png

        1 Reply Last reply Reply Quote 0
        • O
          Orkopaede
          last edited by

          My next test confirms that there is a problem within the PF. I activated another Ethernet port (OPT1) to test the speed directly from the PC through the PF to the Internet. So without a switch or possible defective cable in between. Unfortunately I have also only the 16Mbit here.

          1 Reply Last reply Reply Quote 0
          • O
            Orkopaede
            last edited by

            okay it doesn't seem to be a problem with PF after all.? As a test, I set up several connections at the same time with iperf and the option -P and then I get my 200Mbit. So maybe it's a Windows problem? I do not get it. Any idea, suggestions?

            johnpozJ 1 Reply Last reply Reply Quote 0
            • johnpozJ
              johnpoz LAYER 8 Global Moderator @Orkopaede
              last edited by johnpoz

              @orkopaede have had lack luster success with iperf to the internet. If you put a device on your pfsense wan network and do an iperf what do you get through pfsense.

              What does a normal just web based speedtest show. If you have a hard time putting something on your pfsense wan network to test with. Just create a outbound nat on pfsense to one of your other local networks. So pfsense is both firewalling and natting - this is what it does to your internet connection.

              So doing a iperf to my nas with outbound nat so pfsense is doing exactly what it does to the internet.. I see

              $ iperf3.exe -c 192.168.2.42
              Connecting to host 192.168.2.42, port 5201
              [  5] local 192.168.9.100 port 49470 connected to 192.168.2.42 port 5201
              [ ID] Interval           Transfer     Bitrate
              [  5]   0.00-1.00   sec   115 MBytes   961 Mbits/sec
              [  5]   1.00-2.00   sec   112 MBytes   937 Mbits/sec
              [  5]   2.00-3.00   sec   113 MBytes   946 Mbits/sec
              [  5]   3.00-4.00   sec   112 MBytes   943 Mbits/sec
              [  5]   4.00-5.00   sec   112 MBytes   936 Mbits/sec
              [  5]   5.00-6.00   sec   112 MBytes   944 Mbits/sec
              [  5]   6.00-7.00   sec   112 MBytes   936 Mbits/sec
              [  5]   7.00-8.00   sec   112 MBytes   939 Mbits/sec
              [  5]   8.00-9.00   sec   113 MBytes   945 Mbits/sec
              [  5]   9.00-10.00  sec   112 MBytes   940 Mbits/sec
              - - - - - - - - - - - - - - - - - - - - - - - - -
              [ ID] Interval           Transfer     Bitrate
              [  5]   0.00-10.00  sec  1.10 GBytes   943 Mbits/sec                  sender
              [  5]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec                  receiver
              

              This is for sure close enough to wire speed.. If I take pfsense out of the picture and go directly to the nas over the lan network..

              $ iperf3.exe -c 192.168.9.10
              Connecting to host 192.168.9.10, port 5201
              [  5] local 192.168.9.100 port 49465 connected to 192.168.9.10 port 5201
              [ ID] Interval           Transfer     Bitrate
              [  5]   0.00-1.00   sec   114 MBytes   952 Mbits/sec
              [  5]   1.00-2.00   sec   114 MBytes   962 Mbits/sec
              [  5]   2.00-3.00   sec   113 MBytes   949 Mbits/sec
              [  5]   3.00-4.00   sec   113 MBytes   949 Mbits/sec
              [  5]   4.00-5.00   sec   113 MBytes   949 Mbits/sec
              [  5]   5.00-6.00   sec   113 MBytes   950 Mbits/sec
              [  5]   6.00-7.00   sec   113 MBytes   949 Mbits/sec
              [  5]   7.00-8.00   sec   113 MBytes   949 Mbits/sec
              [  5]   8.00-9.00   sec   113 MBytes   950 Mbits/sec
              [  5]   9.00-10.00  sec   113 MBytes   948 Mbits/sec
              - - - - - - - - - - - - - - - - - - - - - - - - -
              [ ID] Interval           Transfer     Bitrate
              [  5]   0.00-10.00  sec  1.11 GBytes   951 Mbits/sec                  sender
              [  5]   0.00-10.01  sec  1.11 GBytes   949 Mbits/sec                  receiver
              

              Here is the outbound nat I created

              outboundnat.jpg

              Can't really do a test to nas on different network that my pc without doing this without disruption of my network setup. Since the nas has an IP in lan, I run into asymmetrical traffic flow problem - because the nas likes to answer the connection to its 2.42 address via its 9.10 address. This setup makes sure all traffic flows through pfsense, and pfsense nats it, just like it would do to the internet.

              An intelligent man is sometimes forced to be drunk to spend time with his fools
              If you get confused: Listen to the Music Play
              Please don't Chat/PM me for help, unless mod related
              SG-4860 24.11 | Lab VMs 2.7.2, 24.11

              O 1 Reply Last reply Reply Quote 0
              • O
                Orkopaede @johnpoz
                last edited by

                @johnpoz Hi, unfortunately I can't put a device in front of the PF because of the fiber optic router. The router only accepts one MAC to communicate with. If that was your idea? As I said, I get 200Mbit from the PF with Iperf. A web-based speed test gives the same result (16Mbit). My last test with the multiple connections made me think that it might be a Windows or anti-virus software problem. While looking for a windows speed cap I came across this:

                Windows 10 internet speed capped

                With that I get 65Mbit. Which is better. But theoretically it's not the end yet.

                I'll test that with the OUTBOUND NAT.
                Results will follow immediately.

                johnpozJ 1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator @Orkopaede
                  last edited by johnpoz

                  @orkopaede recv window - yup that could have effect for sure..

                  What were your settings? And what did you change them too.. Here are mine

                  $ netsh interface tcp show global                 
                  Querying active state...                          
                                                                    
                  TCP Global Parameters                             
                  ----------------------------------------------    
                  Receive-Side Scaling State          : enabled     
                  Receive Window Auto-Tuning Level    : normal      
                  Add-On Congestion Control Provider  : default     
                  ECN Capability                      : enabled     
                  RFC 1323 Timestamps                 : disabled    
                  Initial RTO                         : 1000        
                  Receive Segment Coalescing State    : enabled     
                  Non Sack Rtt Resiliency             : disabled    
                  Max SYN Retransmissions             : 4           
                  Fast Open                           : enabled     
                  Fast Open Fallback                  : enabled     
                  HyStart                             : enabled     
                  Proportional Rate Reduction         : enabled     
                  Pacing Profile                      : off         
                                                                    
                  

                  BTW - when your testing from client to server in iperf.. Make sure you test just over the local network, where they are on the same - so you can validate you do see wirespeed between the clients. You might also what to check in reverse flow, normally the client is the sender..

                  You can test with -V and see some more details of the test

                  $ iperf3.exe -c 192.168.9.10 -V                                                                          
                  iperf 3.13                                                                                               
                  CYGWIN_NT-10.0-19045 i9-win 3.4.6-1.x86_64 2023-02-14 13:23 UTC x86_64                                   
                  Control connection MSS 1460                                                                              
                  Time: Wed, 22 Feb 2023 13:28:22 GMT                                                                      
                  Connecting to host 192.168.9.10, port 5201                                                               
                        Cookie: grthd7rff3wrvul2smkaytwr6lrnyegxxmp6                                                       
                        TCP MSS: 1460 (default)                                                                            
                  [  5] local 192.168.9.100 port 49589 connected to 192.168.9.10 port 5201                                 
                  Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0   
                  [ ID] Interval           Transfer     Bitrate                                                            
                  [  5]   0.00-1.00   sec   115 MBytes   965 Mbits/sec                                                     
                  [  5]   1.00-2.00   sec   113 MBytes   949 Mbits/sec                                                     
                  [  5]   2.00-3.00   sec   113 MBytes   949 Mbits/sec                                                     
                  [  5]   3.00-4.00   sec   113 MBytes   949 Mbits/sec                                                     
                  [  5]   4.00-5.00   sec   113 MBytes   949 Mbits/sec                                                     
                  [  5]   5.00-6.00   sec   113 MBytes   949 Mbits/sec                                                     
                  [  5]   6.00-7.00   sec   113 MBytes   949 Mbits/sec                                                     
                  [  5]   7.00-8.00   sec   113 MBytes   948 Mbits/sec                                                     
                  [  5]   8.00-9.00   sec   113 MBytes   949 Mbits/sec                                                     
                  [  5]   9.00-10.00  sec   113 MBytes   950 Mbits/sec                                                     
                  - - - - - - - - - - - - - - - - - - - - - - - - -                                                        
                  Test Complete. Summary Results:                                                                          
                  [ ID] Interval           Transfer     Bitrate                                                            
                  [  5]   0.00-10.00  sec  1.11 GBytes   951 Mbits/sec                  sender                             
                  [  5]   0.00-10.01  sec  1.11 GBytes   949 Mbits/sec                  receiver                           
                  CPU Utilization: local/sender 8.7% (5.0%u/3.7%s), remote/receiver 13.6% (1.0%u/12.6%s)                   
                  rcv_tcp_congestion cubic                                                                                 
                                                                                                                           
                  iperf Done.                                                                                              
                  
                  $ iperf3.exe -c 192.168.9.10 -V -R
                  iperf 3.13
                  CYGWIN_NT-10.0-19045 i9-win 3.4.6-1.x86_64 2023-02-14 13:23 UTC x86_64
                  Control connection MSS 1460
                  Time: Wed, 22 Feb 2023 13:29:20 GMT
                  Connecting to host 192.168.9.10, port 5201
                  Reverse mode, remote host 192.168.9.10 is sending
                        Cookie: cvgi4td7ruemmazafk53j3da7qqzy35unrkm
                        TCP MSS: 1460 (default)
                  [  5] local 192.168.9.100 port 49593 connected to 192.168.9.10 port 5201
                  Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
                  [ ID] Interval           Transfer     Bitrate
                  [  5]   0.00-1.00   sec   113 MBytes   948 Mbits/sec
                  [  5]   1.00-2.00   sec   113 MBytes   949 Mbits/sec
                  [  5]   2.00-3.00   sec   113 MBytes   947 Mbits/sec
                  [  5]   3.00-4.00   sec   113 MBytes   949 Mbits/sec
                  [  5]   4.00-5.00   sec   113 MBytes   949 Mbits/sec
                  [  5]   5.00-6.00   sec   113 MBytes   948 Mbits/sec
                  [  5]   6.00-7.00   sec   113 MBytes   949 Mbits/sec
                  [  5]   7.00-8.00   sec   113 MBytes   949 Mbits/sec
                  [  5]   8.00-9.00   sec   113 MBytes   949 Mbits/sec
                  [  5]   9.00-10.00  sec   113 MBytes   949 Mbits/sec
                  - - - - - - - - - - - - - - - - - - - - - - - - -
                  Test Complete. Summary Results:
                  [ ID] Interval           Transfer     Bitrate         Retr
                  [  5]   0.00-10.00  sec  1.11 GBytes   951 Mbits/sec    0             sender
                  [  5]   0.00-10.00  sec  1.10 GBytes   949 Mbits/sec                  receiver
                  snd_tcp_congestion cubic
                  
                  iperf Done.
                  

                  Notice when use the -R it clearly states that 9.10 is the sender.

                  edit: Look how crappy it gets if I turn the window size down to 8k, which could be default..

                  $ iperf3.exe -c 192.168.9.10 -V -w 8k
                  iperf 3.13
                  CYGWIN_NT-10.0-19045 i9-win 3.4.6-1.x86_64 2023-02-14 13:23 UTC x86_64
                  Control connection MSS 1460
                  Time: Wed, 22 Feb 2023 13:32:57 GMT
                  Connecting to host 192.168.9.10, port 5201
                        Cookie: vrzgka56k7hvizu4wdagwtpww4tlc6b6yych
                        TCP MSS: 1460 (default)
                  [  5] local 192.168.9.100 port 49603 connected to 192.168.9.10 port 5201
                  Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
                  [ ID] Interval           Transfer     Bitrate
                  [  5]   0.00-1.00   sec  29.5 MBytes   247 Mbits/sec
                  [  5]   1.00-2.00   sec  29.4 MBytes   247 Mbits/sec
                  [  5]   2.00-3.00   sec  27.6 MBytes   232 Mbits/sec
                  [  5]   3.00-4.01   sec  30.6 MBytes   253 Mbits/sec
                  [  5]   4.01-5.00   sec  31.7 MBytes   269 Mbits/sec
                  [  5]   5.00-6.00   sec  28.6 MBytes   240 Mbits/sec
                  [  5]   6.00-7.00   sec  29.1 MBytes   244 Mbits/sec
                  [  5]   7.00-8.00   sec  32.2 MBytes   270 Mbits/sec
                  [  5]   8.00-9.00   sec  27.6 MBytes   232 Mbits/sec
                  [  5]   9.00-10.00  sec  28.6 MBytes   240 Mbits/sec
                  - - - - - - - - - - - - - - - - - - - - - - - - -
                  Test Complete. Summary Results:
                  [ ID] Interval           Transfer     Bitrate
                  [  5]   0.00-10.00  sec   295 MBytes   247 Mbits/sec                  sender
                  [  5]   0.00-10.00  sec   295 MBytes   247 Mbits/sec                  receiver
                  CPU Utilization: local/sender 24.6% (12.1%u/12.6%s), remote/receiver 22.4% (2.3%u/20.1%s)
                  rcv_tcp_congestion cubic
                  
                  iperf Done.
                  

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                  O 1 Reply Last reply Reply Quote 0
                  • O
                    Orkopaede @johnpoz
                    last edited by

                    @johnpoz Before:

                    TCP Global Parameters
                    ----------------------------------------------
                    Receive-Side Scaling State          : disabled
                    Receive Window Auto-Tuning Level    : disabled
                    Add-On Congestion Control Provider  : default
                    ECN Capability                      : disabled
                    RFC 1323 Timestamps                 : disabled
                    Initial RTO                         : 1000
                    Receive Segment Coalescing State    : enabled
                    Non Sack Rtt Resiliency             : disabled
                    Max SYN Retransmissions             : 4
                    Fast Open                           : enabled
                    Fast Open Fallback                  : enabled
                    HyStart                             : enabled
                    Proportional Rate Reduction         : enabled
                    Pacing Profile                      : off
                    

                    After with: netsh int tcp set global autotuninglevel=normal

                    TCP Global Parameters
                    ----------------------------------------------
                    Receive-Side Scaling State          : disabled
                    Receive Window Auto-Tuning Level    : normal
                    Add-On Congestion Control Provider  : default
                    ECN Capability                      : disabled
                    RFC 1323 Timestamps                 : disabled
                    Initial RTO                         : 1000
                    Receive Segment Coalescing State    : enabled
                    Non Sack Rtt Resiliency             : disabled
                    Max SYN Retransmissions             : 4
                    Fast Open                           : enabled
                    Fast Open Fallback                  : enabled
                    HyStart                             : enabled
                    Proportional Rate Reduction         : enabled
                    Pacing Profile                      : off
                    
                    iperf3.exe -c 192.168.8.248 -V
                    
                    iperf 3.1.3
                    CYGWIN_NT-10.0 WernerLaptop 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
                    Time: Wed, 22 Feb 2023 13:38:39 GMT
                    Connecting to host 192.168.8.248, port 5201
                          Cookie: WernerLaptop.1677073117.500543.72f2e
                          TCP MSS: 0 (default)
                    [  4] local 192.168.8.34 port 2341 connected to 192.168.8.248 port 5201
                    Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test
                    [ ID] Interval           Transfer     Bandwidth
                    [  4]   0.00-1.00   sec   112 MBytes   936 Mbits/sec
                    [  4]   1.00-2.00   sec   113 MBytes   945 Mbits/sec
                    [  4]   2.00-3.00   sec   113 MBytes   949 Mbits/sec
                    [  4]   3.00-4.00   sec   112 MBytes   942 Mbits/sec
                    [  4]   4.00-5.00   sec   113 MBytes   947 Mbits/sec
                    [  4]   5.00-6.00   sec   113 MBytes   949 Mbits/sec
                    [  4]   6.00-7.00   sec   113 MBytes   949 Mbits/sec
                    [  4]   7.00-8.00   sec   112 MBytes   943 Mbits/sec
                    [  4]   8.00-9.00   sec   112 MBytes   941 Mbits/sec
                    [  4]   9.00-10.00  sec   112 MBytes   943 Mbits/sec
                    - - - - - - - - - - - - - - - - - - - - - - - - -
                    Test Complete. Summary Results:
                    [ ID] Interval           Transfer     Bandwidth
                    [  4]   0.00-10.00  sec  1.10 GBytes   945 Mbits/sec                  sender
                    [  4]   0.00-10.00  sec  1.10 GBytes   945 Mbits/sec                  receiver
                    CPU Utilization: local/sender 19.2% (2.1%u/17.1%s), remote/receiver 37.3% (1.8%u/35.7%s)
                    
                    iperf Done.
                    
                    iperf3.exe -c 192.168.8.248 -V -R
                    
                    iperf 3.1.3
                    CYGWIN_NT-10.0 WernerLaptop 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
                    Time: Wed, 22 Feb 2023 13:39:01 GMT
                    Connecting to host 192.168.8.248, port 5201
                    Reverse mode, remote host 192.168.8.248 is sending
                          Cookie: WernerLaptop.1677073141.672333.7cdce
                          TCP MSS: 0 (default)
                    [  4] local 192.168.8.34 port 2351 connected to 192.168.8.248 port 5201
                    Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test
                    [ ID] Interval           Transfer     Bandwidth
                    [  4]   0.00-1.00   sec   112 MBytes   942 Mbits/sec
                    [  4]   1.00-2.00   sec   107 MBytes   898 Mbits/sec
                    [  4]   2.00-3.00   sec   112 MBytes   939 Mbits/sec
                    [  4]   3.00-4.00   sec   113 MBytes   945 Mbits/sec
                    [  4]   4.00-5.00   sec   110 MBytes   919 Mbits/sec
                    [  4]   5.00-6.00   sec   102 MBytes   855 Mbits/sec
                    [  4]   6.00-7.00   sec   109 MBytes   917 Mbits/sec
                    [  4]   7.00-8.00   sec   110 MBytes   923 Mbits/sec
                    [  4]   8.00-9.00   sec  95.4 MBytes   800 Mbits/sec
                    [  4]   9.00-10.00  sec   107 MBytes   899 Mbits/sec
                    - - - - - - - - - - - - - - - - - - - - - - - - -
                    Test Complete. Summary Results:
                    [ ID] Interval           Transfer     Bandwidth       Retr
                    [  4]   0.00-10.00  sec  1.05 GBytes   904 Mbits/sec    0             sender
                    [  4]   0.00-10.00  sec  1.05 GBytes   904 Mbits/sec                  receiver
                    CPU Utilization: local/receiver 53.9% (6.7%u/47.1%s), remote/sender 12.8% (3.3%u/9.5%s)
                    
                    iperf Done.
                    
                    johnpozJ 1 Reply Last reply Reply Quote 0
                    • johnpozJ
                      johnpoz LAYER 8 Global Moderator @Orkopaede
                      last edited by

                      @orkopaede your scaling is set to disabled still..

                      "Receive-Side Scaling State : disabled"

                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                      If you get confused: Listen to the Music Play
                      Please don't Chat/PM me for help, unless mod related
                      SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                      O 1 Reply Last reply Reply Quote 0
                      • O
                        Orkopaede @johnpoz
                        last edited by

                        @johnpoz i see... I changed it also ecn to enabled but still

                        TCP Global Parameters
                        ----------------------------------------------
                        Receive-Side Scaling State          : enabled
                        Receive Window Auto-Tuning Level    : normal
                        Add-On Congestion Control Provider  : default
                        ECN Capability                      : enabled
                        RFC 1323 Timestamps                 : disabled
                        Initial RTO                         : 1000
                        Receive Segment Coalescing State    : enabled
                        Non Sack Rtt Resiliency             : disabled
                        Max SYN Retransmissions             : 4
                        Fast Open                           : enabled
                        Fast Open Fallback                  : enabled
                        HyStart                             : enabled
                        Proportional Rate Reduction         : enabled
                        Pacing Profile                      : off
                        
                        C:\Users\Werner>Z:\Exchange\Firla\Software\iperf\iperf3.exe -c iperf.par2.as49434.net -p 9232 -V
                        iperf 3.1.3
                        CYGWIN_NT-10.0 WernerLaptop 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
                        Time: Wed, 22 Feb 2023 13:47:48 GMT
                        Connecting to host iperf.par2.as49434.net, port 9232
                              Cookie: WernerLaptop.1677073668.275288.50841
                              TCP MSS: 0 (default)
                        [  4] local 192.168.8.34 port 2500 connected to 193.177.162.41 port 9232
                        Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test
                        [ ID] Interval           Transfer     Bandwidth
                        [  4]   0.00-1.00   sec  7.00 MBytes  58.5 Mbits/sec
                        [  4]   1.00-2.01   sec  7.75 MBytes  64.8 Mbits/sec
                        [  4]   2.01-3.01   sec  7.75 MBytes  64.7 Mbits/sec
                        [  4]   3.01-4.02   sec  8.00 MBytes  66.9 Mbits/sec
                        [  4]   4.02-5.00   sec  7.50 MBytes  63.8 Mbits/sec
                        [  4]   5.00-6.00   sec  7.88 MBytes  66.0 Mbits/sec
                        [  4]   6.00-7.01   sec  7.88 MBytes  65.6 Mbits/sec
                        [  4]   7.01-8.00   sec  7.88 MBytes  66.5 Mbits/sec
                        [  4]   8.00-9.00   sec  7.88 MBytes  66.1 Mbits/sec
                        [  4]   9.00-10.01  sec  7.75 MBytes  64.8 Mbits/sec
                        - - - - - - - - - - - - - - - - - - - - - - - - -
                        Test Complete. Summary Results:
                        [ ID] Interval           Transfer     Bandwidth
                        [  4]   0.00-10.01  sec  77.2 MBytes  64.8 Mbits/sec                  sender
                        [  4]   0.00-10.01  sec  77.2 MBytes  64.8 Mbits/sec                  receiver
                        CPU Utilization: local/sender 1.5% (0.3%u/1.2%s), remote/receiver 4.8% (0.3%u/4.4%s)
                        
                        iperf Done.
                        

                        To be honest, I'm still trying to understand how that NAT ... Thing.. works. what you wrote before... :(

                        johnpozJ 1 Reply Last reply Reply Quote 0
                        • johnpozJ
                          johnpoz LAYER 8 Global Moderator @Orkopaede
                          last edited by johnpoz

                          @orkopaede it is just an outbound nat.

                          In my example when pfsense sees traffic to 2.42 from 9.100 it changes the IP from being source of 9.100 to pfsense IP on that interface, in my case 2.253.. Just like what happens when your lan clients goes to something on the internet and pfsense changes the IP to pfsense wan IP (normally some public IP).

                          notice the states when I do that..

                          states.jpg

                          Now when not natting - which pfsense wouldn't do between local networks. See the states

                          nonnatted.jpg

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                          O 1 Reply Last reply Reply Quote 0
                          • O
                            Orkopaede @johnpoz
                            last edited by

                            @johnpoz okay i think i got it:

                            9b68369b-2133-4368-904e-ce53234e883a-grafik.png

                            180c9b1c-0725-4732-bb8d-790560c0554d-grafik.png

                            Speeds are i would say okay the 8.227 is here probably the bottle neck:

                            iperf3.exe -c 192.168.8.227 -V
                            iperf 3.1.3
                            CYGWIN_NT-10.0 WernerLaptop 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
                            Time: Wed, 22 Feb 2023 14:25:03 GMT
                            Connecting to host 192.168.8.227, port 5201
                                  Cookie: WernerLaptop.1677075903.687139.6ae82
                                  TCP MSS: 0 (default)
                            [  4] local 172.28.0.1 port 1188 connected to 192.168.8.227 port 5201
                            Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test
                            [ ID] Interval           Transfer     Bandwidth
                            [  4]   0.00-1.00   sec  94.8 MBytes   794 Mbits/sec
                            [  4]   1.00-2.00   sec  96.9 MBytes   813 Mbits/sec
                            [  4]   2.00-3.00   sec  85.1 MBytes   714 Mbits/sec
                            [  4]   3.00-4.00   sec  89.8 MBytes   753 Mbits/sec
                            [  4]   4.00-5.00   sec  89.5 MBytes   751 Mbits/sec
                            [  4]   5.00-6.00   sec  93.5 MBytes   784 Mbits/sec
                            [  4]   6.00-7.00   sec  68.8 MBytes   576 Mbits/sec
                            [  4]   7.00-8.00   sec  72.8 MBytes   609 Mbits/sec
                            [  4]   8.00-9.00   sec  82.5 MBytes   695 Mbits/sec
                            [  4]   9.00-10.00  sec  95.4 MBytes   800 Mbits/sec
                            - - - - - - - - - - - - - - - - - - - - - - - - -
                            Test Complete. Summary Results:
                            [ ID] Interval           Transfer     Bandwidth
                            [  4]   0.00-10.00  sec   869 MBytes   729 Mbits/sec                  sender
                            [  4]   0.00-10.00  sec   869 MBytes   729 Mbits/sec                  receiver
                            CPU Utilization: local/sender 12.0% (2.0%u/10.0%s), remote/receiver 19.8% (5.2%u/14.6%s)
                            
                            iperf Done.
                            

                            any way it should be enough for the 200Mbit Internet connection

                            johnpozJ 1 Reply Last reply Reply Quote 0
                            • johnpozJ
                              johnpoz LAYER 8 Global Moderator @Orkopaede
                              last edited by johnpoz

                              @orkopaede I don't see any nat happening in your states.. Notice in my states where it shows 192.168.9.100 changed to the wlan interface IP of 192.168.2.253

                              If you had an existing state between those 2 IPs, then the nat wouldn't happen. You need to clear any old states between those IPs and that 5201 port so that the new connection would use the nat.

                              But hard to believe that "nat" alone would be that much of a hit..

                              edit: just for grins.. Could you disable the Kernel PTI.. That for sure could be a hit on performance.. There is almost zero sort of use case where that would be needed on a firewall.. I find it unlikely it could be such a hit to only see 16mbps vs your 200.. But have a hard time coming up with a use case where you would want/need to enable that on a firewall. And it would be a hit, the level of which is hard to say.. But why have anything enabled that would hinder the performance of your box? Unless you have a very good reason to, and have to take the performance hit.

                              https://docs.netgate.com/pfsense/en/latest/config/advanced-misc.html#kernel-page-table-isolation-pti

                              While more secure, this protection can incur a performance penalty. If untrusted users do not have access to run arbitrary code on the firewall, it can be disabled without significant security risk.

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                              O 1 Reply Last reply Reply Quote 0
                              • O
                                Orkopaede @johnpoz
                                last edited by

                                @johnpoz I will give that a try.

                                For the NAT problem. I guess the nat didn't work because everything was or is allowed in the firewall rules for this interface. How would the rules have to be set here so that the NAT can still work? Allow everything except LAN interface where the 8.227 is connected?

                                johnpozJ 1 Reply Last reply Reply Quote 0
                                • johnpozJ
                                  johnpoz LAYER 8 Global Moderator @Orkopaede
                                  last edited by

                                  @orkopaede nothing should have to change in the firewall rules, if the traffic was allowed before. But if you had an existing state, you would need to clear out those old states that were not doing nat.. Or just wait til they go away on their own on timeouts.

                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                  If you get confused: Listen to the Music Play
                                  Please don't Chat/PM me for help, unless mod related
                                  SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                  O 2 Replies Last reply Reply Quote 0
                                  • O
                                    Orkopaede @johnpoz
                                    last edited by

                                    @johnpoz Okay then I'll just reset the stats or wait. I have to go and pick up my child from kindergarten anyway so.
                                    First of all, thank you for your time and help. I'll test further tomorrow. Thank you!

                                    johnpozJ 1 Reply Last reply Reply Quote 0
                                    • O
                                      Orkopaede @johnpoz
                                      last edited by Orkopaede

                                      @johnpoz Finally with the last test

                                      6473ec23-6255-4aa3-b73b-cce4576479ae-grafik.png

                                      Speeds:

                                      iperf3.exe -c 192.168.8.227 -V
                                      iperf 3.1.3
                                      CYGWIN_NT-10.0 WernerLaptop 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
                                      Time: Wed, 22 Feb 2023 15:07:53 GMT
                                      Connecting to host 192.168.8.227, port 5201
                                            Cookie: WernerLaptop.1677078473.811245.0a2fc
                                            TCP MSS: 0 (default)
                                      [  4] local 172.28.0.1 port 5984 connected to 192.168.8.227 port 5201
                                      Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test
                                      [ ID] Interval           Transfer     Bandwidth
                                      [  4]   0.00-1.00   sec  77.4 MBytes   648 Mbits/sec
                                      [  4]   1.00-2.00   sec  81.8 MBytes   686 Mbits/sec
                                      [  4]   2.00-3.00   sec  87.6 MBytes   735 Mbits/sec
                                      [  4]   3.00-4.00   sec  65.5 MBytes   550 Mbits/sec
                                      [  4]   4.00-5.00   sec  96.8 MBytes   812 Mbits/sec
                                      [  4]   5.00-6.00   sec  90.2 MBytes   757 Mbits/sec
                                      [  4]   6.00-7.00   sec  91.9 MBytes   771 Mbits/sec
                                      [  4]   7.00-8.00   sec  75.6 MBytes   635 Mbits/sec
                                      [  4]   8.00-9.00   sec  78.8 MBytes   660 Mbits/sec
                                      [  4]   9.00-10.00  sec  88.4 MBytes   742 Mbits/sec
                                      - - - - - - - - - - - - - - - - - - - - - - - - -
                                      Test Complete. Summary Results:
                                      [ ID] Interval           Transfer     Bandwidth
                                      [  4]   0.00-10.00  sec   834 MBytes   699 Mbits/sec                  sender
                                      [  4]   0.00-10.00  sec   834 MBytes   699 Mbits/sec                  receiver
                                      CPU Utilization: local/sender 11.0% (1.4%u/9.6%s), remote/receiver 2.5% (0.6%u/1.9%s)
                                      

                                      Okay now i have to go.

                                      1 Reply Last reply Reply Quote 0
                                      • johnpozJ
                                        johnpoz LAYER 8 Global Moderator @Orkopaede
                                        last edited by johnpoz

                                        @orkopaede you can always just kill any state in the state table directly. Under diagnostics, state table.

                                        edit:
                                        That seems low for whatever reason.. Notice in mine it was a very small hit to the speed, compared to just lan to lan speed.. And your on an I5.. I would think that is more powerful than my sg4860..

                                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                                        If you get confused: Listen to the Music Play
                                        Please don't Chat/PM me for help, unless mod related
                                        SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                        O 1 Reply Last reply Reply Quote 1
                                        • O
                                          Orkopaede @johnpoz
                                          last edited by

                                          @johnpoz i selected the wrong interface...

                                          bae59868-48e8-403e-9ca1-1d1bdc2aa6d4-grafik.png

                                          LAN not TESTINTERNET

                                          johnpozJ 1 Reply Last reply Reply Quote 0
                                          • johnpozJ
                                            johnpoz LAYER 8 Global Moderator @Orkopaede
                                            last edited by

                                            @orkopaede yeah you need to put the nat on the correct interface for the direction of your traffic flow ;)

                                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                                            If you get confused: Listen to the Music Play
                                            Please don't Chat/PM me for help, unless mod related
                                            SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                            O 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.