Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Issue with throughput with X710-DA2 and PFsense 2.6

    Scheduled Pinned Locked Moved Hardware
    30 Posts 2 Posters 2.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      bawoodruff
      last edited by bawoodruff

      Correct, that was the summed results from iperf of the simultaneous streams.
      Also correct, this box has four igb interfaces. but the point in using this card is that I will be switching ISP's in the near future to which Im going to be using a fiber module for straight from the ONT. In the meantime, I'm trying to get this machine to use that same card with a 1gb ethernet module to save on having to recreate the wheel later when I do switch as far as the majority of the configurations. But I'm not even able to get the 900+Mbps I'm able to get with the UDMP. TBH this is the second card I've purchased, I also tried this with a Chelsio T520-CR with different SFP modules and saw the same results.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Well I would confirm you're able to pass 1G with the igb NICs before going further. That should be no problem for that device. Our own firewalls using that CPU could pass it easily.
        The C2758 with a Chelsio NIC was good for somewhere in the 3-4Gbps range.

        Steve

        1 Reply Last reply Reply Quote 0
        • B
          bawoodruff
          last edited by bawoodruff

          WAN set up on igb2 instead of the ixl interface
          iperf LAN client -> PFsense same result
          Speedtest from PFsense to internet no change also.
          Processor utilization on both are the same as earlier also.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            So that was replacing the LAN interface? Are you able to test igb as WAN?

            1 Reply Last reply Reply Quote 0
            • B
              bawoodruff
              last edited by

              No, that was replacing the WAN Interface

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Hmm, well it could be the LAN side. Try it with igb as both.

                You might also try testing against a local iperf server on the WAN side if you can to rule out anything upstream.

                Steve

                1 Reply Last reply Reply Quote 0
                • B
                  bawoodruff
                  last edited by bawoodruff

                  If I'm ssh'ed into the PFsense box, and running speedtest-cli, and only getting 300-ish download speeds how would that involve the LAN adapter? Outside of being able to SSH into the PFsense box that is.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    It doesn't but speedtest-cli is not an infallible tool, especially at higher speeds. An iperf test would be much better but really should use a local server. It's rare to find a public iperf server that can push 1G in my experience. Near me at least.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • B
                      bawoodruff
                      last edited by bawoodruff

                      My 10Gb SFP+ fiber modules I bought for the ixl interface came in today (weren't supposed to be in till tomorrow). I switched the LAN interface out with the Gbit SFP module that was in there and I'm able to get about 4.5Gbps between the LAN Client and PFsense using multiple streams running:iperf3 -c 192.168.1.1 -P 20 but still only getting 300ish when I try a normal speed test from PFsense to the internet or the LAN Client to the internet.

                      WAN: igb 1Gbit interface
                      LAN: ixl 10Gbit interface

                      Client running iperf3 -> 10Gb LAN->PFsense (iperf3 server) = 4.5ish Gbps
                      ssh -> PFsense -> Speedtest-cli -> WAN -> Internet = 300ish Mb/s
                      client running speedtest-> 10Gb LAN -> PFsense -> WAN -> Internet = 300ish Mb/s

                      The 300Mbps speed was basically the same whether the WAN was on a igb 1Gbit interface or the other ixl interface with a 1Gbit SFP module

                      I've tried multiple speed test sites from the client machine during testing and they all show basically the same speed.

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Hmm, how is the WAN actually connected here?

                        Do you see any collisions or errors on the interfaces?

                        We have seen ISPs that provide a reduced rate profile to devices other than what is registered, which could still be the UDM. Have you tried spoofing the MAC address on WAN to match the UDM?

                        Steve

                        1 Reply Last reply Reply Quote 0
                        • B
                          bawoodruff
                          last edited by

                          Cable Co. ->Docsis3.1 Modem ->Cat6->igb Interface on PFsense box

                          ISP Connection is regulated by MAC address filtering of the Modem itself.

                          I actually already tried cloning the UDMP's MAC on the PFsense box. I saw no change in speed. Just grabbed the IP address last assigned to the UDMP.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            And can I assume a client connected directly to the modem gets a public IP and can see the full bandwidth?

                            1 Reply Last reply Reply Quote 0
                            • B
                              bawoodruff
                              last edited by

                              yup

                              1 Reply Last reply Reply Quote 0
                              • B
                                bawoodruff
                                last edited by

                                Wonder if I backup my current config, and get re-install would make a difference.... Seen some posts involving throughout issues being resolved by this. although this is a rather new install to begin with tbh.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  It's an easy enough test but I wouldn't expect it to make any difference at all.

                                  I would want to test against a WAN side local iperf server to confirm that is capable of passing 1G.
                                  Testing against a remote speedtest server there are a lot of variables.

                                  Steve

                                  1 Reply Last reply Reply Quote 0
                                  • B
                                    bawoodruff
                                    last edited by bawoodruff

                                    Off to try and locate a decent (local to me) iperf server, unless someone knows of any in the Indianapolis area, lol

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      bawoodruff
                                      last edited by bawoodruff

                                      While SSH'ed into the pfSense box and running several tests using"

                                      iperf3 -c speedtest.serverius.net -p 5002 -P 10 -4 -R

                                      shows a bitrate of 280ish Mbps

                                      With the processor cores showing about 20% utilization

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        When I say local I mean actually in your location. When I do this I just use another pfSense box to server as a gateway for the unit under test and run the iperf server on that. That's not ideal but as long as the gateway device is sufficiently powerful it will expose any restrictions on the test device.

                                        Steve

                                        1 Reply Last reply Reply Quote 0
                                        • B
                                          bawoodruff
                                          last edited by bawoodruff

                                          So, I removed all the past settings I had in place and out these in place and reboot.

                                          \boot\loader.conf.local

                                          legal.intel_iwi.license_ack=1
legal.intel_ipw.license_ack=1
net.link.ifqmaxlen=2048
net.isr.defaultqlimit=2048
net.inet.tcp.soreceive_stream=1
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.syncache.bucketlimit=100
net.pf.source_nodes_hashsize=1048576

                                          System Tunables

                                          ev.igb.0.eee_control=0
dev.igb.1.eee_control=0
dev.igb.2.eee_control=0
dev.igb.3.eee_control=0
dev.igb.0.fc=0
dev.igb.1.fc=0
dev.igb.2.fc=0
dev.igb.3.fc=0
net.inet.ip.redirect=0
net.inet6.ip6.redirect=0
net.route.netisr_maxqlen=2048
net.inet.ip.intr_queue_maxlen=2048
net.inet.udp.maxdgram=16384
net.inet.tcp.syncookies=0
hw.intr_storm_threshold=0
kern.ipc.soacceptqueue=2048

                                          And with FQ_Codel queues and Suricata disabled, I now seem to be hitting 900+Mbps
                                          When I run :iperf3 -c proof.ovh.net -p 5201 -P 40 -R
                                          Had to increase the number of streams, whereas before anything past 10 made no difference.

                                          And with FQ_Codel and Suricata enabled, I now seem to be hitting 850+Mbps

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Is that testing from pfSense directly?

                                            You usually don't see any improvement with parallel streams beyond the number of NIC queues. iperf itself is still (deliberately) single threaded so it hits a single core limit there.

                                            Steve

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.