HORRIBLE Speeds with 1.2.3 on PE1950 with Quad Intel (igb)



  • I flew down to our data center recently to decommission our ASA 5510's replacing them with pfSense 1.2.3. Since then, I have noticed that we aren't getting anywhere near the speeds that the ASA had been providing.

    The average total throughput on the network is ~10MB/s, ~94%+ of which is non-crypto.

    When I initiate test downloads of large files, I start out with a ~3MB/s speed which decreases rather steep on a linear scale to ~20KB/s. This is true of HTTP uploads/downloads and scp uploads/downloads.

    Also most of our ~4MB data sets (JSON) get truncated now, making them unusable for our applications.

    I've read a few posts that say setting these to 0 can help with the igb driver:

    dev.igb.1.enable_lro: 0
    dev.igb.1.enable_aim: 0

    If this is something I should attempt, do I truly need to reboot or can I set these in the conf and run a sysctl -p without having downtime?

    Any other thoughts?

    TIA

    …..........................................

    Machine specs are as follows:

    Dell PowerEdge 1950 Gen 3
    Dual Intel Xeon Dual Core 5148's @ 2.33GHz 1333FSB
    8GB Mem (4 x 2GB PC-4200)
    Dual 146GB 15K SAS Drives in RAID 1

    Intel PT 1000 Quad Port Gigabit Ethernet

    igb0: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 1.4.1="">port 0xece0-0xecff mem 0xfd0e0000-0xfd0fffff,0xfce00000-0xfcffffff,0xfd0dc000-0xfd0dffff irq 18 at device 0.0 on pci16
    igb0: Using MSIX interrupts with 3 vectors
    igb0: [ITHREAD]
    igb0: [ITHREAD]
    igb0: [ITHREAD]
    igb0: Ethernet address: 00:1b:21:….
    igb1: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 1.4.1="">port 0xecc0-0xecdf mem 0xfd0a0000-0xfd0bffff,0xfcc00000-0xfcdfffff,0xfd0d8000-0xfd0dbfff irq 19 at device 0.1 on pci16
    igb1: Using MSIX interrupts with 3 vectors
    igb1: [ITHREAD]
    igb1: [ITHREAD]
    igb1: [ITHREAD]
    igb1: Ethernet address: 00:1b:21….
    igb2: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 1.4.1="">port 0xdce0-0xdcff mem 0xfcae0000-0xfcafffff,0xfc800000-0xfc9fffff,0xfcadc000-0xfcadffff irq 16 at device 0.0 on pci17
    igb2: Using MSIX interrupts with 3 vectors
    igb2: [ITHREAD]
    igb2: [ITHREAD]
    igb2: [ITHREAD]
    igb2: Ethernet address: 00:1b:21:….
    igb3: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 1.4.1="">port 0xdcc0-0xdcdf mem 0xfcaa0000-0xfcabffff,0xfc600000-0xfc7fffff,0xfcad8000-0xfcadbfff irq 17 at device 0.1 on pci17
    igb3: Using MSIX interrupts with 3 vectors
    igb3: [ITHREAD]
    igb3: [ITHREAD]
    igb3: [ITHREAD]
    igb3: Ethernet address: 00:1b:21:…

    <snip>dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.4.1
    dev.igb.0.%driver: igb
    dev.igb.0.%location: slot=0 function=0
    dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10d6 subvendor=0x8086 subdevice=0x145a class=0x020000
    dev.igb.0.%parent: pci16
    dev.igb.0.debug: -1
    dev.igb.0.stats: -1
    dev.igb.0.flow_control: 0
    dev.igb.0.enable_lro: 1
    dev.igb.0.enable_aim: 1
    dev.igb.0.low_latency: 128
    dev.igb.0.ave_latency: 450
    dev.igb.0.bulk_latency: 1200
    dev.igb.0.rx_processing_limit: 100
    dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 1.4.1
    dev.igb.1.%driver: igb
    dev.igb.1.%location: slot=0 function=1
    dev.igb.1.%pnpinfo: vendor=0x8086 device=0x10d6 subvendor=0x8086 subdevice=0x145a class=0x020000
    dev.igb.1.%parent: pci16
    dev.igb.1.debug: -1
    dev.igb.1.stats: -1
    dev.igb.1.flow_control: 0
    dev.igb.1.enable_lro: 1
    dev.igb.1.enable_aim: 1
    dev.igb.1.low_latency: 128
    dev.igb.1.ave_latency: 450
    dev.igb.1.bulk_latency: 1200
    dev.igb.1.rx_processing_limit: 100
    dev.igb.2.%desc: Intel(R) PRO/1000 Network Connection version - 1.4.1
    dev.igb.2.%driver: igb
    dev.igb.2.%location: slot=0 function=0
    dev.igb.2.%pnpinfo: vendor=0x8086 device=0x10d6 subvendor=0x8086 subdevice=0x145a class=0x020000
    dev.igb.2.%parent: pci17
    dev.igb.2.debug: -1
    dev.igb.2.stats: -1
    dev.igb.2.flow_control: 0
    dev.igb.2.enable_lro: 1
    dev.igb.2.enable_aim: 1
    dev.igb.2.low_latency: 128
    dev.igb.2.ave_latency: 450
    dev.igb.2.bulk_latency: 1200
    dev.igb.2.rx_processing_limit: 100
    dev.igb.3.%desc: Intel(R) PRO/1000 Network Connection version - 1.4.1
    dev.igb.3.%driver: igb
    dev.igb.3.%location: slot=0 function=1
    dev.igb.3.%pnpinfo: vendor=0x8086 device=0x10d6 subvendor=0x8086 subdevice=0x145a class=0x020000
    dev.igb.3.%parent: pci17
    dev.igb.3.debug: -1
    dev.igb.3.stats: -1
    dev.igb.3.flow_control: 0
    dev.igb.3.enable_lro: 1
    dev.igb.3.enable_aim: 1
    dev.igb.3.low_latency: 128
    dev.igb.3.ave_latency: 450
    dev.igb.3.bulk_latency: 1200
    dev.igb.3.rx_processing_limit: 100</snip></intel(r)></intel(r)></intel(r)></intel(r)>


  • Rebel Alliance Developer Netgate

    The output of the following commands may help:

    The full output of "dmesg"

    "ifconfig -a"

    "vmstat -i"

    "pciconf -lv"



  • Hi there,

    I'm having the same issue on a Dell R300 server with a quad Intel. Used to work fine on a 1.2.2 version but since the upgrade we got a degraded performance on a net to net transfer, no such probleme when transferring over internet.

    Have you managed that issue now?

    r@


Locked