Hardware questions do to poor routing speed



  • I have pfsense 2.1-release running on a atom 330 cpu with 2G RAM.

    I have 2 internet connections 100Mb and 500Mb

    I have no issues pulling 100Mb on the 100Mb line

    on the 500Mb I can only do about 220-250Mb, so I have been looking around here and come to the conclusion that I might need a new MB as the cpu could be the issue,
    but the problem is that usage on the cpu is never over 25%

    If I connect my mac book pro to the same cable I can pull 498Mb so it is not a connection issue.

    any suggestions why I can not get more throughput when there is no CPU load



  • It's been a while since I used a 330, but I remember there being something about them that capped out at about that point.  Might have been the crappy Realtek NICs they all shipped with or just a general lack of CPU grunt.

    EDIT:  Here we go.
    http://forum.pfsense.org/index.php/topic,19865.msg102535.html#msg102535



  • I gave up on that realtek card and added some better NICs but that did not help Intel 82541 as it is a jetway MB and I had a daughter board laying around,

    no sucess :(


  • Netgate Administrator

    Have a look at the per-core CPU usage, use 'top -SH' at the console.
    The 330 is a dual core CPU with hyperthreading so it appears as 4 cores. If one of those cores is maxed out the overall CPU usage may only appear as 25%.

    Steve



  • yep that appear to be the problem :(

    need new fw

    11 root    171 ki31    0K    32K RUN    3  32:35 90.77% idle{idle: cpu3}
      11 root    171 ki31    0K    32K CPU2    2  32:15 72.85% idle{idle: cpu2}
      11 root    171 ki31    0K    32K CPU1    1  32:35 62.60% idle{idle: cpu1}
        0 root    -68    0    0K  128K -      0  1:14 63.87% kernel{em2 taskq}
      11 root    171 ki31    0K    32K CPU0    0  32:12 55.96% idle{idle: cpu0}



  • We may have a real life Atom throughput figure of 220-250Mbps.


  • Netgate Administrator

    Hmm, all of those cores are at least 50% idle. What state was the machine in when those figures were taken?
    Did top show high interior load? Are those pci NICs all on the same bus?

    Steve



  • that is taken when I download with about 220-250Mbps

    all nic are as a daughter board for a jetway MB so they are on the same bus, network config

    lagg 0
    realtek lan 
    intelnet lan

    intel net 100Mbps internet
    intel net 500Mpbs internet


  • Netgate Administrator

    So this box has 2 realtek and 3 intel NICs yes?

    The on board NICs may be on a separate bus in which case you may see different results between different NICs. Which are you testing between? How is your lagg interface configured?

    Steve



  • Current setup is
    1 realtek
    3 intel

    the onboard nic (realtek) and 1 intel on the daughter board is in lagg0

    laggproto lacp
    laggport: em1 flags=1c <active,collecting,distributing>laggport: re0 flags=1c <active,collecting,distributing>1 internet connection on em0 (100Mb)
    1 internet connection on em2 (500Mb)

    test between em2 and lagg0.</active,collecting,distributing></active,collecting,distributing>


  • Netgate Administrator

    Ok. The lagg is only giving you redundancy in that setup, it's probably slowet than a single NIC.
    Try running 'pciconf -lv' to check if they are all on the same bus.

    Steve



  • output:
    re0@pci0:1:0:0: class=0x020000 card=0x816810ec chip=0x816810ec rev=0x02 hdr=0x00
        class      = network
        subclass  = ethernet
    em0@pci0:2:4:0: class=0x020000 card=0x10768086 chip=0x10768086 rev=0x05 hdr=0x00
        class      = network
        subclass  = ethernet
    em1@pci0:2:6:0: class=0x020000 card=0x10768086 chip=0x10768086 rev=0x05 hdr=0x00
        class      = network
        subclass  = ethernet
    em2@pci0:2:7:0: class=0x020000 card=0x10768086 chip=0x10768086 rev=0x05 hdr=0x00
        class      = network
        subclass  = ethernet


  • Netgate Administrator

    Ok so the Intel NICs are on bus 2 and the realtek on bus 1. Additionally the realtek device is pci-e and the Intel NICs are pci. It does depend how those buses are connected upstream.
    Try removing lagg and using using just re0 for LAN.

    Steve



  • when breaking lagg and using re0 as the inside network I was able to pill close to 300Mbps and the cpu usages for the wan interface hit 73%


  • Netgate Administrator

    Ah, some improvement then. Is that 75% overall CPU usage?
    It would be interesting to see the per core usage. The 330 isn't much below the d510 in performance terms and that can manage 485Mbps. That is between PCIe Intel NICs though.

    Steve



  • It is not overall it is on a single core, I will see i I can catch a dump of the load later today



  • im starting to wonder if there is a bad network interface in this box as this morning I can do about 224Mbps but there is no cpu load

    11 root    171 ki31    0K    32K CPU3    3 493:00 100.00% idle{idle: cpu3}
      11 root    171 ki31    0K    32K RUN    2 490:23 95.17% idle{idle: cpu2}
      11 root    171 ki31    0K    32K CPU1    1 492:19 79.30% idle{idle: cpu1}
      11 root    171 ki31    0K    32K CPU0    0 462:42 56.88% idle{idle: cpu0}
        0 root    -68    0    0K  128K -      0  5:19 44.48% kernel{em2 taskq}
      12 root    -68    -    0K  200K WAIT    0  16:14 26.46% intr{irq256: re0}


  • Netgate Administrator

    Hmm, well it's clearly not the CPU that's restricting your throughput.
    Perhaps try disabling hyperthreading in the BIOS since it's clearly not using the virtual cores usefully.
    You could also try disabling anything you're not using in the BIOS, sound card, parallel port etc. One of this devices could be slowing the bus.

    Steve



  • That did not help, everything has been disabled, still only about 220-300Mbps and cpu load not over 50%



  • @armsby:

    That did not help, everything has been disabled, still only about 220-300Mbps and cpu load not over 50%

    With HT disabled you're fully tapping a core at 50% (which you might see as 70/30, 50/50, 90/10, etc.).  I'm still going to go with that's the cap for a 330, particularly since that's the same cap I hit a few years back.