PfSense throtteling WAN bandwidth?



  • pfSense 2.1.5 running on a dual core HP w 2 gigs RAM.  WAN on 1G Intel NIC, LAN on USB 2.0 10/100 RealTec NIC.  Wan is bridged cable modem with 5 up and 50 down.  When lap top plugged directly into cable modem, I get 50 MB down.  LAN client only gets 5 MB down through pfSense.  No load balance/limiter/traffic shaping packages installed installed or enabled.  Only pfBlocker, SquidGuard and OpenVPN.

    What else could cause the low bandwidth on the LAN side?  Could it be USB NIC? USB 2.0 is supposed to 485 MB, and I'm only asking for 50.

    I'm confused.  Any one have a clue?


  • Netgate Administrator

    Could be a connection problem, duplex issue. Check the Status: Interfaces: page for errors.

    Steve



  • Would be interesting to see CPU time during speedtests. USB and RealTek sounds like a one-two punch for lots of interrupts.



  • No errors or dropped packets on NICs.  Interrupts go from 47% to 56% during a speed test, CPU load goes from .35 to .57.  No obvious memory hogs or CPU intensive procs in top.  Running the pfBlocker on or off made no change.


  • Netgate Administrator

    Is it negotiating the speed/duplex correctly on both interfaces? Could be a 10Mb connection.
    I agree that a Realtek USB NIC is pretty much asking for trouble unfortunately.

    Steve


  • LAYER 8 Global Moderator

    I would look to a 10Mb connection as the problem as well.

    So we we are clear your talking 50Mbits per second not 50MBytes – B is Bytes, b is bits - huge difference!!

    You mention squidguard - what speeds do you get without a proxy?



  • I think we were all following right up until you said you were running your LAN on a USB dongle :)  Lets clean up the simple things before we start blaming the OS.  You can't run your LAN on a USB adapter and expect production results.

    Here's a quote from the minimum hardware section:

    "The numbers stated in the following sections can be increased slightly for quality NICs, and decreased (possibly substantially) with low quality NICs. All of the following numbers also assume no packages are installed."

    Unfortunately, a USB NIC is going to fall into that "low quality NIC" category.  Also, every package (e.g. squid, AV, etc) adds potential overhead.

    Yes, the USB 2.0 standard supports theoretical throughput up to 480 Mbit, but real world numbers will tell you that even under ideal conditions you're lucky to see 1/4th of that.  Not to mention, are your USB ports configured in full speed or hi-speed mode?  Also, if you're on an older board, it is possible that your USB ports are v1.1 which hamstrings you even more @ 12 Mbit max.

    Do yourself a favor, don't spend any more time troubleshooting this setup.  Go purchase a Gigabit NIC (most would say preferably intel), remove all your packages and re-do your speedtest with your ISP's speed testing site.  I expect you'll see your full bandwidth.

    Just to give you an example, my system (P4-2.4 Ghz, 512 MB, 40 GB HDD, x2 Intel 10/100 NIC) is pushing 7 up / 100 down without issue.  So, I'm pushing twice the bandwidth with less than half your specs.

    There are several factors that may be contributing to your slow speeds, but we're all reasonably confident that one of them is not PFsense.



  • Dump the USB adapter and use a cheap VLAN switch if possible.


  • Netgate Administrator

    Lets not jump to conclusions here. Yes, USB NICs are to be avoided in general but there are plenty of people out there using them with pfSense successfully. I would expect to get >5Mbps out of even the crappiest thing if it's configured correctly. Perhaps it's falling back to USB 1 mode? Perhaps it's negotiating to 10Mbps? All those things can be discovered with a few simple tests.

    Steve



  • Well - Let me rephrase.  USB will most likely be slow and unreliable.

    Thats a best case scenario.

    Worst case scenario, it will be offline more than online.

    The only thing USB has ever done for me reliably is charge my cellphone.

    I have exactly 1 device that has never been flakey on USB and thats my nextar drive inclosure which for some miraculous reason is reliable on usb.

    Unless money is a huge huge factor, I'd get the usb NIC out of the mix.



  • @jriggin:

    No errors or dropped packets on NICs.  Interrupts go from 47% to 56% during a speed test, CPU load goes from .35 to .57.  No obvious memory hogs or CPU intensive procs in top.  Running the pfBlocker on or off made no change.

    57% cpu time on a dual core cpu is an entire core running at 100%, and current PFSense is mostly single threaded for firewall. If someone with more understanding could correct me if wrong, but I think he's CPU bound from interrupts.


  • Netgate Administrator

    Or two cores each running at 57% or some other combination.
    Run 'top -SH' at the console to see how your CPU is actually loaded across the cores.

    Steve



  • I agree the USB NIC is not an enterprise quality solution.  This is for my lab, and prior to Brighthouse switching out my cable modem (in bridge mode) and upgrading from 2.1.4 -> 2.1.5 I was getting 30 megs from WAN to LAN.  I'm supposed to get 50, therefor the cable modem switch.  I admit it was pretty dumb to change modem and pfSense version at once, but sometimes I do dumb things.

    pfSense runs on Dell Optiplex 755 small desktop that only takes 1/2 height cards.  Any suggestions?  I've tried StartTek, Linksys and D-Link 1/2 height cards and pfSense didn't notice any of them.  Hence the USB.  :(

    Thing is, everything was working until upgrade and new modem.  Is there a way to test bandwidth directly between pfSense and WAN gateway without using LAN NIC?



  • BTW: neither NICs  on auto negotiate.  Intel on 1000TXFull, usb on 100TXFull.  Cores seem equally loaded under all conditions.

    Turning off Squid seems to lower proc lad a few pints but no change in throughput.


  • Netgate Administrator

    Ok so the 2.1.4 to 2.1.5 update was mostly security fixes, I'd be surprised if it affected your USB NIC. The modem change is the likely suspect here. Since it's on the WAN side it may be nothing to do with your USB NIC and we all just jumped on that.  ::)
    Check the output of ifconfig at the console. Check the 'media:' line for your WAN NIC is saying autoselect and is at 100Mbps or more.

    Test the download speed at the pfSense console. This test will eliminate the USB NIC as a source of problems as you suggested.

    [2.1.5-RELEASE][root@pfsense.fire.box]/root(3): fetch -o /dev/null http://cachefly.cachefly.net/10mb.test
    /dev/null                                     100% of   10 MB  780 kBps 00m00s
    
    

    You should see much faster than that, I'm in the UK. Other test file sites might prove better.

    Steve

    Edit: Just seen your other post. Why aren't they on auto-negotiate? That almost certainly the cause of your problem.



  • The NICs were on auto-negotiate by default.  I changed them to see if it would effect the problem.  They auto negotiate to the same thing I had them set to.

    I got slightly slower download than you did.

    [2.1.5-RELEASE][root@pfsense]/root(1):  fetch -o /dev/null http://cachefly.cachefly.net/10mb.test
    /dev/null                                     100% of   10 MB  746 kBps 00m00s
    

  • Netgate Administrator

    Auto-negotiating to 100FD is not the same as setting it manually. Many devices will try to negotiate, fail, and then default to some lower setting like 10HD.

    Can you get a decent download speed from cachefly without the pfSense box in the way? If so then it looks like your WAN connection is at fault.

    Steve



  • My Linksys E2500 wireless router is plugged into Brighthouse modem right next to pfSense WAN.  On wireless, my laptop gets 40 Mbps down as tested at speedtest.net and speedtest.bhn.net.  (Changing the Brighthouse port pfSense is plugged in to makes no difference in results.)  If I plug the same laptop into the LAN port on pfSense, I get 5 Mbps down.



  • My suggestion is get a vlan switch.


  • Netgate Administrator

    Make sure that you can get a significantly higher speed from cachefly directly from your laptop (not though pfSense).
    If that is the case then we have shown that it's the pfSense WAN connection that is as fault and we can try to diagnose it further.

    Steve

    Edit: typo



  • I got 10mb using the file from cachefly on my LAN from the pfSense box:

    [2.1.5-RELEASE][root@pfsense]/root(7): fetch -o /dev/null http://172.16.77.50/10mb.test
    /dev/null                                     100% of   10 MB   10 MBps
    
    

    (That would be over the much maligned USB NIC)

    I also got 10 MB on my laptop plugged into Brighthouse modem.

    It HAS to be the WAN port (which worked fine on 2.1.4) or something in pfSense doesn't like something about the new Brighthouse modem.  But I get the same result in any modem RJ45 port, and any other device connected to the modem gets over 30 MB on speed tests.

    Any other ideas?  Or should I just wipe and re-install from ISO?



  • Well - 10Mb/s is better than NoMb/s I guess.

    And if you connect your laptop directly to the modem, problem disappears?

    Even if you get this working, You will be ok with the loss of 20Mb/s?

    Just a thought - Switch the LAN and WAN interfaces and see what happens.


  • Netgate Administrator

    That's MB/s so for a NIC that's negotiated to 100Mbps it's not that bad.  ;)

    My money is on some basic problem between the NIC and the new modem. Some times two devices don't quite comply with the specs correctly and won't work. These things happen.  ::)
    Simple test. Put a switch in between the modem and the USB NIC on the pfSense box. If it's a layer 1 problem that may prove it.

    Steve



  • Sorry - Mixed my apples and oranges.

    Did he try switching LAN and WAN?


Log in to reply