PFSense 1.2.2 - 30MB Fiber Link - Uploads are being limited to between 3-10MB.



  • Dear PFSense forum,

    This is my first post here and I am sure there is more to come. I like the idea of being able to support PFSense and request features for money, I will be sure to use that in the future.

    Anyway, I inherited the Administration of a small media-rich/web-content/mediafirm, Whom has a 30MB Dedicated line via a CISCO 1811 to Telus Backbone.

    This runs through a box which we've dedicated as our PFSense server. It has 3 NIC's which are Via Rhine III Cards, I plan on building a new PFSense server with Intel Desktop Pro NIC's soon though,

    In the meantime, THERE IS A MAJOR ISSUE! Any network traffic that goes through our NAT/PFSense box, Is severely limited in its upload. I have tried to troubleshoot this and I have plugged in a notebook with no firewall to the CISCO and after assigning it one of our unique IP's, I am able to consistantly hit 30MB Upstream. The PFSense box, however, will not go above 10MB Upstream traffic, In fact, sometimes it doesn't hit 4MB. I am testing to a local host that runs off speedtest.net and it is located a few blocks away from me locally.

    PFSense gets 30-34MB (Burst) Downstream each test, just the upload is where the problem was.

    We noticed our WAN Link was half duplex on PFSENSE, and the counters on the CISCO are specific to full duplex so we modified the config file to reflect a forced full duplex setting and restarted the interfaces but it did not resolve the slow upstream issue. We are running some packges, Like bandwidthD to monitor traffic, but there is presently no Traffic Shaping or QoS enabled.

    Besides this, Nothing has changed on our configuration side of PFsense, and this problem seems to have appeared on its own in the past 3 weeks. Would it be wise to attempt a restore to a month ago as a troubleshooting step, as backups are enabled on PFSense?

    We have a FAILOVER configured to a cable modem which has not failed over for some time, But its there if need be, I know its not the problem because the cable modem wont even hit 1MB Upstream let alone 7.

    I have read about other users having simular issues to the one we are having but there was no updated posts or clear resolutions, and it is business critical that I fix this issue so I am seeking some advice from people here. Where should I begin?

    There are incrementing errors on the LAN In, interface, Not very many.

    We were having problems with Promiscuous mode going on and off multiple times a second but resolved that by removing the rate/bandwidthd package, Which unfortunately still lists in the PFsense control panel but gives a 404 error when you click the link.

    Any help here about why PFSense would be limiting our 30MB Fiber line to less than 10 would be GREATLY appreciated and possibly donated on behalf of me.'

    (If this is the incorrect forum, Please forgive me, I thought NAT Is most specific to the issue I am having, but it may be worth moving this topic to the general forum if that is a better area for this type of post.)

    We are having voodoo problems with this box as of recently, it began to limit our 30MB Dedicated fiber line ($3400/Month) to about 3MB on the upload, But a direct attacted Video Appliance was able to effortlessly hit 30MB Upstream consistantly when not going through PFSense (It's own IP Address, Video Content Server), I can't figure that out but imagine an update might help.

    Can you guys please give me a hand?

    Here is one from a month ago:



  • I would first test with a live boot cd to see if it is a hardware problem. There is always Commercial Support if it's business critical.



  • Cide,

    Did you ever resolve your issue?

    I am having a similar issue with a 10Mbps Fiber Link from XO Communications, Downstream is fine, put our upstream caps at around 3Mbps.

    Hooking a laptop directly the the ethernet handoff works correctly, so I'm assuming it iss something within PFSense 1.2.3 Configuration or possibly hardware related.



  • Check the interfaces under Status > Interfaces. Look for errors, collisions where there shouldn't be.



  • No collisions reported.

    WAN:

    Media  100baseTX <full-duplex>In/out packets 11696702/9921518 (1.68 GB/3.56 GB)
    In/out errors 0/0
    Collisions 0

    LAN:

    Media  100baseTX <full-duplex>In/out packets 12370614/13220069 (3.88 GB/2.90 GB)
    In/out errors 0/0
    Collisions 0

    XO and I set our routers to 100FDX since we were having the same issue with them set to auto-negotiate.</full-duplex></full-duplex>



  • I'd be curious to see what happened if you set both ends to 100/half rather than 100/full.  Also, can you get interface stats from their side, not just yours?



  • It still sounds like a duplex mismatch. It sounds like you have a switch between your WAN and the Cisco router, if that's the case you must force speed and duplex on the switch port, not the router (and if you do on the router, you must on its switch port as well - if you're going to force, make absolutely sure everything is forced, or you will end up with a duplex mismatch and serious performance problems). If you have an unmanaged switch where you can't force the port, you must run autonegotiate on everything.

    I've seen some VIA NICs that refuse to force speed and duplex when it's manually set. In those cases, the interface status showed incorrectly though so that isn't the same case here. I'd switch out the NICs with a different chipset and see what happens, if your switch ports and router are definitely all set correctly.



  • Yeah, I agree.



  • Some more oddities.

    We sent out an email to a  very large amount of individuals on Friday's and I noticed my bandwidth meter's transmit going through the roof.

    It was actually capping around 13Mbps (Our fiber is supposed to cap at 10Mbps)

    I cannot replicate this on speedtest.net or speakeasy.net.

    Another oddity is that we have 4 interfaces. WAN, LAN, DMZ, and Wireless.

    WAN and LAN are pretty self explanatory.

    I have the DMZ setup for Webserver with a public IP, and restrict the DMZ from accessing the LAN.

    The Wireless interface connects directly to a LAN port on an old Westell Wireless HUB.

    The DMZ and Wireless interfaces use the same make/model NIC.

    I connected my laptop the the WLAN and ran a speedtest, and low and behold I am hitting close to our 10Mbps.

    Some details:

    ISP Router - Set for 100FDx

    Cross Over cable directly connected between ISP Router and WAN Interface.

    WAN
    bge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
           options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>media: Ethernet 100baseTX <full-duplex>status: active

    LAN
    bge1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
           options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:22:19:d5:ee:93
           inet 192.168.168.1 netmask 0xffffff00 broadcast 192.168.168.255
           inet6 fe80::222:19ff:fed5:ee93%bge1 prefixlen 64 scopeid 0x4
           media: Ethernet autoselect (1000baseTX <full-duplex>)
           status: active

    DMZ
    em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
           options=19b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4>ether 00:1b:21:39:55:c1
           inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
           inet6 fe80::21b:21ff:fe39:55c1%em0 prefixlen 64 scopeid 0x1
           media: Ethernet autoselect (1000baseTX <full-duplex>)
           status: active

    WIRELESS
    em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
           options=19b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4>ether 00:1b:21:39:57:73
           inet6 fe80::21b:21ff:fe39:5773%em1 prefixlen 64 scopeid 0x2
           inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255
           media: Ethernet autoselect (100baseTX <full-duplex>)
           status: active

    LAN is connected to a Managed HP ProCurve Switch 2810-24G with Duplex settings set to Auto.</full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4></up,broadcast,running,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4></up,broadcast,running,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast>



  • Another update, this appears to only be affecting HTTP Upstream.

    SMTP and FTP are able to consistently hit our upstream cap and maintain it.

    What would be causing only HTTP to perform subpar? We have tested with QoS disabled, and QoS enabled with everything as default and HTTP set to higher priority, no difference.



  • Hmmm, I assume this is bulk SMTP and/or FTP traffic?  What kind of traffic is the HTTP?  Is it bulk too?  Or lots of smaller packets?  Maybe a limitation on the number of packets/sec you can push thru?



  • The email I described was likely 2MB in size, but was sent to about 800 recipients.

    The FTP and HTTP test consisted of uploading the same 1GB file to a hosting company I use. FTP thru FileZilla and HTTP through cPanel.

    FTP hits cap, but HTTP still struggles around 1.5-3Mbps just as illustrated on Speedtest.net and Speakeasy.net.



  • Is it possible to try a different http client?  It would eliminate questions about, say, send window size or whatever…



  • I've run the speedtest.net, speakeasy.net, and a cPanel upload in both IE and Firefox.  :-\



  • Oh, sorry, was going by the comment about the http test being via cpanel.



  • I guess without seeing a wireshark capture, it's really hard to suggest anything more…



  • Are you using any kind of HTTP proxy? (like squid, for example)



  • I had installed squid at one point in time, but removed it.

    I'm almost wondering if one of my packages didn't uninstall properly. I currently do not have any packages installed. I will be performing a fresh install this coming Friday when I have a maintenance window. If I exhibit the same symptoms I may give the BETA a try.


Log in to reply