Bandwidth loss across pfSense?

  • I set up the version 2.1.2 release with a WAN and a LAN interface.  NAT is automatic outbound rule generation.  The WAN is connected to a Comcast cable modem with 100 Mbps down and 20 Mbps up.  When I run a speed test with a client PC on the LAN side I get about 65 Mbps download average over 3 tests.  Connected to the Comcast cable modem directly with the same client PC the average download speed over 3 tests is about 96 Mbps.

    Has anyone else experienced an approximate 33% disparity across their pfSense box?

    I also set up a test bed using JPerf with a switch and three computers.  One PC is the JPerf server representing the WAN, another set up as a pfSense box and the third as the JPerf client on the LAN.

    Connecting the client directly to the server resulted in an average bandwidth of 300 Mbps.
    When I placed the pfSense between the client and server the result was an average bandwidth of 200 Mbps. 
    This also works out to an approximate 33% bandwidth disparity across the pfSense box.

  • LAYER 8 Global Moderator

    What are the hardware specs of your pfsense box?

    What doesn't make a lot of sense is that with your ipferf test you were able to do 200mbps, so you would think it should be able to handle the 100mbps connection of your internet connection.

    I run pfsense in a VM, and I only have the 50/10 package from comcast - but I see 57 to 58mbps down and like 11 to 12 up through pfsense.  So I am getting the bandwidth I pay for.  I could hook up a box on the wan side and repeat your iperf test to see it can do.  I know from lan segment to lan segment through pfsense I have see 500Mbps.  Which I know is a hit since on the same lan segment not going through pfsense I can do really low 900's or really high 800's depending, etc.

    And pfsense is running as a vm with 512MB ram on hp n40L microserver with the vmxnet3 virtual interfaces connected to intel physical nics.

  • Thanks for responding….
    I'm pretty sure it is not the hardware.  It is a physical Dell server with an I3-2100 CPU at 3.10 GHz with 8 GBs of ram and Intel NICs .

    The speed test on the production set up and the JPerf testing set up are two different physical set ups.  Since the one pf box connected to Comcast is in production I set up the JPerf test bed with a different pf box to eliminate any incidental traffic affecting the outcome.

    The JPerf test bed pf box is a physical Dell desktop PC that as a dual core Intel E8400 at 3.00 GHz with 4 GBs of ram and Intel NICs.

    The JPerf test bed is a fresh install and all I changed was to add IP Addresses to the WAN and LAN interfaces.

  • LAYER 8 Global Moderator

    yeah you clearly should be able to do 100mbps with that hardware.  What are the nics - are you connecting at gig or 100?  Possible errors on the connection?  I would think a duplex mismatch would give you worse speeds.  What are the exact nics?

    My point on your test bed was if your seeing 200mbps with that, clearly pfsense can push 200 so why are you not seeing that with your isp connection?  Bad cable?  Something is clearly wrong..

  • The NICs are gig full duplex.  I have a managed switch so I can see that all active connections to the switch are gig full duplex.  Also the port counters on the switch show no in or out errors and no discards.

    The Comcast pf box has a quad gig Intel pro card and the test bed PC has an onboard gig Intel pro and a pci-e gig Intel pro card.

    The pf box can definitely handle more than the 100 Mps from my isp.  When I connect my client directly to the isp side of the switch I am getting close to 100 Mps on the speed test.  Note that when I am running the speed test there are other users connected so I do not expect to get the full 100 to myself.  That is why I set up the test bed.

    Regardless of absolute speeds I still see a difference of about 33% on either set up when I compared the speed connected through the pf box versus connected without the pf box.
    Comcast- WAN side 95/LAN side 65
    Test bed- WAN side 300/LAN side 200

    Any additional thoughts would be appreciated.

  • LAYER 8 Global Moderator

    Well that is odd.. There is clearly going to be a hit your natting and running through a firewall so its not going to be wire speed like your just connected to a switch - but it sure would not be that high of a number.

    I will connect something directly to my cable modem later tonight if I get time to see if I get say 60mbps and only 58mbps through pfsense – but I never looked into it before since I only pay for 50 and seeing almost 60 so I didn't think of there being any sort of significant hit on the performance.

Log in to reply