Internet is slow behind pfsense



  • Hi guys
    i just come across something today,
    when i use my machine hehind pfsense as gateway the internet become slow, when i run a speedtest.net its comes back with 
    and without pfsense its 2.60 Mbps

    i have no packages installed really, no squid.
    firewall rules are basic, just to use the internet

    NICS versions are VMX0 and VMX1,

    Pfsense is on a VMware box.

    thank you


  • LAYER 8 Global Moderator

    well seems your internet is just slow 2.6mbps is not something I would classify as speedy ;)

    so if your seeing the same speeds with or without pfsense - what are you using to judge that it is slow?



  • @johnpoz:

    well seems your internet is just slow 2.6mbps is not something I would classify as speedy ;)

    so if your seeing the same speeds with or without pfsense - what are you using to judge that it is slow?

    home i have two lines coming inside :)
    200 MB/sec and 4MB/sec
    i am using the 4Mbps for the Lab,
    when i send the test using Pfsense the speed is just low till 1Mbps and some is less than 1 MB.
    wihtout pfsense its reach 2 MB

    is this something with jumbo frames?


  • LAYER 8 Global Moderator

    Why would you have jumbo frames.. Do you have jumbo frames on your public side connection?


  • Netgate Administrator

    We have combinations of milli and Mega, bits and Bytes here. Please try to use consistent units especially when discussing bandwidth.  ;)

    1Mbps is very low. Do you have some connection issue? Do you see collisions in Status > Interfaces? What does 'ifconfig -a' report?

    Steve



  • @stephenw10:

    We have combinations of milli and Mega, bits and Bytes here. Please try to use consistent units especially when discussing bandwidth.  ;)

    1Mbps is very low. Do you have some connection issue? Do you see collisions in Status > Interfaces? What does 'ifconfig -a' report?

    Steve

    sorry Steven
    i meant i have home 200Mb/s line home which is working fine without any issues
    i have on my LAB a sport line with 4 Mb /s
    when i am behind the Pfsense i run a test on speedtest.net i get 1Mbps/s which is really low.
    when i am connected directly to the ISP model and i run the speedtest.net i get between 2 and 3.5 Mbps /s

    Ifconfig -a report is

    $ ifconfig -a
    vmx0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=60009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6>ether 00:0c:29:c0:1d:a3
    	inet6 fe80::20c:29ff:fec0:1da3%vmx0 prefixlen 64 scopeid 0x1 
    	inet 192.168.1.60 netmask 0xffffff00 broadcast 192.168.1.255 
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
    	status: active
    vmx1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=60009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6>ether 00:0c:29:c0:1d:ad
    	inet6 fe80::20c:29ff:fec0:1dad%vmx1 prefixlen 64 scopeid 0x2 
    	inet 192.168.6.1 netmask 0xffffff00 broadcast 192.168.6.255 
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
    	status: active
    pflog0: flags=100 <promisc>metric 0 mtu 33144
    pfsync0: flags=0<> metric 0 mtu 1500
    	syncpeer: 224.0.0.240 maxupd: 128 defer: on
    	syncok: 1
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    	options=600003 <rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6>inet 127.0.0.1 netmask 0xff000000 
    	inet6 ::1 prefixlen 128 
    	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 
    	nd6 options=21 <performnud,auto_linklocal>enc0: flags=0<> metric 0 mtu 1536
    	nd6 options=21 <performnud,auto_linklocal>ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    	options=80000 <linkstate>inet6 fe80::20c:29ff:fec0:1da3%ovpns1 prefixlen 64 scopeid 0x7 
    	inet 10.0.8.1 --> 10.0.8.2 netmask 0xffffffff 
    	nd6 options=21 <performnud,auto_linklocal>Opened by PID 1222</performnud,auto_linklocal></linkstate></up,pointopoint,running,multicast></performnud,auto_linklocal></performnud,auto_linklocal></rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6></up,loopback,running,multicast></promisc></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
    

  • LAYER 8 Global Moderator

    so in your OP you stated that speed was the same both with and without pfsense..

    when i run a speedtest.net its comes back with and without pfsense its 2.60 Mbps


  • Banned

    pfSense doesnt slow the internet down….traffic does :D



  • @johnpoz:

    so in your OP you stated that speed was the same both with and without pfsense..

    when i run a speedtest.net its comes back with and without pfsense its 2.60 Mbps

    i meant behind pfsense is slow,
    direct with the ISP modem is fast

    @Supermule:

    pfSense doesnt slow the internet down….traffic does :D

    i love Pfsense more than my wife :) and yes it doesn't slow the internet and i am using it on my production and very happy with it .


  • Netgate Administrator

    Try running a download test instead. You can download a file from a known good source directly on the pfSense box. That way you know if the restriction you're seeing is at the WAN or LAN. For example:

    [2.1.5-RELEASE][root@pfsense.fire.box]/root(1): fetch -o /dev/null http://download.thinkbroadband.com/50MB.zip
    /dev/null                                     100% of   50 MB 8550 kBps
    

    That work well in the UK but you may want to choose some file closer to you. Though at 1Mbps you're probably fine.

    Compare that with downloading the file behind pfSense or directly.

    Steve



  • @Jamerson:

    i love Pfsense more than my wife :)

    :o

    I'd recommend to see if there's an upgrade pack for wife available  ;D ;D ;D ;D


  • Banned

    A more recent model of wife could be good to get the speed you want…



  • @stephenw10:

    Try running a download test instead. You can download a file from a known good source directly on the pfSense box. That way you know if the restriction you're seeing is at the WAN or LAN. For example:

    [2.1.5-RELEASE][root@pfsense.fire.box]/root(1): fetch -o /dev/null http://download.thinkbroadband.com/50MB.zip
    /dev/null                                     100% of   50 MB 8550 kBps
    

    That work well in the UK but you may want to choose some file closer to you. Though at 1Mbps you're probably fine.

    Compare that with downloading the file behind pfSense or directly.

    Steve

    Thank you Steve,

    this behind the Pfsense :

    /root: fetch -o /dev/null http://download.thinkbroadband.com/50MB.zip
    /dev/null                                       8% of   50 MB  169 kBps 05m47s
    

    this directly to the internet :

    etch -o /dev/null http://download.thinkbroadband.com/50MB.zip
    /dev/null                                      53% of   50 MB 1965 kBps 00m14s
    

    PS : i've updated to the 2.2 release


  • Netgate Administrator

    So, are either of those on the pfSense VM itself?


  • LAYER 8 Global Moderator

    And what vm do you have it setup on?  You stated this was a VM.  Workstation, Player, Esxi?  How is it setup?  How much cpu, how much ram - how the nics connected to your physical network/host?  Are they bridged or natted?

    So my pfsense is also vm..
    [2.2-RELEASE][root@pfSense.local.lan]/root: fetch -o /dev/null http://www.wswd.net/testdownloadfiles/512MB.zip
    /dev/null                                      30% of  512 MB 7284 kBps 00m52s

    Needed a bit bigger file 50MB done so fast could get a good speed indication.  And the UK bit far from me, wasn't seeing good speeds there.

    this is from another vm behind pfsense, didn't have fetch so used wget
    135,725,046 7.11MB/s  eta 60s

    Going to need some details of how your VM is setup, etc.



  • @johnpoz:

    And what vm do you have it setup on?  You stated this was a VM.  Workstation, Player, Esxi?  How is it setup?  How much cpu, how much ram - how the nics connected to your physical network/host?  Are they bridged or natted?

    So my pfsense is also vm..
    [2.2-RELEASE][root@pfSense.local.lan]/root: fetch -o /dev/null http://www.wswd.net/testdownloadfiles/512MB.zip
    /dev/null                                      30% of  512 MB 7284 kBps 00m52s

    Needed a bit bigger file 50MB done so fast could get a good speed indication.  And the UK bit far from me, wasn't seeing good speeds there.

    this is from another vm behind pfsense, didn't have fetch so used wget
    135,725,046 7.11MB/s  eta 60s

    Going to need some details of how your VM is setup, etc.

    Hi John thank you for trying to help me fix this.
    the Guest OS :FreeBSD (64-bit)
    Compatibility  ESXI 5.5 ( VM Version 10 )
    VMware Tools : Running version 2147483647
    Pfsense has two Virtuals NICS, VMX3
    NIC 1 is LAN
    NIC 0 is WAN
    2 CPUs
    Memory 2048 MB, ( 112 USED )
    Two Disks : 8 GB SSD for the OS, and 40 GB for cashing " Not configured yet "
    Firewall rules are standart nothing special really, no VLANS.
    NAT Port 443,

    Between Pfsense and Internet there is ISP Modem,

    Thank you

    @stephenw10:

    So, are either of those on the pfSense VM itself?

    yes they are the same VM, only first test is  using Pfsense as it Gateway ( the slow one ) and second test is not ( the fastest one )
    I am not saying pfsense is slowing the traffic, i think there is somewhere miss configuration.
    i am using pfsense on a production and very happy about it .


  • Netgate Administrator

    So run that command on the pfSense VM itself and you will know if the speed restriction is at the WAN or LAN interface.

    Steve



  • @stephenw10:

    So run that command on the pfSense VM itself and you will know if the speed restriction is at the WAN or LAN interface.

    Steve

    i am sorry not sure quiet i understand you,
    what commands do i need to run ?


  • Netgate Administrator

    Run the fetch command at the pfSense console.
    If you see the full download speed there you know the throttling exists between pfSense and the LAN side clients.

    Steve


  • LAYER 8 Global Moderator

    "VMware Tools : Running version 2147483647"

    What tools?  The native tools, there has been nothing but issues with the native tools..  I think that build number reflects 3rd party tools, where did you get that build number?  That is not a valid number.  The current version number of the openvm tools package is 1280544_10 that I see

    I would suggest you do a clean install of pfsense, not install the tools - test speed.  Then install the openvm tools not the native ones.  What instructions did you follow, there are some things that need to be fixed up with the openvm tools to get the shutdown to work correctly, etc.

    When I was playing with the native tools couldn't get anything to work other than ping, etc.

    And what is your esxi host, and how is it connected to the real world?  What vswitch setup do you have?  Where exactly did you do those fetch tests on - you didn't do them on pfsense itself?


  • Banned

    @johnpoz:

    What tools?  The native tools, there has been nothing but issues with the native tools..

    Yeah, that package is totally horrible, best removed altogether from 2.2


  • LAYER 8 Global Moderator

    Not sure where he is getting that build number for his tools, that is not a valid number.

    on pfsense with the openvpn tools install I show this

    [2.2-RELEASE][root@pfSense.local.lan]/root: vmware-toolbox-cmd -v
    9.4.0.25793 (build-1280544)

    On a linux box with the current native tools I show this
    ubuntu:~$ vmware-toolbox-cmd -v
    9.4.11.42879 (build-2400950)

    If I google that buld number for vmware tools shows as unmanaged.. not a standard build number and is just the decimal value of 0x7fffffff, so not sure where or what he has installed for tools if any to be honest.

    Other question I have is what build number of esxi 5.5 - the version of freebsd 10.1, in 2.2 is not fully supported until update 2 of esxi 5.5, build 2068190, the current build number is 2456374 which 4 patches past update 2.

    I am running 2.2 pfsense on vm on esxi with the openvm tools and it is running fantastic..  Other than the apinger showing me that my gateway is like 1.2 ms, I WISH ;) I have really not seen any issues at all.

    My connections in esxi host to the real world is like this - see attached.  So would be curious to see how he has it setup.

    My host has 4 physical nic ports connected to vswitches.  Pfsense has a nic in each vswitch, the wan physical nic is directly connected to my cable modem.  This puts a public IP on pfsense wan.  Then there is wlan and lan and dmz.  The vmkern is on its own nic not connected to pfsense just tied to the lan.  When you share port groups with vmkern and another network (lan say)  seems to slow down performance moving files to and from the datastore - since I have enough phsyical nics why not break it out, etc.

    I run a vlan on the wlan nic in pfsense for guest wireless, etc.  The lan and wlan physical nics connect to a managed switch with vlans for the lan and wlan/guestwlan segments because there a few physical devices on the wlan not just AP and controller, printer makes airprint easier when on the wireless network.  And I put my dvrs which are wired on it as well since makes it easier or ipad app to find them, etc.

    So curious how the OP setup is..  I would assume it should be something very sim to this.






  • I am experiencing a slow down on the wan port when running pfsense 2.2 on esxi 5.5 u2 with open-vm-tools install. Everything seems normal on the lan port. I have tried replacing nic and cables but the results are the same. Cannot find the problem.

    Here are some test results running iperf client on pfsense

    –----------------------------------------------------------
    LAN side iperf server listening on TCP port 5001
    TCP window size: 85.3 KByte (default)

    [  4] local 192.168.5.22 port 5001 connected with 192.168.5.3 port 41971 (pfsense on esxi)
    [ ID] Interval      Transfer    Bandwidth
    [  4]  0.0-10.0 sec  712 MBytes  596 Mbits/sec
    [  5] local 192.168.5.22 port 5001 connected with 192.168.5.1 port 37565 (pfsense on hyperv)
    [  5]  0.0-10.3 sec  684 MBytes  559 Mbits/sec
    [  4] local 192.168.5.22 port 5001 connected with 192.168.5.2 port 51702 (pfsense on atom pc)
    [  4]  0.0-10.0 sec  442 MBytes  370 Mbits/sec

    –----------------------------------------------------------
    WAN side iperf server listening on TCP port 5001
    TCP window size: 85.3 KByte (default)

    [  4] local 192.168.1.22 port 5001 connected with 192.168.1.3 port 39355
    [ ID] Interval      Transfer    Bandwidth
    [  4]  0.0-11.2 sec  7.12 MBytes  5.32 Mbits/sec  <-?????? esxi pfsense running iperf client to wan side server
    [  5] local 192.168.1.22 port 5001 connected with 192.168.1.1 port 54856
    [  5]  0.0-10.9 sec  700 MBytes  536 Mbits/sec
    [  4] local 192.168.1.22 port 5001 connected with 192.168.1.2 port 58495
    [  4]  0.0-10.9 sec  537 MBytes  413 Mbits/sec



  • Hi John,
    i've run the command on the box to detect the install vmware tools

    [2.2-RELEASE][root@firewall.pfsense.lan]/root: vmware-toolbox-cmd -v
    9.4.0.25793 (build-1280544)
    [2.2-RELEASE][root@firewall.pfsense.lan]/root: 
    

    my switch is 1Gigabit switch and all ports are 1GB
    on my ESXI 5.5 i've got 4 1GB NICS, and have all of them on one group.
    all the traffic is travelling on those 4 NICS at once over 1Gigabit switch ( HP 1810-24G v2 ).
    i have the same switch and configuration on production all seems to works fine even faster .
    i have removed the vmware tools and reinstalled them but the issue still exist .

    my ESXI 5.5 is having the latest patches.
    are you saying the pfsesne 2.2 isn't supporting the latest U2 Patches ? because my production is patched Build Number: 2456374


  • LAYER 8 Global Moderator

    "on my ESXI 5.5 i've got 4 1GB NICS, and have all of them on one group."

    What??  That makes NO SENSE – how do you isolate between your wan and your lan on pfsense..

    So you have 1 vswitch? Please post your esxi setup like I did for vswitches and lets see this setup of your 4 nics - did you setup a lagg on your switch?


  • Banned

    Long time no bridge…  ::)



  • @johnpoz:

    "on my ESXI 5.5 i've got 4 1GB NICS, and have all of them on one group."

    What??  That makes NO SENSE – how do you isolate between your wan and your lan on pfsense..

    So you have 1 vswitch? Please post your esxi setup like I did for vswitches and lets see this setup of your 4 nics - did you setup a lagg on your switch?

    between pfsense and internet there is my ISP Modem.
    this is a lab and the VM's are having a static iP, and the ISP Modem DHCP is off.
    so all the traffic is going through 4 NICS ( WAN and LAN ).

    yes using one vSwitch but all the VMS are statics.


  • LAYER 8 Global Moderator

    So your trying to run 2 networks over the same wire (wan and lan)??  Just by changing the ips..  Yeah sorry it doesn't work that way..

    Break out a nic for you WAN, then another one for your LAN put them on 2 different vswitches.  I would also break out your vmkern..  IF you have a managed switch you can use the same switch with vlans.  But your wan connection should really just be direct to your esxi host interface..



  • @johnpoz:

    So your trying to run 2 networks over the same wire (wan and lan)??  Just by changing the ips..  Yeah sorry it doesn't work that way..

    Break out a nic for you WAN, then another one for your LAN put them on 2 different vswitches.  I would also break out your vmkern..  IF you have a managed switch you can use the same switch with vlans.  But your wan connection should really just be direct to your esxi host interface..

    thank John,
    you mean 1 NIC for the WAN and Managenement of the ESXI and 3 Groups NICS for the LAN and traffic.

    Like WAN = gonna use Vlan 2
    and LAN= Vlan 3
    ect.. ?

    According to VMware NIC Teaming is the best practise for ESXI provide reduncency, speed, and increase the netwrok capacity

    isn't John ?


  • LAYER 8 Global Moderator

    dude this is a LAB not some mission critical production setup..  And even if it was you sure wouldn't run 4 nics all to the same vswitch for both wan and lan networks.  Was it a trunk and you were setting up vlans on pfsense?  If so you made no mention of vlans in your setup.

    Break out your NICS  What is your internet speed again, how and the F do you think you need 3 nics to lan??  If you want to do failover then put 2 on each and sure you can share your vmkern port with your lan.  But it hurts performance to and from the datastore for moving files.  This is normally rare - but you have 4 nics to play with.

    Doesn't really matter what vlan numbers you put on them.. They will be physically isolated I would hope - but sure if you have to run your wan through your switch it can be on a different vlan.  Pfsense doesn't really care in that sort of setup.

    I can tell you what vmware doesn't say is try and run 2 different networks over the same freaking wire that is for damn sure..  So what mode did you have these nics in when groups?  teamed/lagged/loadbalanced/failover?  Did you lagg them on the switch?



  • @johnpoz:

    dude this is a LAB not some mission critical production setup..  And even if it was you sure wouldn't run 4 nics all to the same vswitch for both wan and lan networks.  Was it a trunk and you were setting up vlans on pfsense?  If so you made no mention of vlans in your setup.

    Break out your NICS  What is your internet speed again, how and the F do you think you need 3 nics to lan??  If you want to do failover then put 2 on each and sure you can share your vmkern port with your lan.  But it hurts performance to and from the datastore for moving files.  This is normally rare - but you have 4 nics to play with.

    Doesn't really matter what vlan numbers you put on them.. They will be physically isolated I would hope - but sure if you have to run your wan through your switch it can be on a different vlan.  Pfsense doesn't really care in that sort of setup.

    I can tell you what vmware doesn't say is try and run 2 different networks over the same freaking wire that is for damn sure..  So what mode did you have these nics in when groups?  teamed/lagged/loadbalanced/failover?  Did you lagg them on the switch?

    hi John,
    on the switch i have Two Vlans
    Vland 1 Default and Vlan 4 for productions,

    2 NICS for the LAN and 2 for the WAN both are Teamed " Route Based on the original VLAN ID" and both on Fail over.
    the switch is also new not completely configured but have created 1 Extra VLAN,
    if i break out the NICS to two different Wire , should i create a rules on pfsense to be able to communicate with the WAN VLAN too ?
    i want to be able to manage the WAN from the LAN too.


  • LAYER 8 Global Moderator

    Manage the wan from the lan?  What??

    Do you understand what a firewall/router is?

    Please draw up what you want your network to be..  Yes devices connect to your lan can access stuff on your wan.  Out of the box pfsense creates a any any rule for lan to wan (normally internet)  It also out of the box NATs this traffic to look like it came from the IP on the pfsense wan.

    If you want to just use it as a router/firewall and not the nat, you can do that too.  The rules you allow between segments is would determines what you can "manage" ??  This term makes no sense.  Are you talking about accessing pfsense gui or ssh from the wan side?  What is it you want to "manage"



  • @johnpoz:

    Manage the wan from the lan?  What??

    Do you understand what a firewall/router is?

    Please draw up what you want your network to be..  Yes devices connect to your lan can access stuff on your wan.  Out of the box pfsense creates a any any rule for lan to wan (normally internet)  It also out of the box NATs this traffic to look like it came from the IP on the pfsense wan.

    If you want to just use it as a router/firewall and not the nat, you can do that too.  The rules you allow between segments is would determines what you can "manage" ??  This term makes no sense.  Are you talking about accessing pfsense gui or ssh from the wan side?  What is it you want to "manage"

    i want to manage the ISP Modem, Switch and other devices that are running on the WAN
    John i love you man,
    i have 1 NIC on the LAN and 2 NICS on the WAN ( one Group ) , i've noticed the traffic is faster, Internet pages load faster than before !
    Thank you so much for the tip no having two Networks on one wire.
    If i have 3 NICS " for better performance should i group two NICS on the LAN or WAN ?

    you are the man again thank you so much.


  • LAYER 8 Global Moderator

    Again its a router firewall, why would you not be able to access stuff on the wan segment or internet as long as you don't forbid it in the firewall of pfsense.  You access the internet right ;)

    Not really much to "manage" on a modem if you ask me..  What you want to check your signals?  Why would your switch management interface be on the wan??  Put its management IP on the lan side..

    As to your 2 nics on the wan?  Thought your internet speed was 4mbps or something.. WTF you need 2 gig interfaces for??  Failover ok – thought this was a lab do you really need failover redundancy in a lab?  You sure and the hell don't need the extra bandwidth ;)  you do understand 1000mbps is WAY faster than 4Mbps!!  Where are you wanting to go that you think you need a 2 Gig lan super highyway when right after you leave your driveway at 1 gig you hit a gravel road with a speed limit of 4mbps..

    Where do you think you need this "performance"  I would leverage your interfaces for actual segments if needed.. Ie 1 for vmkern, 1 for wan, 1 for lan and last one maybe wireless segment?  Or DMZ segment, or Lan2, etc..  using it for "performance" you will never need or use is misuse.  As to failover - ok if it was production mission critical and that connection can not go done, ok..  But I don't see that in this setup..


  • Netgate Administrator

    1000mbps=1bps.  :P



  • Damn thats fast…. haha


Log in to reply