Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Slow internet with pfsense

    Virtualization
    2
    25
    4556
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      telvenes last edited by

      Hi,

      Do not know if it's the right category, please move if not :)

      I run pfSense virtual on a XenServer and 1Gbit internet and internet is slow.
      It's a little different too how slow it is, but between 10 and 50 MBits down, upload speed is always good at 1gbit.

      If I rebooted pfSense internet is about 1Gbit/1Gbit for about 5 minutes until it becomes slow again. Reboot and its fast again for 5 minutes.

      What I've done with XenServer for both WAN and all LAN:

      xe vif-param-set uuid=VIFUUID other-config:ethtool-tx="off"
      xe vif-param-set uuid=VIFUUID other-config:ethtool-rx="off"

      pkg install xe-guest-utilities

      echo "xenguest_enable="YES"" >> /etc/rc.conf.local
      ln -s /usr/local/etc/rc.d/xenguest /usr/local/etc/rc.d/xenguest.sh

      service xenguest start

      What i have done with pfSense:

      • Diabled all DHCP Servers

      • Created 4 Lan

      • Under Wan, IPv6 configuration type = none

      • Under wan, IPv3 Configuration type = static IPv4

      • Under wan, added Virtual IP/32

      • Under wan, selected gateway

      • Under system->routing, addd gateway default and checked: Use non-local gateway

      • Under Firewall rules, copied LAN default rule to every interface

      • Under Firewall rules, removed ipv6 rule

      • Under all 4 LAN interfaces deactivated ipv6

      • Under DNS Resolver, deaktivated

      • Under DNS Forwarder, activated and selected all lan under interfaces

      • Under Firewall Virtual IP, added 25 Virtual IPs

      • Under advanced ->network: Disabled Hardware Checksum Offloading

      And I know that there issnt something wrong with my Internet line as if I try to add a virtual IP on another vm and then internet is perfect.

      I also tried the same setup with vmware on another server, had the same problem then.

      Here is OVH network configuration guide:
      http://help.ovh.com/BridgeClient

      1 Reply Last reply Reply Quote 0
      • T
        telvenes last edited by

        I'd really appreciate some help, have tried to fix this for 3 months now.

        Tell If any screenshots or more info any tests I can do.

        1 Reply Last reply Reply Quote 0
        • T
          telvenes last edited by

          i have 1 interface for every LAN and i have testing copying a file of 1gb from vlan to vlan and i get speed of 200mb pr sek. so i belive network is ok?

          1 Reply Last reply Reply Quote 0
          • ?
            Guest last edited by

            Do you see any errors with tcpdump, or in the firewall logs? And what about iperf tests?

            1 Reply Last reply Reply Quote 0
            • T
              telvenes last edited by

              I cant se any errors…

              iperf also gets good speed:
              [2.3.2-RELEASE][root@fw-pfSense-01.localdomain]/root: iperf -c ping.online.net
              –----------------------------------------------------------
              Client connecting to ping.online.net, TCP port 5001
              TCP window size: 65.0 KByte (default)

              [  3] local 87.98.128.127 port 5109 connected with 62.210.18.40 port 5001
              [ ID] Interval      Transfer    Bandwidth
              [  3]  0.0-10.0 sec  654 MBytes  548 Mbits/sec

              And from a vm behind pfsense:
              ^Croot@isp1:~# iperf -c ping.online.net
              –----------------------------------------------------------
              Client connecting to ping.online.net, TCP port 5001
              TCP window size: 85.0 KByte (default)

              [  3] local 192.168.99.101 port 45274 connected with 62.210.18.40 port 5001
              [ ID] Interval      Transfer    Bandwidth
              [  3]  0.0-10.0 sec  1.09 GBytes  935 Mbits/sec

              So i get good speed with iperf. but when i try to download http://proof.ovh.net/files/10Gb.dat i get only get between 10 and 100mbit pr sec

              1 Reply Last reply Reply Quote 0
              • T
                telvenes last edited by

                also when downloading http://ping.online.net/10000Mo.dat i only get 20mbit

                1 Reply Last reply Reply Quote 0
                • ?
                  Guest last edited by

                  How big is the TCP window size and how big is the MTU during HTTP?

                  1 Reply Last reply Reply Quote 0
                  • T
                    telvenes last edited by

                    I do not know what MTU is? And TCP window size: 85.0 KByte (default) ?

                    1 Reply Last reply Reply Quote 0
                    • ?
                      Guest last edited by

                      What is the current VIFUUID of your VM interface

                      1 Reply Last reply Reply Quote 0
                      • T
                        telvenes last edited by

                        This is the list:

                        uuid ( RO)                  : 5c40178e-8998-dce9-b2d0-4311a03cdfb0
                                vm-name-label ( RO): fw-pfSense-01
                                        device ( RO): 0
                                          MAC ( RO): 00:50:56:0b:f2:ae
                                  network-uuid ( RO): fbbe8cbd-b8c7-961f-34cc-94d890116d65
                            network-name-label ( RO): WAN

                        uuid ( RO)                  : 5b921ac5-e297-60dd-83dd-3607f053f2fb
                                vm-name-label ( RO): fw-pfSense-01
                                        device ( RO): 4
                                          MAC ( RO): 0a:87:13:7d:bb:eb
                                  network-uuid ( RO): ce166d3a-613a-b364-174f-95f47496b284
                            network-name-label ( RO): ISP

                        uuid ( RO)                  : d3679b9b-4960-7ec8-76d7-48fb7e2fcf22
                                vm-name-label ( RO): fw-pfSense-01
                                        device ( RO): 2
                                          MAC ( RO): a6:0c:62:9d:4b:06
                                  network-uuid ( RO): 426b2541-0140-41dc-ac92-1fb55cecf6d0
                            network-name-label ( RO): BACKUP

                        uuid ( RO)                  : 78c01939-9395-b040-d6ba-ef81c9e51195
                                vm-name-label ( RO): fw-pfSense-01
                                        device ( RO): 1
                                          MAC ( RO): 9a:b7:cc:e6:eb:26
                                  network-uuid ( RO): 26c32f02-31d4-ab1f-a6d6-83a823ce8607
                            network-name-label ( RO): ADMIN

                        uuid ( RO)                  : 9b4da5fe-06d2-8b2f-08f5-e531a6e209cf
                                vm-name-label ( RO): fw-pfSense-01
                                        device ( RO): 3
                                          MAC ( RO): 4e:20:0d:fa:c7:1e
                                  network-uuid ( RO): 84dfa988-3c82-1832-1343-3bcc02944eb5
                            network-name-label ( RO): HOME

                        And also On host server:

                        [root@ns3044318 tmp]# wget -O /dev/null http://proof.ovh.net/files/1Gio.dat
                        –2016-11-23 20:15:57--  http://proof.ovh.net/files/1Gio.dat
                        Resolving proof.ovh.net... 188.165.12.106, 2001:41d0:2:876a::1
                        Connecting to proof.ovh.net|188.165.12.106|:80... connected.
                        HTTP request sent, awaiting response... 200 OK
                        Length: 1073741824 (1.0G) [application/octet-stream]
                        Saving to: `/dev/null'

                        100%[=================================================================================================================================================================================================>] 1,073,741,824  112M/s  in 13s

                        2016-11-23 20:16:10 (79.2 MB/s) - `/dev/null' saved [1073741824/1073741824]

                        On host behind pfsense:

                        administrator@isp1:~$  wget -O /dev/null http://proof.ovh.net/files/1Gio.dat
                        –2016-11-23 20:16:56--  http://proof.ovh.net/files/1Gio.dat
                        Slår opp vertsnavn proof.ovh.net (proof.ovh.net) … 188.165.12.106, 2001:41d0:2:876a::1
                        Kobler til proof.ovh.net (proof.ovh.net)|188.165.12.106|:80 … tilkoblet.
                        HTTP-forespørsel sendt. Venter på svar … 200 OK
                        Lengde: 1073741824 (1,0G) [application/octet-stream]
                        Lagrer til: «/dev/null»

                        /dev/null                                                  100%[========================================================================================================================================>]  1,00G  5,07MB/s    om 3m 42s

                        1 Reply Last reply Reply Quote 0
                        • ?
                          Guest last edited by

                          Strange, it looks like packets are discarded or the MTU/Window/Fragmentation sizes are all wrong to cause this. Is MTR installed? Try MTR'ing to that DL server. Oh, and if you simply ping for 100 times, how are the latency/drops?

                          1 Reply Last reply Reply Quote 0
                          • T
                            telvenes last edited by

                            Running mtr  -w -c 10 -i 1 ping.online.net:

                            Start: Mon Nov 28 20:05:08 2016
                            HOST: fw-pfSense-01.localdomain          Loss%  Snt  Last  Avg  Best  Wrst StDev
                              1.|– 51.255.92.253                      0.0%    10  180.6 182.0 165.6 194.7  9.2
                              2.|-- po110.gra-g2-a75.fr.eu              0.0%    10    0.3  0.3  0.3  0.4  0.0
                              3.|-- ???                                100.0    10    0.0  0.0  0.0  0.0  0.0
                              4.|-- be99-1110.th2-1-a9.fr.eu            0.0%    10    5.5  5.0  4.7  5.5  0.0
                              5.|-- ???                                100.0    10    0.0  0.0  0.0  0.0  0.0
                              6.|-- 45x-s44-2-a9k2.dc3.poneytelecom.eu  0.0%    10    6.9  6.1  5.4  9.4  0.9
                              7.|-- ping.online.net                    0.0%    10    4.9  5.0  4.9  5.4  0.0

                            Host Server

                            –- www.google.com ping statistics ---
                            592 packets transmitted, 592 received, 0% packet loss, time 591891ms
                            rtt min/avg/max/mdev = 4.495/4.598/9.068/0.238 ms

                            Local VM behind pfsense

                            –- www.google.com ping statistics ---
                            619 packets transmitted, 614 packets received, 0.8% packet loss
                            round-trip min/avg/max/stddev = 4.561/4.739/11.052/0.336 ms

                            sometimes i get alot of pageloss on pfsense and 0 on the host

                            1 Reply Last reply Reply Quote 0
                            • ?
                              Guest last edited by

                              Sounds like you may still be having non-ICMP traffic issues, maybe your offloading settings still aren't right. Can you print the ethtool output on the hypervisor side for the VIF's?

                              1 Reply Last reply Reply Quote 0
                              • T
                                telvenes last edited by

                                i am not sure what command i have to run to find this:

                                but i run this script right now and it did not help
                                https://github.com/cloudnull/XenServer-Offloading-Off/blob/master/offloadingoff.sh

                                1 Reply Last reply Reply Quote 0
                                • T
                                  telvenes last edited by

                                  I also have to say i had the same issue on vmware

                                  1 Reply Last reply Reply Quote 0
                                  • ?
                                    Guest last edited by

                                    Have you tried pfSense 2.4 Beta just to see if the newer FreeBSD base makes a difference?

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      telvenes last edited by

                                      Trying to set up tomorrow. but do you think it might be a problem with pfSense or configuration error or problem with OVH?

                                      1 Reply Last reply Reply Quote 0
                                      • ?
                                        Guest last edited by

                                        @telvenes:

                                        Trying to set up tomorrow. but do you think it might be a problem with pfSense or configuration error or problem with OVH?

                                        Others have a working setup on OVH, but it's somewhat tricky with FreeBSD on mass-virtualisation, so it could be an issue with the underlying platform. It's hard to say at this point, but everything still points towards packets being discarded.

                                        1 Reply Last reply Reply Quote 0
                                        • T
                                          telvenes last edited by

                                          i am now running on 2.4, was thinking if i should try without xen-tools first and se how that goes.

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            telvenes last edited by

                                            Did not help. installed xen tools and still same error. could it bee a config error?

                                            1 Reply Last reply Reply Quote 0
                                            • T
                                              telvenes last edited by

                                              I have read over the whole internet on guides to get this working.

                                              And almost everywhere i see this lines:

                                              route add -net 2.3.4.254/32 -iface vmx1
                                              route add default 2.3.4.254

                                              I do not use them, i have only added default gateway and activated gateway outside subnet, is this correct?

                                              1 Reply Last reply Reply Quote 0
                                              • ?
                                                Guest last edited by

                                                @telvenes:

                                                I have read over the whole internet on guides to get this working.

                                                And almost everywhere i see this lines:

                                                route add -net 2.3.4.254/32 -iface vmx1
                                                route add default 2.3.4.254

                                                I do not use them, i have only added default gateway and activated gateway outside subnet, is this correct?

                                                No, that's a somewhat random suggestion and probably has nothing to do with your network.
                                                If you have vmx interfaces, they are not VT-d PCI interfaces and you may end up with the checksum bug as described in the sticky'ed topic (see the main virt forum).

                                                1 Reply Last reply Reply Quote 0
                                                • T
                                                  telvenes last edited by

                                                  no they are xn, but i dont know what more i can check to get this working correctly.

                                                  Anyway, if i reboot my pfsense internet is fast for 10 minutes or something before it goes slower.

                                                  1 Reply Last reply Reply Quote 0
                                                  • T
                                                    telvenes last edited by

                                                    Did some pacage filtering for 10 seconds while trying to download a file, can anyone see something wrong?

                                                    17:23:19.255593 IP MYIP.3389 > 109.247.144.79.65168: tcp 565
                                                    17:23:19.258132 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.266947 IP 109.247.144.79.65168 > MYIP.3389: tcp 117
                                                    17:23:19.267531 IP 191.181.216.88.4243 > 149.202.114.11.23: tcp 0
                                                    17:23:19.286667 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:19.286736 IP MYIP.3389 > 109.247.144.79.65168: tcp 85
                                                    17:23:19.305069 IP 109.247.144.79.65168 > MYIP.3389: tcp 117
                                                    17:23:19.331952 IP6 fe80::2ff:ffff:feff:fffd > ff02::1:ff00:0: ICMP6, neighbor solicitation, who has 2001:41d0:1004:20a7::, length 32
                                                    17:23:19.333436 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:19.338452 IP MYIP2 > 51.255.92.254: ICMP echo request, id 14876, seq 22429, length 8
                                                    17:23:19.340620 IP 51.255.92.254 > MYIP2: ICMP echo reply, id 14876, seq 22429, length 8
                                                    17:23:19.349256 IP MYIP.3389 > 109.247.144.79.65168: tcp 101
                                                    17:23:19.349268 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349271 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349274 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349278 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349287 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349290 IP MYIP.3389 > 109.247.144.79.65168: tcp 97
                                                    17:23:19.349365 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349370 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349395 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349411 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349450 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349456 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349516 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349522 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349557 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349563 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349609 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349616 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349620 IP MYIP.3389 > 109.247.144.79.65168: tcp 485
                                                    17:23:19.349683 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349688 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349718 IP MYIP.3389 > 109.247.144.79.65168: tcp 101
                                                    17:23:19.349770 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349776 IP MYIP.3389 > 109.247.144.79.65168: tcp 241
                                                    17:23:19.349796 IP MYIP.3389 > 109.247.144.79.65168: tcp 1460
                                                    17:23:19.349801 IP MYIP.3389 > 109.247.144.79.65168: tcp 81
                                                    17:23:19.349814 IP MYIP.3389 > 109.247.144.79.65168: tcp 101
                                                    17:23:19.363405 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.395919 IP MYIP.3389 > 109.247.144.79.65168: tcp 853
                                                    17:23:19.400052 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.400921 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.403148 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.403156 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.403195 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.403202 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.422843 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.423399 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.423408 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.423413 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.423416 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.423452 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.423485 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.423493 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.427759 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:19.427827 IP 109.247.144.79.65168 > MYIP.3389: tcp 117
                                                    17:23:19.427944 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:19.436158 IP 109.247.144.79.65168 > MYIP.3389: tcp 117
                                                    17:23:19.442717 IP MYIP.3389 > 109.247.144.79.65168: tcp 1077
                                                    17:23:19.472448 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.489523 IP MYIP.3389 > 109.247.144.79.65168: tcp 645
                                                    17:23:19.518900 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.565971 IP 109.247.144.79.65168 > MYIP.3389: tcp 0
                                                    17:23:19.578572 IP 92.222.185.1 > 149.202.114.9: ICMP echo request, id 22793, seq 1, length 12
                                                    17:23:19.618643 ARP, Request who-has 51.255.92.251 tell 51.255.92.253, length 46
                                                    17:23:19.675378 ARP, Request who-has 51.255.92.246 tell 51.255.92.253, length 46
                                                    17:23:19.832092 ARP, Request who-has 51.255.92.246 tell 51.255.92.253, length 46
                                                    17:23:19.840443 IP MYIP2 > 51.255.92.254: ICMP echo request, id 14876, seq 22430, length 8
                                                    17:23:19.844237 IP 51.255.92.254 > MYIP2: ICMP echo reply, id 14876, seq 22430, length 8
                                                    17:23:20.037704 ARP, Request who-has 51.255.92.242 tell 51.255.92.253, length 46
                                                    17:23:20.061806 IP 51.255.92.253.1985 > 224.0.0.2.1985: UDP, length 20
                                                    17:23:20.075734 ARP, Request who-has 51.255.92.1 tell 51.255.92.253, length 46
                                                    17:23:20.115356 ARP, Request who-has 51.255.92.249 tell 51.255.92.253, length 46
                                                    17:23:20.264026 IP 92.222.184.1 > 149.202.114.13: ICMP echo request, id 50886, seq 1, length 12
                                                    17:23:20.264177 IP 149.202.114.13 > 92.222.184.1: ICMP echo reply, id 50886, seq 1, length 12
                                                    17:23:20.276486 IP 92.222.184.1 > 149.202.114.8: ICMP echo request, id 50886, seq 1, length 12
                                                    17:23:20.310848 IP6 fe80::2ff:ffff:feff:fffe > ff02::1:ff00:0: ICMP6, neighbor solicitation, who has 2001:41d0:1004:20a5::, length 32
                                                    17:23:20.342422 IP MYIP2 > 51.255.92.254: ICMP echo request, id 14876, seq 22431, length 8
                                                    17:23:20.344993 IP 51.255.92.254 > MYIP2: ICMP echo reply, id 14876, seq 22431, length 8
                                                    17:23:20.381787 IP 109.247.144.79.65168 > MYIP.3389: tcp 85
                                                    17:23:20.389901 IP 109.247.144.79.65168 > MYIP.3389: tcp 85
                                                    17:23:20.390052 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:20.394264 IP MYIP.3389 > 109.247.144.79.65168: tcp 85
                                                    17:23:20.397723 IP 109.247.144.79.65168 > MYIP.3389: tcp 85
                                                    17:23:20.409800 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:20.413599 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:20.429884 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:20.429979 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:20.445900 IP 109.247.144.79.65168 > MYIP.3389: tcp 85
                                                    17:23:20.463383 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:20.463520 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:20.477848 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:20.487830 IP MYIP.3389 > 109.247.144.79.65168: tcp 85
                                                    17:23:20.493827 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:20.509964 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:20.510076 IP MYIP.3389 > 109.247.144.79.65168: tcp 0
                                                    17:23:20.525818 IP 109.247.144.79.65168 > MYIP.3389: tcp 101
                                                    17:23:20.527893 ARP, Request who-has 51.255.92.246 tell 51.255.92.253, length 46
                                                    17:23:20.538488 ARP, Request who-has 51.255.92.251 tell 51.255.92.253, length 46

                                                    1 Reply Last reply Reply Quote 0
                                                    • T
                                                      telvenes last edited by

                                                      I have contacted the internet provider, but its i think its strange if its something wrong with the internet? since it works ok with E.G Windows?

                                                      1 Reply Last reply Reply Quote 0
                                                      • First post
                                                        Last post