• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

[solved] significant problem with throughput through our pfsense

Scheduled Pinned Locked Moved General pfSense Questions
8 Posts 4 Posters 3.1k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • U
    um3n
    last edited by Jan 9, 2017, 8:36 AM Jan 4, 2017, 3:00 PM

    Dear all,

    Currently we have a significant problem with throughput through our pfsense.
    Throughput is very small if we send data from one local segment to another local segment and also,
    if we send data over the WAN-Connection, but we have full bandwidth available if we send from any host to one of the pfsense interfaces.
    Throughput through the pfsense firewall is 12.7 Mbits/s, compared to the expected bandwidth of approx.. 9 Gbit/s, which is achieved when testing to the pfsense interfaces.
    We use the pfsense as the central router in our network. All hosts in our network are connected with 10GBE Nic's.
    The pfsense also have a 10GBE WAN P2P connection to a second location. The latency of this connection is 13ms.
    In an attempt to reach full speed over the 10GBE WAN connection we have made some adjustments in the TCP windows size  on the hosts and on the pfsense (incremented to 16B).

    Also, to help find the error we disabled the entire firewall functionality on the pfsense but without any result.

    Please can you investigate and help resolve why throughput through the pfsense is significantly reduced.
    Please see below for information about the configurations.

    With kind regards,

    Teste scenarios with iperf 2

    host a 10.100.4.2/24
    host b 10.100.5.8/24

    pfsense

    10.100.4.1/24
    10.100.5.1/24

    10.100.5.8 ->  10.100.4.1              =  8.60 Gbits/sec
    10.100.4.2 ->  10.100.5.1              =  8.88 Gbits/sec
    10.100.5.8 ->  10.100.4.2              =  12.7 Mbits/sec
    10.100.4.2 ->  10.100.5.8              =  12.7 Mbits/sec

    PFSense Version:

    2.3.2-RELEASE-p1 (amd64)

    10.100.5.1 -> vlan on bxe2
    10.100.4.1 -> vlan on bxe2

    tunables

    net.inet.tcp.recvspace 16777216
    net.inet.tcp.sendspace 16777216
    net.raw.sendspace  16777216
    net.raw.recvspace 16777216
    kern.ipc.maxsockbuf 33554432
    kern.ipc.nmbclusters 1000000

    firewall & nat

    disable firewall true
    disable firewall scrub

    networking

    Disable hardware checksum offload  false
    Enable device polling false
    Disable hardware TCP segmentation offload false
    Disable hardware large receive offload false

    hardware

    Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz
    32 CPUs: 2 package(s) x 8 core(s) x 2 SMT threads
    130GB RAM

    HP FlexFabric 10Gb 2-port 533FLR-T Adapter

    Location Embedded

    Firmware 7.14.79

    2c:44:fd:91:0d:50
    2c:44:fd:91:0d:54

    HP Ethernet 10Gb 2-port 530T Adapter

    Location Slot 3

    Firmware 7.14.79

    14:02:ec💿bd:28
    14:02:ec💿bd:2c

    HP Ethernet 1Gb 4-port 331i Adapter - NIC

    Location Embedded

    Firmware 17.4.41

    1c:98:ec:15:69:2c
    1c:98:ec:15:69:2d
    1c:98:ec:15:69:2e
    1c:98:ec:15:69:2f

    ifconfig

    bxe0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=507bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso>ether 2c:44:fd:91:0d:50
    
    inet6 fe80::2e44:fdff:fe91:d50%bxe0 prefixlen 64 scopeid 0x1
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    bxe1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=507bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso>ether 2c:44:fd:91:0d:54
    
    inet6 fe80::2e44:fdff:fe91:d54%bxe1 prefixlen 64 scopeid 0x2
    
    inet 192.168.11.3 netmask 0xffffff00 broadcast 192.168.11.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    bge0: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
    
    options=c019b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate>ether 1c:98:ec:15:69:2c
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
    
    bge1: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
    
    options=c019b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate>ether 1c:98:ec:15:69:2d
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
    
    bge2: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
    
    options=c019b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate>ether 1c:98:ec:15:69:2e
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
    
    bge3: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
    
    options=c019b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate>ether 1c:98:ec:15:69:2f
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
    
    bxe2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=507bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2 prefixlen 64 scopeid 0x7
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    bxe3: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
    
    options=507bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso>ether 14:02:ec:cd:bd:2c
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (none)
    
    pflog0: flags=100 <promisc>metric 0 mtu 33160
    
    pfsync0: flags=0<> metric 0 mtu 1500
    
    syncpeer: 224.0.0.240 maxupd: 128 defer: on
    
    syncok: 1
    
    enc0: flags=0<> metric 0 mtu 1536
    
    nd6 options=21 <performnud,auto_linklocal>lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    
    options=600003 <rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6>inet 127.0.0.1 netmask 0xff000000
    
    inet6 ::1 prefixlen 128
    
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0xc
    
    nd6 options=21 <performnud,auto_linklocal>bxe2_vlan1016: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan1016 prefixlen 64 scopeid 0xd
    
    inet 10.100.1.1 netmask 0xfffffff0 broadcast 10.100.1.15
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 1016 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan1017: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan1017 prefixlen 64 scopeid 0xe
    
    inet 10.100.4.1 netmask 0xffffff00 broadcast 10.100.4.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 1017 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan1018: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan1018 prefixlen 64 scopeid 0xf
    
    inet 10.100.9.1 netmask 0xffffff00 broadcast 10.100.9.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 1018 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan1019: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan1019 prefixlen 64 scopeid 0x10
    
    inet 10.100.10.1 netmask 0xffffff00 broadcast 10.100.10.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 1019 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan1501: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan1501 prefixlen 64 scopeid 0x11
    
    inet 10.100.5.1 netmask 0xffffff00 broadcast 10.100.5.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 1501 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan1502: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan1502 prefixlen 64 scopeid 0x12
    
    inet 10.100.6.1 netmask 0xffffff00 broadcast 10.100.6.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 1502 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan2518: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan2518 prefixlen 64 scopeid 0x13
    
    inet 10.100.2.1 netmask 0xfffffff0 broadcast 10.100.2.15
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 2518 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan2519: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan2519 prefixlen 64 scopeid 0x14
    
    inet 10.100.3.1 netmask 0xffffff00 broadcast 10.100.3.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 2519 vlanpcp: 0 parent interface: bxe2
    
    bxe2_vlan3015: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 14:02:ec:cd:bd:28
    
    inet6 fe80::1602:ecff:fecd:bd28%bxe2_vlan3015 prefixlen 64 scopeid 0x15
    
    inet 10.100.7.1 netmask 0xffffff00 broadcast 10.100.7.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 3015 vlanpcp: 0 parent interface: bxe2
    
    bxe0_vlan3016: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 2c:44:fd:91:0d:50
    
    inet6 fe80::2e44:fdff:fe91:d50%bxe0_vlan3016 prefixlen 64 scopeid 0x16
    
    inet 10.100.8.1 netmask 0xffffff00 broadcast 10.100.8.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 3016 vlanpcp: 0 parent interface: bxe0
    
    bxe0_vlan3017: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    
    options=303 <rxcsum,txcsum,tso4,tso6>ether 2c:44:fd:91:0d:50
    
    inet6 fe80::2e44:fdff:fe91:d50%bxe0_vlan3017 prefixlen 64 scopeid 0x17
    
    inet 10.100.11.1 netmask 0xffffff00 broadcast 10.100.11.255
    
    nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (10Gbase-T <full-duplex>)
    
    status: active
    
    vlan: 3017 vlanpcp: 0 parent interface: bxe0</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,tso4,tso6></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6></up,loopback,running,multicast></performnud,auto_linklocal></promisc></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso></broadcast,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate></broadcast,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate></broadcast,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate></broadcast,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso,linkstate></broadcast,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso></up,broadcast,running,simplex,multicast>
    
    1 Reply Last reply Reply Quote 0
    • C
      CC
      last edited by Jan 4, 2017, 3:31 PM

      Is this a new setup or working setup that has failed?

      1 Reply Last reply Reply Quote 0
      • H
        Harvy66
        last edited by Jan 4, 2017, 5:16 PM

        It looks like you tested from a Device on Subnet A to PFSense's IP on Subnet B and vice-versa, and got in the 8Gb/s range. Then you tried to test between the two devices in the different subnets and got only in the 13Mb/s range.

        Have you tried placing the two devices on the same subnet and iperf'd between them that way to rule-out an issue between the two devices?

        1 Reply Last reply Reply Quote 0
        • U
          um3n
          last edited by Jan 5, 2017, 9:21 AM

          Hey Guys,

          thank you for responding, we appreciate that you spend your time.
          Of course i will answer your questions to get some more suggestions from you.

          @CC:

          Is this a new setup or working setup that has failed?

          Its a new setup which we are testing here.

          @Harvy66:

          It looks like you tested from a Device on Subnet A to PFSense's IP on Subnet B and vice-versa, and got in the 8Gb/s range. Then you tried to test between the two devices in the different subnets and got only in the 13Mb/s range.

          Have you tried placing the two devices on the same subnet and iperf'd between them that way to rule-out an issue between the two devices?

          Of course we did.
          The two machines are virtual ones, so when they are on the same virtual host we get a throughput about 26Gbit/s. When the machines are on different hosts we get a throughput about 8-9Gbit/s with iperf.

          Here are some more infos

          hostname
          10.100.5.8 -> app0016
          Linux app0016 3.10.0-327.36.3.el7.x86_64 #
          
          tuning für wan:
          
          net.ipv6.conf.all.disable_ipv6 = 1
          net.core.wmem_max=16777216
          net.core.rmem_max=16777216
          net.ipv4.tcp_rmem= 10240  16777216  16777216
          net.ipv4.tcp_wmem= 10240  16777216  16777216
          net.ipv4.tcp_window_scaling = 1
          net.ipv4.tcp_timestamps = 1
          net.ipv4.tcp_sack = 1
          net.ipv4.tcp_no_metrics_save = 1
          net.core.netdev_max_backlog = 5000
          
          host
          10.100.4.2 -> adm0001
          Linux adm0001 3.10.0-327.36.3.el7.x86_64
          
          tuning für wan
          
          net.ipv6.conf.all.disable_ipv6 = 1
          net.core.wmem_max=16777216
          net.core.rmem_max=16777216
          net.ipv4.tcp_rmem= 10240  16777216  16777216
          net.ipv4.tcp_wmem= 10240  16777216  16777216
          net.ipv4.tcp_window_scaling = 1
          net.ipv4.tcp_timestamps = 1
          net.ipv4.tcp_sack = 1
          net.ipv4.tcp_no_metrics_save = 1
          net.core.netdev_max_backlog = 5000
          
          iperf server 10.100.4.2  client 10.100.5.8
          
          [root@adm0001 ~]# iperf -s
          ------------------------------------------------------------
          Server listening on TCP port 5001
          TCP window size: 16.0 MByte (default)
          ------------------------------------------------------------
          [  4] local 10.100.4.2 port 5001 connected with 10.100.5.8 port 46600
          
          [root@app0016 ~]# iperf -c 10.100.4.2
          ------------------------------------------------------------
          Client connecting to 10.100.4.2, TCP port 5001
          TCP window size: 16.0 MByte (default)
          ------------------------------------------------------------
          [  3] local 10.100.5.8 port 46600 connected with 10.100.4.2 port 5001
          [ ID] Interval       Transfer     Bandwidth
          [  3]  0.0-10.0 sec  15.2 MBytes  12.7 Mbits/sec
          
          iperf server 10.100.4.2  client 10.100.5.8 - kleinere window size
          
          [root@app0016 ~]# iperf -c 10.100.4.2 -w 64KB
          ------------------------------------------------------------
          Client connecting to 10.100.4.2, TCP port 5001
          TCP window size:  128 KByte (WARNING: requested 64.0 KByte)
          ------------------------------------------------------------
          [  3] local 10.100.5.8 port 46606 connected with 10.100.4.2 port 5001
          [ ID] Interval       Transfer     Bandwidth
          [  3]  0.0-15.0 sec   215 KBytes   117 Kbits/sec
          
          iperf client 10.100.5.8 -> server 10.100.4.1 (pfsense)
          
          [root@app0016 ~]# iperf -c 10.100.4.1
          ------------------------------------------------------------
          Client connecting to 10.100.4.1, TCP port 5001
          TCP window size: 16.0 MByte (default)
          ------------------------------------------------------------
          [  3] local 10.100.5.8 port 32782 connected with 10.100.4.1 port 5001
          [ ID] Interval       Transfer     Bandwidth
          [  3]  0.0-10.0 sec  9.97 GBytes  8.57 Gbits/sec
          
          1 Reply Last reply Reply Quote 0
          • D
            Derelict LAYER 8 Netgate
            last edited by Jan 5, 2017, 10:00 AM Jan 5, 2017, 9:46 AM

            disable firewall true
            disable firewall scrub

            networking

            Disable hardware checksum offload  false
            Enable device polling false
            Disable hardware TCP segmentation offload false
            Disable hardware large receive offload false

            Have you tried this:

            disable firewall false
            disable firewall scrub false

            networking

            Disable hardware checksum offload  true
            Enable device polling false
            Disable hardware TCP segmentation offload true
            Disable hardware large receive offload true

            With performance problems on certain NICs disabling the offloads is probably what you want, at least when you're looking at pitiful performance like 12.7 Mbits/sec through the firewall. Something is certainly off there. That almost looks like the problem with checksumming going through XenServer.

            It looks like you are "hairpinning" traffic between those 2 subnets on bxe2_vlan1017 and bxe2_vlan1501 so you do realize you will never see the same throughput you see when that is not the case right? Obviously should be seeing better than 12Mbit/s though. Out of curiosity does it perform the same between a VLAN on bxe2 and a VLAN on bxe0? bxe2 and bgeX? bgeX & bgeY?

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • U
              um3n
              last edited by Jan 5, 2017, 10:41 AM

              We will check your suggestions.
              As far we have some results you will here from me.

              We activated the TSO and LRO to get a better performance and that work actually fine. I am not sure if this causes the side effects.
              But i keep you informed after trying.

              1 Reply Last reply Reply Quote 0
              • D
                Derelict LAYER 8 Netgate
                last edited by Jan 5, 2017, 10:42 AM

                We activated the TSO and LRO to get a better performance and that work actually fine.

                Yeah something there is obviously not fine.

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • U
                  um3n
                  last edited by Jan 9, 2017, 8:36 AM Jan 5, 2017, 2:36 PM

                  So… it seems to be the LRO that is decreasing the throughput.
                  I'll keep you informed.
                  *edit:
                  i think that and some performancetuning helped.

                  1 Reply Last reply Reply Quote 0
                  8 out of 8
                  • First post
                    8/8
                    Last post
                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                    This community forum collects and processes your personal information.
                    consent.not_received