Netgate SG-4860: Hardware Limitation or Misconfiguration



  • Hi all

    Please have look at the picture to get an overview of my setup.
    pfSense is the routing instance of both VLAN's 103 & 104.

    drawing_pfsense.png

    Problem description
    If I copy large files from my workstation to the synology nas, cpu usage is very high and throughput is only ~45MByte.
    Ping latency to the internet is increasing dramatically and cpu usage of the pfSense is very high.

    What I did

    1. Separate LAGG for VLAN 103
      I moved VLAN 103 to a separate LAGG (1) that incoming and outgoing traffic is separated physically.

    2. System Tunables
      I have disabled the EEE settings on all interfaces.
      See this post https://community.spiceworks.com/topic/1221309-periodic-packet-loss-constant-carp-switchovers-with-intel-nics-i350-igb for details.

    [2.4.5-RELEASE][root@pfsense]/root: sysctl dev.igb | grep eee
    dev.igb.5.eee_disabled: 1
    dev.igb.4.eee_disabled: 1
    dev.igb.3.eee_disabled: 1
    dev.igb.2.eee_disabled: 1
    dev.igb.1.eee_disabled: 1
    dev.igb.0.eee_disabled: 1
    

    Question
    Is this behavior because of hardware limitations, or do I have a misconfiguration of my setup?
    I thought that a Netgate SG-4860 Appliance can handle 1GByte throughput without any limitations.

    Thanks in advance for your support!

    Some pfSense top -aSH outputs:

      PID USERNAME     PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
       11 root         155 ki31     0K    64K RUN     1 936:47  59.19% [idle{idle: cpu1}]
       11 root         155 ki31     0K    64K CPU3    3 935:25  52.52% [idle{idle: cpu3}]
       11 root         155 ki31     0K    64K RUN     0 937:59  45.01% [idle{idle: cpu0}]
       12 root         -92    -     0K   704K WAIT    0   0:45  27.15% [intr{irq272: igb4:que 0}]
    
    
      PID USERNAME     PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
        0 root         -92    -     0K   864K CPU2    2   2:06 100.00% [kernel{igb3 que (qid 0)}]
       11 root         155 ki31     0K    64K RUN     3 935:26  45.91% [idle{idle: cpu3}]
       11 root         155 ki31     0K    64K RUN     1 936:48  44.13% [idle{idle: cpu1}]
       12 root         -92    -     0K   704K WAIT    0   0:45  42.56% [intr{irq272: igb4:que 0}]
       11 root         155 ki31     0K    64K RUN     0 937:59  35.26% [idle{idle: cpu0}]
    
    
      PID USERNAME     PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
       11 root         155 ki31     0K    64K CPU3    3 935:33  94.23% [idle{idle: cpu3}]
       11 root         155 ki31     0K    64K CPU1    1 936:55  88.77% [idle{idle: cpu1}]
       11 root         155 ki31     0K    64K RUN     0 938:04  70.96% [idle{idle: cpu0}]
       12 root         -92    -     0K   704K WAIT    0   0:49  25.32% [intr{irq272: igb4:que 0}]
       12 root         -92    -     0K   704K WAIT    0   0:15   0.34% [intr{irq267: igb3:que 0}]
    

    Ping from workstation to 8.8.8.8 while copying a large file:

    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3057ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=5ms TTL=52
    Request timed out.
    Request timed out.
    Reply from 8.8.8.8: bytes=32 time=2495ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=1007ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3007ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=9ms TTL=52
    Request timed out.
    Reply from 8.8.8.8: bytes=32 time=2630ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    Reply from 8.8.8.8: bytes=32 time=3ms TTL=52
    

Log in to reply