Passthrough Intel Pro/1000 with ntopng only 500 Mbits/s



  • Hello!
    I've pfSense virtualised over unRaid (it uses KVM) but I'm doing passthrough of an Intel Pro/1000 4 NIC port (shows up as igb0, igb1, etc). Exact model number is Intel PRO/1000VT Quad Port PCIe Gigabit Ethernet Dell P/N 0HM9JY

    Without ntopng I get the full gigabit speed, and it uses around 15% of the CPU (I've 8 cores assigned to the VM).
    0_1551993833994_Screenshot 2019-03-07 at 22.07.18.png

    When enabling ntopng I only get around 500mbits and CPU usage is about 40% Note that I'm not doing routing, the test is in the same subnet
    0_1551993800108_Screenshot 2019-03-07 at 22.13.20.png

    I've tried enabling and disabling the Hardware Checksum Offloading, Hardware TCP Segmentation Offloading and Hardware Large Receive Offloading but no luck.!

    Here's an overview of the system and the services I'm running
    0_1551993915541_Screenshot 2019-03-07 at 22.24.57.png

    Any idea on what I can try to tweak? Definitively it's not because CPU power or RAM so I'm not sure how to proceed :/

    EDIT: I've found out some more interesting things. It looks like I'm unable to get steady download speeds from internet, unless I'm downloading directly on the pfSense machine.

    This is a graph of downloading this file https://speed.hetzner.de/10GB.bin with fetch directly on the pfSense machine
    0_1552039477465_Screenshot 2019-03-08 at 10.56.31.png

    This is the same file downloading it on my server where the pfSense machine is a few seconds later. Note that they have separated physical cards
    0_1552039549408_Screenshot 2019-03-08 at 10.58.38.png

    I've also tried with some other computers on the network and I'm also getting unstable speeds. I believe something is going on my pfSense

    Looked at Tuning and Troubleshooting Network Cards docs, section Card-Specific Issues and was giubg ti add kern.ipc.nmbclusters="1000000" but it seems my /boot/loader.conf already got it

    Thanks!



  • Added this to system tunables

    kern.ipc.nmbclusters="1000000"
    

    Now the graph looks better
    0_1552040546041_Screenshot 2019-03-08 at 11.20.46.png

    But still not steady, not sure if I should duplicate the current value? I have 3GB of RAM and it's 17% full only.



  • Used 4000000 but I'm unable to improve it further. I honestly don't know what I should tweak or what's going on.

    On this image below, orange arrow shows when I downloaded the file from the pfSense VM. Green arrow shows when I used my server (where pfSense is running)
    0_1552041054520_Screen Shot 2019-03-08 at 11.30.00.png

    EDIT:
    Adding a video where you can see how it looks downloading from pfSense vs server (both connected with LAN). Looks like the pfSense is being limited to around 250mb? while the server can download at more speed for a limited amount of time. This is driving me crazy haha
    Video: https://d.pr/v/1OYQeP


Log in to reply