Passthrough Intel Pro/1000 with ntopng only 500 Mbits/s

[bluepr0]

Hello!
I've pfSense virtualised over unRaid (it uses KVM) but I'm doing passthrough of an Intel Pro/1000 4 NIC port (shows up as igb0, igb1, etc). Exact model number is Intel PRO/1000VT Quad Port PCIe Gigabit Ethernet Dell P/N 0HM9JY

Without ntopng I get the full gigabit speed, and it uses around 15% of the CPU (I've 8 cores assigned to the VM).

When enabling ntopng I only get around 500mbits and CPU usage is about 40% Note that I'm not doing routing, the test is in the same subnet

I've tried enabling and disabling the Hardware Checksum Offloading, Hardware TCP Segmentation Offloading and Hardware Large Receive Offloading but no luck.!

Here's an overview of the system and the services I'm running

Any idea on what I can try to tweak? Definitively it's not because CPU power or RAM so I'm not sure how to proceed :/

EDIT: I've found out some more interesting things. It looks like I'm unable to get steady download speeds from internet, unless I'm downloading directly on the pfSense machine.

This is a graph of downloading this file https://speed.hetzner.de/10GB.bin with fetch directly on the pfSense machine

This is the same file downloading it on my server where the pfSense machine is a few seconds later. Note that they have separated physical cards

I've also tried with some other computers on the network and I'm also getting unstable speeds. I believe something is going on my pfSense

Looked at Tuning and Troubleshooting Network Cards docs, section Card-Specific Issues and was giubg ti add kern.ipc.nmbclusters="1000000" but it seems my /boot/loader.conf already got it

Thanks!

[bluepr0]

Added this to system tunables

kern.ipc.nmbclusters="1000000"

Now the graph looks better

But still not steady, not sure if I should duplicate the current value? I have 3GB of RAM and it's 17% full only.

[bluepr0]

Used 4000000 but I'm unable to improve it further. I honestly don't know what I should tweak or what's going on.

On this image below, orange arrow shows when I downloaded the file from the pfSense VM. Green arrow shows when I used my server (where pfSense is running)

EDIT:
Adding a video where you can see how it looks downloading from pfSense vs server (both connected with LAN). Looks like the pfSense is being limited to around 250mb? while the server can download at more speed for a limited amount of time. This is driving me crazy haha
Video: https://d.pr/v/1OYQeP