Pfsense high cpu usage KVM (Unraid)
-
Yes, there is something significantly wrong with your virtualisation setup there. You can pass 250Mbps with a something ancient and slow like a 1st gen APU at 1GHz.
Steve
-
@stephenw10 Poor me then, i will see if i will try some other things with this setup
-
to me the problem should be investigated on the vm side more than from inside pfsense. i see on google that people tend to bridge the interface instead off using the passthrough for unraid.
personally, for example, i was never able to make pfSense work reliable under virtualbox and i had to change the vm to qemu/kvm -
Here a little update: i changed from pfsense to the OPNsense. Kind off the same thing but OPNsense seemed to handle the troughput way better with way lower usage. Right now i am able to run power safe mode (all 8 cores on 1.4Ghz) where 4 cores are for the firewall and get 250mbit without a problem. I am now using this firewall for all the network traffic in my house. So far no issues.
-
same thing here, i'm using intel cpu and yet very high cpu usage.
I have a 4 port NIC, and I passthrough 2 ports to pfSense, 1 port for WAN, and 1 port for LAN.
I saw a comment on reddit says:it sounds like you've got your WAN to one port of your Intel NIC and the LAN to the other port of your Intel NIC... I don't think that's it's intended use. Each physical NIC should be for one purpose, LAN or WAN but not both. Maybe I'm wrong on that but I've always seen Dual or Quad NICs used as all LAN ports. (reddit)
I'm wondering if this really a bad thing? I have other openwrt installed before and never have this issue, or maybe you guys have a workaround to fix this?
-
No that doesn't make any difference. pfSense just sees those as individual NICs.
Steve
-
ok, I found out my network card is using the igb driver, there are some threads point out that sometimes igb cards need some tweaking. so this is not quite a unraid's fault.
-
@tinysnake Have u tried completely disconnecting the NIC from unraid and bound the PCI(E) card to your VM?
See this video for configuration: https://www.youtube.com/watch?v=58tNUx7A3lM -
@BjornStevens yes, I followed his tutorial to passthrough the nics to pfsense.
And I tried using just 1 port for wan and lan with no performance issue, but I don't quite like this setup, will try tweak the igb settings after work. -
Nope, I tried every possible tweak that I can found and with no luck what so ever.
I found a weird thing: the intr process of igb0 and igb1 is ehci and uhci? as far as I know, these are usb thing not a pcie thing?
-
They are sharing the irq with those USB controllers, which is unusual but probably not an issue.
They don't appear to be using MSI/X, did you disable that? They would normally be on their own, much higher, IRQs.
Steve
-
@stephenw10 Yes I disabled MSI/X, like I said, I tried every possible combination of fine tuning and the problem still there. I even bought an other card, and more problem pops up. I think it's time for me to give up trying pfSense, :(
-
You shouldn't need any tweaks to igb really, I would removed all that and recheck.
Just how high a CPU usage are you seeing? Under what traffic conditions?
Steve
-
@stephenw10 I have a i5 9500T, it's base clock is 2200MHz, and I just gave 1 single core to it.
Network wise, I have a 4 port intel 85276 nic, and simply passthrough 2 ports to it, 1 for wan and 1 for lan, without any "tweaks", wan-to-lan cpu usage is 90% at about 100Mbps.
My most successful result is only 1 port for wan and lan, that way 100Mbps traffic don't even take any cpu usage. But I don't like this topology.
I ordered a i350-T4 after 2 days, and I found the pfSense hardly pick them up, either show no port or just a single one.
I even tried OPNSense, and no nic were found either.
Looks like something just don't play along with FreeBSD.
It took me a lot of sleep time to try pfSense, but sadly non of them worked.. -
Hmm, well that's certainly very high. Does it show that CPU usage in pfSense?
Does it make any difference what two NICs you have? Like one pass though and one internal? Or both internal?
-
@stephenw10 weirdly enough, if I add any virtual NIC to pfSense, then I get spam message says "interrupt storm detected", and cpu usage goes to 30% when idling.
By the way I have other routers in the same network for easy access without changing network ip address.
I'm done fiddling around pfSense and break my home network every night, for now.
If I encounter something weird the next time I'll post full detail and ask you for help. Thanks so much for replying over and over! -
No problem. There's certainly something odd happening there. Many of us here run pfSense in Proxmox in all sorts of exotic configurations and see no significant issues, that's also KVM.
Steve
-
I should mention a thing:
because I have problem, so I upgrade unraid to 6.8.0rc, which qemu version is Q35-4.1, not the usual Q35-3.1 that unraid 6.7.2 has.
It's either Q35-4.1 or unraid 6.8.0rc could not pick up the NICs, too bad that I just send the i350-T4 back to the shop, no way to try the better NIC out. -
That's odd. i350 is very widely used. Unless it was one of the many fake Intel cards that are about...
