UPDATE: 2.4.3 is slower than 2.4.2 (Was: Has FAIRQ Behaviour Changed in 2.4.3?)
-
https://atxfiles.pfsense.org/mirror/downloads/old/
I would strongly advise to try all of this on physical hardware. It wouldn't be the first time that someone got tripped up by a hypervisor glitch.
-
Yea, I'll see if I can dig out some actual hardware to try it on.
It's just the old thing of it was working on 2.4.2-P1.
I know because that was the first pfSense version I've deployed and I did extensive testing to make sure my QoS was working correctly and with excellent performance, and it was!
And then after 2.4.3 I suddenly have this bad performance problem.
I agree though, it's possible it's a hypervisor issue or a hardware issue that's crept in somewhere.Thanks for your help.
-
So I've done some more testing - can anyone smarter than me tell me if the below shows that my 2 x vCPUs are being pegged here?
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 155 ki31 0K 32K CPU1 1 291.7H 55.16% [idle{idle: cpu1}] 11 root 155 ki31 0K 32K RUN 0 290.1H 50.37% [idle{idle: cpu0}] 12 root -92 - 0K 400K WAIT 0 213:36 48.55% [intr{irq261: virtio_pci2}] 12 root -92 - 0K 400K WAIT 1 148:00 43.18% [intr{irq264: virtio_pci3}]That's running a 100Mb/s iperf that only gets ~50-70Mb/s with FAIRQ turned on.
With it off:
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 155 ki31 0K 32K CPU1 1 291.7H 91.36% [idle{idle: cpu1}] 11 root 155 ki31 0K 32K RUN 0 290.1H 87.53% [idle{idle: cpu0}] 12 root -92 - 0K 400K WAIT 0 215:07 11.89% [intr{irq261: virtio_pci2}] 12 root -92 - 0K 400K WAIT 1 149:20 8.10% [intr{irq264: virtio_pci3}]That to me says that I'm CPU bound in the first example
Can someone confirm/deny this for me please :)
-
55 & 50% idle isn't that bad for your dual cores. Turning on shaping quadruples the CPU load.
-
Depends on the hardware for CPU usage. I do HFSC+Codel+NAT at line rate gigabit half-duplex around 17% cpu. I can't test full-duplex because I don't have the client hardware to test bidirectionally at line rate.
-
Hardware is a Intel(R) Core(TM) i5-5250U CPU. pfSense is one of two VM's on a Proxmox (kvm) host, cpu type is Host. pfSense is allocated a Gig of memory and using ~25% of that. PTI is turned off, both in the pfSense guest and the vm host. The only extra packages I use are openvpn-export and Avahi.
I have allocated it 2 CPUs in Proxmox, so really it's probably getting a single core with Hyperthreading turned on.
I can see the old pfSense v2.4.2 but is there a way to get the -P1 I was originally using? I assume if I install 2.4.2 and then do an upgrade I'll go to 2.4.3, not -P1?
I think I have an old vm backup somewhere, but would be good to reinstall from proper image.
-
I'm not aware of a way to pull down individual patches for their general releases.
-
OK so I finally got the time this weekend with the kids asleep to install pfSense 2.4.2 (I can't find a way to load -P1 but it doesn't matter)
And with the same config/QoS config, I get the following results (repeatable every time)
pfSense 2.4.2 - Speedtest 92Mb/s Down, 18Mb/s Up
pfsense 2.4.3 - Speedtest 49Mb/s Down, 18Mb/s UpBoth of those are with the same FAIRQ configuration.
So there appears to be some sort of performance regression between 2.4.2 and 2.4.3. What can I do to diagnose this further? I'd log a ticket on Redmine but I can't actually point to a bug and "It's different between versions" isn't something any sane develop can work with.
Does anyone have any suggestions for how to debug this further so can we can bisect it to a FreeBSD or pfSense patch?
-
I'd log a ticket on Redmine but I can't actually point to a bug and "It's different between versions" isn't something any sane develop can work with.
I would do exactly that, just as long as you can faithfully reproduce the condition.
-
Thanks KOM, I have created a ticket here.
I fully expect to get shouted at though :)Thanks for your help.
