[solved] pfSense (2.6.0 & 22.01 ) is very slow on Hyper-V

[Dominixise]

@stephenw10

Hi Steve I can setup a zoom session with you on my server unless that requires paid tech support from netgate?

Dom

[stephenw10]

I'm not the guy for that, I know next to nothing about hyper-v.

[Dominixise]

packetcapture.zip @stephenw10

Could be a Microsoft error too, I remember just after Christmas there was an windows update that cause some Hyper-V errors and maybe some are still present. My Server is up to date.

But heres a packet capture of a speed testlogin-to-view

[Dominixise]

@dominixise
Also pftop while doing a speed testlogin-to-view

[stephenw10]

No actual speedtesting shown there, was it filtered?

All small packets, some MTU issue?

Steve

[Dominixise]

@stephenw10
Sorry i am new to getting logs, try this onepacketcapture (1).zip

[stephenw10]

Nearly!

You probably want to filter by the client IP you are running the test on, if you're capturing on LAN. And set the capture to, say, 5000 packets.

Steve

[Dominixise]

@stephenw10

Okay thanks for the acknowledgment here is the new capture, I had to put the download on my webserver for download since its 5MB

https://zebrita.publicvm.com/files/packetcapture(2).cap

[Dominixise]

@dominixise
Here is another one with just my host ip

https://zebrita.publicvm.com/files/packetcapture(3).cap

[RMH 0]

A bit of digging and it looks like 2 issues to me.

One in Hyper-V which I have now got resolved, fix below (well for me anyhow)
One in pfsense that is missreporting throughput (I can live with that till a fix comes)

For Hyper-V I found this article on RSC https://www.doitfixit.com/blog/2020/01/15/slow-network-speed-with-hyper-v-virtual-machines-on-windows-server-server-2019/
Once I disabled RSC on all virtual switches my speed was back to normal. No restart needed, just go on to Hyper-V host, open powershell and input commands to disable RSC on each virtual switch.

These are commands I used

Get-VMSwitch -Name LAN | Select-Object RSC
Checks status, if true run next command LAN is my vswitch name

Set-VMSwitch -Name LAN -EnableSoftwareRsc $false
This disables RSC, re run first command to confirm it is disabled

If your vSwitch has a space in the name add "" around the name
Get-VMSwitch -Name "WAN #1" | Select-Object RSC

After applying speed is back to normal but pfsense seems to top out showing throughput at 60mb, even though I was getting over 500mb.

Anyhow, hope it helps thers on Hyper-V (this is a 2019 instance of Hyper-V)

[DonZalmrol]

@rmh-0 fantastic find!

I can confirm this has resolved it for me to, I'll leave it as is until a fix comes out.

Speed with RSC enabled:
login-to-view

Speed with RSC disabled:
login-to-view

[Bob.Dig]

Disabling RSC did nothing good for me at least. Problem with super slow SMB-Share over VLAN persists.

[stephenw10]

Mmm, RSC is TCP only so I guess that explains why you saw much better throughput with a UDP VPN.
But I'm unsure how the pfSense update would trigger that...

[DonZalmrol]

@stephenw10 maybe with the new FreeBSD kernel release it activated some incompatible functions for Windows Server on a network card level? e.g. thanks to @RMH-0 I disabled RSC and it returned to a normal level.

I know that about 7 years ago I needed to turn off VMQ in a large environment due to a bug in it that caused the guest VMs to all work incredibly slow...

[RMH 0]

@donzalmrol If you do not mind a quick query as yours is OK now. Do you get a similar representation of throughput in pfsense in comparison to the spee test.

I get the below which is way different. Trying to see if I have another issue or if others have the same.

login-to-view

[PaulPrior]

Disabling RSC did nothing for my environment. Inter-vLAN rate are still a fraction of what they were. Between machines on the same vLAN a file copy takes 3 seconds, between vLANs via the pfSense this jumps to 45-90 minutes.
login-to-view

[PaulPrior]

@paulprior This is a file copy in action between vLANs. There are 10Gb\s virtual adapters!
login-to-view

[PaulPrior]

@paulprior From Windows:
login-to-view

[Bob.Dig]

Maybe they are different problems, I for myself had no problem with my WAN speed from the beginning.

[PaulPrior]

So, disabling RSC has restored the network speed between VMs behind the pfSense and the internet (HTTPS download speeds), but the inter-vlan SMB file copy speeds are awful. Not quite dial-up modem speeds but almost.