[solved] pfSense (2.6.0 & 22.01 ) is very slow on Hyper-V
-
I have same issue as covered here:
Win 2019 Server running Hyper-V
All Ok on 2.5.2 moved to 2.6 and network speed dropped from, 500mb to 2.5mb. I did all the changes suggested here with VMQ etc. Also swapped WAN interface from Intel Pro 1000 PT Dual NIC to onboard Marvell Yukon 88E8059 and no difference.
What did weirdly make a massive difference was using a VPN on my PC, I started NordVPN and my speed jumped back to 488mb. Strange thing was in pfsense it showed my WAN speed as 60mb. Dropping the VPN and speed dropped back bown to a couple of MB.
Essentially the connection via pfsesne 2.6 runs at normal speed if your client is conencting through it on a VPN. All be pfsense thinks it is running at nearly 10x a slower speed.
-
TCP vs UDP maybe? Some hardware off-loading happening?
Do you see error in the interfaces when it's slow?
-
Hi Steve I can setup a zoom session with you on my server unless that requires paid tech support from netgate?
Dom
-
I'm not the guy for that, I know next to nothing about hyper-v.
-
Could be a Microsoft error too, I remember just after Christmas there was an windows update that cause some Hyper-V errors and maybe some are still present. My Server is up to date.
But heres a packet capture of a speed test
-
@dominixise
Also pftop while doing a speed test
-
No actual speedtesting shown there, was it filtered?
All small packets, some MTU issue?
Steve
-
@stephenw10
Sorry i am new to getting logs, try this onepacketcapture (1).zip -
Nearly!
You probably want to filter by the client IP you are running the test on, if you're capturing on LAN. And set the capture to, say, 5000 packets.
Steve
-
Okay thanks for the acknowledgment here is the new capture, I had to put the download on my webserver for download since its 5MB
https://zebrita.publicvm.com/files/packetcapture(2).cap
-
@dominixise
Here is another one with just my host iphttps://zebrita.publicvm.com/files/packetcapture(3).cap
-
A bit of digging and it looks like 2 issues to me.
One in Hyper-V which I have now got resolved, fix below (well for me anyhow)
One in pfsense that is missreporting throughput (I can live with that till a fix comes)For Hyper-V I found this article on RSC https://www.doitfixit.com/blog/2020/01/15/slow-network-speed-with-hyper-v-virtual-machines-on-windows-server-server-2019/
Once I disabled RSC on all virtual switches my speed was back to normal. No restart needed, just go on to Hyper-V host, open powershell and input commands to disable RSC on each virtual switch.These are commands I used
Get-VMSwitch -Name LAN | Select-Object RSC
Checks status, if true run next command LAN is my vswitch nameSet-VMSwitch -Name LAN -EnableSoftwareRsc $false
This disables RSC, re run first command to confirm it is disabledIf your vSwitch has a space in the name add "" around the name
Get-VMSwitch -Name "WAN #1" | Select-Object RSCAfter applying speed is back to normal but pfsense seems to top out showing throughput at 60mb, even though I was getting over 500mb.
Anyhow, hope it helps thers on Hyper-V (this is a 2019 instance of Hyper-V)
-
@rmh-0 fantastic find!
I can confirm this has resolved it for me to, I'll leave it as is until a fix comes out.
Speed with RSC enabled:
Speed with RSC disabled:
-
Disabling RSC did nothing good for me at least. Problem with super slow SMB-Share over VLAN persists.
-
Mmm, RSC is TCP only so I guess that explains why you saw much better throughput with a UDP VPN.
But I'm unsure how the pfSense update would trigger that... -
@stephenw10 maybe with the new FreeBSD kernel release it activated some incompatible functions for Windows Server on a network card level? e.g. thanks to @RMH-0 I disabled RSC and it returned to a normal level.
I know that about 7 years ago I needed to turn off VMQ in a large environment due to a bug in it that caused the guest VMs to all work incredibly slow...
-
V viktor_g referenced this topic on
-
V viktor_g referenced this topic on
-
@donzalmrol If you do not mind a quick query as yours is OK now. Do you get a similar representation of throughput in pfsense in comparison to the spee test.
I get the below which is way different. Trying to see if I have another issue or if others have the same.
-
Disabling RSC did nothing for my environment. Inter-vLAN rate are still a fraction of what they were. Between machines on the same vLAN a file copy takes 3 seconds, between vLANs via the pfSense this jumps to 45-90 minutes.
-
@paulprior This is a file copy in action between vLANs. There are 10Gb\s virtual adapters!
-
@paulprior From Windows:
