MS RDP traffic problems after upgrading to 2.6.0 (with no MS RDGateway involved)
-
@sheprador Did you try disabling UDP and only port forwarding TCP? I'm not even sure 2008 supports UDP for RDP so this may be irrelevant, but it was a problem on Windows 10 across several clients.
pfSense just passes those packets on, it doesn't do anything with them.
And if you're connecting from a server on the network pfSense isn't in the connection flow there. It is just a screen share from the server to which you are connected.
-
Sounds like an asymmetric routing issue somehow.
Traffic that is forwarded through pfSense would be different to traffic going over a VPN and then going from pfSense directly.
Are the RDP servers all on the same subnet internally?
Do you see blocked traffic in the firewall logs?
Steve
-
My first question here would be, is the OpenVPN AS server on the same subnet as the RDP server? Basically, if that is working, does pfSense still touch that traffic? If it's on the same subnet then it's just using layer 2 for that meaning pfSense isn't touching it, but if it is on another subnet and IS working then that does point more towards pfSense being the problem (which still doesn't make sense to me but since I have a very similar setup to yours I'd like to make sure this isn't something that is going to impact my setup when I upgrade lol)
I hope I'm making sense with the above question though, maybe didn't word it super great, but what I was getting at is that if the traffic is tunneled to a VPN server, which resides on a DIFFERENT subnet than the RDP server, and it is working, this makes me think it's less likely to be an actual bug in pfSense and more likely to be something else like a misconfiguration. In theory if there was an issue with RDP traffic in pfSense somehow, you'd see it on interVLAN/subnet routing AND WAN to LAN routing.
Like some others have said this sounds like either some kind of asymmetric routing or RDP server issue, that would be the most likely. Not really sure what pfSense would be doing that could break RDP and not also break tons of other things, it doesn't really see RDP traffic and do something special with it, it's just following the rules put in place. So the most common cause of stuff like this is usually a configuration thing.
Can you confirm this happened like RIGHT after upgrading or was it something you noticed within a day? And are you certain nothing else has changed or even been rebooted since this happened?
-
@planedrop said in MS RDP traffic problems after upgrading to 2.6.0 (with no MS RDGateway involved):
My first question here would be, is the OpenVPN AS server on the same subnet as the RDP server?
Ah, I missed that. Also consider that if that server is behind pfSense is the traffic being forwarded to it in the same way as direct RDP? Hitting the same issue potentially.
-
I've found the cause of this problem thanks to this article: freebsd 12.3 / pfsense 2.6 on hyper-v with low throughput
I am running my pfSense's as VM's on Hyper-V 2019 servers.
I've just learned that both Windows Server >=2019 and FreeBSD 12.3 support Receive Segment Coalescing (RSC), but unfortunately this feature isn't working properly in pfSense 2.6 CE
Ref: Hyper-V RSC support in
hn(4)driver is enabled by default and results in very low throughputTo solve this issue I had to disable RSC on Hyper-V's virtual switches (in my case one for LAN and another one for WAN), using the following PowerShell command
for LAN virtual switch
set-VMSwitch -Name "LAN Virtual Switch" -EnableSoftwareRsc $falseand, for WAN virtual switch
set-VMSwitch -Name "WAN Virtual Switch" -EnableSoftwareRsc $falseI hope that a patch for pfSense 2.6.0 CE will be released.
Thank you!
-
@sheprador Super glad you figured it out, that's good news.
Now, I don't want to come off rude in any way, but this is why I think most professionals would recommend you do NOT virtualize pfSense in an environment like this if at all possible, it's just asking for issues IMO.
For me I won't virtualize 2 things (other than for lab purposes of course), firewalls and NAS devices, other than in some fringe cases.
Might be worth getting a physical box if you can? Seems like you had to deal with a lot of headache due to this.
-
@sheprador
https://docs.netgate.com/pfsense/en/latest/releases/2-7-0.html
“ Fixed: Hyper-V RSC support in hn(4) driver is enabled by default and results in very low throughput #12873”Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
Upvote 👍 helpful posts! -
@planedrop
As per my experience I would point out the following points:-
speaking about hardware: my servers are extremely reliable Dell PowerEdge's R4xx, with redundant power supplies, dedicated high quality NIC's (Broadcom or Intel), high performance disk controllers with integrated battery, and iDrac Enterprise remote access cards (an IPMI card with some adde benefits), fast ECC RAM, VM snapshots.
-
speaking about O/S: Windows Server 2019 (and before they were Windows 2012 R2) have proven to be extremely reliable and Microsoft gives very precise information about the underlying technologies of every O/S version/release well before they'll officially release it. Moreover their O/S versions tend to be quite stable until you decide to upgrade to another major version (e.g. from 2019 to 2022)
So, in my experience, I don't think it would be a real improvement to replace my virtual appliances with physical ones. It would be quite expensive to buy hardwares with the same features and options for being reached remotely (IPMI like). Not to mention that I would need to pay for every extra slot used on my datacenter rack. (A tier 4 datacenter, which has very strict regulations about the quality of the devices they allow to be put on their racks).
Bugs happen, also on physical appliances I sadly may think....
In 10+ year I hade 0 problems before this occurrence with this configuration.Unfortunately pfSense decided to not keep up upgrading the CE version with the same pace they did in the past, otherwise this problem would have been solved more or less one year ago.
Of course I'm reasoning about switching to the Plus edition, but I have some concerns apart the mere costs.
Thanks
-
-
@SteveITS
Thanks Steve,
I've seen....unfortunately no patch for version 2.6.0 and no plans for the release of version 2.7 (or 2.8 maybe?).....Anyway, I love pfSense and I appreciate a lot its extreme quality.
-
@sheprador Well a few things here, because I'm not disagreeing that you're stuff is highly reliable, that isn't the point at all, I'd hope any enterprise environment has reliable servers. And while I would disagree about Windows being all that reliable, it's been better as of late on the server side so I'll give you that too.
The point I get at with things being virtualized is that often times the firewall is the thing you need to have functional if the virtual machine host is having issues of any sort, this is why I keep things separate for firewalls and NAS devices in most scenarios.
If my VM host has some major issue, I can still get into it's IPMI as long as my VPNs on my firewall are working, and I can restore from backups if something major is wrong as long as my NAS is it's own box as well.
It's not that you are likely to have issues, TONS of people have run pfSense and other firewall products in virtual environments, even in production, and never had an issue, so that's not my point at all. My point here is that WHEN you do have an issue it could/is more likely to be really bad.
The personal way I do it is physical firewall and NAS, virtual literally everything else, and IMO this is the most reliable way to do it (again in most cases, there are of course some outliers lol).
Finally I would add that using a free community edition package of software is really not best practice in an enterprise environment either; IMO it should be the full pfSense Plus versions if it's truly a critical environment.
But either way glad you got this resolved!
-
@sheprador said in MS RDP traffic problems after upgrading to 2.6.0 (with no MS RDGateway involved):
@SteveITS
Thanks Steve,
I've seen....unfortunately no patch for version 2.6.0 and no plans for the release of version 2.7 (or 2.8 maybe?).....Well it’s in beta so seems like it’s pretty near.
They can’t patch binary files as easily as PHP code.
Pretty sure RSC and definitely Hyper-V speed has come up before on this forum. It wasn’t mentioned above or we probably would have suggested that earlier. :)
-
Yeah, it's a well known issue in hyper-V. There's a huge thread about it in the virtualization sub.
2.7 should be "real soon now". It's fixed in 23.05 too if you need something now.
-
Thank you all guys for your quick support.
I have one last observation that arises from the fact that some of my larger customers are planning to move their software ecosystems to Azure. At that point, in many scenarios, it will be appropriate (or maybe needed) to use firewalls in the form of VMs.
Azure is largely based upon Hyper-V.
I suspect that for this reason Hyper-V will become a particularly important subject compared to the past.
I don't know... we'll see what the future holds for us.. -
@stephenw10 ...hmmm...I'm unsure if it's a Hyper-V problem or a pfSense's 2.6.0 CE one.....
it seems the latter from what I've understood -
Azure is not affected because it doesn't support RSC.
However the pfSense Plus Azure images are based on 23.05 where this is resolved anyway. So you should be good.
-
@stephenw10 said in MS RDP traffic problems after upgrading to 2.6.0 (with no MS RDGateway involved):
Azure is not affected because it doesn't support RSC
Funny how that works.
