Consistent RDP disconnects
-
We've had an issue for a while now where RDP connections are dropping every few minutes.
It's happening both ways, RDP connections into our office or out from our office.
This also appears to happen regardless of whether or not we connect over a VPN. It also affects Citrix GoToMeeting.
I'm wondering if it's a firewall setting? It happens very consistently every 10 or 15 minutes or so, all throughout the day. And again, it's both ways, and not limited to any one machine in the office. RDP between systems on the internal network are fine.
I thought it could be an internet connectivity issue, so I ran an internet connectivity monitor from this site: https://code.google.com/p/internetconnectivitymonitor/ and had it attempt to connect to google every 5 seconds for 30 minutes, it didn't report any disconnects, and I generally don't have issues with disconnecting from websites. So I don't think it's an internet connectivity issue.
I also ran a couple of trace routes:
From my house to my office:
Tracing route to 75-150-44-1-Oregon.hfc.comcastbusiness.net [75.150.44.1]
over a maximum of 30 hops:
1 7 ms 3 ms 3 ms 10.0.0.1
2 * * * Request timed out.
3 10 ms 12 ms 12 ms xe-0-3-2-32767-sur04.beaverton.or.bverton.comcast.net [68.85.148.249]
4 11 ms 11 ms 12 ms te-3-0-0-ten11.beaverton.or.bverton.comcast.net [68.85.149.58]
5 19 ms 16 ms 20 ms c-98-246-138-191.hsd1.or.comcast.net [98.246.138.191]
6 21 ms 16 ms 15 ms 75-150-44-1-Oregon.hfc.comcastbusiness.net [75.150.44.1]
From my office to my house:
Tracing route to c-67-189-18-29.hsd1.or.comcast.net [67.189.18.29]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 10.0.0.1
2 <1 ms <1 ms <1 ms 10.1.10.1
3 5 ms 5 ms 3 ms 73.90.88.1
4 7 ms 7 ms 40 ms xe-1-1-2-32767-sur04.beaverton.or.bverton.comcast.net [68.85.149.57]
5 7 ms 8 ms 9 ms te-3-0-0-ten07.beaverton.or.bverton.comcast.net [68.85.148.250]
6 17 ms 15 ms 16 ms c-67-189-18-29.hsd1.or.comcast.net [67.189.18.29]
-
Is your pfSense setup behind a VM? Does it hang your local network too?
-
If I RDP onto a machine on the local LAN I don't get the disconnect.
We don't have any VMs in front of the pfSense, at least not to my knowledge.
-
Yeah, I'm having a similar issue, but my pfSense is inside a VM.
And yes, RDP in LAN have no issues, does your network hang when it happens?
-
Nope, no hang on the network when the disconnects happen.
-
Any ideas?
-
So I have a RDP session open to my home workstation almost every single day from work, pretty much all day via openvpn connection (tcp) on pfsense 2.1.3 i386 without any issues. Pfsense is actually running on a VM on esxi 5.5u1
What rdp version are you using are you using 7, 7.1, 8, 8.1 ? What is your client what is your server? Did you enable UDP?
-
We're using RDP version 6.0.6 on the older machines and 6.3.9 on the newer machines. Same thing happens on both.
"What is your client and what is your server?" We've tried this both from client workstations and from servers, to client workstations and to servers. In all cases the same affect occurs, if we connect to a machine outside the office, or a machine outside the office connects to us.
We have UDP enabled on our end but we can't control what happens on the other side.
-
6.0.6 and 6.3.9 is not the protocol version..
For example - see attached.
What do your clients show for their connection.. If you see the little graph bar on the top like in my picture they are using atleast protocol 8 I do believe.
Are you getting errors on the client, the rdp server side or in pfsense? I can tell you I have no issues at all with rdp through pfsense.. So unless your doing something of an odd ball configuration in pfsense I don't think your issue is there. Did you try turning off UDP?
-
I used to RDP (Windows 7 & 8 Remote Desktop) from work to home both "natively" and through VPN. Both "natively" and VPN had to go out through the companies SOCKs proxy and on the home end was pfSense either NAT or VPN. Never had any troubles with long term (day long) connections. For direct (non VPN) connections I used and alternate port (other than the RDP standard 3389). You might try using an alternate port as a test. Say maybe 3390. Maybe there is something conflicting with the standard RDP port.
-
I don't see the little graph bar, the server that our employees RDP into to access the network over VPN is running Server 2008, so you can't force version 8.0 like you can in Win 7 and 2008 R2. I also don't see any options for UDP in the group policy editor for the server.
-
Could it be a state table session duration timeout occurring? (not session inactivity timeout)
-
For direct (non VPN) connections I used and alternate port (other than the RDP standard 3389). You might try using an alternate port as a test. Say maybe 3390. Maybe there is something conflicting with the standard RDP port.
Thanks, I'll try this tonight.
-
"the server that our employees RDP into to access the network over VPN is running Server 2008"
What?? You mean they vpn to your network and then RD to this server? RD is not a vpn connection ;)
I don't understand why would you be using 2008? and not R2? Is your hardware so old that is only 32bit? If you have lots of users that RDP, and its a common part of your business - why would you be running stuff that is so dated? There have been huge improvements in the rdp in 7, 7.1, 8 and 8.1
I do believe 2008 server runs 6.1 which is "4" Versions behind the current protocol.. And from 2008.. Why would you be using such old stuff?
-
Because we dont want to be Microsoft guineapigs ;)
-
Dude your 4 versions behind, I think your safe from being considered a guinea pig. You just have clearly lack of any sort of refresh policy and being cheap.. Your just hurting yourself and users not keeping up with technology.. I could see not running 2012r2 as of yet.. But 2008r2 sp1 that you could update to 8.1 of rdp has been out for what 3 years..
There are many features that will make the experience better for the user using remote desktop in the versions newer than the ANCIENT version your using ;) Plus many security enhancements. If you have lots of RDP users - the use of RemoteFX can make for a much richer faster, better experience for the user.. Which came out in 7.1 – 3 versions back, but your not even there - your 4 Versions behind for gosh sake dude..
-
"the server that our employees RDP into to access the network over VPN is running Server 2008"
What?? You mean they vpn to your network and then RD to this server? RD is not a vpn connection ;)
I don't understand why would you be using 2008? and not R2? Is your hardware so old that is only 32bit? If you have lots of users that RDP, and its a common part of your business - why would you be running stuff that is so dated? There have been huge improvements in the rdp in 7, 7.1, 8 and 8.1
I do believe 2008 server runs 6.1 which is "4" Versions behind the current protocol.. And from 2008.. Why would you be using such old stuff?
Actually most of our servers are running server 2003. Most all of our servers are at least 7 years old. We also still have a few XP machines still in the office. It takes time to upgrade a constant-use production environment. Plus there's compatibility issues to consider (both software and hardware). Don't get me wrong, I want to upgrade everything as soon as I can, but it's not a quick and easy task like upgrading a single home computer.
In the case of VPN connections to our clients, the VPN is what allows us to RDP into their systems in the first place. Also each of those clients could have a completely different firewall configuration. Plus I don't have access to look at their firewalls unless they're using Windows built-in one. I get the same disconnect issue for every client.
As far as the opposite direction, when our employees VPN from home, one of our employees uses the server as their workstation, so they need to RDP into it. They can RDP into it whether or not they're over a VPN, the VPN is just there so that they can do it securely.
But regardless of whether or not they use the VPN to connect to that server it still disconnects every few minutes.
Last night I tried RDPing into a different machine using a different port but had the same issue.
-
"I want to upgrade everything as soon as I can, but it's not a quick and easy task like upgrading a single home computer."
Who said it was - I have worked in Enterprise IT for 20+ years.. Have gone from running IPX to TCP, from running WFW 3.11 boxes with windows NT 3.51 through the whole life span, Shit netware and OS/2 etc.. So I know exactly what is involved with updating a business, not just in one location but across the globe. NT 4, 2k, 2k3, 2k3r2, 2k8, 2k8r2 and now starting to use 2k12.. Sorry but running 2k3 servers in an enterprise/business environment today is just beyond lazy and cheap.. Sorry that is just fact.. You do understand 2k3 is complete EOL here really really soon.. 7/2015 - mainstream support ended back in 2010..
While I can understand budget constraints and hey it works mindset.. You should of been moving off XP years ago – its not like you didn't have a end date for its support years and years ago. Not say you need to be running 8.1 across your enterprise.. But come on using versions of both the server and the client that are not EOL is not crazy talk ;)
What I can tell you is I have never had any issues over pfsense maintaining a connection, even when bouncing off a proxy where the exit point is JAX FL, while I am in Chicago. And enterprise wise using all sorts of vpn connections through my pfsense home connection, be it cisco ipsec, juniper ssl, etc. etc. I RDP into boxes all day long across many firewalls in all different parts of the world, across many different connections and have never seen such an issue. And I quite often have to access servers all over the globe via vpn through pfsense at 2 am in the morning, etc.. And have never had an issue with pfsense disconnecting any sessions. Be it RDP or any other protocol.
What is the error on the client, what is the error on the server, what is the error in pfsense? I would suggest you create a test connection and follow the states in pfsense. As already mentioned are you running out of states? Do you have something running that kills states? Pfsense can kill states on a different things
example
Advanced Firewall/NAT -- Firewall Adaptive Timeouts - have you edited these?
Advanced MISC -- The monitoring process will flush states for a gateway that goes down if this box is not checked. Check this box to disable this behavior. - What are you monitoring for your gateway.. Have you tried turning this feature off.When did this start to happen? You only state
"We've had an issue for a while now where RDP connections are dropping every few minutes."What were you using before pfsense 2.x?? Was there an update to pfsense when this started happening, do you have more than one connection and do failover, policy routing, etc. etc. You mention you don't have issues with websites.. Well website don't really have much issues with creating of new states when you go to a new page or refresh. Where something like Remote Desktop would.
On pfsense what is the current % of your states and what is the total number? What does your MBUF show on the same system information widget?
-
"I want to upgrade everything as soon as I can, but it's not a quick and easy task like upgrading a single home computer."
Who said it was - I have worked in Enterprise IT for 20+ years.. Have gone from running IPX to TCP, from running WFW 3.11 boxes with windows NT 3.51 through the whole life span, Shit netware and OS/2 etc.. So I know exactly what is involved with updating a business, not just in one location but across the globe. NT 4, 2k, 2k3, 2k3r2, 2k8, 2k8r2 and now starting to use 2k12.. Sorry but running 2k3 servers in an enterprise/business environment today is just beyond lazy and cheap.. Sorry that is just fact.. You do understand 2k3 is complete EOL here really really soon.. 7/2015 - mainstream support ended back in 2010..
While I can understand budget constraints and hey it works mindset.. You should of been moving off XP years ago – its not like you didn't have a end date for its support years and years ago. Not say you need to be running 8.1 across your enterprise.. But come on using versions of both the server and the client that are not EOL is not crazy talk ;)
What I can tell you is I have never had any issues over pfsense maintaining a connection, even when bouncing off a proxy where the exit point is JAX FL, while I am in Chicago. And enterprise wise using all sorts of vpn connections through my pfsense home connection, be it cisco ipsec, juniper ssl, etc. etc. I RDP into boxes all day long across many firewalls in all different parts of the world, across many different connections and have never seen such an issue. And I quite often have to access servers all over the globe via vpn through pfsense at 2 am in the morning, etc.. And have never had an issue with pfsense disconnecting any sessions. Be it RDP or any other protocol.
What is the error on the client, what is the error on the server, what is the error in pfsense? I would suggest you create a test connection and follow the states in pfsense. As already mentioned are you running out of states? Do you have something running that kills states? Pfsense can kill states on a different things
example
Advanced Firewall/NAT -- Firewall Adaptive Timeouts - have you edited these?
Advanced MISC -- The monitoring process will flush states for a gateway that goes down if this box is not checked. Check this box to disable this behavior. - What are you monitoring for your gateway.. Have you tried turning this feature off.When did this start to happen? You only state
"We've had an issue for a while now where RDP connections are dropping every few minutes."What were you using before pfsense 2.x?? Was there an update to pfsense when this started happening, do you have more than one connection and do failover, policy routing, etc. etc. You mention you don't have issues with websites.. Well website don't really have much issues with creating of new states when you go to a new page or refresh. Where something like Remote Desktop would.
On pfsense what is the current % of your states and what is the total number? What does your MBUF show on the same system information widget?
Unfortunately I don't have answers for all of these because I've been here less than 6 months. A week or two after I started here (brand new, knew only a little about this stuff) the former IT guy up and quit with no notice and I've been learning on the fly ever since. I've almost sort of got this whole system back to where it should be but there's still a lot to be done.
The issue has been happening for as long as I've worked here at least.
We have a few servers at an offsite datacenter maintained by a third party, when I RDP out from there I never get disconnects but that's a whole different firewall and internet connection.
Under advanced settings in pfsense, the only thing labeled timeout in the Firewall/NAT section that I see is the "reflection timeout" field which is blank.
Under Misc the Gateway Monitoring states box is unchecked.
As for error messages, the only ones I could get are from the Cisco AnyConnect Secure Mobility Client that we use for some of our VPN connections:
Event Type: Error
Event Source: acvpnagent
Event Category: Engineering Debug Details
Event ID: 2
Date: 6/19/2014
Time: 9:22:10 AM
User: N/A
Computer: REMOTEACCESSPC1
Description:
Function: CTunnelProtocolDpdMgr::OnTimerExpired
File: .\TunnelProtocolDpdMgr.cpp
Line: 277
Invoked Function: CTunnelProtocolDpdMgr::handleExpiredDPD
Return Code: -25952246 (0xFE74000A)
Description: TUNNELPROTOCOLDPDMGR_ERROR_NO_DPD_RESPONSE:The secure gateway failed to respond to Dead Peer Detection packets.
DTLS/CDTPEvent Type: Error
Event Source: acvpnagent
Event Category: Engineering Debug Details
Event ID: 2
Date: 6/19/2014
Time: 9:22:10 AM
User: N/A
Computer: REMOTEACCESSPC1
Description:
Function: CTunnelStateMgr::OnTunnelStatusChange
File: .\TunnelStateMgr.cpp
Line: 1309
Invoked Function: Tunnel status change callback status
Return Code: -25952246 (0xFE74000A)
Description: TUNNELPROTOCOLDPDMGR_ERROR_NO_DPD_RESPONSE:The secure gateway failed to respond to Dead Peer Detection packets.
DTLSEvent Type: Warning
Event Source: acvpnagent
Event Category: None
Event ID: 2016
Date: 6/19/2014
Time: 9:22:10 AM
User: N/A
Computer: REMOTEACCESSPC1
Description:
Tunnel level reconnect reason code 6:
Reconnecting due to the disruption of the VPN connection to the secure gateway.
Caching the default reconnect reason for DTLS -
Sorry but running 2k3 servers in an enterprise/business environment today is just beyond lazy and cheap.. Sorry that is just fact.. You do understand 2k3 is complete EOL here really really soon.. 7/2015 - mainstream support ended back in 2010..
Bear in mind that you're likely yelling at the wrong guy about this. I'm in the exact same boat as he is. Ancient 2003 servers running a 2003 AD with Exchange 2003. Pentium-4's all over the place. No budget to change anything, and no authority to do anything that might cause the slightest downtime…. so nothing ever gets upgraded. Yes, it's stupid and lazy and cheap to the point of being miserly, but it is what it is. Management, who wouldn't know a router if it hit them in the head, are confident they know more than you about all of IT. But when their lack of knowledge leads to problems, you should have been prepared for that (with your zero budget and authority...). As long as my paycheque hits the bank when it should, they can do as they please. I'll make my money picking up their pieces.
