Consistent RDP disconnects
-
"I want to upgrade everything as soon as I can, but it's not a quick and easy task like upgrading a single home computer."
Who said it was - I have worked in Enterprise IT for 20+ years.. Have gone from running IPX to TCP, from running WFW 3.11 boxes with windows NT 3.51 through the whole life span, Shit netware and OS/2 etc.. So I know exactly what is involved with updating a business, not just in one location but across the globe. NT 4, 2k, 2k3, 2k3r2, 2k8, 2k8r2 and now starting to use 2k12.. Sorry but running 2k3 servers in an enterprise/business environment today is just beyond lazy and cheap.. Sorry that is just fact.. You do understand 2k3 is complete EOL here really really soon.. 7/2015 - mainstream support ended back in 2010..
While I can understand budget constraints and hey it works mindset.. You should of been moving off XP years ago – its not like you didn't have a end date for its support years and years ago. Not say you need to be running 8.1 across your enterprise.. But come on using versions of both the server and the client that are not EOL is not crazy talk ;)
What I can tell you is I have never had any issues over pfsense maintaining a connection, even when bouncing off a proxy where the exit point is JAX FL, while I am in Chicago. And enterprise wise using all sorts of vpn connections through my pfsense home connection, be it cisco ipsec, juniper ssl, etc. etc. I RDP into boxes all day long across many firewalls in all different parts of the world, across many different connections and have never seen such an issue. And I quite often have to access servers all over the globe via vpn through pfsense at 2 am in the morning, etc.. And have never had an issue with pfsense disconnecting any sessions. Be it RDP or any other protocol.
What is the error on the client, what is the error on the server, what is the error in pfsense? I would suggest you create a test connection and follow the states in pfsense. As already mentioned are you running out of states? Do you have something running that kills states? Pfsense can kill states on a different things
example
Advanced Firewall/NAT -- Firewall Adaptive Timeouts - have you edited these?
Advanced MISC -- The monitoring process will flush states for a gateway that goes down if this box is not checked. Check this box to disable this behavior. - What are you monitoring for your gateway.. Have you tried turning this feature off.When did this start to happen? You only state
"We've had an issue for a while now where RDP connections are dropping every few minutes."What were you using before pfsense 2.x?? Was there an update to pfsense when this started happening, do you have more than one connection and do failover, policy routing, etc. etc. You mention you don't have issues with websites.. Well website don't really have much issues with creating of new states when you go to a new page or refresh. Where something like Remote Desktop would.
On pfsense what is the current % of your states and what is the total number? What does your MBUF show on the same system information widget?
Unfortunately I don't have answers for all of these because I've been here less than 6 months. A week or two after I started here (brand new, knew only a little about this stuff) the former IT guy up and quit with no notice and I've been learning on the fly ever since. I've almost sort of got this whole system back to where it should be but there's still a lot to be done.
The issue has been happening for as long as I've worked here at least.
We have a few servers at an offsite datacenter maintained by a third party, when I RDP out from there I never get disconnects but that's a whole different firewall and internet connection.
Under advanced settings in pfsense, the only thing labeled timeout in the Firewall/NAT section that I see is the "reflection timeout" field which is blank.
Under Misc the Gateway Monitoring states box is unchecked.
As for error messages, the only ones I could get are from the Cisco AnyConnect Secure Mobility Client that we use for some of our VPN connections:
Event Type: Error
Event Source: acvpnagent
Event Category: Engineering Debug Details
Event ID: 2
Date: 6/19/2014
Time: 9:22:10 AM
User: N/A
Computer: REMOTEACCESSPC1
Description:
Function: CTunnelProtocolDpdMgr::OnTimerExpired
File: .\TunnelProtocolDpdMgr.cpp
Line: 277
Invoked Function: CTunnelProtocolDpdMgr::handleExpiredDPD
Return Code: -25952246 (0xFE74000A)
Description: TUNNELPROTOCOLDPDMGR_ERROR_NO_DPD_RESPONSE:The secure gateway failed to respond to Dead Peer Detection packets.
DTLS/CDTPEvent Type: Error
Event Source: acvpnagent
Event Category: Engineering Debug Details
Event ID: 2
Date: 6/19/2014
Time: 9:22:10 AM
User: N/A
Computer: REMOTEACCESSPC1
Description:
Function: CTunnelStateMgr::OnTunnelStatusChange
File: .\TunnelStateMgr.cpp
Line: 1309
Invoked Function: Tunnel status change callback status
Return Code: -25952246 (0xFE74000A)
Description: TUNNELPROTOCOLDPDMGR_ERROR_NO_DPD_RESPONSE:The secure gateway failed to respond to Dead Peer Detection packets.
DTLSEvent Type: Warning
Event Source: acvpnagent
Event Category: None
Event ID: 2016
Date: 6/19/2014
Time: 9:22:10 AM
User: N/A
Computer: REMOTEACCESSPC1
Description:
Tunnel level reconnect reason code 6:
Reconnecting due to the disruption of the VPN connection to the secure gateway.
Caching the default reconnect reason for DTLS -
Sorry but running 2k3 servers in an enterprise/business environment today is just beyond lazy and cheap.. Sorry that is just fact.. You do understand 2k3 is complete EOL here really really soon.. 7/2015 - mainstream support ended back in 2010..
Bear in mind that you're likely yelling at the wrong guy about this. I'm in the exact same boat as he is. Ancient 2003 servers running a 2003 AD with Exchange 2003. Pentium-4's all over the place. No budget to change anything, and no authority to do anything that might cause the slightest downtime…. so nothing ever gets upgraded. Yes, it's stupid and lazy and cheap to the point of being miserly, but it is what it is. Management, who wouldn't know a router if it hit them in the head, are confident they know more than you about all of IT. But when their lack of knowledge leads to problems, you should have been prepared for that (with your zero budget and authority...). As long as my paycheque hits the bank when it should, they can do as they please. I'll make my money picking up their pieces.
-
Also, this issue is happening even on the Win 7 boxes that have fully up to date RDP. So I don't think that has anything to do with this particular issue.
In pfsense:
Under Advanced - Firewall/NAT, under Network Access Translation I have the following settings:
Disable NAT Reflection for port forwards - checked
Reflection Timeout - blank
Disable NAT Reflection for 1:1 NAT - checked
Automatically create outbound NAT rules which assist inbound NAT rules that direct traffic back out to the same subnet it originated from. - unchecked
Under Firewall all check boxes are unchecked, all fields blank, optimization set to "normal"
Anything else I should look at?
-
Did you look up those errors?
he dartbundle files show this error message when the user gets disconnected: TUNNELPROTOCOLDPDMGR_ERROR_NO_DPD_RESPONSE:The secure gateway failed to respond to Dead Peer Detection packets. This error means that the DTLS channel was torn due to dpd failure. This error is resolved by tweaking the dpd keepalives and issuing these commands:
webvpn
svc keepalive 30
svc dpd-interval client 80
svc dpd-interval gateway 80http://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/100597-anyconnect-vpn-troubleshooting.html
Looks to me like your having failure with your VPN, which in turn will cause your RDP to end.
As to the unchecked box for monitor – so did you CHECK it??
Advanced MISC -- The monitoring process will flush states for a gateway that goes down if this box is not checked. Check this box to disable this behavior.
What are you monitoring?? If you miss pings, states can get flushed.. Which would server all connections.
-
Did you look up those errors?
he dartbundle files show this error message when the user gets disconnected: TUNNELPROTOCOLDPDMGR_ERROR_NO_DPD_RESPONSE:The secure gateway failed to respond to Dead Peer Detection packets. This error means that the DTLS channel was torn due to dpd failure. This error is resolved by tweaking the dpd keepalives and issuing these commands:
webvpn
svc keepalive 30
svc dpd-interval client 80
svc dpd-interval gateway 80http://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/100597-anyconnect-vpn-troubleshooting.html
Looks to me like your having failure with your VPN, which in turn will cause your RDP to end.
As to the unchecked box for monitor – so did you CHECK it??
Advanced MISC -- The monitoring process will flush states for a gateway that goes down if this box is not checked. Check this box to disable this behavior.
What are you monitoring?? If you miss pings, states can get flushed.. Which would server all connections.
I looked them up but I can't actually edit the configuration files for the VPN connect. They're downloaded from client sites. I'd have to ask the clients to edit them.
I've checked the tick box now for monitor. I'll see if that changes anything.
-
Wow, I can't believe it! Ticking that box fixed the problem. I had no idea that setting was even there, and had no idea what it did. Now I know.
-
It didn't really fix anything.. What it did is not reset states on loss of contact with your monitor IP. This points to issue with your gateway not answering pings all the time. Actual issue with your internet line, etc.
What are you monitoring? Normally its your gateway.. Does it not answer ping consistently? You would see this in your pfsense logs.. Pick something else to monitor that is past your isp gateway. Quite often they don't answer pings very well.
Other problem with that is if you saturate your line and pings start to fail, then states can get reset..
-
It didn't really fix anything.. What it did is not reset states on loss of contact with your monitor IP. This points to issue with your gateway not answering pings all the time. Actual issue with your internet line, etc.
What are you monitoring? Normally its your gateway.. Does it not answer ping consistently? You would see this in your pfsense logs.. Pick something else to monitor that is past your isp gateway. Quite often they don't answer pings very well.
Other problem with that is if you saturate your line and pings start to fail, then states can get reset..
All I know is before, RDP would lock up and then have to reconnect every few minutes. Now I can go more than an hour and not notice any hang ups.
The gateway is from Comcast, so I wouldn't be surprised if it's not able to be connected to sometimes. I have a Comcast router in my house that I can't even get into the web app of.
-
How do I see or change what I'm monitoring?
-
system routing - click on edit your gateway(s) and you can turn off monitoring - change the probe time. It defaults to 1 every second. You can change it to monitor some other ip than your actual gateway.
Don't you have the gateways widget on your dashboard - this shows you your gateway IP, response time - if online, etc.
-
system routing - click on edit your gateway(s) and you can turn off monitoring - change the probe time. It defaults to 1 every second. You can change it to monitor some other ip than your actual gateway.
Don't you have the gateways widget on your dashboard - this shows you your gateway IP, response time - if online, etc.
I can see them on my dashboard but they've always been up when I log in, and I don't sit there and watch it.
Any suggestions on what I should monitor instead? Can I specify the IP of a website or something?
-
sure - you can monitor anything on the internet that will answer the ping.. So your currently monitoring your actual gateway - what does your quality graph look like - do you see packet loss, really high times?
example see my pfsense graph, and I also run smokeping - which can monitor anything you want to monitor for latency as well.
If your states were resetting because of loss of monitor - your quality graph should show that. If you saturate your line your latency is going to go way up to your gateway. This is going to happen no matter what your monitor out on the internet.. See the last one the 2 spikes – I was maxing out my download and the latency spikes up!! Something like that could cause you grief if your resetting your states and you fill up your pipe with say your rdp traffic, and then reset, etc.
What does your quality graph look like would be a start to seeing if your having issues with monitoring your gateway.
-
Just upgraded from 2.1.1 to 2.1.4… our office moved into a new building and the PFS install there was 2.1.4... after years of great performance, we quickly noticed RDP disconnect before a minute, every time, when going over a VPN connection handled by an internal MS RRAS server. I went through everything I could think of before finally hooking our previous office PFS device and BOOM everything worked just fine. So, now I'm thinking, ok let's upgrade to the latest version while I'm at it... now the constant RDP disconnects return.
Downgrading now, but hey I feel your pain. If there's anything I can do to help troubleshoot this for others, let me know.
-
Just upgraded from 2.1.1 to 2.1.4… our office moved into a new building and the PFS install there was 2.1.4... after years of great performance, we quickly noticed RDP disconnect before a minute, every time, when going over a VPN connection handled by an internal MS RRAS server. I went through everything I could think of before finally hooking our previous office PFS device and BOOM everything worked just fine. So, now I'm thinking, ok let's upgrade to the latest version while I'm at it... now the constant RDP disconnects return.
Downgrading now, but hey I feel your pain. If there's anything I can do to help troubleshoot this for others, let me know.
OK, I spoke too soon. Still had issues. Downgraded to 2.1.0… STILL ISSUES... went through the following settings with success - disable gateway monitors, clear invalid DF bits, disables firewall scrub, bypass firewall rules for traffic on same interface, unchecked the private networks options under wan, disabled all offloading under network interfaces under advanced
After all that, and a full reboot... everything is working. I'll keep an eye on it and slowly undo some of the changes to narrow it down.
-
Just upgraded from 2.1.1 to 2.1.4… our office moved into a new building and the PFS install there was 2.1.4... after years of great performance, we quickly noticed RDP disconnect before a minute, every time, when going over a VPN connection handled by an internal MS RRAS server. I went through everything I could think of before finally hooking our previous office PFS device and BOOM everything worked just fine. So, now I'm thinking, ok let's upgrade to the latest version while I'm at it... now the constant RDP disconnects return.
Downgrading now, but hey I feel your pain. If there's anything I can do to help troubleshoot this for others, let me know.
OK, I spoke too soon. Still had issues. Downgraded to 2.1.0… STILL ISSUES... went through the following settings with success - disable gateway monitors, clear invalid DF bits, disables firewall scrub, bypass firewall rules for traffic on same interface, unchecked the private networks options under wan, disabled all offloading under network interfaces under advanced
After all that, and a full reboot... everything is working. I'll keep an eye on it and slowly undo some of the changes to narrow it down.
Upgraded to 2.1.1 and still running, also crossed the following off the list (offloading under network interfaces can be default, checksum offloading enabled, gateway monitoring can be enabled, disable PF scrubbing does not have to be checked, clear invalid DF bits does not have to be check) which just leaves the bypass firewall rules for traffic on same interface and the unchecked block private networks optoin under wan.
I'll upgrade to 2.1.2 later this week and report back more findings.
-
This post is deleted!
