SG-3100 disconnects every 20min w/ Cisco AnyConnect VPN client

neatneat · Jun 27, 2019, 5:02 AM

I have an sg-3100 at home and whenever I connect to my corporate VPN (using the Cisco AnyConnect client), my connection seems to timeout and have to reconnect every 20 minutes. Everything works fine while connected; it's just the periodic disconnects that is the issue.

This only happens with my sg-3100. I previously was using a virtualized pfSense instance on ESXi, and never encountered this issue. All customized settings are the same across both installs.

Other VPN connections have no issues (these would be non-corporate VPN's like PIA or NordVPN). However, I'm mandated to use this VPN client by corporate. There are no settings for KeepAlive in the client.

I've researched this and found some potential issues related to the Firewall Optimization settings. I tried different values, but issues occurred with the default Normal setting, and persist with Conservative settings as well.

Below are samples of my AnyConnect logs. The disconnects are nearly always 20 min increments, so I'm guessing it's related to some sort of timeout issues.

I'm lost at how to diagnose further, any ideas appreciated. Thanks!

    8:47:52 PM    Ready to connect.
    8:48:00 PM    Contacting https://VPN-domain.com/.
    8:48:07 PM    User credentials entered.
    8:48:28 PM    Establishing VPN session...
    8:48:29 PM    The AnyConnect Downloader is performing update checks...
    8:48:29 PM    Checking for profile updates...
    8:48:29 PM    Checking for product updates...
    8:48:29 PM    Checking for customization updates...
    8:48:29 PM    Performing any required updates...
    8:48:29 PM    The AnyConnect Downloader updates have been completed.
    8:48:29 PM    Establishing VPN session...
    8:48:29 PM    Establishing VPN - Initiating connection...
    8:48:29 PM    Establishing VPN - Examining system...
    8:48:29 PM    Establishing VPN - Activating VPN adapter...
    8:48:29 PM    Establishing VPN - Configuring system...
    8:48:31 PM    Establishing VPN...
    8:48:31 PM    Connected to https://VPN-domain.com/.
    9:01:21 PM    Reconnecting to https://VPN-domain.com/...
    9:02:10 PM    Disconnect in progress, please wait...
    9:02:11 PM    Ready to connect.
    9:13:48 PM    Contacting https://VPN-domain.com/.
    9:13:52 PM    User credentials entered.
    9:14:00 PM    Establishing VPN session...
    9:14:01 PM    The AnyConnect Downloader is performing update checks...
    9:14:01 PM    Checking for profile updates...
    9:14:01 PM    Checking for product updates...
    9:14:01 PM    Checking for customization updates...
    9:14:01 PM    Performing any required updates...
    9:14:01 PM    The AnyConnect Downloader updates have been completed.
    9:14:01 PM    Establishing VPN session...
    9:14:01 PM    Establishing VPN - Initiating connection...
    9:14:01 PM    Establishing VPN - Examining system...
    9:14:01 PM    Establishing VPN - Activating VPN adapter...
    9:14:01 PM    Establishing VPN - Configuring system...
    9:14:03 PM    Establishing VPN...
    9:14:03 PM    Connected to https://VPN-domain.com/.
    9:22:53 PM    Reconnecting to https://VPN-domain.com/...
    9:23:51 PM    Reconnecting to https://VPN-domain.com/...
    9:23:55 PM    Establishing VPN - Examining system...
    9:23:55 PM    Establishing VPN - Activating VPN adapter...
    9:23:55 PM    Establishing VPN - Configuring system...
    9:23:57 PM    Establishing VPN...
    9:23:57 PM    Connected to https://VPN-domain.com/.
    9:44:36 PM    Reconnecting to https://VPN-domain.com/...
    9:44:53 PM    Establishing VPN - Examining system...
    9:44:53 PM    Establishing VPN - Activating VPN adapter...
    9:44:53 PM    Establishing VPN - Configuring system...
    9:44:55 PM    Establishing VPN...
    9:44:55 PM    Connected to https://VPN-domain.com/.
    10:05:36 PM    Reconnecting to https://VPN-domain.com/...
    10:06:14 PM    Reconnecting to https://VPN-domain.com/...
    10:06:18 PM    Establishing VPN - Examining system...
    10:06:18 PM    Establishing VPN - Activating VPN adapter...
    10:06:18 PM    Establishing VPN - Configuring system...
    10:06:20 PM    Establishing VPN...
    10:06:20 PM    Connected to https://VPN-domain.com/.
    10:26:59 PM    Reconnecting to https://VPN-domain.com/...
    10:27:15 PM    Establishing VPN - Examining system...
    10:27:15 PM    Establishing VPN - Activating VPN adapter...
    10:27:15 PM    Establishing VPN - Configuring system...
    10:27:17 PM    Establishing VPN...
    10:27:17 PM    Connected to https://VPN-domain.com/.
    10:47:57 PM    Reconnecting to https://VPN-domain.com/...
    10:48:12 PM    Establishing VPN - Examining system...
    10:48:12 PM    Establishing VPN - Activating VPN adapter...
    10:48:12 PM    Establishing VPN - Configuring system...
    10:48:14 PM    Establishing VPN...
    10:48:14 PM    Connected to https://VPN-domain.com/.
    11:08:54 PM    Reconnecting to https://VPN-domain.com/...
    11:09:32 PM    Reconnecting to https://VPN-domain.com/...

stephenw10 · Jun 27, 2019, 3:17 PM

Hmm, so no errors just 'connected' then 'reconnecting'.

Is there any traffic going over the tunnel during that time?
Does it still disconnect if you leave a pong running across it?

Seems like it might be a firewall state timeout if there's no keep-alive. You can try setting the timeouts to 'conservative':
https://docs.netgate.com/pfsense/en/latest/config/advanced-setup.html#firewall-nat

Steve

neatneat · Jun 27, 2019, 11:13 PM

@stephenw10

Correct, no errors... one minute I'm connected and then web browsing will start to timeout, then a couple seconds later my VPN is trying to reconnect. I've also already tried adjusting the firewall state to conservative but it doesn't improve (I also rebooted the router between the changes).

There are numerous issues online about this specific issue. Are there any timeout defaults in the netgate hardware specific pfSense that could be contributing versus a non-netgate hardware install?

stephenw10 · Jul 3, 2019, 5:47 AM

So changing those state timeouts made no difference at all? Still disconnects every 20mins?

Check the state table to see what states that is opening when it's connected.

Steve

neatneat · Jul 3, 2019, 5:47 AM

@stephenw10
Correct, no difference at all. I've rebooted the router in-between changes of these settings as well.

I've checked the state table, but I'm unsure of exactly what I'm looking for. I noticed a couple connection issues that likely pertain to this issue, but I'm unsure of how to interpret and proceed. Obfuscated snippet below:

WAN IP: 	100.100.100.100
LAN IP: 	192.168.1.10
Corp IP 1: 	150.10.10.10
Corp IP 2: 	150.20.20.20
Corp IP 3:	150.30.30.30


WAN	tcp	100.100.100.100:19020 (192.168.1.10:49197) -> 150.10.10.10:443	SYN_SENT:CLOSED	3 / 0	192 B / 0 B
WAN	tcp	100.100.100.100:42692 (192.168.1.10:49198) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
99_VLAN	tcp	192.168.1.10:49372 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_VLAN	tcp	192.168.1.10:49376 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B

stephenw10 · Jul 5, 2019, 1:33 AM

I expect to see the states open with traffic both ways if the anyconnect tunnel is up and carrying traffic.
If usually uses UDP 443 or falls back to TCP 443 if that's not available. It could be possible to use a custom port though I'm not aware of that.

Steve

neatneat · Jul 5, 2019, 1:33 AM

@stephenw10

The actual state table shows other states with traffic going both ways for the VPN connection.

The above was only a snippet of some of the states; ones that were CLOSED and didn't look to be fully connected (0 bytes sent). I believe these states are what are causing the disconnects after 20min (ie. if these aren't connecting within 20min, kill the active VPN connection and reconnect).

My question is: using these 4 CLOSED states as the potential cause of the problem, what would you suggest I do to further debug? I'm taking a stab in the dark but could it be potential port forwarding issues? Would I set up a port forward for 9997 to my laptop?

Thanks!

stephenw10 · Jul 5, 2019, 4:30 PM

Each outbound connection from your laptop will create a state on the internal interface and a state on the WAN including NAT.
Those 4 closed states are all different though. Different source ports on each one. Was there a matching state for each that was still open?
What I expect to see is a state opened when the VPN connects and held open at least until the tunnel rekeys. If for some reason it's not opening states at that point in one of the interfaces that would obviously be a problem.

Steve

neatneat · Jul 14, 2019, 6:05 AM

@stephenw10 it doesn't look like there were any matching open states for that example. Here's the entire state table of the above example (IPs obfuscated).

150.90.90.90 is the IP that the AnyConnect client is set to connect (https://VPN-domain.com in OP).

WAN IP: 	100.100.100.100
LAN IP: 	192.168.1.10
VPN IP:             150.90.90.90
Corp IP 1: 	150.10.10.10
Corp IP 2: 	150.20.20.20
Corp IP 3:	150.30.30.30


interface	protocol	connection	state	conns	bytes
99_GUEST	tcp	192.168.1.10:49327 -> 150.40.40.40:9997	CLOSED:SYN_SENT	1 / 0	64 B / 0 B
99_GUEST	tcp	192.168.1.10:49329 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49332 -> 150.20.20.20:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49345 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49352 -> 150.40.40.40:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49358 -> 150.40.40.40:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49367 -> 150.20.20.20:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49372 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49376 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49378 -> 150.20.20.20:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49380 -> 150.40.40.40:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49385 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49387 -> 150.20.20.20:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49389 -> 150.40.40.40:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49391 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49399 -> 150.40.40.40:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49405 -> 150.30.30.30:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49408 -> 150.20.20.20:9997	CLOSED:SYN_SENT	8 / 0	512 B / 0 B
99_GUEST	tcp	192.168.1.10:49394 -> 150.40.40.40:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49397 -> 150.20.20.20:9997	CLOSED:SYN_SENT	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49337 -> 70.70.70.70:443	CLOSING:ESTABLISHED	78 / 79	9 KiB / 19 KiB
99_GUEST	tcp	192.168.1.10:49338 -> 70.70.70.70:443	CLOSING:ESTABLISHED	78 / 79	9 KiB / 19 KiB
99_GUEST	tcp	192.168.1.10:49339 -> 70.70.70.70:443	CLOSING:ESTABLISHED	78 / 80	9 KiB / 19 KiB
99_GUEST	tcp	192.168.1.10:49364 -> 150.60.60.60:443	CLOSING:ESTABLISHED	121 / 120	145 KiB / 13 KiB
99_GUEST	tcp	192.168.1.10:49368 -> 150.60.60.60:443	CLOSING:ESTABLISHED	22 / 31	7 KiB / 8 KiB
99_GUEST	tcp	192.168.1.10:49400 -> 150.50.50.50:443	CLOSING:ESTABLISHED	14 / 23	2 KiB / 6 KiB
99_GUEST	tcp	192.168.1.10:49336 -> 70.70.70.70:443	CLOSING:ESTABLISHED	78 / 80	9 KiB / 19 KiB
99_GUEST	tcp	192.168.1.10:49340 -> 70.70.70.70:443	CLOSING:ESTABLISHED	106 / 97	11 KiB / 25 KiB
99_GUEST	tcp	192.168.1.10:49401 -> 150.50.50.50:443	CLOSING:ESTABLISHED	435 / 600	24 KiB / 846 KiB
WAN	tcp	100.100.100.100:64063 (192.168.1.10:49156) -> 2.2.2.23:5223	ESTABLISHED:CLOSING	39 / 35	5 KiB / 6 KiB
WAN	tcp	100.100.100.100:50569 (192.168.1.10:49228) -> 82.82.82.82:443	ESTABLISHED:CLOSING	17 / 21	2 KiB / 8 KiB
WAN	tcp	100.100.100.100:24749 (192.168.1.10:49246) -> 84.84.84.84:443	ESTABLISHED:CLOSING	28 / 33	6 KiB / 8 KiB
WAN	tcp	100.100.100.100:13815 (192.168.1.10:49256) -> 84.84.84.84:443	ESTABLISHED:CLOSING	321 / 397	43 KiB / 406 KiB
WAN	tcp	100.100.100.100:43609 (192.168.1.10:49261) -> 70.70.70.70:443	ESTABLISHED:CLOSING	47 / 54	5 KiB / 30 KiB
WAN	tcp	100.100.100.100:29558 (192.168.1.10:49264) -> 70.70.70.70:443	ESTABLISHED:CLOSING	50 / 56	6 KiB / 23 KiB
WAN	tcp	100.100.100.100:41905 (192.168.1.10:49265) -> 70.70.70.70:443	ESTABLISHED:CLOSING	48 / 54	6 KiB / 30 KiB
WAN	tcp	100.100.100.100:23005 (192.168.1.10:49268) -> 151.101.0.106:443	ESTABLISHED:CLOSING	19 / 26	2 KiB / 6 KiB
WAN	tcp	100.100.100.100:40183 (192.168.1.10:49271) -> 70.70.70.70:443	ESTABLISHED:CLOSING	87 / 84	11 KiB / 37 KiB
WAN	tcp	100.100.100.100:8542 (192.168.1.10:49274) -> 72.72.72.72:443	ESTABLISHED:CLOSING	18 / 25	2 KiB / 7 KiB
WAN	tcp	100.100.100.100:50966 (192.168.1.10:49283) -> 150.60.60.60:443	ESTABLISHED:CLOSING	20 / 27	6 KiB / 7 KiB
WAN	tcp	100.100.100.100:9312 (192.168.1.10:49294) -> 150.60.60.60:443	ESTABLISHED:CLOSING	75 / 74	87 KiB / 10 KiB
WAN	tcp	100.100.100.100:40462 (192.168.1.10:49308) -> 35.35.35.35:443	ESTABLISHED:CLOSING	239 / 100	323 KiB / 11 KiB
WAN	tcp	100.100.100.100:13052 (192.168.1.10:49316) -> 11.11.11.11:443	ESTABLISHED:CLOSING	17 / 24	2 KiB / 8 KiB
WAN	tcp	100.100.100.100:14594 (192.168.1.10:49322) -> 150.50.50.50:443	ESTABLISHED:CLOSING	599 / 1.821 K	33 KiB / 2.57 MiB
WAN	tcp	100.100.100.100:21254 (192.168.1.10:49323) -> 24.24.24.24:443	ESTABLISHED:CLOSING	10 / 19	2 KiB / 7 KiB
WAN	tcp	100.100.100.100:28927 (192.168.1.10:49324) ->  150.70.70.70:443	ESTABLISHED:CLOSING	13 / 18	3 KiB / 6 KiB
WAN	tcp	100.100.100.100:61747 (192.168.1.10:49325) -> 150.80.80.80:443	ESTABLISHED:CLOSING	15 / 24	2 KiB / 11 KiB
WAN	tcp	100.100.100.100:31961 (192.168.1.10:49337) -> 70.70.70.70:443	ESTABLISHED:CLOSING	78 / 79	9 KiB / 19 KiB
WAN	tcp	100.100.100.100:8863 (192.168.1.10:49338) -> 70.70.70.70:443	ESTABLISHED:CLOSING	78 / 79	9 KiB / 19 KiB
WAN	tcp	100.100.100.100:18026 (192.168.1.10:49339) -> 70.70.70.70:443	ESTABLISHED:CLOSING	78 / 80	9 KiB / 19 KiB
WAN	tcp	100.100.100.100:3256 (192.168.1.10:49364) -> 150.60.60.60:443	ESTABLISHED:CLOSING	121 / 120	145 KiB / 13 KiB
WAN	tcp	100.100.100.100:44448 (192.168.1.10:49368) -> 150.60.60.60:443	ESTABLISHED:CLOSING	22 / 31	7 KiB / 8 KiB
WAN	tcp	100.100.100.100:62411 (192.168.1.10:49400) -> 150.50.50.50:443	ESTABLISHED:CLOSING	14 / 23	2 KiB / 6 KiB
WAN	tcp	100.100.100.100:18795 (192.168.1.10:49262) -> 70.70.70.70:443	ESTABLISHED:CLOSING	50 / 54	6 KiB / 21 KiB
WAN	tcp	100.100.100.100:19285 (192.168.1.10:49336) -> 70.70.70.70:443	ESTABLISHED:CLOSING	78 / 80	9 KiB / 19 KiB
WAN	tcp	100.100.100.100:41344 (192.168.1.10:49340) -> 70.70.70.70:443	ESTABLISHED:CLOSING	106 / 97	11 KiB / 25 KiB
WAN	tcp	100.100.100.100:38858 (192.168.1.10:49401) -> 150.50.50.50:443	ESTABLISHED:CLOSING	435 / 600	24 KiB / 846 KiB
WAN	tcp	100.100.100.100:9445 (192.168.1.10:49217) -> 70.70.70.70:443	ESTABLISHED:ESTABLISHED	67 / 72	7 KiB / 36 KiB
WAN	tcp	100.100.100.100:13222 (192.168.1.10:49224) -> 70.70.70.70:443	ESTABLISHED:ESTABLISHED	85 / 88	8 KiB / 32 KiB
WAN	tcp	100.100.100.100:49576 (192.168.1.10:49201) -> 195.195.195.195:443	ESTABLISHED:ESTABLISHED	11 / 12	2 KiB / 6 KiB
99_GUEST	tcp	192.168.1.10:49411 -> 150.90.90.90:443	ESTABLISHED:ESTABLISHED	56 / 55	6 KiB / 15 KiB
WAN	tcp	100.100.100.100:10923 (192.168.1.10:49411) -> 150.90.90.90:443	ESTABLISHED:ESTABLISHED	56 / 55	6 KiB / 15 KiB
WAN	tcp	100.100.100.100:39287 (192.168.1.10:49320) -> 150.90.90.90:443	ESTABLISHED:ESTABLISHED	16 / 19	6 KiB / 6 KiB
WAN	tcp	100.100.100.100:22620 (192.168.1.10:49363) -> 150.80.80.80:443	ESTABLISHED:FIN_WAIT_2	15 / 15	2 KiB / 8 KiB
WAN	tcp	100.100.100.100:12169 (192.168.1.10:49381) -> 41.41.41.41:443	ESTABLISHED:FIN_WAIT_2	33 / 15	34 KiB / 5 KiB
WAN	tcp	100.100.100.100:62159 (192.168.1.10:49290) -> 150.50.50.50:443	ESTABLISHED:FIN_WAIT_2	32 / 32	4 KiB / 35 KiB
99_GUEST	tcp	192.168.1.10:49363 -> 150.80.80.80:443	FIN_WAIT_2:ESTABLISHED	15 / 15	2 KiB / 8 KiB
99_GUEST	tcp	192.168.1.10:49381 -> 41.41.41.41:443	FIN_WAIT_2:ESTABLISHED	33 / 15	34 KiB / 5 KiB
99_GUEST	tcp	192.168.1.10:49382 -> 190.190.190.190:443	FIN_WAIT_2:FIN_WAIT_2	14 / 15	3 KiB / 1 KiB
WAN	tcp	100.100.100.100:32103 (192.168.1.10:49382) -> 190.190.190.190:443	FIN_WAIT_2:FIN_WAIT_2	14 / 15	3 KiB / 1 KiB
99_GUEST	tcp	192.168.1.10:49407 -> 150.90.90.90:443	FIN_WAIT_2:FIN_WAIT_2	28 / 27	8 KiB / 13 KiB
WAN	tcp	100.100.100.100:15685 (192.168.1.10:49407) -> 150.90.90.90:443	FIN_WAIT_2:FIN_WAIT_2	28 / 27	8 KiB / 13 KiB
99_GUEST	tcp	192.168.1.10:49402 -> 180.180.180.180:443	FIN_WAIT_2:FIN_WAIT_2	18 / 20	5 KiB / 8 KiB
WAN	tcp	100.100.100.100:12399 (192.168.1.10:49402) -> 180.180.180.180:443	FIN_WAIT_2:FIN_WAIT_2	18 / 20	5 KiB / 8 KiB
99_GUEST	tcp	192.168.1.10:49403 -> 3.3.3.3:443	FIN_WAIT_2:FIN_WAIT_2	13 / 9	2 KiB / 6 KiB
WAN	tcp	100.100.100.100:50172 (192.168.1.10:49403) -> 3.3.3.3:443	FIN_WAIT_2:FIN_WAIT_2	13 / 9	2 KiB / 6 KiB
99_GUEST	udp	192.168.1.10:55072 -> 192.168.1.1:53	MULTIPLE:MULTIPLE	2 / 2	140 B / 262 B
99_GUEST	udp	192.168.1.10:61843 -> 150.90.90.90:443	MULTIPLE:MULTIPLE	1.397 K / 956	553 KiB / 372 KiB
WAN	udp	100.100.100.100:15097 (192.168.1.10:61843) -> 150.90.90.90:443	MULTIPLE:MULTIPLE	1.397 K / 956	553 KiB / 372 KiB
WAN	tcp	100.100.100.100:59294 (192.168.1.10:49182) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:42144 (192.168.1.10:49187) -> 150.10.10.10:443	SYN_SENT:CLOSED	2 / 0	128 B / 0 B
WAN	tcp	100.100.100.100:58814 (192.168.1.10:49188) -> 150.10.10.10:443	SYN_SENT:CLOSED	2 / 0	128 B / 0 B
WAN	tcp	100.100.100.100:40975 (192.168.1.10:49189) -> 150.10.10.10:443	SYN_SENT:CLOSED	2 / 0	128 B / 0 B
WAN	tcp	100.100.100.100:22169 (192.168.1.10:49190) -> 150.10.10.10:443	SYN_SENT:CLOSED	2 / 0	128 B / 0 B
WAN	tcp	100.100.100.100:23190 (192.168.1.10:49196) -> 150.10.10.10:443	SYN_SENT:CLOSED	3 / 0	192 B / 0 B
WAN	tcp	100.100.100.100:19020 (192.168.1.10:49197) -> 150.10.10.10:443	SYN_SENT:CLOSED	3 / 0	192 B / 0 B
WAN	tcp	100.100.100.100:42692 (192.168.1.10:49198) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:27054 (192.168.1.10:49200) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:42574 (192.168.1.10:49293) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:60912 (192.168.1.10:49304) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:42655 (192.168.1.10:49307) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:48261 (192.168.1.10:49309) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:11488 (192.168.1.10:49321) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:25840 (192.168.1.10:49327) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:10674 (192.168.1.10:49329) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:35184 (192.168.1.10:49332) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:61260 (192.168.1.10:49345) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:18090 (192.168.1.10:49352) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:63274 (192.168.1.10:49358) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:47537 (192.168.1.10:49367) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:50689 (192.168.1.10:49372) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:63540 (192.168.1.10:49376) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:53611 (192.168.1.10:49378) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:56886 (192.168.1.10:49380) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:27042 (192.168.1.10:49385) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:7531 (192.168.1.10:49387) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:61761 (192.168.1.10:49389) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:14040 (192.168.1.10:49391) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:35582 (192.168.1.10:49399) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:40819 (192.168.1.10:49405) -> 150.30.30.30:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:39882 (192.168.1.10:49408) -> 150.20.20.20:9997	SYN_SENT:CLOSED	8 / 0	512 B / 0 B
WAN	tcp	100.100.100.100:53085 (192.168.1.10:49255) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:60795 (192.168.1.10:49310) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:42614 (192.168.1.10:49394) -> 150.40.40.40:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
WAN	tcp	100.100.100.100:45525 (192.168.1.10:49397) -> 150.20.20.20:9997	SYN_SENT:CLOSED	9 / 0	576 B / 0 B
99_GUEST	tcp	192.168.1.10:49361 -> 2.2.2.2:5223	TIME_WAIT:TIME_WAIT	26 / 18	4 KiB / 5 KiB
WAN	tcp	100.100.100.100:21405 (192.168.1.10:49361) -> 2.2.2.2:5223	TIME_WAIT:TIME_WAIT	26 / 18	4 KiB / 5 KiB

Also, I tried experimenting with unchecking "disable firewall scrub" and "ip do-not-fragment compatibility" (suggestions from some additional threads I found) but neither worked.

stephenw10 · Jul 14, 2019, 8:57 PM

I see two matched pairs of states from source ports 61843 and 49411and one unmatched state from 49320. The internal state has closed for that but there is almost no traffic on it.

Is the client reporting a failed connection at that point?

Steve

neatneat · Jul 14, 2019, 8:57 PM

@stephenw10 yes, this state was captured moments after the failed connection occurred.

stephenw10 · Jul 15, 2019, 9:07 PM

Hmm, well nothing there looks unusual except maybe that state on WAN only.
Might need a packet capture to see what's failing there. I don't see any other reports of that mode of failure.

Steve

neatneat · Jul 18, 2019, 4:56 AM

Following up on this in-case others have this same issue:

It turns out there was a filtering issue, but it wasn't from pfSense. I have a Unifi AC-Pro WAP which has a filtering option called Multicast and Broadcast Filtering which is enabled by default on guest networks. Disabling this feature resolved the issue.

stephenw10 · Jul 18, 2019, 9:04 AM

Nice catch! Hard to imagine what the Anyconnect client needed that would be blocked by such a filter. If it was filtering as expected at least.

Steve