Upgrade to 2.01 has many broken connections

cmb

The raw log would be helpful, we're mis-parsing something on the "TCP:lo" lines.

Everything except those is probably no different than it's ever been, those are all FINs or RSTs and just normal out of state traffic that happens on occasion with every firewall. Just retransmits on connections that are already closed trying to close them again. Those cannot be the cause of any connectivity problems.

lfreeman

I'm using default values:
State table size 1207/97000
MBUF Usage 1716/25600

Optimization is set to Conservative.

I'll try to get a packet capture of the TCP:lo lines

jimp

No need for a packet capture, if you see it in the GUI log, go to Diag > Command and run:

clog /var/log/filter.log

Then find the relevant line in the raw log, and copy/paste it here.

lfreeman

Ok.

Firewall log:

May 9 10:35:46 WAN 69.171.228.70:443 50.XX.XX.130:36046 TCP:lo

packet:

May 9 10:35:46 lascolinas pf: 69.171.228.70.443 > 50.XX.XX.130.36046: Flags [R.], cksum 0x76a3 (correct), seq 4021180300:4021180346, ack 165425407, win 0, length 46 [RST+ BIG-IP: [0x116b7f6:165] Flow e]
May 9 10:36:38 lascolinas pf: 00:00:52.481053 rule 1/0(match): block in on bge0: (tos 0x0, ttl 64, id 2239, offset 0, flags [DF], proto TCP (6), length 788)

jimp

those don't appear to be connected lines, the timestamp should match on both.

lfreeman

May 9 10:35:46 lascolinas pf: 00:01:06.694431 rule 1/0(match): block in on bge1: (tos 0x0, ttl 231, id 59683, offset 0, flags [DF], proto TCP (6), length 86)
May 9 10:35:46 lascolinas pf: 69.171.228.70.443 > 50.84.43.130.36046: Flags [R.], cksum 0x76a3 (correct), seq 4021180300:4021180346, ack 165425407, win 0, length 46 [RST+ BIG-IP: [0x116b7f6:165] Flow e]

jimp

Looks like that BIG-IP: bit is throwing off the regex. Might need to try to tighten it up a little.

lfreeman

Ok maybe this is a clue. How do these packets even show up on my LAN interface with the subnet ip of 10.10.16.0/24?

May 9 14:32:02 LAN 22.42.215.213:40849 209.85.145.188:5228 TCP:FPA
May 9 14:31:41 LAN 22.42.215.213:40849 209.85.145.188:5228 TCP:FPA

May 9 14:31:41 lascolinas pf: 00:07:13.689474 rule 1/0(match): block in on bge0: (tos 0x0, ttl 64, id 14828, offset 0, flags [none], proto TCP (6), length 75)
May 9 14:31:41 lascolinas pf: 22.42.215.213.40849 > 209.85.145.188.5228: Flags [FP.], cksum 0xaf5e (correct), seq 3655068907:3655068930, ack 2331443737, win 8011, options [nop,nop,TS val 6610240 ecr 4266176071], length 23
May 9 14:32:02 lascolinas pf: 00:00:20.554728 rule 1/0(match): block in on bge0: (tos 0x0, ttl 64, id 14829, offset 0, flags [none], proto TCP (6), length 75)
May 9 14:32:02 lascolinas pf: 22.42.215.213.40849 > 209.85.145.188.5228: Flags [FP.], cksum 0xa73e (correct), seq 0:23, ack 1, win 8011, options [nop,nop,TS val 6612320 ecr 4266176071], length 23

cmb

which NIC is bge0?

lfreeman

bge0 is the LAN interface. bge1 is the WAN interface.

bge0 is wired to our internal switch and bge1 is directly connected to our TWC fiber router. There is no other route out of the internal network except via bge1.

lfreeman

I turned off our wireless access point and I haven't seen any more strange packets since. The only devices attached wirelessly were smartphones. Maybe a trojan on a phone was trying to DoS someone's server? The source ip is a DoD address.

cmb

In order for logs like you're seeing on the LAN interface to show up, you'd have to somehow have gotten LAN and WAN mixed onto the same subnet. The logs such as:

May 9 14:31:41 lascolinas pf: 00:07:13.689474 rule 1/0(match): block in on bge0: (tos 0x0, ttl 64, id 14828, offset 0, flags [none], proto TCP (6), length 75)
May 9 14:31:41 lascolinas pf: 22.42.215.213.40849 > 209.85.145.188.5228: Flags [FP.], cksum 0xaf5e (correct), seq 3655068907:3655068930, ack 2331443737, win 8011, options [nop,nop,TS val 6610240 ecr 4266176071], length 23

showing blocking of traffic with both source and destination of public IPs indicates that, you'll never see such traffic unless WAN and LAN are somehow interconnected (or a few other possible but much less likely scenarios like some host on LAN with a public IP manually configured). Given it stopped when the wireless AP was turned off, I suspect maybe somehow that was combining your LAN and WAN networks?