States in FIN_WAIT_2:FIN_WAIT_2 when they should not be



  • I am passing traffic from an IPSec VPN connection to a local lan network.  A tcpdump trace on the pfsense LAN side where the server is shows that the client and server are sending the correct FIN/ACK, ACK, FIN/ACK, ACK packets and the states on the server (tested with netstat) are going away properly.  The thing I am trying to figure out is why pfsense 2.0.3 firewall is keeping the states in a FIN_WAIT2:FIN_WAIT2 state instead of a TIME_WAIT state.  FIN_WAIT_2 should mean that a device received an ACK for its sent FIN and is now waiting for a matching FIN from the other side.  A trace on the firewall does show that the FIN/ACK and ACKs to the FIN are being sent though so the firewall should close the state or put them in TIME_WAIT state but that is not happening.

    These connections are TCP tests coming from a loadbalancer based on haproxy btw.  Connections directly from devices in the field seem to go into the correct TIME_WAIT state on the firewall.  Looking at wireshark traces I can't see where the difference is.  The only difference is with the firewall as the FIN_WAIT_2 connections are coming from an IPSec VPN whereas the ones that go into the proper TIME_WAIT state are being load balanced using the built in firewall load balancer (RDR I assume).  Maybe this is an issue with IPSec tunnels with pfsense?

    Keep in mind that the result of this doesn't break anything.  The states are just shown in the incorrect FIN_WAIT_2 state on the firewall instead of TIME_WAIT.  This is going to bug me until I find out the cause though.  I must know! :)

    STATES:
    all tcp 10.x.x.x:9000 <- 10.y.y.y:36148       FIN_WAIT_2:FIN_WAIT_2
       [3704256684 + 5888] wscale 0  [1870992773 + 65535] wscale 7
       age 00:00:16, expires in 00:01:14, 4:3 pkts, 216:168 bytes, rule 122
    all tcp 10.y.y.y:36148 -> 10.x.x.x:9000       FIN_WAIT_2:FIN_WAIT_2
       [1870992773 + 65535] wscale 7  [3704256684 + 5888] wscale 0
       age 00:00:16, expires in 00:01:14, 4:3 pkts, 216:168 bytes, rule 40
    
    TCPDUMP Trace:
    16:08:45.814000 IP 10.y.y.y.36148 > 10.x.x.x.9000: Flags [s], seq 1870992771, win 5840, options [mss 1460,sackOK,TS val 1625713552 ecr 0,nop,wscale 7], length 0
    16:08:45.814229 IP 10.x.x.x.9000 > 10.y.y.y.36148: Flags [S.], seq 3704256682, ack 1870992772, win 16384, options [mss 1460,nop,wscale 0,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
    16:08:45.847842 IP 10.y.y.y.36148 > 10.x.x.x.9000: Flags [.], ack 1, win 46, options [nop,nop,TS val 1625713586 ecr 0], length 0
    16:08:45.847887 IP 10.y.y.y.36148 > 10.x.x.x.9000: Flags [F.], seq 1, ack 1, win 46, options [nop,nop,TS val 1625713586 ecr 0], length 0
    16:08:45.848556 IP 10.x.x.x.9000 > 10.y.y.y.36148: Flags [.], ack 2, win 65535, options [nop,nop,TS val 18189337 ecr 1625713552], length 0
    16:08:45.851434 IP 10.x.x.x.9000 > 10.y.y.y.36148: Flags [F.], seq 1, ack 2, win 65535, options [nop,nop,TS val 18189338 ecr 1625713552], length 0
    16:08:45.885035 IP 10.y.y.y.36148 > 10.x.x.x.9000: Flags [.], ack 2, win 46, options [nop,nop,TS val 1625713623 ecr 18189338], length 0
    
    RULES:
    @40 pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 1158184   Packets: 5595025   Bytes: 639394593   States: 890   ]
      [ Inserted: uid 0 pid 48874 ]
    @122 pass in quick on enc0 inet proto tcp from 10.y.y.y/24 to <ahservers:6> port = 9000 flags S/SA keep state label "USER_RULE: Allow ports from external load balancer"
      [ Evaluations: 255352    Packets: 1556036   Bytes: 135727661   States: 210   ]
      [ Inserted: uid 0 pid 48874 ][/s]</ahservers:6>
    

Log in to reply