Using static route filtering, still having some lag issues



  • Hello,

    Our network has multiple routed subnets (the routed subnets are managed by cisco routers that are just routing, no NAT or firewall).  We use pfSense for our edge firewall/router/gateway (it holds static routes for the other subnets across point-to-points).  I installed 2.0.x (upgraded recently to 2.0.1).  After install, I noticed that pf would drop connections that occurred across routed subnets after a short period of time even when my ruleset didn't indicate it should at all.  The dropped packets were like, ACK, ACK/SYN type TCP packets.

    So I enabled "Bypass firewall rules for traffic on the same interface". under: System > Advanced > Firewall/NAT.  This was a fix to stop the complete drop outs that were happening.

    Now, I am experiencing periods of lag across the routed subnets that I don't experience on the local subnet in this building.  For example, a windows share I use as a test where I rifle through files in the share here at the local subnet and then a remote subnet.  The local subnet functions as it should, responding almost instantly.  The routed subnet after a few file openings / closings, experiences a lag where the window doesn't respond for maybe 20 seconds and then comes back.  Before I used "Bypass firewall rules for traffic on the same interface", the connection would completely drop out at this point.  Now it lags pretty hard and recovers.

    I also used wireshark on the PC that I am invoking the network share.  I see [TCP] retransmissions, [TCP] Duplicate ACK, and [SMB] Close Requests.

    So I post here hoping someone could provide something to test, or maybe some kind of theroy.

    Thank you.

    below is a tcpdump from the pfSense router looking at the host traffic where I am invoking the windows share across the routed subnet.  Somewhere in here the lag/lockups occurr too.

    [2.0.1-RELEASE][root@pf.domain.local]/root(12): tcpdump -s 0 -i em1 host 192.168.10.199

    09:07:46.071249 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 239040, win 64271, length 172SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.071327 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 239040, win 64271, length 172SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.072165 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 239079, win 64232, length 200SMB PACKET: SMBntcreateX (REQUEST)

    09:07:46.072223 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 239079, win 64232, length 200SMB PACKET: SMBntcreateX (REQUEST)

    09:07:46.084547 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [.], ack 240655, win 65535, length 0
    09:07:46.084621 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [.], ack 240655, win 65535, length 0
    09:07:46.084891 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 240655, win 65535, length 194SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.084950 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 240655, win 65535, length 194SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.085075 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 240694, win 65496, length 210SMB PACKET: SMBntcreateX (REQUEST)

    09:07:46.085134 IP 192.168.10.199.3592 > asrv01.domain.local .microsoft-ds: Flags [P.], ack 240694, win 65496, length 210SMB PACKET: SMBntcreateX (REQUEST)

    09:07:46.090163 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 240798, win 65392, length 204SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.090241 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 240798, win 65392, length 204SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.091211 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 240837, win 65353, length 216SMB PACKET: SMBntcreateX (REQUEST)

    09:07:46.091279 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 240837, win 65353, length 216SMB PACKET: SMBntcreateX (REQUEST)

    09:07:46.097373 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241253, win 64937, length 148SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.097586 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241292, win 64898, length 226SMB PACKET: SMBntcreateX (REQUEST)

    09:07:46.322890 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241292, win 64898, length 374SMB PACKET: SMBtrans2 (REQUEST)

    09:07:46.380608 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [.], ack 241292, win 64898, length 0
    09:07:46.380693 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [.], ack 241292, win 64898, length 0
    09:07:46.928814 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241292, win 64898, length 374SMB PACKET: SMBtrans2 (REQUEST)

    09:07:47.131781 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241292, win 64898, length 148SMB PACKET: SMBtrans2 (REQUEST)

    09:07:48.135806 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241292, win 64898, length 522SMB PACKET: SMBtrans2 (REQUEST)

    09:07:48.298917 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241292, win 64898, length 45SMB PACKET: SMBclose (REQUEST)

    09:07:50.558547 IP 192.168.10.199.3592 > srv01.domain.local.microsoft-ds: Flags [P.], ack 241292, win 64898, length 567SMB PACKET: SMBtrans2 (REQUEST)

    Simple network schematic, (yes the point to points are behind the pfsense, the pfsense gateway handles the static routes for the lan subnet)

    <wan><pfsense>– <local lan="">|
                                    |
                                    |
                            <cisco p-t-p="">/             
                          /                 
                    <cisco endpt=""><cisco endpoint="">Remote                  Remote
                          site A                      site B

    ps. I also set "Firewall optimization options to "Conservative".</cisco></cisco></cisco></local></pfsense></wan>



  • How fast are the point to point connections, and how loaded are they? Windows file sharing performance is very poor over anything with > ~20 ms latency by the design the protocol. You can take the firewall out of the equation completely by adding a static route to one of the test hosts, and I expect you'll see the same behavior. It's most likely latency induced from the sounds of it.



  • Ok good point.  Ill do some tests as you suggest.

    The point to points are 3.0mbps up/down.

    Thanks.

    @cmb:

    How fast are the point to point connections, and how loaded are they? Windows file sharing performance is very poor over anything with > ~20 ms latency by the design the protocol. You can take the firewall out of the equation completely by adding a static route to one of the test hosts, and I expect you'll see the same behavior. It's most likely latency induced from the sounds of it.


Locked