Load Balancing (unresolved)



  • Hello,

    I've installed 2.1-BETA1 (amd64) built on Sun May 19 05:38:39 EDT 2013 as a KVM guest on two SmartOS servers using virtio drivers.

    Everything works great; CARP failover works; however, I'm running into a snag using the load balancing service for web traffic.

    The monitor fails for services running locally (on the same hypervisor); however, those same monitors pass on the opposite hypervisor.

    Server A -> Server B … pass
    Server B -> Server A ... pass
    Server A -> Server A ... fail
    Server B -> Server B ... fail

    I am able to contact Server A from the WAN IP Alias... however, if I take Server A down, the fail over doesn't work [as expected].

    All hypervisors and guests can contact both over HTTP.

    pfSense can ping the IP for both http servers; however, it can only connect to port 80 on the "opposite" server.

    For the purpose of this test, I have an allow all rule to make sure that the firewall isn't causing any issues.

    Right now I have two interfaces; one WAN, one LAN, with pfSync running on the LAN until a second NIC comes so I can use a dedicated interface.

    The LAN is a crossover cable between Server A -> Server B.

    The SmartOS option for both my external and internal
    "model": "virtio",
    "allow_ip_spoofing": true,
    "allow_unfiltered_promisc": true

    I have tried changing some of the NAT reflection settings; however, I'm using local IPs that are shared over a cross over interface; those are what I'm having trouble with, so as far as I know, this isn't a NAT issue.

    Below is some output from snoop, with the following setup:

    10.0.0.1 CARP LAN IP
    10.0.0.2 Firewall #1
    10.0.0.2 Firewall #2
    10.0.0.3 Server #1
    10.0.0.4 Server #2

    … HTTP Server 1: 10.0.0.20 [lives on Server #1, with Firewall #1]
    … HTTP Server 2: 10.0.0.21 [lives on Server #2, with Firewall #2]

       10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=7159, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=216, ID=54096, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=40996 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=104, ID=17895, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=11877, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=18122 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=56709, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=40811, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=128, ID=56858, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=12117, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=72, ID=37947, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=41141, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.20    HTTP C port=9163 
       10.0.0.20 -> 10.0.0.2     HTTP R port=9163 
        10.0.0.2 -> 10.0.0.21    HTTP C port=34668 
        10.0.0.2 -> 10.0.0.20    HTTPS C port=52483 
       10.0.0.20 -> 10.0.0.2     HTTPS R port=52483 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
       10.0.0.21 -> 10.0.0.2     HTTP R port=34668 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=43047 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=1000, ID=26510, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.21    HTTP C port=34668 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
        10.0.0.2 -> 10.0.0.21    HTTP HEAD / HTTP/1.0
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
       10.0.0.21 -> 10.0.0.2     HTTP R port=34668 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=43047 
       10.0.0.21 -> 10.0.0.2     HTTP HTTP/1.1 200 OK
       10.0.0.21 -> 10.0.0.2     HTTP R port=34668 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=43047 
        10.0.0.2 -> 10.0.0.21    HTTP C port=34668 
        10.0.0.2 -> 10.0.0.21    HTTP C port=34668 
        10.0.0.2 -> 10.0.0.21    HTTP C port=34668 
       10.0.0.21 -> 10.0.0.2     HTTP R port=34668 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=43047 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=43047 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=43047 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=43047 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=43047 
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=60248, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=72, ID=49470, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=48490 
       10.0.0.20 -> 10.0.0.2     HTTP R port=43673 
       10.0.0.20 -> 10.0.0.2     HTTP R port=54081 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=480, ID=21179, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=9163 
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=21022, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=64755, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=33494, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=8870, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=9163 
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=392, ID=50837, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=11916, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=216, ID=59269, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.20    HTTP C port=54676 
       10.0.0.20 -> 10.0.0.3     HTTP R port=54676 
        10.0.0.3 -> 10.0.0.20    HTTPS C port=43278 
       10.0.0.20 -> 10.0.0.3     HTTPS R port=43278 
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=1000, ID=47175, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.20    HTTP C port=54676 
        10.0.0.3 -> 10.0.0.20    HTTP HEAD / HTTP/1.0
       10.0.0.20 -> 10.0.0.3     HTTP R port=54676 
       10.0.0.20 -> 10.0.0.3     HTTP HTTP/1.1 200 OK
       10.0.0.20 -> 10.0.0.3     HTTP R port=54676 
        10.0.0.3 -> 10.0.0.20    HTTP C port=54676 
        10.0.0.3 -> 10.0.0.20    HTTP C port=54676 
        10.0.0.3 -> 10.0.0.20    HTTP C port=54676 
       10.0.0.20 -> 10.0.0.3     HTTP R port=54676 
       10.0.0.20 -> 10.0.0.2     HTTP R port=7552 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=24381, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=480, ID=55902, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=26538, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=30897, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=299, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=18122 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=72, ID=49818, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=128, ID=35642, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=72, ID=10905, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=128, ID=44985, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=9163 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=30502, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=28411, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=304, ID=53202, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=304, ID=49135, TOS=0x10, TTL=255
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=128, ID=46879, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=216, ID=56952, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.20    HTTP C port=61366 
       10.0.0.20 -> 10.0.0.2     HTTP R port=61366 
        10.0.0.2 -> 10.0.0.21    HTTP C port=18088 
        10.0.0.2 -> 10.0.0.20    HTTPS C port=44455 
       10.0.0.20 -> 10.0.0.2     HTTPS R port=44455 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
       10.0.0.21 -> 10.0.0.2     HTTP R port=18088 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=33698 
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=1000, ID=10667, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.21    HTTP C port=18088 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
        10.0.0.2 -> 10.0.0.21    HTTP HEAD / HTTP/1.0
       10.0.0.21 -> 10.0.0.2     HTTP R port=18088 
       10.0.0.21 -> 10.0.0.2     HTTP HTTP/1.1 200 OK
       10.0.0.21 -> 10.0.0.2     HTTP R port=18088 
        10.0.0.2 -> 10.0.0.21    HTTP C port=18088 
        10.0.0.2 -> 10.0.0.21    HTTP C port=18088 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
        10.0.0.2 -> 10.0.0.21    HTTP C port=18088 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=33698 
       10.0.0.21 -> 10.0.0.2     HTTP R port=18088 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=33698 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=33698 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=33698 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=33698 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
        10.0.0.2 -> 10.0.0.21    HTTPS C port=33698 
       10.0.0.21 -> 10.0.0.2     HTTPS R port=33698 
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=104, ID=35588, TOS=0x10, TTL=255
        10.0.0.2 -> 10.0.0.3     IP  D=10.0.0.3 S=10.0.0.2 LEN=568, ID=51520, TOS=0x10, TTL=255
       10.0.0.20 -> 10.0.0.2     HTTP R port=42370 
       10.0.0.20 -> 10.0.0.2     HTTP R port=22630 
       10.0.0.20 -> 10.0.0.2     HTTP R port=41708 
       10.0.0.20 -> 10.0.0.2     HTTP R port=61366 
        10.0.0.3 -> 10.0.0.2     IP  D=10.0.0.2 S=10.0.0.3 LEN=392, ID=64177, TOS=0x10, TTL=255
    

    Does anyone have any ideas or advice?

    Thanks,
    Jeff



  • It seems like the problem are the checksums.

    Where firewall #1 = 10.0.0.2, and firewall #2 = 10.0.0.3 and webserver #1 10.0.0.20 resides on the same box as firewall #1:

    22:12:32.010453 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25867, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c19b)!)
        10.0.0.20.80 > 10.0.0.2.39941: Flags [S.], cksum 0x1444 (incorrect -> 0xfa4f), seq 3148488349, ack 3149163609, win 64436, options [sackOK,TS val 2690607 ecr 168150,mss 1460,nop,wscale 1], length 0
    22:12:32.681415 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 43406, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.3.6614 > 10.0.0.20.80: Flags [s], cksum 0xfd86 (correct), seq 3712651789, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2469362 ecr 0], length 0
    22:12:32.681518 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 15649, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->e984)!)
        10.0.0.20.80 > 10.0.0.3.6614: Flags [S.], cksum 0x1445 (incorrect -> 0xa48a), seq 3158085937, ack 3712651790, win 64436, options [sackOK,TS val 2690674 ecr 2469362,mss 1460,nop,wscale 1], length 0
    22:12:32.681922 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 53590, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.6614 > 10.0.0.20.80: Flags [.], cksum 0xccfd (correct), seq 1, ack 1, win 520, options [nop,nop,TS val 2469362 ecr 2690674], length 0
    22:12:32.681945 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 85: (tos 0x0, ttl 64, id 17897, offset 0, flags [DF], proto TCP (6), length 71)
        10.0.0.3.6614 > 10.0.0.20.80: Flags [P.], cksum 0xe915 (correct), seq 1:20, ack 1, win 520, options [nop,nop,TS val 2469362 ecr 2690674], length 19
    22:12:32.681965 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15650, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e98b)!)
        10.0.0.20.80 > 10.0.0.3.6614: Flags [.], cksum 0x143d (incorrect -> 0xd33d), seq 1, ack 20, win 64436, options [nop,nop,TS val 2690674 ecr 2469362], length 0
    22:12:32.682179 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 275: (tos 0x0, ttl 64, id 15651, offset 0, flags [DF], proto TCP (6), length 261, bad cksum 0 (->e8b9)!)
        10.0.0.20.80 > 10.0.0.3.6614: Flags [P.], cksum 0x150e (incorrect -> 0xbf75), seq 1:210, ack 20, win 64436, options [nop,nop,TS val 2690674 ecr 2469362], length 209
    22:12:32.682294 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15652, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e989)!)
        10.0.0.20.80 > 10.0.0.3.6614: Flags [F.], cksum 0x143d (incorrect -> 0xd26b), seq 210, ack 20, win 64436, options [nop,nop,TS val 2690674 ecr 2469362], length 0
    22:12:32.682645 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 51925, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.6614 > 10.0.0.20.80: Flags [.], cksum 0xcc1b (correct), seq 20, ack 210, win 518, options [nop,nop,TS val 2469362 ecr 2690674], length 0
    22:12:32.682672 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 58068, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.6614 > 10.0.0.20.80: Flags [.], cksum 0xcc18 (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2469362 ecr 2690674], length 0
    22:12:32.682691 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 8747, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.6614 > 10.0.0.20.80: Flags [F.], cksum 0xcc17 (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2469362 ecr 2690674], length 0
    22:12:32.682709 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15653, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e988)!)
        10.0.0.20.80 > 10.0.0.3.6614: Flags [.], cksum 0x143d (incorrect -> 0xd26a), seq 211, ack 21, win 64436, options [nop,nop,TS val 2690674 ecr 2469362], length 0
    22:12:34.010141 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25868, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c19a)!)
        10.0.0.20.80 > 10.0.0.2.1891: Flags [S.], cksum 0x1444 (incorrect -> 0xc5a3), seq 3153987533, ack 1289269862, win 64436, options [sackOK,TS val 2690807 ecr 170152,mss 1460,nop,wscale 1], length 0
    22:12:34.999957 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25869, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c199)!)
        10.0.0.20.80 > 10.0.0.2.57647: Flags [S.], cksum 0x1444 (incorrect -> 0x0a1a), seq 3156680537, ack 1760254991, win 64436, options [sackOK,TS val 2690906 ecr 171153,mss 1460,nop,wscale 1], length 0
    22:12:37.082452 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 32238, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.43876 > 10.0.0.20.80: Flags [s], cksum 0x0d2b (correct), seq 2576368177, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 172154 ecr 0], length 0
    22:12:37.083080 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 32238, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.43876 > 10.0.0.20.80: Flags [s], cksum 0x0d2b (correct), seq 2576368177, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 172154 ecr 0], length 0
    22:12:37.083101 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25870, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c198)!)
        10.0.0.20.80 > 10.0.0.2.43876: Flags [S.], cksum 0x1444 (incorrect -> 0xab8d), seq 3159267336, ack 2576368178, win 64436, options [sackOK,TS val 2691114 ecr 172154,mss 1460,nop,wscale 1], length 0
    22:12:37.679514 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 25871, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->c1ab)!)
        10.0.0.20.80 > 10.0.0.2.14717: Flags [R.], cksum 0x1430 (incorrect -> 0xf41c), seq 3107978579, ack 1944340388, win 64436, length 0
    22:12:37.849503 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25872, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c196)!)
        10.0.0.20.80 > 10.0.0.2.49704: Flags [S.], cksum 0x1444 (incorrect -> 0x9a67), seq 3124220788, ack 1460435021, win 64436, options [sackOK,TS val 2691191 ecr 159139,mss 1460,nop,wscale 1], length 0
    22:12:37.989482 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25873, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c195)!)
        10.0.0.20.80 > 10.0.0.2.22876: Flags [S.], cksum 0x1444 (incorrect -> 0xfc0f), seq 3140401200, ack 2444052374, win 64436, options [sackOK,TS val 2691205 ecr 165148,mss 1460,nop,wscale 1], length 0
    22:12:38.219441 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25874, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c194)!)
        10.0.0.20.80 > 10.0.0.2.43876: Flags [S.], cksum 0x1444 (incorrect -> 0xab1b), seq 3159267336, ack 2576368178, win 64436, options [sackOK,TS val 2691228 ecr 172154,mss 1460,nop,wscale 1], length 0
    22:12:40.489096 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25875, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c193)!)
        10.0.0.20.80 > 10.0.0.2.43876: Flags [S.], cksum 0x1444 (incorrect -> 0xaa38), seq 3159267336, ack 2576368178, win 64436, options [sackOK,TS val 2691455 ecr 172154,mss 1460,nop,wscale 1], length 0
    22:12:42.018925 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25876, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c192)!)
        10.0.0.20.80 > 10.0.0.2.10312: Flags [S.], cksum 0x1444 (incorrect -> 0x53eb), seq 3151221733, ack 1949110463, win 64436, options [sackOK,TS val 2691608 ecr 169151,mss 1460,nop,wscale 1], length 0
    22:12:42.691805 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 64732, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.3.60491 > 10.0.0.20.80: Flags [s], cksum 0x2cce (correct), seq 1620511003, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2470363 ecr 0], length 0
    22:12:42.691833 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 15654, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->e97f)!)
        10.0.0.20.80 > 10.0.0.3.60491: Flags [S.], cksum 0x1445 (incorrect -> 0xf3a1), seq 3160763727, ack 1620511004, win 64436, options [sackOK,TS val 2691675 ecr 2470363,mss 1460,nop,wscale 1], length 0
    22:12:42.692381 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 51564, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.60491 > 10.0.0.20.80: Flags [.], cksum 0x1c15 (correct), seq 1, ack 1, win 520, options [nop,nop,TS val 2470363 ecr 2691675], length 0
    22:12:42.692416 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 85: (tos 0x0, ttl 64, id 10547, offset 0, flags [DF], proto TCP (6), length 71)
        10.0.0.3.60491 > 10.0.0.20.80: Flags [P.], cksum 0x382d (correct), seq 1:20, ack 1, win 520, options [nop,nop,TS val 2470363 ecr 2691675], length 19
    22:12:42.692436 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15655, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e986)!)
        10.0.0.20.80 > 10.0.0.3.60491: Flags [.], cksum 0x143d (incorrect -> 0x2255), seq 1, ack 20, win 64436, options [nop,nop,TS val 2691675 ecr 2470363], length 0
    22:12:42.692455 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 275: (tos 0x0, ttl 64, id 15656, offset 0, flags [DF], proto TCP (6), length 261, bad cksum 0 (->e8b4)!)
        10.0.0.20.80 > 10.0.0.3.60491: Flags [P.], cksum 0x150e (incorrect -> 0x0e8c), seq 1:210, ack 20, win 64436, options [nop,nop,TS val 2691675 ecr 2470363], length 209
    22:12:42.692574 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15657, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e984)!)
        10.0.0.20.80 > 10.0.0.3.60491: Flags [F.], cksum 0x143d (incorrect -> 0x2183), seq 210, ack 20, win 64436, options [nop,nop,TS val 2691675 ecr 2470363], length 0
    22:12:42.692911 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 37350, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.60491 > 10.0.0.20.80: Flags [.], cksum 0x1b33 (correct), seq 20, ack 210, win 518, options [nop,nop,TS val 2470363 ecr 2691675], length 0
    22:12:42.692939 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 65116, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.60491 > 10.0.0.20.80: Flags [.], cksum 0x1b30 (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2470363 ecr 2691675], length 0
    22:12:42.692958 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 20217, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.60491 > 10.0.0.20.80: Flags [F.], cksum 0x1b2f (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2470363 ecr 2691675], length 0
    22:12:42.692977 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15658, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e983)!)
        10.0.0.20.80 > 10.0.0.3.60491: Flags [.], cksum 0x143d (incorrect -> 0x2182), seq 211, ack 21, win 64436, options [nop,nop,TS val 2691675 ecr 2470363], length 0
    22:12:44.018602 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25877, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c191)!)
        10.0.0.20.80 > 10.0.0.2.57647: Flags [S.], cksum 0x1444 (incorrect -> 0x0694), seq 3156680537, ack 1760254991, win 64436, options [sackOK,TS val 2691808 ecr 171153,mss 1460,nop,wscale 1], length 0
    22:12:45.008389 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25878, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c190)!)
        10.0.0.20.80 > 10.0.0.2.43876: Flags [S.], cksum 0x1444 (incorrect -> 0xa874), seq 3159267336, ack 2576368178, win 64436, options [sackOK,TS val 2691907 ecr 172154,mss 1460,nop,wscale 1], length 0
    22:12:47.090995 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 35164, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.17334 > 10.0.0.20.80: Flags [s], cksum 0x07cd (correct), seq 3163392087, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 173155 ecr 0], length 0
    22:12:47.091591 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 35164, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.17334 > 10.0.0.20.80: Flags [s], cksum 0x07cd (correct), seq 3163392087, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 173155 ecr 0], length 0
    22:12:47.091613 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25879, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c18f)!)
        10.0.0.20.80 > 10.0.0.2.17334: Flags [S.], cksum 0x1444 (incorrect -> 0x7c9a), seq 3162029450, ack 3163392088, win 64436, options [sackOK,TS val 2692115 ecr 173155,mss 1460,nop,wscale 1], length 0
    22:12:47.727980 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 25880, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->c1a2)!)
        10.0.0.20.80 > 10.0.0.2.48412: Flags [R.], cksum 0x1430 (incorrect -> 0xfca8), seq 3110615858, ack 1127984666, win 64436, length 0
    22:12:47.867898 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25881, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c18d)!)
        10.0.0.20.80 > 10.0.0.2.19363: Flags [S.], cksum 0x1444 (incorrect -> 0x23a6), seq 3126935250, ack 407646457, win 64436, options [sackOK,TS val 2692193 ecr 160141,mss 1460,nop,wscale 1], length 0
    22:12:48.007889 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25882, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c18c)!)
        10.0.0.20.80 > 10.0.0.2.59858: Flags [S.], cksum 0x1444 (incorrect -> 0x00c8), seq 3143169063, ack 334752815, win 64436, options [sackOK,TS val 2692207 ecr 166147,mss 1460,nop,wscale 1], length 0
    22:12:48.227839 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25883, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c18b)!)
        10.0.0.20.80 > 10.0.0.2.17334: Flags [S.], cksum 0x1444 (incorrect -> 0x7c28), seq 3162029450, ack 3163392088, win 64436, options [sackOK,TS val 2692229 ecr 173155,mss 1460,nop,wscale 1], length 0
    22:12:50.497501 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25884, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c18a)!)
        10.0.0.20.80 > 10.0.0.2.17334: Flags [S.], cksum 0x1444 (incorrect -> 0x7b45), seq 3162029450, ack 3163392088, win 64436, options [sackOK,TS val 2692456 ecr 173155,mss 1460,nop,wscale 1], length 0
    22:12:52.027301 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25885, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c189)!)
        10.0.0.20.80 > 10.0.0.2.1891: Flags [S.], cksum 0x1444 (incorrect -> 0xbe99), seq 3153987533, ack 1289269862, win 64436, options [sackOK,TS val 2692609 ecr 170152,mss 1460,nop,wscale 1], length 0
    22:12:52.702136 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 27477, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.3.48671 > 10.0.0.20.80: Flags [s], cksum 0x65bc (correct), seq 1332743063, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2471364 ecr 0], length 0
    22:12:52.702159 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 15665, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->e974)!)
        10.0.0.20.80 > 10.0.0.3.48671: Flags [S.], cksum 0x1445 (incorrect -> 0x454d), seq 3163443328, ack 1332743064, win 64436, options [sackOK,TS val 2692676 ecr 2471364,mss 1460,nop,wscale 1], length 0
    22:12:52.703547 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15100, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.48671 > 10.0.0.20.80: Flags [.], cksum 0x6dc0 (correct), seq 1, ack 1, win 520, options [nop,nop,TS val 2471364 ecr 2692676], length 0
    22:12:52.703566 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 85: (tos 0x0, ttl 64, id 37284, offset 0, flags [DF], proto TCP (6), length 71)
        10.0.0.3.48671 > 10.0.0.20.80: Flags [P.], cksum 0x89d8 (correct), seq 1:20, ack 1, win 520, options [nop,nop,TS val 2471364 ecr 2692676], length 19
    22:12:52.703611 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15666, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e97b)!)
        10.0.0.20.80 > 10.0.0.3.48671: Flags [.], cksum 0x143d (incorrect -> 0x7400), seq 1, ack 20, win 64436, options [nop,nop,TS val 2692676 ecr 2471364], length 0
    22:12:52.703670 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 275: (tos 0x0, ttl 64, id 15667, offset 0, flags [DF], proto TCP (6), length 261, bad cksum 0 (->e8a9)!)
        10.0.0.20.80 > 10.0.0.3.48671: Flags [P.], cksum 0x150e (incorrect -> 0x6036), seq 1:210, ack 20, win 64436, options [nop,nop,TS val 2692676 ecr 2471364], length 209
    22:12:52.703764 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15668, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e979)!)
        10.0.0.20.80 > 10.0.0.3.48671: Flags [F.], cksum 0x143d (incorrect -> 0x732e), seq 210, ack 20, win 64436, options [nop,nop,TS val 2692676 ecr 2471364], length 0
    22:12:52.704102 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 58540, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.48671 > 10.0.0.20.80: Flags [.], cksum 0x6cde (correct), seq 20, ack 210, win 518, options [nop,nop,TS val 2471364 ecr 2692676], length 0
    22:12:52.704134 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 53386, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.48671 > 10.0.0.20.80: Flags [.], cksum 0x6cdb (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2471364 ecr 2692676], length 0
    22:12:52.704153 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 65311, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.48671 > 10.0.0.20.80: Flags [F.], cksum 0x6cda (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2471364 ecr 2692676], length 0
    22:12:52.704171 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15669, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e978)!)
        10.0.0.20.80 > 10.0.0.3.48671: Flags [.], cksum 0x143d (incorrect -> 0x732d), seq 211, ack 21, win 64436, options [nop,nop,TS val 2692676 ecr 2471364], length 0
    22:12:54.026987 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25886, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c188)!)
        10.0.0.20.80 > 10.0.0.2.43876: Flags [S.], cksum 0x1444 (incorrect -> 0xa4ee), seq 3159267336, ack 2576368178, win 64436, options [sackOK,TS val 2692809 ecr 172154,mss 1460,nop,wscale 1], length 0
    22:12:55.016854 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25887, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c187)!)
        10.0.0.20.80 > 10.0.0.2.17334: Flags [S.], cksum 0x1444 (incorrect -> 0x7981), seq 3162029450, ack 3163392088, win 64436, options [sackOK,TS val 2692908 ecr 173155,mss 1460,nop,wscale 1], length 0
    22:12:57.099531 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 41719, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.52416 > 10.0.0.20.80: Flags [s], cksum 0x6a0f (correct), seq 2459091740, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 174156 ecr 0], length 0
    22:12:57.100144 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 41719, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.52416 > 10.0.0.20.80: Flags [s], cksum 0x6a0f (correct), seq 2459091740, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 174156 ecr 0], length 0
    22:12:57.100165 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25888, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c186)!)
        10.0.0.20.80 > 10.0.0.2.52416: Flags [S.], cksum 0x1444 (incorrect -> 0x9030), seq 3164735524, ack 2459091741, win 64436, options [sackOK,TS val 2693116 ecr 174156,mss 1460,nop,wscale 1], length 0
    22:12:57.736361 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 25889, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->c199)!)
        10.0.0.20.80 > 10.0.0.2.64807: Flags [R.], cksum 0x1430 (incorrect -> 0xe51d), seq 3113290202, ack 755682554, win 64436, length 0
    22:12:57.876365 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25890, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c184)!)
        10.0.0.20.80 > 10.0.0.2.22582: Flags [S.], cksum 0x1444 (incorrect -> 0xa0b4), seq 3129644688, ack 2800138499, win 64436, options [sackOK,TS val 2693194 ecr 161142,mss 1460,nop,wscale 1], length 0
    22:12:58.016324 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25891, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c183)!)
        10.0.0.20.80 > 10.0.0.2.2567: Flags [S.], cksum 0x1444 (incorrect -> 0x8095), seq 3145892889, ack 3716082355, win 64436, options [sackOK,TS val 2693208 ecr 167149,mss 1460,nop,wscale 1], length 0
    22:12:58.236268 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25892, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c182)!)
        10.0.0.20.80 > 10.0.0.2.52416: Flags [S.], cksum 0x1444 (incorrect -> 0x8fbe), seq 3164735524, ack 2459091741, win 64436, options [sackOK,TS val 2693230 ecr 174156,mss 1460,nop,wscale 1], length 0
    22:13:00.505946 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25893, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c181)!)
        10.0.0.20.80 > 10.0.0.2.52416: Flags [S.], cksum 0x1444 (incorrect -> 0x8edb), seq 3164735524, ack 2459091741, win 64436, options [sackOK,TS val 2693457 ecr 174156,mss 1460,nop,wscale 1], length 0
    22:13:02.035695 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25894, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c180)!)
        10.0.0.20.80 > 10.0.0.2.57647: Flags [S.], cksum 0x1444 (incorrect -> 0xff89), seq 3156680537, ack 1760254991, win 64436, options [sackOK,TS val 2693610 ecr 171153,mss 1460,nop,wscale 1], length 0
    22:13:02.712653 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 62898, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.3.59691 > 10.0.0.20.80: Flags [s], cksum 0xcdd3 (correct), seq 3558731740, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2472365 ecr 0], length 0
    22:13:02.712677 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 15671, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->e96e)!)
        10.0.0.20.80 > 10.0.0.3.59691: Flags [S.], cksum 0x1445 (incorrect -> 0x0a2f), seq 3166236578, ack 3558731741, win 64436, options [sackOK,TS val 2693677 ecr 2472365,mss 1460,nop,wscale 1], length 0
    22:13:02.713031 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 4242, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.59691 > 10.0.0.20.80: Flags [.], cksum 0x32a2 (correct), seq 1, ack 1, win 520, options [nop,nop,TS val 2472365 ecr 2693677], length 0
    22:13:02.713041 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 85: (tos 0x0, ttl 64, id 14147, offset 0, flags [DF], proto TCP (6), length 71)
        10.0.0.3.59691 > 10.0.0.20.80: Flags [P.], cksum 0x4eba (correct), seq 1:20, ack 1, win 520, options [nop,nop,TS val 2472365 ecr 2693677], length 19
    22:13:02.713050 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15672, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e975)!)
        10.0.0.20.80 > 10.0.0.3.59691: Flags [.], cksum 0x143d (incorrect -> 0x38e2), seq 1, ack 20, win 64436, options [nop,nop,TS val 2693677 ecr 2472365], length 0
    22:13:02.713120 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 275: (tos 0x0, ttl 64, id 15673, offset 0, flags [DF], proto TCP (6), length 261, bad cksum 0 (->e8a3)!)
        10.0.0.20.80 > 10.0.0.3.59691: Flags [P.], cksum 0x150e (incorrect -> 0x251c), seq 1:210, ack 20, win 64436, options [nop,nop,TS val 2693677 ecr 2472365], length 209
    22:13:02.713273 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15674, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e973)!)
        10.0.0.20.80 > 10.0.0.3.59691: Flags [F.], cksum 0x143d (incorrect -> 0x3810), seq 210, ack 20, win 64436, options [nop,nop,TS val 2693677 ecr 2472365], length 0
    22:13:02.713617 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 887, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.59691 > 10.0.0.20.80: Flags [.], cksum 0x31c0 (correct), seq 20, ack 210, win 518, options [nop,nop,TS val 2472365 ecr 2693677], length 0
    22:13:02.713639 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 26532, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.59691 > 10.0.0.20.80: Flags [.], cksum 0x31bd (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2472365 ecr 2693677], length 0
    22:13:02.713658 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 55634, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.59691 > 10.0.0.20.80: Flags [F.], cksum 0x31bc (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2472365 ecr 2693677], length 0
    22:13:02.713677 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15675, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e972)!)
        10.0.0.20.80 > 10.0.0.3.59691: Flags [.], cksum 0x143d (incorrect -> 0x380f), seq 211, ack 21, win 64436, options [nop,nop,TS val 2693677 ecr 2472365], length 0
    22:13:04.035391 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25895, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c17f)!)
        10.0.0.20.80 > 10.0.0.2.17334: Flags [S.], cksum 0x1444 (incorrect -> 0x75fb), seq 3162029450, ack 3163392088, win 64436, options [sackOK,TS val 2693810 ecr 173155,mss 1460,nop,wscale 1], length 0
    22:13:05.025220 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25896, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c17e)!)
        10.0.0.20.80 > 10.0.0.2.52416: Flags [S.], cksum 0x1444 (incorrect -> 0x8d17), seq 3164735524, ack 2459091741, win 64436, options [sackOK,TS val 2693909 ecr 174156,mss 1460,nop,wscale 1], length 0
    22:13:07.108055 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 48818, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.27688 > 10.0.0.20.80: Flags [s], cksum 0x0a3b (correct), seq 2213842494, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 175157 ecr 0], length 0
    22:13:07.108669 e2:9d:bf:c3:3e:1e > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 48818, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.2.27688 > 10.0.0.20.80: Flags [s], cksum 0x0a3b (correct), seq 2213842494, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 175157 ecr 0], length 0
    22:13:07.108689 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25897, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c17d)!)
        10.0.0.20.80 > 10.0.0.2.27688: Flags [S.], cksum 0x1444 (incorrect -> 0x40b2), seq 3167417276, ack 2213842495, win 64436, options [sackOK,TS val 2694117 ecr 175157,mss 1460,nop,wscale 1], length 0
    22:13:07.744782 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 25898, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->c190)!)
        10.0.0.20.80 > 10.0.0.2.30661: Flags [R.], cksum 0x1430 (incorrect -> 0x780c), seq 3115990444, ack 2712803019, win 64436, length 0
    22:13:07.944725 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25899, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c17b)!)
        10.0.0.20.80 > 10.0.0.2.55001: Flags [S.], cksum 0x1444 (incorrect -> 0xabce), seq 3132429099, ack 276381974, win 64436, options [sackOK,TS val 2694201 ecr 162143,mss 1460,nop,wscale 1], length 0
    22:13:08.024799 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25900, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c17a)!)
        10.0.0.20.80 > 10.0.0.2.39941: Flags [S.], cksum 0x1444 (incorrect -> 0xec3d), seq 3148488349, ack 3149163609, win 64436, options [sackOK,TS val 2694209 ecr 168150,mss 1460,nop,wscale 1], length 0
    22:13:08.244671 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25901, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c179)!)
        10.0.0.20.80 > 10.0.0.2.27688: Flags [S.], cksum 0x1444 (incorrect -> 0x4040), seq 3167417276, ack 2213842495, win 64436, options [sackOK,TS val 2694231 ecr 175157,mss 1460,nop,wscale 1], length 0
    22:13:10.514341 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25902, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c178)!)
        10.0.0.20.80 > 10.0.0.2.27688: Flags [S.], cksum 0x1444 (incorrect -> 0x3f5d), seq 3167417276, ack 2213842495, win 64436, options [sackOK,TS val 2694458 ecr 175157,mss 1460,nop,wscale 1], length 0
    22:13:12.044125 52:81:4a:5a:68:23 > e2:9d:bf:c3:3e:1e, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 25903, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c177)!)
        10.0.0.20.80 > 10.0.0.2.43876: Flags [S.], cksum 0x1444 (incorrect -> 0x9de4), seq 3159267336, ack 2576368178, win 64436, options [sackOK,TS val 2694611 ecr 172154,mss 1460,nop,wscale 1], length 0
    22:13:12.722822 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 47391, offset 0, flags [DF], proto TCP (6), length 60)
        10.0.0.3.9925 > 10.0.0.20.80: Flags [s], cksum 0xeaa4 (correct), seq 2122573603, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2473366 ecr 0], length 0
    22:13:12.722895 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 15680, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->e965)!)
        10.0.0.20.80 > 10.0.0.3.9925: Flags [S.], cksum 0x1445 (incorrect -> 0x8afa), seq 3168831383, ack 2122573604, win 64436, options [sackOK,TS val 2694678 ecr 2473366,mss 1460,nop,wscale 1], length 0
    22:13:12.723305 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 56376, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.9925 > 10.0.0.20.80: Flags [.], cksum 0xb36d (correct), seq 1, ack 1, win 520, options [nop,nop,TS val 2473366 ecr 2694678], length 0
    22:13:12.723314 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 85: (tos 0x0, ttl 64, id 38851, offset 0, flags [DF], proto TCP (6), length 71)
        10.0.0.3.9925 > 10.0.0.20.80: Flags [P.], cksum 0xcf85 (correct), seq 1:20, ack 1, win 520, options [nop,nop,TS val 2473366 ecr 2694678], length 19
    22:13:12.723322 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15681, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e96c)!)
        10.0.0.20.80 > 10.0.0.3.9925: Flags [.], cksum 0x143d (incorrect -> 0xb9ad), seq 1, ack 20, win 64436, options [nop,nop,TS val 2694678 ecr 2473366], length 0
    22:13:12.723422 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 275: (tos 0x0, ttl 64, id 15682, offset 0, flags [DF], proto TCP (6), length 261, bad cksum 0 (->e89a)!)
        10.0.0.20.80 > 10.0.0.3.9925: Flags [P.], cksum 0x150e (incorrect -> 0xa5e6), seq 1:210, ack 20, win 64436, options [nop,nop,TS val 2694678 ecr 2473366], length 209
    22:13:12.723518 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15683, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e96a)!)
        10.0.0.20.80 > 10.0.0.3.9925: Flags [F.], cksum 0x143d (incorrect -> 0xb8db), seq 210, ack 20, win 64436, options [nop,nop,TS val 2694678 ecr 2473366], length 0
    22:13:12.723906 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 37083, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.9925 > 10.0.0.20.80: Flags [.], cksum 0xb28b (correct), seq 20, ack 210, win 518, options [nop,nop,TS val 2473366 ecr 2694678], length 0
    22:13:12.723933 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 37371, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.9925 > 10.0.0.20.80: Flags [.], cksum 0xb288 (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2473366 ecr 2694678], length 0
    22:13:12.723952 62:f1:0c:32:7c:07 > 52:81:4a:5a:68:23, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 20695, offset 0, flags [DF], proto TCP (6), length 52)
        10.0.0.3.9925 > 10.0.0.20.80: Flags [F.], cksum 0xb287 (correct), seq 20, ack 211, win 520, options [nop,nop,TS val 2473366 ecr 2694678], length 0
    22:13:12.723970 52:81:4a:5a:68:23 > 62:f1:0c:32:7c:07, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 15684, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->e969)!)
        10.0.0.20.80 > 10.0.0.3.9925: Flags [.], cksum 0x143d (incorrect -> 0xb8da), seq 211, ack 21, win 64436, options [nop,nop,TS val 2694678 ecr 2473366], length 0
    				[/s][/s][/s][/s][/s][/s][/s][/s][/s][/s][/s][/s][/s]
    


  • I believe hardware checksum offloading is broken in that circumstance, you'll have to disable it under System>Advanced.



  • It's been disabled from the get go. I checked using ifconfig from the shell and it's definitely off.

    I also reverted back to e1000 instead of virtio drivers to no avail.

    Here is a sample of tcpdump showing the empty checksums (using virtio drivers):

    tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 96 bytes
        10.0.0.21.80 > 10.0.0.3.17833: Flags [S.], cksum 0x0000 (incorrect -> 0x7094), seq 236895230, ack 1405366510, win 64436, options [sackOK,TS val 80053 ecr 64022,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.31085: Flags [S.], cksum 0x0000 (incorrect -> 0x61cf), seq 242098620, ack 1197582769, win 64436, options [sackOK,TS val 80254 ecr 66016,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.36809: Flags [S.], cksum 0x0000 (incorrect -> 0x2df3), seq 244730475, ack 3363313911, win 64436, options [sackOK,TS val 80353 ecr 67017,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.37210: Flags [S.], cksum 0x0000 (incorrect -> 0x3931), seq 247239459, ack 3898515115, win 64436, options [sackOK,TS val 80560 ecr 68018,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.37210: Flags [S.], cksum 0x0000 (incorrect -> 0x38be), seq 247239459, ack 3898515115, win 64436, options [sackOK,TS val 80675 ecr 68018,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.35821: Flags [R.], cksum 0x0000 (incorrect -> 0x6c85), seq 196859152, ack 1541827215, win 64436, length 0
        10.0.0.21.80 > 10.0.0.3.15134: Flags [S.], cksum 0x0000 (incorrect -> 0xaaa7), seq 212994612, ack 395907221, win 64436, options [sackOK,TS val 80785 ecr 55149,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.53201: Flags [S.], cksum 0x0000 (incorrect -> 0xe056), seq 229173355, ack 1811610409, win 64436, options [sackOK,TS val 80788 ecr 61155,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.37210: Flags [S.], cksum 0x0000 (incorrect -> 0x37db), seq 247239459, ack 3898515115, win 64436, options [sackOK,TS val 80902 ecr 68018,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.32078: Flags [S.], cksum 0x0000 (incorrect -> 0x43aa), seq 239598202, ack 3797242161, win 64436, options [sackOK,TS val 81055 ecr 65015,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.36809: Flags [S.], cksum 0x0000 (incorrect -> 0x2a6d), seq 244730475, ack 3363313911, win 64436, options [sackOK,TS val 81255 ecr 67017,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.37210: Flags [S.], cksum 0x0000 (incorrect -> 0x3617), seq 247239459, ack 3898515115, win 64436, options [sackOK,TS val 81354 ecr 68018,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.56747: Flags [S.], cksum 0x0000 (incorrect -> 0x1742), seq 249921820, ack 561857847, win 64436, options [sackOK,TS val 81561 ecr 69019,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.56747: Flags [S.], cksum 0x0000 (incorrect -> 0x16cf), seq 249921820, ack 561857847, win 64436, options [sackOK,TS val 81676 ecr 69019,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.36347: Flags [R.], cksum 0x0000 (incorrect -> 0x5fd8), seq 199520277, ack 3803075384, win 64436, length 0
        10.0.0.21.80 > 10.0.0.3.64226: Flags [S.], cksum 0x0000 (incorrect -> 0x56b6), seq 215618978, ack 3947347370, win 64436, options [sackOK,TS val 81786 ecr 56150,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.13189: Flags [S.], cksum 0x0000 (incorrect -> 0xfb6c), seq 231867792, ack 2708873668, win 64436, options [sackOK,TS val 81789 ecr 62156,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.56747: Flags [S.], cksum 0x0000 (incorrect -> 0x15ec), seq 249921820, ack 561857847, win 64436, options [sackOK,TS val 81903 ecr 69019,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.31085: Flags [S.], cksum 0x0000 (incorrect -> 0x5ac5), seq 242098620, ack 1197582769, win 64436, options [sackOK,TS val 82056 ecr 66016,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.37210: Flags [S.], cksum 0x0000 (incorrect -> 0x3291), seq 247239459, ack 3898515115, win 64436, options [sackOK,TS val 82256 ecr 68018,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.56747: Flags [S.], cksum 0x0000 (incorrect -> 0x1428), seq 249921820, ack 561857847, win 64436, options [sackOK,TS val 82355 ecr 69019,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.25214: Flags [S.], cksum 0x0000 (incorrect -> 0xf50b), seq 252489329, ack 1821663797, win 64436, options [sackOK,TS val 82562 ecr 70020,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.25214: Flags [S.], cksum 0x0000 (incorrect -> 0xf498), seq 252489329, ack 1821663797, win 64436, options [sackOK,TS val 82677 ecr 70020,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.22218: Flags [R.], cksum 0x0000 (incorrect -> 0xf792), seq 202209121, ack 827301017, win 64436, length 0
        10.0.0.21.80 > 10.0.0.3.52932: Flags [S.], cksum 0x0000 (incorrect -> 0x374f), seq 218307818, ack 1352242332, win 64436, options [sackOK,TS val 82786 ecr 57151,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.49933: Flags [S.], cksum 0x0000 (incorrect -> 0x43c4), seq 234501097, ack 1762872308, win 64436, options [sackOK,TS val 82790 ecr 63157,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.25214: Flags [S.], cksum 0x0000 (incorrect -> 0xf3b5), seq 252489329, ack 1821663797, win 64436, options [sackOK,TS val 82904 ecr 70020,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.36809: Flags [S.], cksum 0x0000 (incorrect -> 0x2363), seq 244730475, ack 3363313911, win 64436, options [sackOK,TS val 83057 ecr 67017,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.3.56747: Flags [S.], cksum 0x0000 (incorrect -> 0x10a2), seq 249921820, ack 561857847, win 64436, options [sackOK,TS val 83257 ecr 69019,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.2.8353: Flags [S.], cksum 0x0000 (incorrect -> 0x0b88), seq 254410938, ack 1230268315, win 64436, options [sackOK,TS val 83297 ecr 1404,mss 1460,nop,wscale 1], length 0
        10.0.0.21.80 > 10.0.0.2.8353: Flags [.], cksum 0x0000 (incorrect -> 0x3a3b), seq 1, ack 20, win 64436, options [nop,nop,TS val 83297 ecr 1404], length 0
    
    


  • Have you tried adding any of the following options in your json config.

    "allow_restricted_traffic": true,
    "allow_mac_spoofing": true,
    "allow_dhcp_spoofing": true



  • Thanks for the reply, I was getting nervous I wasn't going to hear back.

    I had not enabled DHCP spoofing. I tried enabling it with both the virtio drivers and e1000 drivers and that didn't help the situation. Below is a copy of my NIC configurations in SmartOS for pfSense VMs:

    pf1

    [
      {
        "interface": "net0",
        "mac": "82:cb:d4:bd:cd:65",
        "nic_tag": "external",
        "gateway": "76.x.xx.193",
        "primary": true,
        "ip": "76.x.xx.195",
        "netmask": "255.255.255.248",
        "model": "e1000",
        "allow_dhcp_spoofing": true,
        "allow_ip_spoofing": true,
        "allow_unfiltered_promisc": true
      },
      {
        "interface": "net1",
        "mac": "e2:9d:bf:c3:3e:1e",
        "nic_tag": "admin",
        "ip": "10.0.0.2",
        "netmask": "255.0.0.0",
        "model": "e1000",
        "allow_dhcp_spoofing": true,
        "allow_ip_spoofing": true,
        "allow_unfiltered_promisc": true
      }
    ]
    

    pf2

        {
          "interface": "net0",
          "mac": "32:9a:35:c7:e2:8b",
          "nic_tag": "admin",
          "gateway": "76.x.x.193",
          "primary": true,
          "ip": "76.x.x.196",
          "netmask": "255.255.255.248",
          "model": "e1000",
          "allow_dhcp_spoofing": true,
          "allow_ip_spoofing": true,
          "allow_unfiltered_promisc": true
        },
        {
          "interface": "net1",
          "mac": "62:f1:0c:32:7c:07",
          "nic_tag": "internal",
          "ip": "10.0.0.3",
          "netmask": "255.0.0.0",
          "model": "e1000",
          "allow_dhcp_spoofing": true,
          "allow_ip_spoofing": true,
          "allow_unfiltered_promisc": true
        }
    

Locked