SG-3100: 2nd new virt interface fails



  • Review: SG-3100 comes w/ 3 interfaces:

    • 0: OPT
    • 1: LAN (4-port switch)
    • 2: WAN

    I'm using the OPT port for a 2nd WAN (deactivated for now). I'm keeping LAN ports 1 & 2 as-is, but making 3 & 4 a couple of different internal networks. I've [physically] labeled the ports thus:

    • WAN: WAN1
    • OPT: WAN2
    • LAN1: LAN - 192.168.1.x
    • LAN2: LAN - 192.168.1.x
    • LAN3: Guest/Wi-Fi - 192.168.3.x
    • LAN4: DMZ - 192.168.2.x

    Using the instructions in Configuring the Switch Ports ("This optional guide shows the steps required to configure the 4 switched Ethernet ports as discrete ports."), I added the VLANs for ports 3 & 4.

    The first time I configured it, I went through all the steps (in order) and set up port 4, then I went back and did the same for port 3. Port 4 worked fine. Port 3 could communicate with its subnet, but nothing else; e.g. I could reach 192.168.3.1 but not .2.1, .1.1, or anything on the WAN. Pings to WAN destinations showed responses coming back to the router, but not being passed back out port 3.

    I checked everything I could think of, finally deleted everything associated with port 3 & recreating it, no dice. (Rebooted a couple of times during testing, just to see if it helped.) Finally, I deleted interfaces for both 3 & 4, and set VLANs back to the default (everything on 1). Couldn't get a DHCP lease (was still attached to port 3) but after reboot it worked fine; I verified that all 4 ports worked correctly.

    I repeated my setup of ports 3 & 4, this time reversing the order (doing 3 then 4 instead of doing 4 then 3) and doing them both at the same time (step 1 for 3 & 4, then step 2...) instead of completing all steps for one port before moving to the other as I did before.

    Imagine my surprise when, this time, port 3 worked, and port 4 didn't! The first port to be set up each time worked, the 2nd one didn't. I know that only 2 coin tosses landing on heads isn't a very strong suggestion that there's something wrong w/ tails, but I haven't found any other link yet. (it's not the particular port, VLAN#, etc.)

    I created a single firewall rule on each interface by copying the default "allow all" rule from LAN & changing the interface in 2 places (where the rule applies and what net to allow traffic from).

    Would y'all take a peek at my screenshots here and see if you spot anything funny? I went over them, and can't tell any difference that would make one work and not the other.
    Selection_001.png
    Selection_002.png
    Selection_003.png
    Selection_004.png
    Selection_005.png
    Selection_006.png
    Selection_007.png
    It's gotta be something stupid, but I can't seem to find it. I'm happy to provide any other info that may be helpful -- just ask!

    Thank you.



  • This is somewhat redundant since I already knew that .2.* devices could talk to .2.1 but not any of the router's other addresses, but...
    I configured an OpenVpn server on the Netgate, and connected to it from the outside. On that connection, I can talk to any of the Netgate's IP addresses, and I can talk to a device on the port 3 network (192.168.3.50) but not the same device on the problem port 4 network (192.168.2.50). This again illustrates that the problem is not limited to communication between the problem port and WAN addresses; it prevents talking to VPN tunnel devices too, i.e. the router will send traffic out port 4 (to 192.168.2.* devices) that originates from 192.168.2.1, but it will not forward packets from any other interface.


  • Netgate Administrator

    Can we see the routing table?

    Maybe the firewall states when you are trying to connect out from port 4?

    What you have configured there looks correct. It must be something we can't see like a subnet problem or a gateway on the wrong interface...

    Steve



  • @stephenw10 Thanks for the suggestions, Steve.
    Here's the IP4 routing table via netstat -rn:

    Destination        Gateway            Flags     Netif Expire
    default            10.1.10.1          UGS     mvneta2
    10.1.10.0/24       link#8             U       mvneta2
    10.1.10.2          link#8             UHS         lo0
    127.0.0.1          link#10            UH          lo0
    192.168.1.0/24     link#2             U       mvneta1
    192.168.1.1        link#2             UHS         lo0
    192.168.2.0/24     link#14            U      mvneta1.
    192.168.2.1        link#14            UHS         lo0
    192.168.3.0/24     link#13            U      mvneta1.
    192.168.3.1        link#13            UHS         lo0
    192.168.5.0/24     192.168.5.2        UGS      ovpns1
    192.168.5.1        link#15            UHS         lo0
    192.168.5.2        link#15            UH       ovpns1
    

    Hmmm, looking at firewall states would've been smart...
    Unfortunately, I had to install the router today as-is (they had failing hardware that was causing extreme trouble) and I was under time pressure and didn't think about setting up a test PC on-site. (Oops!) Pings to the router interface work fine since they're not actually leaving/coming in, so that's not a test source. My contact is neither technical nor terribly motivated right now, so it may take me a while to set something up, since it's in another city and I don't expect to be there again soon. (Today's the 1st time I've left home in 6 weeks.)

    In the meantime, anything else that I ought to show that doesn't require traffic?

    Edit: I just realized that since the problem isn't packets coming into that interface, but rather that packets from other nets won't go out on it, I may be able to get a valid state snapshot even without a test machine -- if I can convince it try even though the Ethernet port's not connected to anything. If I can get it to make an honest attempt to send something from the router itself (which was working) -- maybe I'll have have to somehow stuff a fake entry in the ARP table or something, I don't know; I'm getting goofy and probably shouldn't even be typing until I've slept on it -- but if that'll produce results, then I ought to be able to do the same thing from another interface to see the difference. I'll revisit it tomorrow.


  • Netgate Administrator

    Ok, you seem to have interfaces marked mvneta1. instead of mvneta1.4083 and mvneta1.4084.

    Check the output of ifconfig and of etherswitchcfg.

    Try using some other VLAN ID such as 20, something much lower.

    If it really is failing to create those VLANs there should be errors in the system log.

    Steve



  • The "mvneta1." thing is just netstat truncating the column's displayed width; it would look fine if "mvneta1" was named something short like "en1".
    ifconfig:

    code_textmvneta0: flags=8b02<BROADCAST,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=800bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
    	ether 00:08:a2:10:ee:37
    	hwaddr 00:08:a2:10:ee:37
    	media: Ethernet autoselect (none)
    	status: no carrier
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    mvneta1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM>
    	ether 00:08:a2:10:ee:38
    	hwaddr 00:08:a2:10:ee:38
    	inet6 fe80::208:a2ff:fe10:ee38%mvneta1 prefixlen 64 scopeid 0x2
    	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
    	media: Ethernet 2500Base-KX <full-duplex>
    	status: active
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    mvneta2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=800bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
    	ether 00:08:a2:10:ee:39
    	hwaddr 00:08:a2:10:ee:39
    	inet6 fe80::208:a2ff:fe10:ee39%mvneta2 prefixlen 64 scopeid 0x8
    	inet 10.1.10.2 netmask 0xffffff00 broadcast 10.1.10.255
    	media: Ethernet autoselect (1000baseT <full-duplex>)
    	status: active
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    enc0: flags=0<> metric 0 mtu 1536
    	groups: enc
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    	inet6 ::1 prefixlen 128
    	inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa
    	inet 127.0.0.1 netmask 0xff000000
    	groups: lo
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    pflog0: flags=100<PROMISC> metric 0 mtu 33184
    	groups: pflog
    pfsync0: flags=0<> metric 0 mtu 1500
    	groups: pfsync
    mvneta1.4083: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=3<RXCSUM,TXCSUM>
    	ether 00:08:a2:10:ee:38
    	inet6 fe80::208:a2ff:fe10:ee38%mvneta1.4083 prefixlen 64 scopeid 0xd
    	inet 192.168.3.1 netmask 0xffffff00 broadcast 192.168.3.255
    	groups: vlan
    	vlan: 4083 vlanpcp: 0 parent interface: mvneta1
    	media: Ethernet Other <full-duplex>
    	status: active
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    mvneta1.4084: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=3<RXCSUM,TXCSUM>
    	ether 00:08:a2:10:ee:38
    	inet6 fe80::208:a2ff:fe10:ee38%mvneta1.4084 prefixlen 64 scopeid 0xe
    	inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
    	groups: vlan
    	vlan: 4084 vlanpcp: 0 parent interface: mvneta1
    	media: Ethernet Other <full-duplex>
    	status: active
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    ovpns1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    	options=80000<LINKSTATE>
    	inet6 fe80::208:a2ff:fe10:ee37%ovpns1 prefixlen 64 scopeid 0xf
    	inet 192.168.5.1 --> 192.168.5.2 netmask 0xffffff00
    	groups: tun openvpn
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	Opened by PID 25988
    

    It's not the particular VLAN ID because the same VLAN that failed to work the first time worked the 2nd and visa-versa -- the problem didn't stay w/ the VLAN#

    That "etherswitchcfg" command that you suggested is nice -- I wasn't aware of it -- it displays a lot of pertinent info in a concise, easily read format. Here it is:

    etherswitch0: VLAN mode: DOT1Q
    port1:
    	pvid: 1
    	state=8<FORWARDING>
    	flags=0<>
    	media: Ethernet autoselect (1000baseT <full-duplex>)
    	status: active
    port2:
    	pvid: 1
    	state=8<FORWARDING>
    	flags=0<>
    	media: Ethernet autoselect (1000baseT <full-duplex,master>)
    	status: active
    port3:
    	pvid: 4083
    	state=8<FORWARDING>
    	flags=0<>
    	media: Ethernet autoselect (100baseTX <full-duplex>)
    	status: active
    port4:
    	pvid: 4084
    	state=8<FORWARDING>
    	flags=0<>
    	media: Ethernet autoselect (none)
    	status: no carrier
    port5:
    	pvid: 1
    	state=8<FORWARDING>
    	flags=1<CPUPORT>
    	media: Ethernet 2500Base-KX <full-duplex>
    	status: active
    vlangroup0:
    	vlan: 1
    	members 1,2,5
    vlangroup1:
    	vlan: 4083
    	members 3,5t
    vlangroup2:
    	vlan: 4084
    	members 4,5t
    

    I think that the router engine is correctly tagging the packets that it does send to the switch (mvneta1) because packet sniffing shows that traffic destined for any of the 192.168 nets does appear on its interface and does not show on other interface; that's true both "on the wire" (sniffing from another PC) and with pfSense's own packet capture.

    To confirm, I just set up a pcap on all 3 LAN interfaces:

    • #1.naked mvneta1, ports 1-2, 192.168.1.x
    • #4.mvneta1.4083, port 3, 192.168.3.x
    • #5.mvneta1.4084, port 4, 192.168.2.x (the broken one)

    I then pinged 4 nonexistent addresses:

    • VPN pinged 192.168.3.31
    • pfSense pinged 192.168.3.32
    • VPN pinged 192.168.2.103
    • pfSense pinged 192.168.2.104

    Regardless of the source of the ping, the ARP requests each showed up on their proper VLAN interface, but not the other VLAN. They also both showed up on the "naked" interface*

    *Nice to know that the naked interface is available to sniff all VLANs simultaneously. I did confirm that a packet capture by another machine on the .1.x network (using a managed switch to mirror all of the traffic from the Netgate to the capture machine) can see ARP requests to the .1 network but nothing for .2 or .3, so nothing's inappropriately leaking out onto the wire.

    Side note: More than an hour after I last used it, the Netgate is still sending ARP requests for another junk address (*.101) at irregular intervals averaging maybe a couple of times a minute. Dunno what that's about -- I checked to make sure I didn't have a terminal still pinging it or something -- but I don't think it has any bearing on this discussion.

    Of course, for all I know, now that the device is at a different location and sitting in production, if I actually hooked a device up to the .2.x network, I might find that the previously-observed problem disappeared and it works fine now! :-/

    To find out, I stuffed the ARP cache with:

    # arp -s 192.168.2.222 22:22:22:22:22:22 temp
    

    and then pinged it from an OpenVPN client (192.168.5.2) and then from pfSense. Capture on mvneta1.4084 showed no ARP requests (besides the ones for .101 still coming after more than 2 hrs), showing that my ARP stuffing worked. It didn't show the ICMP ping requests from the VPN client, but it does show the pings originating from the Netgate itself.

    That confirms that the problem still exists; traffic originating from the Netgate goes out appropriately, but packets coming in from another interface aren't going out. If Altogether the evidence (if I'm reading it right) really seems to point to a layer 3 problem; layer 2 seems to be working fine.



  • pfinfo output (copied from Web UI):

    Status: Enabled for 1 days 10:05:33           Debug: Urgent
    
    Hostid:   0x5ce499c9
    Checksum: 0x3625e06152740b04a020d3f4d3978bf0
    
    Interface Stats for mvneta1           IPv4             IPv6
      Bytes In                      1345484027         37139834
      Bytes Out                     6722315697                0
      Packets In
        Passed                         5628568                0
        Blocked                          84230           129353
      Packets Out
        Passed                         7373259                0
        Blocked                              0                0
    
    State Table                          Total             Rate
      current entries                      568               
      searches                        27999438          228.1/s
      inserts                           725104            5.9/s
      removals                          724536            5.9/s
    Source Tracking Table
      current entries                        0               
      searches                               0            0.0/s
      inserts                                0            0.0/s
      removals                               0            0.0/s
    Counters
      match                            1038601            8.5/s
      bad-offset                             0            0.0/s
      fragment                               0            0.0/s
      short                                  0            0.0/s
      normalize                              0            0.0/s
      memory                                 0            0.0/s
      bad-timestamp                          0            0.0/s
      congestion                             0            0.0/s
      ip-option                           6335            0.1/s
      proto-cksum                            0            0.0/s
      state-mismatch                      1985            0.0/s
      state-insert                           0            0.0/s
      state-limit                            0            0.0/s
      src-limit                              0            0.0/s
      synproxy                               0            0.0/s
      map-failed                             0            0.0/s
    Limit Counters
      max states per rule                    0            0.0/s
      max-src-states                         0            0.0/s
      max-src-nodes                          0            0.0/s
      max-src-conn                           0            0.0/s
      max-src-conn-rate                      0            0.0/s
      overload table insertion               0            0.0/s
      overload flush states                  0            0.0/s
    
    states        hard limit   202000
    src-nodes     hard limit   202000
    frags         hard limit     5000
    table-entries hard limit   400000
    
    tcp.first                   120s
    tcp.opening                  30s
    tcp.established           86400s
    tcp.closing                 900s
    tcp.finwait                  45s
    tcp.closed                   90s
    tcp.tsdiff                   30s
    udp.first                    60s
    udp.single                   30s
    udp.multiple                 60s
    icmp.first                   20s
    icmp.error                   10s
    other.first                  60s
    other.single                 30s
    other.multiple               60s
    frag                         30s
    interval                     10s
    adaptive.start           121200 states
    adaptive.end             242400 states
    src.track                     0s
    
    all
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  2                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    enc
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    enc0
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    lo
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    lo0
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  4                 
    	In4/Pass:    [ Packets: 213520             Bytes: 24488193           ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 213576             Bytes: 24495544           ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 3788               Bytes: 954580             ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 3788               Bytes: 954580             ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    mvneta0
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    mvneta1
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  40                
    	In4/Pass:    [ Packets: 5628571            Bytes: 1338740152         ]
    	In4/Block:   [ Packets: 84230              Bytes: 6745249            ]
    	Out4/Pass:   [ Packets: 7373259            Bytes: 6722315697         ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 129353             Bytes: 37139834           ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    mvneta1.4083
    	Cleared:     Fri Apr 17 08:47:48 2020
    	References:  18                
    	In4/Pass:    [ Packets: 206914             Bytes: 20162180           ]
    	In4/Block:   [ Packets: 293                Bytes: 20000              ]
    	Out4/Pass:   [ Packets: 398978             Bytes: 517612077          ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 239                Bytes: 66541              ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    mvneta1.4084
    	Cleared:     Fri Apr 17 08:47:48 2020
    	References:  16                
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 6871               Bytes: 731862             ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    mvneta2
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  43                
    	In4/Pass:    [ Packets: 7906550            Bytes: 7247828665         ]
    	In4/Block:   [ Packets: 67106              Bytes: 7946780            ]
    	Out4/Pass:   [ Packets: 5526528            Bytes: 1318929177         ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 40821              Bytes: 7180189            ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    openvpn
    	Cleared:     Fri Apr 17 08:47:56 2020
    	References:  8                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    ovpns1
    	Cleared:     Fri Apr 17 08:47:56 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 24124              Bytes: 1983411            ]
    	In4/Block:   [ Packets: 10                 Bytes: 520                ]
    	Out4/Pass:   [ Packets: 31686              Bytes: 19154147           ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 11                 Bytes: 1048               ]
    pflog
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    pflog0
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    pfsync
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    pfsync0 (skip)
    	Cleared:     Fri Apr 17 08:47:57 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    tun
    	Cleared:     Fri Apr 17 08:47:56 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    tun1
    	Cleared:     Fri Apr 17 08:47:56 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    vlan
    	Cleared:     Fri Apr 17 08:47:48 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    vlan0
    	Cleared:     Fri Apr 17 08:47:48 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    vlan1
    	Cleared:     Fri Apr 17 08:47:48 2020
    	References:  0                 
    	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
    	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
    	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
    	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
    

  • LAYER 8 Netgate

    That configuration looks correct.

    This is a little tl;dr for me.

    Narrow it down for me, please.

    One thing - specifically and in as much detail as possible (ip addresses not generic terms, etc) - that does not work.

    Thank you.



  • pftop output (copied from web UI) filtering on ICMP and showing pings going to 2.222 from pfSense and from a VPN client. (the pings from pfSense would work if there was something there to talk to).:

    pfTop: Up State 1-4/4 (614), View: default, Order: bytes
    PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
    icmp      Out 10.1.10.2:1487                10.1.10.1:1487                          0:0            33:38:42  00:00:09   474338 13281464
    icmp      In  192.168.5.2:13206             192.168.2.222:13206                     0:0            00:07:42  00:00:10      452    37968
    icmp      Out 192.168.5.2:13206             192.168.2.222:13206                     0:0            00:07:42  00:00:10      452    37968
    icmp      Out 192.168.2.1:47732             192.168.2.222:47732                     0:0            00:00:30  00:00:09       30     2520
    

    Here it is w/ filter "net 192.168.2.0/24" without me doing anything:

    code_textpfTop: Up State 1-2/2 (514), View: default, Order: bytes
    PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
    udp       In  192.168.1.3:61633             192.168.2.101:161              NO_TRAFFIC:SINGLE       00:00:12  00:00:29        2      214
    udp       Out 192.168.1.3:61633             192.168.2.101:161                  SINGLE:NO_TRAFFIC   00:00:12  00:00:29        2      214
    

    Well lookie there, the cause of those lingering ARP queries! .1.3 is a Windows server; I guess it somehow heard that there was supposed to be a 2.101 somewhere and decided to start sending it SNMP queries? Go figure. Getting back on track...

    Here's a couple of snapshots using the same "net 192.168.2.0/24" filter (I manually removed the .101 distractor, but it's otherwise untampered) showing me simultaneously trying to open a TCP connection to .2.222 from both the Netgate and the VPN client:

    code_textpfTop: Up State 1-5/5 (509), View: default, Order: bytes
    PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
    tcp       In  192.168.5.2:34340             192.168.2.222:22222                CLOSED:SYN_SENT     00:00:04  00:00:29        3      180
    tcp       Out 192.168.5.2:34340             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:04  00:00:29        3      180
    tcp       Out 192.168.2.1:39581             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:04  00:00:29        2      120
    
    
    pfTop: Up State 1-5/5 (854), View: default, Order: bytes
    PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
    tcp       Out 192.168.2.1:39581             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:24  00:00:28        7      420
    tcp       In  192.168.5.2:34340             192.168.2.222:22222                CLOSED:SYN_SENT     00:00:24  00:00:21        5      300
    tcp       Out 192.168.5.2:34340             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:24  00:00:21        5      300
    

    I don't see anything wrong at that point -- it seems to claim that it's at least planning to forward the SYN from the VPN client. Any more ideas on how to get a closer peek at where it's being dropped?


Log in to reply