Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-3100: 2nd new virt interface fails

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    9 Posts 3 Posters 459 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      cboling
      last edited by

      Review: SG-3100 comes w/ 3 interfaces:

      • 0: OPT
      • 1: LAN (4-port switch)
      • 2: WAN

      I'm using the OPT port for a 2nd WAN (deactivated for now). I'm keeping LAN ports 1 & 2 as-is, but making 3 & 4 a couple of different internal networks. I've [physically] labeled the ports thus:

      • WAN: WAN1
      • OPT: WAN2
      • LAN1: LAN - 192.168.1.x
      • LAN2: LAN - 192.168.1.x
      • LAN3: Guest/Wi-Fi - 192.168.3.x
      • LAN4: DMZ - 192.168.2.x

      Using the instructions in Configuring the Switch Ports ("This optional guide shows the steps required to configure the 4 switched Ethernet ports as discrete ports."), I added the VLANs for ports 3 & 4.

      The first time I configured it, I went through all the steps (in order) and set up port 4, then I went back and did the same for port 3. Port 4 worked fine. Port 3 could communicate with its subnet, but nothing else; e.g. I could reach 192.168.3.1 but not .2.1, .1.1, or anything on the WAN. Pings to WAN destinations showed responses coming back to the router, but not being passed back out port 3.

      I checked everything I could think of, finally deleted everything associated with port 3 & recreating it, no dice. (Rebooted a couple of times during testing, just to see if it helped.) Finally, I deleted interfaces for both 3 & 4, and set VLANs back to the default (everything on 1). Couldn't get a DHCP lease (was still attached to port 3) but after reboot it worked fine; I verified that all 4 ports worked correctly.

      I repeated my setup of ports 3 & 4, this time reversing the order (doing 3 then 4 instead of doing 4 then 3) and doing them both at the same time (step 1 for 3 & 4, then step 2...) instead of completing all steps for one port before moving to the other as I did before.

      Imagine my surprise when, this time, port 3 worked, and port 4 didn't! The first port to be set up each time worked, the 2nd one didn't. I know that only 2 coin tosses landing on heads isn't a very strong suggestion that there's something wrong w/ tails, but I haven't found any other link yet. (it's not the particular port, VLAN#, etc.)

      I created a single firewall rule on each interface by copying the default "allow all" rule from LAN & changing the interface in 2 places (where the rule applies and what net to allow traffic from).

      Would y'all take a peek at my screenshots here and see if you spot anything funny? I went over them, and can't tell any difference that would make one work and not the other.
      Selection_001.png
      Selection_002.png
      Selection_003.png
      Selection_004.png
      Selection_005.png
      Selection_006.png
      Selection_007.png
      It's gotta be something stupid, but I can't seem to find it. I'm happy to provide any other info that may be helpful -- just ask!

      Thank you.

      C 1 Reply Last reply Reply Quote 0
      • C
        cboling @cboling
        last edited by

        This is somewhat redundant since I already knew that .2.* devices could talk to .2.1 but not any of the router's other addresses, but...
        I configured an OpenVpn server on the Netgate, and connected to it from the outside. On that connection, I can talk to any of the Netgate's IP addresses, and I can talk to a device on the port 3 network (192.168.3.50) but not the same device on the problem port 4 network (192.168.2.50). This again illustrates that the problem is not limited to communication between the problem port and WAN addresses; it prevents talking to VPN tunnel devices too, i.e. the router will send traffic out port 4 (to 192.168.2.* devices) that originates from 192.168.2.1, but it will not forward packets from any other interface.

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          Can we see the routing table?

          Maybe the firewall states when you are trying to connect out from port 4?

          What you have configured there looks correct. It must be something we can't see like a subnet problem or a gateway on the wrong interface...

          Steve

          C 1 Reply Last reply Reply Quote 0
          • C
            cboling @stephenw10
            last edited by cboling

            @stephenw10 Thanks for the suggestions, Steve.
            Here's the IP4 routing table via netstat -rn:

            Destination        Gateway            Flags     Netif Expire
            default            10.1.10.1          UGS     mvneta2
            10.1.10.0/24       link#8             U       mvneta2
            10.1.10.2          link#8             UHS         lo0
            127.0.0.1          link#10            UH          lo0
            192.168.1.0/24     link#2             U       mvneta1
            192.168.1.1        link#2             UHS         lo0
            192.168.2.0/24     link#14            U      mvneta1.
            192.168.2.1        link#14            UHS         lo0
            192.168.3.0/24     link#13            U      mvneta1.
            192.168.3.1        link#13            UHS         lo0
            192.168.5.0/24     192.168.5.2        UGS      ovpns1
            192.168.5.1        link#15            UHS         lo0
            192.168.5.2        link#15            UH       ovpns1
            

            Hmmm, looking at firewall states would've been smart...
            Unfortunately, I had to install the router today as-is (they had failing hardware that was causing extreme trouble) and I was under time pressure and didn't think about setting up a test PC on-site. (Oops!) Pings to the router interface work fine since they're not actually leaving/coming in, so that's not a test source. My contact is neither technical nor terribly motivated right now, so it may take me a while to set something up, since it's in another city and I don't expect to be there again soon. (Today's the 1st time I've left home in 6 weeks.)

            In the meantime, anything else that I ought to show that doesn't require traffic?

            Edit: I just realized that since the problem isn't packets coming into that interface, but rather that packets from other nets won't go out on it, I may be able to get a valid state snapshot even without a test machine -- if I can convince it try even though the Ethernet port's not connected to anything. If I can get it to make an honest attempt to send something from the router itself (which was working) -- maybe I'll have have to somehow stuff a fake entry in the ARP table or something, I don't know; I'm getting goofy and probably shouldn't even be typing until I've slept on it -- but if that'll produce results, then I ought to be able to do the same thing from another interface to see the difference. I'll revisit it tomorrow.

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              Ok, you seem to have interfaces marked mvneta1. instead of mvneta1.4083 and mvneta1.4084.

              Check the output of ifconfig and of etherswitchcfg.

              Try using some other VLAN ID such as 20, something much lower.

              If it really is failing to create those VLANs there should be errors in the system log.

              Steve

              1 Reply Last reply Reply Quote 0
              • C
                cboling
                last edited by

                The "mvneta1." thing is just netstat truncating the column's displayed width; it would look fine if "mvneta1" was named something short like "en1".
                ifconfig:

                code_textmvneta0: flags=8b02<BROADCAST,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
                	options=800bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
                	ether 00:08:a2:10:ee:37
                	hwaddr 00:08:a2:10:ee:37
                	media: Ethernet autoselect (none)
                	status: no carrier
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                mvneta1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
                	options=bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM>
                	ether 00:08:a2:10:ee:38
                	hwaddr 00:08:a2:10:ee:38
                	inet6 fe80::208:a2ff:fe10:ee38%mvneta1 prefixlen 64 scopeid 0x2
                	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
                	media: Ethernet 2500Base-KX <full-duplex>
                	status: active
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                mvneta2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
                	options=800bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
                	ether 00:08:a2:10:ee:39
                	hwaddr 00:08:a2:10:ee:39
                	inet6 fe80::208:a2ff:fe10:ee39%mvneta2 prefixlen 64 scopeid 0x8
                	inet 10.1.10.2 netmask 0xffffff00 broadcast 10.1.10.255
                	media: Ethernet autoselect (1000baseT <full-duplex>)
                	status: active
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                enc0: flags=0<> metric 0 mtu 1536
                	groups: enc
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
                	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
                	inet6 ::1 prefixlen 128
                	inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa
                	inet 127.0.0.1 netmask 0xff000000
                	groups: lo
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                pflog0: flags=100<PROMISC> metric 0 mtu 33184
                	groups: pflog
                pfsync0: flags=0<> metric 0 mtu 1500
                	groups: pfsync
                mvneta1.4083: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
                	options=3<RXCSUM,TXCSUM>
                	ether 00:08:a2:10:ee:38
                	inet6 fe80::208:a2ff:fe10:ee38%mvneta1.4083 prefixlen 64 scopeid 0xd
                	inet 192.168.3.1 netmask 0xffffff00 broadcast 192.168.3.255
                	groups: vlan
                	vlan: 4083 vlanpcp: 0 parent interface: mvneta1
                	media: Ethernet Other <full-duplex>
                	status: active
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                mvneta1.4084: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
                	options=3<RXCSUM,TXCSUM>
                	ether 00:08:a2:10:ee:38
                	inet6 fe80::208:a2ff:fe10:ee38%mvneta1.4084 prefixlen 64 scopeid 0xe
                	inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
                	groups: vlan
                	vlan: 4084 vlanpcp: 0 parent interface: mvneta1
                	media: Ethernet Other <full-duplex>
                	status: active
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                ovpns1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
                	options=80000<LINKSTATE>
                	inet6 fe80::208:a2ff:fe10:ee37%ovpns1 prefixlen 64 scopeid 0xf
                	inet 192.168.5.1 --> 192.168.5.2 netmask 0xffffff00
                	groups: tun openvpn
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                	Opened by PID 25988
                

                It's not the particular VLAN ID because the same VLAN that failed to work the first time worked the 2nd and visa-versa -- the problem didn't stay w/ the VLAN#

                That "etherswitchcfg" command that you suggested is nice -- I wasn't aware of it -- it displays a lot of pertinent info in a concise, easily read format. Here it is:

                etherswitch0: VLAN mode: DOT1Q
                port1:
                	pvid: 1
                	state=8<FORWARDING>
                	flags=0<>
                	media: Ethernet autoselect (1000baseT <full-duplex>)
                	status: active
                port2:
                	pvid: 1
                	state=8<FORWARDING>
                	flags=0<>
                	media: Ethernet autoselect (1000baseT <full-duplex,master>)
                	status: active
                port3:
                	pvid: 4083
                	state=8<FORWARDING>
                	flags=0<>
                	media: Ethernet autoselect (100baseTX <full-duplex>)
                	status: active
                port4:
                	pvid: 4084
                	state=8<FORWARDING>
                	flags=0<>
                	media: Ethernet autoselect (none)
                	status: no carrier
                port5:
                	pvid: 1
                	state=8<FORWARDING>
                	flags=1<CPUPORT>
                	media: Ethernet 2500Base-KX <full-duplex>
                	status: active
                vlangroup0:
                	vlan: 1
                	members 1,2,5
                vlangroup1:
                	vlan: 4083
                	members 3,5t
                vlangroup2:
                	vlan: 4084
                	members 4,5t
                

                I think that the router engine is correctly tagging the packets that it does send to the switch (mvneta1) because packet sniffing shows that traffic destined for any of the 192.168 nets does appear on its interface and does not show on other interface; that's true both "on the wire" (sniffing from another PC) and with pfSense's own packet capture.

                To confirm, I just set up a pcap on all 3 LAN interfaces:

                • #1.naked mvneta1, ports 1-2, 192.168.1.x
                • #4.mvneta1.4083, port 3, 192.168.3.x
                • #5.mvneta1.4084, port 4, 192.168.2.x (the broken one)

                I then pinged 4 nonexistent addresses:

                • VPN pinged 192.168.3.31
                • pfSense pinged 192.168.3.32
                • VPN pinged 192.168.2.103
                • pfSense pinged 192.168.2.104

                Regardless of the source of the ping, the ARP requests each showed up on their proper VLAN interface, but not the other VLAN. They also both showed up on the "naked" interface*

                *Nice to know that the naked interface is available to sniff all VLANs simultaneously. I did confirm that a packet capture by another machine on the .1.x network (using a managed switch to mirror all of the traffic from the Netgate to the capture machine) can see ARP requests to the .1 network but nothing for .2 or .3, so nothing's inappropriately leaking out onto the wire.

                Side note: More than an hour after I last used it, the Netgate is still sending ARP requests for another junk address (*.101) at irregular intervals averaging maybe a couple of times a minute. Dunno what that's about -- I checked to make sure I didn't have a terminal still pinging it or something -- but I don't think it has any bearing on this discussion.

                Of course, for all I know, now that the device is at a different location and sitting in production, if I actually hooked a device up to the .2.x network, I might find that the previously-observed problem disappeared and it works fine now! :-/

                To find out, I stuffed the ARP cache with:

                # arp -s 192.168.2.222 22:22:22:22:22:22 temp
                

                and then pinged it from an OpenVPN client (192.168.5.2) and then from pfSense. Capture on mvneta1.4084 showed no ARP requests (besides the ones for .101 still coming after more than 2 hrs), showing that my ARP stuffing worked. It didn't show the ICMP ping requests from the VPN client, but it does show the pings originating from the Netgate itself.

                That confirms that the problem still exists; traffic originating from the Netgate goes out appropriately, but packets coming in from another interface aren't going out. If Altogether the evidence (if I'm reading it right) really seems to point to a layer 3 problem; layer 2 seems to be working fine.

                C 1 Reply Last reply Reply Quote 0
                • C
                  cboling @cboling
                  last edited by

                  pfinfo output (copied from Web UI):

                  Status: Enabled for 1 days 10:05:33           Debug: Urgent
                  
                  Hostid:   0x5ce499c9
                  Checksum: 0x3625e06152740b04a020d3f4d3978bf0
                  
                  Interface Stats for mvneta1           IPv4             IPv6
                    Bytes In                      1345484027         37139834
                    Bytes Out                     6722315697                0
                    Packets In
                      Passed                         5628568                0
                      Blocked                          84230           129353
                    Packets Out
                      Passed                         7373259                0
                      Blocked                              0                0
                  
                  State Table                          Total             Rate
                    current entries                      568               
                    searches                        27999438          228.1/s
                    inserts                           725104            5.9/s
                    removals                          724536            5.9/s
                  Source Tracking Table
                    current entries                        0               
                    searches                               0            0.0/s
                    inserts                                0            0.0/s
                    removals                               0            0.0/s
                  Counters
                    match                            1038601            8.5/s
                    bad-offset                             0            0.0/s
                    fragment                               0            0.0/s
                    short                                  0            0.0/s
                    normalize                              0            0.0/s
                    memory                                 0            0.0/s
                    bad-timestamp                          0            0.0/s
                    congestion                             0            0.0/s
                    ip-option                           6335            0.1/s
                    proto-cksum                            0            0.0/s
                    state-mismatch                      1985            0.0/s
                    state-insert                           0            0.0/s
                    state-limit                            0            0.0/s
                    src-limit                              0            0.0/s
                    synproxy                               0            0.0/s
                    map-failed                             0            0.0/s
                  Limit Counters
                    max states per rule                    0            0.0/s
                    max-src-states                         0            0.0/s
                    max-src-nodes                          0            0.0/s
                    max-src-conn                           0            0.0/s
                    max-src-conn-rate                      0            0.0/s
                    overload table insertion               0            0.0/s
                    overload flush states                  0            0.0/s
                  
                  states        hard limit   202000
                  src-nodes     hard limit   202000
                  frags         hard limit     5000
                  table-entries hard limit   400000
                  
                  tcp.first                   120s
                  tcp.opening                  30s
                  tcp.established           86400s
                  tcp.closing                 900s
                  tcp.finwait                  45s
                  tcp.closed                   90s
                  tcp.tsdiff                   30s
                  udp.first                    60s
                  udp.single                   30s
                  udp.multiple                 60s
                  icmp.first                   20s
                  icmp.error                   10s
                  other.first                  60s
                  other.single                 30s
                  other.multiple               60s
                  frag                         30s
                  interval                     10s
                  adaptive.start           121200 states
                  adaptive.end             242400 states
                  src.track                     0s
                  
                  all
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  2                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  enc
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  enc0
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  lo
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  lo0
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  4                 
                  	In4/Pass:    [ Packets: 213520             Bytes: 24488193           ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 213576             Bytes: 24495544           ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 3788               Bytes: 954580             ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 3788               Bytes: 954580             ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  mvneta0
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  mvneta1
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  40                
                  	In4/Pass:    [ Packets: 5628571            Bytes: 1338740152         ]
                  	In4/Block:   [ Packets: 84230              Bytes: 6745249            ]
                  	Out4/Pass:   [ Packets: 7373259            Bytes: 6722315697         ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 129353             Bytes: 37139834           ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  mvneta1.4083
                  	Cleared:     Fri Apr 17 08:47:48 2020
                  	References:  18                
                  	In4/Pass:    [ Packets: 206914             Bytes: 20162180           ]
                  	In4/Block:   [ Packets: 293                Bytes: 20000              ]
                  	Out4/Pass:   [ Packets: 398978             Bytes: 517612077          ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 239                Bytes: 66541              ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  mvneta1.4084
                  	Cleared:     Fri Apr 17 08:47:48 2020
                  	References:  16                
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 6871               Bytes: 731862             ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  mvneta2
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  43                
                  	In4/Pass:    [ Packets: 7906550            Bytes: 7247828665         ]
                  	In4/Block:   [ Packets: 67106              Bytes: 7946780            ]
                  	Out4/Pass:   [ Packets: 5526528            Bytes: 1318929177         ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 40821              Bytes: 7180189            ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  openvpn
                  	Cleared:     Fri Apr 17 08:47:56 2020
                  	References:  8                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  ovpns1
                  	Cleared:     Fri Apr 17 08:47:56 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 24124              Bytes: 1983411            ]
                  	In4/Block:   [ Packets: 10                 Bytes: 520                ]
                  	Out4/Pass:   [ Packets: 31686              Bytes: 19154147           ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 11                 Bytes: 1048               ]
                  pflog
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  pflog0
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  pfsync
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  pfsync0 (skip)
                  	Cleared:     Fri Apr 17 08:47:57 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  tun
                  	Cleared:     Fri Apr 17 08:47:56 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  tun1
                  	Cleared:     Fri Apr 17 08:47:56 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  vlan
                  	Cleared:     Fri Apr 17 08:47:48 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  vlan0
                  	Cleared:     Fri Apr 17 08:47:48 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  vlan1
                  	Cleared:     Fri Apr 17 08:47:48 2020
                  	References:  0                 
                  	In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In4/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out4/Block:  [ Packets: 0                  Bytes: 0                  ]
                  	In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
                  	In6/Block:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
                  	Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
                  
                  C 1 Reply Last reply Reply Quote 0
                  • DerelictD
                    Derelict LAYER 8 Netgate
                    last edited by

                    That configuration looks correct.

                    This is a little tl;dr for me.

                    Narrow it down for me, please.

                    One thing - specifically and in as much detail as possible (ip addresses not generic terms, etc) - that does not work.

                    Thank you.

                    Chattanooga, Tennessee, USA
                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                    1 Reply Last reply Reply Quote 0
                    • C
                      cboling @cboling
                      last edited by

                      pftop output (copied from web UI) filtering on ICMP and showing pings going to 2.222 from pfSense and from a VPN client. (the pings from pfSense would work if there was something there to talk to).:

                      pfTop: Up State 1-4/4 (614), View: default, Order: bytes
                      PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
                      icmp      Out 10.1.10.2:1487                10.1.10.1:1487                          0:0            33:38:42  00:00:09   474338 13281464
                      icmp      In  192.168.5.2:13206             192.168.2.222:13206                     0:0            00:07:42  00:00:10      452    37968
                      icmp      Out 192.168.5.2:13206             192.168.2.222:13206                     0:0            00:07:42  00:00:10      452    37968
                      icmp      Out 192.168.2.1:47732             192.168.2.222:47732                     0:0            00:00:30  00:00:09       30     2520
                      

                      Here it is w/ filter "net 192.168.2.0/24" without me doing anything:

                      code_textpfTop: Up State 1-2/2 (514), View: default, Order: bytes
                      PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
                      udp       In  192.168.1.3:61633             192.168.2.101:161              NO_TRAFFIC:SINGLE       00:00:12  00:00:29        2      214
                      udp       Out 192.168.1.3:61633             192.168.2.101:161                  SINGLE:NO_TRAFFIC   00:00:12  00:00:29        2      214
                      

                      Well lookie there, the cause of those lingering ARP queries! .1.3 is a Windows server; I guess it somehow heard that there was supposed to be a 2.101 somewhere and decided to start sending it SNMP queries? Go figure. Getting back on track...

                      Here's a couple of snapshots using the same "net 192.168.2.0/24" filter (I manually removed the .101 distractor, but it's otherwise untampered) showing me simultaneously trying to open a TCP connection to .2.222 from both the Netgate and the VPN client:

                      code_textpfTop: Up State 1-5/5 (509), View: default, Order: bytes
                      PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
                      tcp       In  192.168.5.2:34340             192.168.2.222:22222                CLOSED:SYN_SENT     00:00:04  00:00:29        3      180
                      tcp       Out 192.168.5.2:34340             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:04  00:00:29        3      180
                      tcp       Out 192.168.2.1:39581             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:04  00:00:29        2      120
                      
                      
                      pfTop: Up State 1-5/5 (854), View: default, Order: bytes
                      PR        DIR SRC                           DEST                                   STATE                AGE       EXP     PKTS    BYTES
                      tcp       Out 192.168.2.1:39581             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:24  00:00:28        7      420
                      tcp       In  192.168.5.2:34340             192.168.2.222:22222                CLOSED:SYN_SENT     00:00:24  00:00:21        5      300
                      tcp       Out 192.168.5.2:34340             192.168.2.222:22222              SYN_SENT:CLOSED       00:00:24  00:00:21        5      300
                      

                      I don't see anything wrong at that point -- it seems to claim that it's at least planning to forward the SYN from the VPN client. Any more ideas on how to get a closer peek at where it's being dropped?

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post
                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.