Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Multiple interfaces, VLAN switching

    Scheduled Pinned Locked Moved L2/Switching/VLANs
    13 Posts 4 Posters 1.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      myradon
      last edited by myradon

      Hi guys,

      Recently I got hold of 2 network switches supporting vlan (HPE 1820 24G PoE and HP 1810-24G). I'm using pfsense for several years. Even started with it's origin m0n0wall. I want to automate my house (Home Assistant, Zigbee lights, lightswitches) so isolating IOT-stuff is no luxury. Also a good time to put my dev-webserver in DMZ-subnet.

      I get so strange behaviour when pinging interfaces. Some hosts are reachable, some are not. Actually this quest started when docker container on host htpc couldn't be reached from other subnets. VM Guest in subnet is does ICMP-reply but it's host doesn't. I erased all my rules and set some simple loose "any" rules.

      Network diagram underneath is simplified (actually 2 switches and more hosts. Switches are connected with 2 tagged ports for all 3 vlans);
      home network

      All hosts get DHCP assigned IP. Bridge members don't have any networking rules And System Tunables;

      net.link.bridge.pfil_bridge Packet filter on the bridge interface 1
      net.link.bridge.pfil_member Packet filter on the member interface 0

      For testing purposes firewall rules allow everything;



      Now is the funny part (not).

      • iPhone (wlanguest interface) can ping hosts in other subnets;
        -- Debian VM, Hackintosh
        -- Mini-itx-bak only on vlan-lan (ID 130)
        -- Htpc only one vlan-lan (ID 130)

      • Hackintosh (bridgelan interface) can ping Htpc;
        -- only on vlan-lan (ID 130, obvious same subnet so no routing),
        -- vlan-iot (ID 132) is timeout
        -- vlan-dmz (ID 133) is timeout

      • Hackintosh (bridgelan interface) can ping Debian VM (NIC configured vlan-dmz on host Htpc);

      • Hackintosh (bridgelan interface) can ping Mini-itx-bak;
        -- only on vlan-lan (ID 130, obvious same subnet so no routing),
        -- vlan-iot (ID 132) is timeout
        -- vlan-dmz (ID 133) is timeout

      • Mini-itx-bak (tagged vlans 130,132,133) CAN ping Hackintosh (untagged 130);
        -- From vlan-lan (ID 130, obvious same subnet so no routing),
        -- From vlan-iot (ID 132), brought vlan-lan and vlan-dmz down first to know it's routed by pfsense.
        -- No go from vlan-dmz (ID 133) because; tried to obtain new lease from DHCP-server (previously released lease). ("sudo dhclient -v lan-dmz"); endless DHCP-discover. (Mac is in DHCP static maps)

      • Mini-itx-bak (tagged vlans 130,132,133) CAN ping Debian VM(NIC configured vlan-dmz on host Htpc);
        -- From vlan-lan (ID 130),
        -- From vlan-iot (ID 132), brought vlan-lan down first to know it's routed by pfsense (checked with mtr).

      • Mini-itx-bak (tagged vlans 130,132,133) CAN ping Htpc (tagged 130,132,133);
        -- IP (ID 130),
        -- IP (ID 132)
        -- IP (ID 133) is timeout (rember Guest Debian VM does reply)

      I don't see any logic. Is anybody able to elaborate this one?

      EDIT: changed diagram

      1 Reply Last reply Reply Quote 0
      • johnpozJ
        johnpoz LAYER 8 Global Moderator
        last edited by johnpoz

        Why are you calling what I hope is just pfsense IP on that vlan gateway? You didn't set gateways on these interfaces did you?

        Not sure what you think your accomplishing by pretty much every box on multiple vlans.. Why do you have hosts in 3 different vlans?

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        M 1 Reply Last reply Reply Quote 0
        • M
          myradon @johnpoz
          last edited by myradon

          @johnpoz said in Multiple interfaces, VLAN switching:

          Why are you calling what I hope is just pfsense IP on that vlan gateway?

          Which vlan, which call do you refer to? In opening post I don't see an example of pinging a gateway.

          You didn't set gateways on these interfaces did you?

          I didn't assign any gateways on the (bridge)interfaces. All interfaces 'IPv4 Upstream gateway' set to default 'none'

          Not sure what you think your accomplishing by pretty much every box on multiple vlans.. Why do you have hosts in 3 different vlans?

          Basically host 'mini-itx-bak' is a temporary system to play with multiple vlan-interfaces on a host. I don't want to mess up host Htpc. Both system are running on Ubuntu 18.04 LTS by the way. In the future host Htpc will have all 3 vlan-interfaces; 1 for lan-subnet as the fileserver, 1 to talk to iot-devices + running Home Assistant and 1 for webserver in dmz-subnet (VM guest).

          Does VM host need an IP to get VM Guest assigned an IP? Probably not right. I see with DHCP-discover broadcast to mac ff:ff:ff:ff:ff:ff. So it's a Layer-2 thingy if I'm not mistaking. Then I could ditch 1 IP (htpc, vlan-dmz, id: 133, 192.168.133.132). I ditched 1 IP; htpc, vlan-dmz, id: 133, 192.168.133.132. In the future when I got vlans running like I would like to, host mini-itx-bak will be gone.

          EDIT: I think something is wrong with Ubuntu's new Netplan network configuration. Luckily MacOS also supports VLAN tagging. I've add all 3 vlans to host Hacknitosh and configured switch accordingly. I'm able to ping from iPad in all wifi-subnets to all IP assigned to MacOS vlans. Fropm every vlan I'm able to connect to webserver in DMZ.

          1 Reply Last reply Reply Quote 0
          • M
            myradon
            last edited by

            Okay other thought; why would a vlan-interface reply in subnet and not routed? To it means; vlan is configured correctly on host and switch-port tagging is correct otherwise host don't even respond.

            Here is config and some output;

            raymond@htpc:~$ cat /etc/netplan/config.yaml
            
            network:
              version: 2
              renderer: networkd
              ethernets: 
                enp1s0f0:
                  dhcp4: true
                  optional: true
                enp1s0f1:
                  match:
                    macaddress: "00:26:55:da:9f:9b"
                  set-name: enp1s0f1
                  dhcp4: false
              vlans:
                vlan-lan:
                  id: 130
                  link: enp1s0f1
                  dhcp4: true
                  dhcp4-overrides:
                    route-metric: 100
                vlan-iot:
                  id: 132
                  link: enp1s0f1
                  dhcp4: true
                  dhcp4-overrides:
                    route-metric: 200
                vlan-dmz:
                  id: 133
                  link: enp1s0f1
                  dhcp4: false
            
            raymond@htpc:~$ networkctl
            IDX LINK             TYPE               OPERATIONAL SETUP     
              1 lo               loopback           carrier     unmanaged 
              2 enp1s0f0         ether              no-carrier  configuring
              3 enp1s0f1         ether              degraded    configured
              4 vlan-lan         ether              routable    configured
              5 vlan-iot         ether              routable    configured
              6 vlan-dmz         ether              degraded    
            
            raymond@htpc:~$ networkctl status
            State: routable
                   Address: 192.168.130.133 on vlan-lan
                            192.168.132.133 on vlan-iot
                            169.254.9.221 on enp1s0f0
                            169.254.9.226 on enp1s0f1
                            fe80::226:55ff:feda:9f9b on enp1s0f1
                            fe80::226:55ff:feda:9f9b on vlan-lan
                            fe80::226:55ff:feda:9f9b on vlan-iot
                            fe80::226:55ff:feda:9f9b on vlan-dmz
                   Gateway: 192.168.130.129 on vlan-lan
                            192.168.132.129 on vlan-iot
                       DNS: 192.168.130.129
                            192.168.132.129
            
            JKnottJ 1 Reply Last reply Reply Quote 0
            • JKnottJ
              JKnott @myradon
              last edited by

              @myradon said in Multiple interfaces, VLAN switching:

              To it means; vlan is configured correctly on host and switch-port tagging is correct otherwise host don't even respond.

              Are you enabling VLANs on the devices and then connecting them to a port that's already been assigned to a VLAN? You do one or the other. If you have a managed switch, you configure the port for the VLAN and let the connected device run native LAN. The only time you'd tag a device is if you're connected to an unmanaged switch or running tagged and untagged devices over the same cable. An example of this would be when a computer is plugged into a VoIP phone and then the phone is connected to the switch.

              PfSense running on Qotom mini PC
              i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
              UniFi AC-Lite access point

              I haven't lost my mind. It's around here...somewhere...

              M 1 Reply Last reply Reply Quote 0
              • M
                myradon @JKnott
                last edited by myradon

                @JKnott

                • I've setup 2 hosts with both 3 tagged vlans (ID's 130,132,133) .
                • The switch is configured accordingly (ports tagged id 130,132,133). Also both ports on the switch aren't configured untagged also exlcuded from vlan 1.
                • In my previous post you see output from commands (host htpc); the vlan-interfaces have obtained an IP-lease from Pfsense.
                • VM Guest (Debian) running on Virtualbox on interface "vlan-dmz" will get an IP from Pfsense And does ICMP-reply from all subnets.

                What could be the reason I can't ping host htpc on vlan-iot? It gets it's lease from pfsense. Within subnet it does reply. So to me it sounds like a routing-thing. But I don't understand what's wrong.

                1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator
                  last edited by johnpoz

                  @myradon said in Multiple interfaces, VLAN switching:

                  Gateway: 192.168.130.129 on vlan-lan
                  192.168.132.129 on vlan-iot

                  So which gateway does it use, where you you have your state set.. If you want to multi home boxes... Then your going to run into all kinds of asymmetrical routing issues if your not careful..

                  There is one thing tagging vlans down to a vm host to use for vm's that are on only 1 vlan.. There is another when you start multihoming devices and have them setup with multiple gateways, etc.

                  My point of what your calling gateways.. Don't label it like that!!! That looks like you have a gateway set on pfsense.. Is it pointing to itself, is it pointing to a different IP? Its not a gateway on pfsense, its its IP address - it could be a gateway to something on that network... But you have it listed as gateway on the pfsense box.. Which is confusing to what you actually mean.

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  M 1 Reply Last reply Reply Quote 0
                  • M
                    myradon @johnpoz
                    last edited by

                    Okay I'll change diagram in OP. To it's host in the subnets it's their gateway. I thought the Pfsense-interface is then called the gateway. My bad. Anyway new revision of diagram. Hopefully it's now more clear to you networking guys.

                    diagram v3

                    A clue what's going on?

                    1 Reply Last reply Reply Quote 0
                    • DerelictD
                      Derelict LAYER 8 Netgate
                      last edited by

                      Why are you messing about with /25 subnets?

                      Seems you're over-thinking and over-complicating things.

                      You would also have a much more straightforward design if you got an access point and stopped trying to do Wi-Fi on pfSense.

                      Chattanooga, Tennessee, USA
                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      M 1 Reply Last reply Reply Quote 0
                      • M
                        myradon @Derelict
                        last edited by myradon

                        @Derelict said in Multiple interfaces, VLAN switching:

                        Why are you messing about with /25 subnets?

                        Seems you're over-thinking and over-complicating things.

                        You would also have a much more straightforward design if you got an access point and stopped trying to do Wi-Fi on pfSense.

                        How wil this help me solving the issue? I change the subnetmask to a let say /24, and then? I remove the Wlan-NIC and add a WIFI-router/AP to LAN-interface. It's a bridge again. Just like current setup. Is Pfsense not capable of doing a good job as an AP? For years it has been serving this network in a simply bridged LAN/WLAN-setup along with separate guest-network. The webserver was running in LAN as VM Guest. Ideal? Not from a security standpoint.

                        According to IOT-experts putting IOT-stuff in isolated subnet is best practice; I've got wired Home Assistant host, wireless-devices(Wifi & Zigbee) for Home automation. To me it makes good sense to put them in same subnet so they can easily talk to each other And isolates from LAN-subnet. Because some IOT-devices don't do 802.11n I've add an extra RT3570 Wifi USB-dongle and bridged it too (isn't 100% stable fiddling with settings in Pfsense sometimes dongle goes down; have to reboot). Putting webserver in DMZ also makes good sense to me as security measure.
                        When you wan't to control/access Home Assistant from your laptop, desktop, tablet or phone you have to be able to access that server (pinch a hole in firewall). It has to go from 1 subnet to the other. Because I don't like to run double or triple cabling I'm using VLANs. If I would like to access the webserver from LAN it has to be routed from subnet-lan too. 4 local subnets more or less isolated with different purposes. This my reasoning for current setup. What will make it simpler and do what I have in mind?

                        1 Reply Last reply Reply Quote 0
                        • johnpozJ
                          johnpoz LAYER 8 Global Moderator
                          last edited by johnpoz

                          Not sure why you think you need to bridge anything on pfsense to put wifi in its own vlan... I have multiple vlans that are both wired and wireless for IOT devices and the like... Zero bridges on my pfsense!! Get an AP that does vlans!! Connect it to your switch..

                          As to the /24 vs the /25 or /29s - just easier for humans to see that its a different network when you use /24 is all... Makes no matter from a use standpoint - just common practice - not like your tight on rfc1918 space ;) So why break at the /25 mark or /29.. Why not just use the easier to spot /24

                          If you have an AP that does vlans - you can put any wireless device on any vlan you want.. Without having to touch pfsense or deal with that added complexity and lack of performance that a bridged interface brings.. And gawd the performance for wifi on freebsd is going to be horrible.. You can not do AC that is for sure..

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                          M 1 Reply Last reply Reply Quote 0
                          • M
                            myradon @johnpoz
                            last edited by myradon

                            @johnpoz said in Multiple interfaces, VLAN switching:

                            Not sure why you think you need to bridge anything on pfsense to put wifi in its own vlan... I have multiple vlans that are both wired and wireless for IOT devices and the like... Zero bridges on my pfsense!! Get an AP that does vlans!! Connect it to your switch..

                            I only use bridges when wired and wireless should see each other. That fact doesn't have to do with VLAN by itself. Only when they should be able to use the same physical connection. How do your IOT-devices talking to each other? Aren't you essentially tying together interfaces on VLAN-level on your AP? The Atheros WIFI-nic is pretty stable piece of hardware I would like to remain.

                            As to the /24 vs the /25 or /29s - just easier for humans to see that its a different network when you use /24 is all... Makes no matter from a use standpoint - just common practice - not like your tight on rfc1918 space ;) So why break at the /25 mark or /29.. Why not just use the easier to spot /24

                            Maybe I'm not human ;-)

                            If you have an AP that does vlans - you can put any wireless device on any vlan you want.. Without having to touch pfsense or deal with that added complexity and lack of performance that a bridged interface brings.. And gawd the performance for wifi on freebsd is going to be horrible.. You can not do AC that is for sure..

                            I get your VLAN-setup. In my setup Pfsense doesn't know anything about VLANs (just two extra 50cm patch cables ). Yep 802.11AC would be welcome. Netflix 1080p on iPad does run still fine.

                            Still I don't see how changing into a VLAN-supported AP, Pfsense setup with VLANs along with VLAN-interface firewall-rules will solve my problem of devices not able to ping each other from different subnets. The 2 bridged subnets work fine on Mac-level like your VLAN ID1 combined Wireless/Wired is working. These 2 system tunables should do the trick for me;

                            net.link.bridge.pfil_bridge	Packet filter on the bridge interface	1	 
                            net.link.bridge.pfil_member	Packet filter on the member interface	0
                            

                            TCPdump listening on DMZ-interface. Ping from host LAN-subnet to host DMZ-subnet;

                            16:33:29.065455 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 2, length 64
                            16:33:29.884582 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 3, length 64
                            16:33:29.901840 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 3, length 64
                            16:33:30.886917 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 4, length 64
                            16:33:30.899407 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 4, length 64
                            16:33:31.888232 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 5, length 64
                            16:33:31.898171 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 5, length 64
                            16:33:32.214350 ARP, Request who-has 192.168.133.129 tell 192.168.133.131, length 46
                            16:33:32.214382 ARP, Reply 192.168.133.129 is-at 00:30:18:c7:af:70, length 28
                            16:33:32.892456 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 6, length 64
                            

                            Nice! that's fine.

                            TCPdump listening on BRIDGEDIOT-interface. Ping from host LAN-subnet And from host WLANGUEST to host IOT-subnet;

                            18:19:17.612968 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 28, length 64
                            18:19:18.405515 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 19, length 64
                            18:19:18.614008 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 29, length 64
                            18:19:19.411016 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 20, length 64
                            18:19:19.614242 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 30, length 64
                            18:19:20.416574 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 21, length 64
                            18:19:20.616907 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 31, length 64
                            18:19:21.053445 IP 192.168.132.133.42102 > 239.255.255.250.1900: UDP, length 126
                            18:19:21.421993 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 22, length 64
                            18:19:21.620510 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 32, length 64
                            18:19:22.427412 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 23, length 64
                            18:19:22.553453 IP 192.168.132.133.57621 > 192.168.132.255.57621: UDP, length 44
                            18:19:22.622639 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 33, length 64
                            18:19:23.432543 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 24, length 64
                            18:19:23.624955 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 34, length 64
                            18:19:23.819926 IP 192.168.132.133.48448 > 192.168.132.129.53: UDP, length 48
                            18:19:23.870237 IP 192.168.132.129.53 > 192.168.132.133.48448: UDP, length 88
                            18:19:24.436550 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 25, length 64
                            18:19:24.627276 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 35, length 64
                            18:19:25.442658 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 26, length 64
                            18:19:25.628934 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 36, length 64
                            18:19:26.447828 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 27, length 64
                            18:19:26.630960 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 37, length 64
                            

                            TCPdump on host 'htpc' IOT-subnet on vlan-iot

                            raymond@htpc:~$ sudo tcpdump -i vlan-iot
                            tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
                            listening on vlan-iot, link-type EN10MB (Ethernet), capture size 262144 bytes
                            18:22:22.579128 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
                            18:22:22.582521 IP htpc.35299 > _gateway.domain: 4793+ [1au] PTR? 255.132.168.192.in-addr.arpa. (57)
                            18:22:22.582914 IP _gateway.domain > htpc.35299: 4793 NXDomain* 0/1/1 (116)
                            18:22:22.583162 IP htpc.35299 > _gateway.domain: 4793+ PTR? 255.132.168.192.in-addr.arpa. (46)
                            18:22:22.583536 IP _gateway.domain > htpc.35299: 4793 NXDomain* 0/1/0 (105)
                            18:22:22.585752 IP htpc.40124 > _gateway.domain: 37925+ [1au] PTR? 129.132.168.192.in-addr.arpa. (57)
                            18:22:22.586184 IP _gateway.domain > htpc.40124: 37925 NXDomain* 0/1/1 (116)
                            18:22:22.586278 IP htpc.40124 > _gateway.domain: 37925+ PTR? 129.132.168.192.in-addr.arpa. (46)
                            18:22:27.704279 ARP, Request who-has _gateway tell htpc, length 28
                            18:22:27.704484 ARP, Reply _gateway is-at 02:0f:c1:d7:81:01 (oui Unknown), length 46
                            18:22:47.552069 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
                            18:22:47.552086 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
                            18:22:47.552090 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:22:47.552093 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 477
                            18:22:47.552140 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:22:47.552146 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 457
                            18:22:47.552149 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:22:52.579511 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
                            18:23:17.578857 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
                            18:23:17.578939 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
                            18:23:17.578944 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:23:17.578947 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 477
                            18:23:21.050920 IP htpc.42102 > 239.255.255.250.1900: UDP, length 126
                            18:23:22.580104 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
                            18:23:47.591834 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
                            18:23:47.591851 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
                            18:23:47.859711 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:23:47.859713 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 457
                            18:23:47.859715 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:23:47.859782 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:23:47.859787 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
                            18:23:52.580636 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
                            18:24:08.523712 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
                            18:24:09.524293 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
                            18:24:10.524833 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
                            18:24:11.525775 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
                            18:24:14.648905 IP htpc.50796 > 23-111-157-86.static.hvvc.us.4000: UDP, length 24
                            18:24:14.649509 IP htpc.36129 > _gateway.domain: 41992+ [1au] PTR? 86.157.111.23.in-addr.arpa. (55)
                            18:24:14.759058 IP 23-111-157-86.static.hvvc.us.4000 > htpc.50796: UDP, length 20
                            18:24:15.745194 IP _gateway.domain > htpc.36129: 41992 1/0/1 PTR 23-111-157-86.static.hvvc.us. (97)
                            18:24:15.745231 IP htpc > _gateway: ICMP htpc udp port 36129 unreachable, length 133
                            18:24:17.045432 IP htpc.50796 > 66-165-233-194.static.hvvc.us.4000: UDP, length 24
                            18:24:17.045954 IP htpc.39556 > _gateway.domain: 304+ [1au] PTR? 194.233.165.66.in-addr.arpa. (56)
                            18:24:17.214250 IP 66-165-233-194.static.hvvc.us.4000 > htpc.50796: UDP, length 20
                            18:24:17.640458 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
                            18:24:17.640557 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
                            18:24:17.640564 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:24:17.907664 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:24:17.907666 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 457
                            18:24:17.907668 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
                            18:24:17.926893 IP _gateway.domain > htpc.39556: 304 1/0/1 PTR 66-165-233-194.static.hvvc.us. (99)
                            18:24:17.926929 IP htpc > _gateway: ICMP htpc udp port 39556 unreachable, length 135
                            18:24:19.832258 ARP, Request who-has _gateway tell htpc, length 28
                            18:24:19.832475 ARP, Reply _gateway is-at 02:0f:c1:d7:81:01 (oui Unknown), length 46
                            18:24:22.581881 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
                            

                            TCPdump listening on BRIDGEDIOT-interface. Ping from host LAN-subnet (192.168.130.135) to host IOT-subnet And from host htpc (192.168.132.133) IOT-subnet;

                            20:49:53.179907 LLDP, length 242: switch
                            20:49:53.249628 IP 192.168.132.133.57621 > 192.168.132.255.57621: UDP, length 44
                            20:49:53.275987 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 21, length 64
                            20:49:54.279214 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 22, length 64
                            20:49:55.280684 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 23, length 64
                            20:49:56.282219 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 24, length 64
                            20:49:57.284520 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 25, length 64
                            20:49:58.285231 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 26, length 64
                            20:49:59.288202 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 27, length 64
                            20:50:00.289155 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 28, length 64
                            20:50:01.291632 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 29, length 64
                            20:50:02.293119 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 30, length 64
                            20:50:03.297943 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 31, length 64
                            20:50:04.298194 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 32, length 64
                            20:50:05.298473 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 33, length 64
                            20:50:06.299761 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 34, length 64
                            20:50:07.302180 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 35, length 64
                            20:50:08.307137 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 36, length 64
                            20:50:08.617221 IP 192.168.132.133.33796 > 239.255.255.250.1900: UDP, length 172
                            20:50:09.307815 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 37, length 64
                            20:50:09.617899 IP 192.168.132.133.33796 > 239.255.255.250.1900: UDP, length 172
                            20:50:10.308168 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 38, length 64
                            20:50:10.618888 IP 192.168.132.133.33796 > 239.255.255.250.1900: UDP, length 172
                            20:50:12.436720 IP 192.168.132.133.50046 > 192.168.132.129.53: UDP, length 34
                            20:50:12.436933 IP 192.168.132.133.36811 > 192.168.132.129.53: UDP, length 34
                            20:50:12.476695 IP 192.168.132.129.53 > 192.168.132.133.50046: UDP, length 98
                            20:50:12.476856 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 50046 unreachable, length 134
                            20:50:12.478372 IP 192.168.132.129.53 > 192.168.132.133.36811: UDP, length 34
                            20:50:12.478568 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 36811 unreachable, length 70
                            20:50:13.310931 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 41, length 64
                            20:50:14.315405 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 42, length 64
                            20:50:15.319584 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 43, length 64
                            20:50:16.323632 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 44, length 64
                            20:50:17.325918 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 45, length 64
                            20:50:17.596892 ARP, Request who-has 192.168.132.129 tell 192.168.132.133, length 46
                            20:50:17.596921 ARP, Reply 192.168.132.129 is-at 02:0f:c1:d7:81:01, length 28
                            20:50:18.326224 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 46, length 64
                            20:50:19.329571 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 47, length 64
                            20:50:19.439597 IP 192.168.132.133.59647 > 192.168.132.129.53: UDP, length 34
                            20:50:19.439833 IP 192.168.132.133.42127 > 192.168.132.129.53: UDP, length 34
                            20:50:19.439937 IP 192.168.132.129.53 > 192.168.132.133.59647: UDP, length 98
                            20:50:19.440136 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 59647 unreachable, length 134
                            20:50:19.468550 IP 192.168.132.129.53 > 192.168.132.133.42127: UDP, length 34
                            20:50:19.499906 IP 192.168.132.133.57149 > 192.168.132.129.53: UDP, length 54
                            20:50:20.053295 IP 192.168.132.129.53 > 192.168.132.133.57149: UDP, length 109
                            20:50:20.053495 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 57149 unreachable, length 145
                            20:50:20.334721 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 48, length 64
                            20:50:21.339863 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 49, length 64
                            20:50:22.342437 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 50, length 64
                            20:50:23.250412 IP 192.168.132.133.57621 > 192.168.132.255.57621: UDP, length 44
                            20:50:23.345154 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 51, length 64
                            20:50:23.820895 LLDP, length 242: switch
                            20:50:24.349488 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 52, length 64
                            20:50:25.351755 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 53, length 64
                            20:50:26.352909 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 54, length 64
                            20:50:27.314823 IP 192.168.132.133.53560 > 192.168.132.129.53: UDP, length 34
                            20:50:27.315063 IP 192.168.132.133.38566 > 192.168.132.129.53: UDP, length 34
                            20:50:27.315164 IP 192.168.132.129.53 > 192.168.132.133.53560: UDP, length 98
                            20:50:27.315392 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 53560 unreachable, length 134
                            20:50:27.342517 IP 192.168.132.129.53 > 192.168.132.133.38566: UDP, length 34
                            20:50:27.356815 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 55, length 64
                            20:50:28.358641 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 56, length 64
                            20:50:29.358716 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 57, length 64
                            20:50:30.358828 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 58, length 64
                            

                            I barely understand these dumps. As It's not my profession. On pfsense interface address (192.168.132.129) you can see both ICMP-requests. Somehow host htpc don't respond (from LAN to IOT) and ping from host htpc outbound, as I interpret, get's the pfsense-interface but the host htpc is not reachable; "20:50:27.315392 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 53560 unreachable, length 134"

                            EDIT: By the way no firewall is running on host htpc;

                            raymond@htpc:~$ sudo ufw status
                            Status: inactive
                            raymond@htpc:~$
                            

                            EDIT2: Look ma! When I set interface also into promicious mode I can even see ICMP-request and replies from within subnet;

                            21:39:25.323752 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 7, length 64
                            21:39:25.323971 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 7, length 64
                            21:39:25.443988 ARP, Request who-has 192.168.132.151 tell 192.168.132.133, length 46
                            21:39:25.493993 ARP, Reply 192.168.132.151 is-at a0:56:f3:58:3d:96, length 28
                            21:39:26.324558 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 8, length 64
                            21:39:26.324824 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 8, length 64
                            21:39:27.330165 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 9, length 64
                            21:39:27.330449 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 9, length 64
                            21:39:28.338949 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 10, length 64
                            21:39:28.339240 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 10, length 64
                            21:39:29.344261 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 11, length 64
                            21:39:29.344481 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 11, length 64
                            21:39:29.795392 ARP, Request who-has 192.168.132.129 tell 192.168.132.133, length 46
                            21:39:29.795424 ARP, Reply 192.168.132.129 is-at 02:41:6e:bb:59:01, length 28
                            

                            Shoot me!

                            M 1 Reply Last reply Reply Quote 0
                            • M
                              myradon @myradon
                              last edited by

                              Okay I'll write what's causing this behaviour. It has nothing todo what the different masks, using pfsense as a wireless-bridge, using bridged interfaces, routing or whatever. Basically it because Linux hosts have default kernel-setting "reverse path filtering" enabled. This can be temporary disabled (untill next reboot) by sudo sysctl -w 'net.ipv4.conf.all.rp_filter=0. Now I understand why it happened. So you have to take that into account.

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.