Multiple interfaces, VLAN switching



  • Hi guys,

    Recently I got hold of 2 network switches supporting vlan (HPE 1820 24G PoE and HP 1810-24G). I'm using pfsense for several years. Even started with it's origin m0n0wall. I want to automate my house (Home Assistant, Zigbee lights, lightswitches) so isolating IOT-stuff is no luxury. Also a good time to put my dev-webserver in DMZ-subnet.

    I get so strange behaviour when pinging interfaces. Some hosts are reachable, some are not. Actually this quest started when docker container on host htpc couldn't be reached from other subnets. VM Guest in subnet is does ICMP-reply but it's host doesn't. I erased all my rules and set some simple loose "any" rules.

    Network diagram underneath is simplified (actually 2 switches and more hosts. Switches are connected with 2 tagged ports for all 3 vlans);
    home network

    All hosts get DHCP assigned IP. Bridge members don't have any networking rules And System Tunables;

    net.link.bridge.pfil_bridge Packet filter on the bridge interface 1
    net.link.bridge.pfil_member Packet filter on the member interface 0

    For testing purposes firewall rules allow everything;



    Now is the funny part (not).

    • iPhone (wlanguest interface) can ping hosts in other subnets;
      -- Debian VM, Hackintosh
      -- Mini-itx-bak only on vlan-lan (ID 130)
      -- Htpc only one vlan-lan (ID 130)

    • Hackintosh (bridgelan interface) can ping Htpc;
      -- only on vlan-lan (ID 130, obvious same subnet so no routing),
      -- vlan-iot (ID 132) is timeout
      -- vlan-dmz (ID 133) is timeout

    • Hackintosh (bridgelan interface) can ping Debian VM (NIC configured vlan-dmz on host Htpc);

    • Hackintosh (bridgelan interface) can ping Mini-itx-bak;
      -- only on vlan-lan (ID 130, obvious same subnet so no routing),
      -- vlan-iot (ID 132) is timeout
      -- vlan-dmz (ID 133) is timeout

    • Mini-itx-bak (tagged vlans 130,132,133) CAN ping Hackintosh (untagged 130);
      -- From vlan-lan (ID 130, obvious same subnet so no routing),
      -- From vlan-iot (ID 132), brought vlan-lan and vlan-dmz down first to know it's routed by pfsense.
      -- No go from vlan-dmz (ID 133) because; tried to obtain new lease from DHCP-server (previously released lease). ("sudo dhclient -v lan-dmz"); endless DHCP-discover. (Mac is in DHCP static maps)

    • Mini-itx-bak (tagged vlans 130,132,133) CAN ping Debian VM(NIC configured vlan-dmz on host Htpc);
      -- From vlan-lan (ID 130),
      -- From vlan-iot (ID 132), brought vlan-lan down first to know it's routed by pfsense (checked with mtr).

    • Mini-itx-bak (tagged vlans 130,132,133) CAN ping Htpc (tagged 130,132,133);
      -- IP (ID 130),
      -- IP (ID 132)
      -- IP (ID 133) is timeout (rember Guest Debian VM does reply)

    I don't see any logic. Is anybody able to elaborate this one?

    EDIT: changed diagram


  • LAYER 8 Global Moderator

    Why are you calling what I hope is just pfsense IP on that vlan gateway? You didn't set gateways on these interfaces did you?

    Not sure what you think your accomplishing by pretty much every box on multiple vlans.. Why do you have hosts in 3 different vlans?



  • @johnpoz said in Multiple interfaces, VLAN switching:

    Why are you calling what I hope is just pfsense IP on that vlan gateway?

    Which vlan, which call do you refer to? In opening post I don't see an example of pinging a gateway.

    You didn't set gateways on these interfaces did you?

    I didn't assign any gateways on the (bridge)interfaces. All interfaces 'IPv4 Upstream gateway' set to default 'none'

    Not sure what you think your accomplishing by pretty much every box on multiple vlans.. Why do you have hosts in 3 different vlans?

    Basically host 'mini-itx-bak' is a temporary system to play with multiple vlan-interfaces on a host. I don't want to mess up host Htpc. Both system are running on Ubuntu 18.04 LTS by the way. In the future host Htpc will have all 3 vlan-interfaces; 1 for lan-subnet as the fileserver, 1 to talk to iot-devices + running Home Assistant and 1 for webserver in dmz-subnet (VM guest).

    Does VM host need an IP to get VM Guest assigned an IP? Probably not right. I see with DHCP-discover broadcast to mac ff:ff:ff:ff:ff:ff. So it's a Layer-2 thingy if I'm not mistaking. Then I could ditch 1 IP (htpc, vlan-dmz, id: 133, 192.168.133.132). I ditched 1 IP; htpc, vlan-dmz, id: 133, 192.168.133.132. In the future when I got vlans running like I would like to, host mini-itx-bak will be gone.

    EDIT: I think something is wrong with Ubuntu's new Netplan network configuration. Luckily MacOS also supports VLAN tagging. I've add all 3 vlans to host Hacknitosh and configured switch accordingly. I'm able to ping from iPad in all wifi-subnets to all IP assigned to MacOS vlans. Fropm every vlan I'm able to connect to webserver in DMZ.



  • Okay other thought; why would a vlan-interface reply in subnet and not routed? To it means; vlan is configured correctly on host and switch-port tagging is correct otherwise host don't even respond.

    Here is config and some output;

    raymond@htpc:~$ cat /etc/netplan/config.yaml
    
    network:
      version: 2
      renderer: networkd
      ethernets: 
        enp1s0f0:
          dhcp4: true
          optional: true
        enp1s0f1:
          match:
            macaddress: "00:26:55:da:9f:9b"
          set-name: enp1s0f1
          dhcp4: false
      vlans:
        vlan-lan:
          id: 130
          link: enp1s0f1
          dhcp4: true
          dhcp4-overrides:
            route-metric: 100
        vlan-iot:
          id: 132
          link: enp1s0f1
          dhcp4: true
          dhcp4-overrides:
            route-metric: 200
        vlan-dmz:
          id: 133
          link: enp1s0f1
          dhcp4: false
    
    raymond@htpc:~$ networkctl
    IDX LINK             TYPE               OPERATIONAL SETUP     
      1 lo               loopback           carrier     unmanaged 
      2 enp1s0f0         ether              no-carrier  configuring
      3 enp1s0f1         ether              degraded    configured
      4 vlan-lan         ether              routable    configured
      5 vlan-iot         ether              routable    configured
      6 vlan-dmz         ether              degraded    
    
    raymond@htpc:~$ networkctl status
    State: routable
           Address: 192.168.130.133 on vlan-lan
                    192.168.132.133 on vlan-iot
                    169.254.9.221 on enp1s0f0
                    169.254.9.226 on enp1s0f1
                    fe80::226:55ff:feda:9f9b on enp1s0f1
                    fe80::226:55ff:feda:9f9b on vlan-lan
                    fe80::226:55ff:feda:9f9b on vlan-iot
                    fe80::226:55ff:feda:9f9b on vlan-dmz
           Gateway: 192.168.130.129 on vlan-lan
                    192.168.132.129 on vlan-iot
               DNS: 192.168.130.129
                    192.168.132.129
    


  • @myradon said in Multiple interfaces, VLAN switching:

    To it means; vlan is configured correctly on host and switch-port tagging is correct otherwise host don't even respond.

    Are you enabling VLANs on the devices and then connecting them to a port that's already been assigned to a VLAN? You do one or the other. If you have a managed switch, you configure the port for the VLAN and let the connected device run native LAN. The only time you'd tag a device is if you're connected to an unmanaged switch or running tagged and untagged devices over the same cable. An example of this would be when a computer is plugged into a VoIP phone and then the phone is connected to the switch.



  • @JKnott

    • I've setup 2 hosts with both 3 tagged vlans (ID's 130,132,133) .
    • The switch is configured accordingly (ports tagged id 130,132,133). Also both ports on the switch aren't configured untagged also exlcuded from vlan 1.
    • In my previous post you see output from commands (host htpc); the vlan-interfaces have obtained an IP-lease from Pfsense.
    • VM Guest (Debian) running on Virtualbox on interface "vlan-dmz" will get an IP from Pfsense And does ICMP-reply from all subnets.

    What could be the reason I can't ping host htpc on vlan-iot? It gets it's lease from pfsense. Within subnet it does reply. So to me it sounds like a routing-thing. But I don't understand what's wrong.


  • LAYER 8 Global Moderator

    @myradon said in Multiple interfaces, VLAN switching:

    Gateway: 192.168.130.129 on vlan-lan
    192.168.132.129 on vlan-iot

    So which gateway does it use, where you you have your state set.. If you want to multi home boxes... Then your going to run into all kinds of asymmetrical routing issues if your not careful..

    There is one thing tagging vlans down to a vm host to use for vm's that are on only 1 vlan.. There is another when you start multihoming devices and have them setup with multiple gateways, etc.

    My point of what your calling gateways.. Don't label it like that!!! That looks like you have a gateway set on pfsense.. Is it pointing to itself, is it pointing to a different IP? Its not a gateway on pfsense, its its IP address - it could be a gateway to something on that network... But you have it listed as gateway on the pfsense box.. Which is confusing to what you actually mean.



  • Okay I'll change diagram in OP. To it's host in the subnets it's their gateway. I thought the Pfsense-interface is then called the gateway. My bad. Anyway new revision of diagram. Hopefully it's now more clear to you networking guys.

    diagram v3

    A clue what's going on?


  • LAYER 8 Netgate

    Why are you messing about with /25 subnets?

    Seems you're over-thinking and over-complicating things.

    You would also have a much more straightforward design if you got an access point and stopped trying to do Wi-Fi on pfSense.



  • @Derelict said in Multiple interfaces, VLAN switching:

    Why are you messing about with /25 subnets?

    Seems you're over-thinking and over-complicating things.

    You would also have a much more straightforward design if you got an access point and stopped trying to do Wi-Fi on pfSense.

    How wil this help me solving the issue? I change the subnetmask to a let say /24, and then? I remove the Wlan-NIC and add a WIFI-router/AP to LAN-interface. It's a bridge again. Just like current setup. Is Pfsense not capable of doing a good job as an AP? For years it has been serving this network in a simply bridged LAN/WLAN-setup along with separate guest-network. The webserver was running in LAN as VM Guest. Ideal? Not from a security standpoint.

    According to IOT-experts putting IOT-stuff in isolated subnet is best practice; I've got wired Home Assistant host, wireless-devices(Wifi & Zigbee) for Home automation. To me it makes good sense to put them in same subnet so they can easily talk to each other And isolates from LAN-subnet. Because some IOT-devices don't do 802.11n I've add an extra RT3570 Wifi USB-dongle and bridged it too (isn't 100% stable fiddling with settings in Pfsense sometimes dongle goes down; have to reboot). Putting webserver in DMZ also makes good sense to me as security measure.
    When you wan't to control/access Home Assistant from your laptop, desktop, tablet or phone you have to be able to access that server (pinch a hole in firewall). It has to go from 1 subnet to the other. Because I don't like to run double or triple cabling I'm using VLANs. If I would like to access the webserver from LAN it has to be routed from subnet-lan too. 4 local subnets more or less isolated with different purposes. This my reasoning for current setup. What will make it simpler and do what I have in mind?


  • LAYER 8 Global Moderator

    Not sure why you think you need to bridge anything on pfsense to put wifi in its own vlan... I have multiple vlans that are both wired and wireless for IOT devices and the like... Zero bridges on my pfsense!! Get an AP that does vlans!! Connect it to your switch..

    As to the /24 vs the /25 or /29s - just easier for humans to see that its a different network when you use /24 is all... Makes no matter from a use standpoint - just common practice - not like your tight on rfc1918 space ;) So why break at the /25 mark or /29.. Why not just use the easier to spot /24

    If you have an AP that does vlans - you can put any wireless device on any vlan you want.. Without having to touch pfsense or deal with that added complexity and lack of performance that a bridged interface brings.. And gawd the performance for wifi on freebsd is going to be horrible.. You can not do AC that is for sure..



  • @johnpoz said in Multiple interfaces, VLAN switching:

    Not sure why you think you need to bridge anything on pfsense to put wifi in its own vlan... I have multiple vlans that are both wired and wireless for IOT devices and the like... Zero bridges on my pfsense!! Get an AP that does vlans!! Connect it to your switch..

    I only use bridges when wired and wireless should see each other. That fact doesn't have to do with VLAN by itself. Only when they should be able to use the same physical connection. How do your IOT-devices talking to each other? Aren't you essentially tying together interfaces on VLAN-level on your AP? The Atheros WIFI-nic is pretty stable piece of hardware I would like to remain.

    As to the /24 vs the /25 or /29s - just easier for humans to see that its a different network when you use /24 is all... Makes no matter from a use standpoint - just common practice - not like your tight on rfc1918 space ;) So why break at the /25 mark or /29.. Why not just use the easier to spot /24

    Maybe I'm not human ;-)

    If you have an AP that does vlans - you can put any wireless device on any vlan you want.. Without having to touch pfsense or deal with that added complexity and lack of performance that a bridged interface brings.. And gawd the performance for wifi on freebsd is going to be horrible.. You can not do AC that is for sure..

    I get your VLAN-setup. In my setup Pfsense doesn't know anything about VLANs (just two extra 50cm patch cables ). Yep 802.11AC would be welcome. Netflix 1080p on iPad does run still fine.

    Still I don't see how changing into a VLAN-supported AP, Pfsense setup with VLANs along with VLAN-interface firewall-rules will solve my problem of devices not able to ping each other from different subnets. The 2 bridged subnets work fine on Mac-level like your VLAN ID1 combined Wireless/Wired is working. These 2 system tunables should do the trick for me;

    net.link.bridge.pfil_bridge	Packet filter on the bridge interface	1	 
    net.link.bridge.pfil_member	Packet filter on the member interface	0
    

    TCPdump listening on DMZ-interface. Ping from host LAN-subnet to host DMZ-subnet;

    16:33:29.065455 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 2, length 64
    16:33:29.884582 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 3, length 64
    16:33:29.901840 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 3, length 64
    16:33:30.886917 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 4, length 64
    16:33:30.899407 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 4, length 64
    16:33:31.888232 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 5, length 64
    16:33:31.898171 IP 192.168.133.131 > 192.168.130.135: ICMP echo reply, id 25605, seq 5, length 64
    16:33:32.214350 ARP, Request who-has 192.168.133.129 tell 192.168.133.131, length 46
    16:33:32.214382 ARP, Reply 192.168.133.129 is-at 00:30:18:c7:af:70, length 28
    16:33:32.892456 IP 192.168.130.135 > 192.168.133.131: ICMP echo request, id 25605, seq 6, length 64
    

    Nice! that's fine.

    TCPdump listening on BRIDGEDIOT-interface. Ping from host LAN-subnet And from host WLANGUEST to host IOT-subnet;

    18:19:17.612968 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 28, length 64
    18:19:18.405515 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 19, length 64
    18:19:18.614008 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 29, length 64
    18:19:19.411016 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 20, length 64
    18:19:19.614242 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 30, length 64
    18:19:20.416574 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 21, length 64
    18:19:20.616907 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 31, length 64
    18:19:21.053445 IP 192.168.132.133.42102 > 239.255.255.250.1900: UDP, length 126
    18:19:21.421993 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 22, length 64
    18:19:21.620510 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 32, length 64
    18:19:22.427412 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 23, length 64
    18:19:22.553453 IP 192.168.132.133.57621 > 192.168.132.255.57621: UDP, length 44
    18:19:22.622639 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 33, length 64
    18:19:23.432543 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 24, length 64
    18:19:23.624955 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 34, length 64
    18:19:23.819926 IP 192.168.132.133.48448 > 192.168.132.129.53: UDP, length 48
    18:19:23.870237 IP 192.168.132.129.53 > 192.168.132.133.48448: UDP, length 88
    18:19:24.436550 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 25, length 64
    18:19:24.627276 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 35, length 64
    18:19:25.442658 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 26, length 64
    18:19:25.628934 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 36, length 64
    18:19:26.447828 IP 192.168.131.131 > 192.168.132.133: ICMP echo request, id 0, seq 27, length 64
    18:19:26.630960 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 61189, seq 37, length 64
    

    TCPdump on host 'htpc' IOT-subnet on vlan-iot

    raymond@htpc:~$ sudo tcpdump -i vlan-iot
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on vlan-iot, link-type EN10MB (Ethernet), capture size 262144 bytes
    18:22:22.579128 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
    18:22:22.582521 IP htpc.35299 > _gateway.domain: 4793+ [1au] PTR? 255.132.168.192.in-addr.arpa. (57)
    18:22:22.582914 IP _gateway.domain > htpc.35299: 4793 NXDomain* 0/1/1 (116)
    18:22:22.583162 IP htpc.35299 > _gateway.domain: 4793+ PTR? 255.132.168.192.in-addr.arpa. (46)
    18:22:22.583536 IP _gateway.domain > htpc.35299: 4793 NXDomain* 0/1/0 (105)
    18:22:22.585752 IP htpc.40124 > _gateway.domain: 37925+ [1au] PTR? 129.132.168.192.in-addr.arpa. (57)
    18:22:22.586184 IP _gateway.domain > htpc.40124: 37925 NXDomain* 0/1/1 (116)
    18:22:22.586278 IP htpc.40124 > _gateway.domain: 37925+ PTR? 129.132.168.192.in-addr.arpa. (46)
    18:22:27.704279 ARP, Request who-has _gateway tell htpc, length 28
    18:22:27.704484 ARP, Reply _gateway is-at 02:0f:c1:d7:81:01 (oui Unknown), length 46
    18:22:47.552069 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
    18:22:47.552086 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
    18:22:47.552090 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:22:47.552093 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 477
    18:22:47.552140 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:22:47.552146 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 457
    18:22:47.552149 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:22:52.579511 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
    18:23:17.578857 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
    18:23:17.578939 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
    18:23:17.578944 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:23:17.578947 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 477
    18:23:21.050920 IP htpc.42102 > 239.255.255.250.1900: UDP, length 126
    18:23:22.580104 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
    18:23:47.591834 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
    18:23:47.591851 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
    18:23:47.859711 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:23:47.859713 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 457
    18:23:47.859715 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:23:47.859782 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:23:47.859787 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
    18:23:52.580636 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
    18:24:08.523712 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
    18:24:09.524293 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
    18:24:10.524833 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
    18:24:11.525775 IP htpc.60768 > 239.255.255.250.1900: UDP, length 172
    18:24:14.648905 IP htpc.50796 > 23-111-157-86.static.hvvc.us.4000: UDP, length 24
    18:24:14.649509 IP htpc.36129 > _gateway.domain: 41992+ [1au] PTR? 86.157.111.23.in-addr.arpa. (55)
    18:24:14.759058 IP 23-111-157-86.static.hvvc.us.4000 > htpc.50796: UDP, length 20
    18:24:15.745194 IP _gateway.domain > htpc.36129: 41992 1/0/1 PTR 23-111-157-86.static.hvvc.us. (97)
    18:24:15.745231 IP htpc > _gateway: ICMP htpc udp port 36129 unreachable, length 133
    18:24:17.045432 IP htpc.50796 > 66-165-233-194.static.hvvc.us.4000: UDP, length 24
    18:24:17.045954 IP htpc.39556 > _gateway.domain: 304+ [1au] PTR? 194.233.165.66.in-addr.arpa. (56)
    18:24:17.214250 IP 66-165-233-194.static.hvvc.us.4000 > htpc.50796: UDP, length 20
    18:24:17.640458 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 409
    18:24:17.640557 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 481
    18:24:17.640564 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:24:17.907664 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:24:17.907666 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 457
    18:24:17.907668 IP _gateway.39402 > 239.255.255.250.1900: UDP, length 417
    18:24:17.926893 IP _gateway.domain > htpc.39556: 304 1/0/1 PTR 66-165-233-194.static.hvvc.us. (99)
    18:24:17.926929 IP htpc > _gateway: ICMP htpc udp port 39556 unreachable, length 135
    18:24:19.832258 ARP, Request who-has _gateway tell htpc, length 28
    18:24:19.832475 ARP, Reply _gateway is-at 02:0f:c1:d7:81:01 (oui Unknown), length 46
    18:24:22.581881 IP htpc.57621 > 192.168.132.255.57621: UDP, length 44
    

    TCPdump listening on BRIDGEDIOT-interface. Ping from host LAN-subnet (192.168.130.135) to host IOT-subnet And from host htpc (192.168.132.133) IOT-subnet;

    20:49:53.179907 LLDP, length 242: switch
    20:49:53.249628 IP 192.168.132.133.57621 > 192.168.132.255.57621: UDP, length 44
    20:49:53.275987 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 21, length 64
    20:49:54.279214 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 22, length 64
    20:49:55.280684 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 23, length 64
    20:49:56.282219 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 24, length 64
    20:49:57.284520 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 25, length 64
    20:49:58.285231 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 26, length 64
    20:49:59.288202 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 27, length 64
    20:50:00.289155 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 28, length 64
    20:50:01.291632 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 29, length 64
    20:50:02.293119 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 30, length 64
    20:50:03.297943 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 31, length 64
    20:50:04.298194 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 32, length 64
    20:50:05.298473 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 33, length 64
    20:50:06.299761 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 34, length 64
    20:50:07.302180 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 35, length 64
    20:50:08.307137 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 36, length 64
    20:50:08.617221 IP 192.168.132.133.33796 > 239.255.255.250.1900: UDP, length 172
    20:50:09.307815 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 37, length 64
    20:50:09.617899 IP 192.168.132.133.33796 > 239.255.255.250.1900: UDP, length 172
    20:50:10.308168 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 38, length 64
    20:50:10.618888 IP 192.168.132.133.33796 > 239.255.255.250.1900: UDP, length 172
    20:50:12.436720 IP 192.168.132.133.50046 > 192.168.132.129.53: UDP, length 34
    20:50:12.436933 IP 192.168.132.133.36811 > 192.168.132.129.53: UDP, length 34
    20:50:12.476695 IP 192.168.132.129.53 > 192.168.132.133.50046: UDP, length 98
    20:50:12.476856 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 50046 unreachable, length 134
    20:50:12.478372 IP 192.168.132.129.53 > 192.168.132.133.36811: UDP, length 34
    20:50:12.478568 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 36811 unreachable, length 70
    20:50:13.310931 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 41, length 64
    20:50:14.315405 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 42, length 64
    20:50:15.319584 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 43, length 64
    20:50:16.323632 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 44, length 64
    20:50:17.325918 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 45, length 64
    20:50:17.596892 ARP, Request who-has 192.168.132.129 tell 192.168.132.133, length 46
    20:50:17.596921 ARP, Reply 192.168.132.129 is-at 02:0f:c1:d7:81:01, length 28
    20:50:18.326224 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 46, length 64
    20:50:19.329571 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 47, length 64
    20:50:19.439597 IP 192.168.132.133.59647 > 192.168.132.129.53: UDP, length 34
    20:50:19.439833 IP 192.168.132.133.42127 > 192.168.132.129.53: UDP, length 34
    20:50:19.439937 IP 192.168.132.129.53 > 192.168.132.133.59647: UDP, length 98
    20:50:19.440136 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 59647 unreachable, length 134
    20:50:19.468550 IP 192.168.132.129.53 > 192.168.132.133.42127: UDP, length 34
    20:50:19.499906 IP 192.168.132.133.57149 > 192.168.132.129.53: UDP, length 54
    20:50:20.053295 IP 192.168.132.129.53 > 192.168.132.133.57149: UDP, length 109
    20:50:20.053495 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 57149 unreachable, length 145
    20:50:20.334721 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 48, length 64
    20:50:21.339863 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 49, length 64
    20:50:22.342437 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 50, length 64
    20:50:23.250412 IP 192.168.132.133.57621 > 192.168.132.255.57621: UDP, length 44
    20:50:23.345154 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 51, length 64
    20:50:23.820895 LLDP, length 242: switch
    20:50:24.349488 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 52, length 64
    20:50:25.351755 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 53, length 64
    20:50:26.352909 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 54, length 64
    20:50:27.314823 IP 192.168.132.133.53560 > 192.168.132.129.53: UDP, length 34
    20:50:27.315063 IP 192.168.132.133.38566 > 192.168.132.129.53: UDP, length 34
    20:50:27.315164 IP 192.168.132.129.53 > 192.168.132.133.53560: UDP, length 98
    20:50:27.315392 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 53560 unreachable, length 134
    20:50:27.342517 IP 192.168.132.129.53 > 192.168.132.133.38566: UDP, length 34
    20:50:27.356815 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 55, length 64
    20:50:28.358641 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 56, length 64
    20:50:29.358716 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 57, length 64
    20:50:30.358828 IP 192.168.130.135 > 192.168.132.133: ICMP echo request, id 28422, seq 58, length 64
    

    I barely understand these dumps. As It's not my profession. On pfsense interface address (192.168.132.129) you can see both ICMP-requests. Somehow host htpc don't respond (from LAN to IOT) and ping from host htpc outbound, as I interpret, get's the pfsense-interface but the host htpc is not reachable; "20:50:27.315392 IP 192.168.132.133 > 192.168.132.129: ICMP 192.168.132.133 udp port 53560 unreachable, length 134"

    EDIT: By the way no firewall is running on host htpc;

    raymond@htpc:~$ sudo ufw status
    Status: inactive
    raymond@htpc:~$
    

    EDIT2: Look ma! When I set interface also into promicious mode I can even see ICMP-request and replies from within subnet;

    21:39:25.323752 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 7, length 64
    21:39:25.323971 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 7, length 64
    21:39:25.443988 ARP, Request who-has 192.168.132.151 tell 192.168.132.133, length 46
    21:39:25.493993 ARP, Reply 192.168.132.151 is-at a0:56:f3:58:3d:96, length 28
    21:39:26.324558 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 8, length 64
    21:39:26.324824 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 8, length 64
    21:39:27.330165 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 9, length 64
    21:39:27.330449 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 9, length 64
    21:39:28.338949 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 10, length 64
    21:39:28.339240 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 10, length 64
    21:39:29.344261 IP 192.168.132.151 > 192.168.132.133: ICMP echo request, id 0, seq 11, length 64
    21:39:29.344481 IP 192.168.132.133 > 192.168.132.151: ICMP echo reply, id 0, seq 11, length 64
    21:39:29.795392 ARP, Request who-has 192.168.132.129 tell 192.168.132.133, length 46
    21:39:29.795424 ARP, Reply 192.168.132.129 is-at 02:41:6e:bb:59:01, length 28
    

    Shoot me!



  • Okay I'll write what's causing this behaviour. It has nothing todo what the different masks, using pfsense as a wireless-bridge, using bridged interfaces, routing or whatever. Basically it because Linux hosts have default kernel-setting "reverse path filtering" enabled. This can be temporary disabled (untill next reboot) by sudo sysctl -w 'net.ipv4.conf.all.rp_filter=0. Now I understand why it happened. So you have to take that into account.


Log in to reply