Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue

    Scheduled Pinned Locked Moved
    L2/Switching/VLANs
    7
    247
    59.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      NRgia @johnpoz
      last edited by

      @johnpoz said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

      @nrgia well clearly something is going on, but if 22.05 broke vlans - I would expect the boards would be a fire with people complaining. Even if was only ix interfaces, have to assume your not the only one using ix and vlans that have moved to 22.05

      That's true, that's why I don't know what to say, it's not a no name board. or a realtek driver. But I wonder, what's @stephenw10 conclusion?

      johnpozJ 1 Reply Last reply Reply Quote 0
      • johnpozJ
        johnpoz LAYER 8 Global Moderator @NRgia
        last edited by

        @nrgia I'm with you for sure - one of the more strange threads seen in a really long time. @stephenw10 would be for sure my go to guy on anything like this.

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.03 | Lab VMs 2.7.2, 24.03

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          It's not generally broken. The 7100 uses ix NICs with VLANs by default and that works fine in 22.05. Also my test here with a 4100 using an individual NIC is not broken.

          We know more than just 22.01 worked and 22.05 didn't. The snapshot that first failed was immediately after the patch that went in to fix VLAN0 handling.

          The incoming traffic we see in the pcap in 22.05 is tagged VLAN0 and hence is dropped by pfSense/FreeBSD. That is the expected behaviour.

          The only thing we don't know is where that tag is coming from.
          Experiments seem to show that is isn't coming from the switch (though that still seems most likely) or from the AP.
          Unless I've misunderstood the situation the only thing left is the driver or hardware. Neither of which seem at all likely but in the absence of anything else...

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by stephenw10

            What I expect to see if you have any sort of priotiy tagging on a VLAN is something like this:

            18:23:28.004183 00:90:7f:b4:38:fa (oui Unknown) > 90:ec:77:1f:8a:5f (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 1001, p 4, ethertype IPv4, AP300.stevew.lan > 10.101.0.1: ICMP echo request, id 10296, seq 2, length 64
18:23:28.004212 90:ec:77:1f:8a:5f (oui Unknown) > 00:90:7f:b4:38:fa (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 1001, p 0, ethertype IPv4, 10.101.0.1 > AP300.stevew.lan: ICMP echo reply, id 10296, seq 2, length 64

            There I have set egress tagging in OpenWRT to map 0 to 4 so pfSense sees it on the incoming traffic. But there is only one set of tags.

            N 1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              If we can make the switch or AP apply priority tags we could then see which set of tags they appear on in the 22.05 pcap. I expect it to be on the VLAN20 tag. If it's on VLAN0 tag that might prove what's creating it.

              N 1 Reply Last reply Reply Quote 0
              • N
                NRgia @stephenw10
                last edited by

                @stephenw10 said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                What I expect to see if you have any sort of priotiy tagging on a VLAN is something like this:

                18:23:28.004183 00:90:7f:b4:38:fa (oui Unknown) > 90:ec:77:1f:8a:5f (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 1001, p 4, ethertype IPv4, AP300.stevew.lan > 10.101.0.1: ICMP echo request, id 10296, seq 2, length 64
18:23:28.004212 90:ec:77:1f:8a:5f (oui Unknown) > 00:90:7f:b4:38:fa (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 1001, p 0, ethertype IPv4, 10.101.0.1 > AP300.stevew.lan: ICMP echo reply, id 10296, seq 2, length 64

                There I have set egress tagging in OpenWRT to map 0 to 4 so pfSense sees it on the incoming traffic. But there is only one set of tags.

                Just a question, what we except to happen if set some priority to VLANS form pfSense? I know the documentation says to leave that on zero?

                I don't know how to provide what you want to see.

                stephenw10S 1 Reply Last reply Reply Quote 0
                • N
                  NRgia @stephenw10
                  last edited by NRgia

                  @stephenw10
                  From the switch in order to apply priority tags, I must disable the 802.1p mode, and apply per port, for the AP I don't see any option to apply a priority without buying a switch from Ubiquiti. I can provide you some screenshots.

                  This is how it looks for a VLAN:
                  https://imgur.com/a/3gBCTIL

                  johnpozJ 1 Reply Last reply Reply Quote 0
                  • johnpozJ
                    johnpoz LAYER 8 Global Moderator @NRgia
                    last edited by

                    @nrgia I have unifi APs, and yeah I just looked I do not see anywhere to set priority on a vlan at all.

                    Running the lastest firmware on APs 6.2.27 and 7.2.87 controller software. I could try moving to the new UI vs legacy that I use.

                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                    If you get confused: Listen to the Music Play
                    Please don't Chat/PM me for help, unless mod related
                    SG-4860 24.03 | Lab VMs 2.7.2, 24.03

                    N 1 Reply Last reply Reply Quote 0
                    • N
                      NRgia @johnpoz
                      last edited by NRgia

                      @johnpoz I'm using the new UI, don't bother. If you don't buy the whole ecosystem you can do so much. But a four eyes check doesn't do any harm. Thank you

                      N 1 Reply Last reply Reply Quote 0
                      • N
                        NRgia @NRgia
                        last edited by

                        @nrgia How your networks are configured in the AP, I mean in the Networks tab? Are the VLANS set to VLANs only network? Or how? Thank you

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator @NRgia
                          last edited by

                          @nrgia said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                          Just a question, what we except to happen if set some priority to VLANS form pfSense?

                          Applying it on the VLAN should show in the VLAN tag outbound as expected like:

                          19:16:32.694149 00:90:7f:b4:38:fa (oui Unknown) > 90:ec:77:1f:8a:5f (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 1001, p 4, ethertype IPv4, AP300.stevew.lan > 10.101.0.1: ICMP echo request, id 10347, seq 1, length 64
19:16:32.694172 90:ec:77:1f:8a:5f (oui Unknown) > 00:90:7f:b4:38:fa (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 1001, p 3, ethertype IPv4, 10.101.0.1 > AP300.stevew.lan: ICMP echo reply, id 10347, seq 1, length 64

                          pfSense sets prio 3 and the AP replies prio 4.

                          Try using port mode on your switch then. If you can set anything that appears in the pcap it might be useful.

                          Steve

                          N 1 Reply Last reply Reply Quote 0
                          • N
                            NRgia @stephenw10
                            last edited by NRgia

                            @stephenw10 said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                            ng that appears in the pcap it might be useful.
                            Steve

                            First I selected Normal priority on the switch for the pfSense port and the AP port. The switch don't have numbers. Also I select 3 for VLAN 20 on pfSense.

                            I got this:
                            tcp_dump_vlan_prioriry_pfsense_and_switch.txt

                            I lost connection after doing this, I could only connect to devices via their IPs

                            Then i reset the priority from VLAN 20 from pfSense to zero(0) and I got this:
                            tcp_dump_vlan_prioriry_on_switch.txt

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Those pcaps were on the mirror port again?

                              In both those pcaps there are no packets from pfSense at all....

                              Do you still see outbound packets if you tcpdump on ix2 on pfSense in either of those setups?

                              Steve

                              1 Reply Last reply Reply Quote 0
                              • N
                                NRgia
                                last edited by NRgia

                                @stephenw10 said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                Those pcaps were on the mirror port again?

                                In both those pcaps there are no packets from pfSense at all....

                                Do you still see outbound packets if you tcpdump on ix2 on pfSense in either of those setups?

                                Steve

                                Ok I redone the tests:
                                First test:
                                Steps:

                                1. Set priority to 3 in pfSense to VLAN 20
                                2. Set Normal Priority in the switch to the port where the AP is connected
                                3. Set High priority in the switch to the port where pfSense LAN side is connected.
                                4. Done a tcpdump from pfSense.
                                  Results:
                                  tcpdump_priority_set_to pfsense_and_switch.txt

                                As you can see, only the priority (3) set in pfSense is intercepted.

                                Second test:

                                1. Removed priority 3 set in pfSense to VLAN 20
                                2. Left the Normal Priority in the switch to the port where the AP is connected
                                3. Left the High priority in the switch to the port where pfSense LAN side is connected.
                                4. Done a tcpdump from pfSense.

                                Results:
                                tcpdump_priority_set_to_switch.txt

                                I don't see anu priorities besides (p0) when done from the switch.

                                I hope the steps are ok, now.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Hmm, OK.

                                  Can I assume traffic was failing during those tests?
                                  We see no ARP replies to pfSense's queries and it looks like one way traffic from the clients.

                                  N 1 Reply Last reply Reply Quote 0
                                  • N
                                    NRgia @stephenw10
                                    last edited by NRgia

                                    @stephenw10 said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                    Hmm, OK.

                                    Can I assume traffic was failing during those tests?
                                    We see no ARP replies to pfSense's queries and it looks like one way traffic from the clients.

                                    Correct, I could communicate only via IP with pfSense. Only that worked.

                                    johnpozJ 1 Reply Last reply Reply Quote 0
                                    • johnpozJ
                                      johnpoz LAYER 8 Global Moderator @NRgia
                                      last edited by

                                      @nrgia said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                      Correct, I could communicate only via IP with pfSense. Only that worked.

                                      If you were not getting arp replies then wouldn't work even with IP..

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.03 | Lab VMs 2.7.2, 24.03

                                      N 1 Reply Last reply Reply Quote 0
                                      • N
                                        NRgia @johnpoz
                                        last edited by

                                        @johnpoz said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                        @nrgia said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                        Correct, I could communicate only via IP with pfSense. Only that worked.

                                        If you were not getting arp replies then wouldn't work even with IP..

                                        If you say so

                                        johnpozJ 1 Reply Last reply Reply Quote 0
                                        • johnpozJ
                                          johnpoz LAYER 8 Global Moderator @NRgia
                                          last edited by johnpoz

                                          @nrgia said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                          If you say so

                                          Not something making up dude - if you can not arp for the mac of something, it is impossible to talk to it even with an IP.

                                          Mac is what is used to talk to something on a network.. Just not possible if you do not have the mac to talk to.

                                          As Steve mentioned

                                          We see no ARP replies to pfSense's queries and it looks like one way traffic from the clients.

                                          Look at your arp table on your client, arp -a will show you the full table.. If there is no mac for an IP then your not going to talk to it.

                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                          If you get confused: Listen to the Music Play
                                          Please don't Chat/PM me for help, unless mod related
                                          SG-4860 24.03 | Lab VMs 2.7.2, 24.03

                                          N 1 Reply Last reply Reply Quote 0
                                          • N
                                            NRgia @johnpoz
                                            last edited by NRgia

                                            @johnpoz said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                            @nrgia said in pfSense 22.05 breaks VLANS, restoring pfSense 22.01 fixes the issue:

                                            If you say so

                                            Not something making up dude - if you can not arp for the mac of something, it is impossible to talk to it even with an IP.

                                            Mac is what is used to talk to something on a network.. Just not possible if you do not have the mac to talk to.

                                            As Steve mentioned

                                            We see no ARP replies to pfSense's queries and it looks like one way traffic from the clients.

                                            Look at your arp table on your client, arp -a will show you the full table.. If there is no mac for an IP then your not going to talk to it.

                                            The host, the switch and pfSense have static IP. If it did not work, how I could provide the tcpdump?
                                            As you can see the tcpdump is from ix2 interface,from pfSense, you can see that priority 3 set from pfSense.
                                            Is this discussion productive, in any way ? Do you think I'm lying ?

                                            johnpozJ 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post