Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-1000 VLANs not working unless PROMISC is set

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    16 Posts 2 Posters 861 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      Rocco83
      last edited by

      About the supported mode,
      I was effectively using it.
      Then i have upgraded && possibly changed something, and pfsense got broken.
      So it was at least working :)

      I have made a further test, and this really seems not linked to the LAGG configuration.
      I have removed cpsw1 from the lagg, and moved to another switch physical port.
      Added the VLAN on top of it, and assigned to a network interface in pfsense.

      [root@pf2-tos ~]# ping 172.16.82.241
      PING 172.16.82.241 (172.16.82.241): 56 data bytes
      ^C
      --- 172.16.82.241 ping statistics ---
      4 packets transmitted, 0 packets received, 100.0% packet loss
      [root@pf2-tos ~]# ifconfig cpsw1 promisc
      [root@pf2-tos ~]# ping 172.16.82.241
      PING 172.16.82.241 (172.16.82.241): 56 data bytes
      64 bytes from 172.16.82.241: icmp_seq=0 ttl=64 time=1.447 ms
      64 bytes from 172.16.82.241: icmp_seq=1 ttl=64 time=0.697 ms
      ^C
      --- 172.16.82.241 ping statistics ---
      2 packets transmitted, 2 packets received, 0.0% packet loss
      round-trip min/avg/max/stddev = 0.697/1.072/1.447/0.375 ms
      [root@pf2-tos ~]#

      [root@pf2-tos ~]# ifconfig cpsw1
      cpsw1: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500
      options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
      ether c8:df:84:c1:16:39
      hwaddr c8:df:84:c1:16:39
      inet6 fe80::cadf:84ff:fec1:1639%cpsw1 prefixlen 64 scopeid 0x2
      media: Ethernet autoselect (1000baseT <full-duplex>)
      status: active
      nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      [root@pf2-tos ~]# ifconfig cpsw1.12
      cpsw1.12: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
      options=80000<LINKSTATE>
      ether c8:df:84:c1:16:39
      inet6 fe80::cadf:84ff:fec1:1639%cpsw1.12 prefixlen 64 scopeid 0x10
      inet 172.16.82.242 netmask 0xffffff00 broadcast 172.16.82.255
      groups: vlan
      vlan: 12 vlanpcp: 0 parent interface: cpsw1
      media: Ethernet autoselect (1000baseT <full-duplex>)
      status: active
      nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      [root@pf2-tos ~]#

      So seems not bonded to the bonding configuration (ah, i like stupid wording jokes :D)

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Ah, interesting. Do you know what version you were running when it was working as expected?

        Steve

        1 Reply Last reply Reply Quote 0
        • R
          Rocco83
          last edited by

          I recall that the first issue was back in Apr/2019.
          According to https://docs.netgate.com/pfsense/en/latest/releases/versions-of-pfsense-and-freebsd.html,
          I was moving from 2.4.4-p1 to 2.4.4-p2.
          I'm now on 2.4.4-p3.

          But, I'm not 100% sure about it...

          Back to https://redmine.pfsense.org/issues/7645, what was the changelog for the issue?
          Just wondering to identify if there is a regression.

          Thanks,
          Daniele

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Just spent a while looking for it and failed to track it down. I'll have to ask someone who might know directly.

            1 Reply Last reply Reply Quote 0
            • R
              Rocco83
              last edited by

              Hi there,

              Any news?

              Thank you very much,
              Daniele

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                I've been unable to replicate this in 2.4.4p3 or 2.5.

                [2.4.4-RELEASE][root@ufw3.stevew.lan]/root: ifconfig
cpsw0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether 0c:b2:b7:af:2f:4f
        hwaddr 0c:b2:b7:af:2f:4f
        inet6 fe80::eb2:b7ff:feaf:2f4f%cpsw0 prefixlen 64 scopeid 0x1 
        inet 172.21.16.80 netmask 0xffffff00 broadcast 172.21.16.255 
        media: Ethernet autoselect (1000baseT <full-duplex,master>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
cpsw1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether 0c:b2:b7:af:2f:51
        hwaddr 0c:b2:b7:af:2f:51
        inet 192.168.80.1 netmask 0xffffff00 broadcast 192.168.80.255 
        inet6 fe80::1:1%cpsw1 prefixlen 64 duplicated scopeid 0x2 
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
        groups: enc 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 
        inet 127.0.0.1 netmask 0xff000000 
        groups: lo 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pfsync0: flags=0<> metric 0 mtu 1500
        syncpeer: 224.0.0.240 maxupd: 128 defer: on
        syncok: 1
        groups: pfsync 
pflog0: flags=100<PROMISC> metric 0 mtu 33184
        groups: pflog 
cpsw1.50: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 0c:b2:b7:af:2f:51
        inet6 fe80::eb2:b7ff:feaf:2f51%cpsw1.50 prefixlen 64 scopeid 0x7 
        inet 172.18.10.11 netmask 0xffffff00 broadcast 172.18.10.255 
        groups: vlan 
        vlan: 50 vlanpcp: 0 parent interface: cpsw1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

[2.4.4-RELEASE][root@ufw3.stevew.lan]/root: ping 172.18.10.1
PING 172.18.10.1 (172.18.10.1): 56 data bytes
64 bytes from 172.18.10.1: icmp_seq=0 ttl=64 time=0.621 ms
64 bytes from 172.18.10.1: icmp_seq=1 ttl=64 time=0.972 ms
64 bytes from 172.18.10.1: icmp_seq=2 ttl=64 time=0.491 ms
^C
--- 172.18.10.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.491/0.695/0.972/0.203 ms

                Capturing at the other side:

                16:36:42.590615 0c:b2:b7:af:2f:51 > 00:90:0b:76:8e:52, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 9584, offset 0, flags [none], proto ICMP (1), length 28)
    172.18.10.11 > 172.18.10.1: ICMP echo request, id 52423, seq 628, length 8
16:36:42.590667 00:90:0b:76:8e:52 > 0c:b2:b7:af:2f:51, ethertype 802.1Q (0x8100), length 46: vlan 50, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 59789, offset 0, flags [none], proto ICMP (1), length 28)
    172.18.10.1 > 172.18.10.11: ICMP echo reply, id 52423, seq 628, length 8

                Requiring promiscuous mode like that usually implies the wrong MAC address or at least an unexpected MAC. I wonder if the lagg on there altered it for that driver?

                Steve

                R 1 Reply Last reply Reply Quote 0
                • R
                  Rocco83 @stephenw10
                  last edited by

                  @stephenw10 lagg has been not configured. Without lagg I confirm that this is working

                  My lagg is configred with lacp

                  Thanks,

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Oh, OK so this only a problem over lagg? In your previous post it looked like you removed cpsw1 from the lagg and a VLAN on that port still only worked with promiscuous mode enabled.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • R
                      Rocco83
                      last edited by

                      Shame on myself,

                      I have made a further test, and this really seems not linked to the LAGG configuration.
                      So, i'll try to upgrade it from scratch and let's see.

                      I will keep you posted.

                      Thanks,
                      Daniele

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Ah, OK, so possibly an issue with the test outside of the lagg?

                        I'll see if I can replicate the lagg issue.

                        Steve

                        1 Reply Last reply Reply Quote 0
                        • R
                          Rocco83
                          last edited by

                          Fast forward to Q3 2021, finally i had some time to reproduce everything.

                          • resetted the SG-1000, installed the pfsense plus 21.02.2
                          • bringed up ssh
                          • create a rule to allow everything in floating (with quick)
                          • NOT configured the lagg
                          • LAN on cpsw1
                          • setup one vlan per NIC (cpsw0.10, cpsw1.11)

                          tests:

                          1. reboot
                          2. ping is working on LAN (untagged) but not not VLANs (tagged)
                          3. ifconfig promisc on cpsw0
                          4. ping is working on both cpsw0.10 AND cpsw1.11
                          5. reboot
                          6. ping is working on LAN (untagged) but not not VLANs (tagged)
                          7. ifconfig promisc on cpsw1
                          8. ping is working on both cpsw0.10 AND cpsw1.11

                          reproducer:

                          *** Welcome to Netgate pfSense Plus 21.02.2-RELEASE (arm) on pf2-tos ***

 WAN (wan)       -> cpsw1.11   -> v4: 172.16.81.242/24
 LAN (lan)       -> cpsw1      -> v4: 172.16.8.242/24
 IOT (opt1)      -> cpsw0.10   -> v4: 172.16.80.242/24

                          initial status after boot, ping not working on VLANs, but working on main LAN interface

                          [21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.8.241
PING 172.16.8.241 (172.16.8.241): 56 data bytes
64 bytes from 172.16.8.241: icmp_seq=0 ttl=64 time=1.292 ms
64 bytes from 172.16.8.241: icmp_seq=1 ttl=64 time=0.738 ms

--- 172.16.8.241 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.738/1.015/1.292/0.277 ms 
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.80.241
PING 172.16.80.241 (172.16.80.241): 56 data bytes

--- 172.16.80.241 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.81.241
PING 172.16.81.241 (172.16.81.241): 56 data bytes

--- 172.16.81.241 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root:

                          Setting promisc on cpsw0, ping working on both interfaces

                          [21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ifconfig cpsw0 promisc
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.80.241
PING 172.16.80.241 (172.16.80.241): 56 data bytes
64 bytes from 172.16.80.241: icmp_seq=0 ttl=64 time=1.501 ms
64 bytes from 172.16.80.241: icmp_seq=1 ttl=64 time=0.889 ms

--- 172.16.80.241 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.889/1.195/1.501/0.306 ms
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.81.241
PING 172.16.81.241 (172.16.81.241): 56 data bytes
64 bytes from 172.16.81.241: icmp_seq=0 ttl=64 time=1.062 ms
64 bytes from 172.16.81.241: icmp_seq=1 ttl=64 time=0.892 ms

--- 172.16.81.241 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.892/0.977/1.062/0.085 ms
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root:

                          reboot, to enable promisc on cpsw1

                          ping not working at boot

                          21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.8.241
PING 172.16.8.241 (172.16.8.241): 56 data bytes
64 bytes from 172.16.8.241: icmp_seq=0 ttl=64 time=1.309 ms
64 bytes from 172.16.8.241: icmp_seq=1 ttl=64 time=0.713 ms

--- 172.16.8.241 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.713/1.011/1.309/0.298 ms
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.80.241
PING 172.16.80.241 (172.16.80.241): 56 data bytes

--- 172.16.80.241 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.81.241
PING 172.16.81.241 (172.16.81.241): 56 data bytes

--- 172.16.81.241 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root:

                          ifconfig cpsw1 promisc, ping working

                          [21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ifconfig cpsw1 promisc
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.80.241
PING 172.16.80.241 (172.16.80.241): 56 data bytes
64 bytes from 172.16.80.241: icmp_seq=0 ttl=64 time=1.429 ms
64 bytes from 172.16.80.241: icmp_seq=1 ttl=64 time=0.756 ms

--- 172.16.80.241 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.756/1.092/1.429/0.337 ms
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ping -c2 172.16.81.241
PING 172.16.81.241 (172.16.81.241): 56 data bytes
64 bytes from 172.16.81.241: icmp_seq=0 ttl=64 time=0.993 ms
64 bytes from 172.16.81.241: icmp_seq=1 ttl=64 time=0.823 ms

--- 172.16.81.241 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.823/0.908/0.993/0.085 ms
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root:

                          settings:

                          [21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ifconfig cpsw0
cpsw0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether c8:df:84:c1:16:37
        inet6 fe80::cadf:84ff:fec1:1637%cpsw0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ifconfig cpsw1
cpsw1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether c8:df:84:c1:16:39
        inet6 fe80::cadf:84ff:fec1:1639%cpsw1 prefixlen 64 scopeid 0x2
        inet 172.16.8.242 netmask 0xffffff00 broadcast 172.16.8.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ifconfig cpsw1.11
cpsw1.11: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: WAN
        options=80000<LINKSTATE>
        ether c8:df:84:c1:16:39
        inet6 fe80::cadf:84ff:fec1:1639%cpsw1.11 prefixlen 64 scopeid 0x7
        inet 172.16.81.242 netmask 0xffffff00 broadcast 172.16.81.255
        groups: vlan
        vlan: 11 vlanpcp: 0 parent interface: cpsw1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root: ifconfig cpsw0.10
cpsw0.10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: IOT
        options=80000<LINKSTATE>
        ether c8:df:84:c1:16:37
        inet6 fe80::cadf:84ff:fec1:1637%cpsw0.10 prefixlen 64 scopeid 0x8
        inet 172.16.80.242 netmask 0xffffff00 broadcast 172.16.80.255
        groups: vlan
        vlan: 10 vlanpcp: 0 parent interface: cpsw0
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[21.02.2-RELEASE][root@pf2-tos.gonzaga.retaggio.net]/root:

                          This could be easily reproduced, in theory...

                          1 Reply Last reply Reply Quote 0
                          • R
                            Rocco83
                            last edited by Rocco83

                            On the other side, the tcpdump show:

                            [2.4.5-RELEASE][admin@pf1-tos.domain]/root: tcpdump -i xn0.10 ether host c8:df:84:c1:16:37 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on xn0.10, link-type EN10MB (Ethernet), capture size 262144 bytes
18:05:55.841165 ARP, Request who-has 172.16.80.241 tell 172.16.80.242, length 42
18:05:55.841190 ARP, Reply 172.16.80.241 is-at 00:16:3e:19:25:5f, length 28
18:05:56.850828 ARP, Request who-has 172.16.80.241 tell 172.16.80.242, length 42
18:05:56.850855 ARP, Reply 172.16.80.241 is-at 00:16:3e:19:25:5f, length 28

                            i can't run it on the origin otherwise the ping start working...

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.