• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

SG-3100 switch weird behavior (resolved)

Official NetgateĀ® Hardware
sg-3100 switch arp syslog
3
85
19.0k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M
    mcury @mcury
    last edited by Oct 18, 2022, 3:49 PM

    I'll reinstall my pfsense from scratch to test, but I can't do it right now.
    Then, if the problem happens again, I'll replace this switch.

    dead on arrival, nowhere to be found.

    M 1 Reply Last reply Oct 18, 2022, 3:54 PM Reply Quote 0
    • M
      mcury @mcury
      last edited by Oct 18, 2022, 3:54 PM

      So weird man, a ping fixes it temporarily..
      login-to-view

      dead on arrival, nowhere to be found.

      1 Reply Last reply Reply Quote 0
      • J
        johnpoz LAYER 8 Global Moderator @mcury
        last edited by johnpoz Oct 18, 2022, 3:56 PM Oct 18, 2022, 3:55 PM

        @mcury yeah you shouldn't be seeing those. Hmmmm Even if your nic was in promiscuous mode, that mac shouldn't be sent down the port where the mac is not listed.

        If you had some sort of leak or bridge where the mac was being learned on multiple interfaces that could happen..

        So proper destination mac is your down the trunk (lan4) to the flex mini. But pfsense is also sending it out lan1? But the only place the mac of that pi4 should be seen by pfsense is the lan4 interface, it should never send that mac out lan1, unless there was bridge setup.

        hmmmm strange...

        This a good question for @stephenw10 he would know way more than me on the inter workings of the switch in the 3100. But typically a switch would only send traffic down the interface that the mac is on.

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.7.2, 24.11

        M 1 Reply Last reply Oct 18, 2022, 3:59 PM Reply Quote 0
        • M
          mcury @johnpoz
          last edited by Oct 18, 2022, 3:59 PM

          @johnpoz Exactly, its so weird, that packet should never go to pfsense's LAN1 ..

          I'll try to fix it tonight by reinstalling my pfsense from scratch..
          Then, if the problem happens again, I'll replace this switch..

          dead on arrival, nowhere to be found.

          1 Reply Last reply Reply Quote 0
          • S
            stephenw10 Netgate Administrator
            last edited by Oct 18, 2022, 4:02 PM

            Yeah, it's a pretty basic switch and there's no control over things like the MAC table. That's the only thing I could imagine causing that though.

            If you haven't already try power cycling the 3100 entirely. That should completely reset the switch if it's somehow managed to toggle some flag.

            Steve

            M 1 Reply Last reply Oct 18, 2022, 4:03 PM Reply Quote 0
            • M
              mcury @stephenw10
              last edited by Oct 18, 2022, 4:03 PM

              @stephenw10 hm, I'll try it now a shutdown, remove the power cable, one sec, let me see who is here using the Internet

              dead on arrival, nowhere to be found.

              M 1 Reply Last reply Oct 18, 2022, 4:10 PM Reply Quote 0
              • M
                mcury @mcury
                last edited by Oct 18, 2022, 4:10 PM

                Done, the problem persists..

                1. Halt system and once the shutdown process ended, removed the power cable for a few seconds.

                login-to-view

                login-to-view

                dead on arrival, nowhere to be found.

                1 Reply Last reply Reply Quote 0
                • S
                  stephenw10 Netgate Administrator
                  last edited by stephenw10 Oct 18, 2022, 4:18 PM Oct 18, 2022, 4:17 PM

                  Hmm, the only other thing I could imagine causing this is if something feeding bad data into the switch MAC table. That would have to be the desktop machine.

                  If you run a continuous ping from the RasPi to somewhere that has to be accessed through the 3100 switch, does that prevent the issue?

                  If it does I'd try to find something sending the RasPi MAC from the desktop. Hard to say what that might be.... something reflected perhaps?

                  If you run a pcap on the desktop and filter by the RasPi MAC address whilst the problem is not happening and wait for it to start. The first thing that happens there might be the offending packet.

                  Steve

                  M 1 Reply Last reply Oct 18, 2022, 4:50 PM Reply Quote 0
                  • M
                    mcury @stephenw10
                    last edited by mcury Oct 18, 2022, 5:00 PM Oct 18, 2022, 4:50 PM

                    @stephenw10 said in SG-3100 switch weird behavior:

                    If you run a continuous ping from the RasPi to somewhere that has to be accessed through the 3100 switch, does that prevent the issue?

                    Testing now, ping is running from RPI4 to pfsense.
                    It seems to have stopped, but it may start again soon, so I'll wait a little longer this time.

                    Packet capture set:
                    login-to-view

                    Edit:

                    This is my ARP table (desktop)

                    $ cat /proc/net/arp
                    IP address       HW type     Flags       HW address            Mask     Device
                    192.168.255.252  0x1         0x2         00:11:32:9f:ee:93     *        enp7s0
                    192.168.255.249  0x1         0x2         00:08:a2:0c:c4:1c     *        enp7s0
                    192.168.255.250  0x1         0x2         b8:27:eb:ea:f8:65     *        enp7s0
                    192.168.255.253  0x1         0x2         dc:a6:32:a5:47:19     *        enp7s0
                    

                    dead on arrival, nowhere to be found.

                    M 1 Reply Last reply Oct 18, 2022, 5:30 PM Reply Quote 0
                    • M
                      mcury @mcury
                      last edited by mcury Oct 18, 2022, 5:30 PM Oct 18, 2022, 5:30 PM

                      40 minutes pinging from raspberry pi 4b (192.168.255.253) to pfsense (192.168.255.249) and no problem so far.

                      I have two wireshark windows opened, one monitoring:
                      eth.src == dc:a6:32:a5:47:19 and not tcp.port == 22 and not tcp.port == 9000

                      And the second one monitoring:
                      ip.addr == 192.168.255.253 and not tcp.port == 9000 and not tcp.port == 22

                      dead on arrival, nowhere to be found.

                      M 1 Reply Last reply Oct 18, 2022, 5:47 PM Reply Quote 0
                      • M
                        mcury @mcury
                        last edited by mcury Oct 18, 2022, 5:48 PM Oct 18, 2022, 5:47 PM

                        Dropped the ping and one minute later (or less), the problem starts again:

                        login-to-view

                        desktop ARP table:
                        $ cat /proc/net/arp
                        IP address HW type Flags HW address Mask Device
                        192.168.255.252 0x1 0x2 00:11:32:9f:ee:93 * enp7s0
                        192.168.255.249 0x1 0x2 00:08:a2:0c:c4:1c * enp7s0
                        192.168.255.250 0x1 0x2 b8:27:eb:ea:f8:65 * enp7s0
                        192.168.255.253 0x1 0x2 dc:a6:32:a5:47:19 * enp7s0

                        dead on arrival, nowhere to be found.

                        1 Reply Last reply Reply Quote 0
                        • S
                          stephenw10 Netgate Administrator
                          last edited by Oct 18, 2022, 7:01 PM

                          Hmm, so nothing from the RasPi MAC address at the desktop that might be inserting invalid entries into the switch.

                          It might be worth re-running that test using the RasPi MAC as destination in the filter (or as either).
                          You might catch something arriving using that but a different IP address.

                          Also when this happens do you see traffic being sent only to the desktop? Or is the syslog traffic sent to all the 3100 switch ports? Does it also arrive at the RasPi?

                          Steve

                          M 1 Reply Last reply Oct 18, 2022, 7:17 PM Reply Quote 0
                          • M
                            mcury @stephenw10
                            last edited by Oct 18, 2022, 7:17 PM

                            @stephenw10 It seems that its only going to LAN1..

                            login-to-view

                            raspberry pi 3 in which you see the tcpdump above is connected to the switch unifi mini.

                            Let me perform this test again, but in the NAS which is connected to LAN2 of pfsense, one sec.

                            dead on arrival, nowhere to be found.

                            M 1 Reply Last reply Oct 18, 2022, 7:21 PM Reply Quote 0
                            • M
                              mcury @mcury
                              last edited by mcury Oct 18, 2022, 7:42 PM Oct 18, 2022, 7:21 PM

                              Hmmm, its going to port LAN2 of pfsense too:
                              NAS IP is 192.168.255.252 (tcpdump) (LAN2 of pfsense)

                              On the right, wireshark running on desktop (LAN1 of pfsense)

                              login-to-view

                              dead on arrival, nowhere to be found.

                              1 Reply Last reply Reply Quote 0
                              • S
                                stephenw10 Netgate Administrator
                                last edited by Oct 18, 2022, 8:17 PM

                                Aha, interesting. You wouldn't expect so see it on one of the other Unifi swtch ports because it should only send it out of the port that MAC is connected to. So to the RasPi4 there.

                                The same should be true of the switch in the 3100 The fact it seems to be sending it to all ports implies that it no longer has a an entry for the MAC address in it's table. If it was an incorrect entry as I speculated earlier then it would only send from port 1.
                                Because that traffic is UDP with no replies it never sees any traffic from the RasPi4 to repopulate the table. Is the RasPi configured with a static IP?

                                It seems unexpected that the table entry has expired though. How long does it take to fail after sending some pings approximately?

                                Steve

                                J M 2 Replies Last reply Oct 18, 2022, 8:24 PM Reply Quote 0
                                • J
                                  johnpoz LAYER 8 Global Moderator @stephenw10
                                  last edited by Oct 18, 2022, 8:24 PM

                                  @stephenw10 said in SG-3100 switch weird behavior:

                                  How long does it take to fail after sending some pings approximately?

                                  the default cache in pfsense is like 20 minutes, but maybe not for the switch mac table? Is there anyway to view the switches mac address table?

                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                  If you get confused: Listen to the Music Play
                                  Please don't Chat/PM me for help, unless mod related
                                  SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                  M 1 Reply Last reply Oct 18, 2022, 8:26 PM Reply Quote 0
                                  • M
                                    mcury @stephenw10
                                    last edited by Oct 18, 2022, 8:26 PM

                                    @stephenw10 said in SG-3100 switch weird behavior:

                                    Because that traffic is UDP with no replies it never sees any traffic from the RasPi4 to repopulate the table. Is the RasPi configured with a static IP?

                                    raspberry pi 4b its on dhcp, no services running on it, only graylog, which means that the device only receives UDP data.

                                    It seems unexpected that the table entry has expired though. How long does it take to fail after sending some pings approximately?

                                    I'll try to get that info right now.

                                    dead on arrival, nowhere to be found.

                                    1 Reply Last reply Reply Quote 0
                                    • M
                                      mcury @johnpoz
                                      last edited by Oct 18, 2022, 8:26 PM

                                      @johnpoz said in SG-3100 switch weird behavior:

                                      the default cache in pfsense is like 20 minutes, but maybe not for the switch mac table? Is there anyway to view the switches mac address table?

                                      I'm really missing my old Cisco days, show mac-address table vlan x :)

                                      dead on arrival, nowhere to be found.

                                      J 1 Reply Last reply Oct 18, 2022, 8:28 PM Reply Quote 0
                                      • J
                                        johnpoz LAYER 8 Global Moderator @mcury
                                        last edited by Oct 18, 2022, 8:28 PM

                                        @mcury hehe - yeah would be easy to see then.. Why I like my routers with interfaces, leave the switch ports to the actual switches ;)

                                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                                        If you get confused: Listen to the Music Play
                                        Please don't Chat/PM me for help, unless mod related
                                        SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                        M 1 Reply Last reply Oct 18, 2022, 8:31 PM Reply Quote 1
                                        • M
                                          mcury @johnpoz
                                          last edited by Oct 18, 2022, 8:31 PM

                                          @johnpoz said in SG-3100 switch weird behavior:

                                          @mcury hehe - yeah would be easy to see then.. Why I like my routers with interfaces, leave the switch ports to the actual switches ;)

                                          :) Yes, you have a point there ehhe

                                          tcpdump in pfsense during the DHCP negotiation with raspberry pi 4b

                                          login-to-view

                                          login-to-view

                                          dead on arrival, nowhere to be found.

                                          1 Reply Last reply Reply Quote 0
                                          24 out of 85
                                          • First post
                                            24/85
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.