Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-3100 switch weird behavior (resolved)

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    sg-3100switcharpsyslog
    85 Posts 3 Posters 20.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mcury @johnpoz
      last edited by

      @johnpoz said in SG-3100 switch weird behavior:

      the default cache in pfsense is like 20 minutes, but maybe not for the switch mac table? Is there anyway to view the switches mac address table?

      I'm really missing my old Cisco days, show mac-address table vlan x :)

      dead on arrival, nowhere to be found.

      johnpozJ 1 Reply Last reply Reply Quote 0
      • johnpozJ
        johnpoz LAYER 8 Global Moderator @mcury
        last edited by

        @mcury hehe - yeah would be easy to see then.. Why I like my routers with interfaces, leave the switch ports to the actual switches ;)

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        M 1 Reply Last reply Reply Quote 1
        • M
          mcury @johnpoz
          last edited by

          @johnpoz said in SG-3100 switch weird behavior:

          @mcury hehe - yeah would be easy to see then.. Why I like my routers with interfaces, leave the switch ports to the actual switches ;)

          :) Yes, you have a point there ehhe

          tcpdump in pfsense during the DHCP negotiation with raspberry pi 4b

          593529d9-b4bd-4207-b57d-99248f0dc9c8-image.png

          f2d379a3-e181-4362-b519-7fb40ba9a350-image.png

          dead on arrival, nowhere to be found.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Yeah the pfSense ARP cache expiry time is completely independent of the switch MAC table. I don't believe there's any way to query the switch IC for the table or for the expiry time.

            Steve

            M 1 Reply Last reply Reply Quote 0
            • M
              mcury @stephenw10
              last edited by

              @stephenw10 said in SG-3100 switch weird behavior:

              Yeah the pfSense ARP cache expiry time is completely independent of the switch MAC table. I don't believe there's any way to query the switch IC for the table or for the expiry time.

              Steve

              3 minutes exactly.

              dead on arrival, nowhere to be found.

              M 1 Reply Last reply Reply Quote 0
              • M
                mcury @mcury
                last edited by

                I recorded, not sure if its going to be useful..
                arp_problem.zip

                dead on arrival, nowhere to be found.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Hmm, well that seems very precise. Unlikely to be random then. The Marvell 88E6141 has a 2048 address MAC table. I'm going to assume you don't have >2000 devices!
                  I guess it's feasible something could be generating random MAC continually and filling the table. You would see that in a pcap though.
                  I can't find a value for a default expiry time. I'm not sure why it would expire at all.
                  If the table were being reset I might imagine something else would be reset too. Are you seeing any other traffic interrupted at the 3min mark?

                  Also can you confirm this is just unexpected, it's not actually failing to pass any traffic?

                  Steve

                  M 1 Reply Last reply Reply Quote 0
                  • M
                    mcury @stephenw10
                    last edited by mcury

                    @stephenw10 said in SG-3100 switch weird behavior:

                    Hmm, well that seems very precise. Unlikely to be random then. The Marvell 88E6141 has a 2048 address MAC table. I'm going to assume you don't have >2000 devices!

                    Not even close.. 25 approximately..

                    If the table were being reset I might imagine something else would be reset too. Are you seeing any other traffic interrupted at the 3min mark?

                    No, everything is normal, I noticed it during a packet capture to check something else, otherwise I wouldn't even notice it..

                    Also can you confirm this is just unexpected, it's not actually failing to pass any traffic?

                    No, my network is running perfectly, the only issue is this, it seems that the marvell switch spams the packets to everyone since the mac table expired..
                    I'm thinking here, maybe install something in the Raspberry Pi 4b to force it to use the internet every 3 minutes?
                    Maybe a cron to run a single ping command?

                    dead on arrival, nowhere to be found.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Yes, that would correct it. Doesn't have to be to something external, it just needs to hit the switch in the 3100.
                      You could set the ARP timeout in pfSense to <3mins. That way pfSense will ARP for the RasPi when it times out and the RasPi will respond refilling the switch table.
                      That's an easy test:

                      [22.11-DEVELOPMENT][admin@3100.stevew.lan]/root: sysctl net.link.ether.inet.max_age=120
                      net.link.ether.inet.max_age: 1200 -> 120
                      

                      Steve

                      M 1 Reply Last reply Reply Quote 0
                      • M
                        mcury @stephenw10
                        last edited by

                        @stephenw10 said in SG-3100 switch weird behavior:

                        sysctl net.link.ether.inet.max_age=120

                        done:

                        [22.05-RELEASE][root@pfsense.home.arpa]/root: sysctl net.link.ether.inet.max_age=120
                        net.link.ether.inet.max_age: 1200 -> 120
                        [22.05-RELEASE][root@pfsense.home.arpa]/root: 
                        

                        dead on arrival, nowhere to be found.

                        M 1 Reply Last reply Reply Quote 0
                        • M
                          mcury @mcury
                          last edited by

                          Strange, sometimes it doesn't take 3 minutes for the problem to happen.
                          I'm not sure if the Marvell mac address is really expiring, or if the problem is something else..

                          I reverted the change, to 1200 and configured a cron job in rpi4

                          */1 * * * * /usr/bin/ping 192.168.255.249

                          Lets see how that goes..

                          dead on arrival, nowhere to be found.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by stephenw10

                            I expect that to solve it.

                            This has been interesting, I've never had to look into it too closely before. I can't find a specific value for the 3100 switch but for the switch in the 7100, which is from the same family of devices, the default MAC address aging time is 300s (5 mins). That has a larger table size so 3mins for the 3100 doesn't seem that unreasonable.

                            I suspect this might be simply the traffic pattern you have to the RasPi4. The fact it's mostly UDP where it never sends a reply.

                            Steve

                            M 1 Reply Last reply Reply Quote 1
                            • M
                              mcury @stephenw10
                              last edited by

                              @stephenw10 I believe that is the issue..
                              My cron is smashing pfsense with pings, let me change that cron to 1 minute

                              dead on arrival, nowhere to be found.

                              M 1 Reply Last reply Reply Quote 0
                              • M
                                mcury @mcury
                                last edited by mcury

                                This seems to be enough..

                                * * * * * /usr/bin/ping 192.168.255.249 -c 2
                                

                                dead on arrival, nowhere to be found.

                                M 1 Reply Last reply Reply Quote 0
                                • M
                                  mcury @mcury
                                  last edited by

                                  This solved the problem, 13 minutes of cron job running, no more problems..

                                  Really thanks for the help @stephenw10 and @johnpoz :)

                                  dead on arrival, nowhere to be found.

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Cool. I think I prefer reducing the ARP timeout as a solution. You might try setting that to 1 min and see if that also solves it. That's just a system tunable in pfSense, all in the config.
                                    But either will work fine.

                                    Steve

                                    M 1 Reply Last reply Reply Quote 0
                                    • M
                                      mcury @stephenw10
                                      last edited by mcury

                                      @stephenw10 said in SG-3100 switch weird behavior (resolved):

                                      Cool. I think I prefer reducing the ARP timeout as a solution.

                                      Wouldn't that change the behavior for everything? Like a global setting?

                                      This ARP timeout would only be triggered in case of a host is not "alive" like the raspberry pi 4b we just observed ?

                                      dead on arrival, nowhere to be found.

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        It would be global but 1min is not that unusual. I believe Windows uses 30s.
                                        What would happen is that every minute the RasPi4 entry in the pfSense ARP table would time out. So in order to send syslog traffic to it it will ARP for the IP address and the RasPi4 will respond to that refreshing the MAC table in the switch.
                                        It feels like a cleaner solution to me but if the ping is working for you then there then there no need to change it. It would be interesting to know that works if you're able to test it.

                                        Steve

                                        M johnpozJ 3 Replies Last reply Reply Quote 0
                                        • M
                                          mcury @stephenw10
                                          last edited by

                                          @stephenw10 said in SG-3100 switch weird behavior (resolved):

                                          It would be interesting to know that works if you're able to test it.

                                          Sure, I'll disable the cron job and test it right now, wireshark is already running, one sec

                                          dead on arrival, nowhere to be found.

                                          1 Reply Last reply Reply Quote 1
                                          • johnpozJ
                                            johnpoz LAYER 8 Global Moderator @stephenw10
                                            last edited by johnpoz

                                            @stephenw10 said in SG-3100 switch weird behavior (resolved):

                                            I believe Windows uses 30s.

                                            windows uses a weird way of doing it, they use 30 seconds as base and then add a random multiplier on it.. But you can adjust it if you want.

                                            https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/address-resolution-protocol-arp-caching-behavior

                                            Wouldn't setting a static arp for the rpi4 also solve it? Or that is different than the switch arp cache?

                                            I wish arp in windows actually showed you what was left on the cache, like freebsd, linux should do that too. At least in linux you can use ip -statistics neigh

                                            I don't know of any way to actually view how much time is left on mac that is cached.

                                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                                            If you get confused: Listen to the Music Play
                                            Please don't Chat/PM me for help, unless mod related
                                            SG-4860 24.11 | Lab VMs 2.8, 24.11

                                            M stephenw10S 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.