• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

SG-3100 switch weird behavior (resolved)

Official Netgate® Hardware
sg-3100 switch arp syslog
3
85
18.9k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M
    mcury @mcury
    last edited by Oct 18, 2022, 8:43 PM

    I recorded, not sure if its going to be useful..
    arp_problem.zip

    dead on arrival, nowhere to be found.

    1 Reply Last reply Reply Quote 0
    • S
      stephenw10 Netgate Administrator
      last edited by Oct 18, 2022, 8:46 PM

      Hmm, well that seems very precise. Unlikely to be random then. The Marvell 88E6141 has a 2048 address MAC table. I'm going to assume you don't have >2000 devices!
      I guess it's feasible something could be generating random MAC continually and filling the table. You would see that in a pcap though.
      I can't find a value for a default expiry time. I'm not sure why it would expire at all.
      If the table were being reset I might imagine something else would be reset too. Are you seeing any other traffic interrupted at the 3min mark?

      Also can you confirm this is just unexpected, it's not actually failing to pass any traffic?

      Steve

      M 1 Reply Last reply Oct 18, 2022, 8:49 PM Reply Quote 0
      • M
        mcury @stephenw10
        last edited by mcury Oct 18, 2022, 8:50 PM Oct 18, 2022, 8:49 PM

        @stephenw10 said in SG-3100 switch weird behavior:

        Hmm, well that seems very precise. Unlikely to be random then. The Marvell 88E6141 has a 2048 address MAC table. I'm going to assume you don't have >2000 devices!

        Not even close.. 25 approximately..

        If the table were being reset I might imagine something else would be reset too. Are you seeing any other traffic interrupted at the 3min mark?

        No, everything is normal, I noticed it during a packet capture to check something else, otherwise I wouldn't even notice it..

        Also can you confirm this is just unexpected, it's not actually failing to pass any traffic?

        No, my network is running perfectly, the only issue is this, it seems that the marvell switch spams the packets to everyone since the mac table expired..
        I'm thinking here, maybe install something in the Raspberry Pi 4b to force it to use the internet every 3 minutes?
        Maybe a cron to run a single ping command?

        dead on arrival, nowhere to be found.

        1 Reply Last reply Reply Quote 0
        • S
          stephenw10 Netgate Administrator
          last edited by Oct 18, 2022, 8:57 PM

          Yes, that would correct it. Doesn't have to be to something external, it just needs to hit the switch in the 3100.
          You could set the ARP timeout in pfSense to <3mins. That way pfSense will ARP for the RasPi when it times out and the RasPi will respond refilling the switch table.
          That's an easy test:

          [22.11-DEVELOPMENT][admin@3100.stevew.lan]/root: sysctl net.link.ether.inet.max_age=120
          net.link.ether.inet.max_age: 1200 -> 120
          

          Steve

          M 1 Reply Last reply Oct 18, 2022, 9:02 PM Reply Quote 0
          • M
            mcury @stephenw10
            last edited by Oct 18, 2022, 9:02 PM

            @stephenw10 said in SG-3100 switch weird behavior:

            sysctl net.link.ether.inet.max_age=120

            done:

            [22.05-RELEASE][root@pfsense.home.arpa]/root: sysctl net.link.ether.inet.max_age=120
            net.link.ether.inet.max_age: 1200 -> 120
            [22.05-RELEASE][root@pfsense.home.arpa]/root: 
            

            dead on arrival, nowhere to be found.

            M 1 Reply Last reply Oct 18, 2022, 9:10 PM Reply Quote 0
            • M
              mcury @mcury
              last edited by Oct 18, 2022, 9:10 PM

              Strange, sometimes it doesn't take 3 minutes for the problem to happen.
              I'm not sure if the Marvell mac address is really expiring, or if the problem is something else..

              I reverted the change, to 1200 and configured a cron job in rpi4

              */1 * * * * /usr/bin/ping 192.168.255.249

              Lets see how that goes..

              dead on arrival, nowhere to be found.

              1 Reply Last reply Reply Quote 0
              • S
                stephenw10 Netgate Administrator
                last edited by stephenw10 Oct 18, 2022, 9:12 PM Oct 18, 2022, 9:11 PM

                I expect that to solve it.

                This has been interesting, I've never had to look into it too closely before. I can't find a specific value for the 3100 switch but for the switch in the 7100, which is from the same family of devices, the default MAC address aging time is 300s (5 mins). That has a larger table size so 3mins for the 3100 doesn't seem that unreasonable.

                I suspect this might be simply the traffic pattern you have to the RasPi4. The fact it's mostly UDP where it never sends a reply.

                Steve

                M 1 Reply Last reply Oct 18, 2022, 9:16 PM Reply Quote 1
                • M
                  mcury @stephenw10
                  last edited by Oct 18, 2022, 9:16 PM

                  @stephenw10 I believe that is the issue..
                  My cron is smashing pfsense with pings, let me change that cron to 1 minute

                  dead on arrival, nowhere to be found.

                  M 1 Reply Last reply Oct 18, 2022, 9:21 PM Reply Quote 0
                  • M
                    mcury @mcury
                    last edited by mcury Oct 18, 2022, 9:22 PM Oct 18, 2022, 9:21 PM

                    This seems to be enough..

                    * * * * * /usr/bin/ping 192.168.255.249 -c 2
                    

                    dead on arrival, nowhere to be found.

                    M 1 Reply Last reply Oct 18, 2022, 9:36 PM Reply Quote 0
                    • M
                      mcury @mcury
                      last edited by Oct 18, 2022, 9:36 PM

                      This solved the problem, 13 minutes of cron job running, no more problems..

                      Really thanks for the help @stephenw10 and @johnpoz :)

                      dead on arrival, nowhere to be found.

                      1 Reply Last reply Reply Quote 0
                      • S
                        stephenw10 Netgate Administrator
                        last edited by Oct 18, 2022, 9:55 PM

                        Cool. I think I prefer reducing the ARP timeout as a solution. You might try setting that to 1 min and see if that also solves it. That's just a system tunable in pfSense, all in the config.
                        But either will work fine.

                        Steve

                        M 1 Reply Last reply Oct 18, 2022, 10:00 PM Reply Quote 0
                        • M
                          mcury @stephenw10
                          last edited by mcury Oct 18, 2022, 10:01 PM Oct 18, 2022, 10:00 PM

                          @stephenw10 said in SG-3100 switch weird behavior (resolved):

                          Cool. I think I prefer reducing the ARP timeout as a solution.

                          Wouldn't that change the behavior for everything? Like a global setting?

                          This ARP timeout would only be triggered in case of a host is not "alive" like the raspberry pi 4b we just observed ?

                          dead on arrival, nowhere to be found.

                          1 Reply Last reply Reply Quote 0
                          • S
                            stephenw10 Netgate Administrator
                            last edited by Oct 18, 2022, 10:04 PM

                            It would be global but 1min is not that unusual. I believe Windows uses 30s.
                            What would happen is that every minute the RasPi4 entry in the pfSense ARP table would time out. So in order to send syslog traffic to it it will ARP for the IP address and the RasPi4 will respond to that refreshing the MAC table in the switch.
                            It feels like a cleaner solution to me but if the ping is working for you then there then there no need to change it. It would be interesting to know that works if you're able to test it.

                            Steve

                            M J 3 Replies Last reply Oct 18, 2022, 10:05 PM Reply Quote 0
                            • M
                              mcury @stephenw10
                              last edited by Oct 18, 2022, 10:05 PM

                              @stephenw10 said in SG-3100 switch weird behavior (resolved):

                              It would be interesting to know that works if you're able to test it.

                              Sure, I'll disable the cron job and test it right now, wireshark is already running, one sec

                              dead on arrival, nowhere to be found.

                              1 Reply Last reply Reply Quote 1
                              • J
                                johnpoz LAYER 8 Global Moderator @stephenw10
                                last edited by johnpoz Oct 18, 2022, 10:21 PM Oct 18, 2022, 10:14 PM

                                @stephenw10 said in SG-3100 switch weird behavior (resolved):

                                I believe Windows uses 30s.

                                windows uses a weird way of doing it, they use 30 seconds as base and then add a random multiplier on it.. But you can adjust it if you want.

                                https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/address-resolution-protocol-arp-caching-behavior

                                Wouldn't setting a static arp for the rpi4 also solve it? Or that is different than the switch arp cache?

                                I wish arp in windows actually showed you what was left on the cache, like freebsd, linux should do that too. At least in linux you can use ip -statistics neigh

                                I don't know of any way to actually view how much time is left on mac that is cached.

                                An intelligent man is sometimes forced to be drunk to spend time with his fools
                                If you get confused: Listen to the Music Play
                                Please don't Chat/PM me for help, unless mod related
                                SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                M S 2 Replies Last reply Oct 18, 2022, 10:18 PM Reply Quote 0
                                • M
                                  mcury @johnpoz
                                  last edited by Oct 18, 2022, 10:18 PM

                                  @johnpoz said in SG-3100 switch weird behavior (resolved):

                                  Wouldn't setting a static arp for the rpi4 also solve it? Or that is different than the switch arp cache?

                                  I tried that, actually the static ARP is set right now..

                                  login-to-view

                                  login-to-view

                                  dead on arrival, nowhere to be found.

                                  1 Reply Last reply Reply Quote 0
                                  • M
                                    mcury @stephenw10
                                    last edited by Oct 18, 2022, 10:22 PM

                                    @stephenw10 15 minutes and no problems..

                                    This also works:

                                    sysctl net.link.ether.inet.max_age=60
                                    

                                    dead on arrival, nowhere to be found.

                                    J 1 Reply Last reply Oct 18, 2022, 10:24 PM Reply Quote 0
                                    • J
                                      johnpoz LAYER 8 Global Moderator @mcury
                                      last edited by Oct 18, 2022, 10:24 PM

                                      @mcury so what is better solution? More arps going out for everything ;) Or just pinging pfsense from the rpi4 every minute or 2 minutes..

                                      Weird one for sure..

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                      M 1 Reply Last reply Oct 18, 2022, 10:25 PM Reply Quote 0
                                      • M
                                        mcury @johnpoz
                                        last edited by mcury Oct 18, 2022, 10:26 PM Oct 18, 2022, 10:25 PM

                                        @johnpoz said in SG-3100 switch weird behavior (resolved):

                                        so what is better solution? More arps going out for everything ;) Or just pinging pfsense from the rpi4 every minute or 2 minutes..
                                        Weird one for sure..

                                        ehhe, that is weird indeed.
                                        I'm not sure how it works, but it seems that every packet that goes through the switch reset that ARP timer, so the firewall wouldn't need to broadcast it as often.

                                        dead on arrival, nowhere to be found.

                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          stephenw10 Netgate Administrator @johnpoz
                                          last edited by Oct 18, 2022, 10:26 PM

                                          @johnpoz said in SG-3100 switch weird behavior (resolved):

                                          Wouldn't setting a static arp for the rpi4 also solve it? Or that is different than the switch arp cache?

                                          Yeah, nothing to do with the switch MAC table. That exists only in the switch IC.

                                          So in fact I would expect setting a static ARP to make this worse because it will never expire, pfSense will never ARP for the IP so no responses will be generated.

                                          So if it was still set static I'm surprised that max_age value made any difference. 🤔

                                          M 1 Reply Last reply Oct 18, 2022, 10:27 PM Reply Quote 0
                                          46 out of 85
                                          • First post
                                            46/85
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.