• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

SG-3100 switch weird behavior (resolved)

Official Netgate® Hardware
sg-3100 switch arp syslog
3
85
19.1k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M
    mcury @stephenw10
    last edited by Oct 18, 2022, 10:22 PM

    @stephenw10 15 minutes and no problems..

    This also works:

    sysctl net.link.ether.inet.max_age=60
    

    dead on arrival, nowhere to be found.

    J 1 Reply Last reply Oct 18, 2022, 10:24 PM Reply Quote 0
    • J
      johnpoz LAYER 8 Global Moderator @mcury
      last edited by Oct 18, 2022, 10:24 PM

      @mcury so what is better solution? More arps going out for everything ;) Or just pinging pfsense from the rpi4 every minute or 2 minutes..

      Weird one for sure..

      An intelligent man is sometimes forced to be drunk to spend time with his fools
      If you get confused: Listen to the Music Play
      Please don't Chat/PM me for help, unless mod related
      SG-4860 24.11 | Lab VMs 2.7.2, 24.11

      M 1 Reply Last reply Oct 18, 2022, 10:25 PM Reply Quote 0
      • M
        mcury @johnpoz
        last edited by mcury Oct 18, 2022, 10:26 PM Oct 18, 2022, 10:25 PM

        @johnpoz said in SG-3100 switch weird behavior (resolved):

        so what is better solution? More arps going out for everything ;) Or just pinging pfsense from the rpi4 every minute or 2 minutes..
        Weird one for sure..

        ehhe, that is weird indeed.
        I'm not sure how it works, but it seems that every packet that goes through the switch reset that ARP timer, so the firewall wouldn't need to broadcast it as often.

        dead on arrival, nowhere to be found.

        1 Reply Last reply Reply Quote 0
        • S
          stephenw10 Netgate Administrator @johnpoz
          last edited by Oct 18, 2022, 10:26 PM

          @johnpoz said in SG-3100 switch weird behavior (resolved):

          Wouldn't setting a static arp for the rpi4 also solve it? Or that is different than the switch arp cache?

          Yeah, nothing to do with the switch MAC table. That exists only in the switch IC.

          So in fact I would expect setting a static ARP to make this worse because it will never expire, pfSense will never ARP for the IP so no responses will be generated.

          So if it was still set static I'm surprised that max_age value made any difference. 🤔

          M 1 Reply Last reply Oct 18, 2022, 10:27 PM Reply Quote 0
          • M
            mcury @stephenw10
            last edited by Oct 18, 2022, 10:27 PM

            @stephenw10 said in SG-3100 switch weird behavior (resolved):

            So if it was still set static I'm surprised that max_age value made any difference.

            It is set like that since the beginning, changed yesterday and the problem happened today several times..

            dead on arrival, nowhere to be found.

            1 Reply Last reply Reply Quote 0
            • S
              stephenw10 Netgate Administrator
              last edited by Oct 18, 2022, 10:29 PM

              Did you set that as a static ARP value to try to solve this or has it always been static?

              Because static ARP entries are almost always the wrong choice. That might be the cause here if it was always set static.

              Steve

              M 1 Reply Last reply Oct 18, 2022, 10:31 PM Reply Quote 0
              • M
                mcury @stephenw10
                last edited by mcury Oct 18, 2022, 10:31 PM Oct 18, 2022, 10:31 PM

                @stephenw10 said in SG-3100 switch weird behavior (resolved):

                Did you set that as a static ARP value to try to solve this or has it always been static?

                I always use static, the problem happened, I changed a few IP addresses here and removed the static ARP, it happened a few times after that so I reverted to static ARP.

                If you want, I can remove that entry to test too

                dead on arrival, nowhere to be found.

                1 Reply Last reply Reply Quote 0
                • S
                  stephenw10 Netgate Administrator
                  last edited by Oct 18, 2022, 10:33 PM

                  Yeah, I would test just without the static ARP entry. If you still see the issue test with max_age at 120s again. Anything less than 3mins should prevent it.

                  Steve

                  M 1 Reply Last reply Oct 18, 2022, 10:36 PM Reply Quote 1
                  • M
                    mcury @stephenw10
                    last edited by mcury Oct 18, 2022, 10:36 PM Oct 18, 2022, 10:36 PM

                    @stephenw10 Done, disabled static ARP, left static DHCP mapping, reverted to sysctl net.link.ether.inet.max_age=1200 to test
                    If the problem happens, I'll change it 120 to test.

                    login-to-view

                    dead on arrival, nowhere to be found.

                    M 1 Reply Last reply Oct 18, 2022, 10:40 PM Reply Quote 1
                    • M
                      mcury @mcury
                      last edited by mcury Oct 18, 2022, 10:43 PM Oct 18, 2022, 10:40 PM

                      Oh, just happened..

                      Changing to 120 to test.

                      Edit:

                      The change didn't take effect, had to ping 192.168.255.253 from pfsense, lets see how it goes now.

                      Edit2: Problem is happening again, I suppose 120 is too much, let me try 60.

                      dead on arrival, nowhere to be found.

                      M 1 Reply Last reply Oct 18, 2022, 10:51 PM Reply Quote 0
                      • M
                        mcury @mcury
                        last edited by Oct 18, 2022, 10:51 PM

                        Static ARP or not didn't change anything.

                        ping -c 2 cron job with an interval of 60s works.
                        setting the sysctl net.link.ether.inet.max_age=60 also works.

                        Now depends on what the user prefer, it seems to me that the cron job would be a better approach since it would only affect a single host.

                        dead on arrival, nowhere to be found.

                        1 Reply Last reply Reply Quote 0
                        • S
                          stephenw10 Netgate Administrator
                          last edited by Oct 18, 2022, 10:54 PM

                          Hmm, interesting. Thanks for testing.

                          Yeah entirely up to you really, either should work fine.

                          Steve

                          M 1 Reply Last reply Oct 18, 2022, 10:55 PM Reply Quote 1
                          • M
                            mcury @stephenw10
                            last edited by Oct 18, 2022, 10:55 PM

                            @stephenw10 said in SG-3100 switch weird behavior (resolved):

                            Hmm, interesting. Thanks for testing.

                            Thanks for the help :)

                            dead on arrival, nowhere to be found.

                            J 1 Reply Last reply Oct 19, 2022, 5:56 AM Reply Quote 1
                            • J
                              johnpoz LAYER 8 Global Moderator @mcury
                              last edited by Oct 19, 2022, 5:56 AM

                              @mcury So this is because the mac cache timeout on the switch on the 3100 is short? So a real switch would fix this, or could allow or account for it.

                              Short might be the wrong term, its shorter/different than pfsense arp cache. So when pfsense goes to send this traffic to that mac, that it has in its arp table be it static or not. If no arp went out that the switch could see the response then the switch doesn't know where to send it - so it broadcasts to all its ports..

                              Interesting issue for sure.. I looked on my cisco, and I can alter the mac address table timeout, it defaults to 300 seconds.. You can also set static arp on the switch..

                              That your rpi is so quiet is strange, most everything is a chatty cathy these days. But I always running something on my pi's, ntp so there is always traffic off the network, so it needs to talk to pfsense. And my other pi, while it runs controller software, and I am almost always ssh'd to it because I use it is my linux shell, so that would have keep alives, and I am from another vlan so again it would always be talking to pfsense (its gateway)..

                              I should kill my ssh and see how chatty it is..

                              But this has been a great thread - very interesting root cause of the problem..

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                              M 1 Reply Last reply Oct 19, 2022, 3:22 PM Reply Quote 0
                              • M
                                mcury @johnpoz
                                last edited by mcury Oct 19, 2022, 3:23 PM Oct 19, 2022, 3:22 PM

                                @johnpoz said in SG-3100 switch weird behavior (resolved):

                                So this is because the mac cache timeout on the switch on the 3100 is short?

                                Exactly.

                                That your rpi is so quiet is strange, most everything is a chatty cathy these days. But I always running something on my pi's, ntp so there is always traffic off the network, so it needs to talk to pfsense. And my other pi, while it runs controller software, and I am almost always ssh'd to it because I use it is my linux shell, so that would have keep alives, and I am from another vlan so again it would always be talking to pfsense (its gateway)..

                                Yes, I'm running Graylog server in my raspberry pi 4b, everything else is disabled to spare some RAM, so it is actually only receiving data (UDP), in which there is no acks, just a UDP dump. Switch believes that its a dead device.
                                That ping cron job fixed..

                                But this has been a great thread - very interesting root cause of the problem..

                                Indeed it was, I'm glad we got it figured out, I was almost doing a clean install in my pfsense or changing the switch but that wouldn't solve anything..

                                dead on arrival, nowhere to be found.

                                J 1 Reply Last reply Oct 19, 2022, 3:30 PM Reply Quote 0
                                • J
                                  johnpoz LAYER 8 Global Moderator @mcury
                                  last edited by Oct 19, 2022, 3:30 PM

                                  @mcury so another work around might be to use TCP for your syslog ;) Then there would be acks and switch would know that IP.. I don't believe pfsense can send tcp syslog though, at least not with the default. Not sure if the syslog-ng package would allow for that?

                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                  If you get confused: Listen to the Music Play
                                  Please don't Chat/PM me for help, unless mod related
                                  SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                  M 1 Reply Last reply Oct 19, 2022, 3:40 PM Reply Quote 0
                                  • S
                                    stephenw10 Netgate Administrator
                                    last edited by Oct 19, 2022, 3:33 PM

                                    Yeah, I thought that. You can use syslog-ng to 'proxy' it between udp and tcp if you needed to.

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      stephenw10 Netgate Administrator
                                      last edited by Oct 19, 2022, 3:34 PM

                                      It's interesting that the Unifi switch apparently doesn't time out it's MAC table like that. I don't think you saw those packets on any other port there.

                                      J 1 Reply Last reply Oct 19, 2022, 3:43 PM Reply Quote 0
                                      • M
                                        mcury @johnpoz
                                        last edited by Oct 19, 2022, 3:40 PM

                                        @johnpoz said in SG-3100 switch weird behavior (resolved):

                                        so another work around might be to use TCP for your syslog ;) Then there would be acks and switch would know that IP.. I don't believe pfsense can send tcp syslog though, at least not with the default. Not sure if the syslog-ng package would allow for that?

                                        That would work but I don't think there is TCP for that purpose, at least that I'm aware of.
                                        The raspbery pi 4b is a 4GB model, so its not much for a syslog server in which there is everything indexed for fast search.
                                        It is using elasticsearch for that task.
                                        The ping cron job is pretty light for a network and RAM usage perspective, so I'll stick with this option.

                                        But yes, something running TCP would have solved that too..

                                        Graylog is a very powerful tool to have in the network, take a look at my dashboard :)

                                        login-to-view

                                        I can search everything, things such a mac address, an IP, a port, a device who accessed a network folder, unifi devices logs, use a firewall rule Tracking ID to see if some user matched that rule and what time it happened.. There is no limit, really.

                                        dead on arrival, nowhere to be found.

                                        J 1 Reply Last reply Oct 19, 2022, 3:46 PM Reply Quote 0
                                        • J
                                          johnpoz LAYER 8 Global Moderator @stephenw10
                                          last edited by Oct 19, 2022, 3:43 PM

                                          @stephenw10 maybe the unifi switch uses just a much longer cache timeout value. I would think switch cache timeout would be of decent length to be honest.

                                          My cisco's have a 5 minute default.

                                          login-to-view

                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                          If you get confused: Listen to the Music Play
                                          Please don't Chat/PM me for help, unless mod related
                                          SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                          M 1 Reply Last reply Oct 19, 2022, 3:46 PM Reply Quote 0
                                          62 out of 85
                                          • First post
                                            62/85
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.