Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense email flood issue

    Scheduled Pinned Locked Moved General pfSense Questions
    29 Posts 5 Posters 3.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Samlink @stephenw10
      last edited by

      @stephenw10 they keep adding up, the last mail had:
      -pppoe disconnected
      -pppoe reconnected
      -updated DNS record
      -the start of the reboot (for stopping this email flood)
      -the completion of the bootup

      9ed9db9e-8a46-43e4-955d-ef6a5605643f-immagine.png

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        That's still all inside the last 10mins though. Does it eventually stop sending the oldest alerts? How long does that take?

        S 1 Reply Last reply Reply Quote 0
        • S
          Samlink @stephenw10
          last edited by Samlink

          @stephenw10 I don't know, I had to restart it.
          I'm trying to replicate manually what happened today to see if this behavior keeps happening.

          Another thing I didn't mentioned before: I also have telegram notifications enabled, and they are working normally, they are not being replicated.

          EDIT:
          it's happening again: unplugged the ups (simulating a blackout), unplugged the wan2 eth, plugged in the ups after 15 seconds and then plugged in the wan2.
          It's been 12 min and I received over 60 emails.

          Is there anything I can look while the problem is ongoing?

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Do you see those in /tmp/notices?

            Do you see a huge list of alerts in the GUI?

            S 2 Replies Last reply Reply Quote 0
            • S
              Samlink @stephenw10
              last edited by

              @stephenw10 sorry I had to reboot it and didn't check this. For a little over 20 min I received the same text 98 times. They kept adding up to the same emails.

              Tomorrow I'll do another try and check the tmp folder as you suggested.

              1 Reply Last reply Reply Quote 1
              • S
                Samlink @stephenw10
                last edited by

                @stephenw10
                I didn't found any notices file(s) or directory under /tmp.
                Attached are the tmp list during the problem, after ups and wan2 reconnection and after a complete reboot.
                during.txt after_reboot.txt after.txt

                1 Reply Last reply Reply Quote 0
                • D
                  DT 0
                  last edited by

                  Similar problem here after 2.6.0 update.

                  Triggered by (NUT) UPS CyberPower on battery and/or DynDNS updated IP Address on WAN (pppoe0).

                  Sample Gateway Log Entries:
                  May 18 22:17:54 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:53 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:53 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:52 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:52 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:51 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:51 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:50 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:50 dpinger 26971 WAN_PPPOE : sendto error: 65
                  May 18 22:17:49 dpinger 26971 WAN_PPPOE : sendto error: 65
                  ...and on and on.

                  Only a reboot will stop the emails.
                  Also, the continuous email notification "DynDNS updated IP Address on WAN (pppoe0)" is indicating a stale WAN IP, not the current/updated one.
                  Also, after the 2.6.0 update, I still require the patch in order for my Dynamic DNS Clients (No-IP) to update.
                  I have disabled notifications for the time being.
                  Thanks for your help.

                  1 Reply Last reply Reply Quote 0
                  • S
                    Samlink
                    last edited by

                    Hello, any update/advice on this issue?

                    1 Reply Last reply Reply Quote 0
                    • D
                      DT 0
                      last edited by

                      My issue was solved with a fresh install of 2.6.0 and a restore of the configuration. Only took a few minutes and all is well now. The re-install also gave me the opportunity to update the file system to zfs.

                      S 1 Reply Last reply Reply Quote 2
                      • S
                        Samlink @DT 0
                        last edited by

                        @dt-0 thank you, definitely going to give it a try the next week👍

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Unclear still if you're seeing the same alerts being continually sent or just a massive number of alerts.
                          Have you been able to check the dashboard when this happens yet?

                          Steve

                          S 1 Reply Last reply Reply Quote 0
                          • S
                            Samlink @stephenw10
                            last edited by

                            @stephenw10 I'm receiving the same alerts over and over. Even today, my pppoe went down only 1 time in 6 minutes, but I received the same email every 10 seconds or so.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Check the file: /var/db/notices_lastmsg.txt

                              That file should store the last message sent and prevent sending the same message twice.

                              Steve

                              S 1 Reply Last reply Reply Quote 0
                              • S
                                Samlink @stephenw10
                                last edited by stephenw10

                                @stephenw10 at 7:09 I unplugged ups and wan2 and started flooding.
                                First email I received is from the UPS, followed by the wan2 down:
                                4f2e5292-1863-4019-980f-3735d506ab39-immagine.png
                                179ec81d-6243-402d-aecc-46d3d15a6d11-immagine.png
                                The file notices_lastmsg.txt is updated with:

                                MONITOR: WAN2_PPPOE has packet loss, omitting from routing group WAN_FAILOVER

                                The following minutes the same last email is getting repeated every 10 sec or so, here are the next 3:
                                [Edited]
                                (date in the email is the same 7:09:43, but I received them up until 7:13 before I rebooted).

                                During the flood, the file notices_lastmsg.txt didn't change unless something new happened, only then it changed with the new notification, like at 7:12, where I plugged back in the wan2:
                                f9530525-82b5-4ea5-9632-582db66b0b77-immagine.png
                                Now the emails are being sent also with the new notifications, they are adding up.
                                Afterwards I rebooted and got the last email:
                                6660cc68-ec22-4b0c-bf25-ed4aa226132e-immagine.png
                                This is the content of notices_lastmsg.txt now:

                                7:09:43 MONITOR: WAN2_PPPOE has packet loss, omitting from routing group WAN_FAILOVER
142.250.184.99|xxx.xxx.xxx.xxx|WAN2_PPPOE|10.029ms|1.699ms|24%|down|highloss
7:12:47 MONITOR: WAN2_PPPOE is available now, adding to routing group WAN_FAILOVER
142.250.184.99|xxx.xxx.xxx.xxx|WAN2_PPPOE|11.602ms|0.092ms|0.0%|online|none
7:12:48 DynDNS updated IP Address on WAN2 (pppoe0) to xxx.xxx.xxx.xxx
7:14:01 pfSense is rebooting now.
7:15:18 Bootup complete

                                PS:
                                The ups came back online at 7:12:52, I've got only one email relative to it, ,in the middle of the other wan2 ones:
                                95371b0b-6a63-413c-8b80-fed8efd46ed6-immagine.png

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Hmm, this should prevent that happening unless the notices are sent as 'forced':
                                  https://github.com/pfsense/pfsense/blob/RELENG_2_6_0/src/etc/inc/notices.inc#L334

                                  Probably time to open a bug report and get some developer eyes on it. I failed to replicate it here though.

                                  https://redmine.pfsense.org/

                                  Steve

                                  S 1 Reply Last reply Reply Quote 1
                                  • S
                                    Samlink @stephenw10
                                    last edited by

                                    @stephenw10 I'll briefly explain my setup and how I can replicate the problem.
                                    -2 PPPoE WANs (tier1 and tier2).
                                    -One ups (by cyberpower, I don't think this matter but the user @DT-0 happened to also have one).

                                    Unplug the ups , after about 5 seconds unplug the wan2. Doing so I'm able to replicate the email flood problem.

                                    I'm going to open a bug report for now, thanks

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Are you running NUT or apcupsd? I assume you must be from the alerts you have.

                                      S 1 Reply Last reply Reply Quote 0
                                      • S
                                        Samlink @stephenw10
                                        last edited by

                                        @stephenw10 I'm running NUT.

                                        1 Reply Last reply Reply Quote 0
                                        • D
                                          DT 0
                                          last edited by

                                          The problem persists. Just had the WAN go down for a few minutes and come back up. Email notification was sent to advise "DynDNS updated IP Address on WAN (pppoe0) to" This is where the problem (bug) is -- the same email notification keeps on repeating. Perhaps because this is the last notification sent. Only way to stop is disable SMTP notifications and reboot. I thought a re-install of 2.6.0 solved this but no. This issue was introduced in 2.6.0

                                          My dynamic DNS client is No-IP so I don't know if the bug is unique to No-IP. I am also using NUT (CyberPower).

                                          Also, contrary to the release 2.6.0 notes indicating the No-IP complex password issue was solved, it is not. The patch is still required. I have yet to try another DNS client but I might.

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Looks like the bug was opened here: https://redmine.pfsense.org/issues/13224

                                            I would add any additional debugging info you have there.

                                            Steve

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.