Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    May 2nd Snapshot doesnt work, breaks everything! Beware

    Scheduled Pinned Locked Moved 2.4 Development Snapshots
    57 Posts 15 Posters 10.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      topika
      last edited by

      Same here, DNS Resolver and DHCP not starting after update, update never rebooted and ran half updated for 1-2 days, until i reverted back to stabile 2.4.3 with no problems.

      1 Reply Last reply Reply Quote 0
      • jimpJ
        jimp Rebel Alliance Developer Netgate
        last edited by

        @Dazog:

        Getting this error now in console with build: 2.4.4.a.20180503.1839

        I can replicate that, but it appears harmless: https://redmine.pfsense.org/issues/8497

        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

        Need help fast? Netgate Global Support!

        Do not Chat/PM for help!

        1 Reply Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate
          last edited by

          @maverick_slo:

          Something is definitly wrong with those builds, I cant halt or reboot via gui or ssh…

          Please read the thread. That has been addressed.

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • I
            iqjet
            last edited by

            /etc/rc.reboot
            /sbin/reboot

            This code helped. Runtime now reseted.

            1 Reply Last reply Reply Quote 0
            • T
              tmushy
              last edited by

              I can confirm this issue is still in the latest 5-04 snapshot
              Running the above commands did not help
              /etc/rc.reboot
              /sbin/reboot

              1 Reply Last reply Reply Quote 0
              • L
                LostInIgnorance
                last edited by

                I am still having issues with the new snapshot (5/4) too. I have 3 WAN connections and the two statics still seem to be working, but the appliance I have screeches to a halt after upgrading. I also cannot get a DHCP address on the third connection. Attached is the errors I get on initial boot and sequential reboots result the same.

                ![IMG_20180504_211145 - Copy.jpg](/public/imported_attachments/1/IMG_20180504_211145 - Copy.jpg)
                ![IMG_20180504_211145 - Copy.jpg_thumb](/public/imported_attachments/1/IMG_20180504_211145 - Copy.jpg_thumb)

                1 Reply Last reply Reply Quote 0
                • jimpJ
                  jimp Rebel Alliance Developer Netgate
                  last edited by

                  @tmushy:

                  I can confirm this issue is still in the latest 5-04 snapshot
                  Running the above commands did not help
                  /etc/rc.reboot
                  /sbin/reboot

                  There is no "this issue" in this thread. You need to provide details about exactly what is not working, with console and/or log entries related to the issue.

                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                  Need help fast? Netgate Global Support!

                  Do not Chat/PM for help!

                  1 Reply Last reply Reply Quote 0
                  • jimpJ
                    jimp Rebel Alliance Developer Netgate
                    last edited by

                    @LostInIgnorance:

                    I am still having issues with the new snapshot (5/4) too. I have 3 WAN connections and the two statics still seem to be working, but the appliance I have screeches to a halt after upgrading. I also cannot get a DHCP address on the third connection. Attached is the errors I get on initial boot and sequential reboots result the same.

                    Please at least post the DHCP log and any dhclient entries from it, and anything that looks relevant in the system or routing logs as well.

                    I can't replicate any DHCP client issues here, mine are all working OK.

                    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                    Need help fast? Netgate Global Support!

                    Do not Chat/PM for help!

                    1 Reply Last reply Reply Quote 0
                    • L
                      LostInIgnorance
                      last edited by

                      Unfortunately I had to rebuild it so I could post this so I only have my syslog to go back to. I am using the 2.4.4-DEVELOPMENT (amd64) built on Thu Apr 26 14:32:50 CDT 2018 FreeBSD 11.1-STABLE snapshot to restore to and the C2758 board you used to use. When I upgrade to the latest snapshot, I am unable to do much of anything with the appliance. It looks like it just keeps bouncing the interface for that wan.

                      dpinger: HOME_DHCP 47.34.34.1: sendto error: 65
                      check_reload_status: Configuring interface wan
                      php-fpm[87613]: /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
                      php-fpm[87613]: /rc.newwanip: rc.newwanip: on (IP address: ) (interface: HOME[wan]) (real interface: igb2).
                      php-fpm[87613]: /rc.newwanip: rc.newwanip: Info: starting on igb2.
                      dpinger: HOME_DHCP 47.34.34.1: sendto error: 65
                      kernel: arpresolve: can't allocate llinfo for 47.34.34.1 on igb2
                      dhclient[20017]: exiting.
                      dhclient[20017]: connection closed
                      dpinger: HOME_DHCP 47.34.34.1: sendto error: 65
                      kernel: arpresolve: can't allocate llinfo for 47.34.34.1 on igb2
                      kernel: arpresolve: can't allocate llinfo for 47.34.34.1 on igb2
                      php-fpm[43905]: /rc.linkup: HOTPLUG: Configuring interface wan
                      php-fpm[43905]: /rc.linkup: DEVD Ethernet attached event for wan
                      dhclient: /sbin/route add default 47.34.34.1
                      dhclient: Adding new routes to interface: igb2
                      dhclient: New Routers (igb2): 47.34.34.1
                      dhclient: New Broadcast Address (igb2): 255.255.255.255
                      dhclient: New Subnet Mask (igb2): 255.255.254.0
                      dhclient: New IP Address (igb2): 47.34.X.X
                      charon: 13[KNL] 47.34.X.X appeared on igb2
                      charon: 13[KNL] 47.34.X.X disappeared from igb2
                      dhclient: ifconfig igb2 inet 47.34.X.X netmask 255.255.254.0 broadcast 255.255.255.255
                      dhclient: Starting add_new_address()
                      dhclient: REBOOT
                      kernel: igb2: link state changed to DOWN
                      check_reload_status: Linkup starting igb2
                      HOME_DHCP 47.34.34.1: sendto error: 64

                      1 Reply Last reply Reply Quote 0
                      • L
                        LostInIgnorance
                        last edited by

                        JimP, I can send you a 4m syslog from the time of upgrade if you would like.

                        after thumbing through more of the syslog, it seems pretty consistent on these repeated lines:

                        php-fpm[43905]: /rc.linkup: DEVD Ethernet attached event for wan
                        php-fpm[43905]: /rc.linkup: HOTPLUG: Configuring interface wan
                        charon: 04[KNL] 47.34.X.X disappeared from igb2
                        kernel: arpresolve: can't allocate llinfo for 47.34.34.1 on igb2
                        kernel: arpresolve: can't allocate llinfo for 47.34.34.1 on igb2
                        dpinger: HOME_DHCP 47.34.34.1: sendto error: 65
                        dhclient[20017]: connection closed
                        dhclient[20017]: exiting.
                        kernel: arpresolve: can't allocate llinfo for 47.34.34.1 on igb2
                        dpinger: HOME_DHCP 47.34.34.1: sendto error: 65
                        php-fpm[87613]: /rc.newwanip: rc.newwanip: Info: starting on igb2.
                        php-fpm[87613]: /rc.newwanip: rc.newwanip: on (IP address: ) (interface: HOME[wan]) (real interface: igb2).
                        php-fpm[87613]: /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
                        check_reload_status: Configuring interface wan
                        dpinger: HOME_DHCP 47.34.34.1: sendto error: 65

                        1 Reply Last reply Reply Quote 0
                        • D
                          Dazog
                          last edited by

                          I will have to roll back to April Build until this is fixed.

                          My DHCP connection has the same errors as the poster above.

                          1 Reply Last reply Reply Quote 0
                          • T
                            tmushy
                            last edited by

                            Iv been trying every new development build (didnt try 5-9) and the issue seems to keep happening. I too have to roll back to the last April build

                            What's odd is I ran a virtual appliance and pfsense ran fine in it. Im starting to wonder if its hardware compatibility issues. Im using a quotom box

                            1 Reply Last reply Reply Quote 0
                            • jimpJ
                              jimp Rebel Alliance Developer Netgate
                              last edited by

                              I don't doubt there is a problem here but I need a lot more detail than "it's broken" or "the same errors". Post the errors (even if they are duplicates), log entries, route table contents, anything you can come up with. I need to know exactly what isn't working, with detail. For example: interfaces missing addresses, missing or incorrect routes, services not running (exactly which ones are not running, and any relevant logs from them), and so on.

                              I still can't replicate any issues here in my lab. We might have one person here who is able to replicate this but they're still testing to find out if it's similar, too soon to say if it's related.

                              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                              Need help fast? Netgate Global Support!

                              Do not Chat/PM for help!

                              1 Reply Last reply Reply Quote 0
                              • W
                                whitexp
                                last edited by

                                @tmushy:

                                Iv been trying every new development build (didnt try 5-9) and the issue seems to keep happening. I too have to roll back to the last April build

                                What's odd is I ran a virtual appliance and pfsense ran fine in it. Im starting to wonder if its hardware compatibility issues. Im using a quotom box

                                me too ..

                                pfsense start normal , but no internet connection in pfsense or lan …

                                in logs a lot of  "route has not been found""

                                this happening after update pfsense 2.4 in 05/09 , how you roll back to a old version ?

                                thanks

                                1 Reply Last reply Reply Quote 0
                                • T
                                  tmushy
                                  last edited by

                                  @jimp:

                                  I don't doubt there is a problem here but I need a lot more detail than "it's broken" or "the same errors". Post the errors (even if they are duplicates), log entries, route table contents, anything you can come up with. I need to know exactly what isn't working, with detail. For example: interfaces missing addresses, missing or incorrect routes, services not running (exactly which ones are not running, and any relevant logs from them), and so on.

                                  I still can't replicate any issues here in my lab. We might have one person here who is able to replicate this but they're still testing to find out if it's similar, too soon to say if it's related.

                                  Hello
                                  Id be happy to give you log files of the errors. Im just not sure which ones you want. Can you please tell me the location of the log files. Webgui is not accessible so I would need to pull them by SSH

                                  1 Reply Last reply Reply Quote 0
                                  • L
                                    LostInIgnorance
                                    last edited by

                                    I would love to send in logs as I have a 4m CSV dump from my syslog server, but still I have not been told where to send them. As they are raw dumps, I am not posting them into the forums but would gladly send them to one of the developers.

                                    1 Reply Last reply Reply Quote 0
                                    • jimpJ
                                      jimp Rebel Alliance Developer Netgate
                                      last edited by

                                      I don't need 4M worth of records. I don't have time to sort through all of that. Just the last dozen or so lines of each log file is sufficient.

                                      I think we have a lead on part of the problem, I pushed a fix for one potential path that could break it but there is one other that I haven't tracked down yet.

                                      https://redmine.pfsense.org/issues/8504

                                      More interesting to me now than logs are two things:

                                      1. The <gateways>section of your configuration(s) before and after upgrade, or at least after. You can redact IP addresses but do not alter anything else.
                                      2. Whether or not you have a default route for IPv4 or IPv6 in "netstat -rnW" after upgrade.</gateways>

                                      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                      Need help fast? Netgate Global Support!

                                      Do not Chat/PM for help!

                                      1 Reply Last reply Reply Quote 0
                                      • jimpJ
                                        jimp Rebel Alliance Developer Netgate
                                        last edited by

                                        OK, there are at least three separate issues here from the looks of it:

                                        0. Harmless route errors spamming the console/logs https://redmine.pfsense.org/issues/8497  (Fixed now)
                                        1. An issue with the upgrade code not converting and handling default gateways properly in some cases https://redmine.pfsense.org/issues/8504 (Also fixed)
                                        2. An issue where certain DHCP WANs (igb interfaces at least) constantly link cycle which leads to all sorts of other symptoms (services not running, IP addresses/routes missing, GUI inaccessible, etc) https://redmine.pfsense.org/issues/8506

                                        We're still working on that last one.

                                        Now what I need to know is:

                                        • What hardware are you running where this is happening?
                                        • What type of network interface is it happening to? (Both systems here, and the logs posted in the thread are all igb, but we don't know if that's a coincidence or not)
                                        • Check "clog /var/log/system.log | grep link" and/or "dmesg | grep link" output to see if the link is flapping

                                        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                        Need help fast? Netgate Global Support!

                                        Do not Chat/PM for help!

                                        1 Reply Last reply Reply Quote 0
                                        • T
                                          tmushy
                                          last edited by

                                          Updated to the latestest beta and still getting issues
                                          Im using a Qotom box

                                          May 11 17:55:36 pfSense php-fpm[22628]: /rc.linkup: DEVD Ethernet attached event for wan
                                          May 11 17:55:36 pfSense php-fpm[22628]: /rc.linkup: HOTPLUG: Configuring interface wan
                                          May 11 17:55:37 pfSense kernel: igb0: link state changed to UP
                                          May 11 17:55:37 pfSense kernel: igb0: link state changed to DOWN
                                          May 11 17:55:42 pfSense kernel: igb0: link state changed to UP
                                          May 11 17:55:43 pfSense php-fpm[22628]: /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1526086543] unbound[66133:0] error: bind: address already in use [1526086543] unbound[66133:0] fatal error: could not open ports'
                                          May 11 17:55:43 pfSense kernel: igb0: link state changed to DOWN
                                          May 11 17:55:45 pfSense php-fpm[71870]: /rc.linkup: DEVD Ethernet detached event for wan

                                          Its just looping the same thing over and over

                                          1 Reply Last reply Reply Quote 0
                                          • L
                                            LostInIgnorance
                                            last edited by

                                            JimP, let us know when we can begin testing snapshots again as I can't keep rebuilding and restoring my firewall.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.