Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    TIP: If you have an IPMI motherboard and constantly pull an internal IP on WAN

    Scheduled Pinned Locked Moved
    Hardware
    12
    20
    16.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ?
      A Former User
      last edited by

      This post is deleted!
      1 Reply Last reply Reply Quote 0
      • F
        Finger79
        last edited by

        @razzfazz:

        For the less overkill solution, log on to the IPMI web interface, go to the network settings and disable "network bonding" (which as OP experienced bonds the IPMI NIC to one of the LOM ports by default for some motherboards).

        Thanks.  At this stage I had not gone into the IPMI interface on the mobo and just wanted to get pfSense running.  Thanks for letting me know about a "network bonding" setting!

        1 Reply Last reply Reply Quote 0
        • R
          razzfazz
          last edited by

          @SunCatalyst:

          we at work test hundreds of servers a year (in the lab) … with dedicated IPMI ports.

          NEVER have seen this behavior.

          sounds like a BIOS bug or a configuration error.

          never seen a dedicated ethernet port IPMI card decide cause it doesnt have IP to overlap itself on top of the motherboard
          ethernet ports.

          at work we run 100's of Supermicro servers all with IPMI cards. and i run them personally as well.

          Just because you haven't seen (or configured) it doesn't mean it doesn't exist. I've seen lots of IPMI implementations that can be explicitly configured to share one of the on-board ports if using a dedicated port is inconvenient (random examples of boards I have on hand: Tyan K8HM, Supermicro X8SIL, ASRock E3C226D2I); newer implementations (just checked on both the X8SIL and the E3C226D2I) will also let you configure automatic fail-over from the dedicated management port to the LOM ports (which BTW is tied to having a link, not having an IP). Now, up until recently, I had not seen this as the default behavior, but that's what it is at least on the E3C226D2I, and apparently on the original poster's board as well.

          1 Reply Last reply Reply Quote 0
          • R
            razzfazz
            last edited by

            @Finger79:

            Thanks.  At this stage I had not gone into the IPMI interface on the mobo and just wanted to get pfSense running.  Thanks for letting me know about a "network bonding" setting!

            Note that the setting you're looking for may have different names depending on your BMC. As I mentioned in my previous posts, on some of my boards the corresponding setting is called "LAN interface" (and should be set to "dedicated," not "shared" or "failover," to avoid what you're seeing), and obviously the terminology on yours may be different yet. The general idea is that most IPMI implementations allow you to configure which network port(s) to use, and you'll want to force it to use the dedicated management port only.

            1 Reply Last reply Reply Quote 0
            • ?
              A Former User
              last edited by

              This post is deleted!
              1 Reply Last reply Reply Quote 0
              • R
                razzfazz
                last edited by

                @SunCatalyst:

                i work in one of the top 100 corporations in america. we test more servers in a year (and crush 99% of them) than most people will ever see in there complete lifetime. all of the stuff we test is all Xeon based servers from many manufacturers.

                apparently you didnt read "bios bug OR a configuration error". could be EITHER or BOTH. if you dont know what your doing with IPMI and dont
                RTFM , then its your own fault.

                'nuff said.

                Good for you, but none of that changes the fact that the ability to failover to the LOM ports is a feature that is commonly (and very intentionally) offered by current IPMI implementations, not some kind of "bios bug" nor something that only happens accidentally when you mess up your configuration. Now, obviously the OP wasn't aware that this particular feature was enabled by default on his board, and RTFM would have helped; however, it's just silly to claim that no IPMI implementation would ever behave this way in normal operation.

                1 Reply Last reply Reply Quote 0
                • J
                  josh4trunks
                  last edited by

                  this took a year to bite me! I've been running pfsense on a C2558F for the last year with the igb0 interface attached to my cable modem. The other day I lost internet and couldn't figure out why only this interface couldn't get a real internet IP assigned. ipmi failover hit me today while listening to TechSNAP.

                  What a random turn of events, lol

                  1 Reply Last reply Reply Quote 0
                  • R
                    robi
                    last edited by

                    But why on earth would anybody leave the IPMI port configured as DHCP?
                    If pfSense is the DHCP server, but it's not booted up yet, one might end up not being able to connect to IPMI either, since it's got no valid internal IP. In most cases just setting a normal internal IP address to IPMI, and connecting the port to a switch will save from situations like this…

                    1 Reply Last reply Reply Quote 0
                    • M
                      mifronte
                      last edited by

                      I leave all my IPMI ports configured to DHCP with no problems, yet.  IPMI network config is very sticky, meaning, it will hang unto its DHCP settings until you completely disconnect all power to the motherboard.  AFAIK, even the DHCP lease time expiring doesn't force IPMI to query the DHCP server.

                      I can have pfsense shutdown and still access the IPMI interface.  However, I have yet to try disconnecting the power cord when my pfSense box is shutdown to see what happens when IPMI issue a DHCP request when power is restored.

                      SuperMicro Atom C2758 A1SRI-2758F 16GB
                      2.7.2 (amd64)

                      1 Reply Last reply Reply Quote 0
                      • D
                        doktornotor Banned
                        last edited by

                        And the "advantage" of this setup is exactly what? Prey that it doesn't break in the worst possible moment?

                        1 Reply Last reply Reply Quote 0
                        • P
                          pfcode
                          last edited by

                          Yep, It solved the issue that pfSense won't get WAN IP address from my bridged cable modem (WAN was assigned to igb0) after a shut down. This problem was puzzled me for several days until I saw this post. and now it works. Thanks.

                          Release: pfSense 2.4.3(amd64)
                          M/B: Supermicro A1SRi-2558F
                          HDD: Intel X25-M 160G
                          RAM: 2x8Gb Kingston ECC ValueRAM
                          AP: Netgear R7000 (XWRT), Unifi AC Pro

                          1 Reply Last reply Reply Quote 0
                          • C
                            codyst
                            last edited by

                            @robi:

                            But why on earth would anybody leave the IPMI port configured as DHCP?
                            If pfSense is the DHCP server, but it's not booted up yet, one might end up not being able to connect to IPMI either, since it's got no valid internal IP. In most cases just setting a normal internal IP address to IPMI, and connecting the port to a switch will save from situations like this…

                            This is exactly what I did when I put my system together. I set the bmc ip statically to get 192.168.1.2 and I have had no problems whatsoever.

                            1 Reply Last reply Reply Quote 0
                            • ?
                              Guest
                              last edited by

                              But why on earth would anybody leave the IPMI port configured as DHCP?

                              I consider, any IPMI interface must have his own static IP address.
                              We have all connected to the management VLAN to get a dedicated contact.

                              1 Reply Last reply Reply Quote 0
                              • A
                                arduino
                                last edited by

                                We have dozens of SuperMicro servers and they all do this by default, you must switch to only use the dedicated IPMI interface.

                                If you are using the nic in a virtualization environment, it will be wide open as well.

                                1 Reply Last reply Reply Quote 0
                                • C
                                  cmb
                                  last edited by

                                  @arduino:

                                  We have dozens of SuperMicro servers and they all do this by default, you must switch to only use the dedicated IPMI interface.

                                  Yep, they do. On the ones we sell, as part of the installation process that "feature" gets disabled so it only uses the dedicated IPMI port. It wants to use what most people assign as the WAN port as its fallback, which is potentially a very serious security issue if we didn't configure it more sensibly.

                                  1 Reply Last reply Reply Quote 0
                                  • A
                                    arduino
                                    last edited by

                                    Honestly, there are so many bugs/security issues with Supermicro's IPMI interfaces that I wanted to just disable them all anyway.

                                    1 Reply Last reply Reply Quote 0
                                    • ?
                                      Guest
                                      last edited by

                                      Honestly, there are so many bugs/security issues with Supermicro's IPMI interfaces that I wanted to just disable them all anyway.

                                      In our company we had not only one times a problem with this boards or their IPMI LAN Port
                                      and friends of mine connect them to Aten KVM Switches up to >100 devices with any kind of problem!

                                      For sure I will consider that this ports must be configured as well as all new boards are arriving.

                                      1 Reply Last reply Reply Quote 0
                                      • D
                                        doktornotor Banned
                                        last edited by

                                        @BlueKobold:

                                        In our company we had not only one times a problem with this boards or their IPMI LAN Port

                                        Maybe you just don't know about the problem.

                                        https://www.thomas-krenn.com/en/wiki/Supermicro_IPMI_Security_Updates_July_2014
                                        https://www.thomas-krenn.com/en/wiki/Supermicro_IPMI_Security_Updates_November_2013

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post