Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Broadcom bge / HP 331T error on boot - Attaching PHYs failed

    Scheduled Pinned Locked Moved
    Hardware
    2
    8
    316
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      Casper042
      last edited by

      I have an HP 331T passed through to a QEMU VM, but when the VM boots I get errors as soon as it gets past port 1 (bge0) and it won't discover the other 3 ports on the card.

      I found some info on the main site and added 3 System Tunables, but it doesn't seem to help.

      Anyone have ideas as to how to get this working?
      I know the card in general works because screenshot 3 was taken over the network to the card (VM has zero Virtual NICs)

      https://imgur.com/a/vMBAaJo

      Thanks

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Hmm, so it does attach as expected to the 1st PHY and creates bge0?

        Does it work if those NICs are not passed through as PCIe devices?

        Pictures of the log files are generally far less useful than the actual log text. If only because it's then searchable.

        You could try booting in verbose mode to see if that give you any additional error output.

        Interrupt boot at the loader menu and go to the loader prompt. Enter: boot -v

        Steve

        C 1 Reply Last reply Reply Quote 0
        • C
          Casper042 @stephenw10
          last edited by

          @stephenw10 Thanks Steve, I will grab the logs and then swap the SSD in the machine (don't want to rebuild the bare metal OS just yet) tomorrow and do a fresh install of pfSense on bare metal and get the same boot -v logs from there for comparison.

          Where do I grab the logs from after I boot -v?
          What's the log file name/path? Or is there a spot in the GUI that makes it easy to download?

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            You can see the boot log in the gui or it would be in /var/log/dmesg.boot

            What I meant was do the NICs work if you just add them in the hypervisor with virtual NICs in pfSense?

            1 Reply Last reply Reply Quote 0
            • C
              Casper042
              last edited by

              OK I have NFC what I am doing in regards to getting the Bridge interfaces added to Ubuntu and adding them to the VM.
              I thought I figured it out, but after adding the 2 virtual nics to the VM and booting it, no dhcp address is coming through on LAN interface.

              sudo ip link add brg0 type bridge
              sudo ip link set enp1s0f0 master brg0

              Then I added the Virtual NIC and set brg0 as the Bridge interface,

              Anyway, I am going to go swap the boot drive and try pfSense on bare metal as well.

              Here are the logs before I started trying to revert the vfio changes and add the bridge interfaces and all that above:
              https://github.com/Casper042/pfSense_bge/

              1 Reply Last reply Reply Quote 0
              • C
                Casper042
                last edited by

                So installing Bare Metal seems to work fine.
                Not sure if we should keep this here or punt it to the VM category.

                I have uploaded the logs to the same GitHub as above.
                I installed latest pfSense ISO via Ventoy on a dedicated NVMe drive.
                It detected the onboard Intel and the 4 port Broadcom just fine, zero issues or tunables required.
                I even auto detected the 2 ports (bge0/bge3) during the setup wizard.

                So now I am just wondering what I did wrong on the VM...

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Yup it could be the PCIe pass through just doesn't do all that's required for that NIC. Seems like a virtualization issue as you say.

                  1 Reply Last reply Reply Quote 0
                  • C
                    Casper042
                    last edited by

                    I have a buddy who used to write Linux drivers for a big company, though not sure how much he knows about BSD.
                    I might hit him up in a few days if I can't figure it out.

                    If pfSense somehow supported Docker/Podman this would be a non issue, but since it's BSD kernel I don't see that happening.

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post