Watchguard Firebox M440

pglover19

Attached is the information request. The switch does not work when booting with no OS. Additionally it does not work when booted with pfSense.

Let me know what additional information you need.

0_1539446726619_BootLog with Verbose.txt

stephenw10

Hmm, well that's disappointing. It implies the switch will require some setting up to use at all.

That log doesn't look any more verbose. Is that line still there in /boot/loader.conf?
Try adding boot_verbose="1" to /boot/loader.conf.local

Looking at the connector pin-out the two "PCIe" connectors that join the mainboard to the switch board in fact carry more than just PCIe. You can see they carry the SGMII signals from igb0-3 and the serial console and USB and a few other things. However it does not carry anything that looks like it might be some other way to connect to the switch like a second com port. It might possibly be usb connected, though I would have expected to see something logged. Try running:
usbconfig dump_device_desc

Another thing we might investigate is the switch debug port, J5. That looks like a serial port from the pinout (it's labelled SP2_X) it may be TTL level though. That might give us access to a switch CLI, if it has one.

Steve

pglover19

@stephenw10 said in Watchguard Firebox M440:

Hmm, well that's disappointing. It implies the switch will require some setting up to use at all.

That log doesn't look any more verbose. Is that line still there in /boot/loader.conf?
Try adding boot_verbose="1" to /boot/loader.conf.local

Looking at the connector pin-out the two "PCIe" connectors that join the mainboard to the switch board in fact carry more than just PCIe. You can see they carry the SGMII signals from igb0-3 and the serial console and USB and a few other things. However it does not carry anything that looks like it might be some other way to connect to the switch like a second com port. It might possibly be usb connected, though I would have expected to see something logged. Try running:
usbconfig dump_device_desc

Another thing we might investigate is the switch debug port, J5. That looks like a serial port from the pinout (it's labelled SP2_X) it may be TTL level though. That might give us access to a switch CLI, if it has one.

Steve

Give me 30 minutes and I will post the information requested..

pglover19

There is no loader.conf.local file in the boot directory.

stephenw10

Create that file if it's not there. You can create the file and put that value in it by running:
echo 'boot_verbose="1"' >> /boot/loader.conf.local

pglover19

@stephenw10 said in Watchguard Firebox M440:

Create that file if it's not there. You can create the file and put that value in it by running:
echo 'boot_verbose="1"' >> /boot/loader.conf.local

Ok.. Here are the results of the usbconfig command...

0_1539473923939_usbconfig results.txt

pglover19

@stephenw10 said in Watchguard Firebox M440:

Create that file if it's not there. You can create the file and put that value in it by running:
echo 'boot_verbose="1"' >> /boot/loader.conf.local

Ok.. Here are the results with verbose turned on I hope... Maybe I'm doing something incorrect.

0_1539474519176_BootLog with Verbose_2.txt

stephenw10

Ok, no USB devices we don't expect, shame.

The verbose console output should contain a lot more information than that.
Do you still see that line in the correct file? Here's a box I just booted verbose to test:

[2.4.4-RELEASE][root@5100.stevew.lan]/root: cat /boot/loader.conf.local
boot_verbose="1"

Steve

pglover19

@stephenw10 said in Watchguard Firebox M440:

Ok, no USB devices we don't expect, shame.

The verbose console output should contain a lot more information than that.
Do you still see that line in the correct file? Here's a box I just booted verbose to test:
[2.4.4-RELEASE][root@5100.stevew.lan]/root: cat /boot/loader.conf.local
boot_verbose="1"
Steve

I think I got it. Here is another file. Verbose is turned on.. This is the same size as my first verbose file upload.. Let me know...
0_1539476506397_BootLog with Verbose_3.txt

pglover19

@stephenw10 said in Watchguard Firebox M440:

Ok, no USB devices we don't expect, shame.

The verbose console output should contain a lot more information than that.
Do you still see that line in the correct file? Here's a box I just booted verbose to test:
[2.4.4-RELEASE][root@5100.stevew.lan]/root: cat /boot/loader.conf.local
boot_verbose="1"
Steve

I think I got it now... Here is another version of the log file with verbose turned on...

0_1539477665814_BootLog with Verbose_4.txt

stephenw10

Ah there we go! A lot more information there but unfortunately nothing significant regarding the igb NICs.

Noting a couple of relevant links:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228202
Not directly relevant but describes igb debug mode which we could try here to get more.
If it is that same error it's not surprising considering there is no PHY.

https://www.linuxquestions.org/questions/ubuntu-63/ubuntu-18-igb-driver-problem-on-marvell-88e1680-4175633325/
That is concerning as it shows Ubuntu also fails to attach with it's default driver. Potentially there is some issue with a resource conflict or it could be that the switch requires something before it presents to the NICs correctly. Though it doesn't look like that in the WGOS boot log. Nor does it look like they are using a custom driver there.
In that particular case though the OP has failed to understand how the switch is attached. 88e1680 is irrelevant.

Other possible workarounds we cannot try here as the BIOS is locked; disable PXE, disable VT-d.

If that is a regression it would be worth booting 2.3.5 to see if that behaves the same.

Do you have a TTL serial cable of the sort that can be connected to J5?

Steve

pglover19

@stephenw10 said in Watchguard Firebox M440:

Ah there we go! A lot more information there but unfortunately nothing significant regarding the igb NICs.

Noting a couple of relevant links:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228202
Not directly relevant but describes igb debug mode which we could try here to get more.
If it is that same error it's not surprising considering there is no PHY.

https://www.linuxquestions.org/questions/ubuntu-63/ubuntu-18-igb-driver-problem-on-marvell-88e1680-4175633325/
That is concerning as it shows Ubuntu also fails to attach with it's default driver. Potentially there is some issue with a resource conflict or it could be that the switch requires something before it presents to the NICs correctly. Though it doesn't look like that in the WGOS boot log. Nor does it look like they are using a custom driver there.
In that particular case though the OP has failed to understand how the switch is attached. 88e1680 is irrelevant.

Other possible workarounds we cannot try here as the BIOS is locked; disable PXE, disable VT-d.

If that is a regression it would be worth booting 2.3.5 to see if that behaves the same.

Do you have a TTL serial cable of the sort that can be connected to J5?

Steve

No.. I don't have TTL serial cable. I want to be clear on what you want me to do.. Here is what I think.

In the sys/dev/e1000/e1000_osdep.h set DBG to 1.
Boot from version 2.3.5 (where can I get this version).
Connect TTL serial cable to jumper J5.

pglover19

Going back to the BIOS discussion, it is prompting me for a password to enter the setup. Is there a way to clear or reset the password?

stephenw10

The igb driver with dbg set to 1 would require the module recompiling after making that change. Non-trivial.

The 2.3.5 install image can be found here:
https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-memstick-serial-2.3.5-RELEASE-amd64.img.gz

It would be interesting to see what is on J5. If it was me I would put a scope on it first to check what the voltage level and baud rate is. However most people don't have access to that.
That may be the only way to configure the switch in the absence of a driver for the Marvell PCI interface.

There is probably no way to remove the password from the BIOS. It's set by default so resetting the CMOS for example will not help.

Steve

pglover19

@stephenw10 said in Watchguard Firebox M440:

The igb driver with dbg set to 1 would require the module recompiling after making that change. Non-trivial.

The 2.3.5 install image can be found here:
https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-memstick-serial-2.3.5-RELEASE-amd64.img.gz

It would be interesting to see what is on J5. If it was me I would put a scope on it first to check what the voltage level and baud rate is. However most people don't have access to that.
That may be the only way to configure the switch in the absence of a driver for the Marvell PCI interface.

There is probably no way to remove the password from the BIOS. It's set by default so resetting the CMOS for example will not help.

Steve

For some reason, the M440 is not booting from the 2.3.5 image on the SSD. I selected the ISO image. Not sure what is going on...

Ok.. Maybe I selected the wrong 2.3.5 image.

pglover19

For some reason, pfSense 2.3.5 will not boot at all on the M440. I tried installing this version on a SSD and Hard Drive and the M440 is not booting from the disk (SSD or HD). Installed v 2.4.4 on the SSD and it is booting fine.. Not sure what is going on with version 2.3.5.

stephenw10

Hmm, it's probably a UEFI boot issue. I could believe it only boots UEFI from SATA. Not sure I ever tested that on the m400.

Do you have a spare CF card you can try the 2.3.5 Nano image on?

Steve

pglover19

I guess we are finding out that pfSense will not recognize the 24 ports that is part of the switch. I assume I made a bad purchase and will need to try to sell these units to get my money back. Really disappointing.

pglover19

@stephenw10 said in Watchguard Firebox M440:

Hmm, it's probably a UEFI boot issue. I could believe it only boots UEFI from SATA. Not sure I ever tested that on the m400.

Do you have a spare CF card you can try the 2.3.5 Nano image on?

Steve

I have a Transcend 8GB Compact Flash card. So, I will try that.. What is the version I need to download and install on the CF?

stephenw10

This: https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-2.3.5-RELEASE-2g-amd64-nanobsd.img.gz

Though I just tested it on the m400 and it doesn't boot even from the CF slot. It probably is UEFI only which was in 2.4.

We have only been trying for a few days so far, it took far longer than that for some of the other stuff we now have access to.
However I agree this is not good target really. Certainly there are far easier things to install pfSense on. But if you wan a challenge this looks like it!

There is more stuff we can try.

Steve