Watchguard Firebox T55

perrin

Dear all,

I recently purchased a Watchguard Firebox T55 off eBay for 50Euros. Thought I'd try to get pfSense running on that box. From a first view there are many similarities to the T70 box also discussed here in the forum in this thread. Since @stephenw10 and @networkBob managed to get that one running, i thought my chances are quite good.

First some pictures of the board:
IMG_20210912_112552 (Benutzerdefiniert).jpg IMG_20210908_110628 (Benutzerdefiniert).jpg IMG_20210911_180131 (Benutzerdefiniert).jpg

The board has 5 switch ports which also seem to be driven by a Marvell 88E6176-TFJ2 switch chip. Differences to the T70 is the lack of DIP Switches for the internal Marvell switch chip. Besides that the PCB around the Marvell chips looks quite identical to the T70.

What I managed to do:

Install pfSense 2.5.2 on the internal M-SATA. It boots and recognizes one igb interface igb0, which is the 5th interface of the switch. However it does not detect any link on that interface and there is no other interface detected.
Having read the T70 thread I decided to desolder R614 using hot air to make the Marvell a dumb switch. That changed the behaviour in pfSense so that now it detects a link on the interfaces. igb0 is now usable but however no other interfaces are deteced.
So I resoldered the resistor back in to be able to boot back into the Watchguard OS.

My thought is that the dip switches change inputs to the marvell chip, which in case of the T55 board are "hard wired".
That leads to two possible solutions:

Find out which chip inputs are changed by the dip switches
Try to initialize the Marvell chip the way Watchguard does.

For option 1. I would need @stephenw10 with a multimeter to trace the circuit from the dip switches to the marvell chip. There should be two inputs different in the T70 to the T55.

So I tried analyzing the software...
I have attached the original T55 bootlog in Linux:
T55-Bootlog.txt

Relevant parts for brining up the Marvell seem to be the same as in the T70:

[    4.874009] Running /etc/runlevel/2/S21eth...[    4.392472] Intel(R) Gigabit Ethernet Network 

...

[    8.687953]
[    8.687953] wg_dsa_init: Built Dec 13 2019 12:28:02CPU 3060 SW 1 Flags 29
[    8.687953]
[    8.701907] libphy: Marvell 886176: probed
[    8.706537] wg_dsa_init: mdio found 88E6176
[    8.711218] wg_dsa_init: Rename eth0 -> eth10
[    8.732246] Distributed Switch Architecture driver version 0.1
[    8.738839] mv88e6123_61_65_probe: SW16  88E6176
[    8.748317] wg_dsa_88e617x_init:  phy  0 id  141  eb1
[    8.757725] wg_dsa_88e617x_init:  phy  1 id  141  eb1
[    8.767137] wg_dsa_88e617x_init:  phy  2 id  141  eb1
[    8.776612] wg_dsa_88e617x_init:  phy  3 id  141  eb1
[    8.786555] wg_dsa_88e617x_init:  phy  4 id  141  eb1
[    8.794856] wg_dsa_88e617x_init:  cpu  5 id ffff ffff
[    8.800624] eth10[0]: detected a Marvell 88E6176 switch

...

[    8.924361] dsa_slave_create: eth0 Marvell headers enabled, len 16
[    8.950091] dsa_slave_create: eth1 Marvell headers enabled, len 16
[    8.975667] dsa_slave_create: eth2 Marvell headers enabled, len 16
[    9.002332] dsa_slave_create: eth3 Marvell headers enabled, len 16
[    9.028989] dsa_slave_create: eth4 Marvell headers enabled, len 16
[    9.056313] wg_dsa_init: Split sw10 code 0 dsa ffff88007742b180
[    9.062938] wg_dsa_init: Rename eth10 -> sw10
[    9.082052] device sw10 entered promiscuous mode
[    9.087227] igb_change_mtu: sw10  Max Frame 1532
[    9.092397] igb 0000:02:00.0: changing MTU from 1500 to 1506
[    9.128559] dsa_change_master_mtu: sw10 MTU 1506
[    9.133764] igb_vlan_mode: Blocking HW_VLAN_CTAG_RX on sw10

So I decided to look at the S21eth script that is responsible for bringing up the Marvell:
(The script is for sure copyrighted by Watchguard, so I am not posting the whole script here, but only the relevant parts)
This is from line 315:

-- handle eths for Westport
if ( model == "T70" or model == "T55" or model == "T55-W" ) then
    wglua.touch( "/tmp/model_" .. model )
    wglua.insmod( "/lib/drivers/dsa_core.ko" )
    wglua.insmod( "/lib/drivers/mv88e6xxx_drv.ko" )
    wglua.insmod( "/lib/drivers/wg_dsa.ko" )
    wglua.system( "/sbin/setmacs" )


    -- interim fix before we get the Final BIOS. i.e. BIOS version > v1.2
    fd = io.open( "/sys/firmware/acpi/interrupts/gpe17", "w" )
    if ( fd ~= nil ) then
        fd:write("disable\n")
        fd:close()
    else
        print ("GPE17: COULD NOT apply!")
    end

end

So, this is basically where I am stuck right now.
dsa_core is "distributed switch architecture" of linux and the mv88e6xxx_drv.ko seems to be the relevant driver. Since I am not that experienced with FreeBSD - is there anything comparable to Linux DSA in BSD? Or does somebody have an Idea how to get the Marvell running in BSD?
Really appreaciate your help.
Thanks
Perrin

stephenw10

@perrin said in Watchguard Firebox T55:

...is there anything comparable to Linux DSA in BSD? Or does somebody have an Idea how to get the Marvell running in BSD?

Yes, and it works in a similar way: https://www.freebsd.org/cgi/man.cgi?query=etherswitch

We use that on the devices we ship with an internal switch.

However the problem with using it here is that it requires an NIC driver that exposes the MDIO bus to the OS to allow it to 'see' the switch. The standard igb driver does not do that and we have never found any patches available to do it. Adding that is a non-trivial undertaking!
The same issue exists in a number of other devices including the T70. Forcing the switch to start in unmanaged mode is useful there though as it has other interfaces. It's not really very useful on the T55 though where it has only one.

Doing that does not prevent the WGOS configuring the switch though. I'm surprised you had to replace the resistor to boot it.

There has been further discussion of this in some other threads. Probably most on the T70 thread.

Steve

perrin

@stephenw10 said in Watchguard Firebox T55:

I'm surprised you had to replace the resistor to boot it.

No, I did not have to replace the resistor to boot it. It did not detect the link on the network interface with the resistor connected.

So, your're basically saying, that when i disabled the switch by desoldering the resistor, I made all ports connect to the internal igb0? So essentially the system then only has a "single" port as seen from the firewall? I that were true, it would be useless as a firewall in that mode...

stephenw10

Yes, pretty much. It doesn't disable the switch, it allows it to come up with all ports in a single layer 2, exactly like it would be used in an unmanaged switch.
It probably comes up in port VLAN mode which means you could use still use an external switch and pass VLAN tagged traffic to it. Or use it as a VPN endpoint etc.
But to use it as selfcontained firewall with pfSense will require an igb driver that exposes the MDIO bus.

The only other possibility, as discussed in the T70 thread, is that the switch IC can also pull it's config from an eprom when it powers up. So theoretically it would be possible to set a default config there and have it come up with the ports separated. The T70 has those jumpers that appear to enable/disable that but changing it made no difference so it might have the the ports disabled there anyway. There's no easy way to access the eprom to find out.

Did you remove a heatsink from the switch IC?

Steve

perrin

@stephenw10 said in Watchguard Firebox T55:

Did you remove a heatsink from the switch IC?

Yes, i did. See picture #3 in the first post. The switch ic is the same as in the T70 (Marvell 88E6176), but it seems that the eeprom is not connected (U36 is empty), also it is missing the DIP switches. I can take more pictures of the PCB is necessary...

I had no luck in finding the datasheet and just contacted Marvell to get a copy of the datasheet. I am not very confident to succeed with the inquiry at Marvell....
Let's see. Having the datasheet would give us more options.
Otherwise we could reverse engineer the Watchguard linux kernel modules, but this is also not an easy task...

stephenw10

You wouldn't need to reverse engineer anything. That switch chip is already supported but numerous things including FreeBSD:
https://github.com/freebsd/freebsd-src/blob/master/sys/dev/etherswitch/e6000sw/e6000sw.c#L268

What is needed is a patched igb driver. That was/is used by Stormsheild/Netasq in their hardware running their FreeBSD based OS. That is why that switch driver exists in large part. Note we also contributed to it.
You can see a similar patch applied to OpenWRT to achieve the same thing. See:
https://forum.netgate.com/post/986595

Steve

zanthos

Any updates?

stephenw10

For the igb driver? None I'm aware of.

perrin

@zanthos Sadly, I did not make any real further progress. Writing the igb driver or patching the igb driver is out of my scope as a programmer...
So this projects rests until i find another idea or somebody is willing to help patching the igb driver.....

markusglaser

@stephenw10 Hi, I know this thread is rather old... just wanted to know if there is any new information, and the link for the IGB Patch in OpenWRT is not working anymore ...
Asking because I got a Watchguard T55 for free and wanted to use this one with either OPNSense, PFSense or OpenWRT, whatever is possible to get it running on the device.
Thanks in advance for your help.

stephenw10

No updates AFAIK. There was a recent thread about OpenWRT on the T70 over on their forum that confirmed the e1000 patch to allow switch access is not present there either. Velocloud had patched it them selves and not upstreamed it. Though there's a good chance such a patch would not be accepted upstream anyway.

Steve