-
Hi everyone,
I installed pfSense on a new WatchGuard T70, which is a tabletop form factor with 8 network interfaces, two of which are PoE.
The first 3 interfaces are recognized by pfSense as igb0-2 and operate as expected. The last 5 interfaces are recognized by pfSense only as igb3 but none of these 5 interfaces work (the PoE voltage is present).
Examining the PCB, the first 3 interfaces each have a dedicated Intel chip. The last 5 interfaces appear to be collectively driven by a one large chip with a heatsink. So, I'm thinking maybe a BSD driver is needed in order for the last 5 interfaces to work. At this point, I installed the developer version of pfSense thinking that this might automatically recognize the need for a driver and install the necessary driver, but nothing changed. I have no idea how to force pfSense to install the necessary driver.
Maybe someone has some insight and can help me?
Thank you,
Bob -
Ah, I have wanted to get a look inside one of those for a while. Waaaay outside my curiosity price range though.
You have photos?
The other 5 ports are almost certainly connected via a switch under the heatsink.
Do you see link LEDs connecting to those ports?
The switch is probably not configured by default and so not passing any traffic. It's far more secure to have the switch come up with it's ports disabled so that would be logical.
However if you do see link it's possible the switch is configured with VLANs by default. Does igb3 show as linked and up?
If so try running a packet capture on there with something on the external ports generating some traffic. You might see some tagged traffic come in.
Otherwise you will need to configure the switch and that will depend on what the switch is. If you cannot remove the heatsink you might look at the Watchguard OS boot log for clues.
Steve
-
-
Ooo, fun!
Looks like there's decent labelling on things, including those dip switches. Not sure I can make it out but it looks like SW1 and SW5 are set in their 'default' positions? Slightly unclear if that's '2-wire eprom' or select between 2-wire or eprom.
Were you able to get any sort of boot log?
Did you install to mSATA in something else and move it across or boot from USB?
Steve
-
The mSATA is the boot disk (I had removed it temporarily when I took the photos). It boots quite fast from mSATA. Unfortunately, I made the mistake of overwriting the original mSATA that contained the Watchguard OS, so I destroyed the opportunity to observe the boot log.
As a high-speed 3-interface pfSense box, this thing works very well. But I really would like to get the other ports working. I will do some more testing and report back, along with more photos.
nBob
-
Speculation time: I would guess that setting SW5 to it's alternate position allows the switch to pull it's config from the eeprom, rather than be programmed by the OS. There may not be any config in the there to pull of course.... And it might require the other DIP switches to be set also....
The fact they are labelled MDC/MDIO implies they may allow/disallow programming the switch that way which is commonly how it's done on small switch chips.If the switch ports are all down by default, including the internal one, that may well be what's happening. It just has no config so defaults to disabling the ports.
Steve
-
hi both.
I have a WatchGuard T70 that I'm looking to butcher to install pfsense.
Before I take it apart and start swinging my cleaver is there anything you need from it to help with getting PoE supported?
I'm back at home at the end of the week.
Thanks.
-
The boot log from the Watchguard OS may contain clues about how the switch is configured, so that would be good to see.
If you can avoid overwriting the original OS so we can refer back to it later that would be good. I believe it should boot from any mSATA device. Or even USB if there is no SATA device present.
Steve
-
@chard101 Maybe this information would be useful to you: I pulled the board from the T70 and removed the heatsink. The chip underneath the silver heatsink is a Marvell 88E6176-TFJ2, PAXS390, 4JW, 1631 A1P, TW. I do not have the skillset necessary to load a SOC driver and get the last 5 interfaces to work. The LEDs associated with those interfaces do not illuminate, although the PoE voltage is available and functional on the two PoE ports (6,7) albeit with no data. Thank you.
-
-
-
Ah, some more info there. We can see the headings on the SW1 DIP switch settings. Either I210-88E6176, the default setting and how they are now set, or SoC-88E6176.
So maybe the switch can be configured via one of the igb ports or from a GPIO line on the SoC dircetly. The bootlog from the original OS might provide a clue there.The 2-wire eeprom is almost certainly what the switch pulls it's default config from. As we discussed before it is probably configured to come up with all ports disabled as that is the best option from a security point of view. Then the OS sets up the ports and VLANs as required. However without the eeprom connected there's a good chance it comes up as a dumb 5 port switch which would be much more useful here if we can't control it.
I don't have one of those but if I did I would move the SW5 DIP switches to the other position. And see if that allows the switch to come up with ports enabled.
If course I'm guessing here so the risk is all yours!Steve
-
Evening chaps.
Bob, thank you very much for posting those additional photos and Stephen, thanks for your suggestions. Much appreciated.
I've finally got a new 240GB msata flash card and the other bits I needs for the job. So, I'm going to shutdown the firebox, take out the original card, put the new one in and install pfsense. I bought a caddy that I can put the original card in and hopefully pull the boot log from it. That's the plan.
I'm going to start this adventure tomorrow anyway, its getting late here. I will try adjusting the SW5 DIP switches after I've completed the install, or would you suggest trying to extract the boot log first and post it?
Fingers crossed we can get this cracked.
-
The first thing I would do is boot the original OS with the console connected and copy/paste the boot messages to a file from there.
Then install pfSense to the new mSATA device in something else and swap it into the T70, make sure that boots. Check you see the same things @networkBob did.
Then try booting with the SW5 DIP switches in the alternate position. I believe that will disconnect the EEPROM from the switch IC so it cannot load a config when it powers up. It should then default to being a unmanaged switch. With any luck all ports enabled and connected in the same untagged vlan. I have no way of testing that though so ymmv!
Steve
-
Hi Steve,
I got some time to crack on with this and am now running with pfsense on my T70. I've taken a copy of the origional WG OS bootlog and also the bootlog for pfsense too. Hopefully they are attached to this post and prove useful. I still have the origional WG SSD so I can always hopefully mount it and extract files if needed.
As Bob has reported igb0 to 2 work as expected. I noticed that igb3 comes up with an incomplete MAC address, regardless of how SW5 is set.
igb3: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x1000-0x101f mem 0x80000000-0x800fffff,0x80100000-0x80103fff irq 19 a4
igb3: Using MSIX interrupts with 5 vectors
igb3: Ethernet address: 00:a0:c9:00:00:00If you have any suggestions to try on getting the other ports working that would be apprecaited.
Thanks.
Rich.
-
Ok at least two interesting things there:
[ 3.769850] LED/Reset Button Driver for MB-UP2010W...That's the M440, it uses the same driver for the LEDs/buttons.
[ 9.234620] libphy: Marvell 886176: probed [ 9.239259] wg_dsa_init: mdio found 88E6176 [ 9.243942] wg_dsa_init: Rename eth3 -> eth10 [ 9.267384] Distributed Switch Architecture driver version 0.1 [ 9.273984] mv88e6123_61_65_probe: SW16 88E6176Confirms what the switch is and how it's attached, via the mdio lines on igb3.
The MAC address you see in pfSense is correct, it's not an error reading it. In the original OS each port is addressed via a VLAN and given a separate MAC at that point.
It's using an Insyde BIOS which is more often found in laptops. Unclear if that's good or bad for us, it's different.
Did you actually power cycle the board between moving the DIP switches? That may be required if the switch remains powered in standby.
I assume the ports still did not show link after changing that? And igb3 still shows as down?Steve
-
Hi Steve,
When I flipped the DIP switches I used the power switch at the back to cut the power before switching it back on. I compared it with a bootlog of before I flipped the switches and there was no change, so I set them back again.
Rich.
-
Ah, the only difference in the bootlog might have been something like:
igb3: link state changed to UPBut only then if you had igb3 assigned and enabled.
If you didn't test the external switch ports after doing that then I would test it again. And run
ifconfig -vmaat the CLI to see if that shows any change on igb3.Steve
-
Hi Rich,
Were you able to get any further with the igb3 ports? :)
I will try Stephen's suggestion regarding ifconfig -vma.
Kind regards,
nBob -
I acquired one of these for (probably waaay too much!).
Unfortunately the switch remains stubbornly with all it's ports disabled whatever I have done to it.
They do not seem to come up even for a second at reboot (or complete power cycle) or in the BIOS setup. Or even if you short the CMOS so it doesn't boot at all.
It's interesting. The outside looks very Lanner but the PSU (I have) is from Senao who make their access points.
I was able to confirm he other DIP switches, if you change them from MDIO to SoC the WG OS fails to find the switch and other ports etc.
Steve
