Watchguard XTM 5 Series

mredding

I thought I would share how I got pfSense 2.4 (mostly) running on an SSD on my XTM 5 version 2 box (initially a XTM 515) since I encountered a few snags along the way that I hadn’t seen brought up in this thread. In the previous forty some pages, there were a lot of questions about what the version 2 boxes had under the hood: It’s a Celeron E3400 processor with 2x1GB RAM. As far as I can tell, everything else is similar. My BIOS firmware declared it was “WG BIOS 1.3” on the LCD, which is newer than the 1.2 BIOS that is (modified or otherwise) floating around in this thread. See later on for more on the BIOS.

The only thing I haven’t resolved is that the WAN interface fails to get an IP address on boot. More details at the end; any help would be appreciated.

Anyway, here goes:

Remove the unsupported Cavium card & the 1GB CF card.
Take a Dremel tool to a 2.5” to 3.5” bay adapter to make it fit. Mine had holes that lined up relatively closely with the power supply screws, so after cutting the adapter to size I just drilled those holes out a bit larger.
I hooked up the SSD to my laptop via USB to SATA adapter similar to this one: http://www.newertech.com/products/usb3_universaldriveadap.php
I used VirtualBox on Ubuntu to install pfSense to the hard drive. This presented a few hiccups:

I allowed access to a raw hard disk (/dev/sdc in my case) using this procedure: https://www.serverwatch.com/server-tutorials/using-a-physical-hard-drive-with-a-virtualbox-vm.html However, I needed root permissions to both create the VirtualBox hard drive that pointed to the real drive (sudo VboxManage…) and I also needed to run VirtualBox as root as well for it to work. I’m sure there is a better way to manage permissions and not run as root, but I really wasn’t concerned enough to investigate.

I set up the VM with 2GB of RAM, the same amount I had on the XTM 5.

I enabled the serial console in the VM using a “host pipe” as explained here so I could use it in VirtualBox if necessary: https://www.gonwan.com/2014/04/07/setting-up-serial-console-on-virtualbox/

I also included two network adapters so I could set up WAN and LAN in VirtualBox if necessary. The first, for WAN, I left as NAT, and the second for LAN I created a host-only network on vmnet0 with DHCP disabled. I changed the Host IP to 192.168.56.10 so I could give pfSense 192.168.56.1. See https://www.virtualbox.org/manual/ch06.html for more on VirtualBox networking. I theoretically wouldn’t need these network adapters or the serial console, since others installing previous versions of pfSense to a hard disk simply did so, dropped it in to the Firebox before rebooting, and configured it from there, but…

Setup from the ISO does not enable the serial console by default, which I realized after I had already installed pfSense to the hard drive and tried to boot it on the Firebox. Perhaps there is a way to do this from setup itself or the console after installation, but I couldn’t find it readily. So, I fired up pfSense in VirtualBox, configured the network adapters, and connected to the web interface at https://192.168.56.1/. From there, the serial console can be enabled in System > Advanced. I connected to the host pipe with minicom to test the serial console and reboot. Voila!

After installing the SSD in the Firebox, I grabbed an old Windows XP laptop out of storage that actually had a serial port on it to connect to the serial console. After putting the appropriate settings in PuTTY, I pushed the button, and just got an error “ding.” No error message; just “ding.” So, I dug out a USB to Serial adapter and used my laptop with Ubuntu. Minicom and gtkterm worked generally okay, so out of curiosity I installed PuTTY in Ubuntu. I put in the settings, pushed the button… error “ding.” I’m probably missing something obvious. Anyway, I found that pfSense kept em0 as WAN and em1 as LAN from when I set that up in VirtualBox, so the networks didn’t have to be reconfigured. Everything worked seemingly well.
Install flashrom and LCDproc. I only had to change the driver to the Firebox one and the port to parallel; I left all other LCDproc options on the Web Configurator alone.
I know it’s not completely necessary, but I wanted to unlock the BIOS. However, I didn’t want to flash someone else’s random BIOS I found on a forum! I wanted to modify my BIOS with some random tool I found on the internet instead! ;D So, after finding the now ancient AMIBCP 3.51 (The links in this thread are dead; the link I used was this: https://ulozto.net/!PfXQpYPhn/amibcp-3-51-zip ) all I did was change the access level to 3 and enabled the “Always CF Card Boot” menu item in Advanced. I left everything else alone. I couldn’t find where to mess with the Arm/Disarm LED, I wasn’t sure how to enable speedstep (and later posts make it sound like it doesn’t work anyway,) and I thought decompressing modules was a little complicated and didn’t care what it said on the LCD at boot. My Arm/Disarm LED never lit up either before or after BIOS modification. If there’s other things that could/should be enabled, let me know (Steve?). I have attached both the original BIOS and my modified one to this post in a zip file; as usual, use with extreme caution. I flashed my modified BIOS, pulled the battery for a while, and when I put everything back together and booted it up I had full access to the BIOS menus.

MD5SUMs for the very brave:
8eaeb054452c9b8f6ba98d8a5c99ca9f XTM5v2_BIOS.rom
5599976bee52736c37806fbd8a4af9b7 MJR-BIOS.rom

8] The final hiccup, and why I said it almost works: I connected the XTM 5 to my present router for testing. On boot, it will not get an IP address on the WAN interface. I always have to make it try again somehow (via the web configurator refresh button, for example.) Any thoughts? As a stopgap, I was thinking of writing a script and that pings Google DNS, if it fails, make dhclient get a new DHCP lease on WAN, and have the script run as a CRON job every hour or so. Any help would be appreciated.

Thanks,
Matt

EDIT: I can't speel gud

XTM5v2_BIOS_Files.zip

DeLorean

@mredding:

I know it’s not completely necessary, but I wanted to unlock the BIOS. However, I didn’t want to flash someone else’s random BIOS I found on a forum! I wanted to modify my BIOS with some random tool I found on the internet instead! ;D So, after finding the now ancient AMIBCP 3.51 (The links in this thread are dead; the link I used was this: https://ulozto.net/!PfXQpYPhn/amibcp-3-51-zip ) all I did was change the access level to 3….

I'm also searching my way in Bios modding, but where did you see access level 3 in Amibcp 3.51 ?
I only see : Supervisor, User, Extended user and Reserved
Which Windows version did you use for the Amibcp tool?
I have tested it with Windows 7 and don't know if that make a difference in the working of Amibcp.

Thanks in advance.

Grtz
DeLorean

mredding

@DeLorean:

@mredding:

I know it’s not completely necessary, but I wanted to unlock the BIOS. However, I didn’t want to flash someone else’s random BIOS I found on a forum! I wanted to modify my BIOS with some random tool I found on the internet instead! ;D So, after finding the now ancient AMIBCP 3.51 (The links in this thread are dead; the link I used was this: https://ulozto.net/!PfXQpYPhn/amibcp-3-51-zip ) all I did was change the access level to 3….

I'm also searching my way in Bios modding, but where did you see access level 3 in Amibcp 3.51 ?
I only see : Supervisor, User, Extended user and Reserved
Which Windows version did you use for the Amibcp tool?
I have tested it with Windows 7 and don't know if that make a difference in the working of Amibcp.

Thanks in advance.

Grtz
DeLorean

In the Setup Configuration tab, Under Security, there should be a User Access Level option. Under both Failsafe and Optimal, I changed it from 02 to 03. See also https://forum.pfsense.org/index.php?topic=43574.msg262490.html#msg262490

Good Luck,
Matt

EDIT: I think I found where Steve changed the Arm/Disarm LED settings: Under the BootBlock SIO Table, the 27th, 28th, and 29th SIO Registers listed are 30, F0, and F1, and are changed to 01, CF, and 20 respectively. See the new attachment. I think I might try it later today. Not sure how these values correspond with https://forum.pfsense.org/index.php?topic=43574.msg261279.html#msg261279 though.

Assuming this is right, the only thing I haven't figured out is speedstep, which based on this post it sounds like that's a pretty futile endeavor: https://forum.pfsense.org/index.php?topic=43574.msg740652.html#msg740652. Well, that and changing what the BIOS says it is.

EDIT x2: That worked. The Arm/Disarm LED turns red on boot. Now I just have to figure out how to make pfSense turn it green, and get my WAN working on boot without user intervention.

BIOS-Security.png_thumb

BIOS-LED.png_thumb

DeLorean

Thx mredding

Grtz
DeLorean

mredding

I copied https://sites.google.com/site/pfsensefirebox/home/WGXepc64 to /conf/WGXepc and I wrote a little shell script to check if the network is up and change the Arm/Disarm LED accordingly:


/conf/WGXepc -l off > /dev/null 2>&1
sleep 1
until ping -c 1 8.8.8.8 > /dev/null 2>&1; do dhclient em0 && sleep 9; done
/conf/WGXepc -l green > /dev/null 2>&1

My modified BIOS turns the LED red on boot. If the network is up, the script turns the LED off so you know it's doing something, pings Google DNS, and turns it green. If the ping fails, it asks dhclient to do it's job, waits 9 seconds, and then tries to ping Google again. I have this run as a shellcmd script, and you could theoretically run it every hour or something as a cronjob too, and you'd get a little flash of the LED to let you know it checked and its WAN connection is still up. Even though I wrote this for my specific problem, I think it's a pretty good use of the LED.

I still have no idea why I do not get a WAN IP address on boot automatically. The DHCP system logs shown in the Web Configurator show only DHCPd as doing anything on boot; there are no dhclient log messages unless I invoke it manually via my script or by other means like the refresh button. When I watch the boot process via serial console, it hangs on WAN for about 10 seconds before declaring "…done." I'm really not familiar enough with FreeBSD to dig deeper in the logs to see what's going on without some guidance. Any thoughts would be appreciated.

Thanks,
Matt

Billyboy

@mredding:

I thought I would share how I got pfSense 2.4 (mostly) running on an SSD on my XTM 5 version 2 box (initially a XTM 515) since I encountered a few snags along the way that I hadn’t seen brought up in this thread. hour or so. Any help would be appreciated.
….
On boot, it will not get an IP address on the WAN interface.
...

Thanks,
Matt

EDIT: I can't speel gud

Hi Matt, did you try to pull a cable on the 1Gbit ports to see if the disconnection is recognized in the GUI/Dashboard/Interface. We had problems with this on at least 4 boxes (see posts on page 43) in the 2.3.x release. It took up to 10 minutes before PFSense recognized it. This leads to a nearly not working CARP. Maybe the DHCP problem is caused by this as well…

Kind regards

Billyboy

mredding

@Billyboy:

Hi Matt, did you try to pull a cable on the 1Gbit ports to see if the disconnection is recognized in the GUI/Dashboard/Interface. We had problems with this on at least 4 boxes (see posts on page 43) in the 2.3.x release. It took up to 10 minutes before PFSense recognized it. This leads to a nearly not working CARP. Maybe the DHCP problem is caused by this as well…

Kind regards

Billyboy

Thank you, I hadn't put together that those problems may be related. Alas, the Web Configurator shows pfSense detects the cable disconnect almost immediately, and regains a connection after inserting the cable almost just as quickly, so I guess that's not it.

EDIT: Okay, this is weird, but it's working now. I inserted a blank 4GB CF card into the slot and set the SSD as the primary boot device in the BIOS. I formatted the CF card as UFS and set it to mount read only in /etc/fstab. The idea was I thought it was kind of a waste to have an empty card slot, so I would remount it as read-write when I wanted to copy backups to it. This was the only thing I changed. I have no idea why a CF card in the slot would change how the network behaves; it very well could be a coincidence. I'm going to investigate further, but figured I'd share.

By chance, did you experience your problem with XTM 5s that had empty CF card slots?

EDIT x2: I pulled the CF card, and it is still getting an IP address automatically now. No idea how I fixed it.

dlucas46

@747Builder:

@dlucas46:

Hi all,

For those of you with Xeons that would like coretemp to report the correct temp, you can try this recompiled coretemp module.

I have set the TJMax value to 70c

Remove the png extension and upload to /boot/coretemp2.ko

Chmod 755 coretemp2.ko

In your /boot/loader.conf.local add the following:

coretemp2_load="YES"

Reboot.

You should now have a correct temperature reading. I did this several months ago and its been working fine.

If your CPU is in the same family as L5420 this should also work for you.

dlucas46,

Thank you for providing this. could you also provide the source code patch?

I have recompiled for 2.4 against freebsd 11.1.

The code is as follows (lines 213 - 220) coretemp.c :


 else if (cpu_model == 0x17) {
		switch (cpu_stepping) {
		case 0x6:	/* Mobile Core 2 Duo */
			sc->sc_tjmax = 105;
			break;
		default:	/* Unknown stepping */
			break;
		}

You need to change the tjmax value to 70

The reason this error occurs is because the core2duo and the L series Xeons have the same family id (0x6) and the same model (0x17).

The coretemp module really needs to do some more checking and try and identify the cpu by another value that is unique.

coretemp2.ko.png

Billyboy

@mredding:

@Billyboy:

Hi Matt, did you try to pull a cable on the 1Gbit ports to see if the disconnection is recognized in the GUI/Dashboard/Interface. We had problems with this on at least 4 boxes (see posts on page 43) in the 2.3.x release. It took up to 10 minutes before PFSense recognized it. This leads to a nearly not working CARP. Maybe the DHCP problem is caused by this as well…

Kind regards

Billyboy

By chance, did you experience your problem with XTM 5s that had empty CF card slots?

Yes, I experinced the problem with empty CF card slot. Meanwhile I upgrade to 2.40, the cable pull problem disappeared. DHCP on WAN IF (em5 in my case) works for me.

–- But ---

Since the upgrade (from 2.2.6=>2.3.4=>2.40) I have a very high CPU load while there is no traffic on PFSense. Load average always min. 0,50 and more (0.69, 0.49, 1.19) and a CPU load traveling between 20% and 50%, never below 20%.

For heavens sake, I just updated my test system...

DeLorean

@Billyboy:

Since the upgrade (from 2.2.6=>2.3.4=>2.40) I have a very high CPU load while there is no traffic on PFSense. Load average always min. 0,50 and more (0.69, 0.49, 1.19) and a CPU load traveling between 20% and 50%, never below 20%.

For heavens sake, I just updated my test system…

Do a clean install of pfSense 2.4.0 , updating from a older versions gives more chance for problems then a clean install,
sometimes with a upgrade you take the errors from a previous version over.
Version 2.4.0 runs fine on a XTM 5 box, and the previous problem with not detecting correctly the LAN cable status (connected or not) is fixed in 2.4.0

Grtz
DeLorean

vizi0n

Hi!

I have installed an E8400 in my XTM 510 but I do not see the speedstep option in the bios. Should it be supported? I don't see the CPU temperature either on the LCD

A Former User

This post is deleted!

vizi0n

@747Builder:

@vizi0n:

Hi!

I have installed an E8400 in my XTM 510 but I do not see the speedstep option in the bios. Should it be supported? I don't see the CPU temperature either on the LCD

speedstep doesnt work on the XTM 5. Stephenw10 and I have verified it doesnt work.

you will have to configure the LCD driver to see anything on the LCD.

Ok thanks.

About the LCD, what I didn't clearly meant was that I am getting a cpufreq(4) error when trying to display the temperature on the LCD using the LCDPROC package instead of the actual temperature. I also don't see the temperature in the pfsense homepage.

My bios' ACPI is set to 1.0, should I use something else? Or is it not related at all?

uu0113

Hi Everyone,

I own a Watchguard XTM515 with a different BIOS. The LCD shows WG BIOS 1.3 when booting. The Boot output shows:


AMIBIOS(C)2006 American Megatrends, Inc.
MB-7580 Ver.WD0 04/26/2010
CPU : Intel(R) Celeron(R) CPU        E3400  @ 2.60GHz

I saved the BIOS using the following commands


pkg install flashrom
flashrom -p internal -r xtm515.rom

[2.3.4-RELEASE][root@pfSense.localdomain]/root: flashrom -p internal -r xtm515.rom
flashrom v0.9.9-r1955 on FreeBSD 10.3-RELEASE-p19 (amd64)
flashrom is free software, get the source code at https://flashrom.org

Calibrating delay loop... OK.
Found chipset "Intel ICH7/ICH7R".
Enabling flash write... OK.
Found Micron/Numonyx/ST flash chip "M25P80" (1024 kB, SPI) mapped at physical address 0x00000000fff00000.
Reading flash... done.

The MD5/SHA1 sum is different from what I found in this thread so I assume I have another Version:

MD5: 512514e3fd2ce318be1a6ee8280856d5
SHA1: 683eb4d99d9c2c8188efba637c501c0ac475ee9c

I modded the BIOS above in order to unlock all settings, enable booting from other sources (e.g. USB-Stick) which all works fine. I also modified the table below as mredding suggested but the Arm/Disarm LED does not turn red when booting:

@mredding:

EDIT: I think I found where Steve changed the Arm/Disarm LED settings: Under the BootBlock SIO Table, the 27th, 28th, and 29th SIO Registers listed are 30, F0, and F1, and are changed to 01, CF, and 20 respectively. See the new attachment. I think I might try it later today. Not sure how these values correspond with https://forum.pfsense.org/index.php?topic=43574.msg261279.html#msg261279 though.

This is how I flashed it:


flashrom -p internal -w xtm515_uu0113_mod.rom

[2.3.5-RELEASE][admin@pfSense.localdomain]/root: flashrom -p internal -w xtm515_uu0113_mod.rom
flashrom v0.9.9-r1955 on FreeBSD 10.3-RELEASE-p22 (amd64)
flashrom is free software, get the source code at https://flashrom.org

Calibrating delay loop... OK.
Found chipset "Intel ICH7/ICH7R".
Enabling flash write... OK.
Found Micron/Numonyx/ST flash chip "M25P80" (1024 kB, SPI) mapped at physical address 0x00000000fff00000.
Reading old flash chip contents... done.
Erasing and writing flash chip... Erase/write done.
Verifying flash... VERIFIED.

After this I powered down the Watchguard, removed the battery for 10 minutes, startet again and went into BIOS to change the settings I needed in order to boot from my USB-Stick. This was necessary in order to get pfSense 2.4.1 installed. 2 Settings are crucial for this: "Always boot from CF Card" must be disabled and serial console must be changed to "VT100"… After that pfSense 2.4.1 was easily installed :)

I am attaching my BIOS files if someone needs it or has the same version running.

If anyone can get the Arm/Disarm to work, please let me know.

xtm515.zip
xtm515_uu0113_mod.zip

DeLorean

First i was sceptical about updating the BIOS, fear of a badflash and bricking the firewall,
but since i have a decent JTAG programmer, i update now every XTM5 box that i convert to pfSense.
Through the serial console or with the JTAG programmer.

Since i'm converting these XTM5 boxes , i have done 505, 510, 515, 520, 525 and 530 with WG Bios 1.2 and 1.3,
and the BIOS xtm5_83.rom from Stephenw10 worked on every box and the update is never been a problem.

The MD5/SHA1 sum will always be different due the modifications you do to the file.
Every little modification change The MD5/SHA1 sum.
This MD5/SHA1 check is therefore only usefull to check if the checksum before and after the download is the same,
to eliminate filecoruption during downloading.

Keep in mind, that the saved WG Bios is stored on your medium (CF card) that you used for the BIOS update.
So formatting this CF card or overwriting, will erase your original WG Bios backup.
To backup this file from the CF card to your desktop/laptop , i use the free program WinSCP to login in with SSH (username root, password pfsense)
to save these BIOS backups to your local drive.

Grtz
DeLorean

chpalmer

@DeLorean:

Keep in mind, that the saved WG Bios is stored on your medium (CF card) that you used for the BIOS update.
So formatting this CF card or overwriting, will erase your original WG Bios backup.
To backup this file from the CF card to your desktop/laptop , i use the free program WinSCP to login in with SSH (username root, password pfsense)
to save these BIOS backups to your local drive.

He has attached a copy to his post above. :)

CyberDaddIO

I have an XTM 510 & plan on installing pfSense. At the moment it is still running the Watchguard OS & I am just trying to log into the serial console. If I boot the XTM while connected to the console via Putty I can see it booting, but when the logon prompt appears, I can't type anything. I have tried 2 different console cables & connected to one laptop via serial port & another via USB to serial adapter (both laptops running Windows 10). All produce the same result.

Anyone experienced this?

mredding

@CyberDaddIO:

I have an XTM 510 & plan on installing pfSense. At the moment it is still running the Watchguard OS & I am just trying to log into the serial console. If I boot the XTM while connected to the console via Putty I can see it booting, but when the logon prompt appears, I can't type anything. I have tried 2 different console cables & connected to one laptop via serial port & another via USB to serial adapter (both laptops running Windows 10). All produce the same result.

Anyone experienced this?

I had problems where there was a little hiccup during the transition from the BIOS to the OS, where I wouldn't see the OS booting without unplugging the console cable and plugging it back in, but nothing like you described on this hardware. The only time I've had a problem where I could see everything but couldn't type was with a TTL to USB 6-pin serial converter and a consumer router when the connector to the router's RX pin was loose.
I'm fairly confident I have these cables for connection to the console and they work: https://smile.amazon.com/dp/B00HUZ6OMQ/ref=cm_sw_r_tw_dp_x_bn0cAbTG7TFEB

Hopefully the console connector on your board is okay. If not, there is also a serial port header you can plug a standard motherboard 10-pin to DB9 connector into like this one: https://smile.amazon.com/dp/B01MFBMZZF/ref=cm_sw_r_tw_dp_x_VQ0cAbWJR5987 However, I guess that would map to the second serial port and I'm not sure how you would direct the console to that one. This post all the way back on page 11 shows where it gets plugged into: https://forum.pfsense.org/index.php?topic=43574.msg430594#msg430594

I couldn't get PuTTY to work at all for some reason, but I used both screen & minicom on Ubuntu without issues.

–---

@uu0113:

If anyone can get the Arm/Disarm to work, please let me know.

I'll attach my modified 1.3 BIOS with working arm/disarm LED so you can compare.
md5sum:23f2a6329db762256a03bec8a70bd5d7

–---

BTW, on an unrelated note, I found that Suricata does not work in inline mode on this hardware (not that I expected it to), but does in legacy mode. YMMV.

MJR-BIOSv2.zip

CyberDaddIO

@CyberDaddIO:

I have an XTM 510 & plan on installing pfSense. At the moment it is still running the Watchguard OS & I am just trying to log into the serial console. If I boot the XTM while connected to the console via Putty I can see it booting, but when the logon prompt appears, I can't type anything. I have tried 2 different console cables & connected to one laptop via serial port & another via USB to serial adapter (both laptops running Windows 10). All produce the same result.

Anyone experienced this?

Turned out to be the USB-Serial adapter. I found another one & it works fine with the three console cables I have. The laptop with the serial port has has it on the docking station, so that serial port must also be doing something funky with the pinout.

diesel678

Pfsense 2.4.1 seems to be working fine on XTM 5. You just boot live USB pfsense and then install it to the CF card using a desktop PC. No problems booting so far