Watchguard X750e running pfSense on a SATA hard drive easily with about $10 mods

[DeLorean]

@vizi0n:

@DeLorean:

The x750e with 32Gb SSD connected to a Sata Controller card conversion that i made a few months ago.

Grtz
DeLorean

That is a very nice mod!

Thank you.
It was a bit of struggle to get everything together on 1 plexi plate.
When de power connector is disconnected from the SSD, and after removing the 2 red backplates,
the total conversion can then be slide out.

Grtz
DeLorean

[stephenw10]

Nice.  :D

[kobzar]

In last time a have e error on this board.
Some one have the same issue?
https://forum.pfsense.org/index.php?topic=128143.msg706598#msg706598
I can't diagnose problem to resolve this mistake.

[Fmslick]

I am so happy to find this and that I am not the only one still using this hahaha!!

I just got some COLO on the low low and I have a x550e (post is for a x750e so i'm a lower class :P ) w/16GB flash card in there now but I just got my 160GB IDE H.D.D. that will be install later on this week. However I am having an issue this the LED indicator lights and display, the indicator lights only come on for activity (I guess that is cool) however the display will only show cpu load. I used this https://doc.pfsense.org/index.php/PfSense_on_Watchguard_Firebox#Overview and some other things here on the forum from googling but no luck.

2.3.4-RELEASE (i386) - NANO
built on Wed May 03 15:22:11 CDT 2017

@DeLorean your mod is top of the line :)

[stephenw10]

The lcdproc package should run pretty reliably in 2.3.4 with whatever option screens you selected.

To get the NIC LEDs working as expected you need the modified drivers that set the LED registers correctly. However I've seen some reports of them not running nicely in 2.3.X. YMMV.

Steve

[Fmslick]

@stephenw10:

The lcdproc package should run pretty reliably in 2.3.4 with whatever option screens you selected.

To get the NIC LEDs working as expected you need the modified drivers that set the LED registers correctly. However I've seen some reports of them not running nicely in 2.3.X. YMMV.

Steve

yeah I will have to work on the LCD a bit and I think I am one of them who is going to have the LED's not run nicely :/ I did modified the drivers and they are still the same. Guess ill have to look into this a bit more.  ::)

EDIT/ADD TO:
So I got my IDE drive and installed it, I will make a new post later on to update on the progress

[magicshine]

I wanted to say a big thanks for this thread, it got me thinking and working away.

I have been running a Hyper-V VM of pf as my main firewall/VPN in and out to the datacentre for a while but moving to ESXi and having some VM outages led me to wanting hardware appliance to run pf on to avoid having to talk the better half through booting up clustered storage to get pf online and letting me VPN in to fix issues.

I managed to pick up a WatchGuard X1250E for £125 locally, I then bought a few extras (2x 1GB sticks of PC-8500 RAM, Intel Pentium M 2GHZ CPU, PCIe SATA controller, 4x PCIe flex extender, cables and other misc) and got hacking. I had a stack of 2.5" sata HDD's and SSD's.

I found that NONE of my CF cards would boot the box (being a photographer, I had lots of cards but none old or slow enough (yes I did change the mode, access and head settings)). I made a clone of the original WG image and then flashed the original 512MB card with FreeDOS and flashed the B7 bios.

After much batteling with the CF cards I gave in and soldered a USB extension cable to the mobo front connector. I did have difficulty soldering to the ground pad, so I soldered to the shield of the network ports. I also added power from the ATX connector to sata power lead. I know you can use a USB header and have it running at the front but I wanted to route the USB to one of the rear red expansion plates to run my UPS into eventually.

I then removed the WG PCIe riser and installed my flex extension, added the sata card (mounted to the rear expansion plate as shown below) and by removing the lower plastic card guide, screwed the sata HDD to the existing holes (it needed a gentle file to the rear of the case to fit the HDD in as it is a bit far back).

With the USB connection working, I flashed a 2.1 serial usbboot image to the stick and got pf installed via serial. From here I left it as a pretty vanilla config, plugged into the network and created a quick allow all rule to get internet access on the box. Then performed an online update to 2.3.4 without any notable issues. From here I restored a backup of my operational pf VM and configured everything as needed (interface names were different, some change in rules etc.)

I am running a few extra packages, LCDProc, Snort, Squid and it seems to handle these reasonably well. I have noticed though, sometimes after making a few chages to items such as my multiple OpenVPN clients, there would be multiple PHP processes spawned and the CPU would be battered up at 100%. Restarting LCDProc seemed to resolve this. So for now I will stop the service before making a lot of changes and then run once finished (though, making a lot of changes is a rare thing for obvious reasons).

I have hit an issue today, I normally VPN in via IPSEC over my 4G connection for remote work in the office but in trying to establish the VPN connection today it seems to have half killed the box. I cannot SSH or access the webconfigurator and my OpenVPN connection to the datacentre is down but internet access works fine. That's something to look at during lunch after a drive home.

TL;DR: Saw this and other posts, bought kit, built firewall. Success.

Attached are some images of my install for anyone who is interested. I have recently moved house so the rack is in need of a tidy, but as it is in the garage it is neglected currently due to the new family addition! I have a stack of new hardware to go in.

.jpg)
.jpg_thumb)
.jpg)
.jpg_thumb)
.jpg)
.jpg_thumb)
.jpg)
.jpg_thumb)
.jpg)
.jpg_thumb)
.jpg)
.jpg_thumb)
.jpg)
.jpg_thumb)

[stephenw10]

Nice.  :)

[DeLorean]

@magicshine:

I found that NONE of my CF cards would boot the box (being a photographer, I had lots of cards but none old or slow enough (yes I did change the mode, access and head settings)). I made a clone of the original WG image and then flashed the original 512MB card with FreeDOS and flashed the B7 bios.

If you flashed Bios version 8.1 , you can forget the settings mode, access and head , because this version
automatic configure your CF card.

Grtz
DeLorean

[vizi0n]

@Fmslick:

@stephenw10:

The lcdproc package should run pretty reliably in 2.3.4 with whatever option screens you selected.

To get the NIC LEDs working as expected you need the modified drivers that set the LED registers correctly. However I've seen some reports of them not running nicely in 2.3.X. YMMV.

Steve

yeah I will have to work on the LCD a bit and I think I am one of them who is going to have the LED's not run nicely :/ I did modified the drivers and they are still the same. Guess ill have to look into this a bit more.  ::)

EDIT/ADD TO:
So I got my IDE drive and installed it, I will make a new post later on to update on the progress

I am using these:
http://www.vizi0n.com/watchguard/if_sk.ko
http://www.vizi0n.com/watchguard/if_msk.ko

Solid when link up and no activity
Blinks when there is activity

Works fine on 2.3.4

You can verifiy if the mod is running by running "dmesg | grep LED". You should see an output like this:

[2.3.4-RELEASE][admin@pfSense.localdomain]/root: dmesg | grep LED
mskc0: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0x8000-0x80ff mem 0xd0020000-0xd0023fff irq 16 at device 0.0 on pci1
mskc1: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0x9000-0x90ff mem 0xd0120000-0xd0123fff irq 17 at device 0.0 on pci2
mskc2: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xa000-0xa0ff mem 0xd0220000-0xd0223fff irq 18 at device 0.0 on pci3
mskc3: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xb000-0xb0ff mem 0xd0320000-0xd0323fff irq 19 at device 0.0 on pci4
skc0: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xc000-0xc0ff mem 0xd042c000-0xd042ffff irq 16 at device 0.0 on pci5
skc1: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xc400-0xc4ff mem 0xd0420000-0xd0423fff irq 17 at device 1.0 on pci5
skc2: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xc800-0xc8ff mem 0xd0424000-0xd0427fff irq 18 at device 2.0 on pci5
skc3: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xcc00-0xccff mem 0xd0428000-0xd042bfff irq 19 at device 3.0 on pci5</marvell></marvell></marvell></marvell></marvell></marvell></marvell></marvell> 

[magicshine]

So after running my appliance for a while, it seems to be rather unstable and would fall over daily.
Mostly when getting a hit of traffic around 50Mb/s.
The local console would appear normal, but all network traffic would drop and you would not be able to get into the WebGui. I am going to remove the extra 4 ports on the PCIe card and see if this stabilises it, as I know these have been problematic and it does look like a NIC drop to me.

[DeLorean]

@magicshine:

So after running my appliance for a while, it seems to be rather unstable and would fall over daily.
Mostly when getting a hit of traffic around 50Mb/s.
The local console would appear normal, but all network traffic would drop and you would not be able to get into the WebGui. I am going to remove the extra 4 ports on the PCIe card and see if this stabilises it, as I know these have been problematic and it does look like a NIC drop to me.

I think your problem is the ribbon cable between motherboard
and PCI-E Sata controller.
This ribbon seems to be very thin in comparison with a normal PCI-E 1x Risercable.

Grtz
DeLorean

[stephenw10]

Did you add the msi loader variable for the sk driver?
https://doc.pfsense.org/index.php/PfSense_on_Watchguard_Firebox#Known_Issues_2

That does only affect the expansion card NICs though and you usually see the timeout errors on the console if you are hitting that.

Steve

[magicshine]

Thanks DeLorean / Steve.

I will replace the ribbon to rule it out, however the console remains fully functional and the shell is usable so I doubt it is that.

Steve, there are no timeout messages and having known there was an issue with the 4 extra ports, I configured the LAN interface on SK0 so that I would still be able to access the device, but this drops too. I will have a read through the link and see if I make any progress.

Thanks

[stephenw10]

If all the NICs stop passing traffic you might check for an mbuf limit being hit. You would usually see something logged when that happens.

[2.3.4-RELEASE][root@pfsense.fire.box]/root: netstat -m
11100/1305/12405 mbufs in use (current/cache/total)
2997/799/3796/20758 mbuf clusters in use (current/cache/total/max)
2997/798 mbuf+clusters out of packet secondary zone in use (current/cache)
0/10/10/10378 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/3075 9k jumbo clusters in use (current/cache/total/max)
0/0/0/1729 16k jumbo clusters in use (current/cache/total/max)
9045K/1964K/11009K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/9/6656 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile

Steve

[Fmslick]

I know this is a bit old but i'm just now getting a bit of free time!  ::)

@vizi0n:

@Fmslick:

@stephenw10:

The lcdproc package should run pretty reliably in 2.3.4 with whatever option screens you selected.

To get the NIC LEDs working as expected you need the modified drivers that set the LED registers correctly. However I've seen some reports of them not running nicely in 2.3.X. YMMV.

Steve

yeah I will have to work on the LCD a bit and I think I am one of them who is going to have the LED's not run nicely :/ I did modified the drivers and they are still the same. Guess ill have to look into this a bit more.  ::)

EDIT/ADD TO:
So I got my IDE drive and installed it, I will make a new post later on to update on the progress

I am using these:
http://www.vizi0n.com/watchguard/if_sk.ko
http://www.vizi0n.com/watchguard/if_msk.ko

Solid when link up and no activity
Blinks when there is activity

Works fine on 2.3.4

You can verifiy if the mod is running by running "dmesg | grep LED". You should see an output like this:

[2.3.4-RELEASE][admin@pfSense.localdomain]/root: dmesg | grep LED
mskc0: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0x8000-0x80ff mem 0xd0020000-0xd0023fff irq 16 at device 0.0 on pci1
mskc1: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0x9000-0x90ff mem 0xd0120000-0xd0123fff irq 17 at device 0.0 on pci2
mskc2: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xa000-0xa0ff mem 0xd0220000-0xd0223fff irq 18 at device 0.0 on pci3
mskc3: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xb000-0xb0ff mem 0xd0320000-0xd0323fff irq 19 at device 0.0 on pci4
skc0: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xc000-0xc0ff mem 0xd042c000-0xd042ffff irq 16 at device 0.0 on pci5
skc1: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xc400-0xc4ff mem 0xd0420000-0xd0423fff irq 17 at device 1.0 on pci5
skc2: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xc800-0xc8ff mem 0xd0424000-0xd0427fff irq 18 at device 2.0 on pci5
skc3: <marvell gigabit="" ethernet="" (led="" mod="" 2.2)="">port 0xcc00-0xccff mem 0xd0428000-0xd042bfff irq 19 at device 3.0 on pci5</marvell></marvell></marvell></marvell></marvell></marvell></marvell></marvell> 

Thanks I will give this a try.

PS.
Sorry I didn't post an update like I said, it was late and I was at the DC working and on a bit of a slow night when I did the mod so I forgot to take pic's  :(