WatchGuard Firebox: Core-e and Peak-e series

fibrewire

I currently have 300+ clients streaming whatever at an average of 100MBit across three WAN connections, and everything remains solid without error during spring break. Looks like that bios setting did it for me.

I have discovered a new issue that causes a network interface to hang with Suricata installed. To replicate the issue, create a Suricata monitor on one WAN interface and have it enabled and blocking. When a second Suricata interface is created, it will cause the interface is is created for to error out. This error will occur regardless if Suricata is enabled for the second interface. In my case anything over one Suritata interface crashes, but I also have three WAN interfaces with the same gateway (DHCP address from Time Warner) for my gateway pool.

EDIT: 88.5 MBit is the max sustainable with the 1300MHz Celeron CPU across three WAN connections, the 2.13GHz CPU should be here any day for testing. Despite maxing out the CPU for speedtests, the x1250-e is running solid with hundreds of guests actively using the network. I decided to point Suricata at the internal LAN interface to figure out what is breaking my network :P

EDIT 2: disabling Suricata allows me to max out all three WAN connections and only hit 46% CPU usage. Very interesting…

fibrewire

Although the x1250-e is a heavy-hitter with the 2.13GHz cpu and 2GB of RAM, I still get the occasional interface watchdog timeout issue. I'll update shortly with log info. At this point I would have to recommend against using the Core-e and Peak-e series in a production environment.

fibrewire

I wanted to try a MicroDrive in the Watchguard Firebox, and came across this link - is it really this easy to resolve the elusive "watchdog timeout" issue? I will post the results here.

EDIT: Found the specifics of these tunable options on the "Tuning and Troubleshooting Network Cards" section of the pfSense documentation here

stephenw10

No it isn't, in my opinion. ;)
Not quite sure where that info first appeared from but it was in the main Xe thread for a while. Some of those settings only apply to Realtek or Broadcom cards, pointless here. The others disable msi and msix globally rather than just for msk. The final setting may be worth investigating.
However I've still not seen a timeout with the one recommended setting so I'm clearly not testing as rigorously as you.

Steve

fibrewire

@stephenw10:

However I've still not seen a timeout with the one recommended setting…

I've got 2 servers and one watchguard running pfSense, and somehow in my last reinstall I put the settings into the wrong firewall. Now that the /boot/loader.conf.local on the WatchGuard Firebox reads:

hint.ata.0.mode=PIO4
hw.msk.msi_disable=1

… and my problem hasn't resurfaced for 10 minutes or so, which is better than the 30 seconds before "watchdog timeout" that I was experiencing whenever I connect the guest wireless.

Thank you Steve, now lets see if it stays up until friday :D

stephenw10

Easily done. :)
Yep, that's what my box reads as.

Steve

fibrewire

It's been 6 hours and things are holding steady.

EDIT: 22+ hours, still no issues. I see light at the end of the tunnel! :D

fibrewire

stephenw10: Thanks again! I think that the documentation could be modified to include those two settings as mandatory for the Core-e and Peak-e series ;) I have 5+ days of uptime with hundreds of users, load balancing 3 modems, 2 lans, and one static modem connection carrying dedicated sip trunks, email, webserver traffic, etc. Thank YOU!

I deem this firewall "PRODUCTION READY!"

pfsense.PNG_thumb

stephenw10

Nice. :)
Thanks for the update.
You're right the documentation needs updating badly, it's tripping up a lot of people right now. I'll try and at least remove the parts that are actually wrong this weekend. I confess that supporting pfSense for a living has taken some of my enthusiasm for doing it in my free time! ::)

Steve

fibrewire

Just a quick update before I upgrade to 2.2.3, been up for over 60 days with no problems. A word of advice, make sure multiple internal networks block traffic from each other - it causes the occasional interface to hang in only one direction (receive) from noisy broadcast devices.

Thanks again for everyone who made this possible. pfSense on WatchGuard - a professional combination.

pfsense-MSR.PNG_thumb

fibrewire

I had numerous issues with the firewall because I mistyped a configuration option upon first setup. This setting is not included in any pfSense backup, and must be performed BEFORE the watchguard firebox fully boots pfsense. When booting a fresh CF or Microdrive on a WatchGuard box you need to interrupt the boot loader when it starts counting down from 4. You'll see the OK prompt. At the prompt enter:

set hint.ata.0.mode=PIO4
set hw.msk.msi_disable=1
boot

That will allow the card to boot and you can then add the lines to /boot/loader.conf.local
You can create it and put the lines into it by executing this in the Diagnostics > Command Prompt Execute Shell command box:

/etc/rc.conf_mount_rw
echo 'hint.ata.0.mode=PIO4' >> /boot/loader.conf.local
echo 'hw.msk.msi_disable=1 ' >> /boot/loader.conf.local
/etc/rc.conf_mount_ro

The Hitachi 4GB Microdrives are much faster than any CF card that I've used so far, and don't suffer from write limitations of flash memory (I've had to replace CF several times due to logging wearing out the CF card.) Also, they are $4 apiece on eBay - an actual tiny hard drive! When using a Microdrive, one can set NanoBSD to permanent read/write mode which eliminates slowdowns that users experience with the WebGUI.