Successful Install on Watchguard Firebox X700!

woc38

I'm using pfSense 1.2.3 on a X700 [full install on harddisk] for over a year now, and I'm very pleased with it.
Modifications:

replaced default 256MB module with a 512 MB (Kingston KVR133X64C3/512)
replaced default 3 case-fans with 3 Noiseblocker XM2 fans, used 2 Y-fan-cables and attached them to the fan2 connector on the motherboard
replaced default CPU fan with a similar new Titan radial fan (TFD-B6015M12B) and used a Zalman Fan Mate 2 to control its speed and noise.
replaced the cryptocard with a TP-Link TL-WN861N WL300 (Atheros 5416 chipset), drilled 2 holes and connected 2 U.FL to RP-SMA Pigtails
activated the LCD with the instructions found around here. Did not play with the arm/disarm leds yet.

With the recent 2.0RC's I was thinking about upgrading. I wanted to leave my existing 1.2.3 install intact, so I search for my 4GB Hitachi microdrive:

wrote pfSense-2.0-RC3-4g-i386-20110621-1821-nanobsd.img with Win32DiskImager
placed it in the CF slot in the X700 and it booted fine
via a serial cable I did the initial setup. After the setup, the webinterface is reachable.
via the webinterface I did an upgrade to 2.0-RC3 built Mon Jul 4 17:29:15 EDT 2011
I activated the LCD with stephenw10's package

So far so good. Now I'm configuring and testing this version.

Things I encountered so far:

I'm getting good speeds (98Mbit via FTP) at 50-55% CPU (basic setup, no extra packages or large ruleset)
As others noticed, the console menu via serial cable is not working. After a reboot it stops at "Bootup complete". When rebooting or shutting down, it does not give the root warning message (System going down IMMEDIATELY), but it gives some final messages on the console:
Waiting (max 60 seconds) for system process vnlru' to stop…done Waiting (max 60 seconds) for system process bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 0 done
All buffers synced.
Uptime: 2h41m43s
Rebooting...
It looks like the Firebox is "temporarily" disabling the console output?!?
SSH is working fine. Would like to see this problem solved…
@jdetmold:

anyone else find that a reboot from the gui does nothing on 2.0?

Reboot and shutdown (X700 halts, no power off) from the webinterface and SSH terminal is working, it only takes more then 2 minutes after you answered "yes" before anything happens.
Sometimes the Firebox does not actually reboot after the command via gui or SSH. The console says rebooting… but nothing hapens, no boot messages appear in the console, the LCD keeps showing "pfSense shutting down". Then I need to switch the unit off and back on. Can not reproduce at will, so far it happens at random.
LCD locked up after transferring 2 large files (3 GB each) via FTP. It showed LCDproc Server Cli:0 Scr:0. Other data is not accessible. It happened 1 time, I couldn't reproduce so far.

First questions:

Any news on the console menu problem? Does this problem also exists with the 1.2.3 -> 2.0RC3 upgrade route, or with a HD install?
Any news on the slow reboots? Does it have something to do with uniprocessor or SMP? Which one should I use for these Fireboxes?

Nicklas

Question, I was able to build in a HD in the x700, but before closing it down, I was wondering can I leave the CF card in it?
If I leave it in do I need to change the jumper position (SCF1) to let it boot from the HD? Any help much appreciated :-)

woc38

Just as a follow-up on the things I encountered installing 2.0RC3:

Installed 1.2.3 nanobsd on a microdrive. Boots fine and gives console menu. Did an upgrade to 2.0RC3 via the webinterface. Upgrade was successful, but there is no console menu. It just stops after "Bootup complete".
Installed 2.0RC3 full on a HD (via the USB installer) on a different machine, enabled serial console via the webinterface, edited /etc/fstab and placed the disk in the Watchguard. It boots and starts fine, but also no console menu after "Bootup complete".
The HD supports some SMART (Attributes Data but no error logging or self tests). When I wanted to view all SMART information, the console suddenly gives output:

Bootup complete
ad2: (fw1.dh.local) (FAILURE - SMART status=51 <ready,dsc,error>error=4<aborted>ttyu0)

*** LBA=12734209Welcome to pfSen
se 2.0-RC3-pfSenad2: se (i386) on fw1FAILURE - SMART status=51 <ready,dsc,error>error=4 <aborted>***

WAN (w LBA=12734214an)
-> re5 -> 172.16.0.2 (DHCP)
LAN (lan) -> re1 -> 192.168.10.1

Logout (SSH only) 8) Shell
Assign Interfaces 9) pfTop
[…]
Enter an option:

Then I get a lot of these error messages


calcru: runtime went backwards from 13179 usec to 12445 usec for pid 27125 (sshlockout_pf)
calcru: runtime went backwards from 6291 usec to 6035 usec for pid 26753 (getty)

Also for other services (minicron, cron, dnsmasq, apinger, php, etc)
Does anybody knows what these messages mean? I read something about the clock ticking backwards, but, what to do about it?

After the initial error message I can use the console. A reboot is working at normal speed (do not have to wait 2 minutes before something is actually hapening). After the reboot, the console halts at "Bootup complete" until I invoke a new SMART error.
After some time the console stops responding again. After generating another SMART error I can access it again

My guess is that something is stopping pfSense from writing the menu to the console until some other data is explicitly written to console.
Can anybody confirm the same? Or has anybody a working console right now?</aborted></ready,dsc,error></aborted></ready,dsc,error>

Nicklas

@Krisstian:

someone asked for pictures… this is an Firebox x700

Krisstian, I notice you replaced the FANs, I would like to do that as well, could you please advise which fans you brought and where? Are you pleased with the result is the sound level dropped and nearly silent?

stephenw10

Given that Kristian hasn't posted since March and before that not sinse 2008 you may be waiting a while! :P
However there are a number of posts here about replacing the fans with quieter ones.
Be careful replacing the main cpu fan. The original is a radial fan because the clearance to the top of the case is small. If you use a standard axial fan you may have to cut hole in the case to prevent it being mostly covered. Even if the case top fits it may not allow enough clearance to get sufficient airflow.

Steve

Nicklas

Thanks Steve :-)

I found indeed 2 type of silent fans to replace the 3 fans on the back of my x700.

A) Fractal Design FD 40mm 13.0 dba max. 4000rpm
B) Scythe Mini Kaze 14.0 dba max. 3500rpm

But the CPU FAN it blows it all to the back and no replacement found as yet for that one.

stephenw10

Well woc38 above said:

replaced default CPU fan with a similar new Titan radial fan (TFD-B6015M12B) and used a Zalman Fan Mate 2 to control its speed and noise.

Looks likely.
You could just try a fan controller on the existing fan. What ever you do make sure you monitor the cpu temps.

Steve

woc38

@Nicklas:

Krisstian, I notice you replaced the FANs, I would like to do that as well, could you please advise which fans you brought and where? Are you pleased with the result is the sound level dropped and nearly silent?

I choose these fans:
@woc38:

replaced default 3 case-fans with 3 Noiseblocker XM2 fans, used 2 Y-fan-cables and attached them to the fan2 connector on the motherboard

replaced default CPU fan with a similar new Titan radial fan (TFD-B6015M12B) and used a Zalman Fan Mate 2 to control its speed and noise.

The 3 case-fans are fairly silent. The radial fan does give some noise, but less then the original one. With the Fan mate I can control the noise to a more acceptable level.
It still makes noise though, I can't sleep next to this red machine… I installed mbmon (pkg_add -r mbmon) to monitor temperature and fanspeed.

Some news about 2.0RC3

Today I upgraded to the current snapshot (2.0RC3(i386) built on Wed Aug 3 01:54:50 EDT 2011). Previous version was the snapshot from 2 aug.
After the reboot: Yes! I saw the first userland console output since long time: the console menu seems to work again
Sometimes it seems to hang (non-responsive). After a minute or so, the menu is rebuilding and back
For the first time since 2.0 I get Watchdog timeouts on the LAN interface. There is virtually no traffic: 1 laptop directly connected to the LAN port accessing the webinterface.

I tried different install methods: 2.0RC3 nanobuilds on microdrives, 2.0RC3 on a harddisk with embedded, uni- or SMP kernels, 1.2.3 upgrades. Massive editing of /etc/ttys. The boot procedure and the pfSense startup is always visible through serial console. It always stopped after "bootup complete" (when the system is entering userland?).
This is a HD full install with a uniproc. kernel. Until today there was no console output after the "Bootup complete" message.

There is also a new problem: I can't assign interfaces anymore via the webinterface. I have 2 out of 6 interfaces enabled (WAN and LAN) and I want to add a 3th one (OPT1).
After clicking interfaces - assign - plus-sign the webinterface reports with "Interface has been added" but there is no new interface in the list.
I can add them without trouble through serial console or SSH and after that they are visible in the webinterface.
This problem also existed in the snapshot from 2 aug.

More of you have serial console output after upgrading to the latest snapshot?
And what about assigning interfaces?

stephenw10

Hmm, strange. There doesn't seem to be anything directly related to any of those things in the recent commits list: https://github.com/bsdperimeter/pfsense/commits/master

Steve

woc38

Hmmm,

My joy didn't last for long… After rebooting a few times guess what... no serial console anymore (and no watchdog timeouts).
Seems like the re driver has some effect on the console... After a watchdog error, the console begins to respond (for a while).
Earlier I noticed that a SMART error also triggers the console for a short time.
And this X700 unit also suffers very slow halt times (except when the console is working, then it halts in good old 1.2.3 times).

Time for an other device... I read some things about the X750e series (and a-like). Are these units "problem free"?

stephenw10

@woc38:

I read some things about the X750e series (and a-like). Are these units "problem free"?

They have Marvell Gigabit interfaces. The four built in use the sk driver and work absolutely no problem. The X550e only has these.
The X750e and higher models have four additional interfaces on a plug in card that are supported by the msk driver.
Unfortunately the msk driver in FreeBSD 8.1 is buggy and can lock up. I've only experienced this when I was load testing the box.
The driver has since been patched so it will be fine in pfSense 2.1. The driver could be backported but it's beyond my skills. ::)
The box has some nice features though and is very upgradable. Read through the thread for more.

Steve

kolkjaer

Not Sure I any off you can help…

I have a x700 with a 4GB Cf card where I have flashed pfSense-2.0-RC3-4g-i386-20110621-1821-nanobsd.img
But I can't seem to get it working.
I don't get any console output, only thing that it dose is to flash green in the arm/diarm LED.

Any ideas ?

stephenw10

Any problems flashing it? Try using the 1GB or 2GB image instead, it will definitely fit on your 4GB card.

Steve

kolkjaer

No the flashing went well.

I have in the mean time tryed with the org CF card, and it dose the same.
It has been working up to around midt last week, with both 1.2.3 and 2.0RC3
So looks like there is some thing worng with the box :'(

stephenw10

Test the CMOS battery. Re-seat the CPU and the RAM. Does the LCD show anyting at all?

Steve

kolkjaer

Have tryed all 3 things.
No the LCD just light up.

leefuji

hi all!
I'm just getting into this pfsense/firebox thing and I'm really confused on which Image to use to load up pfsense on my Hitachi 40GB laptop drive which I will install in the x700. Can anyone point me to the right image I need or link.

ps. keep in mind its a clean install of the latest pfsense which I do believe is 2.0RC3 if 'im not mistaken.

Thanks ;D
Lee

stephenw10

The latest version is the daily build from the snapshot server however 2.0r3 is probably fine unless you find a specific bug that needs fixing.

On a hard drive you can use the full install version. The easiest way to do this is boot the install CD, built from the ISO image, in a laptop. Install to the hard drive and then transfer to the firebox. You will need to boot up the laptop into pfsense once in order to turn on serial console. When you first boot the image in the firebox it will probably fail to boot and you will have to manually enter the location of root.

It's really much easier to use a CF card! ;)

Steve

leefuji

@stephenw10,
Cool didn't know about the Snapshot server link .. thanks for that. Sounds good I will download the ISO image file and follow the instructions as exaplained earlier in this post. Now if I wanted to go with the CF would I still use the ISO image or is there a different one that is recommended to be used with a CF card? I read something about an embedded edition but didn't get if that was something CF or HD specific.

Thanks
Lee

stephenw10

If you use a CF card you need to use the NanoBSD image (a special version of FreeBSD for embedded applications). You can find them on the snapshot server or the main download site. E.g. here: http://snapshots.pfsense.org/FreeBSD_RELENG_8_1/i386/pfSense_RELENG_2_0/nanobsd/pfSense-2.0-RC3-1g-i386-20110905-0445-nanobsd.img.gz

You need to write the image directly to the CF card with some appropriate software like physdiskwrite as described here: http://m0n0.ch/wall/physdiskwrite.php

You can use any of the images that will fit on your card (1GB image on a 4GB card is fine) however manufacturers seem to continuously reduce the actual size of their cards so you may find the 4GB image doesn't fit on your 4GB card. Just use the 1GB image. ;) There is almost no advantage to using a bigger image.

Steve