Pfsense on a HP Netserver LPr

ugob

Hi,

I got a robust HP netserver LPr (dual PIII 750, 1GB RAM) and I'd like to use it as a firewall (PfSense). There is no standard IDE controller (only a small IDE interface, for the slim cdrom… I don't think I could connect an IDE-to-compactFlash on this. So I thought I have 2 solutions:

1- Buy a regular IDE controller and an IDE-to-flash adapter, so that I can use a flash card... I don't have neither of them so trying out may make me waste money
2- Run the LiveCD and keep config on a flash card.
3- Install on HDD. Since these are SCSI disks, they are quite expensive, so I'd rather use something else... and... does PfSense support software RAID?

So I tried 2- first, since I had a spare usb PCI card. However, it won't work. The card is a VIA VT6212L and the system kind of hangs just after the GEOM_LABEL. After many minutes, I get some output, lookin like:

umass0 : BBB reset failed TIMEOUT

Then some other errors about bulk-in clear stall failed and bulk-out clear stall failed.

As soon as I remove the USB stick, it boots w/o problem.

What should I do... buy another USB PCI card based on another chipset? Use an alternate boot method?

Thanks,

Ugo

ugob

Replying to myself… I figured out I could try a floppy as media for the config... It is working, but I trust more USB keys than floppies... This firewall will be in a datacenter, so we don't really want anything physical to break. Any ideas regarding my original post?

sai

yes I wouldn't trust floppies either but usb just dont seem to work with freeBSD. The only time you would need the floppy is at boot time but it is not a good solution. I have had too many floppies die on me - just dont use them any more.
I'd go for another machine.

jahonix

If your HP netserver is going to live in a DataCenter, you'll probably like the improved reliability and speed of a small SCSI HD. The latter helps when/if you use squid on that machine. A SCSI HD always is a good investment in reliability.

On the other hand the small IDE interface naturally IS an IDE port. "All" you need is an adequate adapter but I don't know if something like that is readily available or would have to be built by yourself.

Chris

ugob

@jahonix:

If your HP netserver is going to live in a DataCenter, you'll probably like the improved reliability and speed of a small SCSI HD. The latter helps when/if you use squid on that machine. A SCSI HD always is a good investment in reliability.

But does PfSense supports software RAID? I think a LiveCD + floppy is more reliable (at least for running) than a single disk. The only RAID controller I have here is an HP NetRAID 1-M. Will it work?
@jahonix:

On the other hand the small IDE interface naturally IS an IDE port. "All" you need is an adequate adapter but I don't know if something like that is readily available or would have to be built by yourself.

Hmmm, and even if I find a way to get an IDE-to-flash to work there, I still have the problem of the physical location of this thing… it is a 2U server made for only 2 SCSI HDD...

Thanks,

Perry

don't know about raid ( you can always backup the config file ). but you will be better of with 2 pc and carp cluster imo.

ugob

Hmmm, it is a bit expensive to set 2 servers in a datacenter, so I'd rather have only this 2U server, with the most reliable config possible with this kind of hardware.

Thanks,

Matts

How about the Dual CPU ? I have installed it also on a Dual CPU machine and see only one CPU.

ugob

I haven't checked really, but I think it sees only 1 CPU. I don't really care if it sees only one though.

Matts

@ugob:

I haven't checked really, but I think it sees only 1 CPU. I don't really care if it sees only one though.

No I don't care if it sees one, but I want to be sure it uses both.

This is what I see:

kern.smp.cpus: 1

So if it sees one extra CPU for SMP or at total one.

ugob

@ugob:

But does PfSense supports software RAID? I think a LiveCD + floppy is more reliable (at least for running) than a single disk. The only RAID controller I have here is an HP NetRAID 1-M. Will it work?

I just tested and it works with th HP NetRAID 1-M. I'll get 2 small SCSI disks and I'll use that.

Anyone knows how to monitor for failed drives this way? If it was on linux, I'd check /proc, but I don't know about freebsd.

Thanks!

Matts

@ugob:

@ugob:

But does PfSense supports software RAID? I think a LiveCD + floppy is more reliable (at least for running) than a single disk. The only RAID controller I have here is an HP NetRAID 1-M. Will it work?

I just tested and it works with th HP NetRAID 1-M. I'll get 2 small SCSI disks and I'll use that.

Anyone knows how to monitor for failed drives this way? If it was on linux, I'd check /proc, but I don't know about freebsd.

Thanks!

Yes for Linux I know it too, but I will check it.

Were you able to monitor the controller for failing disks ?

ugob

@Matts:

@ugob:

I haven't checked really, but I think it sees only 1 CPU. I don't really care if it sees only one though.

No I don't care if it sees one, but I want to be sure it uses both.

This is what I see:

kern.smp.cpus: 1

So if it sees one extra CPU for SMP or at total one.

Mine sees and uses both.

In top, I can sometimes see process using CPU1 when I hold the spacebar.

From dmesg:```
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 1
cpu1 (AP): APIC ID: 0

ugob

@Matts:

@ugob:

@ugob:

But does PfSense supports software RAID? I think a LiveCD + floppy is more reliable (at least for running) than a single disk. The only RAID controller I have here is an HP NetRAID 1-M. Will it work?

I just tested and it works with th HP NetRAID 1-M. I'll get 2 small SCSI disks and I'll use that.

Anyone knows how to monitor for failed drives this way? If it was on linux, I'd check /proc, but I don't know about freebsd.

Thanks!

Yes for Linux I know it too, but I will check it.

Were you able to monitor the controller for failing disks ?

No, all I found is this:

amrstat is available as a package, but since my pfsense is not connected to the 'net right now, I can't try.

However, since there is no mta on pfsense, another system would have to run a script via ssh and send an e-mail if one drive is failing.

It would be easier using snmp.

Please let me know if you find anything.

http://www.unixadmintalk.com/f41/monitoring-raid-arrays-51889/

ugob

I found something:

# pkg_add -r http://ftp.id.freebsd.org/ports/packages/All/amrstat-20070216.tbz
# rehash
# amrstat
Logical volume 0:       degraded (16.96 GB, RAID1)
Physical drive 0:0      rebuild
Physical drive 0:1      online

Now you can use a remote server to execute amrstat remotely via ssh and grep for 'degraded'. If grep returns 0, send an e-mail.

Ugo

ugob

See this post to follow-up on the monitoring… http://forum.pfsense.org/index.php/topic,5263.0.html

Matts

I have installed the AMRSTAT Port, but I can;t get it working.

FreeBSD is actually quite new for me :) If it was Linux.. it was more simple ;)

ugob

Look http://forum.pfsense.org/index.php/topic,5252.msg31671.html#msg31671

3 simple commands.

Matts

@ugob:

I found something:
# pkg_add -r http://ftp.id.freebsd.org/ports/packages/All/amrstat-20070216.tbz
# rehash
# amrstat
Logical volume 0:       degraded (16.96 GB, RAID1)
Physical drive 0:0      rebuild
Physical drive 0:1      online
Now you can use a remote server to execute amrstat remotely via ssh and grep for 'degraded'. If grep returns 0, send an e-mail.

Ugo

Hi,

I already installed it using this:

pkg_add -r amrstat

This is what I get because I already installed the latest version I thought

# pkg_add -r http://ftp.id.freebsd.org/ports/packages/All/amrstat-20070216.tbz
Fetching http://ftp.id.freebsd.org/ports/packages/All/amrstat-20070216.tbz... Done.
pkg_add: package 'amrstat-20070216' or its older version already installed
#

And I get this now:


# amrstat
open: No such file or directory
# where amrstat
/usr/local/sbin/amrstat
# /usr/local/sbin/amrstat
open: No such file or directory
# /usr/local/sbin/amrstat --help
amrstat: illegal option -- -
usage: /usr/local/sbin/amrstat [-a num] [-b] [-c ctlr|-f dev] [-g] [-l vol]
                [-p drive|-s bus[:target]] [-t usec] [-v]

        -a num          number of retries
        -b              battery status
        -c ctrl         controller ID
        -f dev          device path
        -g              print global parameters
        -l vol          logical volume ID
        -p drive        physical drive ID
        -s bus[:target] SCSI bus (and optinal target)
        -t usec         sleep time between retries
        -v              verbose output
#

ugob

Maybe your RAID controller is not supported by this program. How are labeled your disks?

Matts

@ugob:

Maybe your RAID controller is not supported by this program.

I will check it, I thought it was.
How are labeled your disks?

What do you mean by this ?

ugob

Give me the output of 'df'

Matts

@ugob:

Give me the output of 'df'

/dev/idad0s1a 13133670 75896 12007082 1% /
devfs 1 1 0 100% /dev
/dev/md0 1710 26 1548 2% /var/run
devfs 1 1 0 100% /var/dhcpd/dev

Btw, I was thinking of using smartmontools. After installing this I see that libcam* is missing.

Or we have to get a full install tree, what not should be it.

ugob

@Matts:

@ugob:

Give me the output of 'df'

/dev/idad0s1a 13133670 75896 12007082 1% /
devfs 1 1 0 100% /dev
/dev/md0 1710 26 1548 2% /var/run
devfs 1 1 0 100% /var/dhcpd/dev

Your RAID controller doesn't use the amr driver, so you can't use amrstat

@Matts:

Btw, I was thinking of using smartmontools. After installing this I see that libcam* is missing.

Or we have to get a full install tree, what not should be it.

If you get smartmontools, how will you get notified of a failed disk?

Matts

@ugob:

@Matts:

@ugob:

Give me the output of 'df'

/dev/idad0s1a 13133670 75896 12007082 1% /
devfs 1 1 0 100% /dev
/dev/md0 1710 26 1548 2% /var/run
devfs 1 1 0 100% /var/dhcpd/dev

Your RAID controller doesn't use the amr driver, so you can't use amrstat

@Matts:

Btw, I was thinking of using smartmontools. After installing this I see that libcam* is missing.

Or we have to get a full install tree, what not should be it.

If you get smartmontools, how will you get notified of a failed disk?

Ow damn, just me… Brain Fart ? ;)

ugob

I think I found how to do it, using an Expect script. I haven't tried it yet 'cause I don't have the pfsense machine close, but I will as soon as I can boot it up.

Here is the script:

http://bash.cyberciti.biz/security/sshlogin.exp.php

This script is executed from a remote host, on which expect is installed. I'm just beginning with expect, so I hard-coded all my values in the script. I know this exposes the firewall root password in a file, but I think I can live with that until I find a better way to be alerted when a drive fails.

#! /bin/bash

RETVAL=0

/home/user/expect_amrstat > /home/user/expect_degraded

grep Degraded /home/user/expect_degraded >dev/null

RETVAL=$?
if [ "$RETVAL" = 0 ]; then
        echo "Array degraded on firewall"  |mail -s "Array Degraded" me@mydomain.com
fi

The script /home/user/expect_amrstat simply runs amrstat on the firewall and outputs the result.