Using a hard-disk in a Watchguard Firebox X750e for cache/log storage

Steve Evans

The memory device /dev/md1 is still being created/mounted so as Stephen says you've got an issue with the startup script.

Steve

sg1

No. I see now, that it is incorrect. I still have the original being mounted due to it. My question is, where specifically in the original rc.embedded do I change with the Steve's changes as there are other calls in the script that I am not sure are needed or not. Apologies, just new to the pfsense and Watchguard platforms so I am sure my questions are on the the dumber side…. The original rc.embedded is below:

rc.embedded - embedded system specific startup information

For pfSense

Size of /tmp

USE_MFS_TMP_SIZE=/usr/bin/grep use_mfs_tmp_size /cf/conf/config.xml | /usr/bin/cut -f2 -d'>' | /usr/bin/cut -f1 -d'<'

if [ ! -z ${USE_MFS_TMP_SIZE} ] && [ ${USE_MFS_TMP_SIZE} -gt 0 ]; then
tmpsize="${USE_MFS_TMP_SIZE}m"
else
tmpsize="40m"
fi

Size of /var

USE_MFS_VAR_SIZE=/usr/bin/grep use_mfs_var_size /cf/conf/config.xml | /usr/bin/ cut -f2 -d'>' | /usr/bin/cut -f1 -d'<'
if [ ! -z ${USE_MFS_VAR_SIZE} ] && [ ${USE_MFS_VAR_SIZE} -gt 0 ]; then
varsize="${USE_MFS_VAR_SIZE}m"
else
varsize="60m"
fi

Run some initialization routines

[ -f /etc/rc.d/uzip ] && /etc/rc.d/uzip start

echo -n "Setting up memory disks…"
mdmfs -S -M -s ${tmpsize} md /tmp
mdmfs -S -M -s ${varsize} md /var

Create some needed directories

/bin/mkdir -p /var/db

Ensure vi's recover directory is present

/bin/mkdir -p /var/tmp/vi.recover/
echo " done."

Steve Evans

Here is the complete patched /etc/rc.embedded.

#!/bin/sh
#
# rc.embedded - embedded system specific startup information
# For pfSense

# Size of /tmp
USE_MFS_TMP_SIZE=`/usr/bin/grep use_mfs_tmp_size /cf/conf/config.xml | /usr/bin/cut -f2 -d'>' | /usr/bin/cut -f1 -d'<'`
if [ ! -z ${USE_MFS_TMP_SIZE} ] && [ ${USE_MFS_TMP_SIZE} -gt 0 ]; then
	tmpsize="${USE_MFS_TMP_SIZE}m"
else
	tmpsize="40m"
fi

# Size of /var
USE_MFS_VAR_SIZE=`/usr/bin/grep use_mfs_var_size /cf/conf/config.xml | /usr/bin/cut -f2 -d'>' | /usr/bin/cut -f1 -d'<'`
if [ ! -z ${USE_MFS_VAR_SIZE} ] && [ ${USE_MFS_VAR_SIZE} -gt 0 ]; then
	varsize="${USE_MFS_VAR_SIZE}m"
else
	varsize="60m"
fi

# Run some initialization routines
[ -f /etc/rc.d/uzip ] && /etc/rc.d/uzip start

echo -n "Setting up memory disks..."
mdmfs -S -M -s ${tmpsize} md /tmp

# If a hard disk is installed then mount that on /var
# otherwise use a ramdisk
harddisk="/dev/ad1s1a"
if [ -c $harddisk ]
then
	echo -n "Using /var physical disk..."
	mount -o noatime $harddisk /var

	# Ensure /var/run is removed on boot before daemons are started
	# It should arguably be a tmpfs, but this works fine
	rm -r /var/run

	# sshd won't start if /var/empty exists on boot
	rm -r /var/empty
else
	echo -n "Using /var memory disk..."
	mdmfs -S -M -s ${varsize} md /var

	# Create some needed directories
	/bin/mkdir -p /var/db

	# Ensure vi's recover directory is present
	/bin/mkdir -p /var/tmp/vi.recover/
fi

echo " done."

sg1

Thank you so much Steve! Works perfectly. Thanks for your patience….

atrocity

only my result of a x5500e with a ide SSD from CWC :

/dev/ad1s1a
512 # sectorsize
57872344064 # mediasize in bytes (53G)
113031922 # mediasize in sectors
0 # stripesize
40448 # stripeoffset
112134 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
OW140822AS1514144 # Disk ident.

Seek times:
Full stroke: 250 iter in 0.040570 sec = 0.162 msec
Half stroke: 250 iter in 0.037379 sec = 0.150 msec
Quarter stroke: 500 iter in 0.125276 sec = 0.251 msec
Short forward: 400 iter in 0.093555 sec = 0.234 msec
Short backward: 400 iter in 0.096201 sec = 0.241 msec
Seq outer: 2048 iter in 0.169674 sec = 0.083 msec
Seq inner: 2048 iter in 0.123082 sec = 0.060 msec
Transfer rates:
outside: 102400 kbytes in 1.124830 sec = 91036 kbytes/sec
middle: 102400 kbytes in 1.128009 sec = 90779 kbytes/sec
inside: 102400 kbytes in 1.125042 sec = 91019 kbytes/sec

seems that i don't have to tune anything.
I will make the same test with a x750e soon, will let you know if somebody is interrested :)
Regards

stephenw10

What tool are you testing that with? Did you enable TRIM? Be interesting to see how those figures vary over time once you've moved a few gigs across the drive.

Steve

atrocity

hi,

i simply use this :

diskinfo -tv /dev/ad1

Trim … i did a try to enable it, but our server don't want it.
Don't remember the problem or the error, sorry.

Steve Evans

Good to see how quick this can run. If my ISP increases speed x5 I may have to get an SSD!

Steve

Steve Evans

Having updated to 2.2 the support for my hard-drive now appears rather worse.

The atacontrol utility which did exactly what I needed has been superseded by camcontrol. This reports that the disk is in polled mode.

$ camcontrol negotiate /dev/ada1 -v 
Current parameters:
(pass1:ata0:0:1:0): ATA mode: PIO4
(pass1:ata0:0:1:0): ATAPI packet length: 0
(pass1:ata0:0:1:0): PIO transaction length: 8192
(pass1:ata0:0:1:0): tagged queueing: disabled
ata0: SIM/HBA version: 1
ata0: supports SDTR message
ata0: scan bus sequentially
ata0: HBA engine count: 0
ata0: maximum target: 1
ata0: maximum LUN: 0
ata0: highest path ID in subsystem: 0
ata0: initiator ID: 0
ata0: SIM vendor: FreeBSD
ata0: HBA vendor: ATA
ata0: HBA vendor ID: 0x8086
ata0: HBA device ID: 0x266f
ata0: HBA subvendor ID: 0x8086
ata0: HBA subdevice ID: 0x266f
ata0: bus ID: 0
ata0: base transfer speed: 3.300MB/sec
ata0: maximum transfer size: 131072 bytes

I should be able to set the disk to UDMA33 mode however, this is what I get.

$ camcontrol negotiate /dev/ada1 -v -U -M UDMA33 -a
User parameters:
(pass1:ata0:0:1:0): ATA mode: UDMA2
(pass1:ata0:0:1:0): ATAPI packet length: 0
(pass1:ata0:0:1:0): PIO transaction length: 131072
(pass1:ata0:0:1:0): tagged queueing: enabled
ata0: SIM/HBA version: 1
ata0: supports SDTR message
ata0: scan bus sequentially
ata0: HBA engine count: 0
ata0: maximum target: 1
ata0: maximum LUN: 0
ata0: highest path ID in subsystem: 0
ata0: initiator ID: 0
ata0: SIM vendor: FreeBSD
ata0: HBA vendor: ATA
ata0: HBA vendor ID: 0x8086
ata0: HBA device ID: 0x266f
ata0: HBA subvendor ID: 0x8086
ata0: HBA subdevice ID: 0x266f
ata0: bus ID: 0
ata0: base transfer speed: 3.300MB/sec
ata0: maximum transfer size: 131072 bytes
Unit is not ready
(pass1:ata0:0:1:0): TEST UNIT READY. CDB: 00 00 00 00 00 00 
(pass1:ata0:0:1:0): CAM status: CCB request was invalid
Test Unit Ready failed

I've tried all manner of variations. Any ideas to get camcontrol to do what a simple

atacontrol mode ad1 UDMA6

would do before?

Thanks,

Steve

stephenw10

Haven't you alread set ata0 to PIO4 only though?
The HD and CF are on the same IDE channel right?

Steve Evans

Hi Stephen,

This is what I'm trying to get my head round at the moment, reading the FreeBSD documentation to better understand the nomenclature.

If I boot without specifying

hint.ata.0.mode=PIO4

I see the following on the console

ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <sandisk sdcfh-004g="" hdx="" 6.02=""> CFA-0 device
ada0: Serial Number ABZ042211193008
ada0: 66.700MB/s transfers (UDMA4, PIO 512bytes)
ada0: 3815MB (7813120 512 byte sectors: 16H 63S/T 7751C)
ada0: Previously was known as ad0
ada1 at ata0 bus 0 scbus0 target 1 lun 0
ada1: <st9402115a 3.01=""> ATA-6 device
ada1: Serial Number 5PV09ZED
ada1: 100.000MB/s transfers (UDMA5, PIO 8192bytes)
ada1: 38154MB (78140160 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad1</st9402115a></sandisk>

Rather than

ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <sandisk sdcfh-004g="" hdx="" 6.02=""> CFA-0 device
ada0: Serial Number ABZ042211193008
ada0: 16.700MB/s transfers (PIO4, PIO 512bytes)
ada0: 3815MB (7813120 512 byte sectors: 16H 63S/T 7751C)
ada0: Previously was known as ad0
ada1 at ata0 bus 0 scbus0 target 1 lun 0
ada1: <st9402115a 3.01=""> ATA-6 device
ada1: Serial Number 5PV09ZED
ada1: 16.700MB/s transfers (PIO4, PIO 8192bytes)
ada1: 38154MB (78140160 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad1</st9402115a></sandisk>

As you point out, this setting affects them both. I've been trying to understand what granularity of control the driver gives me. From the ata man page there's the following which suggests per device control, but I don't currently understand what "specified device" means exactly.

@https://www.freebsd.org/cgi/man.cgi?query=ata(4)&sektion=:

hint.ata.X.devX.mode
limits the initial ATA mode for the specified device on the specified channel.

hint.ata.X.mode
limits the initial ATA mode for every device on the specified channel.

I'm not sure how the master/slave IDE arrangement maps onto the above. Any pointers much appreciated!

Thanks,

Steve

stephenw10

Pretty sure the CF slot is always master if a CF card is in it so I would think it's dev0.

On my test box here which I haven't set loader.conf.local on it will boot if I use:

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel] in 4 seconds...

Type '?' for a list of commands, 'help' for more detailed help.
OK set hint.ata.0.dev0.mode=PIO4
OK boot
Booting...

So I'd say you're good with that.

Steve

Steve Evans

Thank you sir, you're a scholar and a gentleman! :)

Your suggestion works a treat. I clearly had a bit of a mental block on what "devX" could be. I'd tried "0", but it hadn't occurred to me to used "dev0". :-[ I'm clearly not familiar enough with FreeBSD naming conventions.

[code]ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <sandisk sdcfh-004g="" hdx="" 6.02="">CFA-0 device
ada0: Serial Number ABZ042211193008
ada0: 16.700MB/s transfers (PIO4, PIO 512bytes)
ada0: 3815MB (7813120 512 byte sectors: 16H 63S/T 7751C)
ada0: Previously was known as ad0
ada1 at ata0 bus 0 scbus0 target 1 lun 0
ada1: <st9402115a 3.01="">ATA-6 device
ada1: Serial Number 5PV09ZED
ada1: 100.000MB/s transfers (UDMA5, PIO 8192bytes)
ada1: 38154MB (78140160 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad1

I'm now getting the performance I need for my disk used for logs and caching.

diskinfo -tv /dev/ad1
/dev/ad1
        512             # sectorsize
        40007761920     # mediasize in bytes (37G)
        78140160        # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        77520           # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        5PV09ZED        # Disk ident.

Seek times:
        Full stroke:      250 iter in   7.926598 sec =   31.706 msec
        Half stroke:      250 iter in   6.203416 sec =   24.814 msec
        Quarter stroke:   500 iter in   9.877891 sec =   19.756 msec
        Short forward:    400 iter in   3.476113 sec =    8.690 msec
        Short backward:   400 iter in   2.381591 sec =    5.954 msec
        Seq outer:       2048 iter in   0.197696 sec =    0.097 msec
        Seq inner:       2048 iter in   0.179296 sec =    0.088 msec
Transfer rates:
        outside:       102400 kbytes in   3.135350 sec =    32660 kbytes/sec
        middle:        102400 kbytes in   3.745807 sec =    27337 kbytes/sec
        inside:        102400 kbytes in   5.382397 sec =    19025 kbytes/sec

Steve</st9402115a></sandisk>

stephenw10

Nice! I'm going to have to try that sometime. :)

Steve

stephenw10

In testing the HD issue I ran that benchmark. Thought my result might ammuse you:

[2.2-RELEASE][root@pfSense.localdomain]/root: diskinfo -tv /dev/ada0
/dev/ada0
        512             # sectorsize
        20003880960     # mediasize in bytes (19G)
        39070080        # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        38760           # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        32K60131T       # Disk ident.

Seek times:
        Full stroke:      250 iter in  11.840086 sec =   47.360 msec
        Half stroke:      250 iter in   8.878876 sec =   35.516 msec
        Quarter stroke:   500 iter in  15.334866 sec =   30.670 msec
        Short forward:    400 iter in   8.639504 sec =   21.599 msec
        Short backward:   400 iter in   7.131472 sec =   17.829 msec
        Seq outer:       2048 iter in   0.282496 sec =    0.138 msec
        Seq inner:       2048 iter in   1.086055 sec =    0.530 msec
Transfer rates:
        outside:       102400 kbytes in  23.521335 sec =     4353 kbytes/sec
        middle:        102400 kbytes in  20.436978 sec =     5011 kbytes/sec
        inside:        102400 kbytes in  48.862760 sec =     2096 kbytes/sec

Something not right. ::) Yet:

[2.2-RELEASE][root@pfSense.localdomain]/root: dmesg | grep ada
ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <toshiba mk2018gap="" m1.42="" a="">ATA-5 device
ada0: Serial Number 32K60131T
ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes)
ada0: 19077MB (39070080 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad0
Trying to mount root from ufs:/dev/ada0s1a [rw]...</toshiba>

Just a knackered old drive? Works fine though, as long as you're not in a hurry!

Steve

Steve Evans

Ouch. Not much use as a cache though as it's no faster than fiber broadband!

Steve

MacUsers

@stephenw10:

[2.2-RELEASE][root@pfSense.localdomain]/root: dmesg | grep ada
ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <toshiba mk2018gap="" m1.42="" a="">ATA-5 device
ada0: Serial Number 32K60131T
ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes)
ada0: 19077MB (39070080 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad0
Trying to mount root from ufs:/dev/ada0s1a [rw]...</toshiba>

I was just looking in some old posts - Does it mean you have TRIM enabled on your TOSHIBA MK2018GAP drive?

stephenw10

No. I'm pretty sure that device is way too old to support TRIM
TRIM only applies to SSDs anyway.

Steve

MacUsers

That's what I knew as well but this line in your dmesg:

ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes)

made me think otherwise.

stephenw10

What part of that implies TRIM?
That's the maximum speed on the interface, which is far, far faster than the drive can manage in this case. ;)

Steve