Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Tutorial: How to configure squid to use a separate hard drive as its cache

    Scheduled Pinned Locked Moved
    Documentation
    1
    1
    28.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      belikeyeshua
      last edited by

      I've spent literally weeks trying to get my pfsense/squid setup finished… and its finally nearly complete! I tell you what, if someone was paying me only $7.00/hour to get this thing set up, they would be spending hundreds.

      Before I get into anything, I'll just say that I'm trying to make this as noob proof as possible. That said, I hope this will help those who are new to FreeBSD/pfsense.

      The hardest part for me was that I was having a very hard time configuring squid to use a different hard drive. As some of you know, using a separate hard drive is faster... and you usually get more space that way too. The problem is, however, that caching on a separate hard drive is not officially supported by pfsense. Therefore, you might run into problems and you might have to spend hours reading the freeBSD handbook and searching through threads, maybe even posting threads... sometimes getting "RTF's." Nevertheless, I think the process is quite straightforward and its definitely worth a try!

      Here it is:

      If you've not done so already, connect your hard drive and configure jumper settings (master, slave) if needed.

      You will need to delete all data from the drive. Then you have to create a slice, or partition. You can do this manually in the console but I've personally ran into many problems doing it that way. A much better way, I've found is to use a pfsense liveCD.

      Insert the CD and reboot the computer. When you get to the part where it asks if you want to enter Recovery Mode or Installer Mode, hit "I" to enter the installer. When you get to the man install screen and it asks you if you want to do a quick install or a custom install, hit 'custom install.' We will not be installing pfsense now, but just partitioning the hard drive.

      Next, it will ask you what hard drive you want to install it on. Be sure to choose the right one. If your drive is a IDE like mine and its a slave, then it will be ad1.  Just keep continuing through the setups... it will go to the disk geometry screen, and you hit "accept this geometry." And then it will want you to create a slice (I think thats what its called), so you'll create one. Then it will want you to install bootblocks. You should just skip this step. Then it will want you to partition it. When you get to this screen, you need to delete the default swap partition and change the mount point to something like /squid/cache or /cache/squid. Or, it might just work with /cache. Then you will hit continue...

      At this point, it will want to format the partition and begin the installation of pfsense. I've found that for some reason, pfsense does not format the partition correctly and it comes back with an error. This might be because it is confused because there is no swap partition, I don't know. But, this problem is easily remedied by manually formatting the hard drive.

      So,  at this point you need to keep on hitting the "return to..." buttons until you are back to the main install page. There you will hit 'reboot' and once its rebooted, remove the CD. Then, it should load back into your pfsense HD install.

      From there, you'll need to hit 8 to enter the shell where you can type in commands. Here, you need to type```
      newfs /dev/ad1s1a

      
After that is finished, we need to edit /etc/fstab. Type "ee /etc/fstab" (without quotes) and add the following line:

      /dev/ad1s1a            /cache/squid/    ufs    rw              1      1

      
Create the /cache directory (for pfsense to mount your hard drive on). Do that with```
mkdir /cache/squid
```again, chaning the "/cache/squid" so that it matches your setup.

Mount the hard drive by typing```
mount -a
```. Then type in df -h to make sure its mounted. it should look something like this:```
# df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad0s1a    1.9G    177M    1.6G    10%    /
devfs          1.0K    1.0K      0B   100%    /dev
/dev/md0       3.6M     28K    3.3M     1%    /var/run
devfs          1.0K    1.0K      0B   100%    /var/dhcpd/dev
/dev/ad1s1a     54G     22M     50G     0%    /cache/squid

      Now all thats left to do is to enter your pfsense GUI as you normally would, and go to Services –> Squid --> Cache Management. Where it asks you where the cache is to be stored, enter "cache/squid"

      Now, if all went well, you should be done! Check and make sure that there are no errors in your System Log... Satus --> System Logs. It may be necessary for you to manually create the swap directories. In which case, go back to the command prompt and type in```
      squid -z

      
Its also a good idea to check and make sure that it create your swap directories. Of course, if squid-z comes back with an error, then it obviously did not. But, if it did then typing in ls /cache/squid will show you something like this:```
# ls /cache/squid
.snap		05		0B		11		17		1D
00		06		0C		12		18		1E
01		07		0D		13		19		1F
02		08		0E		14		1A		swap.state
03		09		0F		15		1B
04		0A		10		16		1C

      If it did not work, it will only show .snap and nothing else. So, if ls /cache/squid shows the above then congratulations! Now the only thing left for you to do is to delete your other squid cache by typing```
      rm -R /var/squid/cache/*

      
Some Troubleshooting:

If you were unable to create swap directories (as I was), it might say something like this:```
# squid -z
2009/12/24 20:56:44| parseConfigFile: squid.conf:62 unrecognized: 'delay_pools'
2009/12/24 20:56:44| parseConfigFile: squid.conf:63 unrecognized: 'delay_class'
2009/12/24 20:56:44| parseConfigFile: squid.conf:64 unrecognized: 'delay_parameters'
2009/12/24 20:56:44| parseConfigFile: squid.conf:65 unrecognized: 'delay_initial_bucket_level'
2009/12/24 20:56:44| parseConfigFile: squid.conf:66 unrecognized: 'delay_access'
2009/12/24 20:56:44| Creating Swap Directories
FATAL: Failed to make swap directory /cache/squid/00: (13) Permission denied
Squid Cache (Version 2.7.STABLE7): Terminated abnormally.
CPU Usage: 0.001 seconds = 0.001 user + 0.000 sys
Maximum Resident Size: 3664 KB
Page faults with physical i/o: 0

      Also, I was getting tons of errors in my system log. Things like:

      Dec 24 20:53:07 	squid[43341]: Squid Parent: child process 43504 started
Dec 24 20:53:08 	squid[43504]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:53:08 	kernel: pid 43504 (squid), uid 62: exited on signal 6
Dec 24 20:53:08 	squid[43341]: Squid Parent: child process 43504 exited due to signal 6
Dec 24 20:53:08 	squid[43341]: Exiting due to repeated, frequent failures
Dec 24 20:53:55 	Squid_Alarm[43599]: Squid has exited. Reconfiguring filter.
Dec 24 20:53:55 	Squid_Alarm[43601]: Attempting restart...
Dec 24 20:53:55 	squid[43608]: Squid Parent: child process 43611 started
Dec 24 20:53:55 	squid[43611]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:53:55 	kernel: pid 43611 (squid), uid 62: exited on signal 6
Dec 24 20:53:55 	squid[43608]: Squid Parent: child process 43611 exited due to signal 6
Dec 24 20:53:58 	Squid_Alarm[43615]: Reconfiguring filter...
Dec 24 20:53:58 	squid[43608]: Squid Parent: child process 43618 started
Dec 24 20:53:58 	squid[43618]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:53:58 	kernel: pid 43618 (squid), uid 62: exited on signal 6
Dec 24 20:53:58 	squid[43608]: Squid Parent: child process 43618 exited due to signal 6
Dec 24 20:53:59 	Squid_Alarm[43693]: Squid has resumed. Reconfiguring filter.
Dec 24 20:54:01 	squid[43608]: Squid Parent: child process 43763 started
Dec 24 20:54:02 	squid[43763]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:54:02 	kernel: pid 43763 (squid), uid 62: exited on signal 6
Dec 24 20:54:02 	squid[43608]: Squid Parent: child process 43763 exited due to signal 6
Dec 24 20:54:05 	squid[43608]: Squid Parent: child process 43768 started
Dec 24 20:54:05 	squid[43768]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:54:05 	kernel: pid 43768 (squid), uid 62: exited on signal 6
Dec 24 20:54:05 	squid[43608]: Squid Parent: child process 43768 exited due to signal 6
Dec 24 20:54:08 	squid[43608]: Squid Parent: child process 43772 started
Dec 24 20:54:08 	squid[43772]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:54:08 	kernel: pid 43772 (squid), uid 62: exited on signal 6
Dec 24 20:54:08 	squid[43608]: Squid Parent: child process 43772 exited due to signal 6
Dec 24 20:54:08 	squid[43608]: Exiting due to repeated, frequent failures
Dec 24 20:54:55 	Squid_Alarm[43880]: Squid has exited. Reconfiguring filter.
Dec 24 20:54:55 	Squid_Alarm[43882]: Attempting restart...
Dec 24 20:54:56 	squid[43889]: Squid Parent: child process 43892 started
Dec 24 20:54:56 	squid[43892]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:54:56 	kernel: pid 43892 (squid), uid 62: exited on signal 6
Dec 24 20:54:56 	squid[43889]: Squid Parent: child process 43892 exited due to signal 6
Dec 24 20:54:59 	Squid_Alarm[43896]: Reconfiguring filter...
Dec 24 20:54:59 	squid[43889]: Squid Parent: child process 43899 started
Dec 24 20:54:59 	squid[43899]: Failed to verify one of the swap directories, Check cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid for the first time.
Dec 24 20:54:59 	kernel: pid 43899 (squid), uid 62: exited on signal 6

      For this, it might be worthwhile to read the squid faq page where it talks about this: http://wiki.squid-cache.org/SquidFaq/TroubleShooting.

      For me, however, that did not solve my problem. I simply had to change privileges of the file by typing in```
      chown proxy:proxy /cache/squid

      
So, I hope this will help other people. I'm sorry, I'm not very good at making how-tos but hopefully this will work. If someone wants to rewrite it and post it in with the pfsense tutorials and howtos then be my guest!
      1 Reply Last reply Reply Quote 3
      • JonathanLeeJ JonathanLee referenced this topic on
      • First post
        Last post