High CPU usage and random Squid misbehaviour



  • Hi Everyone

    First off, I shall introduce myself, as this is my first post in this forum. Not sure if there are many other pfSensers out in Africa, but after a few years working in the corporate sector in the UK, I decided enough was enough, and I now live in Marangu, on the south eastern slopes of Kilimanjaro in Tanzania. Here i run a small computer centre, which serves two purposes, a teaching/learning/resource centre, and also an internet cafe for use by everyone. In Tanzania, we believe we are the highest (altitude) public internet cafe, and have a fairly strong case for saying in the whole of Africa. Our centre resides around 2200m up above the town of Marangu, in a small village called Mshiri.

    I have been here a year, and have played and toiled with several linux based firewall distributions, Untangle, Smoothwall, IPCop and have decided to stay with pfSense, as it gives me the most of what I need, with the lowest maintenance. My network is essentially five desktops all running ubuntu oneiric, using pfSense's voucher system on the captive portal, a few laptops from staff, a wifi access point that directs DHCP etc through to the pfsense box, and then a teaching room with 8 XP machines, that all have internet access without the captive portal.

    THe hardware I run this on is fairly basic, but I believe is more than up to the task, it's a PIII 1000mhz with 512MB ram and 80GB hard drive. Two netwoork cards (both 100mbit oldies I nicked from some old PII HP Vectras), one connected to a Huawei E220 wifi/3G router and one connected to the unmanaged 24 port switch with the D-Link wifi point patched into that also. All runs very smoothly I have to say, and given that we only have EDGE connectivity, my customers all say that the experience is better than that ran by locals in the towns, where they have full 7.2MBps HSUPA - this is all down to pfSense, and Squid, running in "offline" mode and caching everything it sees.

    So on to my query…

    It's been installed and running fine for around two weeks, after a full re-install from scratch on new hardware when I downloaded the 2.0.1 release just after christmas. Since then, Squid seems to decide to not play from time to time, and i noticed yesterday that the CPU usage on the dashboard shows as 100% all the time. SSH into it from one of the ubuntu boxes and I notice that the top 10 or 15 processes are all PHP. If i restart the box, it goes away, and googling this took me to a few different pages on here, relating to Snort (which I don't have installed or use) or idlepoll, which is not switched on.

    The quirkiness of our system is that we are in Africa. The internet is slow, if it works at all, and most disruptively, we have power cuts almost daily. I have protected the switch, pfSense and main networking elements with a surge protector, but our UPS batteries have long since died, and basically, when the power company cut the power to the village, we lose power. We are a UK based charity, but run on very limited budget, and so buying new hardware/batteries/etc is not really an option. What I love about being a g33k is finidn ways to get things working without spending money, hence running fully legal copies of Ubuntu, and installing a 100% free and open source firewall on an otherwise redundant PC.

    WHat troubleshooting steps can I take to see what causes this when it happens? My guess is that it is the squid process tidying up the filesystem, or something similar follwing the loss of power each time it happens, but is there a way to see the progress of such things??

    On a seperate note, I have installed one of these in a local hotel, again, using old hardware to save them money, and now they have a ticketing system for their patrons internet access, but I want to enable Windows Update caching for their office (10 XP machines that have SP3, but nothing newer since then) which only has a ~150kbps satlink. Downloading overnight etc is not a problem, but downloading 10 times is a problem...


  • Netgate Administrator

    Awesome story. Another day in London for me!  ::)

    Shame I can't help you with your main problem but have a look here for your second question:

    http://doc.pfsense.org/index.php/Squid_Package_Tuning#Caching_Windows_Updates

    Steve



  • cheers Steve.

    I have ran through that and then tested on a vanilla XP SP3 box, then went to a second machine, expecting the updates to zoom down, but they were being downloaded at the same pitiful 15-20kb/s as I waited on the first box. It matters not for my centre, as the machines are protected, and viruses etc are something we don't worry about, if we struggle to get facebook loading, I doubt any hackers/etc will be trying to hard to break in.

    I really want to implement this on the Hotel's pfsense box, which is work I am effectively commercially charging for: implementing a decent firewall, captive portal and then some backup/recovery solutions.

    If there is anyone who actually first hand has XP updates working, please post the code you have in the extra options for squid… :)



  • @graigchq:

    I have ran through that and then tested on a vanilla XP SP3 box, then went to a second machine, expecting the updates to zoom down, but they were being downloaded at the same pitiful 15-20kb/s as I waited on the first box.

    The Windows update was still downloading on the first box when you started it on the second box?



  • @graigchq:

    i noticed yesterday that the CPU usage on the dashboard shows as 100% all the time. SSH into it from one of the ubuntu boxes and I notice that the top 10 or 15 processes are all PHP.

    Please post a couple of samples of the output from pfSense shell command top -S -H taken when CPU usage is reported as 100%.



  • @wallabybob:

    The Windows update was still downloading on the first box when you started it on the second box?

    no, it was the next day… the updates took several hours to download, and the box was rebooted in between (we cannot leave it on overnight, as the centre is closed, and power is so erratic)

    I'm out on my bike now in town, but will post up the output from the box when I get back up the mountain :)


Log in to reply