Troubleshooting pfSense computer

Creighton

I've been using pfSense for a month or so now and have loved it so far. I have Load Balancing set up, along with a few other things but nothing too major. Overall my requirements are simple. (take a look at my horribly drawn network map attached to get an idea.)

I've had the settings I like up and running now for a couple weeks with no problems on a Compaq 7000US with 3 netgear cards and 128MB RAM, however as of yesterday I've had to stop using pfSense because the computer it's on frequently crashes at seemingly random times. After leaving it off for a day, I can turn it on and it will work completely fine. The problem is the length of time it runs fine is random. Sometimes it will be for hours, other times minutes. Sometimes when several computers are connected, and sometimes when no computers are connected. I've tried looking at the log file but half the time it's blank, and the other times I don't see anything of importance.

Basically I need some tips on how to troubleshoot pfSense to see if its causing the freezing, or possibly bad hardware, etc. I'm really lost. Thanks in advance for any help, and I apologize if I missed something when I searched before I posted. (I didn't find anything on my noob-level.)
map.jpg_thumb

GruensFroeschli

Sounds like a hardware problem.
Have you tried running a RAM-test? Or a Burn-in-test?

Random crashes can also be the cause of a weak PSU.

To test hardware i usually use http://www.ultimatebootcd.com/

Creighton

Thanks for the reply. I used the UBCD and thoroughly tested the RAM, CPU, and HD in my pfSense computer. Everything passed multiple times. I left pfSense running overnight with no WAN/OPT/LAN connections plugged in (basically just letting the computer run with only a monitor and keyboard attached). It didn't crash/freeze.

So here I am now, back using pfSense and everything seems to be working fine. Of course, as I said before, it crashes randomly so it could happen hours from now. I'm using remote syslogging to try to watch over pfSense's logs to hopefully catch something when it crashes. Good plan or not?

I've also attached some more in-depth information retrieved by phpSysInfo via a screenshot. Hopefully nothings wrong there.

Will post back if/when the computer crashes again.

phpsysinfo.JPG_thumb

cmb

When it happens, can you still get into the web interface? The description you've given thus far is pretty broad, narrowing down things will help determine what is happening.

Jantje

Random crashes could also be a result of faulty capacitors. Open up your computer case and check to see if any of the capacitors look like the ones in the attached picture. Note that the top of a capacitor should always be flat. If it's not, then it needs to be replaced. Although it's often easier just to replace the motherboard entirely.

leaking-capacitors.jpg_thumb

Creighton

@cmb:

When it happens, can you still get into the web interface? The description you've given thus far is pretty broad, narrowing down things will help determine what is happening.

When it crashes, I can't ping it, nor access the web interface. The options menu on the actual monitor attached to the pfSense computer is frozen as well. (Pressing any keys/numbered options does nothing.) I've let it sit like that for a while in hopes of recovering itself, but it never does. A restart would bring it back to working order for a random amount of time. (Minutes to hours)

As of right now, I've had it up and running for 48 hours without a single freeze. Seems odd since I have changed very little of the previous configuration it was running on. This is what I did:

leave it unplugged for a few days
tested the RAM, CPU, etc
let pfSense run disconnected from everything overnight
enabled remote syslog'ing to my personal computer's syslog server & disabled writing log files to local ram disk

Thanks for the help so far. As far as faulty capacitors, I will eventually take a look into that. (I'm scared to turn it off now that it seems to be back to functional working order :) )

Valhalla1

I like your network map, hehe.. just curious, from looking at the map it looks almost like you're using pfsense to load balance your cable modem along with maybe your neighbor's cable modem by tapping into his unsecured wireless.. lol

cheesyboofs

hehe, a bit off topic but that's exactly what I thought.

Creighton

@Valhalla1:

I like your network map, hehe.. just curious, from looking at the map it looks almost like you're using pfsense to load balance your cable modem along with maybe your neighbor's cable modem by tapping into his unsecured wireless.. lol

Exactly (but with permission and its a secure access point where pfsense's WAN2 is the only client). My goal was to basically give RoadRunner the middle finger ;D .

Anyway, back on topic - I removed the CMOS battery and the computer hasn't froze since. Is this some weird coincidence, or could the battery actually negatively effect the system/pfsense? I thought it was only used for the internal clock and BIOS.

cheesyboofs

I seriously doubt it.
More likely you enabled some setting in your BIOS freebsd didn't like, (speed stepping, DRAM speed adjustments) and when you pulled the battery it is now running on the 'fail safe' defaults IMHO :)

MikeyB

Wow i was just about to make a post about this

my pfSense box has just started doing this… random crashes, no web interface, cant ping, but all the link lights are on... i have to reboot the system then its fine again

jsmwalker

I would also third this, been running since setup (only 30days ago) and suddenly in the last 4 days has locked up twice, no web interface, no ping, no console…. very weird.

Anyone?

J

cheesyboofs

@jsmwalker & MikeyB

You guy's really need to edit your posts and state what version you are running, I too have seen very similar lockup's but ONLY on version 1.2.1.

http://forum.pfsense.org/index.php/topic,10214.0.html

The first thing to rule out with lockup's is heat! Has the environment your router is got hotter recently? I know my problem isn't heat related because it doesn't happen under 1.2-Release and the BIOS says the CPU is at 42 degrees.

Cheers

jsmwalker

Hi

Yep sorry about that, had posted on another forum about another issue (RRD graphs stopping) and forgot to put version on this, the version i'm running is 1.2-RELEASE built on Sun Feb 24 17:04:58 EST 2008, I can't imagine it is heat in my case, is in a a/c server room, although not as cool as i'd hope it hasn't been hotter then in the last month.

Is very frustrating, as this is such a great product, and yes there are a few bugs but nothing the distracts from the overall excellence, apart from this one issue :-[

J

MikeyB

I just thought i would post here insted of making a new topic…

its really starting to annoy me. it locks up randomly.... but i know whats doing it.... large downloads.... or 2 or more torrents downloading causing pfsense to lock up, computer is on, link lights are on.... cant ping router though...

im running
1.2-RELEASE
built on Sun Feb 24 17:04:58 EST 2008

every time i start a download of server 2008 from microsoft (1.7GB) it will lock up after a few seconds.

jsmwalker

Hi

Thanks for your idea, but have managed to download Vista Business the other day (2.1Gb) with no problem at all (This was from MS, using webbrowser not download manager, and we dont allow Torrents in the office). Have to say I have moved over to other hardware and since that poin haven't had any lock ups at all, but saying that, didn't have any problems with the oroginal hardware for 30days so can't be sure that this may not happen again.

J