Unstable pfSense configuration
I'm using pfSense since February 2014 and after the latest update and installation of Cron package the system became very unstable, it keeps hanging every 24 hours or so. When it hangs I can't connect to the system at all, no ping, no SSH, nothing, no WebGUI access, nothing. I'm physically away so I can't connect a monitor and check the state, the only option I've got is to call my girlfriend and tell her to power cycle the system. After power cycle, it works well for couple of hours and then it hangs.
Performed "system restore" to an early state before any modification - no effect. Removed Cron - no effect (even worse).
It's a simple configuration: One WAN and One LAN, used primary for OpenVPN server. The WAN is with static IP, LAN side is with DHCP server.
First VPN server is TAP bridged with LAN. Second VPN server is TUN, separate IP subnet from my LAN. All VPN are working fine, no errors, no packets drops or collision.
Basically I could'n t notice any system error. After a hard reset there is nothing in the system logs, showing that there was a problem with my system. That's why I'm lost and I need your help.
Here are the hardware specs:
Motherboard: GIGABYTE GA-C1037UN-EU
RAM: KINGSTON 2GB DDR3 1600 HYPER X
HDD: 16GB Apacer SDM4-M APSDM016G15AN-CCM 22pin 90° Industrial S-ATA DOM
Case: E-mini I5 Black, Brushed Aluminium, 120W DC/DC + 12V/5A adapter, Mini-ITX
Latest pfSense build.
Please, advise me how to start with the troubleshooting. I'll appreciate if you guide me trough, since I'm quite desperate at the moment.
Thanks, I'm looking forward to hearing from you!
What connections do you have into the pfsense box?
Are you using the VPN links to connect in or have you opened HTTPS to the outside (perhaps on a different port?)
Same questions on SSH?
If you're try to connect via VPN, perhaps the OpenVPN connections are dying for some reason.
Is it possible to get your girlfriend to try a few simple commands at the pfsense console when the box dies?
Rather than just reboot, it would be worthwhile to know whether the box is actually still alive and just not communicating or if it has actually hung.
The console screen might have some status messages that would help.
I'm glad to see some replies to my topic. Thank you!
Regarding the connections to my pfSense box, I have set up https and SSH. When the system hangs, there is now way I can connect to it, neither https or SSH. However the system does not shut down itself, because the power button is still on [according to the feedback from my girlfriend].
Tonight hopefully I will guide her over the phone to connect some sort of display and keyboard to the pfSense box and together with her, we will perform some troubleshooting, but before that, can you please advise me what sort of screens and commands to type, in order to get the system state.
This type of freeze with nothing in the logs could be a hardware problem, say due to poor cooling, memory issues, power supply, etc. Do you have any thermal sensors selected (system, advanced, miscellaneous …)? You can see the temp shown on the dashboard, but it doesn't appear that any temperatures are logged in the rrd graphs.
Assuming you are using a pfSense image with VGA output and your machine has a VGA port, simply plug in a monitor & keyboard to the box. Try to do that before she does the reboot. After re-booting, she should see messages scrolling by, and end up with a text menu on the screen. From there, next time it freezes, she can see if it still responds locally.
*** Welcome to pfSense 2.2-ALPHA-pfSense (amd64) on pfSense *** WAN (wan) -> em0 -> v4/DHCP4: 10.5.60.158/23 LAN (lan) -> em1 -> v4: 192.168.1.1/24 0) Logout (SSH only) 8) Shell 1) Assign Interfaces 9) pfTop 2) Set interface(s) IP address 10) Filter Logs 3) Reset webConfigurator password 11) Restart webConfigurator 4) Reset to factory defaults 12) pfSense Developer Shell 5) Reboot system 13) Upgrade from console 6) Halt system 14) Disable Secure Shell (sshd) 7) Ping host 15) Restore recent configuration Enter an option:
Thank you very much for your help!
Fortunately after extensive troubleshooting, I found that my subnet mask on my WAN interface was wrong, because pfSense requires CIDR prefix, I just got it wrong during my initial configuration of my box.
Now after this fix, the box behaves stable. I'll continue monitoring the system, but I think this was the root cause of my problem.
Thanks to all of you, who take time to help me!
Have a nice weekend!