Segmentation fault

[lege]

Greetings,

We are running 4 pfSense firewalls on embedded hardware bought from hacom.net, they're all running pfHacom 1.2.3 (rebranded pfSense 1.2.3) from CF drives (nanobsd)

All of these firewalls seem to have problems after a while and almost every command issued using a shell prompt returns a Segmentation Fault error and doesn't run.

We cannot even reboot them or do anything from the console.

Please note however that the actual firewall/packet filtering/routing works even in this condition. Even the webConfigurator mostly works, and so does CARP and everything else. We have one that's been up for 200 days and is still routing but cannot reboot it from the console, and currently do not have physical access to it.

Here's an example:

pfHacom console setup

---

0)  Logout (SSH only)
1)  Assign Interfaces
2)  Set LAN IP address
3)  Reset webConfigurator password
4)  Reset to factory defaults
5)  Reboot system
6)  Halt system
7)  Ping host
8)  Shell
9)  PFtop
10)  Filter Logs
11)  Restart webConfigurator
12)  pfHacom Developer Shell
13)  Upgrade from console
14)  Disable Secure Shell (sshd)

Enter an option: 8

Segmentation fault (core dumped)

Sometimes the segmentation faults occur right after they boot up, not even the webConfigurator gets a chance to start, so they won't be accessible via http. We'd need to reboot them another time for the error to go away and eventually start working.

Any ideas on what we could do? Also, there's nothing related to these in the System Log when it starts occuring.

[podilarius]

Can you provide details about the hardware you are running this on? What packages are installed?

[lege]

They're:

Mars 1U Server Celeron-M pfSense Appliance
CPU Speed: 1Ghz Celeron-M
Ethernet: 4xIntel Gigabit LAN
Memory: 1GB DDR RAM
Storage: 4GB CF
Mini-PCI: None

The packages are the default, nothing extra has been installed - it's showing AutoConfigBackup and siproxd. It should be vanilla 1.2.3 as far as I can tell.

1.2.3-RELEASE
built on Mon Jan 4 10:56:15 PST 2010