PFSense - Kernel Panic on 2.0.3 - Redundant Firewalls

Dregnus

Good morning,

My apologies if this is the wrong forum. Summary: I tried to implement a redundant PFSense firewall on Dell PowerEdge R210II servers, and ended up with kernel panics.

I recently tried to implement a redundant PFSense Firewall system, following the procedures as outlined in the PFSense book (updated as necessary for 2.x) and used one of our regional offices Internet Upgrade as the time to do it. I managed to load 2.0.3, x64 version of PFSense on both of the servers. I didn't transfer any configs from our old firewall, I rebuilt the rules, aliases, and interfaces from scratch as a lot of IPs changed with the Internet Upgrade. I installed CARP and managed to successfully sync the firewalls. The final step was to re-create the IPSec Tunnels to a few of our other regional offices.

Unfortunately, on the final day, about 2 hours before I had to leave to catch my flight home, the two PFSense boxes began having cascading kernel panics. One device would lock up, switching all configuration to the secondary device as it rebooted. Then that device would lockup almost immediately.

I've been using PFSense firewalls both internally and for several of our customers for about 4 or 5 years now. I've never installed a redundant configuration and haven't experienced any kernel panics.

Hardware we're using is 2 Dell PowerEdge R210II Servers, the relevant specs are below:

8 GB RAM
Intel Core I3-2100 3.10 Ghz, Dual Core
Intel Gigabit ET Quad Port NIC (igb0 - ibg3)
2 Onboard NICs (bce0 and bce1)
500 GB Hard drive, NO RAID

I've attached the kernel panic log. Error is General Protection Fault (Segmentation Error) while in Kernel mode. Process is 12, which as an interrupt for bce1, which is the "LAN" address. But that's about all I can make out from it. I'd be happy to answer any specific questions about the configuration, however I'd like to avoid uploading the actual configuration for obvious reasons.

I'd appreciate any advice anyone can give on this. The two firewalls are 3000 miles away right now, so I can't grab anything on them, however I'm planning on migrating to a redundant firewall solution in our home office and about to start installation and configuration. It's the same hardware, so I'd imagine I should see this same issue. The firewalls ran for 3 days until employees began using the new Internet line, at which point kernel panics occurred almost immediately after the device rebooted.

Thanks,

EDIT: My sincere thanks in advance and thanks for making such an awesome firewall product.
pfsensekernelpanicPart1.txt
pfsensekernelpanicPar2.txt

jimp

http://doc.pfsense.org/index.php/Unexpected_Reboot_Troubleshooting
(see the second main point there)

Dregnus

Looks like I have the double-whammy with both igb interfaces (Intel expansion slot) as well as Broadcom on board. Thanks for the quick response, I'll deploy this on the two firewalls I'm building for the local office. I'll definitely let you know if this fixes the issue.

Thanks