Two PS-Sense server in a virtual enviroment for redunancy

fastcon68 · Dec 28, 2008, 9:09 PM

I was wondering if it was possible to run two PS-Sense server in a virtual enviroment for redunancy?

(OpenVPN - 192.168.32.X)
(WIFI optional Interface - 192.168.31.1)\

/- FW1 (192.168.30.1)–-------------------
/ (SNORT,IPSEC, OpenVPN, NAT services) -----
Internal Network (192.168.30.x) -----< > ---- (Single External IP)
\ (SNORT,IPSEC, OpenVPN, NAT services)------/
- FW2 (192.168.30.2----------------------/

(OpenVPN - 192.168.32.X) /
(WIFI optional Interface - 192.168.31.1)/

I am in the process of installing and configuring OpenVPN, and have other services. I just want to add reducancy to my firewall so that I can upgrade one and or fail it over. I just need some assistance with this.
RC

fastcon68 · Jan 4, 2009, 5:15 AM

I used XEN 5.0 and shut down my production firewall and built a second firewall from the same image. I changed the LAN IP and setup CARP on both firewalls. All rules and other services remain the same.

I added a internal network to XEN called CARP. Only the PF-Sense servers have access to that virtual network. OPT2 on both servers been added.

Utahraptor (production) - OPT2 192.168.17.1
(Virtual IP CARP) - 192.168.17.3

Utahraptor2 (Backup server) - OPT2 192.168.17.2
(Virtual IP CARP) - 192.168.17.4

I am getting the following error messages:
Jan 3 23:38:05 php: : New alert found: An error code was received while attempting XMLRPC sync with username SuperMan https://192.168.17.1:443 - Code 2: Invalid return payload: enable debugging to examine incoming payload

Jan 3 23:38:05 php: : An error code was received while attempting XMLRPC sync with username SuperMan https://192.168.17.1:443 - Code 2: Invalid return payload: enable debugging to examine incoming payload

Jan 3 23:38:05 php: : Beginning XMLRPC sync to https://192.168.17.1:443.

Can anyone give me a hand on getting it fixed?
RC

fastcon68 · Jan 4, 2009, 9:21 PM

I am getting errors with both servers trying to be master's. Can anyone assist me with configuration issue?

(OpenVPN - 192.168.32.X)
(WIFI optional Interface - 192.168.31.1)\

/- FW1 (192.168.30.1)–--------------------
/ (SNORT,IPSEC, OpenVPN, NAT services) --
/--OPT2 - 192.168.17.1 \
/---CARP VIP 192.168.17.3 \
Internal Network (192.168.30.x) -----< | > ---- (Single External IP)
---CARP VIP 192.168.17.4 /
--OPT2 - 192.168.17.2 /
\ (SNORT,IPSEC, OpenVPN, NAT services)----/
- FW2 (192.168.30.2------------------------/

(OpenVPN - 192.168.32.X) /
(WIFI optional Interface - 192.168.31.1)/

dotdash · Jan 5, 2009, 6:37 PM

I'm confused on so many levels I had to post a reply.
I'll pretend I missed the part about running the carp nodes on VMs. I can't fathom that one, and I have no experience with running pfSense in a VM.
But I have run several CARP clusters, and here are some puzzling things:
You don't need a CARP VIP on the SYNC interface.
You need a CARP VIP on the LAN and the WAN side.
You might want to review the tutorial here: http://www.pfsense.org/mirror.php?section=tutorials/carp/carp-cluster-new.htm
There have been some problems syncing when using non-default usernames. I haven't kept up on that, but try changing it back to admin and see if that helps.

fastcon68 · Jan 6, 2009, 1:23 AM

@dotdash:

I'm confused on so many levels I had to post a reply.

I have gotten the first part of the configuration running. I now have a master and and a slave now.

I'll pretend I missed the part about running the carp nodes on VMs. I can't fathom that one, and I have no experience with running pfSense in a VM.

The reason for the CARP cluster is that XEN has a issue from time to time with PF-Sense server. It will crash or fail to reboot correctly. I need it for additional redundancy.

But I have run several CARP clusters, and here are some puzzling things:
You don't need a CARP VIP on the SYNC interface.

I added a internal network under XEN. This is OPT interface 2, it is only accessible by two PF-Sense servers. It is on the 192.168.17.x subnet. The FW1 OPT2 interface is 192.168.17.1 and has the VIP of 192.168.17.2 and FW2 OPT2 interface is 192.168.17.3.

The FW1 with is internal interface of 192.168.30.1 and FW2 has a internal Interface of 192.168.30.2. I have added a SYNC rule on the OPT2 interface and put in the Sync to IP 192.168.30.2 and it appears to be working.

You need a CARP VIP on the LAN and the WAN side.

The WAN side the IP address is staying the same since I only have one external IP.

You might want to review the tutorial here: http://www.pfsense.org/mirror.php?section=tutorials/carp/carp-cluster-new.htm

I been reviewing the tutorial and it was written for a earlier version of PF_Sense

There have been some problems syncing when using non-default usernames. I haven't kept up on that, but try changing it back to admin and see if that helps.

I did change the user name back to the default and it works.

fastcon68 · Jan 6, 2009, 2:01 AM

I posted some responses from dotdash, I still have few things not quite configured correctly.

I created by second PF-Sense machine by copying my first box. The only difference is the IP address and the name of the server.

I have following Settings:
Synchronize Enabled
Synchronize Interface - OPT2
pfSync sync peer IP 192.168.17.2
Synchronize rules
Synchronize NAT
Synchronize IPsec
Synchronize Virtual IPs
Synchronize traffic shaper
Synchronize to IP 192.168.30.2
Remote System Password (username reset to ADMIN and password set to match on both servers

Added Virtual IP to the Master machine
Type = CARP
Address 192.168.17.2 /24
matched the VIP password
VHID group 1
Advertising Frequency 0

Rules
OPT2
All traffic set to pass between servers

When I bring up the second server CARP comes up with FW1 as master and FW2 as backup. However I see two issues at that point I see even with 192.168.14.2 added as a second gateway, I can't access the internet and IPSEC tunnels appear to be up on both firewalls.

I really want to get this running due to my occasional virtual server issue.
Many thanks,
RC