Carp & nat/firewall rules

dbuckle

Hi,

I've built a 2 node WRAP cluster - works fine.

I can't get the services (HTTP, SMTP etc) exposed to the internet via the VIP

Setup:

WAN VIP: 11.12.13.90/24
LAN VIP: 192.168.1.70/24

FW1:
WAN: 11.12.13.80
LAN: 192.168.1.50

FW2:
WAN: 11.12.13.81
LAN: 192.168.1.51

WEB SERVER:
LAN: 192.168.1.12

LAN Firewall rule: I haven't touched this.
Default LAN -> any

I've added a WAN firewall rule to allow all to destination 192.168.1.12 port 80

Advanced Outbound NAT Rule:

Interface: WAN
Source: 192.168.1.0/24
Destination: *
Destination Port: *
NAT Address: 11.12.13.90
NAT Port: *

Port Forward Rule:

Interface: WAN
Proto: TCP
Ext. Port Range: 80
NAT IP: 192.168.1.12 (ext.: 11.12.13.90)
Int. Port Range: 80

CARP sync's ok. Ping to each WAN IP works but ping to WAN VIP loses some (not all) packets.

Do I need a LAN firewall rule?
Pointers to get this working would be much appreciated.

Many Thanks,

hoba

What version are you running?
What CARP Settings did you choose (preemption, loadbalancing…)?
What advertising frequency does your VIP's have at each box?
Did you create the VIPs at the mastersystem with syncing in place so they were generated automatically at the backup or did you create them manually at the backup?
What does status>carp tell you? Is one machine master, the other backup or is something mixed there?

I have setup pfSense with CARP at several productive environments with portforwardings as well as 1:1 NATs and no problem with these. I as well did some VoIP tests with CARP and even wraps. You'll only notice about 1 second silence and the call continues without being dropped.

dbuckle

Running 1.0 BETA 1

I followed the tutorial "building a fully redundant Cluster with 2 pfSense-systems".

So:

Fiirewall 1:

Sync Enabled: yes
Sync Interface: OPT1
Load Balancing: no
Preemption: yes
Sync Rules: yes
Sync Aliases: no
Synce NAT: yes
Sync IPSEC: no
Sync WOL : no
Sync Static Routes : no
Sync Load Balancer : no
Sync Virtual IP's : YES
Sync Traffic shaper : no
Sync to IP: 192.168.200.2

VIP Advertising Freq: 0

Firewall 2:

Sync Enabled: yes
Sync Interface: OPT1
Load Balancing: no
Preemption: YES
Sync Rules: NO
Sync Aliases: no
Synce NAT: yes
Sync IPSEC: no
Sync WOL : no
Sync Static Routes : no
Sync Load Balancer : no
Sync Virtual IP's : NO
Sync Traffic shaper : no
Sync to IP:

VIP Advertising Freq: 100

The sync seems to be working afaik. Rules I create are copied to the backup.
I created the VIPs on the MASTER and they were created on both nodes.
Firewall1 is MASTER for both WAN and LAN VIP's so it looks ok.

Firewall logs and I can see RULE PASS on port 80 to the 192.168.1.12 server but nothing happens after that (no web site appears). Does this mean I have a problem with NAT (outbound?) perhaps?

hoba

Is it possible that the server behind doesn't use the CARP LAN IP as gateway but a real IP of one of the systems?

Also is you WAN IP in a /24 subnet or something smaller? The IP sounds like a testsetup. The VIP and subnet has to be in the same subnet like the real IP of the Interface.

From what you wrote the pfSense config looks ok. Check your Client and server settings, maybe something is using a wrong gateway.

You also might want to rebuild that config with tha latest testing-snapshot: http://pfsense.com/~sullrich/1.0-BETA1-TESTING-SNAPSHOT-1-25-06/pfSense.img
It has several improvements and bugfixes. As you are running an embedded system that means you need to reflash the cf-media. I also would suggest creating the config afterwards by hand and not reimport the old one, just to make sure.

dbuckle

Great! It works!

I hadn't set the gateway IP address of the web server to the LAN VIP - just as you suggested.

Thanks very much for your help with this, hoba

;D

hoba

Great :D. Btw, if you use the pfSense as DHCP as well, there is a field where you can enter the LAN VIP to be handed out as gateway to the clients at the dhcp-server settings page.

dbuckle

I'm not using DHCP in this situation but thanks for the tip.

One thing that threw me too was that I tried using 192.168.0.70 (note the zero) as my LAN gateway VIP as I originally had my LAN on the 192.168.0.x network. Doing this caused lots of BAD GATEWAY error messages. Changing to 192.168.1.70 as the LAN GATEWAY VIP solved this.

hoba

The Bad Gateway messages are cosmetic and they should be there no matter which IP you are using. They appear always when a CARP IP is brought up.