PfSense Failover, using VLANs on LAN interface, problems

nicolas010

Hi all, I am using pfSense 2.0.2 about a month ago and its incredible, but I have this problem. I need to make my 2 pfSense to have CARP failover.
I have follow these two tutorials…

http://pfsense.mirror.range-id.it/tutorials/carp/carp-cluster-new.swf
http://www.howtoforge.com/how-to-configure-a-pfsense-2.0-cluster-using-carp

but both of them are for firewalls pfSense, mine are a slightly different. My pfSense is for giving Internet through Access Point to students, so I have a Captive Portal with RADIUS and RADIUS MAC authentication, certificates, DHCP server for VLANs, an index.php and some other files to complete the captive portal. It works just fine with one pfSense online, but I need to apply CARP to this one. So as you see the tutorials arent the same, because through the LAN interface there are 4 other VLANs to make a difference between students users, teachers users, etc. So I follow up the tutorial the best I could and come to this...

pfSense MASTER pfSense BACKUP
WAN: 200.1.26.101/27 WAN: 200.1.26.102/27
LAN: 10.1.5.1/24 LAN: 10.1.5.2/24
VLANA: 10.130.0.1/22 VLANA: 10.140.0.1/22
VLANB: 10.131.0.1/22 VLANB: 10.141.0.1/22
VLANC: 10.132.0.1/22 VLANC: 10.142.0.1/22
VLAND: 10.133.0.1/22 VLAND: 10.143.0.1/22
SYNC: 1.1.1.1 SYNC: 1.1.1.2

VIPs are
WAN: 200.1.26.112/27
LAN: 10.1.5.10/24

Here is another question, Do I have to create a VIP for each VLAN??? and would it be? i tried but it says that it need a real interface I think.

After this I enable syncronization in the CARP settings as follows.

Synchronize States: CHECK
Synchronize: SYNC
pfsync Synchronize Peer IP: blank

Synchronize Config to IP: 1.1.1.2
Remote system username: admin
remote system password: password

And also I enabled: Synchronize rules, Synchronize NAT, Synchronize VIPs.

after this I check the STATUS -> CARP (failover) page and saw this...
in the MASTER pfSense both VIPs status were MASTER and in the BACKUP pfSense the WAN VIP1 was as backup and the LAN VIP3 was as MASTER as well, so according to the tutorials this should not be this way? I am missing something? I am sorry for the long post but I needed to specific, Thanks before hand.

podilarius

Each interface would have its own IP address like .2 and .3 and they would have a .1 carp VIP. I have not tried with vlans yet. I think I will try that in the lab, just to see. I see myself having this situation coming up soon.

nicolas010

Would you let me know please, I dont know if I have to create a VIP for each VLAN. Also can you tell why or is it ok to have in the backup the LAN VIP as master when it should be backup…

podilarius

the secondary should never be master unless there is a link problem on the primary. I got busy, but I still plan on setting this up.

nicolas010

So, how did it go?

podilarius

testing on this went very well. failing back seems problematic as in download was interrupted, but restarted with no problem. It might be fixed already in 2.0.3 .. will try that after a while.

Update:
Second test was fine … download was not interrupted. Could be the systems I am using though .. they are very old PIII machines.

Update again: 3rd test was fine ... so just a fluke on the first. Attribute that to slow systems I guess.

Final Update: Nope it was storage space on the workstation that was causing the interruption, not the connection.

nicolas010

Great, so the configuration works? could you please tell me how you do it so I can do it here please? ty :D

podilarius

WAN:
Master: x.y.z.254
Secondary: x.y.z.253
CARP VIP: x.y.z.2 (.1 is gateway)
vhid: 1
( I was testing with private IPs with that is behind another pfsense machine)

LAN:
Master: a.b.c.254
Secondary: a.b.c.253
CARP VIP: a.b.c.1
vhid: 2

Cluster (OPT1):
Master:b.c.d.1
Secondary:b.c.d.2
Standard Clustering setup within pfSense thus far. I am not using directed multicast, but direct ip pfsync, though it was just from earlier testing. (with and without).

VLAN13 (OPT2):
Master: c.d.e.254
Secondary: c.d.e.253
CARP VIP: c.d.e.1
vhid: 13

VLAN14 (Opt3):
Master: d.e.f.254
Secondary: d.e.f.253
CARP VIP: d.e.f.1
vhid: 14

All are /24 subnets.

Course I setup advanced outbound NAT rules and firewall rules for each of the subnets. I put a VM behind VLANs 13 and 14 with the CARP VIP as the gateway and started a download. I then rebooted the master FW and watched the secondary take over and then relinquish control back to the primary when it came up. Downloads were not interrupted nor was the continuous ping.

nicolas010

great I really need to do this, well I have some questions about your configuration,

first
Cluster (OPT1):
this would be the SYNC interface right?

second
VLAN13 (OPT2):
why OPT2? did you use a real interface? its a VLAN so you need to create a VLAN over an interface, on wich did you created it? my VLANS are on LAN interface, the 4 of them, how are your vlans configured?

third
same for VLAN14

fourth
how did you configure your outbound nat rules?

fifth
did you configure anything else? like DCHP server? any changes at all?

thank you again :D.

podilarius

first:
opt1 is the cluster interface tied between the 2 FWs via a crossover cable.

second:
huh? that is how you use VLANs. You create them and then assign them to an interface. In this case it was OPT2. You can rename the interface if you like, i could rename opt2 to vlan13 if I liked, I just didn't. It uses the same physical interface as LAN (LAN technically runs on VLAN1). So basically it maps to like:
VLANx -> fxpx -> OPTx. When you assign OPT2, the interface list will show VLAN13 that was just created on top of fxp1. Then you go to opt2 interface configuration to activate it. Then of course assign an interface address. THEN the CARP IP is created and used as the gateway address for that subnet.
Hope that clarifies it.

thrird:
see second.

fourth:
Easy, switch from auto to manual.then create 2 entries for each VLAN. One for port 500 with static port and the other for all other traffic. It is best to just copy all but the localhost subnets for each VLAN subnet. For my purposes, I used just the WAN address as the natted IP. 1:1 and port forward NAT should work with CARP addresses also.

No, I didn't configure DHCP. This would be at a DC for disaster recovery testing, so customers are expected to provide IPs or DHCP server. I suppose it should be possible. I did use DHCP on LAN during the whole test. This is because it is an internal vswitch and would not interfere with the production DHCP server. But that stayed up as well. Course I was not looking for problems on DHCP.

I was configuring a test for my setup

nicolas010

hi again, and thanks for your help, the thing is that I have 4 VLANs on the LAN interface, and each one of them uses DCHP, so I will try to run some test later afternoon and I will post the results,

podilarius

Have to manually set the gateway to CARP address but DHCP seems to work fine.