Proper setup for VIPs on additional routed subnet
I've been looking around a bit and I'm trying to make sure I get this right. I'm about to replace our aging pfSense 1.2 server with one running 2.0 with an additional failover server. I've done the CARP failover setup in our office before, so I'm familiar with the basics. The additional caveat is that we have some additional IP address blocks routed to (what will be) our VIP on the firewall.
When I originally set this up many year ago, I was still getting a handle on some of the intermediate/advanced ideas for networking (I'm more of a sysadmin), so I set it up as follows:
ISP Gateway (x.x.157.17/29) <-> our router WAN (x.x.157.18/29) <-> our router LAN (x.x.154.65/26) <-> pfSense 1.2.3 WAN (x.x.154.66/26) <-> pfSense LAN (10.205.154.66/21)
We have the following two IP blocks routed to us (ISP has the next hop set to x.x.157.18, currently our router WAN)
x.x.154.66/26 (some are used in router LAN/pfSense WAN above)
These IP blocks are set up as proxy ARP VIPs, which are then set up as as 1:1 NAT to ranges in the private IP subnet (10.205.152.0/21).
Now that I understand more about networking, I'm getting rid of the unnecessary router (it was just doing packet forwarding anyway). This will be the new setup:
ISP Gateway (x.x.157.17/29) <-> pfSense 2.0 WAN (CARP x.x.157.18/29) <-> pfSense LAN (CARP 10.205.154.66/21)
My main concern is the proper way to set up the VIPs for the additional subnets, since I will eventually be doing a primary/slave failover setup. From what I read in the forums, proxy ARP should NOT be used for these VIPs in a failover setup, because (I assume) the slave firewall will also be trying to proxy ARP the same IPs. I'm a little confused if this is still the case, considering that you can select the WAN CARP address as the interface when creating the proxy ARP VIP. Wouldn't that only make it proxy ARP the VIPs when the firewall has control of that CARP VIP?
Assuming that I still need to avoid proxy ARP, I assume that the "Other" or "IP alias" VIP types should work fine given that our ISP is routing the two blocks to our WAN CARP VIP, correct? If that is the case, is there an advantage/disadvantage between the two? I've noticed the docs say the primary difference between the two (when excluding the similarities between proxy ARP and Other) is that IP Alias will respond to ICMP requests. Does this mean the ICMP packet will never be passed along to the appropriate server behind the firewall, given that I have a 1:1 NAT configured for the IP?
Ultimately, I want the same setup I have now, but dropping the router and making sure that failover will work for all routed subnets when I get that set up.
Interestingly enough, I have another separate ISP feed configured essentially the same way, minus the router. It has a subnet routed to its WAN CARP VIP, that subnet is assigned as a proxy ARP VIP, and then it's a 1:1 NAT to the private side. When I removed the proxy ARP VIP, I was still able to get to the internal server via the public IP address. I still had the 1:1 NAT configured, but no VIPs. Is this just related to ARP caching on the ISPs router, or does it still work because they are routing the subnet to our WAN CARP VIP as the next hop? If the latter, do I actually NEED to assign the subnet as a VIP, or does the 1:1 NAT magically take care of it?
dotdash last edited by
There's a lot going on in your post, so I'm just going to throw out some thoughts based on my experience.
In a failover setup, all the VIPs need to be CARP if you want them to work on the backup firewall. CARP VIPs need to be on the actual WAN subnet, or pfSense will throw an error. One way around this is to add an alias IP on the WAN of each firewall for each additional subnet, then you could add the CARP VIPs.
So here's what I ended up doing. Since the setup was basically the same as another setup I had done that was working, I figured I'd just mimic that and hope for the best. I don't have the failover configured yet, but here's what I have.
ISP Gateway (x.x.157.17/29) <-> pfSense WAN (CARP x.x.157.18/29) <-> pfSense LAN (CARP 10.205.154.66/21)
x.x.154.66/26 -> 10.x.154.66/26
x.x.210.0/23 -> 10.x.210.0/23
NO VIPs (outside of the CARP WAN and CARP LAN)
I guess since my ISP routes the two subnets (above) to x.x.157.18 as the "next hop", pfSense automatically handles those requests since they match the external IP address of the 1:1 NAT entries. This works out nicely, since I can't see any reason why the subnets wouldn't fail over to the secondary server because neither server is "advertising" that they control the subnets; they just utilize the 1:1 NAT to map them after they receive the packet.
Before I finalized this setup, I took a look at the "Other" and "IP Alias" VIPs, and I noticed those can only be done on individual IP (same thing for CARP). This obviously wouldn't have worked for my setup, because I have well over 500 IP addresses.