Single WAN, Secondary CARP Internet access: How to automate this simple solution?

MrPete

[UPDATE: new info makes this potentially simpler.]

I just learned how to make my secondary CARP have 100% internet access (for package updates and everything else).

I wish this were not so obscure
How do I make this fully automated?
Could this become part of pfSense?

In my case, I'm using one ISP IP, with a pair of local IP's on the WAN interface (192.168.222.2-3)

Solution:
Configure an alternative gateway, and place as Tier 2 in a gateway group:

Simplest for me is referencing the primary LAN interface and its CARP address (192.168.1.1 on my LAN)
Ensure DNS Resolver is set to also allow accessing "upstream" DNS etc using the same interface. Unusual, but it works.
When necessary, I can manually use "route add default 192.168.1.1" whenever pfSense is CARP backup.

Two problems with this:

For some reason, this is broken by XMLRPC updates (I'll write a separate post on this.)
This doesn't work during install... at least not automatically.

a) Add a firewall rule to that interface, allowing the secondary FW to have full Internet access (ie: ALLOW, source: 192.168.222.3, dest: any)
b) The hard part I just now found, which does not survive a reboot:

On secondary, type this into a shell:
route add default 192.168.222.2
(ie the primary WAN CARP IP)

Cool_Corona

Its not real CARP. Its just access to the internet from the 2nd node...

netblues

@mrpete said in Single WAN, Secondary CARP Internet access: How to automate this simple solution?:

b) The hard part I just now found, which does not survive a reboot:

System, routing, static routes, add
:)

MrPete

@cool_corona what is "real carp"? I have real HA running, including xmlrpc settings sync, pfsync for states, and CARP failover.

MrPete

@netblues is this safe to add as a normal permanent static route? If the secondary CARP goes Primary, not sure what happens to this

netblues

@mrpete You said, it doesn't survive reboots.

You need to test all this in any case.
How about routing to the carp vip?

(haven't tested this, though)

Cool_Corona

@mrpete Its a reak HA cluster with failover that survices a reboot.

Running as master/slave.

MrPete

@netblues have to route to something reachable by the backup CARP.
Pretty sure none of the VIP are available sine they are also defined for the backup CARP system (but disabled due to it being secondary.)

I suspect I can use any primary IP. Will do some experiments.

I am looking at this in terms of upgrades:

during a full reinstall of secondary, CARP is not configured.
so the question is, what could either be auto configured, or what could the user easily do during the initial setup, to enable internet access.
- There is a point during pfsense install where the user is given an opportunity to get a shell in single user mode BSD.
- now that I know these extra tidbits, I will try an experiment to reinstall my secondary, and see if I can enable a gateway thru primary from that point.

If this can be done, secondary CARP reinstall would be much smoother: all packages auto reinstalled, etc.

MrPete

@netblues I am thinking instead of static route, shouldn't this be doable as a defined gateway, and placed in a gateway group, so it only goes active when secondary?

I thought I had tried that already.

Time for more experiments. At least I know what I am shooting for now

MrPete

@mrpete @netblues @Cool_Corona

I've updated the OP with results of my first set of experiments.

When I have a chance, I'll redo a full install on secondary CARP and see how that goes.