Suggestion on how to change WAN and it's default gateway
we have a new ISP and want to make it the preferred one and in short time close the connection with the old ISP; we have 2 pfsense vm with VIP and CARP configured, with one as master and the second as slave; I already configured another OPT card on both firewalls, configured their IP and VIP and CARP; on these boxes I have configured some ipsec vpn, some that has the default wan gateway has endpoint; I also have NAT rules and NAT outbound.
What is your suggestion to make the new wan the default route? Simply changing the default gateway? I have issues with old nat and rules configured with the old wan vip address? They continue to work, is it true?
Thanks for suggestion.
I have no experience with CARP so I cannot contribute to that part. For the rest: I think it depends on how you have set up everything. If you have "hard-coded" something regarding the WAN somewhere except on the WAN interface and its firewall rules, you will probably get problems, e.g. if you have tied something (like a service) specifically to the WAN interface or its IP address. You have copied the firewall and NAT rules to OPT, right? Also, if you use IPv6, you have to make sure that your clients use the correct prefix, because your new ISP will probably not route the prefix of the old one.
I don't know if this helps, but some time ago, I extended my pfSense to a Multi-WAN setup with failover from ISP1 to ISP2 as default and failover from ISP2 to ISP1 for some dedicated LANs (e.g. my "Guest" LAN) using gateway groups (the gateway group with failover from ISP1 to ISP2 as default gateway) which worked well for IPv4, I cannot remember big problems during the setup. (Unfortunately haven't, got it done for IPv6 because of e.g. https://redmine.pfsense.org/issues/4881 and https://redmine.pfsense.org/issues/6880 as I have dynamic IPs.)
thanks for you reply. I have no IPv6 implemented on my network. I've read about gateway group and thought to implement it, but I don't know if it is viable for my scenario, but I can try it.
I also thought to implement other 2 boxes to start clean, but this is my last resort :-)
I hope to not have hard coded somewhere, or, I think no. I have not copied until now the rules on the OPT interface because I thought to turn on the new gateway and then move step by step the rules (I also want to use this move to cancel some old rules :-) ); I also thought that chancing the default gateway could not cause issue, simply my wan traffic goes out from the new interfaces and the old rules works bacause the old connection is up and pfsense correctly route the trafffic: in my head this is a theory, but I don't know, maybe not, is correct.
Well, I don't know if I understand you correctly, but when you route all your traffic via OPT, but you don't have NAT rules for OPT, then it won't work. (I assume you have only one public IP address and need NAT to communicate.) However, if you use Automatic or Hybrid Outbound NAT rule generation, you should have the rules for "normal" traffic automatically.
The rest depends on how you have set up your rules. At least you have to make sure that responses for traffic going out via OPT can come back via OPT, because the responses won't come via your old WAN. If you haven't done something special, it should probably work.
I don't know your setup, but you could e.g. start with a less important LAN and for a first test just change in gateway there in the firewall rule that allows the outgoing traffic (e.g. the "Default allow LAN to any rule", if you haven't changed the default setup).
thanks for your reply and suggestion. Yes, I supposed to do an experiment with a less important Lan to try if it works; my setup is with a vip (because I have 2 firewalls) and NAT rules on WAN connection.
I supposed that, even without any rule on the opt, I can change the firewall default gateway and for example navigate to internet from a pc inside the network, because on the lan interface I have the default rule that you said and left the default gateway.
Sincerely my worry is for ipsec VPN where the endpoint is with the old ip and I don't know, even if I suppose that it is so, if the traffic is routed correctly.