Failover for 2 ISP
-
Ideally you'd have both WANs setup on both pfSense boxes, and run CARP on the LAN, WAN, and OPT WAN. Then you will have real redundancy.
Sorry but i didn't get it.. The WAN is already configured on that two pfsense boxes running CARP on LAN. If i will be using CARP on the WAN what ip address should i use since it is a public ip? Thanks
-
Both of your pfSense boxes should have a connection to each WAN. If you don't run CARP on WAN now, you don't need to add that in.
You'd need the master box to have WAN and WAN2, and the backup box to have WAN and WAN2. Depending on your ISP, this may or may not be feasible.
And both boxes will need to have multi-wan setup properly (failover or load balancing pools, policy routing, etc), which is covered on the doc wiki.
-
An important thing to remember is this, somewhat oversimplified:
CARP on pfSense isn't used to handle failing internet connections, it's used to handle failing pfSense boxes - if the hardware goes down, you use CARP to fail from one pfSense box to another. Ā Among other things, CARP allows the two boxes to share one "virtual" IP address - and your other computers use that IP address to get online. Ā If the primary box fails, the secondary one will take that IP address, and you'll be going out to the internet via the secondary.
If you need mutli-WAN support (multiple internet connection failover) only - and not redundancy between multiple pfSense boxes, you can skip the multiple pfSense boxes and CARP entirely, and use both WAN connections on one pfSense box.
Do both of your ISPs - your WAN links - provide you with more than one WAN IP address? Ā This is a requirement for a dual-pfsense box solution to work. Ā If you have two or three IP addresses at least from each provider, everything can be made to work 'as expected.'
If you only have one IP address from each provider, you can't really use multiple pfSense boxes without a lot of hackery.
If you have one IP address from one provider, and multiple IP addresses from the other, you can setup your "primary" pfSense box with both WAN links, and your secondary box with only the one you have multiple IP addresses from.
If there's not already a good diagram describing multi-WAN, I may assemble one.
-
Both of your pfSense boxes should have a connection to each WAN. If you don't run CARP on WAN now, you don't need to add that in.
You'd need the master box to have WAN and WAN2, and the backup box to have WAN and WAN2. Depending on your ISP, this may or may not be feasible.
And both boxes will need to have multi-wan setup properly (failover or load balancing pools, policy routing, etc), which is covered on the doc wiki.
Thank you jimp. Looks like having two WAN on each pfsense box is not feasible yet. But i am planning to build one more pfsense box that will act as a loadbalance. Like for example. The current 2 pfsense boxes will just be a normal router. And the 3rd one will act as the loadbalancer or failover of the two. Do you think it is feasible?
-
An important thing to remember is this, somewhat oversimplified:
CARP on pfSense isn't used to handle failing internet connections, it's used to handle failing pfSense boxes - if the hardware goes down, you use CARP to fail from one pfSense box to another. Ā Among other things, CARP allows the two boxes to share one "virtual" IP address - and your other computers use that IP address to get online. Ā If the primary box fails, the secondary one will take that IP address, and you'll be going out to the internet via the secondary.
If you need mutli-WAN support (multiple internet connection failover) only - and not redundancy between multiple pfSense boxes, you can skip the multiple pfSense boxes and CARP entirely, and use both WAN connections on one pfSense box.
Do both of your ISPs - your WAN links - provide you with more than one WAN IP address? Ā This is a requirement for a dual-pfsense box solution to work. Ā If you have two or three IP addresses at least from each provider, everything can be made to work 'as expected.'
If you only have one IP address from each provider, you can't really use multiple pfSense boxes without a lot of hackery.
If you have one IP address from one provider, and multiple IP addresses from the other, you can setup your "primary" pfSense box with both WAN links, and your secondary box with only the one you have multiple IP addresses from.
If there's not already a good diagram describing multi-WAN, I may assemble one.
Thanks for clarifying overand. However i only have 1 ip addresses on each pfsense box. If you can give create that diagram that would be awesome. But it is not required though.. :)
-
Both of your pfSense boxes should have a connection to each WAN. If you don't run CARP on WAN now, you don't need to add that in.
You'd need the master box to have WAN and WAN2, and the backup box to have WAN and WAN2. Depending on your ISP, this may or may not be feasible.
And both boxes will need to have multi-wan setup properly (failover or load balancing pools, policy routing, etc), which is covered on the doc wiki.
Thank you jimp. Looks like having two WAN on each pfsense box is not feasible yet. But i am planning to build one more pfsense box that will act as a loadbalance. Like for example. The current 2 pfsense boxes will just be a normal router. And the 3rd one will act as the loadbalancer or failover of the two. Do you think it is feasible?
No, because you're back to having a single point of failure. You may as well just have one router connected to both WANs now.
-
Yeah i understand that the single point of failure is the load balancer right? If you were I.. You have limited resources. Would you think having carp on those 2 pfsense boxes make it much more reliable than using a single pfsense box with 2 WANs in it?
-
That depends on what you are worried about most, and the needs of your business. Only you will know that for sure :)
If you have more WAN failures than hardware failures, having a single dual-WAN box might better. You can keep the spare box installed and ready, boot it every now and then and update its config by hand, but not active 24/7. This is usually referred to as a "cold" spare.
That requires manual intervention in the case of a hardware failure, but depending on your business requirements you'd be better off since you could use both WANs all the time. If you don't have to worry about crazy high uptime and someone is usually on-site to handle the switch if a failure happens (or it only matters during business hours) then having a cold spare box is probably fine.
If you really need to use a two-box failover scenario with multi-wan, you'll need to talk to your ISP and work out the necessary details about getting more IPs or allowed connections. If your business needs call for high availability and automated failover, the cost is probably worth avoiding the downtime.
-
I agree that using a load balancer in front of two CARPed pfSense boxes on seperate WAN links doesn't make much sense.Ā As JimP said, if you have a box (the load balancer) that can handle both WAN links itself, and you're using it as a single point of failure, I'd strongly suggest moving to just-one-box.
If you have fairly "high end" switching equipment, you can use VLANs - this is more complex for most people than they'd like - and it requires having a managed switch with VLAN support, but you'll be able to use one NIC.
-
Thanks Guys!
I have tried connecting the 2 ISP on a single pfsense box. And put it via LB pool
Description: ISP Balance
Type: Gateway
Behavior: Load Balance
Monitor IP: 202.164.x.x
Monitor IP: 124.68.x.xDescription: ISP1 Failover
Type: Gateway
Behavior: Fail Over
Monitor IP: 202.164.x.x
Monitor IP: 124.68.x.xDescription: ISP2 Failover
Type: Gateway
Behavior: Fail Over
Monitor IP: 124.68.x.x
Monitor IP: 202.164.x.xI ran ping on the two public ip.. I can ping both of them i tried shutting off the modem of the first ISP and trying ping obviously one of them cannot be pinged. Now i ran the same thing on the second ISP its the same. My problem now is if the first ISP is up i can browse any site that i want but when it is down i cannot browse any site even though i get ping response from the second ISP. Correct me if i am wrong is this a DNS problem? If so how can i resolve this?
Another question if i can give my hands on the resources my current setup with CARP is fine as long as i have a 3 IP on each ISP for the carp+pfsync+load balance/fail over? Can you somehow help me with the diagram? Thanks a Lot.
-
Under firewall rules for LAN, did you change the gateway from "default gateway" to the Load Balancer you created?
-
That i forgot. :) By the way is it okay to load balance if one of the ISP is under PPPoE?
-
Yes, you can load balance between any number of multiple WANs.
-
I don't really see how PPPoE would be a problem - I'm pretty sure I've used PPPoE WAN links with the load balancer before.
Only read the rest if you need to - it's not needed, and can make things more complex:
Just remember - load balancing is 'fair' or 'even' by default - so even if you have one ISP with 6 megabits and one with 1, by default the load-balancing will (roughly) distribute the connections (and by extension the traffic) equally.
If you want to make one WAN link more likely to be used, you can put its entry into the load balancer config more than once - or have a 'ratio' of one to the other (example:Ā two entries for WAN, three for OPT1 to give a 60/40 balance)
By the way is it okay to load balance if one of the ISP is under PPPoE?
-
Thank you for all the reply I really appreciate the help. My last question would be incase i get my hands on resources like additional ip address for my WAN(s) like 3 each. Do i need more LAN cards? currently have 3 on each. Would my current setup will be suffice in order to make it high availability internet and firewall failover? If so can you guys help me with the diagram if its not too much to ask? Thanks thanks!
