My apologies. The switch for that interface needed to be reset to factory defaults. For whatever reason the two interfaces wound up on seperate VLANs, yet they could both reach the gateway (just not one another) with their frames. Bizarre. I cannot even begin to fathom that, but once set to defaults all was well.
Just for the record, my problem is solved. It was a ruling mistake on DMZ, ie. a directed all traffic destined to elswhere then LAN or DMZ to the load balancer (WAN1 + WAN2), but this way the traffic to 224.0.0.x went out to the net.
Thanks for all who tried to help me to solve this problem.
Just noticed: your firewall rules are set to destination any. You should only allow the destination IPs of the servers in the pool. Use an hosts(192.168.1.2, 192.168.2.2) alias and a ports(80, 443) alias to do that with just a single rule.
hm perhaps i need to do some re thinking. I thought i would be able to use something like this http://www.openbsd.org/faq/pf/pools.html for my load balancing solution. I wasn't sure that slbd was the only deamon under BSD which can handle load balancing because i could hardly find any information on it. But if you say so i am sure you are right so i will have to go back to LVS with keepalived under Linux which supports UDP load balancing. With the price of losing the flexibility of CARP which i was beginning to like. I will keep you posted on my project wish me luck. If you have any other ideas please tell me.
PF itself does no availability checking. That's what we use slbd for - it's responsibility is to insert rules into an anchor (slightly different than the pools, but same concept) based on what's actually up. Again, load balancing is easy - availability checking is considerably more difficult and not usually terribly conclusive.
Consider this. UDP is a stateless protocol, it's not required to reply to anything it doesn't understand (TCP at least sends resets!). The way port scanners detect an "open" UDP port is by the lack of an ICMP port unreachable reply. Guess what happens if the box is down? Oh yeah, ALL ports will refuse to return ICMP port unreachable. OK, so now we have to tie in some other means of checking - let's say ICMP. So, now we get if the box is pingable and I'm not getting an ICMP port unreach answer, the daemon on that port must be good right? Bzzt…what if it's just b0rked but still listening (never happen you say? heh, I've had djbdns ick zombify on me and refuse to die - still listening on port 53).
FWIW, even our commercial F5 BIGIP (LTMs now) at work don't load balance (and do availability checking of) UDP - it can't be done reliably. Specific protocols are doable, but UDP in general isn't (consider syslog...you can't send a valid syslog packet to a syslog daemon that will make it reply to you...how do you know it's not b0rked? you don't)
The way I'd design your setup is the following (and it's free advice, so take it for what it's worth)
Two firewalls in an active passive pair with two CARP virtual IPs.
Four PowerDNS servers with one CARP VIP each, active for one, passive for the other three at different skews - this will cover any box failure that might occur.
If daemon failure is a serious concern, then write a dig script on the firewall to dig all four CARP VIPs and check the result, if they're answering, update your DNS server table in PF with the addresses. Alternately, on the machines themselves, you can use ifstated to do essentially the same thing - check to see if it's resolving, if not, set the CARP address to backup and let the other machines duke it out for taking control.
In BETA-2 i used to be able to set a monitor ip under Load Balancer: Pool: Edit in BETA-4 this field is greyed out. If i click on gateway instead of server in the pull down menu type i can set a monitor ip. Is the correct? Have you changed this setting. I upgraded from Beta-2 to Beta-4 and restored my old settings everything else seems to work.
Monitor is only used for gateway type pools. Server pools use the server address and port you put in the pool. For gateways, you may not actually want to monitor the gateway itself, so we provide a monitor ip field so you can choose a different IP to ping. And yes, b2 -> b4 had numerous LB related changes.
I've been playing around with load balancig lately, and it's been going well. What I would like to know is if there is a way to use pfSense on a 3 Interface WRAP box so that I dont need an extra ethernet switch behind the pfSense BOX. I would like to connect the servers that I want to balance directly to the WRAP without using a switch. (I will never need more than 2 servers, if I do, I'll have to use the switch)
Ie. I have this as a current setup..
WAN – Interface0-> PFSense Wrap Box -- Interface0--> Ethernet Switch -> Server 1
--> Server 2
I would like to set up pfsense as strictly a load balancer, and get rid of the Extra Ethernet switch like this:
WAN --Interface0--> PFSense Wrap Box -- Interface1---> Server 1
-- Interface2---> Server 2
Is this possible? Would I just bridge Interface1 and Interface2? Any reason NOT to do this????
I'd put the servers in different subnets if you're going to cross them over to the firewall. The load balancer doesn't require that servers reside on the same subnet. What you want to do is perfectly doable (just don't put too much thought into it…it's really as simple as it sounds).
Just to let you know I found the problem and it wasn't pfsense. the test lan carp ip I chose was in conflict with my wap and I had forgotten I had assigned that ip to the wap. I haven't finished testing yet but pfsense is working beautifully!
Thanks for everyone's help and support, and I've learned a lot about pfsense, carp and pf.
That could be the answer to my problem i will give it a try soon. Thanks a lot for your help.
Make sure you don't have asymmetric routing. You'll need two carp addresses on the INSIDE also with each group of servers using it's respective carp IP as it's return gateway. While pfSense will sync it's state table, it's not instantaneous and I can guarantee issues with out of state packets.
Thanks for all your support guys. I'll stick to my proposed solution which i'm happy to use. I am very impressed by pfSense, and it's my absolute prefered firewall, and i have tested a few. Thank you very much.
Finally… after testing on three motherboard, I can do ping and port forwarding from external to internal machine.
The main problem is in the default gateway of the internal machine. I forgot to add additional gw in the server routing table. ;D ;D ;D
I will switch to pfSense immediately... thanks guys... ;) ;) ;)
We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.
Subscribe to our Newsletter
Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.