Hi,

Just for the record, my problem is solved. It was a ruling mistake on DMZ, ie. a directed all traffic destined to elswhere then LAN or DMZ to the load balancer (WAN1 + WAN2), but this way the traffic to 224.0.0.x went out to the net.
Thanks for all who tried to help me to solve this problem.

hm, this somehow starts to look like more general driver unrelated issues then. I'll forward this to thompsa.

arrggghhhhh !!!**"!@*ç%2@ >:( >:(

i'm an idiot… for testing purposes i had still running an Virtual machine with an 3rd PF running with carp enabled and the same IP address....

:-))  thanks anyway guy's

Just noticed: your firewall rules are set to destination any. You should only allow the destination IPs of the servers in the pool. Use an hosts(192.168.1.2, 192.168.2.2) alias and a ports(80, 443) alias to do that with just a single rule.

@wizard:

hm perhaps i need to do some re thinking. I thought i would be able to use something like this http://www.openbsd.org/faq/pf/pools.html for my load balancing solution. I wasn't sure that slbd was the only deamon under BSD which can handle load balancing because i could hardly find any information on it. But if you say so i am sure you are right so i will have to go back to LVS with keepalived under Linux which supports UDP load balancing. With the price of losing the flexibility of CARP which i was beginning to like. I will keep you posted on my project wish me luck. If you have any other ideas please tell me.

PF itself does no availability checking.  That's what we use slbd for - it's responsibility is to insert rules into an anchor (slightly different than the pools, but same concept) based on what's actually up.  Again, load balancing is easy - availability checking is considerably more difficult and not usually terribly conclusive.

Consider this.  UDP is a stateless protocol, it's not required to reply to anything it doesn't understand (TCP at least sends resets!).  The way port scanners detect an "open" UDP port is by the lack of an ICMP port unreachable reply.  Guess what happens if the box is down?  Oh yeah, ALL ports will refuse to return ICMP port unreachable.  OK, so now we have to tie in some other means of checking - let's say ICMP.  So, now we get if the box is pingable and I'm not getting an ICMP port unreach answer, the daemon on that port must be good right?  Bzzt…what if it's just b0rked but still listening (never happen you say?  heh, I've had djbdns ick zombify on me and refuse to die - still listening on port 53).

FWIW, even our commercial F5 BIGIP (LTMs now) at work don't load balance (and do availability checking of) UDP - it can't be done reliably.  Specific protocols are doable, but UDP in general isn't (consider syslog...you can't send a valid syslog packet to a syslog daemon that will make it reply to you...how do you know it's not b0rked?  you don't)

The way I'd design your setup is the following (and it's free advice, so take it for what it's worth)

Two firewalls in an active passive pair with two CARP virtual IPs.
Four PowerDNS servers with one CARP VIP each, active for one, passive for the other three at different skews - this will cover any box failure that might occur.
If daemon failure is a serious concern, then write a dig script on the firewall to dig all four CARP VIPs and check the result, if they're answering, update your DNS server table in PF with the addresses.  Alternately, on the machines themselves, you can use ifstated to do essentially the same thing - check to see if it's resolving, if not, set the CARP address to backup and let the other machines duke it out for taking control.

--Bill

@wizard:

In BETA-2  i used to be able to set a monitor ip under Load Balancer: Pool: Edit in BETA-4 this field is greyed out. If i click on gateway instead of server in the pull down menu type i can set a monitor ip. Is the correct? Have you changed this setting. I upgraded from Beta-2 to Beta-4 and restored my old settings everything else seems to work.

Monitor is only used for gateway type pools.  Server pools use the server address and port you put in the pool.  For gateways, you may not actually want to monitor the gateway itself, so we provide a monitor ip field so you can choose a different IP to ping.  And yes, b2 -> b4 had numerous LB related changes.

–Bill

@rneily:

I've been playing around with load balancig lately, and it's been going well.  What I would like to know is if there is a way to use pfSense on a 3 Interface WRAP box so that I dont need an extra ethernet switch behind the pfSense BOX.  I would like to connect the servers that I want to balance directly to the WRAP without using a switch.  (I will never need more than 2 servers, if I do, I'll have to use the switch)

Ie.  I have this as a current setup..

WAN – Interface0->  PFSense Wrap Box -- Interface0--> Ethernet Switch -> Server 1
                                                                                                        --> Server 2

I would like to set up pfsense as strictly a load balancer, and get rid of the Extra Ethernet switch like this:

WAN --Interface0-->  PFSense Wrap Box -- Interface1---> Server 1
                                                          -- Interface2--->  Server 2

Is this possible?  Would I just bridge Interface1 and Interface2?  Any reason NOT to do this????

I'd put the servers in different subnets if you're going to cross them over to the firewall.  The load balancer doesn't require that servers reside on the same subnet.  What you want to do is perfectly doable (just don't put too much thought into it…it's really as simple as it sounds).

--Bill

Just to let you know I found the problem and it wasn't pfsense. the test lan carp ip I chose was in conflict with my wap and I had forgotten I had assigned that ip to the wap. I haven't finished testing yet but pfsense is working beautifully!

Thanks for everyone's help and support, and I've learned a lot about pfsense, carp and pf.

;D

I still wonder what should cause the destruction of the config.xml. Keep an eye on it. There might be something else going on which is unrelated to the other error.

@wizard:

That could be the answer to my problem i will give it a try soon. Thanks a lot for your help.

Make sure you don't have asymmetric routing.  You'll need two carp addresses on the INSIDE also with each group of servers using it's respective carp IP as it's return gateway.  While pfSense will sync it's state table, it's not instantaneous and I can guarantee issues with out of state packets.

–Bill

Without having any more data the only thing that I can suggest is making sure you are on the latest testing snapshot:

http://www.pfsense.com/~sullrich/RELENG_1_SNAPSHOT_04-12-2006/

Thats most likely my fault as I added the Sync code. :)

Could we simply HUP slbd on filter reload?

Thanks for all your support guys. I'll stick to my proposed solution which i'm happy to use. I am very impressed by pfSense, and it's my absolute prefered firewall, and i have tested a few. Thank you very much.

//Eskild

@sullrich:

Yes, go ahead and email me the information.

You got mail… :)

/jan

Finally… after testing on three motherboard, I can do ping and port forwarding from external to internal machine.
The main problem is in the default gateway of the internal machine. I forgot to add additional gw in the server routing table.  ;D ;D ;D

I will switch to pfSense immediately... thanks guys... ;) ;) ;)

You can do what you want with it.  The point is to not ask for help when doing things beyond the scope of what we offer support for.

The Bad Gateway messages are cosmetic and they should be there no matter which IP you are using. They appear always when a CARP IP is brought up.

Wouldn't this be the same problem you run into when you try to impose CARP on a set of bridged interfaces?  Basically the deal is that CARP and proxyarp (or bridging) don't play nicely together.

@kikawala:

I did not try 0.94.10, but I did not have any problems with 0.94.12

Known problem, colin is working on configuration sync.