@wizard:
hm perhaps i need to do some re thinking. I thought i would be able to use something like this http://www.openbsd.org/faq/pf/pools.html for my load balancing solution. I wasn't sure that slbd was the only deamon under BSD which can handle load balancing because i could hardly find any information on it. But if you say so i am sure you are right so i will have to go back to LVS with keepalived under Linux which supports UDP load balancing. With the price of losing the flexibility of CARP which i was beginning to like. I will keep you posted on my project wish me luck. If you have any other ideas please tell me.
PF itself does no availability checking. That's what we use slbd for - it's responsibility is to insert rules into an anchor (slightly different than the pools, but same concept) based on what's actually up. Again, load balancing is easy - availability checking is considerably more difficult and not usually terribly conclusive.
Consider this. UDP is a stateless protocol, it's not required to reply to anything it doesn't understand (TCP at least sends resets!). The way port scanners detect an "open" UDP port is by the lack of an ICMP port unreachable reply. Guess what happens if the box is down? Oh yeah, ALL ports will refuse to return ICMP port unreachable. OK, so now we have to tie in some other means of checking - let's say ICMP. So, now we get if the box is pingable and I'm not getting an ICMP port unreach answer, the daemon on that port must be good right? Bzzt…what if it's just b0rked but still listening (never happen you say? heh, I've had djbdns ick zombify on me and refuse to die - still listening on port 53).
FWIW, even our commercial F5 BIGIP (LTMs now) at work don't load balance (and do availability checking of) UDP - it can't be done reliably. Specific protocols are doable, but UDP in general isn't (consider syslog...you can't send a valid syslog packet to a syslog daemon that will make it reply to you...how do you know it's not b0rked? you don't)
The way I'd design your setup is the following (and it's free advice, so take it for what it's worth)
Two firewalls in an active passive pair with two CARP virtual IPs.
Four PowerDNS servers with one CARP VIP each, active for one, passive for the other three at different skews - this will cover any box failure that might occur.
If daemon failure is a serious concern, then write a dig script on the firewall to dig all four CARP VIPs and check the result, if they're answering, update your DNS server table in PF with the addresses. Alternately, on the machines themselves, you can use ifstated to do essentially the same thing - check to see if it's resolving, if not, set the CARP address to backup and let the other machines duke it out for taking control.
--Bill