[Inbound load balancing / server load balancing / relayd] configuration



  • 2.0-RC1 (i386) built on Tue Apr 26 16:24:39 EDT 2011

    I am trying to properly configure relayd to load balance across a number of backend servers.  Eventually, the servers will do both 80 and 443 (SSL), but I'm trying to just get non-encrypted traffic working for now.

    It appears that the /var/etc/relayd.conf file that is auto-generated by pfsense uses the redirect directive.  Using redirect in a load balancing configuration would appear to set us up for a "direct server return" type of route, where the firewall is not doing NAT on the inbound traffic, but simply forwarding packets to the backend servers such that responses do not need to pass back through the load balancer (just the firewall).  If I'm understanding the nuances correctly, this isn't necessarily what I want (and appears to be discouraged by some: http://undeadly.org/cgi?action=article&sid=20080617010016).  It appears that the relay directive would be a better fit for my use case, as it appears that it actually does NAT on the incoming packets such that symmetric routing is enforced.  However, it appears that any edits that I make to  /var/etc/relayd.conf are overwritten any time the GUI is used to change a load balancer setting.

    Questions:

    1. Is my understanding of the difference between redirect and relay options for relayd correct?  Is it intentional that pfsense only seems to support redirect?
    2. Is there a template relayd.conf file somewhere that I can use for persistent changes to the relayd configuration that survive across GUI-based configuration changes?

    Cheers!

    Dean



  • Redirect is working fine for us in the same use case you indicate - http and https balancing.

    I believe that how relayd is implemented on PF that the redirect works as this occurs on the LAN side, so from the client perspective it's still not DSR as all the traffic comes back via PF.  So yes, it's DSR between PF and the back end pool, but not between the PF public and client.

    I could be wrong on this….but it is working great for us.

    The other option would be to get the HAProxy plugin working well again as it gives much more config options.

    John



  • Is it working or not for you?

    DSR is a very specific type of configuration not usually used.



  • John,

    Thanks for your reply.  There doesn't seem to be much information available on this subject, so I appreciate you sharing your success.  I see what you mean about DSR behind the fw only.

    In doing packet sniffing with our relayd configuration, we're seeing the client IP show up at the backend server.  Because the backend servers are dual homed (one DMZ, one LAN), we're having routing issues in that the default gateway is on the LAN side.  The public IP comes through relayd to the backend server via the DMZ and the server wants to reply to it through the LAN port.

    Can you share any details on the routing configuration of your backend servers?

    Ermal,

    It is not working for me due to the routing issue described above.

    Cheers,

    Dean



  • Than you need DSR yourself :)

    Can you please describe your setup better is not clear for me!?



  • Our http and https back ends live in the DMZ only.  PF has a public, DMZ, and LAN interface.  PF is setup to allow only http and https from public->DMZ IP of back ends.

    The back ends use the PF sense DMZ IF IP as the gateway (and also DNS and NTP as we run those services on PF).

    We run 2 PF boxes and the public, DMZ and LAN IP we use is a CARP so if a PF box goes down we fail over.

    For the http and https back ends to get to the LAN (DB and NFS) they traverse the PF boxes to do this.  We have rules set to allow this only from the DMZ IP's -> LAN IP's required.

    This has proven to be a simple and secure setup, that scales well for our use.  We do have a risk that if someone compromises the firewall to DMZ they could get LAN access from the back ends but from my past experience more complex network topologies are not always more secure due to complexity and human error.

    PFsense on mid range hardware has no issues handling the traffic.

    We also use countryblock to help cut down the intrusion attempts.

    John

    @hikeonpast:

    John,

    Thanks for your reply.  There doesn't seem to be much information available on this subject, so I appreciate you sharing your success.  I see what you mean about DSR behind the fw only.

    In doing packet sniffing with our relayd configuration, we're seeing the client IP show up at the backend server.  Because the backend servers are dual homed (one DMZ, one LAN), we're having routing issues in that the default gateway is on the LAN side.  The public IP comes through relayd to the backend server via the DMZ and the server wants to reply to it through the LAN port.

    Can you share any details on the routing configuration of your backend servers?

    Ermal,

    It is not working for me due to the routing issue described above.

    Cheers,

    Dean



  • John, thanks.

    Ermal, apologies.  See below.

    Client
        |
        |
    pfSense
      |      |
      |      |
    DMZ  |
      |    LAN
      |      |
    Webserver

    Remote management via VPN to the LAN.  Load balanced HTTP/HTTPS requests come into the DMZ.  I currently have the LAN as the default gateway, which causes issues for serving since the packets try to leave the webserver via the LAN.  Changing the default gateway to the DMZ fixes the web packet routing issue, but causes problems for the other services that we're running on the LAN.

    (There are other nodes on the LAN which are omitted for clarity)

    Thanks,

    Dean



  • Side question related to my original post:

    In looking at the auto-generated relayd.conf file, I note that for each redirect section there are two identical forward to entries, each with a load balancer entry.  From looking at my webserver logfiles, this appears to generate two check pings per cycle.  Not a huge deal, but just wondering if this is intentional.

    redirect "CCG_VIP" {
      listen on <publicip> port 443
      forward to <ccg_ssl> port 443 check https '/loadbalance.html' host <pubhost.net> code 200 timeout 1000
      forward to <ccg_ssl> port 443 check https '/loadbalance.html' host <pubhost.net> code 200 timeout 1000
    }</pubhost.net></ccg_ssl></pubhost.net></ccg_ssl></publicip>
    

    Cheers,

    Dean



  • FYI: I don't have this in our config…..

    I also just tried on a VM deployment of PF 2.0 RC current build and on a test lbha config don't see it either.

    @hikeonpast:

    Side question related to my original post:

    In looking at the auto-generated relayd.conf file, I note that for each redirect section there are two identical forward to entries, each with a load balancer entry.  From looking at my webserver logfiles, this appears to generate two check pings per cycle.  Not a huge deal, but just wondering if this is intentional.

    redirect "CCG_VIP" {
      listen on <publicip> port 443
      forward to <ccg_ssl> port 443 check https '/loadbalance.html' host <pubhost.net> code 200 timeout 1000
      forward to <ccg_ssl> port 443 check https '/loadbalance.html' host <pubhost.net> code 200 timeout 1000
    }</pubhost.net></ccg_ssl></pubhost.net></ccg_ssl></publicip>
    

    Cheers,

    Dean



  • I do not think the present load balancer can do DSR.
    So either you contact pfSense comercial support to implement DSR, since it is not planned for 2.0, or keep the default gateway at DMZ and provision for the other services.



  • Hi,

    Thanks for your response.  To be clear, I don't want DSR.

    I was initially asking why PFS uses the redirect directive, which seems a lot like DSR behind the firewall.  I was wondering if the relay directive might be a good choice to do more traditional load balancing (i.e. where the public IP source address is NATted to the IP of the load balancer as it is passed to the backend server…and if PFS supported it.

    Thanks!

    -Dean



  • Ah that can be provided from HAproxy.
    I do not think/know relayd can do such thing!


Locked