Load Balancing Internal WebServers for Testing (QA) Purposes - Help

Shel · Jun 24, 2010, 8:19 PM

This type of stuff is not my expertise at all but I have to do it to accomplish my testing goals. So please be forgiving as to my ignorance. I have an internal product that has to be tested in a simulation environment that requires me to put this product on 2 web servers and load balanced. The production environment(externally controlled and maintained) uses pfsense so after finding the pfsense virtual appliance, I though it would be a good idea to use it in my simulation.

I sparked up 2 web servers in VMs on VMWare vSphere. Installed the pfSense appliance on vSphere as well.

In pfSense LAN is set to 192.168.1.75, WAN is set to 192.168.1.162(DHCP)

My 2 Web Servers are 192.168.1.72 and 192.168.1.74

I set up a pool with these to Web Servers in the list with the following settings.
Type: Server
Behavior: Load Balancing
Port: 80
Monitor: TCP

I set up a Virtual Server with the following settings.
IP Address: 192.168.1.162 (it says that this should be the WAN address that is why I set it to 162)
Port: 80
Virtual Server Pool: the one I setup above.

I opened the firewall for this as well.

So all of this is on the same network, but none of my VMs are on the Domain.

I can access this
http://192.168.1.72/Resources/Images/edit.png
and
http://192.168.1.72/Resources/Images/edit.png
But when I use this IP for my virtual server
http://192.168.1.162/Resources/Images/edit.png
I always get a timeout error.

Is there anyone here who can help me? Tell me what I need to know. Is there any other settings I can show you?

Thanks so much for your time.
Shel

wallabybob · Jun 24, 2010, 9:49 PM

A first sight it looks as if you have violated IP configuration rules in two ways:

The pfSense WAN and LAN interfaces may be on the same subnet (192.168,1.162 and 192.168.1.75) but its not certain since you haven't specified the network masks.
The pfSense WAN and virtual appliance have the same IP address (192.168.1.162). Where should a packet destined to 192.168.1.162 be delivered? Pick one OR the other but not both!

It would be helpful (to me) to have a diagram of your intended topology of this configuration, that is something showing how you want them interconnected. Maybe I haven't understood what you are trying to do because I'm unfamiliar with VMWare facilities and terminology.

j2b · Jun 25, 2010, 3:42 AM

Please check your Firewall rules to allow connection on WAN:80, as pfSense by default blocks packets. And take in mind, that if you work on the same subnet, connections directly to your webservers will bypass pfSense. On which LAN is your testing machine?

Shel · Jun 25, 2010, 1:40 PM

Wallabybob,
1/ Yes, both the LAN and WAN interfaces are on the same Subnet mask; 255.255.255.0. This is a problem? What should I be doing here?

2/ Yes, I thought that strange, but the line under the IP Address setting for the Load Balancer: Virtual Server: said "This is normally the WAN IP address that you would like the server to listen on. All connections to this IP and port will be forwarded to the pool cluster." So I set it to the WAN IP address. That is wrong obviously; since if I create another Virtual Server I can't set that one set to 162 as well. So what should I be setting this to?

j2b,
I think I have setup the Firewall rule properly. Here are the settings.
Action: Pass
Interface: WAN
Protocol: TCP
Source: any
Destination: any
Dest port range: from any, to: any
Gateway: default

What LAN is my testing machine on? Everything is on the same LAN, if I understand your question correctly. Testing machines(clients to access the servers) are on the same LAN and the Servers under the Load Balancer, pfSense is also on the same LAN. Is this not appropriate?

Thanks,
Shel

j2b · Jun 25, 2010, 2:31 PM

Comments concerning your answers:

I don't think, that this should be a problem, as your VPS is acting as load balancer, not router. But in any case there could be additional complexity, if you do not disable packet filtering. I didn't test it, but if you load balance traffic between different NICs, than there could be an issues with firewall and routing. In our case we do use load balancing for web cluster, only the difference is that all pool servers' and Virtual Server IPs ar on one NIC - LAN. The problem can arrise, if you use different subnets, and in such case you have to provide additional logics to router, to deal with packets and their destination. I do not think, that this could be considered as IP rule violation. But I am not an expert in these rules.
I do not think, that PS by adding WAN address to VirtualServer is creating additional NIC. Here you state IP which to use for load balancing pool, and it should be present on router. In our case we do use VIP (Virtual IP), as we deploy failover configuration of doubled PS boxes. So to use IP address in Load balancing Virtual Server IP field, we provide additional Virtual IP, but this is not your case.

Sorry, missed your line stating, that FW rules were set. As for sake of partial solution (proof of concept), can you try to use PFSense only with one NIC - LAN, and set Load balancing Virtual Server on LAN IP - x.x.x.75. Due to such changes, there will be no packet filtering or routing involved and in such case we could prove that system does not have errors in configuration outside pfsense.

What does your PFSense Status > Load balancer > Virtual Servers say? If such configuration in pfsense is OK, than it should see load balanced web servers and show their status - Online.

If you still use 2 NICs, which I can not proof, as I didn't tested it, then I think, that some routing/filtering could be involved, and it make things different. Can you ping your web servers from PFSense? (Diagnostics > Ping). Before you change any configuration, can you prove, that webservers could be pinged from PFSense VPS WAN and LAN interfaces?

Did you link all (WAN/LAN) interfaces to the same Virtual Switch? Aren't there any VLan settings, which could differ?

Shel · Jun 25, 2010, 8:18 PM

Not sure I understand all you are saying j2b. So I will give you the information I do understand.

The Status LoadBalancer: Virtual Server shows that the 2 servers in the pool are online

All WAN/LAN interface to the same vSwitch.

I did find this little piece of information about Redirection and Reflection. http://www.openbsd.org/faq/pf/rdr.html#reflect
This could be related to why I cannot get this to work, though, I'm not sure how to do the remedies to states there.

Thanks,
Shel

j2b · Jun 26, 2010, 4:47 AM

It would be great, if you could give more information on what exactly you do not understand from my writing. As it would be helpfull to give a hand for you. I can not assess your level of competence, so it is hard to choose language to help you.

Did you manage to test the same, only on pfSense Virtual Server put address 192.168.1.75, instead of 192.168.1.162?