"Inbound" Load Balancing on LAN - works across VPN but not locally

Briantist

I want to load balancing purely on the LAN. That is, if the LAN subnet is 10.20.30.0/24, and I have two servers on 10.20.30.77 and 10.20.30.88, I want to be able to add them to a pool and create a "virtual server" on 10.20.30.99 that will load balance between them. I can do this, but connections from the LAN don't work. Odd thing is, I have a VPN tunnel to another location, and if I try to connect to 10.20.30.99 on the correct port from across the VPN, it works like it should.

I'd like to know why this is, and if there is any way to fix it. I hope this is the right place, as there isn't a load balancing section on the forums, and most of the load balancing talk in here is for outbound.

Thanks!

Briantist

It just occurred to me that for some reason I might need proxy ARP, but I tried that, tried adding the virtual IP as CARP and as other too but none of them make much practical difference; with any of the options I still can't connect locally, and still can from across the VPN.

I do see a difference in the packet captures though. The packet captures don't seem to capture the VPN traffic no matter what. I have it set on LAN. With no VIP, a connection attempt from the LAN only shows who-has ARP requests in the capture. With proxy ARP, it gives the MAC and then a connection seems to be made to the MAC address but ultimately it doesn't work. With CARP it gives a different MAC, and sends ICMP redirects, but it still doesn't work. "Other" for VIP type seems to do nothing.

GruensFroeschli

I never had such a problem (because i dont balance servers), but i can imagine that this is due to the same problems why NAT doesnt work out of the box on the same interface. (Request comes in on the LAN, response goes out the LAN)
–> you need NAT reflection.
However there is no NAT reflection for balancing-pools.
I dont think there is a workaround on pfSense itself. (I can only think of workarounds with two pfSenses).

Another feasable workaround i can think of is: if you use your own DNS servers you can set up loadbalancing split DNS.

Briantist

The funny thing is that I want to balance these servers specifically for OpenVPN clients, which is working! I'm just worried about the unintended consequences of it not working locally, and I'd also like to be able to use the virtual server address locally even though it isn't strictly necessary at the moment.

When you say two pfSense boxes would two in CARP failover work? At the moment I'm testing this at home where I only have one, but the end result would be run on two in failover, so I'm interested to hear what your ideas were for two.

Thanks!

GruensFroeschli

Hmmm. Not sure if it would work with two pfSense per CARP.
My idea was to have two pfSenses next to each other with both WANs in the same subnet (ok you have that with CARP).

The Users have LAN_A as their default gateway.
You would access the server over the IP of WAN_B
–> Traffic travels from WAN_B to LAN_B.
--> Serverloadbalancing should be possible.
A request from within the LAN appears to WAN_B as if from the IP WAN_A.
At the same time you have to enable source NAT on the pfSense_B (Advanced outbound NAT, create a rule for the LAN interface).
--> Requests from the pfSense_B to the Servers appear as if from an IP within the LAN
--> The server doesnt send the answer to its default gateway.

I havent tried this but it's interresting enough that i might set this up as a test.

I'm not sure how this would work with CARP since then the two pfSenses know about each other.

GruensFroeschli

Now that i think about it some more.
If you add another interface (and another subnet) and move the servers to this separate subnet (basically create a DMZ).
Shouldnt "true" NAT from LAN to OPT be possible?
–> Server loadbalancing should be possible again.

Briantist

@GruensFroeschli:

Now that i think about it some more.
If you add another interface (and another subnet) and move the servers to this separate subnet (basically create a DMZ).
Shouldnt "true" NAT from LAN to OPT be possible?
–> Server loadbalancing should be possible again.

So what I'm gathering here is that the load balancing basically needs NAT to work? Is that correct?

Putting it on a separate subnet is not a bad idea; in fact for this particular application, it makes a lot of sense, since it's going to be "clients" connecting through a VPN. Even though I'm limiting what they can access, I have been mulling around the idea of separating the services they connect to to a completely different network (either with VLAN, or physically separate; it's a tiny network). If this would solve the load balancer issue it would be another reason to do this.