HA Proxy working only internally with a new WAN
-
Hi everyone,
We are recently changing our main WAN with a new provider.
We were able to set up the new interface and the gateway and they seem to be running ok. In order to proceed with testing we tried to create a new HAProxy rule which would be using the new WAN/Gateway. On one of our DNS we created a new record which pointed to the new IP and this rule is redirecting the "clients" to one of our already working services.
It seems like we are able to access this service as long as we are in our internal net but when we try to test it from the outside it doesn't seem to be working.
We are not experts in pfSense and we do not know why this would be happening, if either Interface Assignments need to be changed or modified or if it might be something else.Feel free to ask for any configurations.
Thanks.
Marc.
-
@hyds-devops said in HA Proxy working only internally with a new WAN:
On one of our DNS we created a new record which pointed to the new IP
On a public DNS?
It seems like we are able to access this service as long as we are in our internal net but when we try to test it from the outside it doesn't seem to be working.
Did you add a rule to WAN to pass the traffic?
Other than with port forwarding you need now a rule with the public IP or "This firewall" as destination. -
@viragomann said in HA Proxy working only internally with a new WAN:
On a public DNS?
Yes we added the rule on a public DNS.
@viragomann said in HA Proxy working only internally with a new WAN:
Did you add a rule to WAN to pass the traffic?
We do have a rule to pass the traffic on all WANs to the HAProxy Ports.
Other than with port forwarding you need now a rule with the public IP or "This firewall" as destination.
I do not understand which rule do we need.
Besides this we found out that we are unable to ping our gateway since the ip for it provided by the "dynamic" Gateway (unable to change in PPPoE WANs) is assigning a Private IP outside the range of all our other systems. All our other private IPs are of the range 10.X.X.X.Regards.
-
@hyds-devops said in HA Proxy working only internally with a new WAN:
Besides this we found out that we are unable to ping our gateway since the ip for it provided by the "dynamic" Gateway (unable to change in PPPoE WANs) is assigning a Private IP outside the range of all our other systems.
That's the special feature of PPP connections.
But you have to ensure that the gateway IP is not overlapping one of your local networks.Pretty extraordinary to get a private gateway IP on a PPPoE.
-
@viragomann said in HA Proxy working only internally with a new WAN:
Pretty extraordinary to get a private gateway IP on a PPPoE.
I'm pretty sure it is not overlapping with any of my other local networks since we dont use the range 192.169.X.X at all.
The thing is that we cannot ping the "gateway" IP since it is outside the range of 10.X.X.X. And we don't know if there is a way to possibly either decide the range of the gateway ip or just use the one that appears on "interfaces" (2nd image, which even if it is blurred it shows our public static ip given by the ISP).
-
@hyds-devops
So did you disable the gateway monitoring? Otherwise dpinger would ping the gateway as well to determine its state and would show it as offline if it gets no responses.What shows Status > Gateways?
As mentioned, with PPP the gateway might be outside of your WAN subnet. Basically you get a /32 WAN IP anyway.
Also it might be on the ISPs gateway to response to pings or not. If it doesn't go into the gateway settings and set an alternative public IP for monitoring.
For instance enter 8.8.8.8. This also add an invisible static route for the monitoring IP pointing to the concerned gateway.
This way you can check if the new WAN connection is working without disabling the other WAN. -
Sorry for the late reply. Thanks for answering.
@viragomann said in HA Proxy working only internally with a new WAN:
So did you disable the gateway monitoring? Otherwise dpinger would ping the gateway as well to determine its state and would show it as offline if it gets no responses.
What shows Status > Gateways?We did not disabled monitoring. The Monitor IP for that gateway is 1.1.1.1. This is what show in Status - Gateways:
As mentioned, with PPP the gateway might be outside of your WAN subnet. Basically you get a /32 WAN IP anyway.
Also it might be on the ISPs gateway to response to pings or not. If it doesn't go into the gateway settings and set an alternative public IP for monitoring.
For instance enter 8.8.8.8. This also add an invisible static route for the monitoring IP pointing to the concerned gateway.
This way you can check if the new WAN connection is working without disabling the other WAN.I guess this would mean that the Gateway it's working, but besides this fact we don't seem to be able to get any of the services working with the new WAN. The only tangible difference we can notice between both WANs (have in mind that the other WAN is not PPPoE and it is our first time trying to get a PPPoE WAN working in the office) is that we can ping the "working"/previous WAN Gateway IP but not the new one.
-
@hyds-devops
That the gateway does not responding to pings does not matter.
If the monitoring is working by using an alternative IP in the gateway settings the gateway is functioning.You can verify this by running a packet capture on the new WAN while your ping 1.1.1.1. You should see the ICMP packets there.
-
@viragomann said in HA Proxy working only internally with a new WAN:
rnative IP in the gateway settings the gateway is functioning.
You can verify this by running a packet capturHi,
We could indeed verify that the ICMP packets are there and that the gateway is running, the only thing we do not understand is why would the HAProxy rules regarding that gateway not work outside our local network. -
@hyds-devops
So how did you configure HAproxy?
How are you able to access it from inside your network. Do you use the public host name?The HAproxy frontend has to be configured to listen on the WAN2 address.
And you need a firewall rule on WAN2 to pass access (HTTP, HTTPS or whatever) to this address.Ensure that there is no NAT portforwarding for the WAN2 IP.
Also ensure that there is no firewall pass rule on interface group or a floating rule matching the packets to this IP, and that in the Advanced Settings > Firewall the "reply-to" is not disabled. -
@viragomann said in HA Proxy working only internally with a new WAN:
So how did you configure HAproxy?
How are you able to access it from inside your network. Do you use the public host name?
The HAproxy frontend has to be configured to listen on the WAN2 address.
And you need a firewall rule on WAN2 to pass access (HTTP, HTTPS or whatever) to this address.
Ensure that there is no NAT portforwarding for the WAN2 IP.
Also ensure that there is no firewall pass rule on interface group or a floating rule matching the packets to this IP, and that in the Advanced Settings > Firewall the "reply-to" is not disabled.The HAProxy is listening to the WAN and we have the pass rules for it and we have no portforwarding.
We did some testing this morning enabling the WAN2 as the default gateway and it worked. Sadly all other services regarding WAN1 stopped working so we figured that we can only have 1 gateway working simultaneously (?). We were told by NetGate support that we were able to have both but it might've been a misunderstanding.We scheduled a maintenance for tomorrow morning to migrate all rules/services/DNS records to the new IP/WAN and we think it will resolve our issue. Regarding the HA Proxy and the theme of this thread I think it has been resolved. Thanks @viragomann for all the help.
-
@hyds-devops said in HA Proxy working only internally with a new WAN:
Sadly all other services regarding WAN1 stopped working so we figured that we can only have 1 gateway working simultaneously (?).
No, it's also possible to have incoming traffic on multiple WANs simultaneously.
I noticed that the WAN2 is not your default gateway and the guidelines in my last post above are targeted on this set up.
If all are obeyed access from non-default gateway WAN should work properly. -
@viragomann said in HA Proxy working only internally with a new WAN:
No, it's also possible to have incoming traffic on multiple WANs simultaneously.
I noticed that the WAN2 is not your default gateway and the guidelines in my last post above are targeted on this set up.
If all are obeyed access from non-default gateway WAN should work properly.It is weird, we reviewed all the setting and we do have the HAProxy set to listen to the WAN2 address, we do have the WAN2 pass rules on the firewall, there is now some NAT portforwarding on WAN2 to replicate the behaviour of the WAN1 but it should not interfere with the HAProxy (there was no NAT rules before today) and there is no firewall floating rule. "reply-to" is not disabled.
The only difference I can spot is that the HAProxy firewall rule to pass the traffic is on the "WANs" group instead of on each interface. -
@hyds-devops
As I wrote above, there must no interface group rule passing the incoming traffic on a non-default WAN. The reply-to isn’t working with these rules.Also would not work on version 2.5.1.
-
@viragomann said in HA Proxy working only internally with a new WAN:
As I wrote above, there must no interface group rule passing the incoming traffic on a non-default WAN. The reply-to isn’t working with these rules.
Also would not work on version 2.5.1.Thanks, we will definetly try this out. We will put the current interface group rules to the specific WANs. Will update on Friday since due to critical moments on our services we will not make any changes until then.
-
The issue was resolved following the steps said by @viragomann . The issue was indeed that the rules regarding the HAProxy on the firewall were on the WANs group. We changed them individually on the WAN1 and WAN2 firewall rules and it worked. Thanks.