Connection refused from Apache in DMZ

WhiteTiger-IT · Dec 15, 2020, 4:22 PM

After various tests and having installed pfSense several times in different configurations, I arrived at the configuration described here.

 Internet
     |
     | IP Public Dinamic & Private
 Router
     |192.168.1.1
     |
     |DHCP
   FW1 -OpenVPN----------IP Public Static
     |192.168.101.1
     |
     |DHCP
   FW2 --- DMZ 192.168.203.1 ------ Server Web (192.168.203.111)
    | 192.168.201.1
  LAN

Having only a private Public IP on the router, I was forced to activate a Static IP with OpenVPN.
This IP was used in the DNS A record.
This way, by calling www.mydomain.tld I should see the web server in the DMZ.

With this configuration I can access the Internet, both from the LAN and from the DMZ.
Furthermore, from the LAN, I can access the servers in the DMZ.

But when I call the domain I get "Connection refused" as a result from the web server in the DMZ .
From the LAN of FW2 the web server responds correctly, so on Apache the port 80 is enabled.
I also tried /in FW2) Diagnostic/Test Port indicating as Host name the IP of the internal server and as Source Address all the available options, always receiving "Port: 80 successful"

Apparently NAT and Rules manage to route traffic to the Web Server.

johnpoz · Dec 15, 2020, 4:25 PM

Where exactly are you trying to access it from using what IP?

WhiteTiger-IT · Dec 15, 2020, 4:26 PM

@johnpoz
From another PC connected directly to the Internet without going through my network

johnpoz · Dec 15, 2020, 4:28 PM

Ok so you would have to have 3 port forwards setup. On your edge router/firewall, pointing to fw1 wan IP, and then on fw1 to your fw2, and then on fw2 to your server.

Also if your trying to use 80, you should make sure webgui of pfsense is not using 80.

WhiteTiger-IT · Dec 15, 2020, 4:31 PM

@johnpoz said in Connection refused from Apache in DMZ:

Ok so you would have to have 3 port forwards setup. On your edge router/firewall, pointing to fw1 wan IP, and then on fw1 to your fw2, and then on fw2 to your server.

Also if your trying to use 80, you should make sure webgui of pfsense is not using 80.

However, the traffic goes through OpenVPN.
It certainly goes through the router, but with a different IP than the router's.
I put the rule in OpenVPN_Name

The Web GUI use another port

WhiteTiger-IT · Dec 15, 2020, 4:39 PM

I have entered these rules (related port 80)

On FW1
NAT Port Forward: From OpenVPN_Name to IP WAN Address of FW2
Rule: On OpenVPN_Name from Any to WAN Address of FW2

On FW2
NAT Port Forward: From Wan Address to IP Server Web
Rule: On WAN From Any to IP Server Web

johnpoz · Dec 15, 2020, 4:43 PM

@whitetiger-it Well if your coming in via vpn, then you need to make sure that the 192.168.203 is handed out as a network locally. And you would need to make sure clients of the vpn would resolve whatever fqdn your suing to this 203.111 address.

You need to make sure as well that server firewall allows access from your tunnel network.

Or if your natting at fw2 port forward. And you since you would be coming from rfc1918 you would need to make sure you remove the rfc1918 block on wan which is default.

WhiteTiger-IT · Dec 15, 2020, 4:50 PM

@johnpoz
There are no external OpenVPN clients; pfSense is a VPN client of a remote server that gives me a Static IP.
The client is active, I also see it in the Dashboard.

Debian Server Firewall (UFW) is currently disabled.
I have disabled the lock on rfc1918

But the error remains

WhiteTiger-IT · Dec 15, 2020, 5:34 PM

These are my rules.

FW1

FW2

johnpoz · Dec 15, 2020, 5:48 PM

@whitetiger-it And can you get traffic down to this internal IP your fw2 wan IP even?

So how is that going to work? Im out on the internet 1.2.3.4, or 5.6.7.8 for example

And I hit your public open vpn IP.. lets call it 9.10.11.12

And you send that down to fw2 wanIP and then forward it to your server. How is the return path going to know to go back through the vpn? You would have to source nat this traffic..

What your trying to do is pretty complicated.. And horrible idea to be honest.. Is this some openvpn host you control? Or some vpn service - vpn services rarely support port forwarding.

kiokoman · Dec 15, 2020, 5:59 PM

@whitetiger-it
are you sure the packets is going out from 192.168.203.111 to the openvpn interface?
smell asymmetric from here
i don't see any policy routing
~~the lock on rfc1918 is still visible on your screenshot~~ n.m. it's on FW1

WhiteTiger-IT · Dec 16, 2020, 10:13 AM

@johnpoz said in Connection refused from Apache in DMZ:

What your trying to do is pretty complicated.. And horrible idea to be honest.. Is this some openvpn host you control? Or some vpn service - vpn services rarely support port forwarding.

I agree with you that it is not an easy situation to manage, but I have not found other solutions to remotely access a server located in a network with private IP.
The server must have public access, so I can't activate a classic VPN solution, with the VPN server on the firewall and the VPN client on the user's PC for the simple reason that I don't know the user.
With this service they offer me a public Static IP associated with a VPN client for a small monthly cost.
The user calls a URL associated with this static IP, enters into the VPN and arrives on my firewall.
Not knowing the user, I didn't want to activate the VPN client on the main firewall, but I wanted to let the traffic enter on a firewall placed in front of it (FW1).
In this way FW2 manages the traffic from its WAN as it normally does, without worrying about who is entering and where it is coming from. It could be traffic coming directly from a router, it doesn't know that there is FW1 in between them.

The rules of FW1 only need to route traffic from the VPN to FW2 according to the authorized ports and discarding everything else.
The rules of FW2 are those that protect LAN and DMZ and route authorized traffic on the DMZ.
Obviously the traffic from LAN and DMZ, for example surfing the Internet and updating PCs and servers, go the opposite way:
LAN/DMZ -> FW2 -> FW1 -> Router -> Internet.

The problem in the problem is that I am new to pfSense and I am working on a project that I did not imagine so complex.
I thought I was working on a dynamic IP, but public and instead I later discovered it was private.

If I can't set up it, I will have to delete VPN and FW1 and then move the web server to a VPS on the Internet, but that's not what I wanted to do.

WhiteTiger-IT · Dec 16, 2020, 10:14 AM

@kiokoman said in Connection refused from Apache in DMZ:

@whitetiger-it
are you sure the packets is going out from 192.168.203.111 to the openvpn interface?
smell asymmetric from here
i don't see any policy routing
~~the lock on rfc1918 is still visible on your screenshot~~ n.m. it's on FW1

I didn't understand your suggestion.

johnpoz · Dec 16, 2020, 11:36 AM

Well first thing you need to validate is that traffic is actually getting to fw2 wan IP.. Do a simple sniff and then go to say can you see me . org and try to hit whatever your vpn public IP is on the port (80?) if fw2 never sees this traffic on its wan - then it never going to work..

Again what vpn server/service are you using? There are very limited services that provide port forwarding.. And they sure and the hell are not going to support port 80.. Not without paying extra that is for sure.. There are some that allow you to pick some high port and forward that - but it has to be setup, etc. Keep in mind that vpn services - your public IP is going to be shared with many other vpn users.. They only have 1 port 80 for each public IP, etc.

First thing to validate that traffic is even getting sent down your vpn.. If not coming down the vpn, you can never forward it to fw2, etc.

WhiteTiger-IT · Dec 16, 2020, 11:38 AM

@johnpoz said in Connection refused from Apache in DMZ:

Well first thing you need to validate is that traffic is actually getting to fw2 wan IP.. Do a simple sniff and then go to say can you see me . org and try to hit whatever your vpn public IP is on the port (80?) if fw2 never sees this traffic on its wan - then it never going to work..

OK. Never did it sniff, but it is the right time to learn.
Meanwhile, I was doing something else.
I brought a PC on the switch between the two firewalls, in this way I am downstream of FW1 and upstream of FW2.
From here I try to connect to the web server in the DMZ.
In theory the routing should work because I am not using the VPN but I am in the FW2 WAN network, right?

johnpoz · Dec 16, 2020, 11:57 AM

Not really any routing in such a setup.. You would hit the port you forwarded on fw2 wan IP to your server..

But yes some PC in the wan network of fw2 would need to be able to talk to the server if you ever want something from the internet to talk to it via your vpn connection.

Sniff (packet capture) is under the diagnostic menu..

edit: And again not sure how/why you think just connecting to some vpn service/server is going to expose your IP to the public internet. This would clearly have to be setup on the vpn server side that your connecting too. Before you worry about anything downstream on fw2.. You need to validate that traffic that hits this public IP of your vpn server/service is actually being sent down the tunnel to whatever pfsense vpn client IP is on fw1..

WhiteTiger-IT · Dec 17, 2020, 10:59 AM

@johnpoz
I'm going crazy.
I restored a basic situation and taking it one step at a time.
Now there is only FW2, directly connected to the router.
I can ping from the router to the WAN port of FW2, but I cannot ping it from Internet.
I have set the rule in WAN
ICMP Any
Source & Port Any
Destination & Port: WAN Address Any

On the router I don't see ICMP described and so I did a NAT of port 7 on IP of WAN Address.
For security in the WAN I also created the rule
TCP / UDP
Source & Port Any
Destination & Port: WAN Address 7

But in both cases I can't PING from a PC on the Internet

johnpoz · Dec 17, 2020, 11:23 AM

What is the wan IP of the router? If its cgnat (10.64/10), or rfc1918 then no your not going to be able to reach it from the internet unless your ISP has setup a port forward for you.

I really have no idea what this means?

IP Public Dinamic & Private

Is your connection via PPPoE? And you have a rfc1918/CGnat? Address and then actual public IP for the PPPoE interface?

ICMP has no port.. So it can be difficult to forward. What I would suggest you do for testing is pick some tcp port.. Lets say 4444 or something... Setup that forward on edge router (isp device) to forward to the wan IP of pfsense behind it... Now sniff on pfsense wan interface for that port.. And then go to can you see me . org and generate traffic to that port.. You should see it in your sniff.. Be it pfsense is forwarding or allowing or not, etc..