Can't connect on port 80...

[rcfa]

So I have two pfSense boxes, similarly (but not identically) configured.

One acts as expected: you connect with http, it works and you get redirected to https
On the other, the connection simply times out.

Even Diagnostics > Test Port, with an address of 127.0.0.1 fails on port 80 on that box, while connecting, as it should, on the other.

For testing purposes, I even temporarily added a pass-all rule, no go.

And of course the obvious: System > General Setup and System > Advanced are both 100% identical, except of course for the host name itself

Any ideas how to debug this?

One more thing: both these systems are old, with the configurations having gone through MANY pfSense upgrade cycles, so there's a chance that if there are bugs in pfSense's configuration migration scripts, there might be settings having been carried along that are somewhere hidden in the config.xml that I can't see. I'm not saying that's the issue, I'm just saying it's something to keep in mind.

I need this http redirect to get ACME Let's Encrypt certificates working, that's why I even noticed the problem, as usually I just use https to access the web UI. So, now, on one system I got the certs working, and on the other I can't because the challenge fails due to no response on the http/80

[johnpoz]

@rcfa said in Can't connect on port 80...:

Even Diagnostics > Test Port, with an address of 127.0.0.1 fails on port 80 on that box

Does the box have https redirection off?

[rcfa]

@johnpoz No, it doesn't, that's pretty much the first thing I checked. That's why I mentioned that System > Advanced are 100% identical on both boxes. Hence the mind being boggled. It's as if something were responsible for that setting being ignored.
I even tried turning it on saving it, and then back off, saving it again, to flush any potential weirdness out of the config.xml

[rcfa]

@johnpoz Another point of interest: I even tried setting the Protocol to http, but that made the box unreachable, so I had to use the ssh login to revert to the previous configuration, and that despite a pass-all rule!

[johnpoz]

@rcfa well its not listening on 80 so yeah going to be a problem..

Did you try toggle that?

So I have it off.. So nginx not listing on 80

Then I let it redirect and you see that it is listening on 80

[rcfa]

@johnpoz

[2.7.2-RELEASE][root@myhost]/root: sockstat -L | grep :80
root     lighttpd_p 47508 5   tcp4   10.10.10.1:80         *:*
root     lighttpd_p 47508 11  tcp6   ::10.10.10.1:80       *:*

That's all I get...

..and yes, I toggled that setting, actually more than once.

[johnpoz]

@rcfa well its for sure not going to work if not listening..

I would turn off pfblocker so its not listening on 80, then restart webgui - does it listen then? if not anything in logs when you restart it?

[Gertjan]

@rcfa said in Can't connect on port 80...:

[2.7.2-RELEASE][root@myhost]/root: sockstat -L | grep :80
root lighttpd_p 47508 5 tcp4 10.10.10.1:80 :
root lighttpd_p 47508 11 tcp6 ::10.10.10.1:80 :

That's all I get...

[23.09.1-RELEASE][root@pfSense.bhf.tld]/root: sockstat  | grep :80
root     lighttpd_p 92014 5   tcp6   ::10.10.10.1:80       *:*
root     lighttpd_p 92014 9   tcp4   10.10.10.1:80         *:*
......
root     nginx       5204 10  tcp4   *:80                  *:*
root     nginx       5204 12  tcp6   *:80                  *:*
root     nginx       4900 10  tcp4   *:80                  *:*
root     nginx       4900 12  tcp6   *:80                  *:*
root     nginx       4647 10  tcp4   *:80                  *:*
root     nginx       4647 12  tcp6   *:80                  *:*

I've "lighttpd_p", the pfBlockerng listening on the "VIP" interface 10.10.10.1
And several instances of nginx (the WebGUI webserver) on all the other, the LAN interfaces.

Open a second SSH, option 8, and

tail -f /var/log/system.log /var/log/nginx.log

and in the first SSH console access, use option 11 "Restart webConfigurator".

What do the logs tell you ?

[rcfa]

@johnpoz So, now the fun part after TWICE going through the procedure of 8 steps

1. turning redirect off
2. on the console restarting the web ui (option 11)
3. on the console restarting php-fpm (option 16)
4. on the console restarting the web ui (option 11)
5. turning redirect on
6. on the console restarting the web ui (option 11)
7. on the console restarting php-fpm (option 16)
8. on the console restarting the web ui (option 11)

It's now listening. That's progress. It's also redirecting on the LAN address.

It's still not redirecting on the WAN address, where I really need it.

This is DESPITE pfBlocker being disabled AND in the Firewall Rules every interface having at the topmost possible place a rule
to pass IPv4/IPv6 TCP/UDP traffic from anywhere to ports 22, 80, 443; including on the Floating Rules, where it's even a quick action rule.

Still no go on the WAN interface. Even rebooted the system a couple of times, to make sure it's not some random stuck process...

[2.7.2-RELEASE][root@myhost]/root: sockstat -L | grep nginx
root     nginx       9401 3   stream -> [9169 10]
root     nginx       9401 5   tcp4   *:443                 *:*
root     nginx       9401 6   tcp6   *:443                 *:*
root     nginx       9401 7   tcp4   *:80                  *:*
root     nginx       9401 9   tcp6   *:80                  *:*
root     nginx       9401 11  dgram  -> /var/run/log
root     nginx       9401 12  stream -> [9169 3]
root     nginx       9401 14  dgram  -> /var/run/log
root     nginx       9169 3   stream -> [9401 12]
root     nginx       9169 5   tcp4   *:443                 *:*
root     nginx       9169 6   tcp6   *:443                 *:*
root     nginx       9169 7   tcp4   *:80                  *:*
root     nginx       9169 9   tcp6   *:80                  *:*
root     nginx       9169 10  stream -> [9401 3]
root     nginx       9169 15  dgram  -> /var/run/log
root     nginx       9138 3   stream -> [9169 10]
root     nginx       9138 5   tcp4   *:443                 *:*
root     nginx       9138 6   tcp6   *:443                 *:*
root     nginx       9138 7   tcp4   *:80                  *:*
root     nginx       9138 9   tcp6   *:80                  *:*
root     nginx       9138 10  stream -> [9401 3]
root     nginx       9138 11  stream -> [9401 12]
root     nginx       9138 12  stream -> [9169 3]

[rcfa]

@Gertjan So, if I access the LAN address it redirects, and I can see the corresponding lines in the log.
If I use the WAN address: NOTHING; it's like if I hadn't done anything.

[johnpoz]

@rcfa so show it listening now.. Where are you saying you can not access it from?

I need this http redirect to get ACME Let's Encrypt certificates working

Why would you need that, if webgui is listening on 80, then yeah you would have hard time getting that to work.. Why would you not just use dns to get your acme cert? No need to expose anything to the internet that way.

[rcfa]

@johnpoz The WAN address doesn't react, but the firewall rules say it should be able to access on all interfaces ports 80, 443 and 22.
sss (22) and https (443) work just fine. But http (80) is utterly non-reactive.

[rcfa]

@johnpoz I'll describe in another post how I setup the ACME thing; works just fine on one machine, by the way. Used a non-privileged user with sftp into a chrooted enviroment. So unless chroot and/or scponly are broken, this should be rather safe an approach, at least safe enough for a machine which needs its admin interface open to the internet anyway, because it's on a colocation site far away from where I'd have physical access. (The machines mostly act as VPN-based routers, FW is just an added bonus)

So DNS is way too complicated, until it's migrated, as it's still self-hosted on an old computer, and the various automated DNS interactions aren't an option there...