pfb_dnsnl (pfBlockerNG DNSBL) service won't start

bobslee

@jrey The VIP in browser keeps loading and finally timeout.

Restart:

[23.05.1-RELEASE][USER@pfSense.local.lan]/usr/local/etc/rc.d: ./pfb_dnsbl.sh restart
2023-11-12 20:54:04: (/wrkdirs/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/plugin.c.209) dlopen() failed for: /usr/local/lib/lighttpd/mod_openssl.so Shared object "libssl.so.30" not found, required by "mod_openssl.so"
2023-11-12 20:54:04: (/wrkdirs/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1631) loading plugins finally failed

jrey

@bobslee said in pfb_dnsnl (pfBlockerNG DNSBL) service won't start:

keeps loading and finally timeout.

so no page loads then

@bobslee said in pfb_dnsnl (pfBlockerNG DNSBL) service won't start:

./pfb_dnsbl.sh restart

umm, according to the dependencies, pfBlocker should be using
lighttpd 1.4.72

and your error message for the restart says in part.

www/lighttpd/work/lighttpd-1.4.71

first check the version running - command prompt
lighttpd -v

should resspond
lighttpd/1.4.72 (ssl) - a light and fast webserver

if 1.4.72 (like above ^^)
stop
else
wouldn't hurt to grab a config backup, then
uninstall pfblocker (remove the package) (keep settings)
reboot pfsense
both pfblocker packages should be in the available package
install the non devel version (you're not missing anything)

jrey

@jrey said in pfb_dnsnl (pfBlockerNG DNSBL) service won't start:

according to the dependencies, pfBlocker should be using
lighttpd 1.4.72

My bad, I was looking on a different version of pfsense
under 23.09 the dependency is 1.4.71 so likely not changed from 23.05.1 (but I don't recall)
however under 2.7.1-RC (my sandbox) it is 1.4.72 for the same pfBlocker Version

there are minor differences in some of the other dependencies as well.

Under 23.09
Screen Shot 2023-11-12 at 4.11.46 PM.png

under 2.7.1-RC
Screen Shot 2023-11-12 at 4.12.24 PM.png

service is running on both here

but this:

2023-11-12 20:54:04: (/wrkdirs/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/plugin.c.209) dlopen() failed for: /usr/local/lib/lighttpd/mod_openssl.so Shared object "libssl.so.30" not found, required by "mod_openssl.so"
2023-11-12 20:54:04: (/wrkdirs/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1631) loading plugins finally failed

is likely the root cause why the service isn't starting, it is not seeing what it wants for a successful start.

Both of my instances only report a non-fatal cipher error when running pfb_dnsbl.sh restart but the non-fatal allows it to run, and service to start.

might need to trouble-shoot specifically the error you are getting,

"lighttpd not starting properly with llibssi.so not found error"

jrey

@bobslee

You didn't have a wrong repo branch selected when you did one of those updates you noted above.?

this implies maybe you did.

Before upgrade, "pfBNG-devel" was installed with "keep settings".

bobslee

@jrey Thanks for your effort!

Indeed, my lighttpd version is 1.4.71.

Regarding the update, I just followed the WebGUI update path.
I didn't altered the branch manually.

At this moment he Update window shows:

FreeBSD pkg program/command

More info to elaborate on the issue ...
Maybe following it's related to the openssl issue ?
Just checking here. Does pfSense ship with the pkg command ?

When I run eg pkg info lighttpd I get the output:
ld-elf.so.1: Shared object "libssl.so.30" not found, required by "pkg"

Which steps (of below) to proceed?

1. Reinstall pfBNG ?

Before this post, I already did your previous suggestion.
So maybe try again?

Uninstall pfBNG (keep settings)
Reboot
Install pfBNG non devel.

2. Update to 23.09 ?

Is it stable enough, things won't break?

Regarding backups:
I also use the ABC (auto config backup)
Is it possible to restore/rooback the whole system OS/FreeBSD with the ABC, in worst case?

bobslee

@jrey I noticed following.
However I didn't performed anything (no side effects) on the pfSense CLI yet, to keep it standard.

Probably the pkg command isn't available, instead there's pkg-static (kinda wrapper) ?

I found the documentation section "Troubleshooting Upgrades".
https://docs.netgate.com/pfsense/en/latest/troubleshooting/upgrades.html
Maybe there's useful info regarding the issue (pfBNG + lighttpd + openssl) and deps ?

bobslee

@jrey By the way ...
I doubt whether it's an openssl package issue, because the webserver (Lighttpd) serves HTTPS and also OpenVPN server/client (does use SSL?) still works.

Kinda lost now.

jrey

@bobslee said in pfb_dnsnl (pfBlockerNG DNSBL) service won't start:

When I run eg pkg info lighttpd I get the output:
ld-elf.so.1: Shared object "libssl.so.30" not found, required by "pkg"

Sound about right, if the Repo is pointing at 23.09 which it appears to be in the screen shot showing latest stable version above. Anything installed would be puling from there, not the version you are on which is still. 23.05.1

You might find this thread helpful in this regard.

https://forum.netgate.com/topic/183088/error-libssl-so-30-not-found-when-installing-package?_=1699874971773

bobslee

@jrey Ok that clarifies :)

The system update screen seems a bit awkward to me, to pin the pkg repo that way.
It also suddenly was set to 23.09 (I didn't manually).

Can you recommend how I can proceed quickly and safely ?
Sorry I'm really in a lack of time.

Either, one of below ... ?

(1) Change Branch ?
Should I just change the "Branch" to "Previous Stable Version (23.05)" ?
Does this immediately updates the pkg index ?
Or which action to undertake here ?

(2) Update the systgem (OS + packages) to 23.09 ?

jrey

@bobslee said in pfb_dnsnl (pfBlockerNG DNSBL) service won't start:

It also suddenly was set to 23.09 (I didn't manually).

Interesting before an update, it should always be showing you the version you are currently on and the drop down will have the next (if one is available) or last as selection options.

Screen Shot 2023-11-13 at 7.06.36 AM.png

Clearly the screen capture you provided is showing conflicting information,
a) that you are on the branch "Latest Stable Version (23.09)
b) that you are current at 23.05.1

Seems broken ;-)

If you are formatted ZFS Boot Environments might be your friend ?

I really can't recommend the next best course of action (and there are many) for your particular situation, that is a risk evaluation for each case, you need to make.

If you try to change the Branch here, forward or backward give it several minutes to adjust. I select the branch, navigate to the dashboard, grab a coffee, come back and confirm what the screen says, then proceed if the choice is appropriate. I've never had to select a previous version.

Often times in cases like this a fresh new image (start from scratch) is what I see recommended more often than not.

bobslee

@jrey Thanks for your suggestions and effort !

I really appreciate it !
I understand the dedication in an open source community, as I also develop and maintain a big project.

I feel a bit n00b now.
Apparently the System Update version pinning was the cause of all this!

As you suggested:
I selected the previous stable version (my base system is on).
Waited.
Removed/uninstalled pfBNG
Installed pfBNG

Now the DNSBL service just works !

I feel sorry about your efforts.
But the pfSense UI/UX could be improved here.

jrey

@bobslee

Awesome !

I feel sorry about your efforts.

No problem, it was nice we could troubleshoot down to the actual cause of the DNSBL service not starting.

Even though the DNSBL was by all accounts "running", the service itself didn't show as started. The inability of the start to actually record/report on the lighttpd startup failure. Seems like a problem.

That of course is a completely different issue..

Have a great day!

bobslee

@jrey Indeed, it sometimes takes a few hurdles to find the actual cause is somewhere else.

I don't whether the DNSBL was actually operational, because now the browser responds with pfBNG page when I request the VIP.

Thanks and enjoy your day ! :)

jrey

@bobslee said in pfb_dnsnl (pfBlockerNG DNSBL) service won't start:

I don't whether the DNSBL was actually operational, because now the browser responds with pfBNG

I believe it was running because your screen capture of the dashboard showed packets and your provided DNS response showed 0.0.0.0 as the IP.

But what wasn't working was the web server to display the page if required.
(ie also a silent failure IMHO)

part of the service start sequence it to start lighttpd (but with that failing hard) the start service script saw that as a failure and showed the DNSBL status as stopped.

Needs work. Seems to me the error.log may have been an appropriate place for that hard failure to have been recorded.

Cheers

bobslee

@jrey I agree with your explanation.
My previous conclusion was silly here.. sorry (need a vacation).

The Lighttpd webserver (for WebGUI purpose) indeed isn't the same as the actual DNSBL service.
Thanks for sharpen my brain!

kab43

@jrey turns out i also had to update , i feel so silly thx for the troubleshoot