Java log4j vulnerability - Is pfSense affected ?

[johnpoz]

I am with @mer on this - I am not aware of pfsense using anything that would use this, for sure nothing out of the box that I am aware of.

I do not have any special insight into all the back end stuff, but a simple listing of packages doesn't show anything related to java for sure nor the freebsd log4j packages.

[21.05.2-RELEASE][admin@sg4860.local.lan]/root: pkg info
arpwatch-3.1                   Monitor arp & rarp requests
avahi-app-0.8                  Service discovery on a local network
aws-sdk-php74-3.103.2          PHP interface for Amazon Web Services (AWS)
bash-5.1.8                     GNU Project's Bourne Again SHell
beep-1.0_1                     Beeps a certain duration and pitch out of the PC Speaker
<snipped the rest for brevity>

I don't see how pfsense could be susceptible to this unless someone installed stuff on their own.. Off the top of my head I don't even think there is any official packages that would use this..

[Honest_Matt]

@johnpoz thanks. I’m running a very standard setup, so hopefully okay.

[johnpoz]

@honest_matt I would be more concerned with stuff behind pfsense. That you have open to the internet - or if you have anything local on your network that you would consider possible hostile that would have access to those devices.

I run the unifi controller on a VM, it is not open to the public internet or anything - and I have nothing on my network that I would consider even possible hostile (stuff in the dmz, my iot vlans, etc) that would have access to that. They are on isolated vlans from the unifi controller, etc.

But they released a updated version of the controller to address this issue.

[nimrod]

@johnpoz

I used

pkg info |grep java

And got nothing in return. But that just lists installed packages related to Java. Im worried about deep deep back end that can not be listed by running FreeBSD commands. Now that i think about this, im not even sure if this question is for pfSense dev team. Maybe i should check FreeBSD CVE database instead.

[johnpoz]

@nimrod is it not the log4j-java that is the problem - if so if you do not have that freebsd package installed. I don't see how there could be an issue.

edit:
I don't even see that package or java anything as possible to install from the pfsense repository - so for java and or that log4j-java to be installed on your system it would have to be side loaded, etc..

[21.05.2-RELEASE][admin@sg4860.local.lan]/root: pkg install log4j
Updating pfSense-core repository catalogue...
pkg: https://files00.netgate.com/pkg/pfSense_plus-v21_05_2_amd64-core/packagesite.pkg: Not Found
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pkg: https://files01.netgate.com/pkg/pfSense_plus-v21_05_2_amd64-pfSense_plus_v21_05_2/packagesite.pkg: Not Found
pfSense repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'log4j' have been found in the repositories
[21.05.2-RELEASE][admin@sg4860.local.lan]/root: pkg install java
Updating pfSense-core repository catalogue...
pkg: https://files00.netgate.com/pkg/pfSense_plus-v21_05_2_amd64-core/packagesite.pkg: Not Found
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pkg: https://files00.netgate.com/pkg/pfSense_plus-v21_05_2_amd64-pfSense_plus_v21_05_2/packagesite.pkg: Not Found
pfSense repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'java' have been found in the repositories
[21.05.2-RELEASE][admin@sg4860.local.lan]/root: 

[stephenw10]

@honest_matt said in Java log4j vulnerability - Is pfSense affected ?:

CVE-2021-44228

The only thing it's listed against in FreeBSD is Graylog:
http://vuxml.freebsd.org/freebsd/3fadd7e4-f8fb-45a0-a218-8fd6423c338f.html

pfSense does not ship with graylog. It's not available as a package.

log4j is not even in our repo as Johnpoz showed so you'd need to try very hard to install it.

Steve

[johnpoz]

As a PSA sort of thing - if your taking the time to look to see if you have any susceptible to this issue. It also a good time to just review your overall security stance.

Do you have stuff open to the internet, could it be just turned off? Could it be more restrictive in any way. Limit source IPs? To either known good clients IP or IP ranges, or limit to geoIP based. Are all your clients only in specific geoip area, if so limit to only that region (pfblocker is great for this sort of thing).

Could you use a VPN to access these resources while remote vs just port forwards from the internet.

On your local network segments, does your guest wifi network for example need access to the pfsense web gui? Do any of your local networks need access to your other networks? If they only need access to specific thing in another vlan, make sure your rules only allow that specific thing vs any, etc.

If your just all on 1 flat lan network, is it time you thought about segmenting your different sort of devices to their own vlans, say iot devices, stuff that is available via the public, etc.

Never a bad idea to do an overall review.. Maybe you were allowing something from vlan x to vlan y.. Is that something still in use? If not remove the rule that allowed that, etc.

Are all your things, fully patched, on the most current version.. If your not running current of pfsense for example.. Good time as any to make sure your on current 2.5.2, 21.05.2 etc..

[nimrod]

@stephenw10 said in Java log4j vulnerability - Is pfSense affected ?:

@honest_matt said in Java log4j vulnerability - Is pfSense affected ?:

CVE-2021-44228

The only thing it's listed against in FreeBSD is Graylog:
http://vuxml.freebsd.org/freebsd/3fadd7e4-f8fb-45a0-a218-8fd6423c338f.html

pfSense does not ship with graylog. It's not available as a package.

log4j is not even in our repo as Johnpoz showed so you'd need to try very hard to install it.

Steve

Yup. Just found that same info. Thanks for the confirmation Steve.

@johnpoz

I only use pfBlocker, Snort, and Squid. I have two PC`s connected to pfSense box. One machine is running FreeBSD, the other one is running Linux. No such packages are installed on either of those machines. As far as im concerned, this thread can be marked as solved.

Thank you everyone.

[qctech]

This is maybe an opportunity to educate users a little as well.

I have 20+ years experience in this game but am happy to admit that I don't really know how the log4j library is included in projects.

I am quite capable of logging onto a pfsense box or linux server and checking what packages have been installed by the package manager but I don't know, for example, if log4j could be included as part of another package in such a way as it does not show separately (compiled right in for example).

Confirmation from @stephenw10 and @johnpoz (thanks guys) that the underlying OS is not showing any known problems (short of a package that isn't actually available for pfSense) is great.

Maybe this isn't the right place to discuss how log4j might be included in other packages.
Maybe we can never be 100% sure anyway.
Maybe someone knows how it's normally used and could comment?

[johnpoz]

@qctech said in Java log4j vulnerability - Is pfSense affected ?:

log4j could be included as part of another package in such a way as it does not show separately (compiled right in for example).

While not saying something like that is not possible.. I have never noticed such a thing. If some package wants or needs a library to function, they normally just have the library package as a dependency that is installed along with their package.

It would be bad practice to do such a thing, where you hard code some library into your package. And could cause issues if the the package that includes such library functions was installed along with package that has the library included with it. Say some other package that uses the library package normally as a dependent package.

That for sure would make it harder to find and patch issues with library or other dependencies.

[bmeeks]

@qctech said in Java log4j vulnerability - Is pfSense affected ?:

This is maybe an opportunity to educate users a little as well.

I have 20+ years experience in this game but am happy to admit that I don't really know how the log4j library is included in projects.

I am quite capable of logging onto a pfsense box or linux server and checking what packages have been installed by the package manager but I don't know, for example, if log4j could be included as part of another package in such a way as it does not show separately (compiled right in for example).

Confirmation from @stephenw10 and @johnpoz (thanks guys) that the underlying OS is not showing any known problems (short of a package that isn't actually available for pfSense) is great.

Maybe this isn't the right place to discuss how log4j might be included in other packages.
Maybe we can never be 100% sure anyway.
Maybe someone knows how it's normally used and could comment?

I will try to explain (and rant a little bit as well) --

When you install a package on pfSense or any other system (FreeBSD, Linux, Windows, etc.), it is highly likely that dependent libraries will get pulled in and automatically installed. The Snort and Suricata packages are two great examples that I am familiar with since I maintain both of them and created one of them as well.

Snort formerly had the capability of exporting log data to a sub-system called Barnyard2. Barnyard2 pulled in Snort binary logging data and stored it into a database for easy searching and reporting. The default database used by Barnyard2 was MySQL. So when Snort was compiled, and the Barnyard2 feature was enabled (as it was on pfSense), a MySQL database connectivity client was automatically compiled and installed alongside of Snort and Barnyard2. So you got the Snort binary, the Barnyard2 binary, and a shared system library that allowed Barnyard2 to connect to and send data to a MySQL database engine running on a separate box. But over the years Barnyard2 ceased to be maintained, and that shared MySQL database connector library did not get updated. Turns out there were some security vulnerabilities uncovered in that library later on, and since Barnyard2 was no longer maintained, there was nobody to take charge and update the database client with a patched version. So I made the move some time back to drop Barnyard2 support and thus remove that vulnerable library.

The upstream Suricata team elected to start coding all new protocol decoders/detectors in Rust. So now, when you compile Suricata, it will drag in the full Rust language library. And because Rust itself is just a very basic programming language, it offers pretty much no built-in features for mundane things like math, string handling, file I/O and so forth. As a developer you have to get that by linking to and compiling in utilities created by others (meaning not the Rust core developers). These necessary plumbing parts you must have to do useful things are called Crates in Rust land. All of this (Rust and the Crates) is getting updated very, very frequently. And it makes me quite nervous. But I can't do anything about it because upstream decided Rust was the new magic pixie dust ... .

At least on FreeBSD, these dependencies are managed by the pkg utility. It determines what version of a given shared library an application being installed needs, then it goes and downloads and installs the needed library. Note that it is highly likely that the dependent libraries have, in turn, their own required runtime dependencies. That's why you will sometimes notice, if you pay close attention, that 50 or 60 things might get installed or updated when you install a single package you want. This is why us old guys caution all the newbies that want to install all kinds of third-party packages on their firewall. All those dragged-in dependencies can break something. And they also create a much larger potential attack surface.

Because pkg manages what gets installed and updated on pfSense, you can always run this command to see what is actually installed:

pkg info

Now for the rant part ---

You may have guessed by now that I am an older retired computer coding guy. And in fact, I could possibly be properly called a "curmudgeon". I've been around long enough to remember when Java was created and hailed as the greatest thing since sliced bread. We know now that was a bit of an overstatement. Java was akin to Rust in that you had the core language, but then you used Beans to do useful work. After Java, the next saviour to mankind in the programming world was Ruby and its Gems. But alas, that really didn't work out all that great so along came GO, and now there's Rust, too. Then there are all the scripting thingies such as Lua, Rexx, and the list goes on. Each of these programming tools has its collection of fan boys who fight endlessly over which is best.

I'm not a big fan of this approach of jumping on the next new hula-hoop thing. But new developers love to take the easy road, so they will use the latest "fad" language and then just import tools willy-nilly to do tasks like logging, or connecting to a database, drawing fancy charts and graphs, etc. The problem is you really have no good idea what kinds of security vulnerabilities you are opening up when doing that. That's exactly how something like the log4j2 tool got used all over the place - dragging a critical security flaw along with it.

There was actually a logging package I looked at for Suricata a few years ago that used Java! Soon as I found that out, I dropped it like a hot potato. Who in their right mind would put anything Java on a firewall? What thought process led that utility creator to write a Java-based logging package for firewall security software? If you are going to do that, why don't you just go ahead and include Adobe Flash as well ... .

[tquade]

@nimrod Only log4j2 is vulnerable. The older log4j (no "2") is not vulnerable. But people often say "log4j" to include either version. This is according to Johannes Ullrich, SANS Institute.

Ted Quade

[bmeeks]

@tquade said in Java log4j vulnerability - Is pfSense affected ?:

@nimrod Only log4j2 is vulnerable. The older log4j (no "2") is not vulnerable. But people often say "log4j" to include either version. This is according to Johannes Ullrich, SANS Institute.

Ted Quade

My bad -- fixed it in my post. Thanks for the hint.

[nimrod]

@bmeeks said in Java log4j vulnerability - Is pfSense affected ?:

@qctech said in Java log4j vulnerability - Is pfSense affected ?:

This is maybe an opportunity to educate users a little as well.

I have 20+ years experience in this game but am happy to admit that I don't really know how the log4j library is included in projects.

I am quite capable of logging onto a pfsense box or linux server and checking what packages have been installed by the package manager but I don't know, for example, if log4j could be included as part of another package in such a way as it does not show separately (compiled right in for example).

Confirmation from @stephenw10 and @johnpoz (thanks guys) that the underlying OS is not showing any known problems (short of a package that isn't actually available for pfSense) is great.

Maybe this isn't the right place to discuss how log4j might be included in other packages.
Maybe we can never be 100% sure anyway.
Maybe someone knows how it's normally used and could comment?

I will try to explain (and rant a little bit as well) --

When you install a package on pfSense or any other system (FreeBSD, Linux, Windows, etc.), it is highly likely that dependent libraries will get pulled in and automatically installed. The Snort and Suricata packages are two great examples that I am familiar with since I maintain both of them and created one of them as well.

Snort formerly had the capability of exporting log data to a sub-system called Barnyard2. Barnyard2 pulled in Snort binary logging data and stored it into a database for easy searching and reporting. The default database used by Barnyard2 was MySQL. So when Snort was compiled, and the Barnyard2 feature was enabled (as it was on pfSense), a MySQL database connectivity client was automatically compiled and installed alongside of Snort and Barnyard2. So you got the Snort binary, the Barnyard2 binary, and a shared system library that allowed Barnyard2 to connect to and send data to a MySQL database engine running on a separate box. But over the years Barnyard2 ceased to be maintained, and that shared MySQL database connector library did not get updated. Turns out there were some security vulnerabilities uncovered in that library later on, and since Barnyard2 was no longer maintained, there was nobody to take charge and update the database client with a patched version. So I made the move some time back to drop Barnyard2 support and thus remove that vulnerable library.

The upstream Suricata team elected to start coding all new protocol decoders/detectors in Rust. So now, when you install Suricata, it will drag in the full Rust language library. And because Rust itself is just a very basic programming language, it offers pretty much no built-in features for mundane things like math, string handling, file I/O and so forth. As a developer you have to get that by linking to and compiling in utilities created by others (meaning not the Rust core developers). These necessary plumbing parts you must have to do useful things are called Crates in Rust land. All of this (Rust and the Crates) is getting updated very, very frequently. And it makes me quite nervous. But I can't do anything about it because upstream decided Rust was the new magic pixie dust ... .

At least on FreeBSD, these dependencies are managed by the pkg utility. It determines what version of a given shared library an application being installed needs, then it goes and downloads and installs the needed library. Note that it is highly likely that the dependent libraries have, in turn, their own required runtime dependencies. That's why you will sometimes notice, if you pay close attention, that 50 or 60 things might get installed or updated when you install a single package you want. This is why us old guys caution all the newbies that want to install all kinds of third-party packages on their firewall. All those dragged-in dependencies can break something. And they also create a much larger potential attack surface.

Because pkg manages what gets installed and updated on pfSense, you can always run this command to see what is actually installed:

pkg info

Now for the rant part ---

You may have guessed by now that I am an older retired computer coding guy. And in fact, I could possibly be properly called a "curmudgeon". I've been around long enough to remember when Java was created and hailed as the greatest thing since sliced bread. We know now that was a bit of an overstatement. Java was akin to Rust in that you had the core language, but then you used Beans to do useful work. After Java, the next saviour to mankind in the programming world was Ruby and its Gems. But alas, that really didn't work out all that great so along came GO, and now there's Rust, too. Then there are all the scripting thingies such as Lua, Rexx, and the list goes on. Each of these programming tools has its collection of fan boys who fight endlessly over which is best.

I'm not a big fan of this approach of jumping on the next new hula-hoop thing. But new developers love to take the easy road, so they will use the latest "fad" language and then just import tools willy-nilly to do tasks like logging, or connecting to a database, drawing fancy charts and graphs, etc. The problem is you really have no good idea what kinds of security vulnerabilities you are opening up when doing that. That's exactly how something like the log4j2 tool got used all over the place - dragging a critical security flaw along with it.

There was actually a logging package I looked at for Suricata a few years ago that used Java! Soon as I found that out, I dropped it like a hot potato. Who in their right mind would put anything Java on a firewall? What thought process led that utility creator to write a Java-based logging package for firewall security software? If you are going to do that, why don't you just go ahead and include Adobe Flash as well ... .

I am experienced Arch Linux user, and i know very well what it means to keep things simple and clean. This is why i always install and use only what is needed. The thing with Linux distros and their package managers is that sometimes they mark certain packages as system essential packages and they will not show up when you try to list them with package manager. Probably due to security reasons and to avoid accidental removal.
However, these "hidden" packages do show up once you start full system update, and package managers clearly shows that they need to be updated. Im not use how it works on FreeBSD, but on Arch Linux every single package can be listed with:

sudo pacman -Q >list.txt

And every single package with all its dependencies can be forcefully removed by using:

sudo pacman -Rnsdd <package_name>

Of course, i dont need to point out that this command can be extremely destructiive if you do not know what you are doing.

Its good that there is nothing Java related in pfSense and its packages. All the info you shared here is useful, thank you sir.

[NollipfSense]

@bmeeks said in Java log4j vulnerability - Is pfSense affected ?:

The Snort and Suricata packages are two great examples that I am familiar with since I maintain both of them and created one of them as well.

I knew you were hardcore to the max...had to go off topic to give you a shout-out.

[ramtech]

@bmeeks said in Java log4j vulnerability - Is pfSense affected ?:

The Snort and Suricata packages are two great examples that I am familiar with since I maintain both of them and created one of them as well.

Sorry for the off-topic, but RESPECT @bmeeks ! Great rant, and couldn't agree more. (maybe something to do with us probably being similar vintage and background )

[bmeeks]

@ramtech said in Java log4j vulnerability - Is pfSense affected ?:

@bmeeks said in Java log4j vulnerability - Is pfSense affected ?:

The Snort and Suricata packages are two great examples that I am familiar with since I maintain both of them and created one of them as well.

Sorry for the off-topic, but RESPECT @bmeeks ! Great rant, and couldn't agree more. (maybe something to do with us probably being similar vintage and background )

Ha-ha! !

And all you damn kids get off our lawns! ...

[ramtech]

@bmeeks said in Java log4j vulnerability - Is pfSense affected ?:

Ha-ha! !

And all you damn kids get off our lawns! ...

[darcey]

I saw alot! of this CVE in my suricata (IDS mode) log from earlier today:

ET HUNTING Possible Apache log4j RCE Attempt - Any Protocol lower Bypass (CVE-2021-44228)
ET HUNTING Possible Apache log4j RCE Attempt - Any Protocol upper Bypass (CVE-2021-44228)
ET EXPLOIT Apache log4j RCE Attempt - 2021/12/12 Obfuscation Observed M1 (CVE-2021-44228)
ET EXPLOIT Apache log4j RCE Attempt - lower/upper TCP Bypass (CVE-2021-44228)
ET EXPLOIT Apache log4j RCE Attempt (http dns) (CVE-2021-44228)
ET EXPLOIT Apache log4j RCE Attempt (tcp dns) (CVE-2021-44228)
ET HUNTING Possible Apache log4j RCE Attempt - Any Protocol (CVE-2021-44228)

From the reported src/dst addresses/ports, these alerts seem to be associated with the following traffic:

1. syslog-ng on pfsense sending eve log to logstash on a local elastic server.
2. filebeat on a webserver sending nginx logs to elasticsearch on that same elastic server.

AIUI this particular pfsense installation and syslog-ng aren't vulnerable to the exploit. However the various elastic components involved may be.

This shows the distribution of these alerts:

From this, I'm guessing the rules appeared in my ruleset with a midnight update. Suricata then began furiously detecting and logging (no blocking on the firewall interface concerned). What interests me is how the alerts peak and decay. Then stop altogether despite the fact the same sort of traffic is almost certainly still crossing that interface.

Re log4j, it seems a very popular library. Several netwrok enabled applications I use daily make use of it. Elastic stack being one of them.

[darcey]

It's just occurred to me, the logging of these alerts triggers more of the very same alerts! Is it the case suricata rolls off alerting/logging, hence the pattern seen in the logging volume?
I'm guessing I need to disable my remote syslogging of suricata eve to logstash or disable some of those CVE-2021-44228 rules.