DHCP Broadcasts cluttering up log from LAN

mightykong

I just pushed my pfSense box into production after doing some testing, it replaced an aging NetGear box. I've been checking the logs just to make sure everything is okay, and I'm seeing tons of these messages in the Firewall log:

block Aug 29 09:49:10 LAN 0.0.0.0:68 255.255.255.255:67 UDP
block Aug 29 09:49:28 LAN 0.0.0.0:68 255.255.255.255:67 UDP
block Aug 29 09:49:29 LAN 0.0.0.0:68 255.255.255.255:67 UDP
block Aug 29 09:49:30 LAN 0.0.0.0:68 255.255.255.255:67 UDP

DHCP appears to be working fine on the LAN, so it's not causing any issues, and the rules for the LAN interface right now allow any IPv4 traffic. So, I'm a bit confused as to why this is cluttering up the logs. I saw a post on quieting this noise on the WAN interface, but not sure if I should use that method as I don't want it to affect DHCP internally. Although, as I said it seems to be working fine now.

Anyone have any insight as to why I would be getting this with internally?

badon

I noticed this problem too, after I added an explicit "block everything" rule that could log what was getting blocked instead of letting them go invisibly discarded. There are several special IP's, like DHCP broadcasts in this case, that don't need to go through the firewall (from LAN to WAN), so there is no point in logging each time the firewall blocks them. One easy way to remove them from your logs is to add explicit block rules for each of the IP's you want to "go away", and just don't check the box for logging the matches to the rule. Also, be sure to put the explicit IP blocks before the "block everything" rule ("block everything" should be last). If you don't, they will get logged.

I just sorted all this out in the last few minutes, so I might have overlooked something, but I think this will work for you. I attached a screenshot of what my rules look like at the bottom of the list of rules. I assume you're familiar with how to use the logging checkbox for each rule (or you can figure it out easily), so I didn't bother to make a screenshot of that.

![explicit non-logged block rules to unlog mundane traffic.png](/public/imported_attachments/1/explicit non-logged block rules to unlog mundane traffic.png)
![explicit non-logged block rules to unlog mundane traffic.png_thumb](/public/imported_attachments/1/explicit non-logged block rules to unlog mundane traffic.png_thumb)

johnpoz

So pfsense is not dhcp server for your lan segment?

badon

Actually it is, but for some reason, DHCP still works. I suspect the firewall is blocking the DHCP packets AFTER they reach the DHCP server, because the firewall is only blocking LAN to WAN, not LAN to LAN. I'm guessing that the firewall assumes 255.255.255.255 must be outside the LAN, which is wrong. If you look at this page in your pfSense:

https://192.168.1.1/services_dhcp.php

You will see this:

Available range 192.168.1.1 - 192.168.1.254

So, the available range does not include 255.255.255.255 or 0.0.0.0, and that could be why the firewall assumes it must be beyond the LAN. pfSense probably wrongly assumes those IP's are not truly LAN to LAN, so it logs a block action in the firewall. That could be a bug. I will investigate if the issue has been reported before, and if I'm confident it's a valid unreported bug, I will report it.

badon

I reported the bug here:

Bug #3865: With explicit block-everything rule in firewall it incorrectly blocks LAN to LAN DHCP broadcast 0.0.0.0 and 255.255.255.255 - pfSense - pfSense bugtracker

P3R

@badon:

I suspect the firewall is blocking the DHCP packets AFTER they reach the DHCP server…

I agree.

…because the firewall is only blocking LAN to WAN, not LAN to LAN.

The firewall isn't processing LAN-LAN packets unless they are destined for the firewall (since broadcasts are destined for all nodes on the network, they are also processed by pfSense).

I'm guessing that the firewall assumes 255.255.255.255 must be outside the LAN…

No it doesn't.

If you look at this page in your pfSense:

https://192.168.1.1/services_dhcp.php

You will see this:

Available range 192.168.1.1 - 192.168.1.254

So, the available range does not include 255.255.255.255 or 0.0.0.0, and that could be why the firewall assumes it must be beyond the LAN.

No it's because neither network nor broadcast addresses should ever be handed out to clients by the DHCP server.

The available range information on the DHCP server page isn't documenting what is or isn't within the LAN network, it is telling you what is the possible range of addresses to serve to clients on the network.

pfSense probably wrongly assumes those IP's are not truly LAN to LAN, so it logs a block action in the firewall. That could be a bug.

Let's start with the basics. A router, firewalling or not, should never forward broadcasts to any other of it's interfaces.

The broadcasts are, as they should be, present and kept within your LAN network. They shouldn't be forwarded to other interfaces, so stopping them as your pfSense is doing is correct.

badon

Thank you for the clarification.

johnpoz

Here is the thing - if he is running dhcp server, then there needs to be a rule to allow that into pfsense - so it wouldn't be in the block log

Look in your full rule set
https://doc.pfsense.org/index.php/How_can_I_see_the_full_PF_ruleset
pfctl -sa

So here is sniff from mine showing that dhcp is allowed - so how would it be blocked and logged?

pass in quick on vmx3f1 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
pass in quick on vmx3f1 inet proto udp from any port = bootpc to 192.168.1.253 port = bootps keep state label "allow access to DHCP server"
pass out quick on vmx3f1 inet proto udp from 192.168.1.253 port = bootps to any port = bootpc keep state label "allow access to DHCP server"

My guess would be not actually running dhcp, and something else is dhcp on the network would be how the clients are getting addresses, etc.

Or he has rule setup blocking and its not the default rule blocking.. But if he had a rule blocking then dhcp wouldn't work!! I don't see how he can say dhcp is working, yet the dhcp discover packets are being blocked.

badon

Here's a screenshot of some DHCP leases. I assume this is conclusive evidence that DHCP is truly working despite there being explicit block rules in my config. Of course, I could just as easily make explicit pass rules, because the only objective here is to remove the noise from the firewall logs.

![pfSense DHCP leases.png](/public/imported_attachments/1/pfSense DHCP leases.png)
![pfSense DHCP leases.png_thumb](/public/imported_attachments/1/pfSense DHCP leases.png_thumb)

johnpoz

How can it log something that it clearly passed if you got a lease?

Can you post your output from the pfctl -sa for the rules

And post your actual rules on this lan and if you have any floating rules setup.

badon

Your guess is as good as mine as to how that's happening. I don't have any floating rules. The other rules contain secret information, and the output from pfctl -sa is loaded with more secret information. I could sanitize it before posting it, but I'd be afraid of missing something. Maybe you can set up a test system to see if you can duplicate it? I've duplicated it myself on 2 different machines, so it ought to be straightforward. I will be curious if you can figure out what's happening.

johnpoz

secret info?? Like what that you block outboud from something called lan net to a public IP address??

Secret info – yeah not so much.

I run dhcp on my 32bit 2.1.5 pfsense on multiple segments, 2 wired only and 1 with wireless clients even and have NOT seen this issue not even one hit it firewall logs. I have setup a test pfsense with 64bit 2.1.5 that also provides dhcp to clients and have never seen this issue.

Because it doesn't seem possible - there is a rule when you turn on dhcp that allows the dhcp broadcast to your interface dhcp server is listening on. So if the packet is passed in this rule, how it would it be listed as block in another rule??

Seems highly highly unlikely -- if is a was bug in code, then you would think lots of people would be reporting this. I have only seen this 1 thread with any such info. So would seem like something specific to your setup.

If you do not want to put the "secret" firewall rules in a public forum - then feel free to PM them to me and can take a look see. But without seeing your rules there is nothing else to do here from my point of view.

if you have duplicated this on 2 setups up pfsense - then I would think the default rules you believe are the issue. Or are you setting them up with your "secret" rules? I would suggest you create a clean install with NO "secret" rules and do you get the error then?

so you see in my output from pfctl -sa the rules for the dhcp.

bootpc = port 68, and bootps = 67

So that rule says let ANY source from port 68 using UDP to 255.255.255.255 (full broadast) to 67 (bootps) allow/pass So if this rule allows the traffic that you are showing blocked. How would some other rule block the same traffic?

The next rule says hey if its direct traffic from same ports to the IP address of my interface that dhcp server is running on also allow it. This would be part of the dhcp process. But from 0.0.0.0 to 255.255.255.255 is a dhcpdiscover packet. And has to be allowed in for new clients to get a lease.

Have you modified these rules at a low level - are they not in there any you had to create some rule to allow dhcp to work? What are these modifications or rules you have created?

Have you somehow disabled this rule and only have your direct rule - so renews would work? Please validate the rules I show are in your pfctl -sa output. That clearly is not any sort of "secret" info ;)

P3R

Okay, this issue is more complicated than what I understood initially. :-[

Since I was having the same block log entries I enabled [b]Status, System Logs, Settings, Filter descriptions and enabled it. That way I could find out it is the block bogon rule causing the log entry.

Then I found that 0.0.0.0/8 is part of the bogon list, which explains the block.

Since I also have working pfSense DHCP servers on all my local networks, I'm now also interested in finding out exactly how the broadcast packets can still reach port 67 in pfSense despite being logged as blocked. :o :o :o

I can see two reasons for why this behaviour isn’t noticed by everyone and haven’t been discussed previously: not everyone enables bogon blocks on internal interfaces and even when doing so, DHCP still works…

P3R

@badon:

I just sorted all this out in the last few minutes…

Your workaround doesn’t work because any non-logging rules will be below the bogon rule that is the culprit.

I have so far only come to think of two possible workarounds to get rid of the log entries:

Go to Status, System Logs, Settings, Log Firewall Default Blocks and untick the “Log packets blocked by 'Block Bogon Networks' rules” option.
Go to Interfaces, LAN and untick the "Block bogon networks" option.

Unfortunately both of my workarounds have other consequences also.

johnpoz

Why would anyone block bogon on the lan side?? The rules on the lan would be limited to source networks of your lan - so what would be the point of blocking bogon? Your already blocking EVERYTHING that does not match your lan segment as source..

P3R

@johnpoz:

Why would anyone block bogon on the lan side??

Let me speculate here: as a newcomer it is hard to immediately understand each and every setting in detail and it may be tempting on initial installation to add something that seemingly increases the protection.

The rules on the lan would be limited to source networks of your lan - so what would be the point of blocking bogon? Your already blocking EVERYTHING that does not match your lan segment as source..

Your posts here given me a much better understanding of how pfSense works and what you say makes sense but the configuration is still possible.

In my opinion the only interesting question is why that configuration leads to a situation where the behaviour is in conflict with our present understanding of the software design. How can the DHCP server actually still work despite the critical initiating broadcast being reported as blocked?

johnpoz

"to add something that seemingly increases the protection."

Not sure I would agree with newcomer to "pfsense" - new comer to firewalls and network as a whole maybe? ;) Bogon is not setup on the lan interfaces out of the box if I recall - such a rule is pointless with the default rule on the lan interface.

With the way dhcp works - a client will send out discover that will be a broadcast to 255.255.255.255 from 0.0.0.0 - I agree with you if that traffic blocked the dhcp server should never see the discover, and then could never send an offer for the client to accept.

Even on a renew the source would still be 0.0.0.0 – possible some clients send a renew it would be from source IP of the client?? I am not sure on every single client out there, but in general from the protocol level if I recall correctly he renew would be sourced from 0.0.0.0.

In such a case if that traffic is blocked it should not be possible to get a dhcp lease - if pfsense is actually blocking it as it reports, then it should not be possible. If there is a rule that allows the traffic as I posted from the pfctl -sa output it should never get to a block rule to be logged that it was blocked.

What I can do when I get a chance is enable bogon on my test segment while doing a sniff and see exactly what is going.. If in fact the discover is listed as blocked but not then that is a huge bug if you ask me.

P3R

@johnpoz:

With the way dhcp works - a client will send out discover that will be a broadcast to 255.255.255.255 from 0.0.0.0 - I agree with you if that traffic blocked the dhcp server should never see the discover, and then could never send an offer for the client to accept.

Yet, that is what I'm and badon is seeing.

I redid the test once again with the following output:

16:40:19.967957 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
16:40:19.969717 IP 192.168.11.1 > 192.168.11.14: ICMP echo request, id 32978, seq 0, length 28
16:40:20.972225 IP 192.168.11.1.67 > 192.168.11.14.68: UDP, length 300
16:40:21.854685 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 308
16:40:21.858652 IP 192.168.11.1.67 > 192.168.11.14.68: UDP, length 300

And

In such a case if that traffic is blocked it should not be possible to get a dhcp lease - if pfsense is actually blocking it as it reports, then it should not be possible.

Exactly, and that is the only reason I still find it interesting to participate in this thread.

What I can do when I get a chance is enable bogon on my test segment while doing a sniff and see exactly what is going..

That is what I have done so, please do.

Bogon.JPG_thumb

johnpoz

Ok so I turned on bogon, and I enabled logging of bogon.. And now you have my attention - clearly something is ODD here..

So if you look at /tmp/rules.debug – this should show you order of the processing of the rules

So scroll down and you find

block bogon networks (IPv4)

http://www.cymru.com/Documents/bogon-bn-nonagg.txt

block in log quick on $DMZ from <bogons>to any label "block bogon IPv4 networks from DMZ"

block bogon networks (IPv6)

http://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt

block in log quick on $DMZ from <bogonsv6>to any label "block bogon IPv6 networks from DMZ"
antispoof for vmx3f3

allow access to DHCP server on DMZ

pass in quick on $DMZ proto udp from any port = 68 to 255.255.255.255 port = 67 label "allow access to DHCP server"
pass in quick on $DMZ proto udp from any port = 68 to 192.168.3.253 port = 67 label "allow access to DHCP server"
pass out quick on $DMZ proto udp from 192.168.3.253 port = 67 to any port = 68 label "allow access to DHCP server"

So clearly blocking bogon's is before the dhcp rules.. and the "quick" is set -- so that means no other rules are processed and should be dropped and should not get to dhcp server..

And I do see it in the log with the exact timestamp of my dhcp lease - see attached.

If you do a pfctl -s rules

block drop in log quick on vmx3f3 from <bogons>to any label "block bogon IPv4 networks from DMZ"</bogons>
block drop in log quick on vmx3f3 from <bogonsv6>to any label "block bogon IPv6 networks from DMZ"
block drop in on ! vmx3f3 inet from 192.168.3.0/24 to any
block drop in inet from 192.168.3.253 to any
block drop in on vmx3f3 inet6 from fe80::20c:29ff:fe1e:18c2 to any
pass in quick on vmx3f3 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
pass in quick on vmx3f3 inet proto udp from any port = bootpc to 192.168.3.253 port = bootps keep state label "allow access to DHCP server"
pass out quick on vmx3f3 inet proto udp from 192.168.3.253 port = bootps to any port = bootpc keep state label "allow access to DHCP server"

Its listed above the dhcp rules as well

Going to need one of the developers to let us know how it is that we get a log of something being blocked, yet still passed?? And from what I know of the way the rules are done - it looks like that should just be dropped and should never get to the dhcp server. That is kind of big issue if you ask me.

But this really should never come in to play really - because why would bogon be enabled on LAN interface? Your rules should limit to source of your LAN network, or sub networks even. Everything else would be blocked by the default block rule.

showsblocked.png_thumb </bogonsv6></bogonsv6></bogons>

badon

@johnpoz:

Why would anyone block bogon on the lan side?? The rules on the lan would be limited to source networks of your lan - so what would be the point of blocking bogon? Your already blocking EVERYTHING that does not match your lan segment as source..

P3R, thanks for chiming in with excellent info that helps us make progress on figuring all this out. I think I had reached my limit on how much I could help here.

johnpoz, correct me if I'm wrong, but I think another thing to consider is the possibility that "LAN side" packets never even touch pfSense if there is a managed switch or router in the network somewhere that can route packets to the correct LAN port immediately without needing to spam the other devices and network segments with packets that are not intended for them. In other words, blocking bogon networks on the LAN side may not even be effective, even if there were some good reason to do that, which we aren't aware of at the moment.

If so, then that may motivate us to question why such a feature exists in pfSense, and thus cause us to consider the possibility of removing the feature, or adding some documentation (preferably next to the checkbox itself) that explains the circumstances when the feature may not work as expected.

Another thing to consider is whether 0.0.0.0 and 255.255.255.255 are truly bogon IP's, because of their universal deployment as a standard for DHCP services. I'm not aware of any other uses for those IP's, but I'm skeptical that there are any "official" standardization rules that define them, so they might be a de facto standard in practice solely because that's what everyone decided to use those IP's for. Determining whether those DHCP IP's are a standard of some kind might aid a decision of whether there is something here that needs to be fixed (like documentation, bugs, feature enhancements, etc).