Firewall log showing strange "pass" entry
-
Hmmm . . . Telnet (TCP/23) from Africa (105.212.87.78).
How did you determine that there is no Rule 4294967295?
-
Perhaps I was incorrect in understanding how rules are numbered in pfsense, I think I do have about 230 rules total, including pfSense system default rules, and addition of user rules – and that may have led me to incorrect to assume there was no rule 4294967295 -- perhaps it could be found with the forum's assistance.
In my past experience reading of the pfsense firewall logs, I had seen plenty of "match,block" on both (WAN/LAN) interfaces, but this was the first one I had seen that came from the WAN that said "short,pass" and I'm just trying to make sense of it and understand it better.
Mar 25 14:42:05 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,239,10359,0,DF,6,tcp,36,105.212.87.78,[MYIPADDRESS],2123,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
I know pfsense firewall defaults to blocking all incoming firewall port scanniing request, and I do add one rule to block port 1-65535 on the WAN firewall IP (I know it is redundant) to just count how much background scanning traffic (packets) that are just always there. I want to emphasize that I do not have any open ports exposed to the Internet, including the port 23 the offender was trying to access.
I stand corrected on possibility of the existence the rule 4294967295, but I don't have user firewall rules that just says pass (external IP) to LAN ok because I know that's a no-no.
– and I'm running 2.4.2-Release-p1 amd64 / FreeBSD 11.1-RELEASE-p6) I think that's the latest version for everything
-
What does running this show:
pfctl -vvsr | grep -A3 4294967295
-
I have rebooted the firewall after adding an alias for ASN block for AS16637 (102 IP ranges) and a corresponding floating blocking rule with that Alias.
https://exchange.xforce.ibmcloud.com/search/AS16637
A few hours ago, someone suggested I try :
pfctl -vvsr | grep "short"
and see what was matched, but nothing came of it.
and I know you already know the response for the following
pfctl -vvsr | grep -A3 4294967295
was just empty response as well (since I rebooted). But, I think I have learned something from this conversation, and try to record responses of
pfctl -vvsr | grep -A3 RULE_NUMBER_AS_DISPLAYED_FROM_FIREWALL_LOG
&
pfctl -vvsr | grep RULE_NUMBER_AS_DISPLAYED_FROM_FIREWALL_LOG
and have that handy if it ever happens again.
Thank you for responding and for helping identifying additional steps I need to take to gather needed info to ask better questions.
-
… What do you know..
Mar 25 21:02:12 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x28,,242,20338,0,DF,6,tcp,36,45.6.195.22,70.44.54.50,55118,8291,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
and…
[2.4.2-RELEASE][admin@pfSense.localdomain]/tmp: pfctl -vvsr | grep -A3 4294967295
[2.4.2-RELEASE][admin@pfSense.localdomain]/tmp: -
It just occurred to me that 4294967295 is one less than 232 (4294967296).
I don't know what the significance of that might be but it is strange. Perhaps an overflow of some sort.
-
Yeah I am not sure that is actually indicating passed traffic at all. I don't know that I have ever seen a log message with an error code like that.
-
Maybe the re0 interface (or whatever it's connected to) is misbehaving.
Any I/O errors on Status > Interfaces?
-
Interesting theory.. I went and checked and it's been 23 hours since the reboot – and the Status for WAN says"
WAN Interface (wan, re0)
Media 1000baseT <full-duplex,master>In/out packets 2461287/1308634 (2.76 GiB/106.79 MiB)
In/out packets (pass) 2461287/1308634 (2.76 GiB/106.79 MiB)
In/out packets (block) 22309/5486 (12.81 MiB/391 KiB)
In/out errors 0/0
Collisions 0LAN Interface is solid (re1) with no errors nor collisions
Since the reboot, there has been 5 additional instances recorded in the firewall log
Mar 26 08:25:40 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,239,28320,0,DF,6,tcp,36,109.73.184.113,[MYIPADDRESS],60964,8291,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
Mar 26 11:19:30 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,241,37101,0,DF,6,tcp,36,168.228.165.29,[MYIPADDRESS],1454,8291,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
Mar 26 12:33:37 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,243,19749,0,DF,6,tcp,36,88.220.191.225,[MYIPADDRESS],29755,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
Mar 26 15:07:07 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,239,2173,0,DF,6,tcp,36,109.73.184.235,[MYIPADDRESS],20600,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
Mar 26 16:23:28 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,242,54321,0,none,6,tcp,36,61.140.124.185,[MYIPADDRESS],45174,22,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
and, I had tried before to try to correlate but there is no "rule" found from:
pfctl -vvsr | grep -A3 4294967295
I do have and use Alias(es) for the purpose of ASN based blocking, for example, Facebook ASN blocking
whois -h whois.radb.net – '-i origin AS32934' | grep ^route | grep -v route6 | cut -d" " -f7 > ./fbblock.txt
I primarily build aliases to block the "ASN of the "usual suspects of persistent hacking/probing" to cut down on the firewall log "noise", as I'm unlikely to have the need to visit networks of or websites located in ... say.. AS9808 or AS4837
I don't know if there are other debugging options that can be turned on to track this, but all the monitoring indicates no "States" or connections are being successfully established by these incoming attempts, as checked by Diagnostics>States>States and filtered by IP of offender on "all" interface.
Thanks for the responding with ideas to check, but as of now, I still have not found the culprit of the strange error message, so I am proceeding cautiously.</full-duplex,master>
-
Good that there are no I/O errors but it's unlikely that your log entries reflect what's really happening.
Maybe the only way to find out is to packet capture everything on the re0 interface until you see another of those log entries.
If the packets are bad there probably isn't anything you could reliably use to filter the capture, so the output could be quite large.
EDIT: This is probably the source of this crap. Three out of five of the IP addresses you mention above come back with a Mikrotik login prompt.
-
Of course, this doesn't solve the riddle of the rule number being 232-1 or the "pass" entry.
I turned on logging for the default block rules and saw more than 100 attempts to connect to ports 21, 22, 23, 8291 in the space of two hours. However, none of my logs had those strange entries shown in yours : oddball rule number, "short", "pass" or "bad hdr length…"
I'm still on 2.3.5 :-[ Perhaps there was a regression in log parsing after that.
-
I would pcap port 23 on WAN and see what wireshark thinks about it.
-
I would pcap port 23 on WAN and see what wireshark thinks about it.
Yeah, that or 8291.
I updated to 2.4.2_1 after my last post and there are still plenty of blocked 21, 22, 23, and 8291 connections. I don't see those weird entries in my logs though, so it doesn't look like the packets are malformed.
-
Let's say I want to follow the advice and do some packet capture stuff, how would I do it?
I will try to read up and see if I can set it up, but given the previous log entry, and the new ones below,
I would appreciate some examples of what to use, either through the GUI or from the command line tailored to narrow down as much as possible.And , some quick updates.
Mar 26 19:01:31 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,240,22619,0,DF,6,tcp,36,91.109.193.174,
[MYIPADDRESS],54443,8291,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',One of the previous "offender" was blocked with my new rule 58 (a new rule specifically created only to block everything coming from those previously identified "short,pass" IP addresses), I don't know if this yields any additional information or new information I can use to help packet capture beside to try to packet filter on port 23 or on port 8291.
Mar 27 00:30:10 pfSense filterlog: 58,,,1522081663,re0,match,block,in,4,0x0,,239,43226,0,DF,6,tcp,40,109.73.179.128,[MYIPADDRESS],5233,23,0,S,781346768,,16060,,
Mar 27 00:52:35 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,239,27500,0,DF,6,tcp,36,105.212.92.243,[MYIPADDRESS],24625,8291,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
Until I receive replies and guidance, I'm off to read about how to do packet filtering and save the results to disk.
-
Diagnostics > Packet capture
Install WireShark on your desktop to read the PCAP file that is generated/downloaded.
Now I'm curious about why the log from your new rule is showing no strange stuff.
-
Your always going to see a shit ton of noise.. Part of the reason I turned off logging default rule and just put in my own rule to log SYN only… I don't need to see all the udp noise, etc. Or anything that is out of state, etc. But I do like to keep an eye on the top ports... Yeah 23, 22 all going to be heavy hitters..
Do you have any odd ball forwards or floating rules? There are things that are looked at before firewall rules, etc..
https://doc.pfsense.org/index.php/Firewall_Rule_Processing_Order
If your really curious I would post up your full rule set.
https://doc.pfsense.org/index.php/How_can_I_see_the_full_PF_rulesetOr if you not open to posting that public.. Sure there are a few of here that that can be trusted you could PM it too.. If you don't feel comfortable with what might be listed in it.. We could then clean it up with any sort of public IPs and repost it for the board to look over, etc.
Part of the reason posting is want to get informed on updates to this thread. Off the top I have nothing.. If your saying you searched your rules for that ID and not seeing it.. at a loss..
How about atleast a screen shot of your wan, floating and forward rules.. To go along with your firewall rules if you don't feel like posting the full set..
-
There's nothing too exciting there I'm afraid.
screenshots of floating rules and wan rules as example, my floating rules goes a lot further below, mostly composed of ASN alias that you can compile checking the IBMcloud.com page doing query of the individual ASN and IP ranges (which I slightly massage, sort into an individual ASN Alias. Rule description basically is "ASN#-Nation-B(lock)-Start_BLOCKDATE.
the "shortpassblock" is currently de-activated so I can try to packet capture some data. and before you ask about 2 copies of say FB rules, I alternate them like two light switches (turn 2nd FB rule on, activate), then turn off 1st FB rule, then reload filter rules, to basically reset packet count.
There is no port forwarding, only pfB DNSBL for malicious ad blocking etc.
Last example shows a mouseover over an ASN alias, shows in the first IP description that shows the "Offender IP" that caused the ASN to be blocked.
-
Sorry, uploaded the wrong image. Here is the Floating rule one.
-
[Edit]: Please see follow-up post #24
[edit] … Learning how to packet capture deleted ...
Below is the "new firewall log entry"
Mar 28 01:11:32 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,242,25830,0,DF,6,tcp,36,168.228.167.124,[MYIPADDRESS],32068,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
-
[Edit] Removed trial packet capture info .. Please see captured Wireshark info a few posts down.
