Firewall log showing strange "pass" entry
-
I would pcap port 23 on WAN and see what wireshark thinks about it.
Yeah, that or 8291.
I updated to 2.4.2_1 after my last post and there are still plenty of blocked 21, 22, 23, and 8291 connections. I don't see those weird entries in my logs though, so it doesn't look like the packets are malformed.
-
Let's say I want to follow the advice and do some packet capture stuff, how would I do it?
I will try to read up and see if I can set it up, but given the previous log entry, and the new ones below,
I would appreciate some examples of what to use, either through the GUI or from the command line tailored to narrow down as much as possible.And , some quick updates.
Mar 26 19:01:31 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,240,22619,0,DF,6,tcp,36,91.109.193.174,
[MYIPADDRESS],54443,8291,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',One of the previous "offender" was blocked with my new rule 58 (a new rule specifically created only to block everything coming from those previously identified "short,pass" IP addresses), I don't know if this yields any additional information or new information I can use to help packet capture beside to try to packet filter on port 23 or on port 8291.
Mar 27 00:30:10 pfSense filterlog: 58,,,1522081663,re0,match,block,in,4,0x0,,239,43226,0,DF,6,tcp,40,109.73.179.128,[MYIPADDRESS],5233,23,0,S,781346768,,16060,,
Mar 27 00:52:35 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,239,27500,0,DF,6,tcp,36,105.212.92.243,[MYIPADDRESS],24625,8291,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
Until I receive replies and guidance, I'm off to read about how to do packet filtering and save the results to disk.
-
Diagnostics > Packet capture
Install WireShark on your desktop to read the PCAP file that is generated/downloaded.
Now I'm curious about why the log from your new rule is showing no strange stuff.
-
Your always going to see a shit ton of noise.. Part of the reason I turned off logging default rule and just put in my own rule to log SYN only… I don't need to see all the udp noise, etc. Or anything that is out of state, etc. But I do like to keep an eye on the top ports... Yeah 23, 22 all going to be heavy hitters..
Do you have any odd ball forwards or floating rules? There are things that are looked at before firewall rules, etc..
https://doc.pfsense.org/index.php/Firewall_Rule_Processing_Order
If your really curious I would post up your full rule set.
https://doc.pfsense.org/index.php/How_can_I_see_the_full_PF_rulesetOr if you not open to posting that public.. Sure there are a few of here that that can be trusted you could PM it too.. If you don't feel comfortable with what might be listed in it.. We could then clean it up with any sort of public IPs and repost it for the board to look over, etc.
Part of the reason posting is want to get informed on updates to this thread. Off the top I have nothing.. If your saying you searched your rules for that ID and not seeing it.. at a loss..
How about atleast a screen shot of your wan, floating and forward rules.. To go along with your firewall rules if you don't feel like posting the full set..
-
There's nothing too exciting there I'm afraid.
screenshots of floating rules and wan rules as example, my floating rules goes a lot further below, mostly composed of ASN alias that you can compile checking the IBMcloud.com page doing query of the individual ASN and IP ranges (which I slightly massage, sort into an individual ASN Alias. Rule description basically is "ASN#-Nation-B(lock)-Start_BLOCKDATE.
the "shortpassblock" is currently de-activated so I can try to packet capture some data. and before you ask about 2 copies of say FB rules, I alternate them like two light switches (turn 2nd FB rule on, activate), then turn off 1st FB rule, then reload filter rules, to basically reset packet count.
There is no port forwarding, only pfB DNSBL for malicious ad blocking etc.
Last example shows a mouseover over an ASN alias, shows in the first IP description that shows the "Offender IP" that caused the ASN to be blocked.
-
Sorry, uploaded the wrong image. Here is the Floating rule one.
-
[Edit]: Please see follow-up post #24
[edit] … Learning how to packet capture deleted ...
Below is the "new firewall log entry"
Mar 28 01:11:32 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,242,25830,0,DF,6,tcp,36,168.228.167.124,[MYIPADDRESS],32068,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
-
[Edit] Removed trial packet capture info .. Please see captured Wireshark info a few posts down.
-
Dude just download the packet capture from pfsense and open it with wireshark or post up the pcap file you download.. What you have posted is just gibberish not going to help anyone figure out anything.
You can run packet capture without being logged in… just come back to it latter and stop it, etc.
-
Hi. I am posting a small progress note.
After some trial and errors, and some hit & misses, I think I have figured out a way to finally grabbing the packets to be examined, I will try to write them down so maybe if people have trouble like mine they will have something to refer to, so they can do the same.
1. I downloaded and installed latest stable version of Wireshark (for Win 7)
2. Learned how to copy a file remotely from pfSense box to local box, so I can move the packetcapture.cap to be examined by Wireshark.
https://doc.pfsense.org/index.php/HOWTO:_Access_pfSense_filesystems_remotely_with_scp
3. Learning to use tcpdump to capture the packets from pfSense box.
the most trouble for me initially was there was no predictability where the packet would be coming from, and I created a bunch of ssh windows each have:/usr/sbin/tcpdump -i re0 -p -c 100 -s 0 -w /tmp/105-212-92-243.cap ip and tcp and port 8291 and host 105.212.92.243
using previous offender IPs, and I would see no traffic from it, but a new one would come in like:
Mar 29 21:07:56 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,56,29244,0,none,6,tcp,36,155.133.83.54,[myIPAddress],40011,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
so, I finally settled on using:
/usr/sbin/tcpdump -i re0 -p -s 0 -w /tmp/port8291packetcapture.cap ip and tcp and port 8291 or 23 or 22
and was able to successful moved a copy of packet capture to examine on Wireshark.
Now that I have cast my net correctly, now I just have to wait.
With the example sample Wireshark screenshot, Once I grab the appropriate packets, what do I need to share here and How would I do it? I think I tried to upload the previous "packetcapture.cap" file but the forum would not accept it.
if you look at the screengrab posted here, what would be the relevant information to post in the next followup, once I have the data, or how would I process the data and post follow up to it?
[Edit] Small correction editing out my IP address on attached picture. The attached picture was to confirm that packets on Ports (22,23,8291) was being captured, and it seems to be working. Now I can wait for the pfSense filter log to show me another packet came in then I can correlate and post follow-up.
-
First thing to do is get a packet matched up with a log entry.
-
Got it !!!
I have absolutely no idea what this means… but here it is. Please let me know if this helps. I have the file on hand if someone needs it.
Mar 29 23:40:23 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,231,17518,0,DF,6,tcp,36,185.51.112.191,[myIPADDRESS],13794,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
-
WireShark also sees the packet as malformed and pfSense is reporting that: '[bad hdr length 20 - too long, > 16]'
When logging this same sort of traffic, I haven't seen any malformed packets at all. Given that you're not seeing any I/O errors on that interface, it's seems to me that something upstream is corrupting the packets.
You are also getting that weird rule number and the "short,pass" reason and action. That's probably just a result of the fact that the packet is corrupt but it would be good to know.
The MAC address translation in the second WireShark window says that packet came from a Cisco (maybe Linksys) device. Do you have something from one of those manufacturers in front of your pfSense box? If so, it would be interesting to see whether the other poster with this "bad hdr" error has one of those as well. Although, he wasn't hitting the "short,pass" or rule number problems.
It really doesn't look like anything is getting in to your network, despite that "pass".
It might be worth backing up your config, install a fresh copy of 2.4.3 and restoring the config to that.
-
I am feeling better today after I read your reply, because while I felt there was no unauthorized traffic coming from WAN, I could not see the underlying conditions nor the data causing the erroneous log entry, and the pfSense logger was causing me anxiety. The mysterious "rule set" number which I knew didn't exist did not help in the matter, either.
To address the "MAC address translation" question, I generate a new random MAC address and use it to "spoof" in the WAN interface periodically to obtain a different IP address once in a while from my ISP. My thinking is that if I got a new IP address, it will make me a more elusive hacking target than if I stayed at one IP address 100% of the time for years on end. I think that the "source" is the "real" MAC address of the WAN interface of my pfSense box. Would this thinking be correct?
I was thinking of updating to pfSense v2.4.3 yesterday, but, I didn't know my issue was relating to these 2 topics listed in the new release notes:
Added firewall rule tracking ID to rule list (in counter tooltip) and firewall rule edit page #8348
Fixed cases where automatic or scripted rules were not getting tracking IDs #8353Now that I have more clarity, I went ahead and did a upgrade this morning (option #13 - upgrade from console), and I will keep an eye out on the pfSense logger.
If the upgrade path doesn't seem to resolve the issue, I may do a clean install in the future. Meanwhile, I'll keep an eye out on the pfSense logger and try to continue to packet capture on those 3 ports.
If anyone have more guidance based on reading the Wireshark data, please let me know.
-
"I generate a new random MAC address and use it to "spoof" in the WAN interface periodically to obtain a different IP address once in a while from my ISP."
WTF??? Really??? Stop smoking whatever it is your smoking.. Nobody gives 2 shits what your IP is… Your not on an episode of Mr. Robot...
As to pulling a sniff from pfsense - all you need to do is click the freaking download button for gosh sake... Either open it directly or save it and open it later, etc..
Why don't you actually post the capture? Vs some screen shot? If your worried about your ip you an always obfuscate that
-
Sounds like you're pretty clicky-clicky and have probably shot yourself in the foot somehow.
-
It seems that even though I had upgraded now to the latest version of pfSense software (2.4.3-RELEASE (amd64)), the error (see below) persists.
Sorry. until 2 days ago, I had never used Wireshark, nor did I really needed to know scp, packet capture technique, and forensic packet analysis.
That said, in the spirit of trying to move forward to increase understanding – and hopefully increase security for all using pfSense,
Mar 30 13:22:20 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x8,,240,49160,0,DF,6,tcp,36,105.212.95.78,24.115.69.86,31586,23,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
The two enclosed .pcap files (which I just learned to mark and export into .pcap) are attached. I thought that by posting the screen grab late last night it would be enough for people to look and infer what the problem was.
Nevertheless, here are the two files you asked for. The "marked-033018.pcap" goes with the filterlog in this message; and "329packetcapture.pcap" was the one mentioned in follow-up reply #24.
I look forward to reading your analysis.
Enjoy.
[edit] fixed small spelling error. -
Good that there are no I/O errors but it's unlikely that your log entries reflect what's really happening.
Maybe the only way to find out is to packet capture everything on the re0 interface until you see another of those log entries.
If the packets are bad there probably isn't anything you could reliably use to filter the capture, so the output could be quite large.
EDIT: This is probably the source of this crap. Three out of five of the IP addresses you mention above come back with a Mikrotik login prompt.
Very interesting read with the Mikrotik you found biggsy.
Makes a lot of sense too. Capture file has Iran, Brazil, Cambodia, Albania, China, Sweden, Korea, Japan, Turkey, South Africa, wew!
Ports 22, 23, 8291. All trying to start handshake sending SYN packets within minutes. I did not see any SYN ACK going out so I guess PfSense is just humming along in the storm, 8)
vt44 did you read it. You may have missed it in the rush. By any chance you running a mikrotik that let out a call for a bot party at your place. Probably not but I would get rid of the redundant blocking and simplify things. Looks confusing to me.
Just a thought. -
I'm running a ZOTAC ZBOX nano CI323 with Intel N3150 chip with only pfSense software, so I'm definitely not using Mikrotik. It looks like the Mikrotik party has moved to Poland & Ukaine and added port 2000, too? I did not have these two packets (nor port 2000) in particular captured, but will modify my filter if to grab the future data if anyone is interested in seeing the pcap file.
Mar 30 23:09:59 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,244,33870,0,DF,6,tcp,36,176.110.150.46,[myipaddress],26210,2000,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',
Mar 31 05:05:55 pfSense filterlog: 4294967295,,,0,re0,short,pass,in,4,0x0,,243,59793,0,DF,6,tcp,36,195.214.197.90,[myipaddress],32909,2000,-4,S,errormsg='[bad hdr length 20 - too long, > 16]',–-
On the topic of routers, I was actually using Google OnHub in 2016 before before I decided to "roll my own router" after reading the Ars Technica's article about it, and the overwhelming reason for the change was not because the OnHub was not a good router, but I wanted more transparency. The Onhub was changed to bridge mode and is working as a dedicated wireless AP, but still is a candidate as my back-up router should it becomes necessary.
Was Mr Derelict addressing this sentence to me? If he could clarify what he was asking I would appreciate it.
"Sounds like you're pretty clicky-clicky and have probably shot yourself in the foot somehow."
I did want to mention that while the pfSense GUI packet capture will allow multiple IP addresses to be specified, it does not allow for the capture of multiple ports at the same time (I tried the "22 23") -- and that was why at the end I could not use "clicky-click" interface to do packet capture and had to rely on "tcpdump" from the command line in the end. And, since there was no "click to download capture", I had to figure out to learn use "scp" to moves the packet capture files over.
-
Pretty much everyone on the net is seeing traffic from these bots.. But you seem to be the only one with the odd pass rule. Most likely due to some oddball setup you have done.
I am seeing hundreds of blocks to 8291 port.. And one to 23..
Here is one with the odd headers
Mar 29 17:04:02 filterlog: 131,,,1512833215,igb1,match,block,in,4,0x0,,229,24172,0,DF,6,tcp,40,105.212.82.37,64.53.xx.xx,30634,8291,-4,S,errormsg='[bad hdr length 24 - too long, > 20]',
Mar 30 20:24:02 filterlog: 131,,,1512833215,igb1,match,block,in,4,0x0,,238,58316,0,DF,6,tcp,40,198.162.199.56,64.53.xx.xx,2514,23,-4,S,errormsg='[bad hdr length 24 - too long, > 20]',
Why don't you actually post your full rules so we can see what your doing that is odd..
https://doc.pfsense.org/index.php/How_can_I_see_the_full_PF_ruleset
