Midstream Packets Being Dropped During Unencrypted File Transfer [AFP/SMB/Raw NC]

CoderChris

I have a Netgate 2100 box running 22.05-RELEASE. I have had the box in production since the beginning of the year with nearly zero issues.

A few days ago a user reported that she was having issues transferring a PDF file from her Mac computer to our Windows File Server. She reported and I confirmed that this one particular file was causing the issue. She was using the AFP file protocol, so I tried swapping her to the SMB protocol to the same results. Then I tried using a 'raw' transfer with Netcat between a Mac and Linux VM I run on the same file server with the same result. It should be noted at this point that the File Server runs Sophos Endpoint Server Protection, but the VM does not.

Additionally, her Mac is at one site and the file server is at another site. The two sites are tied together with IPSEC site-to-site VPN. The firewall rules between the two site's subnets are set to be very permissive and we have not had any issues with the network in the past month that this has been setup.

With more testing I found that if I encrypted the file via openSSL or just compressed it would transfer successfully. Since the Mac was at one site and the Windows File Server was at a second site I copied the file to an independent VPS server I own and copied it in via Netcat to the LinuxVM again, but again with the same failure.This in effect cuts out the site-to-site component, but as you can see I get the same result.

For clarity the Network path from the external VPS (test case that fails) to the file server is: VPS -> INTERNET -> Spectrum Fiber Modem [no Spectrum router bridge, connects directly to Netgate] -> Netgate 2100 -> Ubiquiti ES -> Cisco Switch -> VM Server.

Then I setup wireshark on my File Server and on her Mac and watched the SMB traffic. I found that the file always failed when the exact same binary fragment was sent:

[Dump from hexdump of relevant section in PDF file]

0140f30 c361 f67c 3f39 7982 73ec b38f e67e 7b45
0140f40 b32b 92c9 cade f26c 7f64 3eb3 3be7 1df0
0140f50 4e9e e157 3c7b 6d9f 36dd de3d 63eb 7ec9
0140f60 5b7b 3c79 df6e 936f edc4 b9ef 3e3c 78df
0140f70 f1e4 f7f6 6ba7 ee54 7bf6 37d7 d891 1cef
0140f80 66de 7bef e6cd 3b76 de7c 6757 efb4 5d76
0140f90 9bcd a71f 4a5a 4a4a 4a4a 4a4a 4a4a 4a4a <-Right Here
0140fa0 4a4a 4a4a 4a4a 4a4a 4a4a 4a4a 4a4a 4a4a
*
01410f0 4a4a 4a4a 4a4a 4a4a 4a4a 1f5d bb8d 3c39
0141100 db6e 77d6 d78e cd27 ed77 27ef f1c1 f15e
0141110 9bcb f889 7cef cde5 dec1 71d2 5c0e fbd8
0141120 3fed f312 63f1 b4c7 4bdc c5cf 3ec0 f1d0
0141130 7c1e 5cfc c62d efbb 6e3e e230 ea7d bbad
0141140 175f 37df dfdc be3f bded 6f77 cdb5 dbd8
0141150 dbb5 ff01 9fd6 73f7 f2b0 4f77 dd71 615f
0141160 7cc3 39f6 823f ec79 8f73 5eb3 197b ec17
0141170 b9af de92 7bca 2399 c7ec ceb9 07fc e5dc
0141180 176e fbe4 e96e 93b7 6edd bbf6 7bbb 1b4f
0141190 bc79 f5e4 eddb 00ff aa27 77df 4fae bd53
01411a0 dcef bf9e c4b7 ab1d bd8b e2d8 de1c 73c3
01411b0 79df b6bd bde7 7b73 ce5d 00ff d99b f76e
01411c0 de9c 636f efb4 5df6 cbce 00f4 0000 0000
01411d0 0000 0000 0000 0000 0000 0000 0000 0000
*
0141220 0000 0000 3d3f 6ba7 b5b5 efe1 f276 6e7a
0141230 dbf5 f9e4 230e e5be c6cd 67dc 72bd e1e6

From my limited knowledge of binary exploitation and buffer overflows, I know that attackers will try and use a know set of characters such as repeating 'A's in recognizance to flush out memory overwrites. So maybe this reads as something like that, but I do not have enough knowledge on the topic to do more than guess.

Lastly, I brought my laptop to the same site as the VM/File Server and performed some tests where I copied the file via SMB3 and NC and both worked successfully. The network path of the laptop to the file server is: Ubiquiti AP -> Ubiquiti ES -> Cisco Switch -> Server; I do not think it even touches the Netgate Firewall that is on the edge of the LAN facing the internet.

My working theory is that something about this file, likely the particular section listed above, is tripping a DPI trigger for a potential attack. Though the rub and where the case has gone cold is currently I am not running any kind of applications on my Netgate 2100 that to my knowledge do DPI. For reference here are all the packages on my Firewall:

From my knowledge neither of these do any form of DPI that could block this kind of traffic nor am I aware of any pfSense native features that do this kind of filtering. I am aware of packages such as SNORT and Suricada that could do this, but they are obviously not installed.

Also I have passed the file through virustotal to make sure there isn't anything obviously suspect about it, but no dice:

[https://www.virustotal.com/gui/file/03e6150c0c3af8416b74cd3b9a1c08b3d2cc396e4201c62475798b33b2a2fac4](link url)

Lastly, I have checked firewall logs for any obviously blocked traffic coming from the IPs in question. I double checked that my block rules have logging turned on and I have found no indications in the system logs of traffic being blocked through normal routing rules.

At this point I would appreciate any insights from anyone more knowledgeable than me for where to look next for what is causing this blockage. Additional points for if anyone can actually analyze the binary data determine if it is actually malicious, though I have strong doubts that it is.

Thanks!
Chris C.

stephenw10

Hmm, well that's an odd situation!

So to be clear in the last test case how was the traffic passing through the 2100? Like a port forward to the Linux VM? No VPN involved?

Does it fail if you just send a small section of that file containing the failure code?
If so perhaps we can get a copy of it to try to replicate it.

The next thing I would try to do is replicate it locally between two interfaces on the 2100. Between two VLANs for example.

You might also try uploading it directly to the 2100 or downloading it.

Since it doesn't seem to matter whether the incoming interface is a VPN or not I'd suggest it's more likely failing as it leaves onto the LAN unencrypted.

Steve

CoderChris

@stephenw10

Heh at least I am not only one that think it is odd.

Yes, clarifying when I do the VPS -> VM transfer I port forward port 8080 on my firewall to the open internet then use netcat to raw transfer the bytes from the VPS to the VM via the port forward.

I appreciate the new angles of attack, when I am at work tomorrow I will try these and report back.