pfSense Plus block file upload

Lucas Rey

Hello community,
I recently upgrade to pfSense plus, and everything seemed to be fine. After a few days I notice a strange behaviour, pfSense doesn't allow me to upload file to service e.g. mega.io or 1fichier.com, etc. Simply it reset (block) the connection, while I'm able to download from these hosts without any issue.

Since I'm using these services to exchange big files with colleagues, I rollback to pfSense C.E. and everything worked again. Plus and C.E. have absolutely the same config!

So it seems there is something that block, in pfSense+, the file upload, but I'm not able to identify any rule or service that are blocking such kind of traffic.

This is a very strange bahviour, and it drove me crazy until I rollback. Here below an example of the behaviour.
rar files doesn't work, while zip files works perfect and upload start without issue, it's crazy, isn't it?
alt text

I would like to use pfSense+ but I have to solve this issue, any clue?

Thank you
Lucas

johnpoz

@lucas-rey So how exactly is pfsense blocking your traffic inside a https tunnel I would hope but only rar files not zip.. That is crazy for sure because there just isn't a way for pfsense to have a clue to the file type. Or even what your doing inside a https tunnel.. I moves the packets.. It allows the ports..

Not even the ips could see the difference between a zip and a rar..

Lucas Rey

Well, I don't think the issue is related to extension, I tried with other zip file with the same result (no upload).
I really don't understand how pfSense+ can block upload, while pfSense CE have no issue.

And it's confirmed that is pfSense the root cause, since I also tried to bypass the firewall connecting my PCs directly to the ISP Modem, and everything works as expected.

Maybe some new security service on pfSense+ ?

johnpoz

@lucas-rey said in pfSense Plus block file upload:

Maybe some new security service on pfSense+ ?

No.. How exactly is a L3 firewall that filters on port and IPs involved in a conversation suppose to look inside your encrypted https tunnel and see oh we don't allow rar files.. There is just no way.. Sorry.. Just doesn't work that way..

Would be like a mailman saying sorry can not deliver this box for you, when the box is the same size, same weight, same color, with the same to and from address on them. But somehow looks inside the boxes and says oh sorry you have burrito in this one vs a taco - sorry I can't deliver that.

Lucas Rey

I understood your point of view, but let's forget now zip vs rar.
The fact here is that pfSense+ doesn't allow me to upload file.

As I wrote, bypassing firewall everything works, and also coming back to pfSense CE I can upload file again. Plus and CE have exactly the same config, so there is something that block upload on Plus, that's it.

johnpoz

@lucas-rey said in pfSense Plus block file upload:

but let's forget now zip vs rar.

And how are we suppose to do that - that is what you stated is happening.. But there is no way for pfsense to distinguish such a difference..

So how could pfsense be the issue - you need to look elsewhere to what your problem is.. Not saying your not having an issue, but you thinking it could be pfsense version that somehow can tell the difference in what your doing inside a https tunnel has lead you down the wrong path in your troubleshooting..

Here is a fix for your problem - don't upload rar files ;) You say zip works, so use zip then..

So take your testupload file and just rename it to .zip vs .rar - it now works? But somehow pfsense is stopping it when its named .rar?

Lucas Rey

@johnpoz Again, please forget rar and zip, if I rename rar to zip, the upload still doesn't start. So the issue is not in filename. In my first post I only did an example, saying that "BIGIP-16.1.3.3-0.0.3.LTM.qcow2.zip" seems works without apparent reason.

As I wrote multiple times, the root cause is for sure pfSense+, if I'll don't find the reason I can stay with pfSense CE that works perfect, anyway, I would like to use the plus version.

johnpoz

@lucas-rey here you go I just uploaded a rar file to that site you said pfsense was blocking your uploads too..

Not having any issue.. On 23.01

edit: So it seems that when sending a file, the url changes and can be different

click once and

https://ru-3.site.com

do it another time and now

https://up2.site.com/

Maybe your blocking one of their urls, or having a hard time connecting to one of those, or maybe one of their sites is having issues.. etc..

But pfsense 2.6 vs 23.01 wouldn't change anything in your network connection. Possible your getting a different IP from your ISP, like when you said you connected to your modem, etc.. But that sort of issue connecting to a specific different IP or url you have to resolve could be problematic and source of your issue - but that again is not a 23.01 vs a 2.6 thing..

Gertjan

@lucas-rey

Just uploaded a 450 Mbytes file to my drive storage @Google.
Using 23.01 on a SG4100.

Btw : everything is a file : a web page you look at, the mail you send to some mail server, the content that you upload to your wordpress site, the movies you share with utorrent, whatever.

Not being able to "upload", afaik, it has been seen before. Like MTU issues, asymmetric routing, etc.

The good news is : we both use 23.01, so it's not the pfSense code, as it is byte by byte identical.
Our settings are not.
If pfSense had an issue with 'uploading', this forum would explode with over 100 000 pfSense complaining users right now.
Entire companies would come to a stand still.
That would not have been happening unnoticed.

SteveITS

@lucas-rey There are only a few ways pfSense can block anything such as firewall rule, DNS, pfBlocker, or IDS/Snort. Per your post you haven’t added firewall rules.

Is DNS working for that site at the time? In 23.01 there are several posts about DNS problems. If you are forwarding disable DNSSEC.

Are you using any packages?

Lucas Rey

Wait! I'm not saying that pfSense+ has an issue, I'm trying to explain that MY pfSense has an issue probably due to a wrong setting, and I'm kindly asking a clue where the problem could be.

What I'm not explain is that between pfSense CE and pfSense Plus, the configuration is absolutely the same, so why pfSense plus doesn't work? I'll try to de-activate selectively each services currently active on pfSense plus to try to discover where the issue is. Maybe in the proxy? ClamAV block such domains? I don't know. The fact is that MINE pfSense Plus doesn't work while pfSense 2.6 CE works perfect!

johnpoz

@gertjan said in pfSense Plus block file upload:

Just uploaded a 450 Mbytes file to my drive storage

But was it a rar file? ;) heheheh

Lucas Rey

@johnpoz said in pfSense Plus block file upload:

But was it a rar file? ;) heheheh

There is no need to be sarcastic, and defend pfSense software as if it were your personal product. I wrote in this community because I have a problem, and I was hoping someone can suggest a tips or a clue. I never said/wrote that pfSense software itself has an issue.

However, I finally identified where is the issue. It's the squid proxy server. If I disable it, the upload works without issue.
That's strange because I have it also on pfSense 2.6, while with 23.01 I got the upload issue.
Now the hard thing is to discover where is the problem since there are tons of setting there.

johnpoz

@lucas-rey said in pfSense Plus block file upload:

I never said/wrote that pfSense software itself has an issue.

How is that?

@lucas-rey said in pfSense Plus block file upload:

And it's confirmed that is pfSense the root cause

Glad you found your problem - maybe its just me, but first step in troubleshooting would be to disable any sort of packages your running like ips or proxy.. And you didn't even mention this.. Just stating that pfsense+ is the problem..

Lucas Rey

@johnpoz said in pfSense Plus block file upload:

Just stating that pfsense+ is the problem..

Sure, in my network, the upload problem is given by pfSense. I never wrote that pfSense software is broken, but that MY pfSense have something wrong, it is better that way?

skogs

~Generally speaking~ ... unless you have an exceptionally special use case ... get rid of the proxy and clamav.

Your internet is fast enough you don't need a proxy.
Most malware lazy enough to be sent in the clear and let clamav actually look at it isn't much of a threat anyway.

Encryption is mostly standard now and pretty much makes both of these products useless. Hence the previous discussion about how it is impossible for pfsense to see inside the encrypted tunnel.

I'm cynical ... I say 80% chance the files got blocked by the clamav because loaded with trojans. We got bigger problems than uploads not working. :)

NollipfSense

@lucas-rey said in pfSense Plus block file upload:

the upload problem is given by pfSense. I never wrote that pfSense software is broke

Re-read you first post and the above...you implied that it's pfSense. Glad you realize it wasn't.

Lucas Rey

@skogs said in pfSense Plus block file upload:

~Generally speaking~ ... unless you have an exceptionally special use case ... get rid of the proxy and clamav.
Your internet is fast enough you don't need a proxy.
Most malware lazy enough to be sent in the clear and let clamav actually look at it isn't much of a threat anyway.
Encryption is mostly standard now and pretty much makes both of these products useless. Hence the previous discussion about how it is impossible for pfsense to see inside the encrypted tunnel.
I'm cynical ... I say 80% chance the files got blocked by the clamav because loaded with trojans. We got bigger problems than uploads not working. :)

You are right, I always had a proxy in my network, but now that I have a fiber with 2,5Gbit throughput maybe is time to switch off the proxy :)

@nollipfsense said in pfSense Plus block file upload:

Re-read you first post and the above...you implied that it's pfSense. Glad you realize it wasn't.

Why isn't pfSense the issue? I removed the problem just disabling ClamAV, and antivirus is a pfSense module, isn't it? So, in my point of view, upload issue is caused by a specific pfSense configuration/module in my personal environment, and this after upgrading 2.6 to 23.01. Probably something goes wrong, and probably if I install a fresh pfSense 23.01 version everything will work, but as I said, after upgrade I got this issue, and luckily I had identified it in ClamAV. Why this happen, I don't know.

Gertjan

@lucas-rey said in pfSense Plus block file upload:

Why this happen, I don't know

Like pfBlockerng, ClamAV uses 'rules'.
Based on IP addresses, ports used, and whatever it can find in the Ethernet packet headers (all the bits, flag etc), it applies the 'rules' and then decides.
Like pfBlockerng, ClamAV does nothing by default.
Then the admin drops in, and start activating 'rule sets'. These rule sets are not made by 'Netgate'. Like the IP and DNSBL feeds of pfBlockerNG. You just have to 'trust' them.
It happens all the time : 'something' (a rule) actually blocks traffic that you do not want it to block.

I'm not a ClamAV user myself, but I'm pretty sur ClamAV logs every decision it takes. Like pfBlockerNG. Check these ClamAV logs, and you will know what rule did block you upload. Disable (or edit ?) the rule, and you'll be fine.
Tools like ClamAV needs to be checked all the time for false positives.
Anyway, glad you have the issue cleared

bmeeks

@lucas-rey said in pfSense Plus block file upload:

Why isn't pfSense the issue?

Veteran pfSense users tend to be a little sensitive to broad accusations made against the software without warrant. Here's what I mean ---

First, the official pfSense software is divorced from the packages. The available packages are, by and large, created and maintained by volunteer developers who have no association with Netgate and the pfSense team, and they are not paid for their efforts creating and maintaining a package. Over time these volunteer developers come and go. That can leave a given package orphaned with no developer support. In the case of a few more popular packages, the Netgate team might step in and provide some rudimentary support of those packages. But that is very rare.

It has become quite frequent for a user to start a thread (much like yours began) with a blanket statement saying "pfSense is blocking ..." without providing any additional details about which add-on packages are installed. pfSense itself, installed directly from a USB memstick image or ISO, and configured via the setup wizard is not going to block anything outbound. Providing the user only supplies the requested information (IP addresses for interfaces and assigning interfaces to LAN and WAN), then things will just 100% work. And they will continue to work through later upgrades.

The problems happen when users install add-on packages and/or start to monkey with default settings (DNS Resolver being a favorite place for folks to start tampering without a firm knowledge of what they are doing). This can definitely lead to problems, but the problems in this case are not "pfSense" so much as they are user-inflicted by the user not fully understanding what they are changing or by them installing an add-on package.

If you install packages that are designed to intercept and block things, then when something is blocked or stops working, the very first place you need to look for the problem is that add-on package you installed! Installed packages result in "non-default" installations. pfSense with installed packages is NOT the same as pfSense with no packages.

Here is an example. A user will install a package like pfBlockerNG. That package is designed from the get-go to block stuff using lists of IP addresses. So, soon after installing the package and configuring a bunch of "block lists", the user notices that a number of their favorite websites no longer work properly or won't load at all. The user creates a new topic here on the forums but titles it "pfSense is blocking some websites" and never mentions anywhere in the post they installed and configured the pfBlockerNG package. Veteran users know that generally speaking pfSense doesn't do that (block some websites while allowing others). But with the user supplying no helpful context (such as installed packages), then an argumentative game of whack-a-mole ensues as the folks trying to help have to guess what it might be or what the user has misconfigured.

So back to your case. If you had started your post by saying "I have pfSense version x.xx and I am running the following packages...", the initial responses you received would likely have been quite different. Why you ask? Because veteran users know that a vanilla pfSense install will not just block some particular file type. It can't even do that if you want it to. That can only happen if some add-on package is installed that provides extra capability. By telling them upfront what add-on packages you have installed, they can better tailor their troubleshooting suggestions.

I know from attempting to support the packages I maintain, that it eventually gets somewhat tiring to have to drag information out of users piece-by-piece when trying to sort out a problem. You tend to get frustrated and a bit irritable. It is much easier when the pertinent information is divulged up front. So, in your case it appears the clamAV package was the culprit. But clamAV is not a native part of pfSense. A user must manually install and configure it. And remember packages are maintained by non-Netgate/pfSense developers, so support issues with version upgrades can most certainly appear. But those problems are not the fault of pfSense. They are a natural consequence of installing and using a third-party add-on package supported by a volunteer developer.

The users here will be glad to help you with packages they may be familiar with. There are also a few sub-forums dedicated to specific packages or package categories. Posting a question there can result in much faster help.