pfSense Plus block file upload
-
@lucas-rey said in pfSense Plus block file upload:
but let's forget now zip vs rar.
And how are we suppose to do that - that is what you stated is happening.. But there is no way for pfsense to distinguish such a difference..
So how could pfsense be the issue - you need to look elsewhere to what your problem is.. Not saying your not having an issue, but you thinking it could be pfsense version that somehow can tell the difference in what your doing inside a https tunnel has lead you down the wrong path in your troubleshooting..
Here is a fix for your problem - don't upload rar files ;) You say zip works, so use zip then..
So take your testupload file and just rename it to .zip vs .rar - it now works? But somehow pfsense is stopping it when its named .rar?
-
@johnpoz Again, please forget rar and zip, if I rename rar to zip, the upload still doesn't start. So the issue is not in filename. In my first post I only did an example, saying that "BIGIP-16.1.3.3-0.0.3.LTM.qcow2.zip" seems works without apparent reason.
As I wrote multiple times, the root cause is for sure pfSense+, if I'll don't find the reason I can stay with pfSense CE that works perfect, anyway, I would like to use the plus version.
-
@lucas-rey here you go I just uploaded a rar file to that site you said pfsense was blocking your uploads too..
Not having any issue.. On 23.01
edit: So it seems that when sending a file, the url changes and can be different
click once and
https://ru-3.site.com
do it another time and now
https://up2.site.com/
Maybe your blocking one of their urls, or having a hard time connecting to one of those, or maybe one of their sites is having issues.. etc..
But pfsense 2.6 vs 23.01 wouldn't change anything in your network connection. Possible your getting a different IP from your ISP, like when you said you connected to your modem, etc.. But that sort of issue connecting to a specific different IP or url you have to resolve could be problematic and source of your issue - but that again is not a 23.01 vs a 2.6 thing..
-
Just uploaded a 450 Mbytes file to my drive storage @Google.
Using 23.01 on a SG4100.Btw : everything is a file : a web page you look at, the mail you send to some mail server, the content that you upload to your wordpress site, the movies you share with utorrent, whatever.
Not being able to "upload", afaik, it has been seen before. Like MTU issues, asymmetric routing, etc.
The good news is : we both use 23.01, so it's not the pfSense code, as it is byte by byte identical.
Our settings are not.
If pfSense had an issue with 'uploading', this forum would explode with over 100 000 pfSense complaining users right now.
Entire companies would come to a stand still.
That would not have been happening unnoticed.No "help me" PM's please. Use the forum, the community will thank you.
Edit : and where are the logs ?? -
@lucas-rey There are only a few ways pfSense can block anything such as firewall rule, DNS, pfBlocker, or IDS/Snort. Per your post you haven’t added firewall rules.
Is DNS working for that site at the time? In 23.01 there are several posts about DNS problems. If you are forwarding disable DNSSEC.
Are you using any packages?
-
Wait! I'm not saying that pfSense+ has an issue, I'm trying to explain that MY pfSense has an issue probably due to a wrong setting, and I'm kindly asking a clue where the problem could be.
What I'm not explain is that between pfSense CE and pfSense Plus, the configuration is absolutely the same, so why pfSense plus doesn't work? I'll try to de-activate selectively each services currently active on pfSense plus to try to discover where the issue is. Maybe in the proxy? ClamAV block such domains? I don't know. The fact is that MINE pfSense Plus doesn't work while pfSense 2.6 CE works perfect!
-
@gertjan said in pfSense Plus block file upload:
Just uploaded a 450 Mbytes file to my drive storage
But was it a rar file? ;) heheheh
-
@johnpoz said in pfSense Plus block file upload:
But was it a rar file? ;) heheheh
There is no need to be sarcastic, and defend pfSense software as if it were your personal product. I wrote in this community because I have a problem, and I was hoping someone can suggest a tips or a clue. I never said/wrote that pfSense software itself has an issue.
However, I finally identified where is the issue. It's the squid proxy server. If I disable it, the upload works without issue.
That's strange because I have it also on pfSense 2.6, while with 23.01 I got the upload issue.
Now the hard thing is to discover where is the problem since there are tons of setting there. -
@lucas-rey said in pfSense Plus block file upload:
I never said/wrote that pfSense software itself has an issue.
How is that?
@lucas-rey said in pfSense Plus block file upload:
And it's confirmed that is pfSense the root cause
Glad you found your problem - maybe its just me, but first step in troubleshooting would be to disable any sort of packages your running like ips or proxy.. And you didn't even mention this.. Just stating that pfsense+ is the problem..
-
@johnpoz said in pfSense Plus block file upload:
Just stating that pfsense+ is the problem..
Sure, in my network, the upload problem is given by pfSense. I never wrote that pfSense software is broken, but that MY pfSense have something wrong, it is better that way?
-
~Generally speaking~ ... unless you have an exceptionally special use case ... get rid of the proxy and clamav.
Your internet is fast enough you don't need a proxy.
Most malware lazy enough to be sent in the clear and let clamav actually look at it isn't much of a threat anyway.Encryption is mostly standard now and pretty much makes both of these products useless. Hence the previous discussion about how it is impossible for pfsense to see inside the encrypted tunnel.
I'm cynical ... I say 80% chance the files got blocked by the clamav because loaded with trojans. We got bigger problems than uploads not working. :)
-
@lucas-rey said in pfSense Plus block file upload:
the upload problem is given by pfSense. I never wrote that pfSense software is broke
Re-read you first post and the above...you implied that it's pfSense. Glad you realize it wasn't.
-
@skogs said in pfSense Plus block file upload:
~Generally speaking~ ... unless you have an exceptionally special use case ... get rid of the proxy and clamav.
Your internet is fast enough you don't need a proxy.
Most malware lazy enough to be sent in the clear and let clamav actually look at it isn't much of a threat anyway.
Encryption is mostly standard now and pretty much makes both of these products useless. Hence the previous discussion about how it is impossible for pfsense to see inside the encrypted tunnel.
I'm cynical ... I say 80% chance the files got blocked by the clamav because loaded with trojans. We got bigger problems than uploads not working. :)You are right, I always had a proxy in my network, but now that I have a fiber with 2,5Gbit throughput maybe is time to switch off the proxy :)
@nollipfsense said in pfSense Plus block file upload:
Re-read you first post and the above...you implied that it's pfSense. Glad you realize it wasn't.
Why isn't pfSense the issue? I removed the problem just disabling ClamAV, and antivirus is a pfSense module, isn't it? So, in my point of view, upload issue is caused by a specific pfSense configuration/module in my personal environment, and this after upgrading 2.6 to 23.01. Probably something goes wrong, and probably if I install a fresh pfSense 23.01 version everything will work, but as I said, after upgrade I got this issue, and luckily I had identified it in ClamAV. Why this happen, I don't know.
-
@lucas-rey said in pfSense Plus block file upload:
Why this happen, I don't know
Like pfBlockerng, ClamAV uses 'rules'.
Based on IP addresses, ports used, and whatever it can find in the Ethernet packet headers (all the bits, flag etc), it applies the 'rules' and then decides.
Like pfBlockerng, ClamAV does nothing by default.
Then the admin drops in, and start activating 'rule sets'. These rule sets are not made by 'Netgate'. Like the IP and DNSBL feeds of pfBlockerNG. You just have to 'trust' them.
It happens all the time : 'something' (a rule) actually blocks traffic that you do not want it to block.I'm not a ClamAV user myself, but I'm pretty sur ClamAV logs every decision it takes. Like pfBlockerNG. Check these ClamAV logs, and you will know what rule did block you upload. Disable (or edit ?) the rule, and you'll be fine.
Tools like ClamAV needs to be checked all the time for false positives.
Anyway, glad you have the issue cleared
-
@lucas-rey said in pfSense Plus block file upload:
Why isn't pfSense the issue?
Veteran pfSense users tend to be a little sensitive to broad accusations made against the software without warrant. Here's what I mean ---
First, the official pfSense software is divorced from the packages. The available packages are, by and large, created and maintained by volunteer developers who have no association with Netgate and the pfSense team, and they are not paid for their efforts creating and maintaining a package. Over time these volunteer developers come and go. That can leave a given package orphaned with no developer support. In the case of a few more popular packages, the Netgate team might step in and provide some rudimentary support of those packages. But that is very rare.
It has become quite frequent for a user to start a thread (much like yours began) with a blanket statement saying "pfSense is blocking ..." without providing any additional details about which add-on packages are installed. pfSense itself, installed directly from a USB memstick image or ISO, and configured via the setup wizard is not going to block anything outbound. Providing the user only supplies the requested information (IP addresses for interfaces and assigning interfaces to LAN and WAN), then things will just 100% work. And they will continue to work through later upgrades.
The problems happen when users install add-on packages and/or start to monkey with default settings (DNS Resolver being a favorite place for folks to start tampering without a firm knowledge of what they are doing). This can definitely lead to problems, but the problems in this case are not "pfSense" so much as they are user-inflicted by the user not fully understanding what they are changing or by them installing an add-on package.
If you install packages that are designed to intercept and block things, then when something is blocked or stops working, the very first place you need to look for the problem is that add-on package you installed! Installed packages result in "non-default" installations. pfSense with installed packages is NOT the same as pfSense with no packages.
Here is an example. A user will install a package like
pfBlockerNG. That package is designed from the get-go to block stuff using lists of IP addresses. So, soon after installing the package and configuring a bunch of "block lists", the user notices that a number of their favorite websites no longer work properly or won't load at all. The user creates a new topic here on the forums but titles it "pfSense is blocking some websites" and never mentions anywhere in the post they installed and configured the pfBlockerNG package. Veteran users know that generally speaking pfSense doesn't do that (block some websites while allowing others). But with the user supplying no helpful context (such as installed packages), then an argumentative game of whack-a-mole ensues as the folks trying to help have to guess what it might be or what the user has misconfigured.So back to your case. If you had started your post by saying "I have pfSense version x.xx and I am running the following packages...", the initial responses you received would likely have been quite different. Why you ask? Because veteran users know that a vanilla pfSense install will not just block some particular file type. It can't even do that if you want it to. That can only happen if some add-on package is installed that provides extra capability. By telling them upfront what add-on packages you have installed, they can better tailor their troubleshooting suggestions.
I know from attempting to support the packages I maintain, that it eventually gets somewhat tiring to have to drag information out of users piece-by-piece when trying to sort out a problem. You tend to get frustrated and a bit irritable. It is much easier when the pertinent information is divulged up front. So, in your case it appears the
clamAVpackage was the culprit. ButclamAVis not a native part of pfSense. A user must manually install and configure it. And remember packages are maintained by non-Netgate/pfSense developers, so support issues with version upgrades can most certainly appear. But those problems are not the fault of pfSense. They are a natural consequence of installing and using a third-party add-on package supported by a volunteer developer.The users here will be glad to help you with packages they may be familiar with. There are also a few sub-forums dedicated to specific packages or package categories. Posting a question there can result in much faster help.
-
@bmeeks said in pfSense Plus block file upload:
Veteran pfSense users tend to be a little sensitive to broad accusations made against the software without warrant. Here's what I mean ---
I understand, and agree. But let me explain my point of view. I'm not a security expert, not a newbie too anyway. What I did is just upgrade from 2.6 to pfSense plus 23.01 and from this point, I got upload issue. From my preliminary investigation, I see that rolling back to 2.6 or skip the pfSense network at all, everything worked fine. So, yes, my first question was: "Why pfSense block my upload"? Then after further investigation I discovered the issue in ClamAV. Still anyway don't understand why in 2.6 I got no issue, btw.
Probably I was wrong, and considered pfSense a box that contains everything, included ClamAV. That's why I wrote MY pfSense have an issue. That's it :)Thank you anyway for your long explanation post.
-
@lucas-rey said in pfSense Plus block file upload:
Still anyway don't understand why in 2.6 I got no issue,
There are a number of changes in 23.01 (and pfSense 2.7 CE DEVEL) compared to pfSense 2.6. One huge change is the move from FreeBSD 12.3-STABLE to 14-CURRENT. Another big change is the move from PHP 7.4 to PHP 8.1.
In your case with
clamAV, my suspicion would be an issue perhaps with the move from FreeBSD 12.3-STABLE to 14.0-CURRENT.But I stand by my original post -- when you install an add-on package whose job is to block stuff, then anytime something stops working the very first place to investigate is that add-on blocking package. Try disabling it to see if the block goes away. If it does, you've found the culprit and can troubleshoot accordingly. Netgate does not test packages for upgrade compatibility. That falls upon the volunteer package developers. Only in rare instances will Netgate step up and modify a package's code base.
-
@bmeeks said in pfSense Plus block file upload:
when you install an add-on package whose job is to block stuff,
Here is the problem, if i had installed a package or a new module, for sure my first investigation would have been redirected to such package. Because I upgraded a working pfSense, and I got a system that start to have issue without any reason. Later I realize that the only external module I have enabled in the past was the proxy server, but the first post here was written to ask for advice on which section I can start to investigate, sure, I didn't mean that pfSense software doesn't work at all :)
-
I think people are arguing semantics. :)
@lucas-rey
In addition to the above, note upgrading pfSense also upgrades any installed packages. This is why Netgate recommends uninstalling packages before upgrade. So regardless of pfSense, clamAV was also likely updated to a newer version. -
@bmeeks
First of all, I stand corrected : pfBlockerNG, by default, right after installing, does contain an 'example' DNSBL feed, probably the "StevenBlack" list.
This means these will get blocked for DNS resolution.@Lucas-Rey
There are packages listed in here System > Package Manager > Available Packages that add a functionality, like "Notes".
Some make more info pfSense available, like Cron.
And some really do interact upon the traffic flowing trough the router/firewall.Btw : upgrading from pfSense 2.6.x to 23.01, afaik, doesn't interact with the traffic. Neither the fact that pfSense used PGP 7.4 before, and now 8.x. Upgrading a package, any package, doesn't change a thing.
But : these packages, like pfBlockerNG, ClamAV, and other use rules or feeds or whatever externally available info that is sourced by .... people and sources completely unknown to pfSense (Netgate) and the package (authors) used.
And soon as you start to use these packages, you have to baby-sit them, as "the rule set" used can react upon traffic any time.
I'm not exaggerating : every morning, coffee first and then you inspect the blocked or 'event' list of the package.The very first day you installed pfSense, you found no firewall rules on the WAN interface, and just one pass all rule on the LAN.
Nothing was filtered. Security was also easy : nothing comes in except what you (the human) takes in by visiting a site, and getting it some content.
Now you want to block access to some sites or some content : you use ClamAV, so you started to use automation.
False positives is now a thing, and surely not an exception, so, you - the admin - have work to do : check what the package does/did. Your system will be as secure as the level of your understanding of how it works.
Also : security can never be automated 100 % as long as humans are involved.
