Squid slow speeds and timeouts get fixed after reboot
-
Hello guys i have this problem with squid server, i have it setup without any cache option, antivirus is off, just a squid server for my vpn so i can use my home ip while on the go.
Squid works really great for a while, after it gets really slugish, like i have to wait 1-2 mint for a page to response ! everything gets back to normal after i reboot my pfsense and works great for a day or two then again i have really slow speeds so i have to reboot again.
What could cause this issue ?
i am running pfSense 2.4.2 on Firebox XTM 5 with Samsung PRO SSD 256GB, 2GB ram and CPU E7200 @ 2.53GHz.
-
Nobody ?
-
Could be memory pressure and swapping. Check your page file usage when it's slow?
-
Squid works really great for a while, after it gets really slugish, like i have to wait 1-2 mint for a page to response ! everything gets back to normal after i reboot my pfsense and works great for a day or two then again i have really slow speeds so i have to reboot again.
What could cause this issue ?
Your description doesn't really provide clear indication that Squid is the root cause. The 1-2 minutes for a page response sounds like you could be hitting a protocol time-out, with the connection succeeding on retry.
Rebooting resets the whole platform, so this doesn't help isolate if squid is the sole cause. I would suggest that the next time it occurs, try only restarting the squid service and see if that clears up the issue or it still persists until a full reboot of pfSense.
-
Hello thanks all for your help, i will answer all questions in order to find the problem that i am facing.
1. I am not using swap, i have a 256GB SSD and 2GB of ram, the ram never gets passed 20% at a max load, usually its on 10-15% max i saw was 18%
2. Restarting only squid does not help, only rebooting whole system makes it work back.
3. NAT works perfectly all the time, only squid starts to lag i had the same settings on pFsense 2.3.5 and squid worked perfectly after i clean install of 2.4.2 i did configure squid the exact same way i had it on 2.3.5 but this thing happens :(Please help me fix it.
-
If i setup proxy in my browser and go to google.com i get this: Waiting for http://google.com/… for around 2-3 minutes then it loads the browser circle still spins and after 2 more mins it finishes to load. Total of 5 minutes to wait for a complete load of google. This happens to any other website i am using. If i remove proxy and browse on NAT it all loads instantly.
-
Hi shimano, I don’t' have a solution, but I think I ran into a very similar problem myself, and wanted to share a few more details about the problem as I observed it so far.
I've been downloading a lot of Linux ISO's, so lots of large, long running downloads, and was having problems with the file downloads that should only be taking several minutes were taking hours to days to download, from my linux hosts. I also have a dual boot system with windows. When I booted into windows, the problem as so bad that windows would not even fetch updates, and web pages would start to load then ultimately time-out.
While trying to do more testing to characterize what was going on, I also noticed that, on linux, when my file downloads were running slow I could load & refresh web pages seeming ok, but things like ISO file downloads or PDF files displayed in a browser were significantly slower to the extend that some pages that normally only take a second or two to load, were taking minutes. On my dual boot box I found after booting into windows, everything internet was so slow that even web pages were not loading or refreshing, even though a linux host would load the same web pages seemingly fine. Basically windows seems to be more impacted by this than linux.
Like you, restarting squid did not alleviate the problem, but all of this goes away when squid proxy is disabled and pfSense is used as a pure router, (what your referring to as NAT).
The nature of the traffic behaviour I'm seeing is indicative that there is some form of selective throttling taking place as after each reboot of pfSense. I can reproduce a short burst of high rate traffic then after a period of bytes, (which happens quicker than 1-2 minutes with lots of files downloading simultaneously), watch the traffic suddenly fall off to just a few KBs/s, while other things like web pages seemed to download normally (from linux hosts).
I don't have any traffic throttling enabled, and verified that squid's bandwidth throttling settings were all set to default / disabled, Yet still the only way I can get full throughput bandwidth is to not proxy through squid.
For added note, my configurations were built from scratch using 2.4.x releases, so I think that rules out a version migration issue.
Anyway, for now, the problem as I'm seeing it, is so bad that I had to completely disable squid so I could get some work done. But I did want to follow up to share that you are not alone in your observations of bandwidth throughput problems with squid proxying.
-
Hello TechyTech,
Thank you very much for your kind replay, your problem is exactly 100% same as mine ! i just tested from a centos machine and everything seems to work fine…
I also used a nintendo console to connect to the internet and browse and even play online games and everything seems ok !
But the windows machines that i actually need to work with squid don't work at all, pages load after 1-2 minutes and restarting squid does nothing, restarting the whole gateway seems to have it work for a few minutes like you said..
I really don't see why this is happening, before upgrading to 2.4.x i had 2.3.5 and squid worked so beautiful and perfect i really don't know why i've upgraded i had no issue at all i just wanted to have the latest stable software but it seems its still a beta and not so stable after all..
Lets hope there will be a solution for this asap and we both get our hardware to work properly !
-
just a squid server for my vpn so i can use my home ip while on the go.
Why does squid have anything to do with this?
-
Because i need to use my squid when in VPN so i can get out to internet.
-
Why?
-
Because i can't have NAT enabled and squid cache helps since all the people who are using squid are browsing mainly the same websites and that would help a lot.
-
Anybody has any news regarding this issue ?
-
Anybody has any news regarding this issue ?
Hi Shimano,
Thank you for providing the additional details. I see from your replies that you are experiencing much the same issues as me, including the variations of behaviour across multiple platforms. I see no need to further badger you with interrogatory questions about why you're using a caching proxy, just to have you answer with the overtly obvious purpose of what a caching proxy does, so I'll move on to focus on the technical aspects of TCP/IP that would exhibit this type of performance behaviour differences between a router and a proxy, based on some years of experience working in multi-platform / multi-os environments and seeing this problem crop up when mixing various network connectivity topologies.
The basic jist is that although TCP/IP is the communication protocol that is common to all devices involved, how each OS & device implements it's version of TCP/IP tends to be unique; with Windows being a common culprit to suffer the most issues when something doesn't work right.
What I'm eluding to is a quite common problem in a mixed platform environment when using a proxy vs. a pure switch / router. To be more specific about the TCP/IP protocols involved in this type of problem are things like MTU/MSS size, Path MTU discovery (PMTUD) (aka Black Hole Detection), traffic congestion controls, etc.
First, to understand the difference in behaviour between using squid and pfSense as a pure router; I'll clarify that Squid Proxy is not a router, so trying to compare pfSense acting as a pure router vs. a caching proxy is going to lead to chasing red herrings.
To explain a bit more detail; When only routing, TCP/IP protocol negotiation takes place between the client and the server, transiting the router. So things like PMTUD and congestion controls are negotiated end to end (client <-> server).
But a proxy is an intercessor in the communication links between the client and server, such that the communication links are split up into two portions; the client to the proxy, and the proxy (acting as an external client on behalf of the internal client) to the server (client <-> proxy <-> server).
So, when the client negotiates it's TCP session, when transiting through a router, there is a single TCP session between client and server, but when using a proxy you have two separate TCP session, one from the client to the proxy, the other from the proxy to the server. Both of these sessions must independently negotiate their TCP session parameters.
Thus it is in the nature of the differences of various OS TCP/IP implementations is where all the fun begins.
Squid is known for not playing well with PMTUD, especially if there is an upstream link, such as a VPN, involved that may be reducing the expected packet sizes, or interfering with PMTUD. As well, Windows is somewhat notorious for signification performance issues if PMTU / Black Hole Detection, et all, can not be negotiated correctly. Which is why Windows seems to be more impacted than 'nix clients. And on occasion, you may even need to disable PMTUD / Black Hole Detection when using a proxy, to work around packet sizing issues that may not be otherwise resolvable.
And that's the short answer.
Much more involved answer includes digging into the protocol handshake to verify if PMTU discovery, among other things, is taking place or not when squid is used. This involves doing some diagnostic testing to verify that TCP/IP packets are not being fragmented upstream, as well as packet traces of the communications from squid to the target server to see if proper packet sizing negotiations are taking place or being interfered with in some way.
Since these are pretty common problem, there's already quit a bit of info on the internet about dealing with PMTU / Black Hole Detection problems that you can search for. To get you started, here's a good article describing these type of issues with squid & windows.
https://squidproxy.wordpress.com/2007/06/05/thinsg-to-look-at-if-websites-are-hanging/
Anyway these types of problems tend to be site / link specific, so you'll have to do some digging on your end to verify if this is the problem that is occurring for you.
And if you still need help diagnosing your issue, I would suggest that you provide more detail about your use case configuration, and how you are using your VPN link, in order for others to understand where your in/out packet flows are between your internal, external & VPN networks, and better understand where protocol negotiation issues may be occurring.
