• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

PfSense 2.2.3 - Internet is very slow via Squid3

Cache/Proxy
12
32
20.7k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • G
    gdsnytech
    last edited by Jun 29, 2015, 3:52 PM

    Hello all,

    I had several crash issues after upgrading to pfsense 2.2.3 from 2.2.2. I was able to resolve from of the packages issues by reinstalling and reboot. However the internet via squid3 was not working. Though my setting were still the same. Nothing changed but the pfsense upgrade

    I decided to do a fresh install of pfsense 2.2.3. Afterwards i installed Squid3, SquidGuard, LightSquid, Sarg, Snort, VHosts, Cron, bandwidthD. ntopng

    I had several package install time-outs. i.e. snort, ntopng but was able to get them to fully install after several retries. I had no issues installing Squid3 and SquidGuard.

    As of now i currently have my settings for both Squid3 and SquidGuard set as i had them in pfsense 2.2.2 but the internet is very slow. DNS is set to internal dns, google and opendns in Squid3.

    Is anyone else having this issue? Any ideas of what might be causing it?

    FYI: I am only using pfsense as a web content filter via squid3, squidGuard and snort. Firewall filtering is currently turned off along with WAN disabled. DMZ and LAN are bridged.

    1 Reply Last reply Reply Quote 0
    • H
      heper
      last edited by Jun 29, 2015, 4:10 PM

      fterwards i installed Squid3, SquidGuard, LightSquid, Sarg, Snort, VHosts, Cron,bandwidthD. ntopng

      none of the above are known for their stability and reliability (especially, short after a release).

      I would start with one package at a time and see what the culprit is.

      most likely culprits: LightSquid, Sarg, SquidGuard, VHosts, bandwidthD

      1 Reply Last reply Reply Quote 0
      • K
        KOM
        last edited by Jun 29, 2015, 4:38 PM

        Focus on squid3 and snort.  I can't possibly imagine how a log analyzer like Lightsquid or Sarg would slow down his web browsing since they only run ever now and then.

        1 Reply Last reply Reply Quote 0
        • G
          gdsnytech
          last edited by Jun 29, 2015, 8:20 PM

          I have removed all of the packages listed and tried installing squid 2 but i'm still experiencing the same lag.

          Is it possible that it could be a disk read error issue?

          1 Reply Last reply Reply Quote 0
          • J
            jimp Rebel Alliance Developer Netgate
            last edited by Jun 29, 2015, 8:25 PM

            Some have found that the disk changes we made for sync mode have slowed down squid by default.

            If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

            mount -o nosync /
            

            Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

            Need help fast? Netgate Global Support!

            Do not Chat/PM for help!

            1 Reply Last reply Reply Quote 0
            • K
              KOM
              last edited by Jun 29, 2015, 8:55 PM

              Squid2 is crap and very old.  Stick to squid3.

              1 Reply Last reply Reply Quote 0
              • G
                gdsnytech
                last edited by Jun 30, 2015, 1:30 AM

                @jimp:

                Some have found that the disk changes we made for sync mode have slowed down squid by default.

                If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

                mount -o nosync /
                

                I forgot to mention that the pfsense install is on a RAID 1

                but that command seems to have did it. HTTP requests seems to be more responsive! THANK YOU SIR! ;-)

                1 Reply Last reply Reply Quote 0
                • G
                  gdsnytech
                  last edited by Jun 30, 2015, 6:35 PM

                  How do i make this mount option stick on fstab? After reboot the mount -o nosync option goes away.

                  1 Reply Last reply Reply Quote 0
                  • D
                    doktornotor Banned
                    last edited by Jun 30, 2015, 8:12 PM

                    @gdsnytech:

                    How do i make this mount option stick on fstab? After reboot the mount -o nosync option goes away.

                    Would help to re-read the post quoted. It's already written there.

                    1 Reply Last reply Reply Quote 0
                    • G
                      gdsnytech
                      last edited by Jul 1, 2015, 12:05 AM

                      That is what i did. For some reason it wasn't sticking after rebooting. I had to keep going into /etc/fstab to make the change. But it seems to be sticking now.

                      1 Reply Last reply Reply Quote 0
                      • H
                        hbc
                        last edited by Jul 1, 2015, 7:53 AM

                        @jimp:

                        Some have found that the disk changes we made for sync mode have slowed down squid by default.

                        If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

                        mount -o nosync /
                        

                        Could this change to disk sync also be the issue for my listening queue problem with squid? –> https://forum.pfsense.org/index.php?topic=95873.0

                        At least as soon as I disable disk sync, the listen queue size drops to zero. I compared my /etc/fstab from backup with current one and this sync options wasn't present before.

                        Maybe this option creates big performance impact on RAID systems. My secondary CARP node - also running 2.2.3 and sync option enabled - does not have this problem, but doesn't have RAID either.

                        1 Reply Last reply Reply Quote 0
                        • G
                          gdsnytech
                          last edited by Jul 1, 2015, 6:39 PM Jul 1, 2015, 5:29 PM

                          @hbc:

                          Could this change to disk sync also be the issue for my listening queue problem with squid? –> https://forum.pfsense.org/index.php?topic=95873.0

                          At least as soon as I disable disk sync, the listen queue size drops to zero. I compared my /etc/fstab from backup with current one and this sync options wasn't present before.

                          Maybe this option creates big performance impact on RAID systems. My secondary CARP node - also running 2.2.3 and sync option enabled - does not have this problem, but doesn't have RAID either.

                          It is definitely a RAID issue. Squid caching performance suck on a RAID. Especially with the 'mount sync' option. Just edit your /etc/fstab file with mount option 'nosync' with 'Edit File' in 'Diagnostics'. Save and reboot the box. When it comes back check fstab to make sure that the change is still there.

                          Other mod changes that i made were to 'Systsm> Advance> System Tunables>'

                          Tunable name                    Value
                          vfs.read_max            from 32 to 128

                          https://doc.pfsense.org/index.php/Squid_Package_Tuning

                          Created
                          kern.ipc.nmbclusters        32768

                          created the above ONLY if you are using the 'diskd' 'Hard disk cache system' setup in Squid. Which requires you to copy ipcs and ipcrm from a FreeBSD 10.1 ISO /usr/bin/ to pfsense /usr/local/bin

                          also reboot and check after reboot to make sure that the settings are still there.

                          Squid is 'flying right now'. Well sort of. But the 'nosync' boot option in fstab does make a BIG different.

                          I am running squid in a production environment.

                          1 Reply Last reply Reply Quote 0
                          • M
                            mesro09
                            last edited by Jul 4, 2015, 6:25 PM

                            i have already this problem but i dont have any raid config
                            everything was good working until i update from 2.2.2 to 2.2.3 and intennet put so slow now i am finding solution i already tried to reinstall squid and squidguard but nothing is solved now i dont have any filter
                            i installed new one with fresh install pfsense 2.2.3 but with fresh install i have problem too after restart pfsense squid and squidguard stops and cant start again (squid-1): The redirector helpers are crashing too rapidly, need help!
                            my system was squid3 and squidguard-dev amd64 squid non transparent (with wpad)
                            i hope to solve
                            second time i did big problems (first when i update to 2.2.2 rebooting time delay so much) and 2.2.3 squid problems
                            i think it is time to change FW with others

                            1 Reply Last reply Reply Quote 0
                            • G
                              gdsnytech
                              last edited by Jul 7, 2015, 9:28 AM

                              @mesro09:

                              i have already this problem but i dont have any raid config
                              everything was good working until i update from 2.2.2 to 2.2.3 and intennet put so slow now i am finding solution i already tried to reinstall squid and squidguard but nothing is solved now i dont have any filter
                              i installed new one with fresh install pfsense 2.2.3 but with fresh install i have problem too after restart pfsense squid and squidguard stops and cant start again (squid-1): The redirector helpers are crashing too rapidly, need help!
                              my system was squid3 and squidguard-dev amd64 squid non transparent (with wpad)
                              i hope to solve
                              second time i did big problems (first when i update to 2.2.2 rebooting time delay so much) and 2.2.3 squid problems
                              i think it is time to change FW with others

                              Just try the suggestion above and make the change on fstab.

                              1 Reply Last reply Reply Quote 0
                              • M
                                Mr. Jingles
                                last edited by Jul 7, 2015, 9:35 AM

                                @jimp:

                                Some have found that the disk changes we made for sync mode have slowed down squid by default.

                                If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

                                mount -o nosync /
                                

                                My squid3 is not fast either. But I don't like risking disk corruption, possibly destroying pfSense. That is: what are the odds disk corruption will appear, Jim? Relevant variables? A thumb number (0,005% or 60%)?

                                6 and a half billion people know that they are stupid, agressive, lower life forms.

                                1 Reply Last reply Reply Quote 0
                                • K
                                  KOM
                                  last edited by Jul 7, 2015, 1:23 PM

                                  DNS issues can also make squid look slow.  Shell in and run:

                                  squidclient -h LAN_IP_Address -p 3128 mgr:info

                                  Then read the report, paying special attention to the Median Service Times section.  Look for anything that seems large as compared to the others.

                                  1 Reply Last reply Reply Quote 1
                                  • M
                                    Mr. Jingles
                                    last edited by Jul 7, 2015, 3:26 PM

                                    @KOM:

                                    DNS issues can also make squid look slow.  Shell in and run:

                                    squidclient -h LAN_IP_Address -p 3128 mgr:info

                                    Then read the report, paying special attention to the Median Service Times section.  Look for anything that seems large as compared to the others.

                                    I do not want to hijack this thread, so if I have to create a new thread (similar problem), I will, just let me know  :-[

                                    [quote]
                                    squidclient -h localhost  -p 3128 mgr:info
                                    Sending HTTP request … done.
                                    HTTP/1.1 200 OK
                                    Server: squid
                                    Mime-Version: 1.0
                                    Date: Tue, 07 Jul 2015 15:23:33 GMT
                                    Content-Type: text/plain
                                    Expires: Tue, 07 Jul 2015 15:23:33 GMT
                                    Last-Modified: Tue, 07 Jul 2015 15:23:33 GMT
                                    X-Cache: MISS from squid
                                    X-Cache-Lookup: MISS from squid:3128
                                    Connection: close

                                    Squid Object Cache: Version 3.4.10
                                    Build Info:
                                    Start Time:    Tue, 07 Jul 2015 13:39:21 GMT
                                    Current Time:  Tue, 07 Jul 2015 15:23:33 GMT
                                    Connection information for squid:
                                            Number of clients accessing cache:      2
                                            Number of HTTP requests received:      1037
                                            Number of ICP messages received:        0
                                            Number of ICP messages sent:    0
                                            Number of queued ICP replies:  0
                                            Number of HTCP messages received:      0
                                            Number of HTCP messages sent:  0
                                            Request failure ratio:  0.00
                                            Average HTTP requests per minute since start:  10.0
                                            Average ICP messages per minute since start:    0.0
                                            Select loop called: 609439 times, 10.259 ms avg
                                    Cache information for squid:
                                            Hits as % of all requests:      5min: 23.7%, 60min: 12.2%
                                            Hits as % of bytes sent:        5min: 37.0%, 60min: 4.2%
                                            Memory hits as % of hit requests:      5min: 0.0%, 60min: 8.9%
                                            Disk hits as % of hit requests: 5min: 0.0%, 60min: 8.9%
                                            Storage Swap size:      6668 KB
                                            Storage Swap capacity:  0.0% used, 100.0% free
                                            Storage Mem size:      4620 KB
                                            Storage Mem capacity:    0.2% used, 99.8% free
                                            Mean Object Size:      12.97 KB
                                            Requests given to unlinkd:      0
                                    Median Service Times (seconds)  5 min    60 min:
                                            HTTP Requests (All):  0.10857  0.32154
                                            Cache Misses:          0.12783  0.37825
                                            Cache Hits:            0.00000  0.07014
                                            Near Hits:            0.00000  0.22004
                                            Not-Modified Replies:  0.05633  0.05633
                                            DNS Lookups:          0.01940  0.02683
                                            ICP Queries:          0.00000  0.00000
                                    Resource usage for squid:
                                            UP Time:        6252.187 seconds
                                            CPU Time:      73.719 seconds
                                            CPU Usage:      1.18%
                                            CPU Usage, 5 minute avg:        0.89%
                                            CPU Usage, 60 minute avg:      1.27%
                                            Maximum Resident Size: 178544 KB
                                            Page faults with physical i/o: 0
                                    Memory accounted for:
                                            Total accounted:        7844 KB
                                            memPoolAlloc calls:    246629
                                            memPoolFree calls:    255812
                                    File descriptor usage for squid:
                                            Maximum number of file descriptors:  58977
                                            Largest file desc currently in use:    51
                                            Number of file desc currently in use:  32
                                            Files queued for open:                  0
                                            Available number of file descriptors: 58945
                                            Reserved number of file descriptors:  100
                                            Store Disk files open:                  0
                                    Internal Data Structures:
                                              569 StoreEntries
                                              537 StoreEntries with MemObjects
                                              536 Hot Object Cache Items
                                              514 on-disk objects

                                    What would you make of this, KOM?

                                    Thank you  :P

                                    6 and a half billion people know that they are stupid, agressive, lower life forms.

                                    1 Reply Last reply Reply Quote 0
                                    • K
                                      KOM
                                      last edited by Jul 7, 2015, 4:19 PM

                                      Everything looks normal.  I think your problem is disk-related as you suspected.

                                      1 Reply Last reply Reply Quote 0
                                      • G
                                        gdsnytech
                                        last edited by Jul 7, 2015, 6:35 PM

                                        @Mr.:

                                        @jimp:

                                        Some have found that the disk changes we made for sync mode have slowed down squid by default.

                                        If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

                                        mount -o nosync /
                                        

                                        My squid3 is not fast either. But I don't like risking disk corruption, possibly destroying pfSense. That is: what are the odds disk corruption will appear, Jim? Relevant variables? A thumb number (0,005% or 60%)?

                                        I have made the change in fstab both for work (two sites) and home setup and it is working fine. Just make the change in fstab and then reboot.

                                        1 Reply Last reply Reply Quote 0
                                        • M
                                          mesro09
                                          last edited by Jul 10, 2015, 8:19 PM

                                          hello
                                          let me explain you this changes for squid caches but i dont use squid for cache contens my config is 0
                                          so can i make this changes ?¿?

                                          1 Reply Last reply Reply Quote 0
                                          3 out of 32
                                          • First post
                                            3/32
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.