Proxy causing time outs.



  • Hi everyone.

    I have been running pfsense for a little while and pretty happy with it. Now I am in need to filter for certain computers.

    So I installed squid3 and Dansguardian to get a transparent filtered proxy.

    I am aiming to have a IP range which is filtered and the rest of the ip range is unfiltered.

    However as soon as I start squid3 every single website times out. I have tired reinstalling the package, Restarting pfsense to no avail. All settings in squid3 are default. Ive done some googleing and i haven’t found anything that is fixing the issue.

    Can anyone suggest what I might be missing etc. In previous versions i just installed squid and dansguardian and it worked straight away on the whole ip range.

    Thanks in advance,
    Mr.B



  • Will I need to make a rule in the pfsense box to send traffic to the proxy port?

    Ive been looking and thinking of the best way to filter traffic to a LAN ip range (ranges if i add a guest network). Im wondering if this is possible to do via DNS somehow and forget dansguardian.

    I am thinking a proxy cache will be nice for some better performance and bandwidth etc etc.



  • You can do advanced URL filtering based on ACL groups with squidGuard, but you have to get squid3 working first.  First, are you running it in standard mode or transparent mode?



  • Hi there,

    Ill do some looking into squidgaurd as soon as i have got the proxy going.

    I have only tried using the proxy (squid3) in transparent mode. To be honest testing the proxy in standard mode didn’t cross my mind at all. Ill test it in standard mode and see how it goes, I will add any error message's (From squid logs) that may appear in the standard mode and transparent mode.

    If both fail would it be worth trying squid2?

    Please let me know if doing this is not the efficient / correct way of finding the squid fault.

    The pfsense box is a Dell Optiplex 170L (2.8 P4, 1GB ram, 2x Intel Pro 100, 80GB HDD) so im running the i386 version of pfsense. I will admit the pfsense box is getting old but I have never really been in a situation where the CPU & RAM is on high load etc. So I haven’t seen the need to replace it with something newer.



  • I forgot to mention i did switch on dynamic caching when i installed squid3 first time. However i switched it off when the websites were timing out.

    I would like to enable anti-virus on the proxy for some extra security as well. But i think getting the essentials working first is the way to go.



  • Ok I did some tests while no one else was on the network.

    I installed squid3 and enabled it in transparent mode. I also enabled logging to grab any messages.

    Once it was enabled i reload a bunch of random tabs in my FireFox session. It seem now a few website's work. (Very few at the moment.) I can get access to :
    Facebook
    Google
    Youtube
    & This forum

    They are the only ones ive found to be working so far. The proxy seems to be helping the performance on those sites. I should of installed it a long time ago if the performance is like that on all sites. (I ont mind if there is a slight performance loss if the filter and av is on.)

    However when trying to get to sites which is timing out there is nothing appearing on the real time logs. Which is something I find strange.

    Now in my mind the fact those few sites seem to be working, Would it be a minor config that i dont know about to get everything else working / loading correctly?



  • No, it should just work.  If HTTPS sites are working, HTTP sites should be no problem.  Don't use squid2, stay on squid3.  When the timeouts are happening, you can shell in and run:

    squidclient -h squid_ip_address -p 3128 mgr:info

    look for the Median Service Times, and look a a large number that would indicate a slowdown.  Also, look at /var/log/squid/access.log and cache.log for clues.



  • Thanks for the command to type in. I used this command on the pfsense box via ssh and got this:

    Sending HTTP request ... done.
    HTTP/1.1 403 Forbidden
    Server: squid/3.4.10
    Mime-Version: 1.0
    Date: Mon, 27 Apr 2015 15:45:11 GMT
    Content-Type: text/html
    Content-Length: 3082
    X-Squid-Error: ERR_ACCESS_DENIED 0
    Vary: Accept-Language
    Content-Language: en
    X-Cache: MISS from localhost
    X-Cache-Lookup: NONE from localhost:3128
    Via: 1.1 localhost (squid/3.4.10)
    Connection: close
    
    <title>ERROR: The requested URL could not be retrieved</title>
    
    # ERROR
    
    ## The requested URL could not be retrieved
    
    * * *
    
    The following error was encountered while trying to retrieve the URL: [cache_object://10.0.0.1/info](cache_object://10.0.0.1/info)
    
    > **Access Denied.**
    
    Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.
    
    Your cache administrator is [admin@localhost](mailto:admin@localhost?subject=CacheErrorInfo%20-%20ERR_ACCESS_DENIED&body=CacheHost%3A%20localhost%0D%0AErrPage%3A%20ERR_ACCESS_DENIED%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Mon,%2027%20Apr%202015%2015%3A45%3A11%20GMT%0D%0A%0D%0AClientIP%3A%2010.0.0.1%0D%0A%0D%0AHTTP%20Request%3A%0D%0AGET%20%2Finfo%20HTTP%2F1.0%0AHost%3A%2010.0.0.1%0D%0AUser-Agent%3A%20squidclient%2F3.4.10%0D%0AAccept%3A%20*%2F*%0D%0AConnection%3A%20close%0D%0A%0D%0A%0D%0A).
    
    * * *
    
    Generated Mon, 27 Apr 2015 15:45:11 GMT by localhost (squid/3.4.10)
    
    

    So after seeing the access is denied and tired adding the ip range of the whole network to use the squid proxy (ACL). So in my case its 10.0.0.0/8 (I know i don’t need range that big but I started testing with DHCP etc years ago and kept with it.). But i am still getting time outs. :-( .

    The Clients on the network are Mac, Linux, Andriod Devices, iOS Devices and Windows with the same result.



  • I have no idea what command you entered and what all that output is.  However, it seems that you aren't allowed to use Squid.  By default, the LAN network for pfSense is Squid's allowed network range.  I have seen cases where you uninstall and then reinstall and it's working, so I would try that because it should be working for you.



  • Ive found this on the cache.log's.

    2015/04/27 21:49:23| pinger: Initialising ICMP pinger ...
    2015/04/27 21:49:23|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:49:23| pinger: Unable to start ICMP pinger.
    2015/04/27 21:49:23|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:49:23| pinger: Unable to start ICMPv6 pinger.
    2015/04/27 21:49:23| FATAL: pinger: Unable to open any ICMP sockets.
    2015/04/27 21:49:54 kid1| Starting Squid Cache version 3.4.10 for i386-portbld-freebsd10.1...
    2015/04/27 21:49:54| pinger: Initialising ICMP pinger ...
    2015/04/27 21:49:54|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:49:54| pinger: Unable to start ICMP pinger.
    2015/04/27 21:49:54|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:49:54| pinger: Unable to start ICMPv6 pinger.
    2015/04/27 21:49:54| FATAL: pinger: Unable to open any ICMP sockets.
    2015/04/27 21:50:00| pinger: Initialising ICMP pinger ...
    2015/04/27 21:50:00|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:50:00| pinger: Unable to start ICMP pinger.
    2015/04/27 21:50:00|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:50:00| pinger: Unable to start ICMPv6 pinger.
    2015/04/27 21:50:00| FATAL: pinger: Unable to open any ICMP sockets.
    2015/04/27 21:50:05| pinger: Initialising ICMP pinger ...
    2015/04/27 21:50:05|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:50:05| pinger: Unable to start ICMP pinger.
    2015/04/27 21:50:05|  icmp_sock: (1) Operation not permitted
    2015/04/27 21:50:05| pinger: Unable to start ICMPv6 pinger.
    2015/04/27 21:50:05| FATAL: pinger: Unable to open any ICMP sockets.
    
    


  • The other 2 log file's appear to be empty.



  • I'd blow it away and redo it from scratch.  You have some weird problem going on.



  • Ill give that a try.

    I know i can backup DHCP settings etc etc but what would i select to backup usernames and password for my isp etc. In my mind it will make life a little faster when ive got time to reinstall.



  • Do a backup via Diagnostics - Backup/Restore, but set it to not backup packages.  Your pfSense settings will be saved but the package details will be lost.  That allows you to really start fresh but not have to reconfigure the basics.



  • I have just wiped and started again. Bootable USB FTW!!

    All I have done is restore settings (for DHCP etc), go to squid3 in the packages, installed it, enabled it in transparent mode, checked if squid was running with top and timeout :-( .

    EDIT : However i have noticed the same site as before are still working with the proxy cache.



  • I don't know if transparent mode is yet working on squid3.  Some others have been complaining it's broken.



  • I can try squi 3 in standard mode and see if it works (Slandered mode for me is It required username and password).

    Will i need to set a user-name and password for each user or computer?



  • ??? You don't need to specify a username and password in standard mode.  You do, however, need to either manually configure the client to use the proxy, or implement WPAD so the client can find it automatically.  Transparent mode is convenient but useless for HTTPS, which is the way everything is going these days.



  • Will WPAD work for all OS's? So it will autocratically detect for Linux, Mac, Andriod?

    & would the settings return to default (ie no proxy etc) when the device leaves etc.

    I am also looking at the captive portal which will help security as well.

    I am testing the proxy with a Ubuntu VM before I block port 80 and redirect everything to the proxy.



  • Is there another way to do this without a need a for a username and password?

    I think i have found it. On the proxy setting I have set the authentication method to none in the Proxy server - Authentication.

    On my test VM (Ubuntu Mate 15.04. Kill two birds with one Stone. Test Ubuntu MATE 15.04 and test the proxy. LOL.) I have set the gateway ip and squid port, And it seems to be working. The increased speed on web pages is noticeable.

    WPAD i think is needed.



  • Will WPAD work for all OS's? So it will autocratically detect for Linux, Mac, Andriod?

    Windows, Mac, Linux - yes.  Android - no.

    & would the settings return to default (ie no proxy etc) when the device leaves etc.

    Yes.



  • Brilliant, I will look into that.

    All of the Android devices that connect to the network are known / allowed. So i do not see any reason i cant leave port 80 open to a static ip range (Unless i want to filter them. Then ill have to think of something.).

    The proxy was working for about half (test VM) and then it was timing out again. So i am currently going through the process of diagnosing it again.

    I did test the captive portal again and i love the idea of it and i am planning to use it in a different case if & when i need it. For what i need at the moment it is not necessary.



  • Autodetection of web proxies has been around for at least a decade.  That's what that Automatically detect settings checkbox does in Windows' Control Panel - Internet Options - LAN Settings - Automatic configuration.  I've seen a bug report filed against Android to get Google to add WPAD support but it hasn't happened yet.  I don't know about iPhone.



  • Can I make a firewall rule to redirect traffic from port 80 / 8080 to the proxy port (Default 3128)? This should then start filtering for all devices such as Android, iOS etc.

    So far I am loving SquidGaurd and have started restricting access to my needs. I am half tempted to make a group for my server's and tighten security a little (Block everything and allow domain's that I want such as pfsense.org, ubuntu.com etc etc).



  • Can I make a firewall rule to redirect traffic from port 80 / 8080 to the proxy port (Default 3128)? This should then start filtering for all devices such as Android, iOS etc.

    That's essentially what Transparent Proxy does.  You're still going to have Man in the Middle warnings on all HTTPS sites if you do that.  Leave it up to the user to manually config the proxy settings if his device doesn't support auto-detection.



  • Ok in that case ill take a look at implementing WPAD soon. I might end up doing it when no one is online to make life a little peaceful.

    Is there a plan to add WPAD into the WebUI (Config etc).

    How would I redirect traffic to work with the proxy. Its something ive never tried before and i would like to give it a try and learn what it does etc first hand.





  • Right WPAD seems to be working for Windows only. So what i have decided to do is manually add the proxy on static PC's and devices and ill continue to find a good way to make a transparent proxy working.

    I have considered a second PC just for the proxy but ive not worked out the most efficient way of setting it up.



  • Should work fine on Windows, Mac and Linux clients.  Only mobile seems to still be an issue.