Squid-reverse



  • Hi !

    the squid-reverse package is a replacement for the "normal" squi package since pfSense 2.0 and combines reverse functionality with the normal squid caching proxy.

    you can use the squid-reverse package to replace the squid package when you're using squid in pfSense 2.0. the configuration should be kept.

    squid-reverse is not available in pfSense 1.x.

    i'll bump the squid version in squid-reverse to squid 3.x when squid 3.x is running stable…



  • Could you post a sample configuration?

    I've been trying on and off to get this working for months, and still can't.

    Everything looks right, but it just won't forward anything!



  • Hi !
    You are trying to use the reverse part and it does not work ?
    First:
    Did you add Firewall-Rules from ANY to WAN-Address for 80 / 443 ?

    The three config fields are as follows:

    HOST_SSL;192.168.1.1;443;HTTPS
    HOST;192.168.1.1;80;HTTP

    WEBAPP_SSL;faq;https://gw.domainname.com
    WEBAPP;faq;http://gw.domainname.com

    HOST_SSL;WEBAPP_SSL
    HOST;WEBAPP

    here it works great !



  • Are there instructions anywhere, or do I simply follow something like this? http://wiki.squid-cache.org/SquidFaq/ReverseProxy

    Thanks,

    Mark



  • Hi !
    the packages should be self-explanatory, under each input field there are explanations…

    for further held, please ask ;-)



  • I've configured it like you suggested, and all I get when I try to browse to a page on it is:

    While trying to retrieve the URL: http://wi.atlantis.me.uk/

    The following error was encountered:

    Access Denied.
    Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.



  • is your subnet allowed under access control ?
    or any destimation blocked ?



  • Ive left everything on default except the reverse proxy section, should i change anything on the other tabs?

    Also, on your URI Definitions, what does the faq part mean?



  • you should check the access tab if your subnet is allowed and if there are any sites blocked…

    the faq reflects the uri- after the fqdn http://server.domain.tld: for http://server.domain.tld/faq

    FAQ_HTTP;faq;http://server.domain.tld will be http://server.domain.tld/faq



  • Sorted it.

    I was trying to publish the root of the site.

    Turns out you have to put a * in there for that.

    So, my config looks like this:

    Peer Definitions:
    prometheushttp;192.1.22.6;80;HTTP

    URI Definitions:
    atlantisweb;;http://www.atlantis.me.uk
    atlantisweb;
    ;http://atlantis.me.uk
    atlantiswi;*;http://wi.atlantis.me.uk

    ACL Definitions:
    prometheushttp;atlantisweb
    prometheushttp;atlantiswi

    I added my subnet into the top box in access control.

    Then I enabled logging in the general settings, SSH'd to the box and entered the shell.

    I ran tail -F /var/squid/logs/access.log so i could see all the incoming HTTP requests.

    Now to get OWA, Outlook anywhere and active sync working over HTTPS.

    Any ideas if this can do other HTTPS streaming things? I have a citrix secure gateway server that uses HTTPS to connect on port 443. It's not a web page though. I guess it's similar to activesync. At the moment it's running on 4430 but i'd like to run that through squid too.



  • @Sam0r:

    I have a citrix secure gateway server that uses HTTPS to connect on port 443. It's not a web page though. I guess it's similar to activesync.

    If its not http, you may need to use haproxy or native pfSense load balancer to balance tcp connections.



  • Actually I've just realised a day after getting it working that it doesn't support Exchange 2010 Web Services, this makes the package totally useless for me.

    I just want a reverse proxy, like in forefront TMG/ISA Server!



  • @Sam0r:

    Actually I've just realised a day after getting it working that it doesn't support Exchange 2010 Web Services, this makes the package totally useless for me.

    I just want a reverse proxy, like in forefront TMG/ISA Server!

    I have it working with varnish, haproxy and apache.

    To get balance with https without having certificate issues, you may need a wildcard certificate.

    Varnish does all http balance/cache
    Haproxy does the https balance
    Apache has the certificates and mod_security



  • I think I'll just go back to using Forefront TMG.

    As good as pfsense  is, it does't work for me. I need something up and running, and with documentation, not something put together by people in their spare time with next to no documentation.

    No offence to the community, it's a great work in progress, but its not for me.

    thanks for your time.


  • Rebel Alliance Developer Netgate

    @Sam0r:

    I think I'll just go back to using Forefront TMG.

    As good as pfsense  is, it does't work for me. I need something up and running, and with documentation, not something put together by people in their spare time with next to no documentation.

    No offence to the community, it's a great work in progress, but its not for me.

    thanks for your time.

    There are so many things wrong with that statement I don't know where to begin. But you are right, there is no one perfect solution for everyone, use whatever works best for you.



  • @jimp:

    There are so many things wrong with that statement I don't know where to begin. But you are right, there is no one perfect solution for everyone, use whatever works best for you.

    I second that.

    pfSense works great to me.



  • I think a big difference has to do with the scale of such setups:

    iirc marcelloc is overseeing a large-scale setup (Exchange 2010 with tens of thousands of mailboxes), so he can probably justify spending many hours to intimately learn those different packages in order to integrate and properly test them.

    Someone with a much smaller installation, say 100-200 users, may just want a reverse-proxy solution that "simply works" and offers commercial support, because he's probably busy with a dozen other IT-related subjects.

    So, as jimp noted, there is no one perfect solution for everyone.



  • @dhatz:

    I think a big difference has to do with the scale of such setups:

    iirc marcelloc is overseeing a large-scale setup (Exchange 2010 with tens of thousands of mailboxes), so he can probably justify spending many hours to intimately learn those different packages in order to integrate and properly test them.

    Someone with a much smaller installation, say 100-200 users, may just want a reverse-proxy solution that "simply works" and offers commercial support, because he's probably busy with a dozen other IT-related subjects.

    So, as jimp noted, there is no one perfect solution for everyone.

    You are 100% right.
    All features that I needed in pfsense that was not part of it, I have published to help many others to reach same result with less effort.

    Seeing Sam0r difficult on get a simple web proxy solution, maybe I can improve varnish package to require less configuration or dependencies for example.


  • Rebel Alliance Developer Netgate

    maybe have a wizard to setup exchange forwarding in Varnish. Steps through and asks, host name, IP, etc.

    No need to dumb down the whole GUI just find a way to make some common tasks easier.



  • @jimp:

    maybe have a wizard to setup exchange forwarding in Varnish. Steps through and asks, host name, IP, etc.

    No need to dumb down the whole GUI just find a way to make some common tasks easier.

    great idea!  :)

    I'll try it when I finish dansguardian.



  • Hey guys,

    I'm very new to pfSense, but I like the box and packages :)

    EDIT #2:
    Sorry… My fault. haven't seen it... squid.inc.. now it works like a charm :) I really like this box

    I also use squid as reverse-proxy to get access to OWA and ActiveSync. My main problem is, that I had to manually edit the .conf, because I need more than one https port. Everything is working great, until I reboot pfSense…

    What I found in the forum is, that this seems to be a general problem. But how can I fix it?! I already added "-f /path/to/my/conf.conf" to the startup script in /usr/local/etc/rc.d/squid.sh, but this won't work. Squid startsup with the "empty" config in /usr/local/etc/squid.

    Could someone please point me to the right direction, so the config will survive a reboot of pf Sense?

    Thanks in advance

    EDIT:
    pfSense 2.0.1 release and squid 2.7.9_2



  • It's just the lack of documentation that frustrates me.

    If the documentation had said "To forward the root directory of a website, insert a * in the URI." That would've saved me weeks.

    If I had weeks to spend on this I would, because I like what you guys do, we use untangle in some setups, because the OpenVPN works a treat. Others we use pfsense where we need a simple gateway, and in our enterprise setups we use TMG.

    I desperately wanted to prove that I could use pfsense in an enterprise rig, but I don't have the time to do it myself, or the funding to pay someone else to do it.

    Like i said, its the documentation that always falls sort when it comes to open source software, this isn't just a dig at pfsense, most open source software has this issue. It's easy to see why, documenting things is the boring bit. But to be successful it needs to be done.


  • Rebel Alliance Developer Netgate

    The base system is fairly well documented, but some packages lack it here and there. Squid-reverse (and varnish) are relatively new, and they are packages, so they tend to be less documented than the base system itself.



  • When I was testing varnish on my box.. I was confused and varnish's website was really no help but I posted questions on the forum. Marcelloc replied within hours to help me out.. Took a couple of days but he helped me out and made changes to the package as we found road blocks.



  • Something That helped me a lot during package devel was "googling" for recomended setup, tutorial as well documentation.

    Varnish itself is difficult to setup, gui helps But you still need to know about varnish.

    Sorry for the poor documentation. I alway try to include hints and link to documentation. I'm not That good on tutorials.

    If you still want to try varnish, use forum to post questions. I'll do my best to help you.



  • Heyho,

    thanks for this great package!

    If needed, I could help extending the gui setup of squid-reverse to support more options of squid…?!



  • Dear all,

    I have managed to setting squid-reverse properly. It works for two domain to 2 webservers.
    How can I manage to get all other domain to go to one server without having to list all the domains in the setting?



  • @Sam0r:

    Actually I've just realised a day after getting it working that it doesn't support Exchange 2010 Web Services, this makes the package totally useless for me.

    publishing /EWS* does not help…
    any ints ?



  • @Hobby-Student:

    If needed, I could help extending the gui setup of squid-reverse to support more options of squid…?!

    you're welcome  ;)



  • @trendchiller:

    @Sam0r:

    Actually I've just realised a day after getting it working that it doesn't support Exchange 2010 Web Services, this makes the package totally useless for me.

    publishing /EWS* does not help…
    any ints ?

    next version will support EWS :)



  • Im having abit of a problem.

    I want all subdomains for one domain going to one ip. And another much like it but a different domain.
    And let the target server handle subdomains.

    Much like pseudo config below;

    HOST1;192.168.1.1;80;HTTP
    HOST2;192.168.1.2;80;HTTP

    WEBAPP1;;http://.domainname1.com
    WEBAPP2;;http://.domainname2.com

    HOST1;WEBAPP1
    HOST2;WEBAPP2

    Meaning all requests to a.domainname1.com and b.domainname1.com goes to HOST1.  And c.domainname2.com, d.domainname2.com goes to HOST2.

    How can i do above scenario? im having no luck ;(  getting alot of squid access control problems



  • @danno:

    Im having abit of a problem.

    I want all subdomains for one domain going to one ip. And another much like it but a different domain.
    And let the target server handle subdomains.

    Much like pseudo config below;

    HOST1;192.168.1.1;80;HTTP
    HOST2;192.168.1.2;80;HTTP

    WEBAPP1;;http://.domainname1.com
    WEBAPP2;;http://.domainname2.com

    HOST1;WEBAPP1
    HOST2;WEBAPP2

    Meaning all requests to a.domainname1.com and b.domainname1.com goes to HOST1.  And c.domainname2.com, d.domainname2.com goes to HOST2.

    How can i do above scenario? im having no luck ;(  getting alot of squid access control problems

    To answer my own question;
    It's not harder then adding another "." infront of the "*" like this;

    WEBAPP1;;http://**..**domainname1.com



  • Hi again!

    Another problem,  this time with basic auth. For some reason its turned off with squid.

    See this info;

    If the content on the web servers is password protected then you need to tell the proxy to trust your web server with authentication credentials. This is done via the login= option to cache_peer. Normally you would use login=PASS to have the login information forwarded. The other alternatives is meant to be used when it's the reverse proxy which processes the authentication as such but you like to have information about the authenticated account forwarded to the backend web server.

    From http://wiki.squid-cache.org/SquidFaq/ReverseProxy

    Basically "login=PASS" flag is needed in the conf file(/usr/local/etc/squid/squid.conf)

    cache_peer 10.168.5.13 parent 80 0 proxy-only no-query login=PASS originserver name=MYHOST1

    I tried edit the confi file and restart(/usr/local/etc/rc.d/squid restart)  and my basic auth on webpage starteed working again.

    We need a flag in the UI for this, editing the conf file manually it not a good idea.



  • it's in the next version…
    already fixed...
    just reinstall the package



  • @trendchiller:

    it's in the next version…
    already fixed...
    just reinstall the package

    i installed package 2-3 days ago, using squid-reverse 2.7.9_2

    browsing thru /usr/local/pkg/squid.inc  i can see "login=PASS" in https peers, but not for http

    I added "login=PASS"  to this code;

    if (($cfg[0]) != '' && ($cfg[1]) != '' && ($cfg[2]) != ''){
                        $conf .= "cache_peer {$cfg[1]} parent {$cfg[2]} 0 proxy-only no-query login=PASS originserver ";

    and it does what i want  :)



  • that's also what i did ;-)





  • yes…
    it's comitted...



  • squid-reverse 3.1.10_02 is released now based on squid 3.1.19
    the features are mostly the same…
    ews is supported but still a bit buggy due to some squid issues, which are known to the squid-team and a ticket is opened for this... (it seems that just Apple-devices are concerned - the connection lasts longer to be established... but then works... tested with imac and outlook 2011 for mac)

    have fun !



  • Nice work trendchiller!!

    I haven't removed squid3 to try squid-reverse yet.. Probably will once I hear some feedback from other users. I did replace the binaries from squid3 with squid 3.1.19.. Received some ssl errors but was able to fix by running '/usr/local/libexec/squid/ssl_crtd -c -s /var/squid/lib/ssl_db' after creating dir /var/squid/lib

    Question, i've been using pound as a reverse proxy for over a year now because it can also handle https/ssl traffic. I'm confused if squid's reverse proxy function can do https/ssl. In your example from page 1, looks like it does but haven't noticed anyone trying it. Cause if it does, I can get rid of pound and use this package as a proxy/reverse-proxy server.  Let me know, thanks in advance

    Stephen


Log in to reply