Varnish



  • Varnish is a very fast reverse proxy, it can reduce 10 to 30 times your web server load.

    Version 0.8.6.x for pfsense implements new features and many bug fixes.

    Fell free to test and speed up your site.



  • marcelloc,

    Just want to say thanks for porting this to i386 also! I'm not ready to go amd64 on my box yet, dont want to lose a years worth of RRD data.

    I've been using pound as my reverse-proxy and it works pretty good but wanted to try Varnish since its a package for pfSense.



  • Thank you for your feedback.

    Varnish is not easy to configure, so I've tried to improve GUI for a easy and user friendly configuration.



  • @marcelloc:

    Thank you for your feedback.

    Varnish is not easy to configure, so I've tried to improve GUI for a easy and user friendly configuration.

    Your welcome, packages like this make pfSense even more great as a fw/edge router

    I've noticed its not easy to configure… I've tried a few times and can't seem to get it to work... Got as far to have it redirect traffic to an embedded device but it didn't display the page.. Like the java-script didn't go thru. Tried to have it go to a windows IIS server... that was a go-no... My IIS server picks the web site based on the FQDN, Varnish kept saying that the backend is down.

    I just need time to mess around and look at example configs to get it running for me..

    Been hoping someone would create a package for pound... its really easy to install and configure... Down the road i may try to create my own interface within pfSense for it... but that road may be a long way away.



  • configure your web serves on backends with no map url.

    on settings, enable varnish, set listening port, enable all basic options.
    memory cache is faster, go as far as your system memory can.
    at load balance, include the host header of your site, with client balance method and whitch backends hosts to use.






  • Thank you! That was very helpful, I wasn't using LB to get it to work.. I was mapping URLs on the backend page… I am having an issue tho. It seems that every request is going to  the first backend I create. My test setup has 2 backends. In LB i added 3 entries, 2 goto the same backend depending on the url I type.. At first i thought it was a wildcard but all traffic goes to the same backend.. I would like to use a wildcard for one backend as this is the windows box with different sites on it. Below is my config.. Going to test more in case i'm doing something wrong

    btw, i'm using port 81.. I created a nat/fw rule to port forward port 81 to 127.0.0.1:81 and i'm using i386

    
    # Varnish configuration file
    # Automatically generated by the pfSense package system
    
    # This file is located in /var/etc/default.vcl
    
    sub vcl_error {
    	set obj.http.Content-Type = "text/html; charset=utf-8";
    synthetic {"
    
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    
    <title>"} obj.status " " obj.response {"</title>
    
     **We are very sorry but an error occurred during this request.** 
    
    Please press refresh in your browser to try again.
    
    Varnish Error "} obj.status " " obj.response {"
    
    "} obj.response {"
    
    ### Guru Meditation:
    
    XID: "} req.xid {"
    
    "};
    return(deliver);
    
    }
    
    backend TED5000BACKEND {
    	.host = "192.168.0.15";
    	.port = "80";
    	.first_byte_timeout = 5s;
    	.connect_timeout = 2s;
    	.probe = {
    		.url = "/Footprints.html";
    		.interval = 3600s;
    		.timeout = 5s;
    		.window = 5;
    		.threshold = 1;
    	}
    }
    
    backend WHSBOXBACKEND {
    	.host = "192.168.0.100";
    	.port = "80";
    	.first_byte_timeout = 5s;
    	.connect_timeout = 2s;
    	.probe = {
    		.url = "http://xxxx.homeip.net:81";
    		.interval = 3600s;
    		.timeout = 5s;
    		.window = 5;
    		.threshold = 1;
    	}
    }
    
    director TED5000 client {
    	{
    		.backend = TED5000BACKEND;
    		.weight = 100;
    	}
    }
    
    director TED client {
    	{
    		.backend = TED5000BACKEND;
    		.weight = 100;
    	}
    }
    
    director MainWeb client {
    	{
    		.backend = WHSBOXBACKEND;
    		.weight = 100;
    	}
    }
    
    sub vcl_recv {
    
    	#BASIC VCL RULES
    	#Disable session cache
    	if (req.http.Cookie && req.http.Cookie ~ "(JSESSION|PHPSESSID)"){return(pass);}
    	if (req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache"){return(pass);}
    
    	#Disable post cache
    	if (req.request == "POST"){return(pass);}
    
    	#Enable image cache
    	if (req.request=="GET" && req.url ~ "\.(css|js|gif|jpg|jpeg|bmp|png|ico|img|tga|wmf)$") {remove req.http.cookie;return(lookup);}
    
    	#Fix gzip compression
    	if (req.http.Accept-Encoding) {
    	if (req.url ~ "\.(gif|jpg|jpeg|bmp|png|ico|img|tga|wmf|gz|tgz|bz2|tbz|mp3|ogg)$") {remove req.http.Accept-Encoding;}
    	else if (req.http.Accept-Encoding ~ "gzip") {set req.http.Accept-Encoding = "gzip";}
    	else if (req.http.Accept-Encoding ~ "deflate") {set req.http.Accept-Encoding = "deflate";}
    	else {remove req.http.Accept-Encoding;}}
    
    	#Be rfc2616 compliant
    	if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" &&
    	    req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") {return(pipe);}
    
    	#set X-forward
    	set req.http.X-Forwarded-For = client.ip;
    
    	if (req.http.host == "ted5000.xxxx.homeip.net") {
    		set req.backend = TED5000;
    		set req.http.host = "ted5000.xxxx.homeip.net";
    	} else if (req.http.host == "ted.xxxx.homeip.net") {
    		set req.backend = TED;
    		set req.http.host = "ted.xxxx.homeip.net";
    	} else if (req.http.host == "xxxx.homeip.net") {
    		set req.backend = MainWeb;
    		set req.http.host = "xxxx.homeip.net";
    	} else if (req.http.host == "") {
    		set req.backend = TED5000BACKEND;
    		set req.http.host = "";
    	} else if (req.http.host == "") {
    		set req.backend = WHSBOXBACKEND;
    		set req.http.host = "";
    	} 
    
    	if (req.request != "GET" && req.request != "HEAD") {
    		return(pipe);
    	}
    	return(lookup);
    }
    
    sub vcl_pipe {
    
    	# If we don't set the Connection: close header, any following
    	# requests from the client will also be piped through and
    	# left untouched by varnish. We don't want that.
    	set req.http.connection = "close";
    	# Note: no "pipe" action here - we'll fall back to the default
    	# pipe method so that when any changes are made there, we
    	# still inherit them.
    
    }
    
    


  • varnish has two backend check methods, probe url and request

    To use probe url, put a /test.htm. Varnish will check helth on http:/ip_address/teste.htm

    To use a full html request put a complete url  like http://www.mysite.com/test.htm (important for my preg_match check)
    In request method, varnish send a http request to to your site, with full header definition.
    It is usefull  when using host header or mod_security on your web server.

    Varnish by default listen on all interfaces, you do not need to nat from wan to localhost



  • I removed the NAT statement and opened port 81 on the WAN address. The probe checks are working for me, i increased their time and it seems it was checking every couple of seconds. I have one that check for a html page and other that does a FQDN request.

    The problem i'm seeing is that every request is defaulting to the first backend server i created. They are not going to the second backend server even tho the FQDN matches in the LD setup. Could it be that i'm using the same domain? xxxx.homeip.net?  I'm trying to get ted.xxxx.homeip.net to go to backend server 1; photo.xxxx.homeip.net and xxxx.homeip.net to go to backend server 2. Backend server 1 is an embedded device that only has its web interface. While backend server 2 has multiple web sites based on the host request.

    thanks in advance



  • try setting only backend2 in LB to see if its helth check is working

    If you get only 503 error, it means that backend2 helth check is not working.

    The client director in load balance sends a client ip address to the same web server.

    if you don't have sessions, try random or round-robin.

    if you are using fqdn in backend probe url, see in 'view configuration' if you have probe_url='www.xxx.home.net' or .request.

    You will need to include a page to test ex: 'www.xxx.home.net/test.htm' to get a .request working.



  • The health checks have always been fine.. It just doesn't want to route traffic to the second backend server.. I've played with all the settings and was only able to kinda to get it work by deleting the LBs, and adding the url on the backend page with the regex option but it doesn't display the page correctly. Now if i use LB and make that backend first, the page is displayed correctly.

    I feel that the order in the If statment need to be adjusted in the config or because its the same domain it matches the first time and doesn't go down the list to check host names in the FQDN

    Config:

    
    # Varnish configuration file
    # Automatically generated by the pfSense package system
    
    # This file is located in /var/etc/default.vcl
    
    sub vcl_error {
    	set obj.http.Content-Type = "text/html; charset=utf-8";
    synthetic {"
    
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    
    <title>"} obj.status " " obj.response {"</title>
    
     **We are very sorry but an error occurred during this request.** 
    
    Please press refresh in your browser to try again.
    
    Varnish Error "} obj.status " " obj.response {"
    
    "} obj.response {"
    
    ### Guru Meditation:
    
    XID: "} req.xid {"
    
    "};
    return(deliver);
    
    }
    
    backend WHSBOXBACKEND {
    	.host = "192.168.0.100";
    	.port = "80";
    	.first_byte_timeout = 5s;
    	.connect_timeout = 2s;
    	.probe = {
    		.request =
    			"GET /myip.php HTTP/1.1"
    			"Host: xxx.homeip.net:81"
    			"Connection: Close";
    		.interval = 1s;
    		.timeout = 1s;
    		.window = 5;
    		.threshold = 5;
    	}
    }
    
    backend TED5000BACKEND {
    	.host = "192.168.0.15";
    	.port = "80";
    	.first_byte_timeout = 5s;
    	.connect_timeout = 2s;
    	.probe = {
    		.url = "/Footprints.html";
    		.interval = 1s;
    		.timeout = 1s;
    		.window = 5;
    		.threshold = 5;
    	}
    }
    
    director MainWeb client {
    	{
    		.backend = WHSBOXBACKEND;
    		.weight = 100;
    	}
    }
    
    director PhotosWeb client {
    	{
    		.backend = WHSBOXBACKEND;
    		.weight = 100;
    	}
    }
    
    director TED5000 client {
    	{
    		.backend = TED5000BACKEND;
    		.weight = 100;
    	}
    }
    
    director TED client {
    	{
    		.backend = TED5000BACKEND;
    		.weight = 100;
    	}
    }
    
    sub vcl_recv {
    
    	#BASIC VCL RULES
    	#Disable session cache
    	if (req.http.Cookie && req.http.Cookie ~ "(JSESSION|PHPSESSID)"){return(pass);}
    	if (req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache"){return(pass);}
    
    	#Disable post cache
    	if (req.request == "POST"){return(pass);}
    
    	#Enable image cache
    	if (req.request=="GET" && req.url ~ "\.(css|js|gif|jpg|jpeg|bmp|png|ico|img|tga|wmf)$") {remove req.http.cookie;return(lookup);}
    
    	#Fix gzip compression
    	if (req.http.Accept-Encoding) {
    	if (req.url ~ "\.(gif|jpg|jpeg|bmp|png|ico|img|tga|wmf|gz|tgz|bz2|tbz|mp3|ogg)$") {remove req.http.Accept-Encoding;}
    	else if (req.http.Accept-Encoding ~ "gzip") {set req.http.Accept-Encoding = "gzip";}
    	else if (req.http.Accept-Encoding ~ "deflate") {set req.http.Accept-Encoding = "deflate";}
    	else {remove req.http.Accept-Encoding;}}
    
    	#Be rfc2616 compliant
    	if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" &&
    	    req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") {return(pipe);}
    
    	#set X-forward
    	set req.http.X-Forwarded-For = client.ip;
    
    	if (req.http.host == "xxx.homeip.net") {
    		set req.backend = MainWeb;
    		set req.http.host = "xxx.homeip.net";
    	} else if (req.http.host == "photos.xxx.homeip.net") {
    		set req.backend = PhotosWeb;
    		set req.http.host = "photos.xxx.homeip.net";
    	} else if (req.http.host == "ted5000.xxx.homeip.net") {
    		set req.backend = TED5000;
    		set req.http.host = "ted5000.xxx.homeip.net";
    	} else if (req.http.host == "ted.xxx.homeip.net") {
    		set req.backend = TED;
    		set req.http.host = "ted.xxx.homeip.net";
    	} 
    
    	if (req.request != "GET" && req.request != "HEAD") {
    		return(pipe);
    	}
    	return(lookup);
    }
    
    sub vcl_pipe {
    
    	# If we don't set the Connection: close header, any following
    	# requests from the client will also be piped through and
    	# left untouched by varnish. We don't want that.
    	set req.http.connection = "close";
    	# Note: no "pipe" action here - we'll fall back to the default
    	# pipe method so that when any changes are made there, we
    	# still inherit them.
    
    }
    
    

    Log of it going to wrong backend:

    
    0 Backend_health - WHSBOXBACKEND Still healthy 4--X-RH 5 5 5 0.002687 0.002806 HTTP/1.1 200 OK
        0 Backend_health - TED5000BACKEND Still healthy 4--X-RH 5 5 5 0.080917 0.041358 HTTP/1.1 200 OK
        0 CLI          - Rd ping
        0 CLI          - Wr 200 PONG 1312995470 1.0
        0 Backend_health - WHSBOXBACKEND Still healthy 4--X-RH 5 5 5 0.009365 0.004446 HTTP/1.1 200 OK
        0 Backend_health - TED5000BACKEND Still healthy 4--X-RH 5 5 5 0.027403 0.037869 HTTP/1.1 200 OK
        0 Backend_health - WHSBOXBACKEND Still healthy 4--X-RH 5 5 5 0.002578 0.003979 HTTP/1.1 200 OK
        3 SessionOpen  c xx.xx.xxx.xxx 47164 :81
        3 ReqStart     c xx.xx.xxx.xxx 47164 1374470270
        3 RxRequest    c GET
        3 RxURL        c /
        3 RxProtocol   c HTTP/1.1
        3 RxHeader     c Host: ted.xxx.homeip.net:81
        3 RxHeader     c User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
        3 RxHeader     c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
        3 RxHeader     c Accept-Language: en-us,en;q=0.5
        3 RxHeader     c Accept-Encoding: gzip, deflate
        3 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
        3 RxHeader     c DNT: 1
        3 RxHeader     c Connection: keep-alive
        3 RxHeader     c Cookie: ASP.NET_SessionId=axcuvqsrwb2ga0azruivwovm
        3 VCL_call     c recv
        3 VCL_return   c lookup
        3 VCL_call     c hash
        3 VCL_return   c hash
        3 Hit          c 1374470251
        3 VCL_call     c hit
        3 VCL_return   c deliver
        3 Length       c 151
        3 VCL_call     c deliver
        3 VCL_return   c deliver
        3 TxProtocol   c HTTP/1.1
        3 TxStatus     c 302
        3 TxResponse   c Found
        3 TxHeader     c Cache-Control: private
        3 TxHeader     c Content-Type: text/html; charset=utf-8
        3 TxHeader     c Location: https://ted.xxx.homeip.net/remote
        3 TxHeader     c Server: Microsoft-IIS/7.5
        3 TxHeader     c X-AspNet-Version: 4.0.30319
        3 TxHeader     c X-Powered-By: ASP.NET
        3 TxHeader     c Content-Length: 151
        3 TxHeader     c Date: Wed, 10 Aug 2011 16:57:52 GMT
        3 TxHeader     c X-Varnish: 1374470270 1374470251
        3 TxHeader     c Age: 74
        3 TxHeader     c Via: 1.1 varnish
        3 TxHeader     c Connection: keep-alive
        3 ReqEnd       c 1374470270 1312995472.138645411 1312995472.139071941 0.001661062 0.000177383 0.000249147
        0 Backend_health - TED5000BACKEND Still healthy 4--X-RH 5 5 5 0.026161 0.034942 HTTP/1.1 200 OK
        3 Debug        c "herding"
        0 Backend_health - WHSBOXBACKEND Still healthy 4--X-RH 5 5 5 0.002626 0.003641 HTTP/1.1 200 OK
        0 Backend_health - TED5000BACKEND Still healthy 4--X-RH 5 5 5 0.026089 0.032729 HTTP/1.1 200 OK
        0 CLI          - Rd ping
        0 CLI          - Wr 200 PONG 1312995473 1.0
        0 Backend_health - WHSBOXBACKEND Still healthy 4--X-RH 5 5 5 0.002592 0.003379 HTTP/1.1 200 OK
        0 Backend_health - TED5000BACKEND Still healthy 4--X-RH 5 5 5 0.025051 0.030809 HTTP/1.1 200 OK
        0 Backend_health - WHSBOXBACKEND Still healthy 4--X-RH 5 5 5 0.002898 0.003259 HTTP/1.1 200 OK
        0 Backend_health - TED5000BACKEND Still healthy 4--X-RH 5 5 5 0.026342 0.029692 HTTP/1.1 200 OK
        0 Backend_health - WHSBOXBACKEND Still healthy 4--X-RH 5 5 5 0.002051 0.002957 HTTP/1.1 200 OK
        0 CLI          - Rd ping
        0 CLI          - Wr 200 PONG 1312995476 1.0
        0 Backend_health - TED5000BACKEND Still healthy 4--X-RH 5 5 5 0.025777 0.028713 HTTP/1.1 200 OK
    
    

    I'll take a lot at varnish site sometime today and see if they have any pointers to help us out



  • I will do some tests today too.

    Backend and load balance definitions are set after basic rules.

    Probably that's the mistake.

    I will push a new release fixing some bugs, including this.



  • I've pushed package version 0.8.7 with many improvements and bug fixes.

    test and see if now your setup works as you expect

    some improvements

    • better backend configuration with grace time and regex options if you don't plan to use Load balance.
    • better Lb settins with regex for host/url and grace time.
    • restart on erro if you get error from apache, varnish retries x times before sending error for user.
    • saintmode and grace time for better cache control.
    • advanced startup option for a very custom and advanced setup.


  • thanks marcelloc! I don't see your updates yet but i think one of the core dev has to merge them in. looking github, it states changes were made about 5 hours ago.. when i check what changes were made, its one jimp did 20 hours ago…

    looking forward to testing this



  • This is working much better… I was able to get it to work the way I want it to somewhat... I'm not keen on the backend server checks because if a server prompts you for a username and password, it fails the check... No biggy for now.

    For the main webserver, I added the host names within the Backend setup. For the other devices I had to use the LB setup but i'm able to have FQDNs route to correct the IP now. It seems to be working as a reverse-proxy now... I think some tweeks is still needed for the recerse-proxy function but i have to finish reading the Varnish docs first.

    I think a good tweak is to add to both to the Backend tab and the LB tab, a way to change the the order of the entries.



  • Hi all,

    Varnish package 0.8.8 is out whith fail over for load balance pools.

    There is also a new option to force "no cache" to a specific mapping or director.

    att,
    Marcello Coutinho



  • Thanks for the update Marcello!

    I did notice a bug with the package. If I disable varnish and restart the router, varnish starts-up again. The file varnish.sh should be deleted when you disable varnish to prevent it from starting when you reboot the router. Once you re-enable varnish, it should create the file and startup.



  • Thanks again for your feedback, I will check this option.



  • **varnish 0.8.9 is out.

    Actual state:**
    Good for production servers.
    Just looking if are any main options for gui until release candidate.

    main improvements:
    Per user session cache control
    better static cache options
    new sync gui
    Application and client cache control are now respected by varnish.

    main fixes:
    JSESSION verification before cookies are set.
    reload sync between varnish boxes works again.
    varnish will not start after a reboot if it is disabled.

    TODO:
    PURGE gui options
    listen on interface options
    a way to change backends and directors order in gui
    ssl suport via 'hidden' pound reverse proxy (maybe)

    att,
    Marcello Coutinho



  • Thanks marcelloc for all your help. This thread has been very valuable in getting the service up and running correctly. I agree with previous post about somehow sorting the order of entries for the director.



  • I seem to have found a minor bug in the GUI

    In the Custom VCL tab, whatever is input into the vcl_pipe_early windows i written into the vcl_fetch_early window as well as the vcl_pipe_early window. That code is also put into the vcl_fetch clause of the config file.

    Same with vcl_pipe_late -> vcl_fetch_late

    I have tested on two different install ins both FireFox and Safari.  PfSense ver 2.0 Varnish package 0.8.9.1

    The Varnish package works great otherwise.



  • thanks for your feedback, i'll take a look

    EDIT

    The Custom options are placed correctly in varnish conf file.

    Can you check this on your setup?



  • Varnish 0.8.9.2 is out, now with dashboard widget.



  • I've sucessfully compiled varnish 3.0.2 to pfsense.

    So, packager version for varnish 2.1.5 gets its release status and now varnish3 is out in beta stage.

    There are many improvements between 2.1.5 and 3.0.2, one of the major features included is streaming support.

    Take a look at varnish website to see how this tool can help you.

    If you have time, give it a try and feedback. ;)



  • Hi Marcelloc, I tried to get this working last night as I stated in my other post "Topic: Using DNS in PFSense to split traffic based on host request".

    I feel like I understand the principle pretty well but I can not get it to work. The attachment shows my config file. I was not sure what to do with the NAT settings so I tried with them enabled and disabled but it still did not work.

    The only options I setup where two Backends and two LB Directors. Am I missing something?

    config.txt



  • check backend status on varnish widget.

    You must disable nat for port 80, varnish will do the job.

    Until you get this working, publish varnish on port 81 for example, create a wan rule to permit port 81 communication and do tests.


Locked