Host header forwarding

baron164 · Jan 24, 2012, 3:02 PM

I am running several test servers at my home. Including Exchange and Sharepoint, these require https and http respectively to work properly. I want to deploy a Windows Home Server but for it to work externally it needs both https and http forwarded to it. I've thought about using a Forefront TMG server so I can do web filtering and have the Forefront server forward https requests to the correct server based on what host header is used. However I'm wondering if pfSense has something that might do that so I don't need to deploy yet another server.

marcelloc · Jan 24, 2012, 5:19 PM

host header based redirect can be done with varnish in http protocol.

https can be done with apache, but https certificates are assigned to ip address on server configuration.

you will need a wildcard certificate to reach this config without any certificate alerts to clients.

baron164 · Jan 25, 2012, 3:00 PM

I installed Varnish and I've been trying to get it to work with little luck. I want it to do host redirection for 3 different sites but it only wants to redirect the first site in the list.

marcelloc · Jan 25, 2012, 3:17 PM

Did you disabled nat on port 80 before starting varnish tests?

baron164 · Jan 25, 2012, 3:34 PM

I set a NAT rule to forward port 80 to the pfsense box itself. I had deleted all the 80 NAT/Rules but it wasn't doing anything until I added the one NAT 80 rule.

marcelloc · Jan 25, 2012, 3:37 PM

@baron164:

I set a NAT rule to forward port 80 to the pfsense box itself. I had deleted all the 80 NAT/Rules but it wasn't doing anything until I added the one NAT 80 rule.

varnish must works without nat on same port it listen on.

maybe you have a non working configuration so varnish could not start.

try to start via console to see startup erros.

/usr/local/etc/rc.d/varnish.sh start

baron164 · Jan 25, 2012, 3:52 PM

I don't know how to do that, I tried accessing the console with SSH but it won't let me start the service. I removed the 80 Nat and disabled and re-enabled the varnish service and now the primary redirect which is to /owa doesn't work but if I can hit the root directory

marcelloc · Jan 25, 2012, 3:50 PM

what errors do you get on console?

take a look on this post, may help you

http://forum.pfsense.org/index.php/topic,45496.0.html

baron164 · Jan 25, 2012, 4:00 PM

Here is the config

Varnish configuration file

Automatically generated by the pfSense package system

This file is located in /var/etc/default.vcl

sub vcl_error {
if (obj.status == 503 && req.restarts < 1) {
restart;
}

set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<title>"} obj.status " " obj.response {"</title>

We are very sorry but an error occurred during this request.

Please press refresh in your browser to try again.

Varnish Error "} obj.status " " obj.response {"

"} obj.response {"

Guru Meditation:

XID: "} req.xid {"

"};
return(deliver);

}

backend WebsiteBACKEND {

used in catch_all

.host = "10.10.0.8";
.port = "80";
.first_byte_timeout = 300s;
.connect_timeout = 25s;
.probe = {
.url = "/";
.interval = 1s;
.timeout = 1s;
.window = 5;
.threshold = 5;
}
}

backend WHSBACKEND {

used in catch_all

.host = "10.10.0.4";
.port = "80";
.first_byte_timeout = 300s;
.connect_timeout = 25s;
.probe = {
.url = "/";
.interval = 1s;
.timeout = 1s;
.window = 5;
.threshold = 5;
}
}

backend ExchangeBACKEND {

used in catch_all

.host = "10.10.0.6";
.port = "80";
.first_byte_timeout = 300s;
.connect_timeout = 25s;
.probe = {
.url = "/";
.interval = 1s;
.timeout = 1s;
.window = 5;
.threshold = 5;
}
}

sub vcl_recv {

#BASIC VCL RULES SETTING
#set client balance identity
set client.identity = req.url;

#set X-forward
set req.http.X-Forwarded-For = client.ip;

if (req.http.host == "www.mydomain.com") {
set req.backend = WebsiteBACKEND;
}
else if (req.http.host == "user.domain2.com") {
set req.backend = WHSBACKEND;
}
else if (req.http.host == "mail.mydomain.com/owa") {
set req.backend = ExchangeBACKEND;
}

#respect client wish to refresh the page
if (req.http.Pragma ~ "no-cache")
{
return(pass);
}

#BASIC VCL RULES ACTIONS
#Disable session cache
if (req.http.Cookie && req.http.Cookie ~ "(PHPSESSID|phpsessid)") {
return(pass);
}
if (req.http.Cookie && req.http.Cookie ~ "(JSESSION|jsession)") {
return(pass);
}
if (req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
return(pass);
}
#Enable static cache
if (req.request=="GET" && req.url ~ ".(css|js|txt|zip|pdf|rtf|flv|swf|html|htm)$") {
unset req.http.cookie;
return(lookup);
}
if (req.request=="GET" && req.url ~ ".(gif|jpg|jpeg|bmp|png|ico|img|tga|wmf|mp3|ogg)$") {
unset req.http.cookie;
return(lookup);
}
if (req.request != "GET" && req.request != "HEAD") {return(pipe);}

return(lookup);
}

sub vcl_pipe {

If we don't set the Connection: close header, any following

requests from the client will also be piped through and

left untouched by varnish. We don't want that.

set req.http.connection = "close";

Note: no "pipe" action here - we'll fall back to the default

pipe method so that when any changes are made there, we

still inherit them.

}

sub vcl_fetch {

#Disable cache when backend is starting a session
if (beresp.http.Set-Cookie && beresp.http.Set-Cookie ~ "(PHPSESSID|phpsessid)") {
return(pass);
}
if (beresp.http.Set-Cookie && beresp.http.Set-Cookie ~ "(JSESSION|jsession)") {
return(pass);
}
if (beresp.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
return(pass);
}

Varnish respects the wishes of the backend application.

if (beresp.http.Pragma ~ "no-cache" || beresp.http.Cache-Control ~ "(no-cache|no-store|private)") {
return(pass);
}
## If the request to the backend returns a code other than 200, restart the loop
## If the number of restarts reaches the value of the parameter max_restarts,
## the request will be error'ed. max_restarts defaults to 4. This prevents
## an eternal loop in the event that, e.g., the object does not exist at all.
if (beresp.status != 200 && beresp.status != 403 && beresp.status != 404 &&
beresp.status != 303 && beresp.status != 302 && beresp.status != 301 && beresp.status != 401 ) {
restart;
}

return(deliver);
}

sub vcl_deliver {
##set resp.http.X-Served-By = server.hostname;
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
set resp.http.X-Cache-Hits = obj.hits;
} else {
set resp.http.X-Cache = "MISS";
}
return(deliver);
}

marcelloc · Jan 25, 2012, 4:24 PM

  if (req.http.host == "www.mydomain.com") {
      set req.backend = WebsiteBACKEND;
   } 
      else if (req.http.host == "user.domain2.com") {
      set req.backend = WHSBACKEND;
   } 
      else if (req.http.host == "mail.mydomain.com/owa") {
      set req.backend = ExchangeBACKEND;
   }

move more specific redirect to the first backend config

it should looks like this:

  if (req.http.host == "mail.mydomain.com/owa") {
      set req.backend = ExchangeBACKEND;
   } 
      else if (req.http.host == "user.domain2.com") {
      set req.backend = WHSBACKEND;
   } 
      else if (req.http.host == "www.mydomain.com") {
      set req.backend = WebsiteBACKEND;
   }

also, paste here the return from
/usr/local/etc/rc.d/varnish.sh start

baron164 · Jan 25, 2012, 4:33 PM

Where should I run /usr/local/etc/rc.d/varnish.sh start ? If I SSH into the box and go to option 8 (Shell) it won't let me. If I got to the developer shell then I can enter the command and nothing happens.

marcelloc · Jan 25, 2012, 4:51 PM

it should return this:

/usr/local/etc/rc.d/varnish.sh start


kern.ipc.nmbclusters: 65536
sysctl: kern.ipc.nmbclusters: Invalid argument
kern.ipc.somaxconn: 16384 -> 16384
kern.maxfiles: 131072 -> 131072
kern.maxfilesperproc: 104856 -> 104856
kern.threads.max_threads_per_proc: 4096 -> 4096
storage_malloc: max size 2048 MB.
Using old SHMFILE

If you get any erros, it will show too.

baron164 · Jan 25, 2012, 4:55 PM

I was able to run the stop/start command finally.

It gives me the following:

kern.ipc.nmbclusters: 65536
sysctl: kern.ipc.nmbclusters: Invalid argument
kern.ipc.somaxconn: 16384 -> 16384
kern.maxfiles: 131072 -> 131072
kern.maxfilesperproc: 104856 -> 104856
kern.threads.max_threads_per_proc: 4096 -> 4096
storage_malloc: max size 512 MB.
Using old SHMFILE

marcelloc · Jan 25, 2012, 4:58 PM

So, It's running.

Go on dashboard, add varnish widget to see backend status

they may look like this:

baron164 · Jan 25, 2012, 5:02 PM

Varnish is not among the list of widgets to add.

marcelloc · Jan 25, 2012, 5:03 PM

What varnish package version are you using? :o ???

baron164 · Jan 25, 2012, 5:03 PM

2.1.5 pkg v.1.0

marcelloc · Jan 25, 2012, 5:04 PM

That's the last one.

The widget must be there.

status-> dashboard -> plus button -> Varnish

baron164 · Jan 25, 2012, 5:06 PM

Here is the list I have.

Available Widgets

Captive Portal Status
Carp Status
Gateways
Gmirror Status
Installed Packages
Interface Statistics
Interfaces
Ipsec
Load Balancer Status
Firewall Logs
OpenVPN
Picture
Rss
Services Status
System Information
Traffic Graphs
Wake On Lan

baron164 · Jan 25, 2012, 5:30 PM

So after all this I re-added the Port 80 Nat rule that forwards 80 back into the pfsense box and everything started working properly. Well for the most part at least. I still have some bugs to work out but the sites are all showing up now.

I'm getting messages like this occasionally now when I try to visit the sites.

We are very sorry but an error occurred during this request.

Please press refresh in your browser to try again.

Varnish Error 503 Service Unavailable

Service Unavailable

Guru Meditation:

XID: 966503285