WPAD questions and problems

KOM

But now i wonder if it makes sense to use wpad cause you still have to configure the browser or devices cause autodiscover is not on by default.

All major browsers have shipped with auto-discovery enabled for several years now. Mobile devices may vary, but the trend is to enable auto-discovery.

I think it does not make a difference if i go to the browser settings to select autodiscover or to put a ip and port in there?

Sure, if you only have a tiny number of devices to worry about. Get back to me on your method when you have a LAN with a couple of hundred/thousand clients.

Then i was reading you can port forward any traffic directed at port 80 to the proxy (Squid). So why should i use wpad (which does not work) - i'm missing something?

What you're describing is called Transparent Mode. It's great until you need to intercept HTTPS traffic, which involves installing a trusted certificate on every client and then doing what's essentially a Man in the Middle attack.

Trust me, WPAD is the way to go. How is it not working for you?

MrGlasspoole

@KOM:

All major browsers have shipped with auto-discovery enabled for several years now.

All tutorials say the opposite.

@KOM:

What you're describing is called Transparent Mode.

This tut about wpad still says NAT redirect at the bottom: http://irj972.co.uk/articles/pfSense-WPAD-PAC-configuration

I did setup the vHosts package with:

wpad.mydomain.net

I have:

wpad.pfsense.mydomain.net
wpad.mydomain.net

in the DNS resolver.
In the DHCP server LAN i have:

252 txt http://wpad.mydomain.net/wpad.dat
252 txt http://wpad.mydomain.net/wpad.da
252 txt http://wpad.mydomain.net/proxy.pac

I can download the files if i point the browser to:

http://wpad.mydomain.net/wpad.dat

But Firefox and IE do not use the file with auto-discovery.

KOM

All tutorials say the opposite.

I'm not really looking to argue. I'm trying to help you with knowledge that I have which I know works for a fact and is recommended by every other volunteer here.

All of the major browsers most certainly do work with WPAD. I have seen a few instances where a user's browser had to be manually set from auto-detect to specified server and port, but this is rare in my experience. I don't know why you went wandering off with some random tutorial when pfSense already has a full document on how to get it working which I know works because I used it myself.

WPAD Autoconfigure for Squid

This tut about wpad still says NAT redirect at the bottom:

It is idiotic to have a proxy server online but leave ports 80 and 443 open. If users can go around the proxy then what is the point of having it in the first place?

MrGlasspoole

@KOM:

I don't know why you went wandering off with some random tutorial when pfSense already has a full document on how to get it working

The pfSense docu was the first thing i was looking at.
But it was not working and for a newbie allot of the docu is not clear/specific enough.
I'm not the only one who has problems get it running and after searching the forum i found
out that it is recommended to use a second webserver and lighttpd because if not you have
to use a certificate.

The tutorial i linked is just something i found when i was searching the web to find out
why i can't get it running.

As i wrote i can download the file with

http://wpad.mydomain.net/wpad.dat

but auto-discovery is not using it.

Why is that?
How can i figure out what the problem is?
How do i make sure clients do not bypass the proxy?

KOM

How do i make sure clients do not bypass the proxy?

Create a Ports Alias (Firewall - Aliases - Ports) called WebPorts or WWW_Ports and set it to 80,443. Create a firewall rule on LAN that blocks Source Any, Destination Any, Destination port range (Other) and then put your alias in the red box beside the (Other) combobox. See attached.

but auto-discovery is not using it. Why is that? How can i figure out what the problem is?

First block off the ports as shown above. Then manually set your browser to the proxy to ensure the proxy is working by going to a few sites. Can you show me your wpad.dat file? Loading the file and processing the file are two different things. If you have a bug in your code, then it won't work.

pf1.png_thumb
pf2.png_thumb

MrGlasspoole

Thanks KOM

The proxy is working if i use

http://wpad.mydomain.net/wpad.dat

in Firefox "Automatic proxy configuration URL"
and test it with http://www.lagado.com/proxy-test

wpad.dat:

function FindProxyForURL(url,host)
{
return "PROXY 192.168.0.1:3128";
}

KOM

The auto-discovery should work if you have a DNS entry for the host WPAD on your local domain, or a DHCP 252 entry in DHCP. On your DNS server, create a WPAD host entry and point it to your pfSense LAN IP address. Then every browser set to automatic discovery should be able to find it since they do a DNS lookup on wpad.YourDomain.foo and then load the wpad.dat file via HTTP from that host.

MrGlasspoole

As you can see in my second post the DNS and DHCP entries are already there.
I added now the Firewall rules - if i do that my Internet stops working… (sure Firefox is set to the proxy).

KOM

If you do an nslookup on WPAD, does it resolve to the proxy LAN address?

MrGlasspoole

nslookup wpad.mydomain.net:

Server:  pfsense.mydomain.net
Address:  192.168.0.1

Name:    wpad.mydomain.net
Address:  192.168.0.1

nslookup wpad:

Server:  pfsense.mydomain.net
Address:  192.168.0.1

Name:    wpad
Address:  192.168.0.1

nslookup wpad.pfsense.mydomain.net:

Server:  pfsense.mydomain.net
Address:  192.168.0.1

Name:    wpad.pfsense.mydomain.net
Address:  192.168.0.1

But if something would be wrong here why is the proxy working as long as i not block http in the firewall?

KOM

Because something isn't working. If the browser is set to auto-detect, then it will try to go straight out the gateway. If it can't, then it tries to detect the proxy using WPAD. When you unblock LAN, it can go straight out. When you block LAN, it can't go out so it tries to detect the proxy and use it. This is where your problem is. Either the browser isn't detecting the proxy at all, or it is and the proxy isn't working. Is your WebGUI using HTTP or HTTPS? If I remember, you can't use pfSense under HTTPS to host the WPAD file.

q54e3w

you can't use the default lightppd intstance to serve the WPAD file, its tied up to port 80 for serving webconfigurator stuff. You need the second lightppd instance. If you webconfigurator is running on port 80 and not a custom port you won't be able to bind it so you need a custom port for the webconfiguator lightppd instance in order for the second lightppd to server the file on port 80.

whats the output of "ps aux | grep "light"?

KOM

you can't use the default lightppd intstance to serve the WPAD file

I'm fairly sure that you can, considering that's exactly how I'm doing it and that's how it's documented ;D

From WPAD Autoconfigure for Squid

"Now upload that file to pfSense or another locally accessible web server with scp, or create it using the built-in file editor. The file must go in /usr/local/www/…"

Port 80 isn't "tied up" with WebGUI. It will serve the GUI as the default page, but if you give it an explicit URL then it will serve anything, including wpad.dat.

q54e3w

yes, you are right, sorry. I dont allow HTTP to my pfsense box. I'll keep out of it, you carry on….I suspect you are nearly there :)

KOM

yes, you are right, sorry.

Hey, no problem. I've never let a lack of knowledge or incorrect information stop me from trying to help someone. Even when I am wrong (and I've been wrong in these forums many times), I learn something. It bruises the ego a bit, but you become better for it. Thanks for contributing. A community is only as strong as its members.

MrGlasspoole

Step by step…

Forget the wpad for a while.
What i was saying is:
If i enable the firewall rule to block http and use 192.168.0.1:3128 (not wpad) in the browser then the internet stops working.
If i disable the rule then it works again and uses the proxy.

KOM

If you block 80/443 and manually set your browser to use the proxy at the specified address:port and nothing works and you're positive you didn't make a typo, your Squid install is broken. Look in your System log, as well as /var/squid/logs/access.log and cache.log.

Can you please remind me as to what version of pfSense and Squid you are using?

MrGlasspoole

Ok, found out something.
The whole time i was just using google for testing.
But this time i used another site and it's just HTTPS (google) that is not working if i enable the firewall rule.
HTTP works with pointing the browser to address:port.
BUT if i set the browser to auto-discovery then also HTTP is not working.

pfSense 2.2
squid3 3.4.10_2 pkg 0.2.6

KOM

Do you run IPv6?

MrGlasspoole

IPv6 is none in LAN and WAN.

And the output of ps aux | grep "light" is:

root    17216   0.0  0.3  50796   5888  -  S    12Feb15     0:25.97 /usr/local/sbin/lighttpd -f /var/etc/
root    89081   0.0  0.2  40392   4416  -  S    12Feb15     0:21.07 /usr/local/sbin/lighttpd -f /var/etc/
root    94646   0.0  1.5 232612  31596  -  I    10:10AM     0:00.22 php-fpm: pool lighty (php-fpm)
root    49032   0.0  0.1  18884   2356  0  S+   10:13AM     0:00.00 grep light

As i said i use the vHost package to host the wpad that i can run the WebGUI over HTTPS
but don't need a certificate for the wpad.

Tomorrow when i have time i will make some tests step by step with checking the logs after every step.
But today i have a date and no time ;-)