WPAD Setup help [Solved]

aGeekhere · Jun 4, 2015, 5:18 AM

Hi chris4916 I will try to be more clear with the current issue I have.

The wpad (goal is to allow local sites to bypass the proxy but make all external sites to go through the proxy)

function FindProxyForURL(url, host) {
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
        return "DIRECT";

    return "PROXY 192.168.1.1:3128";
}

Current status
pfsense is now passing the wpad onto the browsers (chrome and ie for now through the DNS forwarder).

http://www.lagado.com/proxy-test shows that the browsers are connected through a proxy (good)

sites in the squidguard blocklist are being blocked for both http and https connections (good)

Current issue
Normal websites (sites which are not blocked form squidguard) are not loading correctly (Images are missing, layout is not fully loaded and some sites do not load at all).

"with this wpad example the issue is local address are blocked" mean?
I was just outlining that if you use that example wpad from https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid that local addresses are blocked and by blocked i mean I get a proxy error (can't remember what the error was).

What you are describing shows that your WPAD set-up is working as expected
That is correct the WPAD is now working (however now have another issue with webpages not loading correctly)

if you set-up proxy.pac to do direct for internal servers, you can reach it.
I can reach them now by using the following wpad

function FindProxyForURL(url, host) {
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
        return "DIRECT";

    return "PROXY 192.168.1.1:3128";
}

Let me know if I need to expand on anything

chris4916 · Jun 4, 2015, 5:34 AM

That's clearer (I hope :))

So your current issue has nothing to do with WPAD but with proxy (Squid) that is not loading pages as expected (or Squidguard blocking some parts of these pages)
Did you look at squid log content already?

Keep in mind that quite often web pages as you see it through your browser are built from different sources.
I would suggest that you perform some tests without any Squidguard rules or even Squid ACL, just to ensure that from HTTP standpoint, everything is OK btween your browser and external web servers, then introduce Squid ACL if needed and Squidguard rules so that you can, perhaps, determine, what's wrong here.

aGeekhere · Jun 4, 2015, 6:18 AM

I have updated the first post, I now use the strikethrough to show old settings. Very strange issue this is now, squidguard is not showing any blocked items for non blocked sites and even if I turn it off or allow all to squidguard I get the same issue (sites not loading correctly, missing content, not loading at all AND very very slow webpage loading times).

Stuck atm (I wounder if it is squid3 that is the issue);

Any ideas on what I should try next?

chris4916 · Jun 4, 2015, 6:35 AM

@aGeekHere:

I have updated the first post, I now use the strikethrough to show old settings.

From personal viewpoint, this is not something I use to do because it makes next posts most of the time out of context.

Very strange issue this is now, squidguard is not showing any blocked items for non blocked sites and even if I turn it off or allow all to squidguard I get the same issue (sites not loading correctly, missing content, not loading at all AND very very slow webpage loading times).

Stuck atm (I wounder if it is squid3 that is the issue);

I'm not using Squid on pfSense but reading various threads here and there, it looks like there is a significant amount of problems with Squid on pfSense 2.x
At least, you have clarified that Squidguard is not the issue here, neither WPAD ;)
Now you could perhaps focus on Squid logs…

aGeekhere · Jun 4, 2015, 9:21 AM

reinstalled squid gui components, cleared the cache (one the user computers and squids cache), and now it works :)

Will test it for a few days to see how i go.

;D

[Update]
If i have no issues with it i will write up a how to

aGeekhere · Jun 17, 2015, 6:31 AM

I just want to update something here.
if you are having issues with webpages only half loading (which is what I had), then you must first delete your squid cache then it should fix that issue.

KOM · Jun 17, 2015, 1:12 PM

if you are having issues with webpages only half loading

This can also by symptomatic of an IP6/IP4 preference issue with dual-stack sites.

aGeekhere · Jul 19, 2015, 1:21 AM

Hi all,

I found an issue where some programs have an issue with the global "automatically detect settings" option (some programs with this issue do not even have a proxy setting setup page) and still want to go through port 80 (I think this could also be what is causing downthemall to only connect to 1 segment because when I open port 80 I can connect to more segments).

So sometimes I have to reopen port 80 to let the traffic through temporary.

I read somewhere that a NAT rule could be created to catch any leftover traffic going to port 80 and redirect it to port 3128.

Tired playing around with a NAT rule
LAN - TCP - LAN address - 80 - WAN address - 80 - 192.168.1.1 - 3128
And in rules i put it at the top.

However the traffic is not redirected.
Tested with a troubled program using

no proxy
automatically detect settings
And even setting the proxy setting with the port set to 80

Any thoughts on this?

User43617 · Jul 20, 2015, 3:49 AM

@aGeekHere:

~~2. Disable DNS Resolver~~ Updated needs to be on

3.Configure DNS Service
add new Host Overrides
Host: wpad
Domain: mylocaldomain.local
IP Address: 192.168.1.1
Description: WPAD Autoconfigure Host
save
Updated change Listen Port to 3128
Enable DNS forwarder
save

Quick question, for clarification on the above quote, please:
I currently have DNS Resolver off and DNS forwarder on with the above Host Override configured on the DNS Forwarder.

Which one did you apply the Host Override configuration?

Which one did you change to listen on port 3128 (same as Squid3, unless you changed that. I don't think you did, based on your proxy.pac code).

I add the Firewall rules for blocking HTTP and HTTPS and I can't load internet pages. I can still get to local servers. Nothing's being filtered by Squidguard. Or, at least, it's not showing up in the logs. Wondering if that's my problem now.

aGeekhere · Jul 21, 2015, 12:59 AM

It is the DNS forwarder that needs the Host Override configuration.

irajames · Aug 29, 2015, 8:27 AM

@User43617:

@aGeekHere:
~~2. Disable DNS Resolver~~ Updated needs to be on

3.Configure DNS Service
add new Host Overrides
Host: wpad
Domain: mylocaldomain.local
IP Address: 192.168.1.1
Description: WPAD Autoconfigure Host
save
Updated change Listen Port to 3128
Enable DNS forwarder
save
Quick question, for clarification on the above quote, please:
I currently have DNS Resolver off and DNS forwarder on with the above Host Override configured on the DNS Forwarder.

Which one did you apply the Host Override configuration?

Which one did you change to listen on port 3128 (same as Squid3, unless you changed that. I don't think you did, based on your proxy.pac code).

I add the Firewall rules for blocking HTTP and HTTPS and I can't load internet pages. I can still get to local servers. Nothing's being filtered by Squidguard. Or, at least, it's not showing up in the logs. Wondering if that's my problem now.

I'm also wondering do I put listen to 3128 in DNS Forwarder or in DNS Resolver?

aGeekhere · Aug 29, 2015, 9:28 AM

I'm also wondering do I put listen to 3128 in DNS Forwarder or in DNS Resolver?

see above post :)

And while I am here, there is an issue that remains with this setup is that programs that have no proxy settings and want to connect directly will get blocked by the port 80 rule.

If someone has any advice on how to solve this i'll update the first post.

KOM · Aug 30, 2015, 1:55 AM

If someone has any advice on how to solve this i'll update the first post.

Add a firewall rule(s) to allow specified LAN IPs to connect to specified destination IPs via 80. Put it above the HTTP/S block. Use an alias to hold the LAN IPs of your clients that have apps that don't support proxy, and an alias to hold all the IPs of the servers they need to talk to. That's it.

aGeekhere · Aug 30, 2015, 2:55 AM

and an alias to hold all the IPs of the servers they need to talk to

The problem is first finding the server that they need to talk to, then updating that rule when they change it. If the destination server is dynamic then you will never know the destination server.

But I guess that is all we can do.

KOM · Aug 31, 2015, 1:55 PM

But I guess that is all we can do.

Them's the breaks.

enrique.perezrul · Sep 17, 2015, 2:15 AM

@aGeekHere:

~~2. Disable DNS Resolver~~ Updated needs to be on

3.Configure DNS Service
add new Host Overrides
Host: wpad
Domain: mylocaldomain.local
IP Address: 192.168.1.1
Description: WPAD Autoconfigure Host
save
Updated change Listen Port to 3128
Enable DNS forwarder
save

This is working on my machine but I will like to know if you can help me understand why do you need to enable DNS forwarder and set listening port to 3128. I have pfsense 2.2.4 DNS Resolver is enable by default and DNS forwarder is disable. I have created the host override on DNS resolver for it to work and enable DNS Forwarder service on port 3128.

Also why do you need the host override if dhcp is configured with the ip address and not a domain name? I will appreciate if you help me understand this.

aGeekhere · Sep 17, 2015, 3:20 AM

Hi enrique.perezrul

Hmm some hard questions there, will do my best, lets start with reading from the wiki

https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid

WPAD will take the domain name given to the machine, likely assigned by DHCP, and prepend wpad.. If the domain is example.com, it will look for wpad.example.com. This task may be accomplished with the DNS Forwarder/DNS Resolver in pfSense or with another internal DNS server used by client PCs.

why do you need to enable DNS forwarder

Because you need to create a Host Override for the wpad.

…set listening port to 3128

Because I wanted to use port 53 for the DNS Resolver (Will need to look more into why I used port 3128)

Also why do you need the host override if dhcp is configured with the ip address and not a domain name

Some web browsers use DNS to configure them others use dhcp so both are needed.

maverik1 · Sep 17, 2015, 7:32 PM

Looking for a well documented guide or video to help configure wpad. Do any exist?

chavarriaa · Sep 17, 2015, 7:43 PM

@maverik1:

Looking for a well documented guide or video to help configure wpad. Do any exist?

With this Post i configurate WPAD, but if you have problems, create your own post. Problems never are the same.

KOM · Sep 17, 2015, 7:45 PM

Looking for a well documented guide or video to help configure wpad. Do any exist?

Look at the post above yours, specifically the link to WPAD Autoconfigure for Squid.