Wpad, squid3 not working

Legion

It's nothing except what's in the docs. I.e.:


function FindProxyForURL(url,host)
 {
 return "PROXY LAN RFC 1918 address:Squid port";
 }

obviously with my system's settings substituted in there, the same IP address and port as when I configure the client computer to manually use the proxy.

The wpad.dat file is correctly served if I point my browser at http://wpad.mydomain.com/wpad.dat. But with auto-detect it's like it never reads wpad.dat and never tries (or knows to try) to access my squid port and everything is blocked by my 80/443 rules.

mendilli

create three files named wpad.dat, wpad.da and proxy.pac in www folder?

set your firewall-rules, dns and dhcp settings as below, and remember you may have trouble with firefox auto network settings, so use explorer or chrome just to be sure

![firewall rules.jpg](/public/imported_attachments/1/firewall rules.jpg)

![firewall rules.jpg_thumb](/public/imported_attachments/1/firewall rules.jpg_thumb)

Legion

Thanks mendilli, I'll give it a try.

I created the three files in /usr/local/www, wpad.dat, wpad.da or proxy.pac (the last two both symbolic links) with 755 permissions (from memory?).

I had the DNS forwarder options set like that.

I think my firewall LAN rules are the same as that, but I'll check.

The dhcp custom boot options, are they exposed by default or did you have to do something like this? I don't remember seeing that option in the dhcp server GUI.

Regardless, I tried to edit a similar custom-proxy-server option in after reading an Ubuntu/dansguardian/squid/wpad article only to find it didn't work (with manual editing of dhcpd.conf, which was then overwritten as I changed configs, so not much use doing it that way). And I only tried to instruct it to serve wpad.dat, not wpad.da or proxy.pac.

Do you also need to instruct lighttpd of the mime type associations for the three files?

mendilli

The dhcp custom boot options, are they exposed by default or did you have to do something like this? I don't remember seeing that option in the dhcp server GUI.

they are not set by default, when you click on additional boot options you can add them one by one pressing (+) button.to use this you must enable dhcp on lan interface of course.clients using dhcp (auto network configuration) can get proxy informaiton via dhcp, I use it this way and it works

Legion

Hmm, still not working. Same settings as you:

When all that failed, I even tried a NAT rdr:

Still no luck, although with the rdr rule set I did get some Squid logs happening, bit like this:

This screenshot was actually taken a bit later and possibly shows some progress. Right before that screenshot, I had deleted my NAT rdr, deleted my LAN rules and started to apply them to the physical LAN interface, part of the bridge. Rules seem to have some effect when placed on that interface instead of (or as well as?) the bridged interface. I need to turn off the system now but will experiment more later.

From the last messing around I did, I suspect it's all down to my bridged interface, rules and NAT and I'll have to try and find a combination that works.

I'm using Chrome.

Legion

Still not working.

Squid works fine if I manually set the proxy to Squid's IP and port.

As soon as I select auto-configure and disable the explicit proxy setting, it falls over.

I tried to add rules above my "block all from LAN to port 80 and 443", rules that say "pass any from Squid to 80 or 443" and "pass any from LAN to Squid". I don't know that this is necessary.

One thing that struck me as noteworthy - in trying to lock my network down I've set static dhcp mappings for all my devices. The advanced options of "custom-proxy-server" pointing to wpad.dat only apply to the dynamic range of IP addresses that by nature of the configuration page can't overlap with the static IP addresses. So I can't allocate a static IP that sets the custom-proxy-server attribute?

As a second resort, I have the DNS forwarding override pointing to wpad, but I'd prefer to have it done by the dhcp server if it's at all possible.

Legion

I changed this client computer to remove the static dhcp mapping and use a dynamically mapped dhcp address. Still doesn't work in auto-detect mode, still works fine with manually set Squid.

Legion

Today's debugging efforts added to my diary thread.

Set Chrome to auto-detect proxy.

Add mimetype assignments for .dat and .da to /etc/inc/system.inc (they only exist for .pac by default).

Restart lighttpd.

Explicitly set rules on my bridged LAN (wlan and lan bridged) to block dest 80 and 443.

–""-- to pass to dest Squid IP/port.

Did 1000 other things.

Fail.

If I listen on tcpdump:

tcpdump -i em2 dst port 3128

with auto-detect set, there's dead silence broken by a very, very occasional packet. If I listen with the proxy set to Squid IP/port, constant traffic.

It's like auto-detect either fails to find wpad.dat, wpad.da or proxy.pac or fails to parse them correctly or some other stupid mistake I've made. It would be good if I could get some kind of feedback as to if the file is accessed (like a trace statement to stdout or a date-of-last-access via ls or anything).

Legion

I found the

stat -f %Sa wpad.dat

command, and it shows that wpad.dat, wpad.da and proxy.pac aren't touched at all with auto-detect set, using Chrome or IE. I disabled captive portal, still no luck. I'm considering now deleting my bridge and trying to set LAN == the LAN interface alone.

Surely I'm not trying anything super weird that no-one's tried before? A search reveals nothing but I'm wasting hours and hours on this and essentially stabbing in the dark with no results.

Legion

Uninstalled Squid3, tried Squid2, no resolution.

Tried a combination of:

NAT LAN subnet http to Squid
NAT LAN subnet https to Squid
pass LAN subnet to Squid
block LAN subnet to http
block LAN subnet to https
pass WAN subnet to http
pass WAN subnet to https

still works fine with proxy explicitly set, still does nothing with auto-detect. Wpad.dat not accessed at all.

marcelloc

wpad via https server will not work, Your firewall rules show pfsense on 443 with automatic redirect from 80 to 443.

Install package filer to edit your wpad files, you can edit it via gui and save on backup files.

On my setups, I configure pfsense gui using only https and another lighthttpd daemon only for wpad.

Legion

That makes sense!

I'm curious which rules show pfsense is on 443? Or is it implied by some of the rules that are only present with that config?

You're right, of course. I set the gui to use 443 and ssh in as well for all my editing/config/management. But I looked over this thread again and can't see any signs obvious to my noob eye that I'm on 443.

I'll try it again tonight with your suggestions and see how it goes. Otherwise I had almost resigned myself to just configuring all my devices to manually set the proxy ip/port.

Legion

@marcelloc:

On my setups, I configure pfsense gui using only https and another lighthttpd daemon only for wpad.

Any more information you care to provide on this subject would be much appreciated. I've just done a bunch of reading on lighttpd and there isn't much out there on multiple running instances. There's internet discussion of lighttpd listening on multiple ports, which would involve editing system.inc. There's discussion of redirection as config options, where I could specifically redirect https://my_lan/wpad_or_proxy to http://… But not much of an instance of lighttpd running just to serve wpad as well as the default that serves the rest of pfsense.

marcelloc

@Legion:

I'm curious which rules show pfsense is on 443? Or is it implied by some of the rules that are only present with that config?

Anti lock rule on lan ;)

marcelloc

@Legion:

Any more information you care to provide on this subject would be much appreciated.

basic steps:

disable on system->advanced redirect option form http to https
copy web configurator file to a new one (cp /var/etc/lighty-webConfigurator.conf /var/etc/lighty-proxy-wpad.conf for example)
edit new file to listen it on port 80 and change http dir to for example /usr/local/www/wpad
copy your wpad/pac files to /usr/local/www/wpad
start it with /usr/local/sbin/lighttpd -f /var/etc/lighty-proxy-wpad.conf
check/create a firewall rule that allow access to lighthttp listening ip:port

optional/additional steps

create a script to check if wpad lighthttp daemon is up and start it if it's down
install package filer to edit files via gui and keep it on pfsense xml backup

Legion

Thanks so much for your help marcelloc.

@marcelloc:

disable on system->advanced redirect option form http to https

Done.

@marcelloc:

copy web configurator file to a new one (cp /var/etc/lighty-webConfigurator.conf /var/etc/lighty-proxy-wpad.conf for example)

Done, cp'd to /usr/local/www/wpad/lighty-proxy-wpad.conf because I noticed a pfsense reboot wiped /var/etc/lighty-proxy-wpad.conf

@marcelloc:

edit new file to listen it on port 80 and change http dir to for example /usr/local/www/wpad

Done, and commented out all the ssl stuff. Pointed to my bridged LAN ip:80.

@marcelloc:

copy your wpad/pac files to /usr/local/www/wpad

start it with /usr/local/sbin/lighttpd -f /var/etc/lighty-proxy-wpad.conf

check/create a firewall rule that allow access to lighthttp listening ip:port

Yep. Although as it turns out I don't need the LAN rule. I just disabled it and traffic continues to pass. Maybe one of my other rules is allowing it? I'm not sure which one though.

@marcelloc:

create a script to check if wpad lighthttp daemon is up and start it if it's down

Like a cron job?

@marcelloc:

install package filer to edit files via gui and keep it on pfsense xml backup

I did it all with vi over putty while I mess around, but the backup idea is good.

Some good news and some bad. The good is - it works!

The bad news:

Opera sort-of works. You have to check a box for auto-detect proxy but then fill in a pac file location anyway. Might as well just point it towards the proxy manually.
Firefox has two settings - use system proxy settings (doesn't work if "system" is set to auto-detect), or auto-detect proxy settings (works)
IE9 works (or it did initially but then stopped - can't be bothered working out why).
Chrome works

Chrome's my browser of choice so that's OK. But the problems with other browsers worries me that some of the other devices going on my network might not work.

marcelloc

@Legion:

The bad news:

Opera sort-of works. You have to check a box for auto-detect proxy but then fill in a pac file location anyway. Might as well just point it towards the proxy manually.

Firefox has two settings - use system proxy settings (doesn't work if "system" is set to auto-detect), or auto-detect proxy settings (works)

IE9 works (or it did initially but then stopped - can't be bothered working out why).

Chrome works

Chrome's my browser of choice so that's OK. But the problems with other browsers worries me that some of the other devices going on my network might not work.

Are you using dhcp or dns auto detect proxy configuration?

Legion

Both.

marcelloc

@Legion:

Both.

Try only dns. I did some tests only with dns instead of both and the result was better.

Legion

I might try this weekend but I've moved onto a new challenge now - pfsense -> dansguardian -> squid with wpad.

First attempts failed (wpad pointing to DG port, Squid as parent, NAT rdr http to DG (I think wpad should do this anyway?), NAT rdr Squid to DG, LAN passing DG, few other rules. I can see http traffic hitting DG but it doesn't seem to then pass on to Squid.

If I get that working I'd like to add on pfBlocker and then CaptivePortal/FreeRADIUS2.

Legion

pfSense -> DG -> Squid3 now working via DHCP/wpad and assorted rules (NAT rdr squid port to DG, LAN pass to lighttpd serving wpad, LAN pass any to DG, LAN block http and https), with a few issues to resolve.

My main hurdle was thinking to make Squid listen on pfSense's box's IP and localhost (previously just pfSense's IP), and make DG's parent proxy IP localhost instead of pfSense box IP.

My main issues with DG I'll ask about in the appropriate thread.