Squid external Raspberry PI WPAD lighttpd server Guide

JonathanLee

Hello fellow Netgate community members, I wanted to share a simple how to guide for creating a WPAD server on a Raspberry Pi Zero 2 W.

First install a headless Raspberrian OS on a fresh card.

Second ssh into the new WPAD server.

ssh username@192.168.1.6 or whatever you set your wpad IP address to.

Second as you will have this behind the proxy set your environments to utilize the proxy system.

First navigate to the following path

/etc/apt/apt.conf.d

run sudo nano 10proxy

inside this new file add the following

Acquire::http::Proxy "http://192.168.1.1:3128/";
Acquire::https::Proxy "http://192.168.1.1:3128/";

ctl x save the file

next run the following command

sudo nano /etc/environment

add the following config lines

http_proxy="http://192.168.1.1:3128/"
https_proxy="http://192.168.1.1:3128/"
no_proxy="localhost,127.0.0.1"

save the file also

reboot the system

now after reboot run

apt update
apt upgrade

once this is completed reboot again run the following.

sudo apt install lighttpd -y

after this is installed take a system on the same lan and navigate to the address of the raspberry pi box http://192.168.1.6 as an example you should see a splash screen if not you have to make sure it is accessible.

After it is accessible navigate to the following path

/var/www/html

sudo rm the file to delete the file that is inside this folder this is the default webserver file.

Now let's make your wpad files.

sudo nano proxy.pac

inside this add the following this is my wpad or add your own version

function FindProxyForURL(url, host) 
{
url = url.toLowerCase();
host = host.toLowerCase();

if (isPlainHostName(host)) 
{
  return 'DIRECT';
}

if (isResolvable(host)) 
{
var hostIP = dnsResolve(host);

if (isInNet(hostIP, '0.0.0.0', '255.0.0.0') || isInNet(hostIP, '10.0.0.0', '255.0.0.0') ||
isInNet(hostIP, '127.0.0.0', '255.0.0.0') || isInNet(hostIP, '169.254.0.0', '255.255.0.0') ||
isInNet(hostIP, '172.16.0.0', '255.240.0.0') || isInNet(hostIP, '192.168.0.0', '255.255.0.0') ||
isInNet(hostIP, '198.18.0.0', '255.254.0.0') || isInNet(hostIP, '224.0.0.0', '240.0.0.0') ||
isInNet(hostIP, '240.0.0.0', '240.0.0.0')) 
{
  return 'DIRECT';
}

if (false) 
{
  return 'DIRECT';
}
}

if (url.substring(0, 5) == 'http:' || url.substring(0, 6) == 'https:' ||
url.substring(0, 4) == 'ftp:' ||  url.substring(0, 7) == "gopher:") 
{
  return 'PROXY 192.168.1.1:3128';
}

return 'DIRECT';
}

after
lets create linker files

sudo ln -s /var/www/html/proxy.pac /var/www/html/wpad.dat
sudo ln -s /var/www/html/proxy.pac /var/www/html/wpad.da

you should have three files. Now if you navigate to http://192.168.1.10/proxy.pac it should download the file.

Next add the following DHCP options in your pfsense

3 option 252 with string and the path to your wpad I am using 192.168.1.6 as an example.

"http://192.168.1.6/proxy.pac"
"http://192.168.1.6/wpad.dat"
"http://192.168.1.6/wpad.da"

After this also add a DNS entry to help resolve the wpad

to test this you would do

nslookup wpad

and it should always respond with a wpad address.

Now change your system that is the road warrior to auto proxy to test if it works. Once it does lock the WPAD out of using the internet as it only serves one purpose handing out your wpad files It doesn't really need internet.

Please let me know if I missed something. This way you can have https only for the firewall and still use the older wpad protocol for auto configurations. I wish they would update the WPAD protocol to make a better version again, that has nothing to do with this.

JonathanLee

@JonathanLee I was so tried of manually setting this every day, this fixed all my issues for traveling laptops. Please if you have any security recommendations for lighttpd let me know. Right now I have disabled all WAN access and it also can not access the proxy or cache.

JonathanLee

I wanted to share some more info on security hardening your wpad after

Added by gstrauss 4 days ago

Security hardening of any webserver starts with restricting permissions to only what is needed.

If you do not need the webserver to run as root, then you should run the webserver as a less-privileged user, e.g. I believe Raspberry Pi Zero runs lighttpd as user www-data by default, but you should check. The Debian-based Raspberry Pi sets up various permissions and locations for lighttpd to write access and error logs, jobs to clean up temp files, etc, so changing user under which the webserver runs requires additional steps to get back the functionality. Prefer to use www-data.

If you do not need the webserver to be public-facing, make sure to configure the webserver and/or firewall so that the webserver can only serve requests from the local networks.

If you do not need the webserver to do anything besides access files read-only, then you should might consider making the document root owned by a different user and read-only to webserver user.

For lighttpd, if you are only serving read-only files, then you should restrict the size of HTTP requests to a low (non-zero) number. (0 disables the limit) See server.max-request-size
For resource-constrained servers like the Raspberry Pi Zero, you might tune the server to reduce the chance that malicious clients can deny service to others. However, if you're serving wpad, that should be on an internal network, not internet-facing, so you should configure lighttpd to listen only on your internal network IP, and not on a public-facing IP. You could also configure lighttpd and/or your firewall to only allow access to port 80 via the local network.

If you do not need access logs, then you might disable access logging in lighttpd to reduce resource usage.
If you are only serving static files, you might reduce connection timeouts since you expect lighttpd to serve files very quickly.
If you are only serving wpad, then you might reduce the number of keep-alive requests allowed per client before lighttpd closes the connection.

Besides running as non-root, and listening and serving clients only from local network (not internet), which are strongly recommended for security hardening, the rest is resource tuning for availability and performance. Still, even without extra tuning on a Raspberry Pi Zero, you should find that lighttpd can serve thousands of requests per second for a small, static wpad file (proxy.pac or wpad.dat)

See WikiStart and links to
Docs_ConfigurationOptions
Docs_ResourceTuning
Docs_Performance

Side note I really do not understand why WPAD has never been updated to something like WPAD2.0 protocol because of the associated risks with it. It seems from a security perspective that big tech should update this older protocol.

Look up "zero trust architecture" in a search engine.
If you the clients already have pre-installed an SSL certificate for the the proxy you assign, and only uses https, then a malicious wpad won't be able to direct the client to send http requests through a rogue server without certificate failures.

Still, on new networks, if you have not already pre-configured (more secure), then many architectures follow TOFU (trust on first use) principles.

JonathanLee

security hardening continued for wpad server

sudo nano /etc/lighttpd/lighttpd.conf

add the following

first line blocks all access but local network with a negated != rule. Get it Netgated rule :) funny funny.

Ok ok and after get more granular and add rules so local network can only access the wpad files and nothing else on the external web server done.

also add a shorter connection timeout

$HTTP["remoteip"] != "192.168.1.0/27" {
url.access-deny = ( "" )
}
}

$HTTP["url"] =~ "^/wpad.dat" {
$HTTP["remoteip"] == "192.168.1.0/27" {
}
else {
url.access-deny = ( "" )
}
}

$HTTP["url"] =~ "^/proxy.pac" {
$HTTP["remoteip"] == "192.168.1.0/27" {
}
else {
url.access-deny = ( "" )
}
}

$HTTP["url"] =~ "^/wpad.da" {
$HTTP["remoteip"] == "192.168.1.0/27" {
}
else {
url.access-deny = ( "" )
}
}

This helped me alot first block all but local network accessing it after make it only allow local to wpad files tested ok with many tests

JonathanLee

Also some changes per lighted recommendations

server.max-request-size     = 8
server.tag                  = ""
server.range-requests       = "disable"
server.max-connections      = 12
connect-timeout             = 2

JonathanLee

Also you can connect a LCD HAT to this from wave share and program it to show who is logged in.

I have the code adapted to output w in a loop so I can see users logged in to the wpad pretty cool