Squid external Raspberry PI WPAD lighttpd server Guide with Raspberry Zero LCD HAT code.

JonathanLee

Hello fellow Netgate community members, I wanted to share a simple how to guide for creating a WPAD server on a Raspberry Pi Zero 2 W.

First install a headless Raspberrian OS on a fresh card.

Second ssh into the new WPAD server.

ssh username@192.168.1.6 or whatever you set your wpad IP address to.

Second as you will have this behind the proxy set your environments to utilize the proxy system.

First navigate to the following path

/etc/apt/apt.conf.d

run sudo nano 10proxy

inside this new file add the following

Acquire::http::Proxy "http://192.168.1.1:3128/";
Acquire::https::Proxy "http://192.168.1.1:3128/";

ctl x save the file

next run the following command

sudo nano /etc/environment

add the following config lines

http_proxy="http://192.168.1.1:3128/"
https_proxy="http://192.168.1.1:3128/"
no_proxy="localhost,127.0.0.1"

save the file also

reboot the system

now after reboot run

apt update
apt upgrade

once this is completed reboot again run the following.

sudo apt install lighttpd -y

after this is installed take a system on the same lan and navigate to the address of the raspberry pi box http://192.168.1.6 as an example you should see a splash screen if not you have to make sure it is accessible.

After it is accessible navigate to the following path

/var/www/html

sudo rm the file to delete the file that is inside this folder this is the default webserver file.

Now let's make your wpad files.

sudo nano proxy.pac

inside this add the following this is my wpad or add your own version

function FindProxyForURL(url, host) 
{
url = url.toLowerCase();
host = host.toLowerCase();

if (isPlainHostName(host)) 
{
  return 'DIRECT';
}

if (isResolvable(host)) 
{
var hostIP = dnsResolve(host);

if (isInNet(hostIP, '0.0.0.0', '255.0.0.0') || isInNet(hostIP, '10.0.0.0', '255.0.0.0') ||
isInNet(hostIP, '127.0.0.0', '255.0.0.0') || isInNet(hostIP, '169.254.0.0', '255.255.0.0') ||
isInNet(hostIP, '172.16.0.0', '255.240.0.0') || isInNet(hostIP, '192.168.0.0', '255.255.0.0') ||
isInNet(hostIP, '198.18.0.0', '255.254.0.0') || isInNet(hostIP, '224.0.0.0', '240.0.0.0') ||
isInNet(hostIP, '240.0.0.0', '240.0.0.0')) 
{
  return 'DIRECT';
}

if (false) 
{
  return 'DIRECT';
}
}

if (url.substring(0, 5) == 'http:' || url.substring(0, 6) == 'https:' ||
url.substring(0, 4) == 'ftp:' ||  url.substring(0, 7) == "gopher:") 
{
  return 'PROXY 192.168.1.1:3128';
}

return 'DIRECT';
}

after
lets create linker files

sudo ln -s /var/www/html/proxy.pac /var/www/html/wpad.dat
sudo ln -s /var/www/html/proxy.pac /var/www/html/wpad.da

you should have three files. Now if you navigate to http://192.168.1.10/proxy.pac it should download the file.

Next add the following DHCP options in your pfsense

3 option 252 with string and the path to your wpad I am using 192.168.1.6 as an example.

"http://192.168.1.6/proxy.pac"
"http://192.168.1.6/wpad.dat"
"http://192.168.1.6/wpad.da"

After this also add a DNS entry to help resolve the wpad

to test this you would do

nslookup wpad

and it should always respond with a wpad address.

Now change your system that is the road warrior to auto proxy to test if it works. Once it does lock the WPAD out of using the internet as it only serves one purpose handing out your wpad files It doesn't really need internet.

Please let me know if I missed something. This way you can have https only for the firewall and still use the older wpad protocol for auto configurations. I wish they would update the WPAD protocol to make a better version again, that has nothing to do with this.

JonathanLee

@JonathanLee I was so tried of manually setting this every day, this fixed all my issues for traveling laptops. Please if you have any security recommendations for lighttpd let me know. Right now I have disabled all WAN access and it also can not access the proxy or cache.

JonathanLee

I wanted to share some more info on security hardening your wpad after

Added by gstrauss 4 days ago

Security hardening of any webserver starts with restricting permissions to only what is needed.

If you do not need the webserver to run as root, then you should run the webserver as a less-privileged user, e.g. I believe Raspberry Pi Zero runs lighttpd as user www-data by default, but you should check. The Debian-based Raspberry Pi sets up various permissions and locations for lighttpd to write access and error logs, jobs to clean up temp files, etc, so changing user under which the webserver runs requires additional steps to get back the functionality. Prefer to use www-data.

If you do not need the webserver to be public-facing, make sure to configure the webserver and/or firewall so that the webserver can only serve requests from the local networks.

If you do not need the webserver to do anything besides access files read-only, then you should might consider making the document root owned by a different user and read-only to webserver user.

For lighttpd, if you are only serving read-only files, then you should restrict the size of HTTP requests to a low (non-zero) number. (0 disables the limit) See server.max-request-size
For resource-constrained servers like the Raspberry Pi Zero, you might tune the server to reduce the chance that malicious clients can deny service to others. However, if you're serving wpad, that should be on an internal network, not internet-facing, so you should configure lighttpd to listen only on your internal network IP, and not on a public-facing IP. You could also configure lighttpd and/or your firewall to only allow access to port 80 via the local network.

If you do not need access logs, then you might disable access logging in lighttpd to reduce resource usage.
If you are only serving static files, you might reduce connection timeouts since you expect lighttpd to serve files very quickly.
If you are only serving wpad, then you might reduce the number of keep-alive requests allowed per client before lighttpd closes the connection.

Besides running as non-root, and listening and serving clients only from local network (not internet), which are strongly recommended for security hardening, the rest is resource tuning for availability and performance. Still, even without extra tuning on a Raspberry Pi Zero, you should find that lighttpd can serve thousands of requests per second for a small, static wpad file (proxy.pac or wpad.dat)

See WikiStart and links to
Docs_ConfigurationOptions
Docs_ResourceTuning
Docs_Performance

Side note I really do not understand why WPAD has never been updated to something like WPAD2.0 protocol because of the associated risks with it. It seems from a security perspective that big tech should update this older protocol.

Look up "zero trust architecture" in a search engine.
If you the clients already have pre-installed an SSL certificate for the the proxy you assign, and only uses https, then a malicious wpad won't be able to direct the client to send http requests through a rogue server without certificate failures.

Still, on new networks, if you have not already pre-configured (more secure), then many architectures follow TOFU (trust on first use) principles.

JonathanLee

security hardening continued for wpad server

sudo nano /etc/lighttpd/lighttpd.conf

add the following

first line blocks all access but local network with a negated != rule. Get it Netgated rule :) funny funny.

Ok ok and after get more granular and add rules so local network can only access the wpad files and nothing else on the external web server done.

also add a shorter connection timeout

$HTTP["remoteip"] != "192.168.1.0/27" {
url.access-deny = ( "" )
}
}

$HTTP["url"] =~ "^/wpad.dat" {
$HTTP["remoteip"] == "192.168.1.0/27" {
}
else {
url.access-deny = ( "" )
}
}

$HTTP["url"] =~ "^/proxy.pac" {
$HTTP["remoteip"] == "192.168.1.0/27" {
}
else {
url.access-deny = ( "" )
}
}

$HTTP["url"] =~ "^/wpad.da" {
$HTTP["remoteip"] == "192.168.1.0/27" {
}
else {
url.access-deny = ( "" )
}
}

This helped me alot first block all but local network accessing it after make it only allow local to wpad files tested ok with many tests

JonathanLee

Also some changes per lighted recommendations

server.max-request-size     = 8
server.tag                  = ""
server.range-requests       = "disable"
server.max-connections      = 12
connect-timeout             = 2

JonathanLee

Also you can connect a LCD HAT to this from wave share and program it to show who is logged in.

I have the code adapted to output w in a loop so I can see users logged in to the wpad pretty cool

JonathanLee

Continued additional security for your WPAD:

I have added an LCD hat to the Zero Pi that uses an adapted vendor provided example python code file. This is so that the screen will also show if any users are logged into the WPAD server. Again, this is very simple to get you into programing a display, it is simple we are adding a string for users logged in to display and adding a service.

Here is a how to guide for this also now that your WPAD is configured and running let us add the LCD hat.

Supplies:
Zero LCD HAT (A)

https://www.amazon.com/LCD-HAT-Secondary-Compatible-Zero/dp/B0DKF8L7VF

You will have to add the GPIO standoff as it is not included in the Zero 2 W you have to manually add the GPIO header so the LCD hat can be plugged in first.

Ok so you got it connected follow the guide for configuration of this.
https://www.waveshare.com/wiki/Zero_LCD_HAT_(A)
I have included the code here also with my adapted steps.

#Enable SPI
sudo raspi-config nonint do_spi 0
#Install Python3 
sudo apt-get update
sudo apt-get install python3-pip
sudo apt-get install python3-pil
sudo apt-get install python3-numpy
sudo apt install python3-luma.oled
sudo apt install python3-luma.lcd
sudo apt install python3-RPi.GPIO
sudo apt install python3-spidev
#Get the Demo Files
sudo apt-get install unzip -y
sudo wget https://files.waveshare.com/wiki/Zero-LCD-HAT-A/Zero_LCD_HAT_A_Demo.zip
sudo unzip ./Zero_LCD_HAT_A_Demo.zip
cd Zero_LCD_HAT_A_Demo

you need to add "dtoverlay=spi1-1cs" to the config.txt file for opening the SPI device.

sudo nano /boot/firmware/config.txt


#adapt the demo file 
sudo cp CPU.py CPU2.py
sudo nano CPU2.py

Adapt the CPU2.py file to activate the middle screen and add the users section.

This is a simple addition do not reinvent the program just activate the new screen with the code given add the string for the output of "w" so that you can always see the current users logged in.

#!/usr/bin/python
# -*- coding: UTF-8 -*-
#import chardet
import os
import sys
import time
import logging
import spidev as SPI
import subprocess
sys.path.append("..")
from lib import LCD_0inch96
from lib import LCD_1inch3
from lib import Gain_Param
from PIL import Image,ImageDraw,ImageFont
import RPi.GPIO as GPIO
import re
import math

# Raspberry Pi pin configuration:
RST_0 =24
DC_0 = 4
BL_0 = 13
bus_0 = 0
device_0 = 0

RST_1 =23
DC_1 = 5
BL_1 = 12
bus_1 = 0
device_1 = 1

RST = 27
DC = 22
BL = 19
bus = 1
device = 0

logging.basicConfig(level=logging.DEBUG)

try:
    disp_0 = LCD_0inch96.LCD_0inch96(spi=SPI.SpiDev(bus_0, device_0),spi_freq=10000000,rst=RST_0,dc=DC_0,bl=BL_0,bl_freq=1000)
    disp_1 = LCD_0inch96.LCD_0inch96(spi=SPI.SpiDev(bus_1, device_1),spi_freq=10000000,rst=RST_1,dc=DC_1,bl=BL_1,bl_freq=1000)
    disp = LCD_1inch3.LCD_1inch3(spi=SPI.SpiDev(bus, device),spi_freq=10000000,rst=RST,dc=DC,bl=BL)

    gain = Gain_Param.Gain_Param()

    disp.Init()
    disp_0.Init()
    disp_1.Init()

    disp.clear()
    disp_0.clear()
    disp_1.clear()

    disp.bl_DutyCycle(100)
    disp_0.bl_DutyCycle(100)
    disp_1.bl_DutyCycle(100)

    image1 = Image.new("RGB", (disp_0.width, disp_0.height), "WHITE")
    draw = ImageDraw.Draw(image1)
    Font2 = ImageFont.truetype("../Font/Font00.ttf",7)
    while True:
        #IP
        ip = gain.GET_IP()
        Font1 = ImageFont.truetype("../Font/Font00.ttf",15)
        draw.text((5, 0), 'IP : '+ip, fill = 0x3cbdc4,font=Font1)

        #time
        time_t = time.strftime("%H:%M:%S", time.localtime())
        time_D = time.strftime("%Y-%m-%d ", time.localtime())
        draw.text((5, 25), "Data: "+time_D, fill = 0x46D017,font=Font1)
        draw.text((5, 50), "Time: "+time_t, fill = 0xf7ba47,font=Font1)
        disp_0.ShowImage(image1)
        draw.rectangle((0,0,disp_0.width,disp_0.height),fill = "WHITE") #Cache area covered with white

        #CPU usage
        CPU_usage= os.popen('top -bi -n 2 -d 0.02').read().split('\n\n\n')[0].split('\n')[2]
        CPU_usage= re.sub('[a-zA-z%(): ]','',CPU_usage)
        CPU_usage= CPU_usage.split(',')
        CPU_usagex =100 - eval(CPU_usage[3])
        draw.text((5, 0), "CPU Usage: " + str(math.floor(CPU_usagex))+'%', fill = 0x0b46e3,font=Font1,)

        #Message
        user = subprocess.run(['w'], stdout=subprocess.PIPE).stdout.decode('utf-8')
        image3 = Image.new("RGB", (disp.width, disp.height), "WHITE")
        draw1 = ImageDraw.Draw(image3)
        draw1.text((5, 0), "WPAD SERVER: \n\n"+user, fill = 0x3cbdc4, font=Font2)
        disp.ShowImage(image3)
        draw1.rectangle((0,0,disp.width, disp.height), "WHITE")

        #TEMP
        temp_t = gain.GET_Temp()
        draw.text((5, 25), "Temp: "+str(math.floor(temp_t))+'℃', fill = 0x0088ff,font=Font1)

        #System disk usage
        x = os.popen('df -h /')
        i2 = 0
        while 1:
            i2 = i2 + 1
            line = x.readline()
            if i2==2:
                Capacity_usage = line.split()[4] # Memory usage (%)
                Hard_capacity = int(re.sub('[%]','',Capacity_usage))
                break

        draw.text((5, 50), "Disk Usage: "+str(math.floor(Hard_capacity))+'%', fill = 0x986DFC,font=Font1) # BGR

        disp_1.ShowImage(image1)
        draw.rectangle((0,0,disp_0.width,disp_0.height),fill = "WHITE")
        time.sleep(0.01)


    disp_0.module_exit()
    disp_1.module_exit()
    logging.info("quit:")
except IOError as e:
    logging.info(e)
except KeyboardInterrupt:
    disp_0.module_exit()
    disp_1.module_exit()
    logging.info("quit:")
    exit()

save your adapted CPU2.py example file.

Test with

sudo python CPU2.py

It should run on and run, if not check connections.

Now that it is working, we need to create a service so that it will run every time you reboot or turn on your Raspberry pi.
You might have to add sudo chmod to the .py file to add +x to it to make it executable.

#Navigate to:
cd /lib/systemd/system/
#create a new service
sudo nano display.service
#add the following into the service file. Make sure your working directory is where the python code is.

[Unit]
Description=Display
After=network.target

[Service]
User=root
WorkingDirectory=/usr/Zero_LCD_HAT_A_Demo/python/example/
ExecStart=/usr/bin/python3 /usr/Zero_LCD_HAT_A_Demo/python/example/CPU2.py
Restart=always
RestartSec=3
StandardOutput=file:/var/log/example_service_output.log
StandardError=file:/var/log/example_service_error.log

[Install]
WantedBy=multi-user.target

#save this ctl x

#Run the following 
sudo systemctl daemon-reload
sudo systemctl enable display.service
sudo systemctl start display.service
#check for errors
sudo systemctl status display.service
#It should display this if it works:
● display.service - Display
     Loaded: loaded (/lib/systemd/system/display.service; enabled; preset: enabled)
     Active: active (running) since Wed 2024-12-18 06:45:23 PST; 6h ago
   Main PID: 507 (python3)
      Tasks: 6 (limit: 178)
        CPU: 2h 18min 29.878s
     CGroup: /system.slice/display.service
             └─507 /usr/bin/python3 /usr/Zero_LCD_HAT_A_Demo/python/example/CPU2.py

Dec 18 06:45:23 Zero systemd[1]: Started display.service - Display.

#Once it says enabled run
sudo reboot
#the dispaly should always run now.

That is it you should now know who is logged into the WPAD at all times, if your not in a SSH session into it the screen show should no users. Meaning that it will only change when someone is on a terminal session of your pi. The screen file is very basic but it can help push you to customize it yourself.

JonathanLee

Keep in mind you will have to install an SSL certificate on the Raspberry PI to download the GIT hub file and program. You have to copy the SSL file over to the pi by way of a USB drive and reconfigure your ca certificates for that, or just connect the pi to a non-proxied system to get your files after set it back. If you are like me you just add the certificate so you can access the proxy.

To add the firewalls certificate used with the proxy first download it directly from the firewall so you know it is yours.
after

sudo mkdir /media/usb
sudo mount /dev/sda2 /media/usb -o umask=000
cd /media/usb 
#find your file from the flash drive lets call it CA-Cert.crt
#copy it to the locations is it needed at
sudo cp CA-Cert.crt /etc/ssl/certs
sudo cp CA-Cert.crt /usr/share/ca-certificates/
sudo cp CA-Cert.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

This should fix your certificate issue to use wget while in the proxy.

Final step we need to make sure that once inside the proxy that you can't call the wpad, meaning that the WPAD is a one-way ticket that the proxy or it's cache cannot access it for added security.

Add this to custom options on the Squid configuration, it is not really needed I add this for fear of container issues,

acl wpad urlpath_regex ^/wpad.dat$
acl wpad urlpath_regex ^/proxy.pac$
acl wpad urlpath_regex ^/wpad.da$
#deny_info 200:/etc/squid/wpad.dat wpad # this is if you manage wpad from squid not needed here
reply_header_access Content-Type deny wpad