WPAD Block Port 80 Rule is blocking all of my traffic

[KOM]

I replaced my actual SSH port with 22 so I wouldn't be posting my SSH port online.

Don't bother. If it's accessible via WAN then it's probably being port-scanned a 100 times a day anyway.  Trust your security.  If you're really paranoid then use 2-factor with a loin and certificate.

OK, sorry for the confusion with your logs.

squid -k check returns nothing. I have no idea what this is doing or what I'm expecting to see here?

Geez, my brain is off today.  Try:

squid -k parse

You should get a long list of output.  Look for warnings or errors.

Are you running WebGUI in HTTP or HTTPS mode?  Is pfSense the web server serving your WPAD files?

[pfBasic]

OK thanks, the results for that are attached.

I switched the WebGUI to http for WPAD, and pfSense is the server for the WPAD files.

[squid -k parse results.txt](/public/imported_attachments/1/squid -k parse results.txt)

[KOM]

There appear to be no problems with your squid configuration, according to your output.

[pfBasic]

Could it be something wrong with the way pfSense is serving up the WPAD files? Or the way I have it setup?

The proxy.pac is located in "/usr/local/www/proxy.pac" and is linked to a wpad.dat & wpad.da file in the same directory.

They all contain the basic configuration:
function FindProxyForURL(url,host)
{
return "PROXY 192.168.1.1:3128";
}

It seems weird to me that traffic doesn't show up on squid when a computer is setup to autoconfigure, but it does when I point the computer to the pfSense box manually?

It also seems weird that clamAV doesn't work on http/s but does on http with an explicit proxy.

[KOM]

Looks like you're doing everything right.  I don't know why it doesn't work for you.  I don't run AV on the firewall – too slow and I suspect the defs aren't as up to date as a commercial provider.  Get ClamAV off the firewall and use a decent client AV package, if required.

[pfBasic]

I'll do that, it does allow things down. Right now I'm using it as an easy way to see if I'm under squids umbrella.

Do you know if there's any way for me to check pfsense's file server?

[KOM]

Do you know if there's any way for me to check pfsense's file server?

Eh, what?

Maybe this might be the issue?

http://kb.k12usa.com/Knowledgebase/Proxy-Auto-Detect-WPAD-Issues-With-IE-Windows-7

[pfBasic]

I'll look into that but I'm getting the same results across several OS's.

I was just wondering if there was a way for me to see what's happening when a client requests a WPAD or proxy file from the pfsense box.

[KOM]

I was just wondering if there was a way for me to see what's happening when a client requests a WPAD or proxy file from the pfsense box.

Go to console and look at /var/log/nginx.log and nginx-error.log.

[pfBasic]

nginx log is large but a notepadd++ search for "wpad" "proxy" ".pac", etc. returns nothing.

nginx error log only has 10 lines:

2016/08/23 15:18:46 [error] 29178#100114: send() failed (54: Connection reset by peer)
2016/08/23 15:26:53 [alert] 29178#100114: close() socket failed (9: Bad file descriptor)
2016/08/25 00:02:41 [error] 32286#100113: send() failed (54: Connection reset by peer)
2016/08/25 00:05:56 [error] 32286#100113: send() failed (54: Connection reset by peer)
2016/08/25 00:05:56 [error] 32286#100113: send() failed (54: Connection reset by peer)
2016/08/25 00:06:33 [error] 32286#100113: send() failed (54: Connection reset by peer)
2016/08/25 00:08:24 [error] 32286#100113: send() failed (54: Connection reset by peer)
2016/08/25 00:08:24 [error] 32286#100113: send() failed (54: Connection reset by peer)
2016/08/25 00:09:36 [error] 32286#100113: send() failed (54: Connection reset by peer)
2016/08/25 00:11:06 [error] 32541#100082: send() failed (54: Connection reset by peer)

I did find some information that might be useful in chrome though, By going to chrome://net-internals/#proxy and chrome://net-internals/#events I was able to see what's going on in chrome when it requests the files and these are the entries I'm seeing on autoconfigure (when the proxy fails).

On the #proxy page I see this:

Effective proxy settings

Use DIRECT connections.
Source: SYSTEM
Original proxy settings

Auto-detect
Source: SYSTEM

On the #events page there are thousands of entries but here are the ones that seem relevant:

230858: URL_REQUEST
http://192.168.1.1/proxy.pac
Start Time: 2016-08-25 00:42:56.954

t=6927008 [st=  0] +REQUEST_ALIVE  [dt=400]
t=6927008 [st=  0]    URL_REQUEST_DELEGATE  [dt=0]
t=6927008 [st=  0]   +URL_REQUEST_START_JOB  [dt=400]
                      --> load_flags = 176 (BYPASS_PROXY | DISABLE_CACHE | DISABLE_CERT_REVOCATION_CHECKING)
                      --> method = "GET"
                      --> priority = "LOWEST"
                      --> url = "http://192.168.1.1/proxy.pac"
t=6927008 [st=  0]      URL_REQUEST_DELEGATE  [dt=0]
t=6927008 [st=  0]      URL_REQUEST_DELEGATE  [dt=0]
t=6927008 [st=  0]     +HTTP_STREAM_REQUEST  [dt=400]
t=6927008 [st=  0]        HTTP_STREAM_REQUEST_STARTED_JOB
                          --> source_dependency = 230859 (HTTP_STREAM_JOB)
t=6927408 [st=400]        CANCELLED
t=6927408 [st=400]     -HTTP_STREAM_REQUEST
t=6927408 [st=400] -REQUEST_ALIVE

230864: URL_REQUEST
http://wpad/wpad.dat
Start Time: 2016-08-25 00:42:57.354

t=6927408 [st=    0] +REQUEST_ALIVE  [dt=21039]
t=6927409 [st=    1]    URL_REQUEST_DELEGATE  [dt=0]
t=6927409 [st=    1]   +URL_REQUEST_START_JOB  [dt=21038]
                        --> load_flags = 176 (BYPASS_PROXY | DISABLE_CACHE | DISABLE_CERT_REVOCATION_CHECKING)
                        --> method = "GET"
                        --> priority = "LOWEST"
                        --> url = "http://wpad/wpad.dat"
t=6927409 [st=    1]      URL_REQUEST_DELEGATE  [dt=0]
t=6927409 [st=    1]      URL_REQUEST_DELEGATE  [dt=0]
t=6927409 [st=    1]     +HTTP_STREAM_REQUEST  [dt=21038]
t=6927409 [st=    1]        HTTP_STREAM_REQUEST_STARTED_JOB
                            --> source_dependency = 230865 (HTTP_STREAM_JOB)
t=6948447 [st=21039]        HTTP_STREAM_REQUEST_BOUND_TO_JOB
                            --> source_dependency = 230865 (HTTP_STREAM_JOB)
t=6948447 [st=21039]     -HTTP_STREAM_REQUEST
t=6948447 [st=21039]   -URL_REQUEST_START_JOB
                        --> net_error = -118 (ERR_CONNECTION_TIMED_OUT)
t=6948447 [st=21039]    URL_REQUEST_DELEGATE  [dt=0]
t=6948447 [st=21039] -REQUEST_ALIVE
                      --> net_error = -118 (ERR_CONNECTION_TIMED_OUT)

230865: HTTP_STREAM_JOB
http://wpad/
Start Time: 2016-08-25 00:42:57.355

t=6927409 [st=    0] +HTTP_STREAM_JOB  [dt=21038]
                      --> alternative_service = "Uninitialized :0"
                      --> original_url = "http://wpad/"
                      --> priority = "LOWEST"
                      --> source_dependency = 230864 (URL_REQUEST)
                      --> url = "http://wpad/"
t=6927409 [st=    0]    TCP_CLIENT_SOCKET_POOL_REQUESTED_SOCKET
                        --> host_and_port = "wpad:80"
t=6927409 [st=    0]   +SOCKET_POOL  [dt=21038]
t=6948447 [st=21038]      SOCKET_POOL_BOUND_TO_CONNECT_JOB
                          --> source_dependency = 230866 (CONNECT_JOB)
t=6948447 [st=21038]   -SOCKET_POOL
                        --> net_error = -118 (ERR_CONNECTION_TIMED_OUT)
t=6948447 [st=21038]    HTTP_STREAM_JOB_BOUND_TO_REQUEST
                        --> source_dependency = 230864 (URL_REQUEST)
t=6948447 [st=21038] -HTTP_STREAM_JOB

230866: CONNECT_JOB
wpad:80
Start Time: 2016-08-25 00:42:57.355

t=6927409 [st=    0] +SOCKET_POOL_CONNECT_JOB  [dt=21038]
                      --> group_name = "wpad:80"
t=6927409 [st=    0]   +SOCKET_POOL_CONNECT_JOB_CONNECT  [dt=21038]
t=6927409 [st=    0]     +HOST_RESOLVER_IMPL_REQUEST  [dt=1]
                          --> address_family = 0
                          --> allow_cached_response = false
                          --> host = "wpad:80"
                          --> is_speculative = false
t=6927409 [st=    0]        HOST_RESOLVER_IMPL_IPV6_REACHABILITY_CHECK
                            --> cached = true
                            --> ipv6_available = true
t=6927409 [st=    0]        HOST_RESOLVER_IMPL_CREATE_JOB
t=6927409 [st=    0]        HOST_RESOLVER_IMPL_JOB_ATTACH
                            --> source_dependency = 230867 (HOST_RESOLVER_IMPL_JOB)
t=6927410 [st=    1]     -HOST_RESOLVER_IMPL_REQUEST
t=6948447 [st=21038]   -SOCKET_POOL_CONNECT_JOB_CONNECT
                        --> net_error = -118 (ERR_CONNECTION_TIMED_OUT)
t=6948447 [st=21038] -SOCKET_POOL_CONNECT_JOB

230867: HOST_RESOLVER_IMPL_JOB
wpad
Start Time: 2016-08-25 00:42:57.355

t=6927409 [st=0] +HOST_RESOLVER_IMPL_JOB  [dt=1]
                  --> host = "wpad"
                  --> source_dependency = 230866 (CONNECT_JOB)
t=6927409 [st=0]    HOST_RESOLVER_IMPL_JOB_STARTED
t=6927409 [st=0]   +HOST_RESOLVER_IMPL_PROC_TASK  [dt=1]
t=6927409 [st=0]      HOST_RESOLVER_IMPL_ATTEMPT_STARTED
                      --> attempt_number = 1
t=6927409 [st=0]      HOST_RESOLVER_IMPL_JOB_REQUEST_ATTACH
                      --> priority = "LOWEST"
                      --> source_dependency = 230866 (CONNECT_JOB)
t=6927410 [st=1]      HOST_RESOLVER_IMPL_ATTEMPT_FINISHED
                      --> attempt_number = 1
t=6927410 [st=1]   -HOST_RESOLVER_IMPL_PROC_TASK
                    --> address_list = ["192.168.1.1:0"]
t=6927410 [st=1] -HOST_RESOLVER_IMPL_JOB

230868: SOCKET
wpad:80
Start Time: 2016-08-25 00:42:57.356

t=6927410 [st=    0] +SOCKET_ALIVE  [dt=21037]
                      --> source_dependency = 230866 (CONNECT_JOB)
t=6927410 [st=    0]   +TCP_CONNECT  [dt=21037]
                        --> address_list = ["192.168.1.1:80"]
t=6927410 [st=    0]     +TCP_CONNECT_ATTEMPT  [dt=21037]
                          --> address = "192.168.1.1:80"
t=6948447 [st=21037]     -TCP_CONNECT_ATTEMPT
                          --> os_error = 10060
t=6948447 [st=21037]      SOCKET_CLOSED
t=6948447 [st=21037]   -TCP_CONNECT
                        --> net_error = -118 (ERR_CONNECTION_TIMED_OUT)
t=6948447 [st=21037] -SOCKET_ALIVE

230869: CONNECT_JOB
wpad:80
Start Time: 2016-08-25 00:42:57.606

t=6927660 [st=    0] +SOCKET_POOL_CONNECT_JOB  [dt=21000]
                      --> group_name = "wpad:80"
t=6927660 [st=    0]    BACKUP_CONNECT_JOB_CREATED
t=6927660 [st=    0]   +SOCKET_POOL_CONNECT_JOB_CONNECT  [dt=21000]
t=6927660 [st=    0]     +HOST_RESOLVER_IMPL_REQUEST  [dt=0]
                          --> address_family = 0
                          --> allow_cached_response = false
                          --> host = "wpad:80"
                          --> is_speculative = false
t=6927660 [st=    0]        HOST_RESOLVER_IMPL_IPV6_REACHABILITY_CHECK
                            --> cached = true
                            --> ipv6_available = true
t=6927660 [st=    0]        HOST_RESOLVER_IMPL_CREATE_JOB
t=6927660 [st=    0]        HOST_RESOLVER_IMPL_JOB_ATTACH
                            --> source_dependency = 230870 (HOST_RESOLVER_IMPL_JOB)
t=6927660 [st=    0]     -HOST_RESOLVER_IMPL_REQUEST
t=6948660 [st=21000]   -SOCKET_POOL_CONNECT_JOB_CONNECT
                        --> net_error = -118 (ERR_CONNECTION_TIMED_OUT)
t=6948660 [st=21000] -SOCKET_POOL_CONNECT_JOB

230870: HOST_RESOLVER_IMPL_JOB
wpad
Start Time: 2016-08-25 00:42:57.606

t=6927660 [st=0] +HOST_RESOLVER_IMPL_JOB  [dt=0]
                  --> host = "wpad"
                  --> source_dependency = 230869 (CONNECT_JOB)
t=6927660 [st=0]    HOST_RESOLVER_IMPL_JOB_STARTED
t=6927660 [st=0]   +HOST_RESOLVER_IMPL_PROC_TASK  [dt=0]
t=6927660 [st=0]      HOST_RESOLVER_IMPL_ATTEMPT_STARTED
                      --> attempt_number = 1
t=6927660 [st=0]      HOST_RESOLVER_IMPL_JOB_REQUEST_ATTACH
                      --> priority = "LOWEST"
                      --> source_dependency = 230869 (CONNECT_JOB)
t=6927660 [st=0]      HOST_RESOLVER_IMPL_ATTEMPT_FINISHED
                      --> attempt_number = 1
t=6927660 [st=0]   -HOST_RESOLVER_IMPL_PROC_TASK
                    --> address_list = ["192.168.1.1:0"]
t=6927660 [st=0] -HOST_RESOLVER_IMPL_JOB

230871: SOCKET
wpad:80
Start Time: 2016-08-25 00:42:57.606

t=6927660 [st=    0] +SOCKET_ALIVE  [dt=21000]
                      --> source_dependency = 230869 (CONNECT_JOB)
t=6927660 [st=    0]   +TCP_CONNECT  [dt=21000]
                        --> address_list = ["192.168.1.1:80"]
t=6927660 [st=    0]     +TCP_CONNECT_ATTEMPT  [dt=21000]
                          --> address = "192.168.1.1:80"
t=6948660 [st=21000]     -TCP_CONNECT_ATTEMPT
                          --> os_error = 10060
t=6948660 [st=21000]      SOCKET_CLOSED
t=6948660 [st=21000]   -TCP_CONNECT
                        --> net_error = -118 (ERR_CONNECTION_TIMED_OUT)
t=6948660 [st=21000] -SOCKET_ALIVE

I'm not really sure what I'm reading here but it looks like the connection is timing out and it retries a few different ways but it never works.

Any new guidance based on this?

[KOM]

No idea but perhaps you're having the issue of nginx not being able to serve the files due a MIME type issue?

https://forum.pfsense.org/index.php?topic=109190.0

[pfBasic]

Thank you, I did this and it's the closest thing to working squid that I've seen yet. With these changes both lagado and chrome detect the proxy settings.
Unfortunately, squid (or at least squidguard) doesn't work.

If port 80 is open, then proxy files are transmitted, autoconfigure is completed and lagado and chrome report using the proxy settings. However, everything appears to be bypassing the proxy somehow? Apparently nginx is opening its listening port (nmap reports port 80 on my pfSense box opened by nginx) on my LAN, because with that configuration enabled, port 80 is opened, if I change the listen port, then that port is open (my rules block ports 80 and 443 except in a few specific circumstances). I still don't understand how this is allowing SSL (443 is closed, nginx didn't open it, and nmap doesn't report it opened) but not applying squidguard rules to SSL?

If I close port 80 after the proxy file has been downloaded, then it simply destroys the internet connection,

I tried forwarding all port 80/443 traffic to 127.0.0.1 on 3128 to force http&/s traffic to squid, but that didn't work either.

Any suggestions?

At this point I'd also be interested in a way to use shallalist on pfBlockerNG…. pfBNG does everything that I want from squid except shallalist, and it just works with no issues.