Please help me research odd problem with mismatched SSL certificates on HTTPS requests, not running squid

BassStation70

I've recently installed pfSense CE and it has been running well for about a month, but I see an odd problem that looks like pfSense is proxying HTTPS sessions for https://js-agent.newrelic.com/XXXX.js files. This is causing browser errors (un-trusted SSL certificate and/or mismatched SSL certificate).

My question is where to look to see why this is happening, and try to keep it from occurring...

I am running Version: 2.7.0-RELEASE (amd64)
Added Packages: nmap, pfBlockerNG, Service_Watchdog, snort.
I am NOT running squid or haproxy.

When I first saw this issue, the SSL certificate being returned to the browser for the affected URLs was coming from CN=<hostname>-pfBNG-DNSBL-#####, so I looked at pfBlockerNG for the cause. I didn't find anything that looked to be related in the configs, but I did try stopping pfBlockerNG and also stopped snort to see if that was releated. The problem persisted, but interestingly, it switched to using the SSL certificate which I installed for the Admin interface. It still looks like the issue is coming from pfSense, but not from these added packages. This looks to me like there's a selective SSL proxy running in pfSense based on previous experience, but I don't know if that is actually what is happening.

Unfortunately, I don't know what search terms to use to try to understand this. Every search points to squid or haproxy, neither of which I am running. Firewall logs don't show anything at all for this traffic, and I have run out of places to look.

If you can help point me in the right direction to understand this better, I would really appreciate it.

stephenw10

Looks like pfBlocker is serving a replacement object instead of that scrip when it blocked it. newrelic.com is commonly blocked to prevent data collection so it's probably in one of the lists you have loaded.

Steve

johnpoz

@BassStation70 said in Please help me research odd problem with mismatched SSL certificates on HTTPS requests, not running squid:

js-agent.newrelic.com

And what does that resolve to on a client? from a cmd line on a client do a dns query for that with your fav tool, nslookup, dig, host, etc.. or if your using firefox you can look in the firefox dns cache right after you see that error.

about:networking#dns

With Steve, this screams being blocked by pfblocker - and your redirecting to a block page because of it..

CN=<hostname>-pfBNG-DNSBL-#####

Is pretty much smoking gun, but the dns query will show you why its happening.. instead of resolving to what the real IP of that is, its resolving to the pfblocker vip, or pfsense IP, etc.

That fqdn is common block, I block it locally here - but I don't do redirection of blocked stuff, I just hand the client all 0's

;; QUESTION SECTION:
;js-agent.newrelic.com.         IN      A

;; ANSWER SECTION:
js-agent.newrelic.com.  2       IN      A       0.0.0.0

BassStation70

@stephenw10 Thanks for the reply. It makes sense to me that pfBlocker would block that content, but serving the https content itself isn't what I expected at all. I guess I need to understand what it is doing more than I currently do. Your comment helps me know where to look, which was my question.

@johnpoz

C:\>nslookup js-agent.newrelic.com
Server:  FW1.XXXXX.local
Address:  10.10.10.1

Name:    js-agent.newrelic.com
Address:  10.5.5.1

10.5.5.1 is the Virtual IP Address for pfBlockerNG/DNSBL. The DNS server is pfSense, with Forwarding mode enabled, if that matters.

The certificate returned for https://js-agent.newrelic.com/ is still CN=<hostname>-pfBNG-DNSBL-#####

FWIW, I disabled pfb_dnsbl and pfb_filter for the last several hours. During that time, I cleaned up the pfSense firewall rules for my different networks and VLANs. Somehow the underlying problem went away, which mean the original site now works, even with pfBlocker services re-enabled.

I'll consider your suggestion to just block the site altogether rather than letting pfBlockerNG handle it using default rules.

Thanks again for your help. SWMBO is happy for the moment now that she is able to do some online shopping...

BassStation70

I guess if I had clicked through, it would have told me where to look.

SteveITS

@BassStation70 yep. PfB can’t spoof HTTPS certs with valid certs so your browser will show the warning.

Re: still working, could be DNS caching on your device, or a list update, etc.