website security problems

bPsdTZpW

@jc1976 I suspect DNS is serving incorrect addresses, or possibly some security software is acting as a MITM and is not handling amazon.com's redirect correctly. What's the output if you nslookup amazon.com , then nslookup www.amazon.com ? Is the former IP in a range owned by Amazon (lookup at https://whois.arin.net )?

FWIW, I do not see this problem when enabling HTTPS-only mode in FF (v94.0).

stephenw10

Mmm, I could certainly believe DNS shenanigans.

Try just resolving locally, stop sending your queries to Cloudflare.

jc1976

@stephenw10

Hey fellas! really, thank you SO much for your input in all this!

the issue also came up on edge and chrome. i only have firefox and edge on my home workstation. I tried it all on my laptop as well and had the same issues.

regarding resolving locally instead of cloudflare, i'm a bit uncertain what you mean by that. DNS settings are:

127.0.0.1
1.1.1.1
1.0.0.1

I don't use dnsmasq (forwarder), just unbound.

disabling https: only mode in all windows (under firefox) seems to have things behaving more normally, but again, i don't have ANY services enabled (suricata, pfblocker, squid/clamav). so who knows what will happen if i were to re-enable those.

I also don't have any rules setup, so i don't know what is causing my trading apps to hang upon startup, at least not at the moment because again, anything that would block them has been shut down..

i've even shut down my windows defender firewall to make sure that couldn't be causing the hang-up.. it's all very weird.

Johnpoz mentioned filtering 301 redirection.. i'm gonna do my research and find out how to figure that out because this whole thing feels like a filtering issue, i just don't know how to go about figuring that out (still learning).

i trying to explain to our consultant at work what goes on..
-i opened up firefox to DDG.
-searched for ad blocker test
-given a list of sites to test ad blocks.
one of which i used is "https://canyoublockit.com"

clicking on the link for https://canyoublockit.com yields nothing.. the site just doesn't open. clicking multiple times does nothing, it just sits there as if i didn't click on anything. sometimes it would give me the security error... but otherwise nothing happens.. then if i close out and come back, maybe the site would open.. it's random..

stephenw10

@jc1976 said in website security problems:

I have cloudflare setup for dns over tls

If you are doing that then you're sending DNS queries to them rather than resolving locally. Assuming it's setup correctly.

What error do you see in Edge? Or any other browser on any other client?

Steve

jc1976

@stephenw10 it's not even the browsers, it's my software.

i have an old crappy cisco rv042 from long ago as a backup.
right now, i have that plugged in because aside from websites not loading, even my trading apps are hanging.

i have EVERY service/package shut down so there shouldn't be anything blocking anything, other than the default ruleset that blocks all incoming connections.

I'm going to have to reinstall it all from scratch when i get a chance.
really aggravating.

stephenw10

Backup your config, restore to defaults, re-test. It's easy to restore the config if it doesn't help.

jc1976

@stephenw10 yup!

i'm happy to say that my GF left for the weekend so performing a ground-up reload of pfsense is what i'll be doing tonight after i get home from work.

I'll keep you posted.

Thanks again for all your input!

jc1976

@jc1976
looks like i'm back in business..

gparted the drive and reloaded pfsense from scratch. I've got the base config down, only modifications done are to dns over tls with cloudflare..

so far, so good.. very snappy and the throughput is where it's supposed to be..

did a backup.. not sure if i wanna bother adding packages yet.. it's nice to have it back up and running. just reeeaaaaally despise the ads and marketing from my devices..

nimrod

@jc1976

I dont see any reason why you should not install pfBlockerNG.

jc1976

@nimrod well, i started off with installing squid..

i ended up canning squid/clamav because even in it's basic installation it was causing issues.

for good measure i also rebuilt the tcpip stack and winsock in my windows 10 pro install..

i spent a while working through basic rule creation to make sure they wouldn't interfere with my apps, that way i know if i have any issues when i install suricata and pfblocker i'll have a better idea of who's causing problems, should any arise.

What are peoples experience with the feodo, c2c(?), sslblacklist options that are part of the suricata install?

do they cause a lotta false positives?

johnpoz

@jc1976 said in website security problems:

i ended up canning squid/clamav

Ah you were running through a proxy - that for sure could cause speed issue.. Because your not just routing traffic your processing all of it as the proxy, etc.

pfblocker shouldn't be able to cause slowdown - since its just doing dns.. and filtering what dns you want to filter, it has nothing to do with the actual processing of the traffic through pfsense.

jc1976

@johnpoz nah not through a proxy.

i am SO FED UP WITH THIS!! it just doesn't work!

i thought it was my pc, so i switch... same thing.. i'm running pfsense stock out of the box, only change is that i have dns over tls enabled with cloudflare..

just got a gl.inet that i was going to set up for my folks.. maybe i'll take that.. i dunno.. tired of dealing with this..

sorry for the venting

johnpoz

@jc1976 said in website security problems:

i have dns over tls enabled with cloudflare..

And do your security issues go away if you just let pfsense do dns? I have been using pfsense for 10+ years and there is really nothing in pfsense that would do this..

As I went over going to amazon.com without a www gets you a redirection 301 that tells your browser to go to www.amazon.com etc.c.

Pfsense has ZERO to do with that - ZERO!! There is nothing other than IPS or a proxy that would have anything to do with any of that.. Are you trying to go through cloudflare proxy to get to these sites?

stephenw10

@stephenw10 said in website security problems:

Mmm, I could certainly believe DNS shenanigans.

Try just resolving locally, stop sending your queries to Cloudflare.

Yup.

bPsdTZpW

@stephenw10 said in website security problems:

@stephenw10 said in website security problems:

Mmm, I could certainly believe DNS shenanigans.

Try just resolving locally, stop sending your queries to Cloudflare.

Yup.

Yup.

jc1976

@bpsdtzpw
ok folks, i think i've figured it out...

firstly, i apologize profusely for venting.. I was really pushed to my limits with this thing.. i love pfsense and will always support it.

unfortunately, A) i don't know how to get in touch with an actual "engineer" at comcast/xfinity, and B) i dunno if that would even matter...

if you think i'm a conspiracy theorist, well....?? check this out.

I believe comcast is blocking dns over tls on alternate dns providers.

i don't have any issues using the ip of other providers, however once i enable DoT (out port 853) all the problems start happening. I tried quad9, and at first all things sprang back to life.. a hour later it got wonky again. far better than with cloudflare, but still isn't working properly..
All web browsing works properly if i enable DoH in my browser. My problem is my apps. I have a few big packages that i use for trading futures and those are running into problems. once i disable DoT the issues abate...

Some might think "well, just stop using DoT or just use Comcasts resolvers... My response is: i pay for this connection.. i should be able to use it however i want, PERIOD! Privacy is a RIGHT! and if we want to use another dns provider, we have THE RIGHT to do so and our isp does NOT have the right to stop us!

johnpoz

@jc1976 said in website security problems:

I believe comcast is blocking dns over tls on alternate dns providers.

That is a pretty big claim.. And your dot client should complain that the cert of the server its talking to does validate..

Blocking 853 would be one thing - but seems your saying its intercepting and sending you wrong info? A simple way to validate dot is with the kdig util

example here is to google dns that support dot

root@NewUC:/tmp# kdig -d @8.8.8.8 +tls-ca +tls-host=dns.google.com www.pfsense.org
;; DEBUG: Querying for owner(www.pfsense.org.), class(1), type(1), server(8.8.8.8), port(853), protocol(TCP)
;; DEBUG: TLS, imported 128 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, CN=dns.google
;; DEBUG:      SHA-256 PIN: 65dUhctCga6DL7Xr0hDFi+V8V/3asGMbMO7Hg3MKhTY=
;; DEBUG:  #2, C=US,O=Google Trust Services LLC,CN=GTS CA 1C3
;; DEBUG:      SHA-256 PIN: zCTnfLwLKbS9S2sbp+uFz4KZOocFvXxkV06Ce9O5M2w=
;; DEBUG:  #3, C=US,O=Google Trust Services LLC,CN=GTS Root R1
;; DEBUG:      SHA-256 PIN: hxqRlPTu1bMS/0DITB1SSu0vd4u/8l8TjPgfaAp63Gc=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted. 
;; TLS session (TLS1.3)-(ECDHE-X25519)-(RSA-PSS-RSAE-SHA256)-(AES-256-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 13175
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 512 B; ext-rcode: NOERROR
;; PADDING: 404 B

;; QUESTION SECTION:
;; www.pfsense.org.             IN      A

;; ANSWER SECTION:
www.pfsense.org.        300     IN      A       208.123.73.69

;; Received 468 B
;; Time 2021-11-22 13:23:39 CST
;; From 8.8.8.8@853(TCP) in 64.3 ms
root@NewUC:/tmp#

Not a fan of dot or doh - but I am with you that ISP shouldn't be doing anything with my traffic other than routing it.. I normally block dot and doh locally so that none of my devices attempt to use it.. But that is my call, not the isp call, etc.

stephenw10

Mmm, exactly that^.

I could believe something is blocking port 853 entirely but not that it's intermittently blocked or only for some types of queries. Because, yeah, the latter implies something is intercepting the TLS and MITMing it.
It seems more likely that Cloudflares DoT resolvers respond differently in some way. Though I'm not sure why that would be.

I personally have DoT configured to Google DNS just to test that it works and have never seen an issue.

Steve

johnpoz

Here is doing such a test to cloudflare

root@NewUC:/tmp# kdig -d @1.1.1.1 +tls-ca +tls-host=cloudflare-dns.com  www.pfsense.org
;; DEBUG: Querying for owner(www.pfsense.org.), class(1), type(1), server(1.1.1.1), port(853), protocol(TCP)
;; DEBUG: TLS, imported 128 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, C=US,ST=California,L=San Francisco,O=Cloudflare\, Inc.,CN=cloudflare-dns.com
;; DEBUG:      SHA-256 PIN: RKlx+/Jwn2A+dVoU8gQWeRN2+2JxXcFkAczKfgU8OAI=
;; DEBUG:  #2, C=US,O=DigiCert Inc,CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1
;; DEBUG:      SHA-256 PIN: e0IRz5Tio3GA1Xs4fUVWmH1xHDiH2dMbVtCBSkOIdqM=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted. 
;; TLS session (TLS1.3)-(ECDHE-X25519)-(ECDSA-SECP256R1-SHA256)-(AES-256-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 54600
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 1232 B; ext-rcode: NOERROR
;; PADDING: 404 B

;; QUESTION SECTION:
;; www.pfsense.org.             IN      A

;; ANSWER SECTION:
www.pfsense.org.        300     IN      A       208.123.73.69

;; Received 468 B
;; Time 2021-11-22 14:48:04 CST
;; From 1.1.1.1@853(TCP) in 40.1 ms
root@NewUC:/tmp#

stephenw10

Indeed. And testing pfsense.org returns exactly the same thing. As expected.