Issue with network and Gmail and other Google pages

codybadger

I'm having an odd issue here.

Sometimes the check all box, individual check boxes, refresh button, and 3 vertical dot button in gmail are either completely missing, or don't do anything when clicked:

Sometimes I can't click on any of the buttons on the top right of gmail (user account (my pic), other apps (9 dots)). Like, they'll be there, but there's no menu when I click.

Sometimes these buttons are missing or unclickable when replying to a message:

Sometimes the page will seemingly load for minutes on end

Sometimes some google documents will open but then after about 30 seconds, they'll say they lost connection, need a refresh, and then reopen at the top of the doc.
Sometimes google image searches are unclickable
Sometimes I can't click on google calendar events to see the detailed information
Some websites can't run certain apps/scripts.

Sometimes this is an issue, and sometimes it's not.

I set up my pfsense box and unifi AP's about two months ago, and everything was working fine, as far as I could tell. Recently we've been having trouble on all of our devices. I didn't make any changes, but this started happening about a week ago. I ignored it for a few days, thinking it may be an issue with the ISP, but it persisted. Then, I decided to switch from my main home network that has pfblocker and runs a VPN to a different network I have (pfsense vlan) that's just open to the internet. This (or hotspotting via cell phone) allows us to do whatever task we were trying to do.

I haven't tried isolating the problem to ad blocker vs vpn, but I'm guessing it's one or the other, perhaps.

Here are my firewall rules for my main network:

And here are my rules for the network that has reliably passed traffic this week:

I'm somewhat new to networking.

JKnott

@codybadger said in Issue with network and Gmail and other Google pages:

I'm somewhat new to networking.

So, why do you think pfSense is the cause? Those web sites use https, which means they are encrypted and pfSense doesn't see anything in those pages, etc.. All it's doing is passing IP packets, which contain that encrypted traffic.

codybadger

I don't know what the cause is. pfsense could be one. I don't think pfsense is the cause more than i think it could be an issue with my ISP or with my unifi hardware or with my chrome setup. I figured since i was switching to a network with different firewall rules, a good place to start would be looking at my firewall. But if you suspect something else, I'm open to many rabbit holes, not just the one I mentioned. @JKnott do you have any helpful ideas to prove or disprove anything?

JKnott

@codybadger

There are 2 possible causes, the web site and the browser. There is no other point where anything other than encrypted traffic is visible. I had a similar experience recently. I was using Google Meet to have a video chat with my friends. We noticed that some of us had "Presentations" available, some had "Captions" and some had both. I found using the Chrome browser provided both, but Firefox might have issues with one or the other.

If you're going to work with networks, you're going to have to learn to isolate problems.

NollipfSense

@codybadger You have pfBlockerNG enabled and running ... maybe, you enabled a feed that blocks certain aspects of Google mail. I have Gmail and never had those issues you're experiencing ... I using Mac though.

JKnott

@NollipfSense said in Issue with network and Gmail and other Google pages:

@codybadger You have pfBlockerNG enabled and running ... maybe, you enabled a feed that blocks certain aspects of Google mail. I have Gmail and never had those issues you're experiencing ...

Given that all pfSense sees is a stream of encrypted data, how could that possibly have any effect?

I using Mac though.

That's OK. We all have our faults.

NollipfSense

@JKnott said in Issue with network and Gmail and other Google pages:

Given that all pfSense sees is a stream of encrypted data, how could that possibly have any effect?

That's why I said maybe ... I have noticed that Google will have multiple IPs similar to the original established connection one feeding port 443 when using Android based device. Come to think about it, it seems that the OP issues are browser related.

JKnott

@NollipfSense

There's no maybe about it. Port 443 is https, which is encrypted http. So yeah, I'd expect to see it being used on sites that use https.

Fire up Wireshark or use Packet Capture to capture port 443 traffic, to see what you get.

Also, doesn't pfBlockerNG block DNS requests? Once the OP has reached Google, DNS is no longer part of the equation, though it might block requests from the pop up ads, etc..

bmeeks

@codybadger said in Issue with network and Gmail and other Google pages:

I don't know what the cause is. pfsense could be one. I don't think pfsense is the cause more than i think it could be an issue with my ISP or with my unifi hardware or with my chrome setup. I figured since i was switching to a network with different firewall rules, a good place to start would be looking at my firewall. But if you suspect something else, I'm open to many rabbit holes, not just the one I mentioned. @JKnott do you have any helpful ideas to prove or disprove anything?

My bet is your issue is related to the ad blocking you are doing with pfBlocker. Your VPN could also come into play.

I'm not here to bash tools such has pfBlocker or Snort or Suricata (I maintain the Snort and Suricata packages for full disclosure). All of these types of packages can cause problems by over-aggressive blocking of traffic. Both need regular maintenance tuning by whitelisting IP addresses or disabling certain rules to prevent issues such as you are seeing. These packages (pfBlocker and Snort/Suricata) regularly download updated lists. In the case of pfBlocker, it's updated IP address lists. For Snort and Suricata, it is updated detection rules. In either case, one of those updates might result in something getting blocked today that was not being blocked yesterday because of a recent change in the list content that was downloaded. Thus things that were "working" suddenly "quit working" for no apparent reason and without you doing anything manually.

So begin by disabling pfBlocker on your firewall and see how things work. If you have success, then turn pfBlocker back on and start to examine each pfBlocker alert to see if you can find what IP address, when blocked, gives you an error. Then whitelist that IP address.

Now let's talk about your VPN. There are web sites out there that will actually refuse to allow traffic coming from known VPN IP address blocks. I doubt Google is one of those, but just be aware there are some out there. Some sites, whether rightly or wrongly, consider "VPN for privacy" to really be a form of "camouflage for nefarious activity" and thus block access from known VPN IP farm addresses. So just keep that in mind as you are troubleshooting. Might try your Google connections outside of the VPN tunnel to see if that works.

But my bet is a pfBlocker list is blocking something those web pages need in order to function. However, @JKnott has a valid point that it can also sometimes be the browser itself. If you want to use Google features (Gmail, Google Docs, etc.), you would be best served by also using Chrome as your browser when accessing those sites.

JKnott

@bmeeks

I've been around long enough to remember the browser wars, when Microsoft tried to corrupt http with Internet Explorer. They'd come up with their own way of doing things that would break other browsers such as Netscape. Fortunately, those days are long behind us, especially since Bill & Steve left. Even still, there are differences between browsers. For example, when I log into my pfSense box with Firefox, it remembers my ID and password. Chrome doesn't. Both are running on Linux.

JKnott

@bmeeks

BTW, I find it a bit much when people load up something with all sorts of crap and then complain something doesn't work properly, when the problem is caused by some of the crap they've install on top of that something.

bmeeks

@JKnott said in Issue with network and Gmail and other Google pages:

@bmeeks

I've been around long enough to remember the browser wars, when Microsoft tried to corrupt http with Internet Explorer. They'd come up with their own way of doing things that would break other browsers such as Netscape. Fortunately, those days are long behind us, especially since Bill & Steve left. Even still, there are differences between browsers. For example, when I log into my pfSense box with Firefox, it remembers my ID and password. Chrome doesn't. Both are running on Linux.

Me, too. I finally settled on Chrome for both my Windows and Linux machines for the last three years or so. Although I will say Microsoft Edge is not too bad. It is certainly heads and shoulders above Internet Explorer.

I use Chrome on Windows and Linux with the uBlock Origin and AdBlock for YouTube extensions to snuff out ads.

bmeeks

@JKnott said in Issue with network and Gmail and other Google pages:

@bmeeks

BTW, I find it a bit much when people load up something with all sorts of crap and then complain something doesn't work properly, when the problem is caused by some of the crap they've install on top of that something.

Yep! Many users get carried away with adding tons of lists to pfBlocker or enabling all of the rules in Snort or Suricata. Both can lead to lots of headaches, especially for a new user of those kinds of tools.

Gertjan

@codybadger said in Issue with network and Gmail and other Google pages:

I'm having an odd issue here.
SUBJECT : Issue with network and Gmail and other Google pages

Better add the missing part to the subject lien .... Chrome (made by ...and controlled by ... ;) ).
Now you have the best of world's biggest 'publicity' company in one bucket.

An then you placed another bucket in front of it, called "pfBlockerNG" and fill it up with feeds that mostly block ... pub sites.

Well ...
The non-technical answer might be : lock up a cat and dog in a room, and be surprised that your here a lot of fighting in there.
.... and some one is gona pay the vet real soon ...

"pfBlockerNG" is like a big gun, pointed downwards - for safety, and not loaded (no feeds). That's how it's installed.
Then the admin starts to load it with 'feeds' without aiming (I mean : selecting the right feeds).
"pfBlockerNG" is in auto trigger mode.
=> Your own feet start to hurt terribly ......

Ok, I'll be more serious.
When you even think that "pfBlockerNG" is blocking something it shouldn't : go to the Firewall > pfBlockerNG > Reports > Alerts page. Check the Deny and DNSBL list.
You'll see the most incredible URL's being blocked.
And before you say : I never visited these sites, know that your browser does so, your TV also, your phones is actually doing so, etc etc. Mail clients, Google web pages etc etc pull in the most incredible content from even more incredible sources (hosts - the URL's).

Now for the bad news : "pfBlockerNG" is a nice tool and it's even free. But good results are not free at all : it needs a lot of your time so you can learn how to use it - you even have to know how it works, so you can instruct it to work correctly for you.

Btw : I'm not against Google - I consider myself even a 'fan' of that company. It can be very invasive, which needs some control. And that's ok for me, I'm a spare-time firewall admin, not a full time bicycle repair guy.
( and yes, no worries, I love bicycles to - I'm from 'holland')

edit : @bmeeks said the same thing, using far less words ...

codybadger

Ok, got it. Thanks all for your input. I'll look into the pfBlockerNG lists and see if I can locate the problem. Just seems so odd that there would be something in there that specifically blocks the check boxes in gmail, for example. The tough part about troubleshooting this is that it doesn't happen all of the time. Sometimes, everything loads properly.

FWIW , I have a desktop PC running windows, android phones, iphones, and macbooks all using chrome that show this issue.

@JKnott said in Issue with network and Gmail and other Google pages:

BTW, I find it a bit much when people load up something with all sorts of crap and then complain something doesn't work properly, when the problem is caused by some of the crap they've install on top of that something.

You think a VPN and pfblocker is "all sorts of crap?" and you think me trying to learn how to resolve the issue is "complaining?"

JKnott

@codybadger said in Issue with network and Gmail and other Google pages:

You think a VPN and pfblocker is "all sorts of crap?" and you think me trying to learn how to resolve the issue is "complaining?"

I wasn't speaking just of you, it's a common problem. However, when resolving a problem, you have to keep things simple and then try to determine what introduces the problem. For example, if you had just plain pfSense, without pfblocker, I doubt you would have seen any issue, as pfSense simply cannot even see the things you're complaining about, as all it sees are encrypted packets. Therefore it is not the cause of the problem. On the other hand, pfblocker might, if what you're seeing is the result of it blocking some stuff etc.. You have to break the problem into pieces to see what makes sense and what doesn't.

bmeeks

@codybadger: I'm not sure what your level of IT and web technology expertise is, so you may already understand what I'm about to tell you.

When you visit a web site or use any web-based application, things that load into your browser as you navigate the site and/or use tools in the web-app such as pushbutton or checkbox icons come from all over the place. They do not necessarily all come from the same IP address. For example, some static content text and images may come from one IP address (server) while javacript to operate buttons and other active elements on the page may be called in from a completely different server at a completely different IP address.

So think about this situation when tools such as pfBlocker are enabled. It could be that the server address where some web application is pulling in its required javascript just so happens to be in the same IP netblock as other servers that serve up javascript used for pulling down browser ads. pfBlocker might have that entire IP netblock in one of its ad block lists. Thus it will blindly block all access to IP addresses in that netblock, and one of those IP addresses might be the one your browser is trying to pull down javascript code from to make one of those buttons work in your mail client (just a hypothetical example, but you get the idea).

To see what is actually happening you need to examine everything your pfBlocker setup is blocking. Could be something there is what is causing the web application issues. Of course as a first test, simply turn off pfBlocker completely and see if the apps work then. If they do, you have your answer as to where the issue resides.

NollipfSense

@JKnott Wireshark - well, since I upgraded Catalina recently, the app won't run and has been investigating. What I meant earlier, I have observe live a connection to Google services with an IP and port 443, then once that connection established, then lots of UDP ports get opened on the same exact IP.

JKnott

@NollipfSense said in Issue with network and Gmail and other Google pages:

then lots of UDP ports get opened on the same exact IP

Those are not part of whatever Google app. They could be something to do with the ads and stuff, but anything directly having anything to do with the Google service would be encrypted, which means you won't see any ports other than 443.

BTW, what's "Catalina"?

bmeeks

@JKnott said in Issue with network and Gmail and other Google pages:

BTW, what's "Catalina"?

I know just enough about Apple to recognize that as the name of their lastest OS update. From visiting other forums I frequent for other technologies (video and audio, for example), I've learned that a lot of older applications for Apple hardware have trouble running under the latest Catalina OS. Kind of like the old Windows 95 versus Windows XP thing from long ago.