Google G-Suite App Issues
-
Hello all,
Last year I deployed a Netgate XG-7100 unit at a small school I work at. I apologize for the lengthy explanation; however, this is weird issue. The unit is running the following services:
- DHCP
- DNS
- SquidGuard
Recently, I have had intermittent issues with G-Suite apps on iOS and Android devices only. The apps state they do not have internet access. However, everything works as desired on the device’s web browser. When I am trying to log into the Google Docs app on multiple devices, I am prompted with an SSL error. I replicated this issue with multiple devices (2 iPads, and 1 iPhone.) I ensured the applications on all devices were up to the current release. Additionally, on LTE or Hotspot the devices can successfully log in and access contents. All the devices are receiving the correct network parameters supplied by DHCP. There has been no configuration change made to the system.
I tried the following steps with no success:
- Restarted then Disabled SquidGuard and tried again, no luck.
- Next I reset the DNS resolver, no luck.
- Restarted DHCP service
- Then rebooted the entire system.
Any advice would be greatly appreciated.
Patrick
-
Tidied up your question to make it more readable.
Check the Squid logs for 409 errors. If you see them it's almost certainly a DNS issue:
https://docs.netgate.com/pfsense/en/latest/cache-proxy/squid-troubleshooting.html#sites-not-loading-with-splice-error-409-in-access-logSteve
-
Steve,
Thank you very much, I will look into that. Sorry my question posted weird in the forum initially.
Patrick
-
@stephenw10 So after reviewing the logs I see numerous 409 error messages. I have attached a screenshot of the log. Can you please confirm that I am looking at the correct table?
I followed the directions to clear the cache. However, I still have the same issue. So it appears my next option is to uninstall SquidGuard and reinstall?
Thanks,
Patrick Dowd
-
Yup that's the 409 log. It's caused because that clients have requested that fqdn at a different IP than Squid resolves it to.
You have to make sure Squid and clients are using the same DNS server to stand the best chance of avoiding it. I would advise you using the DNS resolver in pfSense only for both.Steve
-
@stephenw10 So DHCP gives client machines a DNS server of the default gateway which is 192.168.1.1 . Under general setup, I have the DNS servers set to Google's DNS addresses. So what you are saying is that my DNS servers in there should be set to what my clients are using (192.168.1.1)?
Thanks for the Help,
Patrick
-
It depends what you have the DNS resolver set to do in Services > DNS Resolver.
If it's set to forwarding mode it will be using the servers in General Setup but the default there is to resolve directly.
I suggest removing those google servers. And uncheck
DNS Server Overridein that same section. Then it will only use the resolver which also caches the result so both Squid and clients should see the same IP.Test it from Diag > DNS lookup. The only nameserver listed should be 127.0.0.1.
Steve
-
@stephenw10 Thank you very much for that info. This afternoon I finally had some downtime after hours to try changing the config. I have gone through and removed Google DNS addresses. The DNS Server Override and Forwarding mode options were already unchecked. When I run the test in DNS Lookup It does show the only entry as being 127.0.0.1.
Unfortunately, this has not solved the problem that I am experiencing still with iOS and Android devices. I still observe the 409 error codes in the real-time log within squid. I followed the instructions on clearing the cache and have had no luck fixing the intermittent problem.
I noticed within the DHCP server setup that there is an option to leave the DNS server blank and the system will hand out the default DNS address. Could that possibly be my issue?
I really appreciate your help in troubleshooting this issue.
Patrick
-
If the DHCP server is set to 192.168.1.1 anyway that will be the same thing.
You may be seeing some hard coded DNS servers in IOS or Android. You might try running a packet capture for port 53 on one of the affected devices to see if they are reaching some external server directly.
If they are you could try redirecting DNS requests to Unbound:
https://docs.netgate.com/pfsense/en/latest/dns/redirecting-all-dns-requests-to-pfsense.htmlSteve
-
@stephenw10 Thank you for sending that to me. I added the NAT rule this morning. I have attached a picture of the rules. Is this setup in the correct order for the rule to be applied?
I am still having the same residual issues, I have been running WireShark on the network and notice now that the network is being spammed with MDNS queries.
From what I read this protocol is a collective DNS server between computers on the network. We are a mostly Apple campus. Am I correct to suspect this as being a culprit causing DNS interference and should block the port number 5353?
Thanks again,
Patrick
-
mDNS should not effect global DNS like that. It's probably not an issue.
The firewall is OK but it needs a port forward to actually redirect the traffic. Can we see that too?
Steve
-
@stephenw10 Ok thank you. I have attached the picture of the port forwarding down below.
Before I left I made a breakthrough on the issue. I was going through the squid configuration and deselected the "Enable SSL Filtering". Everything after that point worked and seemed to resolve the issue. However, doing so completely disabled the filtering of websites.
The setting below was also set to Splice All. The next option is Splice Whitelist, Bump otherwise. So it's clearly something wrong with the splicing that was occurring. What options do I have at this point?
Thanks,
Patrick
-
Ok, that looks correct.
I don't see any states or packets on the firewall rule against that so there might not have been any DNS traffic there. If you enable logging on the firewall rule you can review it later.
The issue only really occurs with https traffic simply because all the large CDNs are almost exclusively https on the modern internet. It you disable https filtering none of that traffic is inspected so it never resolves differently.
Steve
-
@stephenw10 Perfect thank you! So my settings within Squid have not changed so what variables could have caused the splice setting to have this effect on Google? Is there any workaround to fix Google apps while preserving HTTPS filtering?
Edit: Also I avoided the Secondary option within squid which would require CA's to be applied to computers. We support a BYOD and that I think would become an operational headache.
Thanks,
Patrick
-
The only fix I'm aware of for those 409 errors is to make sure everything is resolving to the same IP and that is usually accomplished by using the same DNS server. It's likely to become more if an issue as more things start to use DNSoverHTTPS etc.
Steve
-
Hi all,[link text]
Finally this issue is the same i have link link text it summarized as following :
google suit and whatsapp and other applications using websocket will be cut off by squid ssl filtering which i believe this is a bug with the squid that's lead me to find other solution rather than pfsense to filter https websites until pfsense solve the issue -
That looks like a completely different problem unless you were also seeing 409 errors. I don't see that anywhere.
-
@stephenw10 So if my DNS config is correct as I have verified, this is the now a squid issue. Is there a way to add an exception to the Man In the Middle configuration? Could this have been caused by an automatic update to the squid package?
Is there another package that can do content filtering that won't be affected by DNS over HTTPS?
Thanks,
Patrick
-
@msaeed said in Google G-Suite App Issues:
link text
Dear msaeed,
My issue has been based exclusively on Google application suite on iOS and Android devices only. All other internet functionalities on the device as well work fine just with the apps. I checked my DNS routing and all was correct. I finally disabled Squid SSL Filtering and the problem has been resolved. The issue is in doing so my content filter is now down.
-
The only other option for web filtering is doing it via DNS using DNS Blacklist in pfBlocker-ng. That is quite effective and a lot lighter than Squid. Devices using DNSoH or DNSoT will bypass the filtering but will not fail to connect.
https://docs.netgate.com/pfsense/en/latest/packages/pfblocker.html
Steve
