pfSense Blocking Roborock app

bmeeks

@kahodges1721:
Are you running pfSense CE or pfSense Plus? Those have different versions of the unbound DNS resolver, and the version in the current pfSense Plus (1.15.0) has some issues sometimes chasing CNAME records and/or resolving IPv6 stuff. IPv6 does not appear to figure in your problem, though, since those domains resolve to IPv4 addresses.

I also need to ask one more time. When you say you reinstalled pfSense and chose the defaults, does that mean all of the following statements are true?

You installed fresh from either an ISO image or USB memstick image and took all the offered defaults during installation, changing nothing other than probably answering "No" for anything IPv6 related.
You did absolutely nothing to DNS. You did not type in any DNS server IP addresses anyplace at any time during or after the install procedure. DNS is 100% stock, out-of-the-box.
You answered "Yes" during the installation when prompted whether to enable DHCP on the LAN. You took any offered defaults for that configuration as well.
You did NOT restore any kind of previous configuration backup either during the install procedure nor after from the DIAGNOSTICS menu.
You neither added to nor edited any of the firewall rules that came as part of the default out-of-the-box install.
You have not installed (or restored) any add-on packages.

I want to make sure the firewall is truly in a default configuration, because that usually just works for everything.

I can successfully navigate to a web page at both hostnames you provided (usiot and euiot). I get a blank, white page in both instances which I assume is expected as that is really an API for authentication and not for HTML display.

I would also try really basic things like can you ping both host names from the DIAGNOSTICS menu of pfSense? Try using the ping -s 1460 option use a maximum size ICMP packet.

kahodges1721

@bmeeks that’s interesting it worked for you and disappointing bc I was hoping it wouldn’t work for anyone on pfsense haha hmm…

As far as defaults yes it was a fresh download of pfsense CE. All defaults were selected with DHCP on lan. I did enter a dns address of 8.8.8.8 for primary and 8.8.4.4 for secondary. This is also the dns addresses used on the NETGEAR. Nothing else was touched at all. Tested the app and it failed. I then upgraded it to pfsense plus which is what I typically run and same result. I haven’t found a way to install plus directly for some reason so maybe installing CE first is causing something? But what still doesn’t make sense is why it broke in the first place. Nothing about dns or pfsense was touched at all in the past like 6-8 months. I’ve had the same settings for a while and just let it go.

bmeeks

@kahodges1721 said in pfSense Blocking Roborock app:

I did enter a dns address of 8.8.8.8 for primary and 8.8.4.4 for secondary.

Remove those. You do not need them. Out of the box unbound is configured to resolve in pfSense. That means it talks to the root DNS servers and walks down the tree to find IP addresses for hostnames. Also that is quite a bit more "private" than handing Google all of your DNS traffic so they can log where you go and target you with ads and otherwise profile you. Putting in those servers is not "default" . They likely are not the problem here, but that is an unnecessary change to pfSense. Leave the DNS boxes under SYSTEM > GENERAL SETUP blank. Also go back and make sure the DNS server boxes in the DHCP server configuration page are blank. That way, DHCP will hand your clients pfSense as their DNS server (which is generally what you want).

One other thing, make sure the DNS Server Override box is unchecked on the SYSTEM > GENERAL SETUP page!

kahodges1721

@bmeeks again I can’t thank you enough for the help. Unfortunately I had to call it quits for the day and get ready for work tomorrow. I will give it a go again tomorrow after work and update

bmeeks

Your DNS configuration in pfSense is needlessly complex. You have 4 name servers listed. You should only have 127.0.0.1 showing there as the default configuration for pfSense uses unbound running in resolver mode. Your DNS configuration is NOT default. You said earlier you did a fresh install and had everything set for defaults.

Have you enabled "Forwarding" under SERVICES > DNS Resolver?

Have you checked the DNS Server Override box under SYSTEM > General Setup?

What are those two 173.44.120.88 and 173.44.120.89 DNS servers? Are they perhaps from your ISP? You certainly do not want those in the mix in my opinion.

Your DNS setup is NOT default! This could be a source of problems.

Notice in my screenshot for resolving the usiot.roborock.com host there is only a single DNS server listed. It's 127.0.0.1, which is the local unbound daemon running in resolver mode on my firewall. You need to do what I mentioned earlier:

Go to SYSTEM > General Setup and remove all the DNS server IP addresses you entered there (I think you said they were 8.8.8.8 and 8.8.4.4). There should be nothing at all in those DNS server boxes. Leave them blank.
Be sure the DNS Server Override checkbox immediately below on the same page is unchecked (as in cleared or not checked).
Go to the DHCP Server settings under SERVICES > DHCP and remove anything you typed into the DNS server settings there to be served to clients. Make sure all the DNS boxes are empty. That way, DHCP will serve clients the pfSense firewall's LAN IP as the DNS server they are to use. That is what you want.
Go to SERVICES > DNS Resolver > General Settings and uncheck everything on that page except two boxes. Those are Enabled at the top, and DNSSEC about halfway down. Make sure both of those are checked.

After you make all these checks/changes, I would reboot the firewall to be 100% sure the new settings take effect. After rebooting the firewall, toggle your phone's the Wi-Fi off and back on so it will pick up the new DNS settings.

When you make multiple DNS servers available, your clients do not use them in the order you list them. Instead, clients randomly choose one and will continue to use just that one until it fails to respond. Only then will they try another one in the list. This is especially true of Windows machines. This can cause a really confusing mess if the various DNS servers return different results for some reason such as a misconfiguration on one of them. Stuff can then work on one client but fail with another on the same LAN because they choose different DNS servers from the available list to ask for name resolution. The DNS Lookup utility on pfSense will on purpose try every configured DNS server. But normal clients don't behave that way.

kahodges1721

@bmeeks I have a configuration file saved that when I leave I from my computer I load up so it puts the firewall port forwarding and static IPs back into place so the unraid server and various other things work as they should. When I ran that testing it was back after I reloaded the file. When I sit down for testing I have a usb stick that I toss onto the server and start from scratch. That said more information is coming to lights on the integration site. More people are coming forward with the same issue. As it stands now the thought is that the integration flooded the server with MQTT requests to the roborock server so they might have put a ban in place to stop this. If this is true what I don’t understand is why changing the firewall would allow me to get around the ban. Is it possible that the ban is linked to the hwid or MAC address of the PC so when swapping it out it’s no longer banned? That could explain what’s going on here possibly

bmeeks

@kahodges1721 said in pfSense Blocking Roborock app:

@bmeeks I have a configuration file saved that when I leave I from my computer I load up so it puts the firewall port forwarding and static IPs back into place so the unraid server and various other things work as they should. When I ran that testing it was back after I reloaded the file. When I sit down for testing I have a usb stick that I toss onto the server and start from scratch. That said more information is coming to lights on the integration site. More people are coming forward with the same issue. As it stands now the thought is that the integration flooded the server with MQTT requests to the roborock server so they might have put a ban in place to stop this. If this is true what I don’t understand is why changing the firewall would allow me to get around the ban. Is it possible that the ban is linked to the hwid or MAC address of the PC so when swapping it out it’s no longer banned? That could explain what’s going on here possibly

No, a MAC address never goes beyond the local network. It is remotely possible that your ISP gives you a different dynamic WAN IP address when you change devices due to the MAC they can see on your WAN interface port (that goes to their modem), and the Roborock server has that IP banned. You can easily test that by configuring pfSense to clone the MAC address of your Netgear's WAN port.

But I think your firewall is not truly default, or else you have me very confused by some of your statements. You tell me it is default, but then you say this:

@kahodges1721 said in pfSense Blocking Roborock app:

I have a configuration file saved that when I leave I from my computer I load up so it puts the firewall port forwarding and static IPs back into place so the unraid server and various other things work as they should.

If you mean you are uploading a stored configuration to pfSense, then that is not what I consider "default". I outlined very clearly in my previous post what "default" means. It seems to me that you are bringing back some erroneous configuration when you upload that stored file to the firewall. And what do you mean about the USB stick? How does that figure in? It seems we are going around in circles here .

Jarhead

@kahodges1721 Power up the old router, find the MAC of the WAN port and copy it.
In pfSense go to Interfaces/WAN. Enter the old routers MAC in the MAC Address field.
Release/Renew WAN address.

kahodges1721

@bmeeks I think there is just some confusion. Sorry for that. The usb has the most current iso file for pfsense. When I’m playing with the setting and working on the issue of the app not connecting I plug in the usb and load a fresh copy of pfsense onto the computer and run full default config (minus I was entering the 8.8.8.8 and 8.8.4.4 into dns which I won’t moving forward) when I’m no longer working on the issue and leave for the day I load the saved configuration file so things like my Plex server and other integrations that required static ips or port forwarding rules work properly. The screenshots from my phone I sent last night were after I loaded the config file and quit for the day. That is why I had to send them via Imgur bc I didn’t have access to the pc. I won’t be able to revert back to full defaults until I get home from work this afternoon and can continue to test the issue. Hopefully that helps to clear up my process with this a little bit.

That said it seems more and more are showing to have the same issue most with various different routers. Tonight I will load a fresh copy of pfsense and change the mac address as defined above and just test that next.

Please let me know if it’s still not clear and I can try to explain in even more detail.

kahodges1721

@jarhead I plan to give this a try tonight. Will update if this solved anything. I do have a 4 port NIC with 2 not being used on my pfsense machine. In theory I could also just swap the interface to a different port and if it has anything to do with the MAC address this should fox the issue also right? That would be the easier of the two options but I don’t mind booking back up the NETGEAR router if I need to.

Jarhead

@kahodges1721 That would work but you already know the IP you get with the old router works so that would be the best bet.
Try a different port first if you want, just note the IP you get as is, then with the new port. I would also connect the old router and note that IP.
They might have blacklisted a range if they are actually blocking IP's.

bmeeks

@kahodges1721 said in pfSense Blocking Roborock app:

@bmeeks I think there is just some confusion. Sorry for that. The usb has the most current iso file for pfsense. When I’m playing with the setting and working on the issue of the app not connecting I plug in the usb and load a fresh copy of pfsense onto the computer and run full default config (minus I was entering the 8.8.8.8 and 8.8.4.4 into dns which I won’t moving forward) when I’m no longer working on the issue and leave for the day I load the saved configuration file so things like my Plex server and other integrations that required static ips or port forwarding rules work properly. The screenshots from my phone I sent last night were after I loaded the config file and quit for the day. That is why I had to send them via Imgur bc I didn’t have access to the pc. I won’t be able to revert back to full defaults until I get home from work this afternoon and can continue to test the issue. Hopefully that helps to clear up my process with this a little bit.

That said it seems more and more are showing to have the same issue most with various different routers. Tonight I will load a fresh copy of pfsense and change the mac address as defined above and just test that next.

Please let me know if it’s still not clear and I can try to explain in even more detail.

MQTT is, at its core, just TCP packets exchanged with a server either on port 443 or 8883. From the packet captures you posted earlier, I recall seeing connection attempts being made to port 443 of the IPs that were returned from the usiot.roborock.com query. pfSense would not be able to distinguish MQTT from HTTPS. It would simply see traffic outbound to port 443 and let it go when the default rules are in place on LAN and WAN.

kahodges1721

@bmeeks ok so I came home today to begin testing of the changing of the NIC port. Lo and behold it just started working. So I think that solves it. Something on the integration was causing a flood of traffic and ended up getting something banned. The good news is it seems it was temporary. I feel like an idiot spending so much time and effort trying to solve it but in the end I got to really deep dive into various settings. I Learned a lot more about DNS and how it should be set up properly and other various items that I can’t say I knew anything about before so it isn’t all bad! I want to thank you specifically for sure and everyone else that helped support this! We might not of figured out the root cause but man you all helped me learn a lot!

Max 7

@kahodges1721 you need to allow TCP port 8883 and TCP 443