Ecobee thermostat can’t connect to servers
-
@ezhawk Might help to identify what the Cisco router is?
Reaching out to ecobee is not a bad idea either. If the device has not been rebooted, they should be able to see past logs which may contain insights into the issue.
It's very important you provide them with incident time stamps. Asking for escalation to tier 2 may also help as this issue is indeed quite bizarre.
Technically both pfsense and other devices adhere to rfc standards, but im confident there's nuanced differences in implementation.
For those of us here it would also help to have a clear topology of your network hardware. From the above, im not clear if pfsense is out of the loop entirely when the cisco device is present.
-
@GPz1100
Really simple but this is Topo. I have to reboot the Ecobee to get it to switch paths. Same internal subnet, but I use a different default gateway for the IP on the Cisco to send traffic out via the Cisco. All my network traffic other than the Ecobee goes through pfSense and I don't have any issues with anything.Both Layer 3's links are Layer 3 on the pfSense and Cisco, so it isn't something like a loop taking place. The Cisco was only added after months of trying to figure out why the pfSense is blocking.
-
@ezhawk so where you say "modem" you actually mean gateway.. Because if that was an actual modem.. Traffic out of pfsense to the internet would have a different public IP then traffic flowing through the cisco.
Either your natting these connections at your "modem" (so its not a modem) so you only have 1 public IP seen by the internet or your isp is providing you with multiple public IPs.
-
@johnpoz
It is just a dumb standard ISP D3.1 cable modem that is baselined so it does nothing. Both my pfSense and Cisco have their OWN UNIQUE public IP. NAT is taking place on the pfSense and Cisco respectively. -
@ezhawk said in Ecobee thermostat can’t connect to servers:
Both my pfSense and Cisco have their OWN UNIQUE public IP
And that was my point a while back.. But you dismissed it saying the path is the same - when clearly its NOT!
If you can tell me why when my internet path is 100% the same from Cisco or the pfSense
Clearly that is not the case if your public IP that ecobee sees is different..
My point is ecobee server(s) sending you a fin,ack has nothing to do with what pfsense could be or not doing to the traffic.. They clearly said hey buddy I am done with this conversation.
-
@johnpoz If you want to blame the IP, then you need to explain to me why when the Ecobee goes through the Cisco it can work perfectly for months at a time without a single issue on the same IP. But when traffic for the Ecobee goes out via the pfSense the bast case is it'll work for 1-14 days then just stop. Traffic will restore if I spoof a new mac on the pfSense to get a new IP and then it'll work for 1-14 days and then stop working again. 100% repeatable.
Please explain in specific detail how the specific digits of the IP address that are different between the pfSense and the Cisco are causing traffic to fail. Even more so, the IP the Cisco gets is literally 1 digit away from the one the pfSense has.
Please explain how because IP x is on the pfSense it has issues working with this specific thing while IP y on the Cisco doesn't. And before you tell me they are in different subnets, one is the .128 and one is the .129 in a /22.
Here, I'll even give you what we all know Ecobee would say to start with. "Your connection works on the Cisco router, but doesn't on the pfSense? That seems to be an issue with how the pfSense is processing traffic since you can connect via Cisco and never have an issue. We would recommend working with pfSense. If you couldn't connect with both routers then it could be a device or server issue."
If the convo between Ecobee Servers and my Ecobee can complete without issue on the Cisco. Why does it fail on the pfSense? The communication from my Ecobee to the servers goes to the same destination every single time. If it is the source IP, then why when the source is the Cisco router does it work for months. but if the source is from the pfSense it'll work initially, then fail and won't be able to restore? I've had multiple diff IPs on the pfSense over time via spoofing a new mac. It is the same thing over and over and over. Works for a bit then fails. I've never not once had communication fail on the Cisco and my Cisco has had the same public IP the entire time.
No matter what I do, the only thing that can get the connection to fail is the introduction of the pfSense into the routing path.
There is something that gets completely torn down on the pfSense when a new public IP is received but after a bit becomes stale or hung and isn't getting updated properly. Or the pfSense itself is doing something to the traffic. that is causing it to fail.
-
out of curiosity have you checked with ecobee ?
"ecobee has determined that a very small percentage of Smart Thermostats may experience difficulty connecting to our servers, leading to disconnection issues. If your thermostat has exhibited this problem, please use the serial number checker below to see if your device qualifies for our Connectivity Support Program."
maybe you have and I just missed the reference or mention in this thread
here is the link to check your serial number
https://support.ecobee.com/s/esp/smart-thermostat-connectivity-issues
-
@jrey said in Ecobee thermostat can’t connect to servers:
https://support.ecobee.com/s/esp/smart-thermostat-connectivity-issues
That's no longer a thing anymore according to Ecobee.
And to no one's surprise, Ecobee says it is a problem with the pfSense since everything works fine when the pfSense is bypassed. They could see everything just fine until the pfSense was put in line.
-
@ezhawk said in Ecobee thermostat can’t connect to servers:
And to no one's surprise, Ecobee says it is a problem with the pfSense
Just to be clear (and I leave it after that), my suggestion to contact support was not to blame them but get help from them, getting information about connection made by pfSense and if Ecobee was blocking that connection at some point. I suspect you won't get that info in a chat from a 1st level support.
Your package capture shows a 'Client Hello' from the Ecobee device when connecting to idt.ecobee.com and in a normal conversation, the answer would be a 'Server Hello' from the Ecobee server. But instead Ecobee ends the connection (the connection to home-fw.ecobee.com get a 'Server Hello', seems successful).
-
I am sorry to chime in. By any chance, do you have a process set up on pfsense that periodically and with reasonably high rate sends packets to Ecobee servers? For example, (and only for example) if you have configured Ecobee server IP address for Gateway monitoring purpose in pfSense, then by default pfSense sends ICMP echo requests to that address every 0.5 second. That stream of packets could be enough for Ecobee server to declare DoS attack and blacklist your IP address.
You may want to capture all traffic between pfSense and Ecobee server IP address on WAN interface to check for that.
This may lead nowhere, but it is worth to check, in my opinion.
I do have Ecobee-3 thermostats and I use pfSense (Netgate 4200) and I never had any problem.
-
@akochetkov
No, I don't have any process or service on pfSense set up to monitor the Ecobee IPs. There is a pcap earlier in the thread that I pcap'd my WAN to the Ecobee subnet. And even when the Ecobee can't connect to the Ecobee servers, it can still ping Ecobee.com. -
@ezhawk said in Ecobee thermostat can’t connect to servers:
And even when the Ecobee can't connect to the Ecobee servers,
where did you show this - what you showed was them sending you a fin,ack - this is clearly connecting to what you posted.. If there is something that you can not actually connect to you have not shown that.. Where is the traffic you see hit pfsense lan but not get sent out the wan.
What exactly do you think pfsense could do to the encrypted traffic via a https tunnel, that would force them to send you a fin,ack?
-
@ezhawk I think you need to make the pfsense/cisco comparison be more apples to apples. That is, your entire network should be behind the cisco router while the ecobee is connected. Otherwise, there may be traffic on the pfsense side that does not exist when behind the cisco.
-
That's a great point
@ezhawk said in Ecobee thermostat can’t connect to servers:
NAT is taking place on the pfSense and Cisco respectively.
it's been a while since I had to play with Cisco gear, but it requires specific NAT configuration as I recall -
where the pfSense should just work with no special config for a device like the ecobee, maybe, however this is a case where one of the "it should just work" on pfSense needs a specific setup.The reasons ecobee might be rejecting is because of the NAT setting or lack there of on pfsense ?
Have you configured anything specific for NAT setup on pfSense?
perhaps you could show us the Cisco config for this NAT ? and tell us if you are using default setting on the pfSense box or trying to follow a Cisco style and set something up ?
Have you simply just tried another network cable ? stranger things have been known to happen - is the cable showing you all the settings you would expect from pf to modem ?
As has been suggested by many - there is no obvious reason it shouldn't just work.
it will be the ecobee server - like every other device making the original request to a server, if they are telling you to go away (and they are) there is something about the packet they don't like.Finally if you search the forum for pfSense vs Cisco you will find a number of previous threads (some really old) but you might review them anyway (if you haven't already) and see if anything stands out.
-
@jrey said in Ecobee thermostat can’t connect to servers:
I think you need to make the pfsense/cisco comparison be more apples to apples. That is, your entire network should be behind the cisco router while the ecobee is connected. Otherwise, there may be traffic on the pfsense side that does not exist when behind the cisco.
Already done and to no great surprise, everything works perfectly on the Cisco including the Ecobee.
@ezhawk said in Ecobee thermostat can’t connect to servers:
NAT is taking place on the pfSense and Cisco respectively.
The reasons ecobee might be rejecting is because of the NAT setting or lack there of on pfsense ?
Please provide details of what you might think this is. Even when the Ecobee can't connect to the Ecobee servers, it can ping Ecobee.com from the device itself, so NAT is working.
Have you configured anything specific for NAT setup on pfSense?
No, nothing is specifically configured. And Cisco NAT is dirt simple, all you do is define the inside and outside interface and one line for nat-overload and the ACL for internal subnets.
perhaps you could show us the Cisco config for this NAT ? and tell us if you are using default setting on the pfSense box or trying to follow a Cisco style and set something up ?
Cisco
!
interface GigabitEthernet0/0.999
ip nat inside
interface GigabitEthernet0/1
ip nat outside
!
ip nat translation timeout 1000
ip nat inside source list 101 interface GigabitEthernet0/1 overload
!
access-list 101 remark NAT_Pool_Extended
access-list 101 permit ip 10.0.0.0 0.0.0.255 any
access-list 101 permit ip 172.16.21.0 0.0.0.255 any
access-list 101 permit ip 10.1.0.0 0.0.0.255 any
access-list 101 permit tcp 10.1.0.0 0.0.0.255 any
access-list 101 permit ip 10.99.0.0 0.0.0.255 any
Nat on both is just default.Have you simply just tried another network cable ? stranger things have been known to happen - is the cable showing you all the settings you would expect from pf to modem ?
yeah i've done all that L1 ts.
-
@ezhawk said in Ecobee thermostat can’t connect to servers:
Even when the Ecobee can't connect to the Ecobee servers, it can ping Ecobee.com from the device itself
Your package capture show (as I mentioned in my post before) that it seems that the Ecobee can connect to fw-home.ecobee.com but idt.ecobee.com closes the connection.
Maybe another question: how would you like pfSense to solve it? It won't work without someone who got that issue.
And according to your package capture it does not affect all ecobee.com servers but the idt.ecobee.com. I think that is something to consider.
-
@patient0 Worth exploring tunneling idt.ecobee.com (or all *.ecobee.com) traffic via a vpn. OP says he has no vpn. Cloudflare warp is free and will suit this purpose. I offered some guidance on the previous page how to set it up.
-
@ezhawk said in Ecobee thermostat can’t connect to servers:
No, nothing is specifically configured.
Ok, was just curious if you have configured anything "special" under any of the 4 tabs found under Firewall -> NAT
seems that is not the case..
however also sounds like maybe the overload setting function on the cisco ?
There are numerous posts like this. Maybe in this case you just have to "help" pfsense build the right packet the looks correct for the ecobee server and have it stop telling you to go away ?
https://forum.netgate.com/topic/189729/nat-external-ip-vs-internal-ip-cisco-vs-pfsense
I don't have a ecobee so ... can't really see first hand what is going on out of the box so to speak ...
- the other option presented of using Cloudflare wrap would likely also work.
-
@jrey I don't think this is a nat issue. It would either work or not work, but not likely be intermittent. Indeed there's some traffic ecobee server is receiving which causes it to end the connection.
I have the premium (and the ecobee3 lite before) with no such issues behind pfsense.
@ezhawk how is the ecobee wifi configured? Different vlans for different interfaces? Anything else sharing ecobee's?
-
Does the cisco randomise the source port in NAT?
Perhaps Ecobee periodically does something that assumes a source port and when that fails it marks it as bad. Unlikely but easy enough to test with a static port rule.
Can you spoof the WAN MAC on the Cisco so it has the IP pfSense is currently using where Ecobee is failing. See if it then starts working behind the Cisco. That would prove it's not the Ecobee server blocking the IP.
