@shaungehring This sounds similar to an arp cache issue we had. We could not connect, ping it, then all was good. The network team did something to the arp cache on a switch to resolve it. I do not have details as it was many years ago.
Maybe that will get you in the right direction.