Blocking of specific services when connecting openVPN via ISP but not 4G/LTE
-
Hello guys,
I've had a really bizarre problem that I just cannot fathom out.
I am a remote worker who uses OpenVPN to connect to the Head Office pfSense box. We have one of the 'official' boxes, the SG-2440 https://www.pfsense.org/products/product-family.html#sg-2440. It is fully up to date with 2.2.5-RELEASE (amd64) / FreeBSD 10.1-RELEASE-p24.
Certain services provided on the Head Office LAN appear to be blocked. There seems to be no reason to why specific machines and specific services on those machines should appear as blocked. It seems random.
For example, I can MS Remote Desktop to 'Server 1', but I cannot use its SMB file shares. It will authenticate my access to the files shares, present me with a list of shares, but upon connection, it will not let me see any content in the share.
Another machine 'Server 2', I can both MS Remote Desktop to, and connect to SMB file shares quite happily.
Another machine 'Server 3', I can authenticate for SMB file shares and MS Remote Desktop, but I cannot get any further (no content).
I can SSH into the pfSense box, but I can't https into the Admin Web UI.
These machines are all on the same Head Office LAN sub-net. Turning off the firewalls on these Windows servers makes no difference.
I have no issue connecting to ANY services that are connected to the head office by a separate IPSec connection (to Azure - different sub-net). All the machines running in the Azure cloud (private VNet) are accessible.
Now the really weird thing…
If instead of connecting from my home with the Virgin Media Business UK ISP connection (cable modem), I connect via my mobile's 4G / LTE - with the exact same openVPN config and client - everything works. I can see, and interact with all the network services at Head Office LAN (and the IPSec Azure ones).
How on Earth can this be happening? The OpenVPN tunnel (UDP) appears to come up in exactly the same way with the Virgin and Vodafone mobile connection when comparing logs.
Could virgin be doing some deep packet inspecting and blocking stuff? How could they do that when it is encrypted, and why would it only block certain things? You would think if Virgin was blocking something, it would block it universally?
I am going more than a little mad trying to work this one out! If anyone can give me a clue where to start, it would be greatly appreciated. As you can imagine, I have poured over routing tables, logs, firewall settings, and they all look fine. If they were the problem, why would it work over Vodafone's network but not Virgin's? The inbound rule for the OpenVPN on the WAN interface is source any/any UDP.
Please put me out of my misery (I've been at this for days!),
Mark. -
Without knowing what address ranges we're talking about here, there are two things that come to mind, and that's where I'd start looking…
-
Your home IP range is somehow partly overlapping your office IP range.
-
Your ISP connection somehow has a broken MTU path discovery causing large packets to fail.
-
-
Hello awebster,
Thanks for replying.
'Your home IP range is somehow partly overlapping your office IP range.'
Home is 192.168.1.0/24
Office is 192.168.168.0/22
OpenVPN tunnel network is 172.16.2.0/24
Azure VNets 10.0.2.0/24, 10.6.0.0/24'Your ISP connection somehow has a broken MTU path discovery causing large packets to fail.'
How would I go about diagnosing / testing this idea?
Thanks a lot for your help!
Mark. -
I should also add that I tried TCP mode, instead of UDP, and it failed in exactly the same way.
Cheers,
Mark. -
Ok, so this isn't an OpenVPN specific issue :-(
I set up a PPTP VPN server on the same pfSense server at head office.
It misbehaves in exactly the same way as the OpenVPN connection.
Random services not available on the head office LAN when connecting over Virgin ISP with PPTP.
All services available on the head office LAN when connecting over Vodafone mobile 4G / LTE network with PPTP.
Could this suffer from the same MTU issue that you described?
Cheers,
Mark. -
Sure sounds like an MTU issue.
You can start reading about it here:
http://www.znep.com/~marcs/mtu/
http://packetlife.net/blog/2008/aug/18/path-mtu-discovery/
Or any of the many links available googling Path MTU Discovery.But first make sure that you don't have ICMP filtering going on in your own environment.
-
Thank you! :D
It definitely is MTU.
I experimented by putting 'tun-mtu 1300' into my client config… and everything works!
I cannot see any ICMP filtering on my home LAN, router or modem, and I can't see anything like that on the pfSense box either. So I am not sure why it isn't being detected.
A quick Google of 'Virgin Media mtu' has a lot of chatter about what the real maximum is, with no definite outcome.
I will increase my tun-mtu setting to see what the maximum really is. I shall continue to look at why the MTU path discovery isn't working.
I cannot thank you enough. I have been banging my head off of this issue for a couple of weeks now, and it has been driving me crazy.
Effectively, it is all working now.
Thanks again!
Mark. -
Just for completeness, I'd like to report that it seems that setting tun-mtu 1387 is the highest I can go before failure.
Cheers,
Mark.