ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN

CrystalFire

Hello Everyone,
I need some assistance in resolving issue that I am facing while trying to access a specific region restricted website - Hotstar India. When I try to access it by using ExpressVPN app, it just goes through fine - opens up and content plays too, however when I try to access it via pfSense OpenVPN - it just doesn't load completely.

It just stops there. I have tried accessing some video contents using direct link as well but they don't play as well and I get VPN/proxy error(Playback Error: You seem to be using a vpn or proxy. Please turn off any of these services and try again. [HWEB_474]).

All other websites load fine, so the configuration does look good. Also, the dnsleaktest results are good too - only 1 server is found.

Tried ExpressVPN IP & DNS Leak Test as well - which is all good and gives same result.

So, I am connected to right IP region, with no leaks.

I have tried working with ExpressVPN support team to resolve this, but they provide limited assistance in regards to pfSense so are unable to help with it.

Would someone be able to help me with resolving this issue please.

vjizzle

Hi,

Glad to found this post. I am experiencing somewhat the same issue. I have expressvpn running on pfsense and when I enable DNS resolver mode I am not able to sign in on the Netflix app. When I enable DNS resolver Forwarding mode to 1.1.1.1 and 9.9.9.9 Netflix works (kinda...). So now in Dns forwarding mode when I open Netflix app on iPhone or AppleTV, most of the times I am missing lots of series in Netflix My List and Continue Watching list. Connecting to other Expressvpn locations does not resolve this issue.

The weird thing is when I connect to the same location using the expressvpn app on my iPhone (for example Amsterdam) Netflix works fine and everything is in place. How can we get the same connection properties on pfsense as the Expressvpn app uses? Expressvpn support was not able to help here sadly. Help and suggestions are welcome.

bingo600

Does the APP and pfSense use (end up with) the same Public (express vpn) ip
Some IP's are blacklisted by ie. Netflix.

/Bingo

CrystalFire

@vjizzle Based on your description of the issue, I think you could be having DNS leak - run "Extended Test" on https://dnsleaktest.com/ and see if you see just 1 server (as in my original post) or multiple. If you are seeing multiple then its your pfSense configuration for ExpressVPN that is not correct.

And in case if you are seeing just 1 server/IP & correct country based on the location you are connecting to then it could be what @bingo600 mentioned in their post - to resolve this try reconnecting multiple times until you are able to get the public IP that works (if it works fine on your ExpressVPN App).

Also, the way now Netflix works around VPN users is it hides the content when it detects the connection from vpn - so that explains why you are missing the contents.

The issue that I had posted in my original post was a bit different and I still don't know what is the reason for it to not work. However, from all the time that I spent on it, what I did discover was the particular streaming service works fine with exact same configuration from pfSense when I use a different VPN provider (PureVPN) - so I suspect it is definitely something to do with ExpressVPN.

While we are at it, what I have also discovered is pfSense and ExpressVPN don't go well together - there is some considerable performance degradation as well (in terms of speed) if you compare the speed test outcome when connected to ExpressVPN via their app vs pfSense.

So, if your reason of using pfSense is just for ExpressVPN then I would recommend that you move to OpenWrt instead - ExpressVPN works perfectly on it without any glitches, there are no performance issues and its very easy to switch countries/VPN Provider with just clicks (i.e. no re-configurations required) after configuring it once.

Let me know how you go!

jrandol561

@CrystalFire said in ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN:

@vjizzle Based on your description of the issue, I think you could be having DNS leak - run "Extended Test" on https://dnsleaktest.com/ hotstar customer service and see if you see just 1 server (as in my original post) or multiple. If you are seeing multiple then its your pfSense configuration for ExpressVPN that is not correct.

And in case if you are seeing just 1 server/IP & correct country based on the location you are connecting to then it could be what @bingo600 mentioned in their post - to resolve this try reconnecting multiple times until you are able to get the public IP that works (if it works fine on your ExpressVPN App).

Also, the way now Netflix works around VPN users is it hides the content when it detects the connection from vpn - so that explains why you are missing the contents.

The issue that I had posted in my original post was a bit different and I still don't know what is the reason for it to not work. However, from all the time that I spent on it, what I did discover was the particular streaming service works fine with exact same configuration from pfSense when I use a different VPN provider (PureVPN) - so I suspect it is definitely something to do with ExpressVPN.

While we are at it, what I have also discovered is pfSense and ExpressVPN don't go well together - there is some considerable performance degradation as well (in terms of speed) if you compare the speed test outcome when connected to ExpressVPN via their app vs pfSense.

So, if your reason of using pfSense is just for ExpressVPN then I would recommend that you move to OpenWrt instead - ExpressVPN works perfectly on it without any glitches, there are no performance issues and its very easy to switch countries/VPN Provider with just clicks (i.e. no re-configurations required) after configuring it once.

Let me know how you go!

OpenWrt works great, thanks. I had the same problem until I found this thread

vjizzle

Hi!

Thanks for all the feedback guys. I actually went and involved ExpressVPN support in this matter and we did some extensive testing.

We first I made sure that pfSense was only sending DNS queries using the VPN tunnels. I set DNS Resolver to Resolve (not forward!) and using the Diagnostics - States I monitored closely what was happening. I also monitored using dnsleasktest.com and made sure I had not DNS Leaks. ExpressVPN also provides a webpage where you can test if you have a DNS leak. So all was well and still on several ExpressVPN exit point (The Netherlands, Germany, France, Belgium) I was not able to access my complete Netflix library. Also the "Continue watching" list in Netflix was not there anymore. But now some other weird thing started to happen also. I was also missing thumbnails in Netflix! This happened on my iPhone, AppleTV and also webbrowsers. Rebooting devices didn't work. Connecting to other ExpressVPN servers didn't solve the thumbnail problem also.

The support at ExpressVPN didn't know why this was happening because when I connect to ExpressVPN using their app on my iPhone to those same locations like I do with OpenVPN client on pfsense, everything was working fine. All the content on my Netflix library was there and also all the thumbnails were visible in Netflix. But the difference here is that the app uses other IP's then what you get when you are using OpenVPN client on pfsense. According to ExpressVPN support the IP's resolved by the ExpressVPN app cannot be used with OpenVPN client in pfsense.

Then ExpressVPN support suggested I use Google or Cloudflare DNS. I set that up on pfsense and enabled Forwarding mode in DNS Resolver. Immediately all my thumbnails were working and visible in Netflix! Off course now DNSLeaktest.com and also the ExpressVPN DNS leak website shows Cloudflare and Google DNS when I do the DNS leaktest. The next step ExpressVPN suggested is trying different servers to connect to from pfsense and find one which was working with Netflix and showed my complete Netflix library and also my "Continue watching" list. After a while (and testing a lot of servers) I did find a few! :).

So now I have those specific ExpressVPN servers configured in pfsense OpenVPN client and also I am using DNS forwarding to Google and Cloudflare. My Netflix is working fine with this setup and my complete library is visible.

To mitigate the DNS leak issue I have enabled TLS dns communication in DNS Resolver and also I have set the VPN connection as the gateway for the DNS servers I configured in the System -> General tab.

ExpressVPN support told me they will look into this. I am using pfSense because I find it very feature rich and I am using several features in there. I think I could find them also in OpenWRT. But can you confirm that when you are using OpenWRT with ExpressVPN OpenVPN client configuration, you can connect to any ExpressVPN servers and have everything in Netflix running fine? Does is show the same behaviour as I experience with the ExpressVPN app? And how have you guys setup the DNS configuration in OpenWRT? Using DNS Resolver or forwarding? I'd like to hear more about this :)!

CrystalFire

@jrandol561 glad to hear that!

CrystalFire

@vjizzle Sorry my email filter had moved this notification to a wrong folder and hence missed your message.

I will post you screenshots of my pfSense configuration later today so that its up and running for you.

OpenWRT works great with ExpressVPN and is pretty easy to setup. Yep, your experience on OpenWRT will be same as you find it with their app - everything in Netflix running fine including the internet speed over VPN. To answer rest of your questions I will need to look at the OpenWRT configuration too as it has been long since I had set it up - I will only be able to do that later today.

By the way, I have installed both pfSense and OpenWRT on same machine so that I can toggle between them depending on my need.

Cheers

Gertjan

@you all :

Keep in mind : Netflix asks 'especially invited' people to subscribe to every possible (popular) VPN supplier, and to use this VPN connection to connect to their services. These subscriptions, time spend, the Netflix account, etc is fully refunded by Netflix.
People are encouraged to contact the VPN tech support of the VPN to see if they can make it work.

If they manage to make it work : Netflix gives an extra bonus, and add the IP that 'works' to their IP proxy black list =>

VPN suppliers sell IPv4's - probably entire networks, buy them back, change their IPv4 between countries, etc.
Everybody understands that, if they could add "works with Disney Netflix etc" to their publicity, this would help selling subscriptions.

Just to re explain the 'why' part :
Ge-restricted access to content is most often based on contracts implying huge amounts of money. The 'majors' dict the rules here :they can - and will - decide which 'video' can be shown to using which channel in which country.
Because they own the content.

vjizzle

@Gertjan:

Yes indeed I know about the "secret agents" method Netflix and other major media networks use. That's why I am glad to use a VPN (or any means necessary) to "preserve" my internet freedom. This is one of the major reasons why internet piracy is flourishing btw and as long as doing business in this old fashioned way of geo-restricting content continues, so will the fight for internet freedom. Fighting for freedom is basically programmed in our genetic code :).

vjizzle

@CrystalFire

Thanks for taking your time and I am looking forward to the screenshots!

Gertjan

@vjizzle said in ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN:

I know about the "secret agents" method Netflix

Secret ?
If you were a member of the direction of "Universal Pictures", or "Disney" or any other big one you focus on one person : the share holder. That might you you and me, btw.
Thus, you negotiate with every "view channel", like your national TC channels, VOD channels, DVD rent and sell channels the maximum amount of money.
Their business is making money, nothing else.
I'm pretty sure, up until now, my words are not unknown so not secret to you.

It's a very close step to understand why Disney, Netflix, etc want to know who - and from where - accesses their content.
Right ?

How do they get all these IP's from ExpressVN, NrdV*N etc ? They call them and ask ? Maybe, but I guess not ... The method I mentioned above is the next-best option for them : hire a small squad that does the job for you.

We, as end users would like to access and the see content we paid for, using the methods we choose.
They, on the other side, have also their selling conditions.
As usual, the latter prevails. No big deal, you can/ should choose YOUR VPN supplier which matches YOUR needs ^^

Free tip : use the old way of choosing a product : make a list with mandatory needs and optional needs. Then you look up all the suppliers that matches your list. From what's left : take others aspect into account like colour, price and other subjective aspects.
You will never be deceived.

Btw : IMHO : "Netflix", to name just one, proposes a product that does NOT fall under the "Internet Freedom" rights, laws, and principals. Movies, series, are created, have owners, and solely exist for one reason : making money - they are not open in some kind, neither tend to be.
Be honest : they offer a product you don't really need to live or exist. It's just a thing of today, that din"t exist yesterday, and probably will not exist in the future.

@vjizzle said in ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN:

programmed in our genetic code

I hope so ;)
You agree with me that this "code" is hackable, and that the word 'money' has something to do with it.
If you find a model where you could create series like Games of Thrones and make it accessible for everybody and everywhere, I'll be your very first client - promised.
Or, do it the soft way : stop buying products from the majors and you see, they will change their model very fast.

Btw : I'm just a pfSense user - and an occasional ordinary Netflix user. Still waiting for season 9 - Walking Dead.

vjizzle

@Gertjan

Thanks for sharing your thoughts with us @Gertjan and indeed nothing new under the sun so-to-speak. But we are getting a little bit off topic here I think.

I still want to know what is different in pfSens compared to OpenWRT for OpenVPN client config and DNS leaks. Looking forward to the screenshots from @CrystalFire.

Gertjan

@CrystalFire said in ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN:

by using ExpressVPN app, it just goes through fine - opens up and content plays too, however when I try to access it via pfSense OpenVPN - it just doesn't load completely.

Even if your are using the same ~~URL~~ => IP ( !) for ExpressVPN app and the OpenVPN client, nothing will guarantee you what the outgoing == your actual WAN IP - as seen by Netflix - is.

One IP in their IP list can be listed, the other one not. It's purely random.

Maybe the App uses a privately known list with VPN server IP's, not known to the public - and the OpenVPN style of connections another list with IP per country.
Maybe the App uses another WAN style network ... etc.

About the OpenVPN client : we / they all use the same open source OpenVPN, because why re invent the wheel as a good product already exist ?
Still, the App could be a mode made protocol.
At least, my iPhone ExpressVN is based on OpenVPN - the ExpressVN servers are based on a (modified by them to enforce accounting) OpenVPN.

There is no such thing as
OpenVPN client 2.4.9 works and OpenVPN client 2.3.2 doesn't work.

Your favourite VOD service can only see : your WAN IP and port. It can not see what is behind it.

Remember : when the VPN tunnel comes up, traffic starts to flow. For this VOD traffic, actually any type of traffic, the presence of the tunnel is not detectable for the destination address. [edit : the MTU might change]
IP packets, headers, payload, everything is the same. VPN tunnel ought to be transparent for the application (OS) on both sides. What changes on the client side is : the routing table changes. But this will not be visible to the destination side.
There are no bits or bytes or other info in the traffic (headers or payload) that says "this packet was handled by OpenVPN 2.4.9" - and the server is "OpenVPN 2.4.8".

Take a look at https://serverfault.com/questions/480060/how-do-i-tell-if-all-traffic-is-going-though-the-vpn

pfSense uses : OpenVPN (server) :

[2.4.5-RELEASE][admin@pfsense.internal-wall.net]/root: openvpn --help
OpenVPN 2.4.9 amd64-portbld-freebsd11.3 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA]                                                                          [AEAD] built on May  4 2020
....

Pippin

@Gertjan said in ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN:

[edit : the MTU might change]

https://medium.com/@ValdikSS/detecting-vpn-and-its-configuration-and-proxy-users-on-the-server-side-1bcc59742413
https://github.com/ValdikSS/p0f-mtu-script
http://witch.valdikss.org.ru/

vjizzle

@Gertjan said in ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN:

At least, my iPhone ExpressVN is based on OpenVPN - the ExpressVN servers are based on a (modified by them to enforce accounting) OpenVPN.

This is exactly what I mean! The ExpressVPN app on the iphone is somehow always working with all VOD's. So the mtu detection (or VPN detection) as explained by @Pippin (great read btw) is somehow mitigated for if you use the dedicated VPN app. Off course there is always the IP-restriction but let's pretend that this is not the issue here.

I know that from 1st hand experience that the NordVPN app in iPhone is showing the exact same behaviour as the ExpressVPN app. Everything on Netflix, Disney+ or other VOD services cannot detect the VPN as long as you use the dedicated VPN app.

So how can we have this same experience from pfSense and the OpenVPN client configuration there? What combination of settings on pfSense can achieve the same level experience level as those dedicated VPN apps? Or at least come close...

Yes I understand that those VPN apps are maybe heavily modified versions of some VPN protocol and not opensource. But then again as @CrystalFire said earlier in this post, OpenWRT seems to be able to achieve that same level as the dedicated VPN apps.

CrystalFire

@vjizzle Below screenshots are of pfSense configuration for ExpressVPN. Just getting you the relevant ones, let me know if you want screenshots for any other page. Hope you are able to match it against your configuration and fix your issue.

You don't need to have the X greyed WAN config.

Configure only one OpenVPN Client here - ignore my greyed one in below screenshot.

So, hopefully this is all you need to fix the configuration at your end and get ExpressVPN up & running for your purpose.

However, like I had mentioned earlier as well, use OpenWrt if you are trying to achieve ExpressVPN app kind of experience.

Cheers

vjizzle

@CrystalFire :

Thanks for sharing your setting!

When I disable DNS Forwarding in the resolver (like you have) and select my VPN interfaces as outbound interface in DNS resolverI always have a DNS leak whenever I restart pfSense. dnsleaktest.com shows me DNS from my ISP. I have to restart unbound service manually after every reboot to make it use the VPN interface for resolving. This happens on every reboot / boot. And every time it can be fixed by manually restarting unbound. After that no dns leak until my pfSense restarts :(.

I have been playing with the outbound NAT rules but I can't seem to fix this. Does this sound familiar for anyone?

--edit typos

charles_moody

@vjizzle following this thread because of the same problems with expressvpn.

Even if it's not a technical fix you could use a ups so the pfsense box wont't go down

vjizzle

@vjizzle said in ExpressVPN - Geo-restricted website works when connected through their app over OpenVPN UDP but not through pfSense OpenVPN:

@CrystalFire :

Thanks for sharing your setting!

When I disable DNS Forwarding in the resolver (like you have) and select my VPN interfaces as outbound interface in DNS resolverI always have a DNS leak whenever I restart pfSense. dnsleaktest.com shows me DNS from my ISP. I have to restart unbound service manually after every reboot to make it use the VPN interface for resolving. This happens on every reboot / boot. And every time it can be fixed by manually restarting unbound. After that no dns leak until my pfSense restarts :(.

I have been playing with the outbound NAT rules but I can't seem to fix this. Does this sound familiar for anyone?

--edit typos

This is why I have setup DNS servers in the General Setup tab in pfsense and then assigned the VPN interface as the gateway. Then I have Forwarding enabled in DNS Resolver with SSL/TLS. In this way on every (re)boot of pfSense I can make sure that there is not DNS leak with the DNS ip of my internet provider. Off course dnsleaktest.com us showing multiple cloudflare and quad9 dns servers, but I can live with that.

BTW I am running 3 VPN client connections to ExpressVPN and using a Routing group I am able to have a "fail-over" situation because I sometimes find that my ExpressVPN client disconnects. Those VPN interfaces are the only ones selected in DNS Resolver as outgoing interface. This setup has been tested with only 1 VPN client and shows the same behaviour.

So with this configuration and by trying different ExpressVPN servers I found some of them working fine with Netflix. But I suppose just a matter of time that those servers are also backlisted :P.

The permanent solution here seems to be OpenWRT. For me that is a no go because pfSense is so much more user friendly. Plus I am using Suricata to protect internet exposed servers and with OpenWRT that is a p.i.t.a. to configure.