OpenVPN on pfSense within the ISP's router
-
Hi All,
I am new to pfSense in general. I have a lot of time searching but cannot seem to find a solution. I am starting to think that my overall approach may be flawed. Any advice is appreciated.
I have a windows server running a vpn client (PIA). I have a 1 Gb/s connection to my ISP. I discovered that I was really only getting 200mb/s. I assumed this was due to traffic at PIA. I recently found out that its a performance issue with the windows version of openvpn or at very least my server. So I built a pfsense box to run the open VPN client. I can get 900 Mb/s over VPN within my network and I have seen 800 Mb/s using speedtest out to the real world.
I wanted to run my server into pfsense and then into my ISPs router. (ultimately I want to replace my ISPs router with pfsense but there are further issues that make this hard so that will have to wait for another day)
I have tried running pfSense as a router with an openVPN client to PIA attached to a LAN port on the ISPs router. This works great but the other devices on the WAN side of pfSense couldn't access the server very well despite very relaxed firewall rules and port forwarding I added. Specifically my TV could not see the Serviio (DLNA) server on the windows box. I tried PIMD and IGMP Proxy but neither would work for me.
So I tried putting all the pfSense interfaces into a bridge and turn off DHCP and NAT so I got the client IPs from the ISP's router. This worked great and the TV could now see the serviio server. openVPN looked to be sending the traffic down the tunnel but PIA's website was still showing it could see my real IP and not my VPN's IP.
I found several posts talking about this as a routing issue with the response route:
- https://forum.netgate.com/topic/25168/openvpn-return-packets
- https://forum.netgate.com/topic/52146/pfsense-as-a-passive-firewall-and-openvpn-end-point
I could not find anything directly talking about how to fix this. I don't mind keeping trying but I would like to make sure that I am at least going in the right direction. All I really want is to encrypt the traffic out of the one server but still have the rest of the local device be able to access it. I am not sure if I can put everything behind the pfsense server without majorly re-cabling the whole house. Does any one have any advice?
Thank you in advance!
P.s. I am loving messing around with pfsense and have learned a lot from this forum
-
Hi,
To clarify : you want to use a VPN > OpenVPN > Clients you created to access a paid VPN service on the intrenet so a part or all of the outgoing traffic goes through that VPN service ? Or only the traffic of your server ?
-
Hi Gertjan,
Yes, I currently have pfSense set up using VPN > OpenVPN > Clients pointing to a paid VPN provider. I'd love to have all the outbound traffic on my network going through vpn but there are just too many issues. I'd be happy if just the traffic from my server to the internet went through the VPN.
I am starting to think I am trying to use pfSense incorrectly as a whole and need to rethink my approach.
Thanks,
David -
@Lumpsalot said in OpenVPN on pfSense within the ISP's router:
I'd be happy if just the traffic from my server to the internet went through the VPN.
The quick and easy solution : because you only want one device to use your VPN, install their "app", have it connect at boot, and you're done.
Next best is setting up pfSense to have it route all traffic over the VPN, which runs over your WAN.
I advise you to take a look at the official videos (Youtube > Netgate) about the subject.
For example : VPN as a WAN : https://www.youtube.com/watch?v=lp3mtR4j3Lw
Take note : every VPN supplier is/can use it's own specialities and exceptions and etc ...
VPN suppliers are not all equal : there is the price .... and a whole lot of other factors that should be considered before choosing one.Most VPN suppliers also have support/manual/forum/FAQ that mention a step by step how to set it up on pfSense.
When you have that working, you can do one more final (more complicated ?! ^^) step : have only one device using the VPN connection and not the other devices on your LAN. -
Thanks Gertjan,
I previously (currently?) had it set up using the VPN's client app. The performance on windows is pretty poor. basically this is what started this whole thing. My friend had the same suggestion as you had but he said I should scrap the windows box in favour of linux since there doesn't seem to be a performance issue of openvpn on linux
I guess the solution would be to put all my local traffic behind the pfsense box. I was trying to avoid this since the TV box provided by the ISP may be tricky to get working from behind another firewall.
There are some great threads about how to replace my ISP's router (bell home hub 3000) with pfsense but I have the wrong type of GPON sfp module provided by the ISP. Apparently, to get around this I would have to solder a mod on the network card and dont think I want to go that far.
I may just leave my setup as it is currently since it is currently "working". I just wish I could get the full performance out of the bandwidth I have.
Thanks,
David -
@Lumpsalot said in OpenVPN on pfSense within the ISP's router:
windows is pretty poor.
What is poor ?
Pulling hundreds of MBytes/sec through Soho devices is quiet possible, true.
But a mere 100 Megabit/sec is very feasible for any low bud PC, even it it is running Windows whatever version.Keep in mind that OpenVPN and the entire SSL libs and programs for Linux and Windows devices are all "the same" machine code. OpenVPN, by itself, would use any AES-NI CPU capabilities available.
Do not forget that throughput is dictated by the VPN supplier. They can say they can handle "xx Mbytes/sec" for you, but there are factors (like how many clients are on there servers ?).Your pfSense device is AES-NI capable ?
It would help - but not "triple" your throughput.@Lumpsalot said in OpenVPN on pfSense within the ISP's router:
since the TV box provided by the ISP may be tricky to get
Yeah, the TV Box (from your ISP ?) wouldn't work well if at all if it passes through a VPN.
-
Hi Gertjan,
There were several posts under the PIA (VPN) forum around "why am I not getting GB/s VPN under windows" (not an exact name, I am not able to find them now). The conclusion was that there was a limitation for the TAP driver used by openvpn under windows.
my windows server has an i5-4670k (running stock but with AES-ni). When I first found out that my VPN provider was not the bottleneck, I built a pfsense image using the same hardware (different hard drive). The hard part was that my 10g card was not supported out of the box with freebsd but I was able to find a driver someone had compiled for the same version psfense uses. I did a test using the speedtest.net app on windows and got around 200Mb/s. I then immediately switched to the pfsense boot drive I made (with vpn confirmed as running) and ran the speedtest through a client pc attached to pfsense and got 800Mb/s (same hardware different OS). I ran these back and forth like this at different times of day (I get the best speeds at 6:00am my time). I get similar speeds (200ish) with the other PCs on my network so I doubt its just the build on my server
I was getting constant results but the actual numbers I saw varied wildly. The best i got was 250 on windows and the worst i got on pfsense was 600. Because it was hard to get consistent results testing against the real world. I then set up a test setup with a new machine (i3-7100 also supporting AES-ni) as my primary pfsense "vpn client" server.
here was my 4 box test setup:
PC1 --> pfsense vpn client --> pfsense vpn server --> PC2
using iperf3 from pc1 to pc2 via the pfsense vpn tunnel, I was able to get 909 Mb/s as a max speed. without vpn enabled I was getting 945 ish. so I was happy that pfsense on my hardware could do the speeds I wanted (as an aside, I am not really sure why i am not getting full 945Mb/s as the network cards can do full speed and the CPUs did not look to be doing more then 20% load. I guess there is a bottleneck elsewhere?)
to make a long story short, I am convinced the issue is with the windows implementation of something in the stack as pfsense and the hardware don't seem to be an issue. I think my next step is to try and duplicate the functionality of my windows server under linux (or maybe freebsd?) and then test the performance that way. the issue is I have no knowledge of linux and will need to learn that from scratch too.
about the TV box. the issue there is I have 1 ethernet cable running from the TV room to my router. both the Smart TV and the ISP's TV box run on the same physical cable (via a switch behind the TV). the TV box needs to be outside the VPN and the smart TV needs to be on the same subnet as the Serviio (DNLA) box or discovery doesn't work any more. I am thinking of running a new cable for the TV. I am asking myself if I can be bothered?
-
@Lumpsalot said in OpenVPN on pfSense within the ISP's router:
I am not really sure why i am not getting full 945Mb/s
For one just the actual overhead of the tunnel.. How is this even a question to be honest? Are you thinking you should see 920 vs your 909 or something? You have added overhead, there is no possible way you would be able to see 945 its always going to be X - overhead when you do something like a tunnel.
-
@johnpoz said in OpenVPN on pfSense within the ISP's router:
@Lumpsalot said in OpenVPN on pfSense within the ISP's router:
I am not really sure why i am not getting full 945Mb/s
For one just the actual overhead of the tunnel.. How is this even a question to be honest? Are you thinking you should see 920 vs your 909 or something? You have added overhead, there is no possible way you would be able to see 945 its always going to be X - overhead when you do something like a tunnel.
Hi Jonpoz,
I would expect the overhead to show up in terms of latency. ie. if it takes an additional 50 ms to encrypt the package, then the package would be delayed by 50ms as long as the RATE the encryption was capable of was still happening at the same or higher bits / second rate.
Ie. if we can get 95 mb/s on a 100mb lan and my pfsense server can handle 900mb/s I would expect the over all throughput to be at the lesser of the two between the rate of the network and the rate the encryption (plus some latency). Hence, I would expect the throughput to be 95 mb/s limited by the network as there would be plenty of encryption power to spare.
In terms of the rate my pfsense is encrypting at, there appears to be more clock cycles un-used, the network has additional capacity. maybe the memory rate is saturated? maybe the bus? maybe there is logic in the openvpn implementation?
somewhere in the determination of the rate of the encryption, there is a rate determining step. is this step something we have the ability to effect? (over clocking, faster memory, bios, config etc...) or is it innate to the technology stack we are implemented on?
I don't have enough 10g cards to test this out but I would expect the same throughput if I ran the whole test at 10g. honestly I really don't know. Let me know if I am looking at this wrong. I have 2 10g cards right now and another 2 on the way. let me know if you can think of a way to test this out using less then 6 cards. (pc1 to server1 = 2 cards, server 1 to server 2 = 2 more. server 2 to pc2 2 more cards for a total of 6).
-
Doesn't work that way... The overhead is MORE data!!! Over a specific bandwidth and latency. Your checking your payload speed..
The overall speed of what the nic and cpu can process has not changed..
Think of it this way... If the airport says your bag can weigh 50lbs.. And your bag weighs 1 pound you can bring 49 lbs of stuff inside the bag... The total weight is 50 lbs.. But if your bag weighs 2 lbs, you can only bring 48 pounds of stuff..
Iperf is measuring the stuff.. When you use a heavier bag, ie the tunnel then your stuff is less..
-
thanks Johnpoz!
I didn't understand that. The only issue now is that I thought I was maxing out my capacity based on my hardware. I will need to see if I can scare up some more 10g cards to see where it caps out. I wanted to upgrade my Internet to 1.5gb/s but didn't pursuit it since i though 900 was the best I could do.
The annoying part is the ISP's router only has 1gb ports (hence wanting to replace it with pfsense).
we may be a little off topic now, can any one suggest a path I should pursue?
- bridge all the port on pfsense and see if I can figure out how to get the return traffic from the VPN to route properly?
- put only the server behind the pfsense firewall and see if I can get the DLNA discovery on the Smart TV to work through the firewall (pimd?)
or 3) redo the entire network behind the pfsense box and fix the cabling
or 4) something else?Thanks
David -
@Lumpsalot said in OpenVPN on pfSense within the ISP's router:
bridge all the port
That would Never be something I would ever suggest - what do you hope to happen by bridging?
3 seems like the best option to me - but maybe I missed something about what exactly your trying to do..
-
HA! I have seen that people really hate the idea of using pfsense as a switch. I think the shortest way of describing what I am trying to do is to move openVPN off my windows server and have it running somewhere between the server and my ISP's router.
Ultimately I'd like to replace the existing router with pfsense but its a little too much for me right now. for the mean time I think you are right and 3) is the best option, I would have to run some additional cable. I was hoping there was a work around
thanks
David -
Using a "tap" VPN to connect to a VPN supplier ? Strange.
I can understand the usage of tap when you call into your network from the outside. It should enable remote users to "see" LAN devices "as the Windows Network Explorer " does.
But that usage is based on a heavily fckd up notion of using networked devices. Microsoft learned us to "see" (enumerate) devices on our LAN in a GUI way. using ancient protocols like NETBui etc. That was ok in that time, but these protocols are not - or very hard - rotatable.
LAN based, limited, protocols have / had a huge advantage : no need to make users aware of firewalls, security issues, etc.
But you are using tap to go connect to a VPN supplier ... I admit not understanding that.On the other hand, with these typical ISP-build devices like TV decoders (nad ISP phones) that only work behind a ISP router/modem then you should put the decoder behind the ISP router, using your 'original' WAN so it can do it thing. The other device hooked up to your your ISP router/modem would be pfSense, with a VPN-out over WAN so all your pfSense LAN's devices are behind the VPN WAN IP.
Some setup is needs to make communication happening between your ISP's TV decoder and the LAN network.Also : keep in mind that VPN was meant to access remote networks for 'administration' purposes, typically a company need. It doesn't emulate well the classic "Windows look and feel" behaviour of a local LAN network. VPN can work just fine @home for classic tunnelled "Internet access" but covering all the needs like using your XBox while flying above the Atlantic is still "close to rocket science".
Btw : I saw a this video several days ago - it doesn't help you with your VPN questions, but would help to integrate a max. True : The guy didn't explain if his TV decoder (modem router ?) was still working / pfSense should be setup so it emulates the ISP router, that had the fibre connection initially.
-
cool thanks! I am not able to make any changes to my network right now since we are working from home (uggg all the schools are closed likely for a while too). If I break the network I will be in big trouble. I will let you know how it goes probably this week end. fingers crossed
-
I watched the video you linked. this is pretty much exactly what I ultimately want to do. I have 2 of the cards he used in the first attempt on the way. and my GPON module is the same as the one he has (nokia). apparently, there is a pin on the module that needs to be held to ground. They recommend soldering the test pad for "pin 4" (i think) on the back of the card to ground. I suspect if he did this it would have worked for him.
any way pretty cool
