[SOLVED] Slow PIA VPN connection on pfsense 2.4b
-
EDIT
This was solve by going back to Pfsense 2.3.3 stable.
EDITHello
I recently bought a pfsense box and upgraded my internet connection to 100/100mbit from 30/30mbit
at first running the PIA VPN on pfsense was impossible because with the standard settings described in the PIA guide I only got around 0.5mbit download speed. Then I looked here on the pfsense forums and it seems there are a few others who have had speed issues as well.I found that I should add this:
fast-io
sndbuf 524288
rcvbuf 524288This gave me an increase to a maximum of 75/25mbit
but I'm still missing the last 25/75mbits and since I have no clue what the things I added to the client does I have no idea how to proceed.I have also tried disabling the PIA client in pfsense and testing with the PIA pc client and here it is maxing out my internet connection.
Its a homemade mini ITX pc with the following specs:
Asrock Intel J3455-ITX
2x4GB HyperX 1866MHz RAM
Pfsense 2.4bI'm running with same encryption level on both clients.
![PIA Settings 01.jpg](/public/imported_attachments/1/PIA Settings 01.jpg)
![PIA Settings 01.jpg_thumb](/public/imported_attachments/1/PIA Settings 01.jpg_thumb)
![PIA Settings 02.jpg](/public/imported_attachments/1/PIA Settings 02.jpg)
![PIA Settings 02.jpg_thumb](/public/imported_attachments/1/PIA Settings 02.jpg_thumb)
![Speedtest straight.JPG](/public/imported_attachments/1/Speedtest straight.JPG)
![Speedtest straight.JPG_thumb](/public/imported_attachments/1/Speedtest straight.JPG_thumb)
![Speedtest PCVPN.jpg](/public/imported_attachments/1/Speedtest PCVPN.jpg)
![Speedtest PCVPN.jpg_thumb](/public/imported_attachments/1/Speedtest PCVPN.jpg_thumb)
![Speedtest VPN.JPG](/public/imported_attachments/1/Speedtest VPN.JPG)
![Speedtest VPN.JPG_thumb](/public/imported_attachments/1/Speedtest VPN.JPG_thumb) -
No one? :'(
I just did a restore to a basic setup with WLAN I had of pfsense 2.4 and then updated to the latest build from the 19th of April.
Then tried going through the PIA setup guide once again, but I have the same problem as before.I have attached a screenshot of the speed I'm getting.
Please let me know if you need more information regarding other parts of my pfsense settings.
![Speedtest VPN default.JPG](/public/imported_attachments/1/Speedtest VPN default.JPG)
![Speedtest VPN default.JPG_thumb](/public/imported_attachments/1/Speedtest VPN default.JPG_thumb) -
Try these things one at a time and individually. If they don't help individually try them in combinations.
Under System / Advanced / Miscellaneous > Cryptographic Hardware : make sure you have AES-NI selected.
try using a different PIA server
try using the webconfigurator cert for your client certificate
try removing all of your custom options except "remote-cert-tls server"
try using port 1194
-
Try these things one at a time and individually. If they don't help individually try them in combinations.
Under System / Advanced / Miscellaneous > Cryptographic Hardware : make sure you have AES-NI selected.
try using a different PIA server
try using the webconfigurator cert for your client certificate
try removing all of your custom options except "remote-cert-tls server"
try using port 1194
AES-NI was selected.
Switching to port 1194 "broke" the VPN, meaning I was running on my ISP again.
There are no other certs available for me to choose other than the PIA cert i created.
Removing all custom options except "remote-cert-tls server" made the connection slow again.
The server I have selected is maxing my connection when I run it on my pc client, should I still try another server?
Another thing I've noticed is that my RTT is around 7-15 ms shouldn't it be less than 0? thought I might mention it, if it could help out.
![No VPN.JPG](/public/imported_attachments/1/No VPN.JPG)
![No VPN.JPG_thumb](/public/imported_attachments/1/No VPN.JPG_thumb)
![PIA PC VPN CLIENT.JPG](/public/imported_attachments/1/PIA PC VPN CLIENT.JPG)
![PIA PC VPN CLIENT.JPG_thumb](/public/imported_attachments/1/PIA PC VPN CLIENT.JPG_thumb)
![PFSENSE PIA VPN.JPG](/public/imported_attachments/1/PFSENSE PIA VPN.JPG)
![PFSENSE PIA VPN.JPG_thumb](/public/imported_attachments/1/PFSENSE PIA VPN.JPG_thumb)
![PFSENSE PIA VPN CUSTOMS REMOVED.JPG](/public/imported_attachments/1/PFSENSE PIA VPN CUSTOMS REMOVED.JPG)
![PFSENSE PIA VPN CUSTOMS REMOVED.JPG_thumb](/public/imported_attachments/1/PFSENSE PIA VPN CUSTOMS REMOVED.JPG_thumb)
-
No RTT can never be 0, it will always be greater.
Still try another server.
The webconfigurator certificate is on pfsense by default. If you really don't have it something is probably wrong, and you should do a clean install and restore config.
-
No RTT can never be 0, it will always be greater.
Still try another server.
The webconfigurator certificate is on pfsense by default. If you really don't have it something is probably wrong, and you should do a clean install and restore config.
oops what I ment was around 0.. I've seen tutorials on youtube where its been around 0.2 as i remember.
I found the certificate, tried it out, but didn't help. Will try a few different servers now.
-
sub 50ms RTT is pretty good for a VPN connection. sub 10ms is good for a normal WAN. Sub 1ms you will probably only ever see on your local network.
Honestly, your VPN settings look like they should work just fine. The fact that you could get higher speeds with tweaking on that server means it can give you those speeds.
I would still try a few different servers.But I think something is wrong with your box. Probably a config somewhere. That hardware is confirmed to get high VPN throughput.
Try backing up your config, and doing a clean install. Setup just the VPN and see if it works for you. If so then try restoring the config.xml
-
Ok so I have now tried 5 EU servers and one US server without any difference, they are all around 40-60/15-40mbps. (it seems to be my max speed today)
as per usual I turned off the pfsense vpn and tried without any vpn and also the pc client, with the same results as always = around 100/100mbpsAlso tried to remove:
fast-io
sndbuf 524288
rcvbuf 524288it made it worse every single time, around 5mbps down.
sub 50ms RTT is pretty good for a VPN connection. sub 10ms is good for a normal WAN. Sub 1ms you will probably only ever see on your local network.
Perfect, these numbers I'm well within of.
Honestly, your VPN settings look like they should work just fine. The fact that you could get higher speeds with tweaking on that server means it can give you those speeds.
I would still try a few different servers.But I think something is wrong with your box. Probably a config somewhere. That hardware is confirmed to get high VPN throughput.
Try backing up your config, and doing a clean install. Setup just the VPN and see if it works for you. If so then try restoring the config.xml
Would like to do that, but is it possible without a screen and keyboard? ::) I borrowed those two items when installing them last time, since my only pc is a laptop.
If not I will have to ask around ;D
-
Yeah you'll need a keyboard and screen.
You can try backing up your config, then Diagnostics / Factory defaults.
That will only fix it if it's a weird setting you've placed somewhere that's causing problems.
If somethings corrupted with the instal it won't help you, you'll need clean install for that.What other packages are you running?
-
Yeah you'll need a keyboard and screen.
You can try backing up your config, then Diagnostics / Factory defaults.
That will only fix it if it's a weird setting you've placed somewhere that's causing problems.
If somethings corrupted with the instal it won't help you, you'll need clean install for that.What other packages are you running?
I thought so much.. Will have to ask around then :)
Now that we are speaking about the possible reason being the install, I do remember that the system sometimes hangs and gives an error message while loading pfsense from a reboot. Never gave it much thought as it was the 2.4 beta and trying to run it mirrored on two USB's.
I unfortunately cannot remember what the error message was. but probably something about it couldn't find or load something.
I was running Suricata, but as it is very CPU hungry, I reverted back to a former backup before it was installed, to see if that was the problem.
-
Yeah you shouldn't be getting error messages even on 2.4 during boot.
I also run 2.4.0 BETA on USB sticks in raidz2 with a RAM disk and get no error messages.
-
Yeah you shouldn't be getting error messages even on 2.4 during boot.
I also run 2.4.0 BETA on USB sticks in raidz2 with a RAM disk and get no error messages.
Hopefully this is the reason (knock on wood) :D
So since I have 2 USB3 ports and 2 USB2 port available, how do I go about installing?
Because I've read somewhere here on the forum that USB3 ports are not a good idea.. -
I haven't heard that?
USB3.0 drives are often not recommended as install media because they tend to get hotter, but I don't know if that even matters.
I would leave the drives you'll install to in the 2.0 slots and put the one with the image in whatevers left over.
-
I haven't heard that?
USB3.0 drives are often not recommended as install media because they tend to get hotter, but I don't know if that even matters.
I would leave the drives you'll install to in the 2.0 slots and put the one with the image in whatevers left over.
Read it over on the hardware forum, but I cant find it now.
It might just be because they get hot.
Will try your suggestion(s) tomorrow, have ordered a screen and keyboard and if all fails with the usbs, I have a 256gb ssd I can donate to the "cause".
-
Do you use a RAM disk?
-
-
So during the installation of pfsense 2.4 I ran into the issue I also had the first time I installed it and that was that the installer hanged after I selected to reboot. See picture.
I waited aprox 10-15 mins for it to reboot and then forced it by unplugging the router. Is this normal, should I have waited longer?
In the next picture, this always show up during reboot. Is this normal?
![Dump devices does not exist.jpg_thumb](/public/imported_attachments/1/Dump devices does not exist.jpg_thumb)
![Dump devices does not exist.jpg](/public/imported_attachments/1/Dump devices does not exist.jpg) -
The "no suitable dump device found" error just means that you don't have swap, which is fine as long as that's the way you installed.
I'd try it again and leave it for 20-30min.
There are a few issues with reboot floating around, and there's an issue where reboot can hang for ~20min because it can't install packages but it will eventually continue the boot and then you can get packages installed once boot is complete. The 20 minute hang issue hopefully gets fixed….
So reinstall, let it start rebooting and just walk away, for a good bit of time.
For your hardware I don't think it's an issue but make sure you have latest BIOS/UEFI/firmware installed.
-
The "no suitable dump device found" error just means that you don't have swap, which is fine as long as that's the way you installed.
I did install without swap, so I can ignore that message in the future :)
I'd try it again and leave it for 20-30min.
There are a few issues with reboot floating around, and there's an issue where reboot can hang for ~20min because it can't install packages but it will eventually continue the boot and then you can get packages installed once boot is complete. The 20 minute hang issue hopefully gets fixed….
So reinstall, let it start rebooting and just walk away, for a good bit of time.
It just started its 20mins reboot countdown, so I'm crossing my fingers :)
For your hardware I don't think it's an issue but make sure you have latest BIOS/UEFI/firmware installed.
I am all ready, so shouldn't be a problem.
-
I'm now on 120 mins, no sign of it wanting to reboot..
-
So am I doing something wrong with the install?
Downloading the latest memstick image from here https://snapshots.pfsense.org/amd64/pfSense_master/installer/?C=M;O=D unpacking it, then using Win32DiskImager to make the bootable usb stick. Booting it up and following your guide https://forum.pfsense.org/index.php?action=thankyoupostlist;topic=126597.0;msg=699155 and choosing 2 disk mirror.
-
I don't think so, there have been multiple users reporting reboot issues. I've never encountered it though so I don't really know how to help except pointing you to these.
https://forum.pfsense.org/index.php?topic=128577.msg712180#msg712180
https://forum.pfsense.org/index.php?topic=126520.msg698661#msg698661
-
I don't think so, there have been multiple users reporting reboot issues. I've never encountered it though so I don't really know how to help except pointing you to these.
https://forum.pfsense.org/index.php?topic=128577.msg712180#msg712180
https://forum.pfsense.org/index.php?topic=126520.msg698661#msg698661
Thanks, its good to know I'm not the only one.
Will try to install pfsense 2.4 to my SSD instead, hopefully it will work and then when a fix has been implemented I will go back to the sticks.
-
Honestly if you have an SSD laying around you are better off using that.
In my ZFS Guide I do mention installs to USB sticks, but not because they are better. I mention it because it is a cost saving feature that might enable someone to afford it that otherwise couldn't. There are other reasons to install to USB, but generally speaking if you have an SSD definitely use the SSD.
USB drive installs need you to adjust things to make them last that you wouldn't ever have to worry about with an SSD.
The only advantage they have over SSDs is price, and how common they are (just about anyone can pull an SSD out of a drawer and install pfSense to their machine.
-
Honestly if you have an SSD laying around you are better off using that.
In my ZFS Guide I do mention installs to USB sticks, but not because they are better. I mention it because it is a cost saving feature that might enable someone to afford it that otherwise couldn't. There are other reasons to install to USB, but generally speaking if you have an SSD definitely use the SSD.
USB drive installs need you to adjust things to make them last that you wouldn't ever have to worry about with an SSD.
The only advantage they have over SSDs is price, and how common they are (just about anyone can pull an SSD out of a drawer and install pfSense to their machine.
Ja I totally understand, was just hoping to save the SSD for other projects. Its a bit overkill to have a 256gb disk in a router system IMO ;D
and as I said in my former post, I will properbly go back to the USB's if I hear news that the issue is fixed. -
Oh btw how would you configure the install with an SSD? how big of a swap size, if any?
-
I don't think the boot issue is USB specific, others have reported the issue on SSD/HDD.
Swap is normally double your RAM, I believe that's the default setting.
Defaults will work great.
-
I don't think the boot issue is USB specific, others have reported the issue on SSD/HDD.
Swap is normally double your RAM, I believe that's the default setting.
Defaults will work great.
hmm, for me the change to SSD worked, it rebooted straight away, with no issues.
In that case, I will need to add some more swap.
Will test VPN tommorrow.
-
So got OpenVPN configured this morning and with the standard settings, its slow as usual 5/10-30mbit.
With my added settings I'm hitting a max of 84mbps.
I switched to pfsense monitor for reference as it seems Speedtest.net is all over the place..
I tried to set it up as you described it in another thread. https://forum.pfsense.org/index.php?topic=128230.15Then do your best to max out your bandwidth, Steam downloads usually have great bandwidth and they have free titles (DOTA 2 is pretty big and free so it will run for long enough to see it on RRDs).
You have a pretty beefy connection so you might also stream a bunch of UHD youtube videos, I think you can search for even 5k and 8k content that will really suck down some bandwidth!Anyawys, after you max out the connection for 5-10 minutes,
go to Status / Monitoring and set it up like so:
System > Processor on one side
Traffic > WAN on the other side
1 Hour, 1 Minute, Line, On, Never
De-select everything on the graph except:
user util
nice util
system util
interrupt
inpass total
outpass totalScreenshot the graph and data summary with your mouse hovering over a point on the graph where your bandwidth is maxed out to display the stats you selected and post it up here.
That will give no bullshit real world VPN throughput:CPU usage data (assuming you are piping all of your traffic out through a VPN client as you stated).
I know that's all a very specific request, but it would be greatly appreciated!
I'm thinking its pretty good, though it sucks I know it could be a bit better.
Should I maybe give up and try running two OpenVPN clients? or is there still more I can tinker with?![Steam 2nd install SSD VPN.jpg](/public/imported_attachments/1/Steam 2nd install SSD VPN.jpg)
![Steam 2nd install SSD VPN.jpg_thumb](/public/imported_attachments/1/Steam 2nd install SSD VPN.jpg_thumb) -
I don't know if I've asked this already but what NIC are you using?
The CPU is obviously working just fine at ~17% for 86Mbps VPN throughout.
With this being a clean install it should be maxing your connection.
-
I don't know if I've asked this already but what NIC are you using?
The CPU is obviously working just fine at ~17% for 86Mbps VPN throughout.
With this being a clean install it should be maxing your connection.
I'ts running an IBM intel i340-T4 quad
Maybe its totally obvious and I just dont see it, but where do you see the 17% CPU usage?
EDIT: Found it, adding the % together ::)btw I got this magnificent reply from PIA support, which really answered all my technical questions in which I specifically told them that I could get 100/100 on my pc client and that my router was suppose to handle well beyond 100mbit…
Thanks for getting back to us.
You can expect to see at least a 10-15%* speed drop from the results you get when testing "disconnected" to our servers on our network page here: https://www.privateinternetaccess.com/pages/network/
- Typically it will be drop between 15-50% for computers and 25-75% (or more, depending on the router's capability) drop for routers.
- The higher encryption that you use, the more overhead that would be added slowing the connection. This can certainly be worsened by connecting to gateways that have additional routing latency or have a lot of traffic on them at the time.
- Our servers also have a 1 gigabit connection (for each server) shared among the customers connecting to the server. That in mind, we wouldn't normally expect you to reach higher than 50-100mbps.
We apologize for the inconvenience.
Let us know if you have anymore questions.
-
Yeah that's just a canned response.
I really have no idea why you aren't getting line speeds. You should be from what I can tell.
Maybe someone else can chime in here?
All I can siggest is playing around with the settings?
Maybe try LZ4v2, try no compression, try disabling NCP? Really idk though. I've run almost an identical setup on a J3355 and got line speeds at 150/10 no problems, no adding custom options.
-
I really have no idea why you aren't getting line speeds. You should be from what I can tell.
No worries, I'm just glad you want to try and help.
and we did fix one issue with the installer :DAll I can siggest is playing around with the settings?
Maybe try LZ4v2, try no compression, try disabling NCP? Really idk though. I've run almost an identical setup on a J3355 and got line speeds at 150/10 no problems, no adding custom options.
I tried different compressions and disabling NCP, without much difference although for the worse.
![Steam pfsense VPN SSD no compression.jpg_thumb](/public/imported_attachments/1/Steam pfsense VPN SSD no compression.jpg_thumb)
![Steam pfsense VPN SSD no compression.jpg](/public/imported_attachments/1/Steam pfsense VPN SSD no compression.jpg)
![Steam pfsense VPN SSD adaptive LZO.jpg](/public/imported_attachments/1/Steam pfsense VPN SSD adaptive LZO.jpg)
![Steam pfsense VPN SSD adaptive LZO.jpg_thumb](/public/imported_attachments/1/Steam pfsense VPN SSD adaptive LZO.jpg_thumb)
![Steam pfsense VPN SSD LZO4v2.jpg](/public/imported_attachments/1/Steam pfsense VPN SSD LZO4v2.jpg)
![Steam pfsense VPN SSD LZO4v2.jpg_thumb](/public/imported_attachments/1/Steam pfsense VPN SSD LZO4v2.jpg_thumb)
![Steam pcvpn SSD.jpg](/public/imported_attachments/1/Steam pcvpn SSD.jpg)
![Steam pcvpn SSD.jpg_thumb](/public/imported_attachments/1/Steam pcvpn SSD.jpg_thumb)
![Steam straight SSD.jpg](/public/imported_attachments/1/Steam straight SSD.jpg)
![Steam straight SSD.jpg_thumb](/public/imported_attachments/1/Steam straight SSD.jpg_thumb) -
Could you explain me how to setup two VPN's as one? or do you have a link to a guide?
Just want to try it out and see if that gets me closer to the 100 mark. :)
-
Could you explain me how to setup two VPN's as one? or do you have a link to a guide?
Just want to try it out and see if that gets me closer to the 100 mark. :)
What hardware is your pfsense box running on?
What PIA setup guide are you using, and are you connecting to the "strong crypto" gateways or the standard PIA gateways with less encryption?
Its entirely possible that your CPU can't process the encryption faster than 75mbps on the throughput. This explains why you see full line rates when running it on a PC, and slower rates when its running on pfsense.
If you're connecting to the stronger encryption gateways, the only thing you can do to improve your throughput is to start connecting to the default (lower encryption) ones.
-
What hardware is your pfsense box running on?
Asrock J3455-ITX
2x4gb Hyperx DDR3L 1866MHz
256gb SSDAccording to the synthetic benchmarks I have done, it should be able to handle up to 280mbps over VPN.
What PIA setup guide are you using, and are you connecting to the "strong crypto" gateways or the standard PIA gateways with less encryption?
If you're connecting to the stronger encryption gateways, the only thing you can do to improve your throughput is to start connecting to the default (lower encryption) ones.
Its the standard pfsense guide, with 128bit encryption found on their website here: https://www.privateinternetaccess.com/pages/client-support/pfsense
It gives me roughly 5mbps download.
I have then added:
fast-io sndbuf 524288 rcvbuf 524288
which improves that figure to 86mbps
-
J3455 can definitely do a lot more then ~80Mbps.
Here are some instructions for gateway groups with VPN:
https://forum.pfsense.org/index.php?topic=115992.msg652957#msg652957 -
So after a lot of trial and error, it seems I have gotten two vpn clients up and running in a gateway group.
The speeds are finally at maximum! But the latency seems to have gone up and so webpages seems to be loading slower than before. Is this a trade off with this configuration?
https://ipleak.net/ seems fast and reports my ip and dns servers to be the correct for my VPN choice, with no exceptions.
But I'm a little worried that I'm maybe running traffic around the VPN with these speed and the low CPU usage..?
I made a lot of back and forth settings changes and think I might be better of restoring from a backup and trying again tomorrow.
EDIT
It seems I have the same problems with everything showing up offline as pigbait on page 2.
Also bretthoward sums up pretty much what I'm experiencing on the same page as well.![Steam VPNGG SSD.jpg](/public/imported_attachments/1/Steam VPNGG SSD.jpg)
![Steam VPNGG SSD.jpg_thumb](/public/imported_attachments/1/Steam VPNGG SSD.jpg_thumb) -
So after a lot of trial and error, it seems I have gotten two vpn clients up and running in a gateway group.
The speeds are finally at maximum! But the latency seems to have gone up and so webpages seems to be loading slower than before. Is this a trade off with this configuration?
https://ipleak.net/ seems fast and reports my ip and dns servers to be the correct for my VPN choice, with no exceptions.
But I'm a little worried that I'm maybe running traffic around the VPN with these speed and the low CPU usage..?
I made a lot of back and forth settings changes and think I might be better of restoring from a backup and trying again tomorrow.
EDIT
It seems I have the same problems with everything showing up offline as pigbait on page 2.
Also bretthoward sums up pretty much what I'm experiencing on the same page as well.you can verify if traffic isnt being passed thorugh the VPN setup by going to diagnostic -> packet capture -> wan and leave the default options. Launch the packet cap, then do a bunch of broswing/speed tests. I'd recommend keeping the capture UNDER 5 SECONDS, otherwise youre going to be reading through a LARGE packet cap log.
Once you think youve generated enough traffic, stop the packet cap and read through the connections. If you see anything exiting your wan interface and headed to hosts other than your VPN provider, you've got a routing leak.
Its worth mentioning since I'm unaware of how your setup is configured, that a multi wan (in this case its multi WAN, because youve got multiple VPN gateways traffic can exit) setup can cause havoc on session tracking for websites if youre set to round robin. You'll want traffic headed to websites to always leave through the same gateway, therefor its always returning via the same route.
Since your procs can handle line speeds, its likely your speed issue is due to the gateway youre heading to. PIA aggregates i think 10 users per IP (which their servers have a 1gbps connection, so 100mbps per user in a perfect world), so you might just be on a node that has heavy traffic.
-
PIA doesn't have a 100Mbps per user cap.
It's common to get much more than that. The highest I think I've seen reported on here was in the 600Mbps range on a single instance.
Using gateway groups as is works just fine, you don't need to do anything funky with your website traffic or session tracking at all. You're unnecessarily overcomplicating it.
Looking for anything going to !PIA_IP on pcap will only work if you are routing all of your traffic to the VPN, most people do not do this because many services don't work over VPN.