slow downloads or packet loss on WAN
-
pfSense 2.4.5 is running in Hyper-V, IDS is disabled.
-
I installed a new pfSense on VMware WS, problem is the same.
-
I did more testing (ISO downloads, not speedtests; 20 MB/s max) and found out that,
if I am not connected to any VPN (NordVPN), I only get half download speed.
if I am connected via a VPN-Client on Windows, I only get half download speed.
if I am connected via the VPN-Client in pfSense I get fullspeed.
if I am directly connected to the Modem and don't use pfSense at all and also not connected to any VPN, I get fullspeed.So something is seriously flawed on WAN, maybe in all the senses (and or with virtualization).
Its very sad.
-
@Bob-Dig said in Packet loss with VPN Software running:
I did more testing (ISO downloads, not speedtests; 20 MB/s max) and found out that,
if I am not connected to any VPN (NordVPN), I only get half download speed.
if I am connected via a VPN-Client on Windows, I only get half download speed.
if I am connected via the VPN-Client in pfSense I get fullspeed.
if I am directly connected to the Modem and don't use pfSense at all and also not connected to any VPN, I get fullspeed.So something is seriously flawed on WAN, maybe in all the senses (and or with virtualization).
Its very sad.
All problems still persist with 2.4.5p1.
Slow Download speed, tested with downloading ISOs, or massive packet loss shown under Gateways.
No problems without pfSense and using the modem directly. -
Ok, it seems related to how I made my dns request. For some time now, I defined in the resolver as Outgoing Network Interfaces only the vpn-clients. This is probably why I only got full speed over the vpn connection in pfsense, because DNS is used for optimal peering these days with CDNs.
It also made sense that I got fullspeed, when I was directly connected to the modem, because then there was no dns connections trough any vpn clients, only the ISP DNS.
So DNS-requests and downloads should go through the same interface for optimal performance. Lets say: they should be symmetric.Only the packetloss shown under gateways is not explained by this.
Edit: Explanation wrong, problem still persist, with pfsense slow, without faster...
-
@Bob-Dig said in slow downloads or packet loss on WAN:
Ok, it seems related to how I made my dns request. For some time now, I defined in the resolver as Outgoing Network Interfaces only the vpn-clients. This is probably why I only got full speed over the vpn connection in pfsense, because DNS is used for optimal peering these days with CDNs.
It also made sense that I got fullspeed, when I was directly connected to the modem, because then there was no dns connections trough any vpn clients, only the ISP DNS.
So DNS-requests and downloads should go through the same interface for optimal performance. Lets say: they should be symmetric.Only the packetloss shown under gateways is not explained by this.
Edit: Explanation wrong, problem still persist, with pfsense slow, without faster...
Do you have the option for "Don't Pull Routes" checked as part of the VPN setup? If not, you need to check that box.
With that box unchecked, when you configured your VPN client it would have pulled routes from your VPN provider and that will likely change the default routing on your firewall. The end result is that all traffic, even unencrypted traffic with your VPN "off", is still routed to your VPN provider's network. I'm betting that is your issue. Your VPN provider likely treats incoming traffic differently when it is not encrypted; perhaps routing it to the Internet over a more congested circuit. Pretty much every VPN provider's setup instructions skip that little vital point because they probably want to see "all" of your traffic. That gives them more of your browsing data to monetize...
.Once you check that box, you will need to properly configure policy routing on your firewall by creating the appropriate rules. Google "policy routing on pfSense" to find a plethora of tutorials on this.
-
@bmeeks said in slow downloads or packet loss on WAN:
Do you have the option for "Don't Pull Routes" checked as part of the VPN setup? If not, you need to check that box.
Hey @bmeeks, great to have you here.
That box is checked on all the clients and I do policy routing already. It is driving me nuts.
First I only saw that packet loss, much later I found out, that my download speed is less with pfSense then with a direct connection. And for that to find out, you have to try it first.
So yesterday I pulled out my old Asus RT-AC56U, flashed johns Merlin fork on it and saw a nice realworld download speed.
But I had used split DNS a lot and the rest of my network still was in the "pfsense mode"... so I reactivated the pfsense-vm behind my Asus, made double-NAT and used the pfSense as the DNS-Server for my hole network.
But then I saw somewhat slower dl-speed again. So I thought, ok, must be DNS. Did a rebuild of my old setup with different DNS only to find out, that all the problems still there.
-
You need to slow down with all the changes and rebuilds and instead think through a logical methodical troubleshooting process.
First up is you mentioned pfSense is on a VM while the Asus was bare-iron. Comparing a VM with bare iron does not always tell you much because now you have the unknowns of the hypervisor's operating system, its virtual NIC drivers and the actual hardware the hypervisor is running on all thrown into the mix. Any single one of those, or a combination of several, could be impacting throughput.
How are you testing with the Asus box? Are you using a separate machine such as a bare-iron desktop or a laptop for the speed tests? If so, that is taking the hypervisor's network performance out of the mix, so even though pfSense is not in the loop neither is the hypervisor.
Do you have a spare bare-iron machine you can install pfSense on and test with that? That will show that pfSense is fully capable.
If not, then you need to first determine if your hypervisor is at fault. Shutdown the pfSense VM and construct a VM desktop client on the hypervisor ( maybe a Windows PC or even Linux). Do speed tests to/from that virtual client through your Asus router. Do you get full speed then? If not, then you know the hypervisor environment is where the problem is and you can concentrate your efforts there. What hypervisor are you running?
-
@bmeeks I use Hyer-V, but had also tested with VMware WS. Problem is, I don't even remember what the problem was I was looking for, dl-speed or packetloss but it also was flawed. And I don't have a bare metal machine for pfSense.
And problem with the speed test by downloading ISOs, they were constant if testes directly after each other, but with hours between, it could also be the other side...But I feel like giving up, really am tired of testing and the asus is my only access point (or router), so switching modes just for testing is also cumbersome.
-
@Bob-Dig said in slow downloads or packet loss on WAN:
@bmeeks I use Hyer-V, but had also tested with VMware WS. Problem is, I don't even remember what the problem was I was looking for, dl-speed or packetloss but it also was flawed. And I don't have a bare metal machine for pfSense.
And problem with the speed test by downloading ISOs, they were constant if testes directly after each other, but with hours between, it could also be the other side...But I feel like giving up, really am tired of testing and the asus is my only access point (or router), so switching modes just for testing is also cumbersome.
Hyper-V is not exactly the most well regarded hypervisor environment. ESXi is the big dog in hypervisors, and for a good reason ...
.Now with that said, Hyper-V can work, but you have to realize that virtual machines must use virtual network drivers for the guest VMs to interact with. Those virtual network drivers behave differently with various guest operating systems. Obviously Microsoft is going to optimize their virtual network drivers to work best with Windows-based guest operating systems. Optimizing for FreeBSD (or even Linux), is not so high on their list. It's likely a case of "it works and doesn't crash, so let's call it good!".
To narrow down your problem with dropped packets and slow speeds (by the way, those two symptoms are intertwined), you will need to first make sure your virtual machine environment is able to achieve the required performance. That means testing to and from a virtual machine client running on the hypervisor.
-
@bmeeks Yes I did this and speed is slow there too. But what blows my mind, tested this again same seconds ago, using "WAN" in pfSense, slow speed. Using OVPN-Client in pfSense , high speed (NordVPN). Also connecting directly to modem, high speed. How can this be, virtualization can't be the problem if it works top with the OVPN-Client...
I have a tv-card with drivers only for Windows, that's why I "have to" use Windows as host.
-
@Bob-Dig said in slow downloads or packet loss on WAN:
@bmeeks Yes I did this and speed is slow there too. But what blows my mind, tested this again same seconds ago, using "WAN" in pfSense, slow speed. Using OVPN-Client in pfSense , high speed (NordVPN). Also connecting directly to modem, high speed. How can this be, virtualization can't be the problem if it works top with the OVPN-Client...
i have a tv-card with windows only drivers, that's why I "have to" use Windows as the host.
If it works at full speed with the VPN enabled and at reduced speed with the VPN "off", then it is a routing problem. With the VPN "off", pfSense has less work to do since there is no encryption. Thus the only possible variable in my view is your routing. Something changes with your packet routing on the Internet when you enable versus disable the VPN cilent. Concentrate your search there. Begin by doing a simple traceroute between your box and speed test server (or ISO download site you are using). See what the differences are in the route with VPN enabled versus disabled.
-
@bmeeks Does tracert (windows) use ports?
Anyway, I did two test with pfSense and two with Windows to an AWS IP... I can't see nothing.
