Very slow database access when connected via VPN
-
Hello, we have PFSense 2.7.0-RELEASE (amd64) on bare metal install and OpenVPN.
Our customer uses a local application that uses a database on a server that is also local.
When the customer's PC is at the company, we don't use VPN and the access is excellent.
Eventually, that customer takes his PC home-office, and then, on the VPN, database access is very, very slow. Sometimes the local application becomes unresponsive and you have to forcefully close it. The customer's internet connection is via gigalan ethernet cable with speeds above 400 Mbps.
We use other programs with the VPN using RemoteApp and we don't have this problem. But the problematic app support is poor and we couldn't turn it into RemoteApp either.
Also, we couldn't find VPN connection parameters that could cause this slowdown.
Has anyone been through this situation? can you help me? -
What is in front of the WAN interface of pfSense ?
If it's an ISP-like router : you can test things locally and fact-check that "OpenVPN" is the issue.Replace the straight WAN cable netween pfSense and the ISP router with a switch. Hook up your ISP device, pfSense and your Customer PC to this switch
Fire up the OpenVPN client and use the WAN IP of pfSense as a destination.
Now you will see the real throughput of your switch (a Gigabit probably), the pfSense NIC (1 GBit also ?) so you will see what the Customer PC OpenVPN app on one side, and the OpenVPN server on pfSense really can handle.Btw : For me, for years, my ISP gave me "20 Mbits/sec down and a mere 2 Mbis/sec up", so that was my bottleneck.
Other surprises are possible, see for example openvpn smb not working although I presume you connection is more TCP based.
pfSense has more then "OpenVPN". You could also use IPSEC, Tailscale, etc. -
@Gertjan Thanks for your attention.
Simplifying the customer's enterprise network topology:
ISP router (600/300 Mbps link)
> PF Sense (as Firewall and DHCP provider)
>Gigalan switch
> File Server
> Application Server (includes the aforementioned database application)
>local computersWho is physically in the company does not use VPN, but is always "below" the network formed by PF Sense.
PF Sense WAN and LAN adapters are gigalan
OpenVPN is configured as UDP over IPv4 (IPv4 is still common here in Brazil). When we set up the VPN Server, it was mentioned that TCP connections are slower for our purpose.
From what I understand, the suggested test is to reverse the PF Sense Server with the switch in the topology. And that?
In case it would be
ISP -> SWITCH -> PF SENSE -> Local PCsI don't see this configuration with good eyes, even for testing.
-
Are multiple clients over the VPN seeing the same poor performance when accessing the database?
Have you checked if perhaps the MTU and/or MSS would need to be adjusted? Have you played with those settings?
Is the client connected over PPPoE at their home? -
@michmoor Yes, we've adjusted the MTU a few times.
All customers who access this specific application complain about the slowness.
What really stands out is that the RemoteApp applications are OK, as well as File Server access, for example.
The problem program is from a third-party company that refuses to assume that it is their application that is bad. We can't refute them without making sure our company's infrastructure is OK -
@Suporte-Speedtech What kind of database? Something like Access would read the database file over the VPN. It would also be prone to corruption if the link was broken. Client/server SQL should be better.
300 Mbps upload from the server is 1/3 gigabit speed but shouldn't be terribly noticable. SMB though can be slow over VPN, and in particular this spring there's been ongoing reports of slow VPN on Windows 11 but IIRC that was with Windows' IPSec.
RemoteApp is like Remote Desktop in that it runs the software in the office so there is no latency to the files.
-
@SteveITS Access DB.
It's not complicated for us to change the database files, but at the moment we don't have authorization to do so.I'm checking every PF Sense rule that could interfere with the communication, but so far nothing that makes sense.
I'm going to "clone" their installation to a virtual machine and use it as a lab. If I discover something, I'll post it here.
-
The fact that this only occurs over VPN is interesting.
Something similar happened a long time ago where an application was sending out packets with the DF bit set but also those packets were sent exceeding the VPN MTU. So clients were stuck with poor performance.
The solution was to have the devs disable the DF bit on all outgoing packets. Problem solved.
So i would advise to do a pcap first off the NIC of the database. See what is being sent. Go from there. -
@michmoor what does iperf3 show?
-
Have a look on the Latency, and how the App works, go for a pcap.
If the start use 10k queries to the DB, on lan site no problem but on VPN site with 20-30ms it takes Min to start.