Strange problem with OVPN speed - almost there - please help

[mumin50]

Hallo All..

I received my OVH dedicated server… it gives me 1000/1000Mbs connection
I use 50/50Mbs fiber connection.

Booth sides using ESXi and Pfsense 2.1.3

I set up an OVPN TAP tunnel between on port 1199 UTP
So it looks like

10.0.0.0/24 <50/50><ovpn 1199="" tunnel="" utp=""><1000/1000>10.0.0.0/24

everything works ok but when i upload something (SMB) to my server i get ~6-10Mbs and when i download something i max out on 50Mbs

I tried few things already
1)IP Fastforwarding 1
2)Enable TCP Inflight mode 0
3)added mssfix 1400
4)tried diferent ports (1194-1200, 443, 1500)
5)tried on TCP
6)tested both connections with speedtest.net and both seams to be ok
7) enough CPU power (4 cores 2GB ram and not more than 5 % usage when uploading)

Any clue what i do wrong???

Marcin</ovpn>

[NetBandit]

I'm having problems with OpenVPN as well.

Can you post a portion of your OpenVPN log file during the time you are experiencing the issue?

pfSense interface:  Status | System Logs | OpenVPN

Let's see if there is anything there to point us in the right direction.
-nb

[mumin50]

ok i found some reasons that can be problem  8)
My speeds are now 25/25Mbps on 50/50mb/s connection

logs look ok :
May 20 11:33:23 openvpn[4182]: Peer Connection Initiated with [AF_INET]yyy.yyy.yyy.yyy:58313
May 20 11:33:22 openvpn[4182]: TCPv4_SERVER link remote: [AF_INET]yyy.yyy.yyy.yyy:58313
May 20 11:33:22 openvpn[4182]: TCPv4_SERVER link local (bound): [AF_INET]xxx.xxx.xxx.xxx:1199
May 20 11:33:22 openvpn[4182]: TCP connection established with [AF_INET]yyy.yyy.yyy.yyy:58313
May 20 11:33:17 openvpn[4182]: Listening for incoming TCP connection on [AF_INET]xxx.xxx.xxx.xxx:1199
May 20 11:33:17 openvpn[3549]: /usr/local/sbin/ovpn-linkup ovpns1 1500 1595 init
May 20 11:33:17 openvpn[3549]: TUN/TAP device /dev/tap1 opened
May 20 11:33:17 openvpn[3549]: TUN/TAP device ovpns1 exists previously, keep at program end
May 20 11:33:17 openvpn[3549]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
May 20 11:33:17 openvpn[3549]: OpenVPN 2.3.2 amd64-portbld-freebsd8.3 [SSL (OpenSSL)] [LZO] [eurephia] [MH] [IPv6] built on Mar 27 2014
May 20 11:33:17 openvpn[63762]: SIGTERM[hard,] received, process exiting
May 20 11:33:17 openvpn[63762]: /usr/local/sbin/ovpn-linkdown ovpns1 1500 1595 init
May 20 11:33:17 openvpn[63762]: event_wait : Interrupted system call (code=4)

But first thing first what i did till now:

1. OVH is limiting UTP -  F@#!  >:( to 20Mbps , so i was needing to switch to TCP
2. Turned off LZO compresion
3. Enabled IP Fastforwarding 1  (gived 25% speed)
4. Enabled TCP Inflight mode 0 (gived extra speed to ~18Mbps)
5. Added  "mssfix" to adv config of ovpn (this did a trick now 25/25 Mbps)

now i see one extra problem and when i get this done i will max out  ;)

SO the problem is :
When i started uploading a file my WAN is transfering 50Mbps where my bridged interface is only 25Mbps… (look attachment )
SO i did pocket capture and looks ok
**SO IMO there is a problem with fragmentation…
Packets are fragmented in 2 parts and then encrypted.

Any help please  :'(**

edit:
I found adv option "tcp-nodelay" for letancy but i get
May 20 11:48:38 openvpn[35938]: Exiting due to fatal error
May 20 11:48:38 openvpn[35938]: Assertion failed at helper.c:533

[NetBandit]

You have the same problem as the rest of us:

https://forum.pfsense.org/index.php?topic=75989.0
https://forum.pfsense.org/index.php?topic=76735.0
https://forum.pfsense.org/index.php?topic=77169.0

This is what my bandwidth graph looks:

Get the same message in the log.  OpenVPN restarts itself, and it repeats all over again.

So far there hasn't been a bug submitted for this issue, but I'm certain that this is the issue, and I'd consider it critical.

-nb

[mumin50]

NetBandit:

Yes and No

I think that my problem lies more with OVPN MTU and fragmenting pockets that are going via tunnel.
Because it is maxing on WAn to 50Mbps what is ok, but lan side sends 25Mbps
So it looks like 1 package is fragmented and sent inside 2 packages. Because of this instead sending 1 package in 1 package via tunel it sends it in 2 doubling usage.
But i miss knowledge (for now :) ) how to do…
Correct me if wrong...

i also worked a little workaround about TCP_NODELAY statement for TCP
I found that tcp-nodelay option is just connection of two commands (https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage)

socket-flags TCP_NODELAY
push "socket-flags TCP_NODELAY"

after entering those in adv setting it seams to work , but i don't see much latency decrease

Correct me if wrong...

[NetBandit]

@mumin50:

Because it is maxing on WAn to 50Mbps what is ok, but lan side sends 25Mbps

Are you using VLANs by chance?

-nb

[mumin50]

Yes I do use vlans

Wan side is on vlan

[NetBandit]

There is a bug where the traffic on VLAN links is reported as twice as much:
https://redmine.pfsense.org/issues/3314

Separately, the WAN links where you are using your OpenVPN across may increase in latency as traffic increases.
This causes the gateway monitoring agent to react adversely.

I've had some success in increasing the latency monitoring thresholds in those links:
System | Routing
Edit the respective gateway, press 'Advanced'
Change the latency thresholds (I used 750ms/1000ms)
This was definitely helpful

However, there may be a bug in play:
@cmb:

If you don't have any gateway alarms, there is a potential second cause I just fixed. Edit /etc/rc.newwanip and find where it has curnwanip in there. Replace that with curwanip (just remove the n), save, might want to reboot afterwards just to be 100% sure nothing is using the old code. Be careful when editing any code like that. That made things think your WAN IP had changed in cases where it hadn't, so it did things like restart VPNs where it was unnecessary.

I'm going to revert the latency settings to default and test the /etc/rc.newwanip fix tonight.

See this thread for more info:
https://forum.pfsense.org/index.php?topic=76735.0;topicseen
-nb