Win error moving large files across tunnel

pecker88

2 pfSense boxes, both running 2.3.2. Each box has 4GB of memory and quad core Intel Atom CPU.

Site1: 25x5 mbps broadband connection. The LAN at this site is running an Ubuntu Server, Samba configured.

Site2: 100x100 mbps dedicated fiber. The LAN at this site has a workstation.

For months the workstation at Site2 was able to browse and copy/paste files from the Site1 samba server, any size of file, it copied successfully using windows explorer.

Starting yesterday, Site2 cannot copy a file over 20MB in size. Windows explorer throws a "unexpected network error" at almost the exact same time with every attempt.

Packet capture at Site1 just shows 1445 packet sizes, until it errors out, then the packet sizes go down to 100 or so.

I recently added a system tunable on Site1, kern.ipc.nmbclusters = 500000. Not sure this would have an effect.

Any advice? I out of ideas. thanks.
Edit:

I just copied a 100MB file that originated at Site1 to Site3 25x5 mbps connection. Site3 is connected to Site1 over the same openVPN tunnel, however Site1 and Site3 are MUCH closer geographically compared to Site1 and Site2.

So, the problem seems to be high latency between Site1 and Site2, how could I account for this?

johnpoz

when you say high latency your talking what exactly?

SMB over really any latency other than lan is going to suck!!! Are you talking 5, 10, 30, 100ms?

You can get some relief with wan optimizer like riverbed or something.. But smb is so freaking chatty that over any real latency its going to just plain suck..

pecker88

Ping results from Site2 –> Site1 smb server:

ping 192.168.44.201 -l 1500
Reply from 192.168.44.102: bytes=1500 time=78ms TTL=62
Reply from 192.168.44.102: bytes=1500 time=72ms TTL=62
Reply from 192.168.44.102: bytes=1500 time=72ms TTL=62
Reply from 192.168.44.102: bytes=1500 time=73ms TTL=62

Ping results from Site3 –> Site1 smb server:

ping 192.168.44.201 -l 1500
Reply from 192.168.44.102: bytes=1500 time=34ms TTL=62
Reply from 192.168.44.102: bytes=1500 time=39ms TTL=62
Reply from 192.168.44.102: bytes=1500 time=38ms TTL=62
Reply from 192.168.44.102: bytes=1500 time=36ms TTL=62

I don't understand how the Site2 issue copying lg. files from Site1 could just change suddenly like that. This hasn't been a problem for 4+ months that the Site2–Site1 openVPN connection has existed, just started yesterday. Weird.

johnpoz

did the latency spike up? From what it was before..

pecker88

No, it's always been around 75ms.

We have a roadwarrior in Site4 that ALWAYS had this issue with pulling files from Site1, so months ago I configured a separate openVPN server at Site1 for them using TCP and a different port.

I just switched Site2 to use the TCP openVPN server and now I can pull large files without issue, albeit much slower then UDP.

Just can't figure out why this Site2 UDP problem started now, but it seems to be latency induced.?.?