Slow Speed Through VLAN

stevencavanagh

Hi,

Having browsed through this forum plus those for Synology I'm struggling to solve this issue........

I have the following scenario, where a Synology NAS DS920+ is on its own VLAN (VLAN20) and I'm accessing from the main VLAN (default) without issue.

The Pfsense box is:-

(CPU Type Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz
Current: 3405 MHz, Max: 3400 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No

Pfsense ---> Draytek P1280 switch (MS1) ---> Draytek P1280 switch (MS2)

Connection from Pfsense is a LAG, as is connection from MS1 to MS2 (both running at 1GB

The NAS is connected directly to MS2
The PCs are connected directly to MS1

However, when I run Iperf3 from Windows PCs (2 of) to the NAS I get approx the same values as below, which is clearly slow.

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 6.12 MBytes 51.3 Mbits/sec
[ 4] 1.00-2.01 sec 6.12 MBytes 51.1 Mbits/sec
[ 4] 2.01-3.01 sec 6.00 MBytes 50.1 Mbits/sec
[ 4] 3.01-4.00 sec 5.88 MBytes 49.9 Mbits/sec
[ 4] 4.00-5.00 sec 6.00 MBytes 50.2 Mbits/sec
[ 4] 5.00-6.01 sec 6.00 MBytes 49.9 Mbits/sec
[ 4] 6.01-7.01 sec 6.00 MBytes 50.7 Mbits/sec
[ 4] 7.01-8.00 sec 6.00 MBytes 50.6 Mbits/sec
[ 4] 8.00-9.01 sec 6.00 MBytes 50.1 Mbits/sec
[ 4] 9.01-10.01 sec 5.88 MBytes 48.9 Mbits/sec

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.01 sec 60.0 MBytes 50.3 Mbits/sec sender
[ 4] 0.00-10.01 sec 59.9 MBytes 50.2 Mbits/sec receiver

Given that I get the same reading from 2 different PCs on different ports then ~I am assuming the CAT 6 cables from PCs to MS1 are ok. I have also changed the 4 LAG cables from MS1 to MS2 as well as the LAG cables from MS2 to NAS.

I have checked all the settings in the NAS that are being recommended.

Any thoughts on what could be causing it or where to go next as I'm slowly losing the will to live!

Thanks
Steve

stevencavanagh

Just another quick question.

I understand how VLANs work in Pfsense and have mine set up fine with the appropriate rules in place.

However, as I understand it, it would be better to do the inter-VLAN routing at switch level (L3) to get faster speeds. Although at the moment I have 2 managed switches (Draytek P1280), I don't believe these are capable of Inter-VLAN routing. I have a P2280x on the way, which is but will need another at some point.

How do I do the inter-VLAN routing? Probably easy enough in the switch but what if anything will I need to change in Pfsense. Do the currently created 7 VLANs remain and just the relevant inter-VLAN firewall rules removed.

If someone could enlighten me in this area, it would be appreciated.

Thanks
Steve

SteveITS

@stevencavanagh since you mentioned Windows look into disabling RSC on that PC.

planedrop

To answer your second question first, the inter-VLAN routing is going to require a switch that can do it, and it's best to think of the switch as behaving basically as another router, they are layer 3 aware (or layer 2 if you're talking TCP/IP instead of OSI) and are making traffic decisions based on IP headers rather than MACs.

While you are right, in larger setups, using Layer 3 switches is the way to go for faster routing, there isn't a reason pfSense should be limiting you this much, I've pushed line rate via inter-VLAN on pretty low end hardware before (10GbE being line rate) so it's possible.

Now to get into your actual issue here, first, did you try iperf with the -P command to add more threads? Could help troubleshoot things if you say do -P 8, in theory it should be higher by quite a bit (in total) but if not then something may be going on.

Additionally, why put the NAS on it's own VLAN? Most typical is to expose the NAS on the subnets/VLANs you need it so you can deal with things from a layer 2 perspective. (not sure if Synology allows, but then you'd disable any management functions on the non-management VLANs).

Have you done a pcap of this yet to see if there is maybe something funky going on, lots of retransmits or something?

stevencavanagh

@planedrop

Hi,

I did try Iperf with the -P command and no difference in speed, see below:-

ID] Interval Transfer Bandwidth
sender
receiver
sender
receiver
sender
receiver
sender
receiver
sender
receiver
sender
receiver
sender
receiver
sender
receiver
sender
receiver

I put the NAS on its own VLAN as I only wanted certain devices to connect, some on the main LAN and a couple of firesticks from the IOT VLAN and would use firewall rules based on their static IPs. However, if there is a better way then great!

I hadn't done a pcap but have now with results below. Can't see anything wrong with it:-

13:52:43.055299 IP 192.168.0.207.56503 > 192.168.20.200.5201: tcp 1460
13:52:43.055553 IP 192.168.20.200.5201 > 192.168.0.207.56503: tcp 0
13:52:43.055611 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 158
13:52:43.055742 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055869 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055872 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055874 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055877 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055879 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055881 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055884 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055886 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055888 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055890 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055893 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055896 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055993 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1460
13:52:43.055995 IP 192.168.0.207.56503 > 192.168.20.200.5201: tcp 1460
13:52:43.055999 IP 192.168.0.207.56503 > 192.168.20.200.5201: tcp 1460
13:52:43.056002 IP 192.168.0.207.56503 > 192.168.20.200.5201: tcp 1460
13:52:43.056003 IP 192.168.0.207.56503 > 192.168.20.200.5201: tcp 1460
13:52:43.056176 IP 192.168.0.207.53600 > 192.168.60.3.9000: tcp 0
13:52:43.056183 IP 192.168.20.200.5201 > 192.168.0.207.56500: tcp 0
13:52:43.056303 IP 192.168.20.200.5201 > 192.168.0.207.56503: tcp 0
13:52:43.056304 IP 192.168.0.207.53600 > 192.168.60.3.9000: tcp 0
13:52:43.056353 IP 192.168.0.207.56500 > 192.168.20.200.5201: tcp 1460
13:52:43.056757 IP 192.168.0.207.56503 > 192.168.20.200.5201: tcp 1460
13:52:43.056881 IP 192.168.0.207.56503 > 192.168.20.200.5201: tcp 1460
13:52:43.056965 IP 192.168.20.200.5201 > 192.168.0.207.56504: tcp 0
13:52:43.057007 IP 192.168.60.3.9000 > 192.168.0.207.53600: tcp 1079
13:52:43.057288 IP 192.168.0.207.56504 > 192.168.20.200.5201: tcp 1460
13:52:43.057341 IP 192.168.0.207.53600 > 192.168.60.3.9000: tcp 0
13:52:43.057413 IP 192.168.0.207.56504 > 192.168.20.200.5201: tcp 1460
13:52:43.057415 IP 192.168.0.207.56504 > 192.168.20.200.5201: tcp 1460
13:52:43.057417 IP 192.168.0.207.56504 > 192.168.20.200.5201: tcp 1460
13:52:43.057467 IP 192.168.20.200.5201 > 192.168.0.207.56501: tcp 0
13:52:43.057807 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460
13:52:43.057932 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460
13:52:43.057935 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460
13:52:43.058116 IP 192.168.20.200.5201 > 192.168.0.207.56500: tcp 0
13:52:43.058443 IP 192.168.0.207.56500 > 192.168.20.200.5201: tcp 1460
13:52:43.058567 IP 192.168.0.207.56500 > 192.168.20.200.5201: tcp 1460
13:52:43.058569 IP 192.168.0.207.56500 > 192.168.20.200.5201: tcp 1460
13:52:43.058654 IP 192.168.20.200.5201 > 192.168.0.207.56501: tcp 0
13:52:43.058964 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460
13:52:43.059089 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460
13:52:43.059118 IP 192.168.20.200.5201 > 192.168.0.207.56500: tcp 0
13:52:43.059365 IP 192.168.20.200.5201 > 192.168.0.207.56501: tcp 0
13:52:43.059542 IP 192.168.0.207.56500 > 192.168.20.200.5201: tcp 1460
13:52:43.059667 IP 192.168.0.207.56500 > 192.168.20.200.5201: tcp 1460
13:52:43.059669 IP 192.168.0.207.56500 > 192.168.20.200.5201: tcp 1460
13:52:43.059708 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460
13:52:43.059726 IP 192.168.20.200.5201 > 192.168.0.207.56501: tcp 0
13:52:43.059833 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460
13:52:43.059837 IP 192.168.0.207.56501 > 192.168.20.200.5201: tcp 1460

Steve

stevencavanagh

@SteveITS said in Slow Speed Through VLAN:

@stevencavanagh since you mentioned Windows look into disabling RSC on that PC.

I tried to disable RSC but kept getting this error.........

C:\Windows\System32>Powershell Disable-NetAdapterRsc -Name Ethernet
Disable-NetAdapterRsc : No MSFT_NetAdapterRscSettingData objects found with property 'Name' equal to 'Ethernet'.
Verify the value of the property and retry.
At line:1 char:1

Disable-NetAdapterRsc -Name Ethernet

  + CategoryInfo          : ObjectNotFound: (Ethernet:String) [Disable-NetAdapterRsc], CimJobException
  + FullyQualifiedErrorId : CmdletizationQuery_NotFound_Name,Disable-NetAdapterRsc

planedrop

I think my method here would be to give the NAS an interface on the LAN (not sure if the NAS can be multi-homed) so full speeds can be reached, if Synology lets you filter by IP then you can do that (for example I do IP allow whitelisting on my TrueNAS box).

But for IoT, I agree, keeping it segmented is the way to go.

Still should be seeing better speeds than this though.

What is your WAN speed? Any chance it's like 1 gigabit and you can get that full speed through it? Just trying to find more info to help identify the problem area.

stevencavanagh

@planedrop

Could put the NAS on the LAN, which should speed things up but would still have slow speeds to IOT stuff. Think I can block IP addresses on Synology firewall but it isn't currently enabled.

Would I see better speeds given that there are currently 7 VLANs plus LAN on the same interface?

I do have 3 spare interfaces available though (igb1, igb4 & igb5)

WAN speed is only around 60MB currently and the speed across the LAN VLAN is fine :-

Steve

planedrop

I don't think the interface/VLAN count is the issue, I've run environments with like 20 VLANs without issue before (all on the same physical interface).

I'm sure there is something we are missing, but not sure what, this is an odd one.

Maybe something about the LAG is mucking things up, not sure why that would be the case though, have never had issues with LAG setups on pfSense myself (but I admittedly haven't done it a ton).

Your hardware is fine too, I've pushed 10 gig (inter-VLAN, not WAN) through lower-end hardware than this.

Do you have more devices you can do any testing with? Maybe a couple Linux machines you can plug into other switch ports and see how it goes?

stevencavanagh

@planedrop

Unfortunately, I don't have any Linux machines and apart from one other PC (windows) which is also on the LAN, the rest are mobiles / Ipad and other IOT stuff etc so a bit stuffed really.

Don't think I have an issue with LAG as all the LAGs are showing 1GB speed and there are 3 of them (Pfsense to Draytek switch 1), (Draytek switch 1 to Draytek switch 2) and Draytek switch 2 to NAS).

Steve

stevencavanagh

Think the LAG between the 2 switches is working as I configured a new AP on the second switch, connected android phone and ran Iperf3 to a Windows PC on switch one. Results on the phone were:-

Transfer 2.00 MBytes
Bandwidth 563 Mbits/sec

This was on wifi 6

This is a similar result to being connected to an exact copy of the AP but on the first switch.