Very Poor Performance on VLAN Routing
-
It could have been w. jumboframes , as i ran Jumbo for a short time.
Then i decided i didn't need Jumbo on my home network, due to many of my "home appliances" not supporting it. And disabled it site wide.I just reran an iperf to show the OP , that there isn't much difference between pure L2 , or L3 with pfSense as Vlan router.
Switches HP1820 Cat5e Linked
Linux server Deb10 - Realtec NIC - (iperf -s) : 192.168.x.y Linux WS Linux Mint - Intel 82579LM - (iperf -c) : 192.168.x.x or 10.x.x.x Client & Server On same Vlan (Pure L2) # iperf -t60 -i10 -c 192.168.x.y ------------------------------------------------------------ Client connecting to frodo, TCP port 5001 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.x.x port 58296 connected with 192.168.x.y port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.08 GBytes 931 Mbits/sec [ 3] 10.0-20.0 sec 1.08 GBytes 931 Mbits/sec [ 3] 20.0-30.0 sec 1.09 GBytes 933 Mbits/sec [ 3] 30.0-40.0 sec 1.09 GBytes 934 Mbits/sec [ 3] 40.0-50.0 sec 1.08 GBytes 929 Mbits/sec [ 3] 50.0-60.0 sec 1.08 GBytes 929 Mbits/sec [ 3] 0.0-60.0 sec 6.50 GBytes 931 Mbits/sec # Client & Server on different Vlans # iperf -t60 -i10 -c 192.168.x.y ------------------------------------------------------------ Client connecting to frodo, TCP port 5001 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.x.x.x port 33834 connected with 192.168.x.y port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.08 GBytes 930 Mbits/sec [ 3] 10.0-20.0 sec 1.08 GBytes 928 Mbits/sec [ 3] 20.0-30.0 sec 1.08 GBytes 927 Mbits/sec [ 3] 30.0-40.0 sec 1.08 GBytes 926 Mbits/sec [ 3] 40.0-50.0 sec 1.08 GBytes 929 Mbits/sec [ 3] 50.0-60.0 sec 1.08 GBytes 926 Mbits/sec [ 3] 0.0-60.0 sec 6.48 GBytes 927 Mbits/sec #
No pfSense IGBx ethernet tuning at all.
Edit: pfSense CPU load during xfer 23..29%
/Bingo
-
@kdb9000 said in Very Poor Performance on VLAN Routing:
As far as I can tell with Ubiquiti. The ports were setup as an Aggregate across the 3 ports on the Switch.
Which mode was configured on the PFsense side? Which mode was configured on the Ubiquity switch?
-
My settings are the same as what you have in the image (disabled for the two offloading, checksum is the only offloader enabled).
I haven't gotten the iperf working on the Synology (not sure which one I need to install). I did use the Windows version between two computers on different VLANS.
Main > Gaming
Main VLAN>iperf3.exe -c 192.168.13.235 Connecting to host 192.168.13.235, port 5201 [ 4] local 192.168.10.60 port 57740 connected to 192.168.13.235 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 4.50 MBytes 37.7 Mbits/sec [ 4] 1.00-2.00 sec 38.0 MBytes 319 Mbits/sec [ 4] 2.00-3.00 sec 56.1 MBytes 470 Mbits/sec [ 4] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec [ 4] 4.00-5.00 sec 56.9 MBytes 477 Mbits/sec [ 4] 5.00-6.00 sec 55.1 MBytes 462 Mbits/sec [ 4] 6.00-7.00 sec 57.8 MBytes 484 Mbits/sec [ 4] 7.00-8.00 sec 58.4 MBytes 490 Mbits/sec [ 4] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec [ 4] 9.00-10.00 sec 55.4 MBytes 464 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 494 MBytes 414 Mbits/sec sender [ 4] 0.00-10.00 sec 494 MBytes 414 Mbits/sec receiver iperf Done. Gaming VLAN>iperf3.exe -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.10.60, port 57739 [ 5] local 192.168.13.235 port 5201 connected to 192.168.10.60 port 57740 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.01 sec 4.00 MBytes 33.2 Mbits/sec [ 5] 1.01-2.00 sec 32.3 MBytes 274 Mbits/sec [ 5] 2.00-3.00 sec 56.1 MBytes 471 Mbits/sec [ 5] 3.00-4.00 sec 56.0 MBytes 469 Mbits/sec [ 5] 4.00-5.00 sec 56.8 MBytes 476 Mbits/sec [ 5] 5.00-6.00 sec 55.1 MBytes 462 Mbits/sec [ 5] 6.00-7.00 sec 57.7 MBytes 484 Mbits/sec [ 5] 7.00-8.00 sec 58.5 MBytes 491 Mbits/sec [ 5] 8.00-9.00 sec 56.2 MBytes 471 Mbits/sec [ 5] 9.00-10.00 sec 55.4 MBytes 464 Mbits/sec [ 5] 10.00-10.11 sec 6.01 MBytes 475 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.11 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.11 sec 494 MBytes 410 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5201 -----------------------------------------------------------
And then from Gaming > Main
Gaming VLAN>iperf3.exe -c 192.168.10.60 Connecting to host 192.168.10.60, port 5201 [ 4] local 192.168.13.235 port 64557 connected to 192.168.10.60 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 34.2 MBytes 287 Mbits/sec [ 4] 1.00-2.01 sec 33.1 MBytes 277 Mbits/sec [ 4] 2.01-3.00 sec 33.6 MBytes 284 Mbits/sec [ 4] 3.00-4.00 sec 32.6 MBytes 273 Mbits/sec [ 4] 4.00-5.00 sec 29.5 MBytes 247 Mbits/sec [ 4] 5.00-6.01 sec 31.6 MBytes 265 Mbits/sec [ 4] 6.01-7.00 sec 33.0 MBytes 278 Mbits/sec [ 4] 7.00-8.00 sec 31.6 MBytes 265 Mbits/sec [ 4] 8.00-9.00 sec 33.0 MBytes 277 Mbits/sec [ 4] 9.00-10.00 sec 32.1 MBytes 269 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 324 MBytes 272 Mbits/sec sender [ 4] 0.00-10.00 sec 324 MBytes 272 Mbits/sec receiver iperf Done. Main VLAN>iperf3.exe -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.13.235, port 64556 [ 5] local 192.168.10.60 port 5201 connected to 192.168.13.235 port 64557 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 33.3 MBytes 280 Mbits/sec [ 5] 1.00-2.00 sec 32.5 MBytes 273 Mbits/sec [ 5] 2.00-3.00 sec 33.6 MBytes 282 Mbits/sec [ 5] 3.00-4.00 sec 32.6 MBytes 273 Mbits/sec [ 5] 4.00-5.00 sec 29.3 MBytes 246 Mbits/sec [ 5] 5.00-6.00 sec 31.7 MBytes 266 Mbits/sec [ 5] 6.00-7.00 sec 33.1 MBytes 278 Mbits/sec [ 5] 7.00-8.00 sec 31.5 MBytes 264 Mbits/sec [ 5] 8.00-9.00 sec 33.1 MBytes 277 Mbits/sec [ 5] 9.00-10.00 sec 32.1 MBytes 269 Mbits/sec [ 5] 10.00-10.04 sec 1.58 MBytes 304 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.04 sec 324 MBytes 271 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5201 -----------------------------------------------------------
Nothing in the network as changed since I posted the initial thread, and the SMB traffic between the VLANs is still very poor compared to what the iperf test shows (which I have seen some people the iperf test isn't a very good test).
-
Iperf on pfsense is not a good test no.
But client to client through pfsense is good test.
So what do you see from client to client on the same network? Because those speeds are terrible for gig.. You should be seeing high 800's to low 900s for sure..
What specific model of nas do you have? And I can help you figure out which synology iperf you want. For example on my ds918 its the apollolake..
-
@marvosa said in Very Poor Performance on VLAN Routing:
@kdb9000 said in Very Poor Performance on VLAN Routing:
As far as I can tell with Ubiquiti. The ports were setup as an Aggregate across the 3 ports on the Switch.
Which mode was configured on the PFsense side? Which mode was configured on the Ubiquity switch?
Aggregate is what is it called on the Ubiquiti side, LAGG is what it is called on the pfSense side. On the pfSense side, the protocol was LACP. Ubiquiti doesn't have any other options to change for the Aggregate (aside from setting Link Speed and how many ports are in the Aggregate). At this time, I am not running LAGG on pfSense. Instead the 3 connections are individual with different VLAN's attached to them.
-
What does an L2 iperf report ?
I mean server & client on the same subnetAre you running hairpin / "On a Stick" when doing the inter Vlan xfers ?
I have divided my Vlans across two pfSense interfaces.
And made sure my Server and (cabled) Client vlans are on separate IGBx interfaces./Bingo
-
This post is deleted! -
@johnpoz said in Very Poor Performance on VLAN Routing:
Iperf on pfsense is not a good test no.
But client to client through pfsense is good test.
So what do you see from client to client on the same network? Because those speeds are terrible for gig.. You should be seeing high 800's to low 900s for sure..
What specific model of nas do you have? And I can help you figure out which synology iperf you want. For example on my ds918 its the apollolake..
I haven't been able to test that yet. Having issues getting iperf on the Synology. I will say, when the Synology was on a different network (had one called Server before I moved the Synology) I had a lot of issues with transferring files to it and even using OwnCloud (which is hosted on the Synology). Backup using Veeam was also an issue (similar to what I am seeing with the on in the Gaming VLAN). Once it was moved to the Main VLAN, all of those issues went away (so going from Layer 3 to Layer 2). The number of hops and the setup of the Synology (other then the IP) has not changed.
-
@bingo600 said in Very Poor Performance on VLAN Routing:
What does an L2 iperf report ?
I mean server & client on the same subnetAre you running hairpin / "On a Stick" when doing the inter Vlan xfers ?
I have divided my Vlans across two pfSense interfaces.
And made sure my Server and (cabled) Client vlans are on separate IGBx interfaces./Bingo
It was "On a Stick" and worked without issue for a long time. It was only recently it started acting up. I have since spread out the VLANs onto the other interfaces, although Main and Gaming at on the same interface. When I tried to move it, pfSense was having issues with the routing (it still said it was on the one interface when I had moved it to another interface) and was blocking the traffic (at least outbound from the VLAN, inbound to the VLAN worked fine).
-
@kdb9000 which specific nas do you have - can lookup up which version of the software you need. I have ds918 which is the apollolake software..
-
@johnpoz said in Very Poor Performance on VLAN Routing:
@kdb9000 which specific nas do you have - can lookup up which version of the software you need. I have ds918 which is the apollolake software..
DS1817+
-
@kdb9000 said in Very Poor Performance on VLAN Routing:
Nothing in the network as changed since I posted the initial thread, and the SMB traffic between the VLANs is still very poor compared to what the iperf test shows (which I have seen some people the iperf test isn't a very good test).
I hate it when people are using SMB as ANY kind of network performance test.
SMB performance depends on the Server CPU load , and disk load at the exact moment.Then i end up having people blaming the network , for their lousy overcomitted Virtual server, with mechanical disks
/Bingo
-
@kdb9000 said in Very Poor Performance on VLAN Routing:
although Main and Gaming at on the same interface. When I tried to move it, pfSense was having issues with the routing (it still said it was on the one interface when I had moved it to another interface) and was blocking the traffic (at least outbound from the VLAN, inbound to the VLAN worked fine).
So the above iperfs you showed , are using "On a stick" (same) interface.
As it's Main -> Gaming , and reverse ?/Bingo
-
@bingo600 said in Very Poor Performance on VLAN Routing:
@kdb9000 said in Very Poor Performance on VLAN Routing:
Nothing in the network as changed since I posted the initial thread, and the SMB traffic between the VLANs is still very poor compared to what the iperf test shows (which I have seen some people the iperf test isn't a very good test).
I hate it when people are using SMB as ANY kind of network performance test.
SMB performance depends on the Server CPU load , and disk load at the exact moment.Then i end up having people blaming the network , for their lousy overcomitted Virtual server, with mechanical disks
/Bingo
I monitored the Synology system, it was basically idle when I tried doing the backup and/or file transfers. OwnCloud doesn't use SMB (at least when uploading through the web page and I do not believe so with the Windows Client), and was also problematic. The Veeam backup does use SMB, and while monitoring the performance of the computer and the storage there wasn't anything that would cause the transfer to be slow. To also add to that, when I did the VLAN setup on the Synology (one VLAN in the same as the system backing up, the other required me to go to pfSense) it would preform at the 500 KB/s going across the VLAN compared to the MB/s I see when I went to it directly on Layer 2. These test were done one right after the other (not at the same time).
I will also point out that I do not have any Virtual Servers (unless you want to count what I am running in Docker as a Virtual Server) in play with this setup. If you want to blame Docker for the OwnCloud part, I had slow transfers, interrupted transfers, and issues when using it over Layer 3. I did not have any issues when I switch it to Layer 2.
-
@bingo600 said in Very Poor Performance on VLAN Routing:
@kdb9000 said in Very Poor Performance on VLAN Routing:
although Main and Gaming at on the same interface. When I tried to move it, pfSense was having issues with the routing (it still said it was on the one interface when I had moved it to another interface) and was blocking the traffic (at least outbound from the VLAN, inbound to the VLAN worked fine).
So the above iperfs you showed , are using "On a stick" (same) interface.
As it's Main -> Gaming , and reverse ?/Bingo
Yes, until I can get pfSense to correctly move the VLAN to another interface. I am Working from Home, so it isn't very easy to reset my Router/Firewall at this time.
-
My SMB "rant" was not meant for you, in particular.
It was gathered , from many job debug situations , where 95% of the SMB tests , were proven wrong by iperf. But it takes a lot of convincing to get a M$ Admin to accept that iperf is the way to go, when testing network performance. -
@bingo600 said in Very Poor Performance on VLAN Routing:
My SMB "rant" was not meant for you, in particular.
It was gathered , from many job debug situations , where 95% of the SMB tests , were proven wrong by iperf. But it takes a lot of convincing to get a M$ Admin to accept that iperf is the way to go, when testing network performance.I wasn't sure, but I know some people might pick it up and run with it. And I know what you mean, we have to fight with out Database people about the Storage system (they keep blaming performance issues on Storage when we do not see any issues related to it). We did find issues with the Databases that we brought to their attention, and after that most of the issues stopped or it wasn't as bad as it was.
-
Did you see this one
https://www.reddit.com/r/synology/comments/f2s6nv/only_getting_1gbit_transfer_speeds_rather_than/Something about SMB signing
-
@bingo600 said in Very Poor Performance on VLAN Routing:
Did you see this one
https://www.reddit.com/r/synology/comments/f2s6nv/only_getting_1gbit_transfer_speeds_rather_than/Something about SMB signing
SMB Signing is related to Domains (as far as I can tell and something that was mentioned in the Reddit you sent). One of the Synology troubleshooting guides mentioned SMB Signing as well, but it was only if the Synology was part of a Domain setup.
-
Re iperf install
https://www.synology.com/en-global/knowledgebase/DSM/tutorial/Compatibility_Peripherals/What_kind_of_CPU_does_my_NAS_have
DS1817+ Intel Atom C2538 Quad Core 4 Yes Avoton DDR3 2/8 GB
Do you see any Avoton arch - iperf packages
-
@bingo600 said in Very Poor Performance on VLAN Routing:
DS1817+ Intel Atom C2538 Quad Core 4 Yes Avoton DDR3 2/8 GB
http://www.jadahl.com/iperf-arp-scan/DSM_6.2/iperf_avoton-6.2_3.7-1.spk
Install that spk, then ssh to your nas, then just run iperf3 -s
Then hit your nas IP from linux or windows client also running iperf.. I compile iperf3 for windows myself..
Here is that
iperf3.9_64.zip -
@bingo600 said in Very Poor Performance on VLAN Routing:
Re iperf install
https://www.synology.com/en-global/knowledgebase/DSM/tutorial/Compatibility_Peripherals/What_kind_of_CPU_does_my_NAS_have
DS1817+ Intel Atom C2538 Quad Core 4 Yes Avoton DDR3 2/8 GB
Do you see any Avoton arch - iperf packages
@johnpoz said in Very Poor Performance on VLAN Routing:
@bingo600 said in Very Poor Performance on VLAN Routing:
DS1817+ Intel Atom C2538 Quad Core 4 Yes Avoton DDR3 2/8 GB
http://www.jadahl.com/iperf-arp-scan/DSM_6.2/iperf_avoton-6.2_3.7-1.spk
That one worked. Is there a specific way to turn on the server part (do I need to do it from the command line or is it just always running)?
-
see my post above yours ;) I edited it.
from the client you would run iperf3 -c ipaddressofnas
edit: To @bingo600 rant on smb tests.. I really agree there... While slow smb transfers of can be a symptom of network problems. It doesn't prove that there is a network issue..
If you do an iperf and its slow - then that can explain your slow smb. If your your iperf test shows the network is fine for speed, then your slow smb problem is something else..
The iperf you showed are not normal for a gig network... So something is clearly not right there..
But lets see test from device A to B on the same network - this should be 940ish maxed out if everything is perfect.. But as long as your in the 800's - could just be driver issue, etc.. Maybe you need to do some tweaking of the driver settings or something. But if you were seeing 800ish on your network... You should be seeing more than 80MBps -- should be 100 or so.. And you sure shouldn't be seeing wild fluctuations in the transfer
-
@johnpoz said in Very Poor Performance on VLAN Routing:
see my post above yours ;) I edited it.
from the client you would run iperf3 -c ipaddressofnas
Not sure which post you are referring to. When I try and run the test to the Synology, I get connection refused (I have it installed and it says it is running).
-
And are you running firewall on your NAS?
-
@johnpoz said in Very Poor Performance on VLAN Routing:
And are you running firewall on your NAS?
No, it is disabled.
-
@kdb9000 said in Very Poor Performance on VLAN Routing:
@johnpoz said in Very Poor Performance on VLAN Routing:
see my post above yours ;) I edited it.
from the client you would run iperf3 -c ipaddressofnas
Not sure which post you are referring to. When I try and run the test to the Synology, I get connection refused (I have it installed and it says it is running).
Now .. Your pfSense or your NAS isn't blocking the iperf packages ?
I usually "cheat" and run iperf as root ... Not that that would help if iptables is blocking./Bingo
-
Are you running it through pfsense? - port 5201 is the default port.. That will have to be allowed, but refused points to OS sending back icmp saying NO. Ie an OS firewall. If your trying to use a 2.x client on your test client the port is different.. You need to use iperf3 on both client and server - the nas version is 3.7, I uploaded windows 3.9 version which work fine together.
-
@johnpoz said in Very Poor Performance on VLAN Routing:
Are you running it through pfsense? - port 5201 is the default port.. That will have to be allowed, but refused points to OS sending back icmp saying NO. Ie an OS firewall. If your trying to use a 2.x client on your test client the port is different.. You need to use iperf3 on both client and server - the nas version is 3.7, I uploaded windows 3.9 version which work fine together.
@bingo600 said in Very Poor Performance on VLAN Routing:
@kdb9000 said in Very Poor Performance on VLAN Routing:
@johnpoz said in Very Poor Performance on VLAN Routing:
see my post above yours ;) I edited it.
from the client you would run iperf3 -c ipaddressofnas
Not sure which post you are referring to. When I try and run the test to the Synology, I get connection refused (I have it installed and it says it is running).
Now .. Your pfSense or your NAS isn't blocking the iperf packages ?
I usually "cheat" and run iperf as root ... Not that that would help if iptables is blocking./Bingo
It is from the system on the same network (so not pfSense at this point). Running a "netstat -nato" on the Synology, I don't see it listening on port 5201. I am using 3.1.3 on Windows.
-
@kdb9000 said in Very Poor Performance on VLAN Routing:
I don't see it listening on port 5201
Then its not running..
You see this?
ash-4.3# iperf3 -s ----------------------------------------------------------- Server listening on 5201 -----------------------------------------------------------
From another ssh prompt, you should see it running
netstat -anl | grep 5201 tcp6 0 0 :::5201 :::* LISTEN
-
The "server usually" shows what port it listens on
iperf3 -s ----------------------------------------------------------- Server listening on 5201 -----------------------------------------------------------
-
@johnpoz said in Very Poor Performance on VLAN Routing:
@kdb9000 said in Very Poor Performance on VLAN Routing:
I don't see it listening on port 5201
Then its not running..
You see this?
ash-4.3# iperf3 -s ----------------------------------------------------------- Server listening on 5201 -----------------------------------------------------------
It said running in the GUI. I asked if I needed to run it from the Synology, but you just said to see the post above mine.
-
Is there an echo in here
-
hehee - Yeah something getting missed. I clearly gave instructions in my post
And I also uploaded the lastest 3.9 version for windows.. not sure why you would be running some old 3.1 version ;) Are you on linux - if so I Know they have later than 3.1 available
-
@KDB9000
@johnpoz said in Very Poor Performance on VLAN Routing:@bingo600 said in Very Poor Performance on VLAN Routing:
DS1817+ Intel Atom C2538 Quad Core 4 Yes Avoton DDR3 2/8 GB
http://www.jadahl.com/iperf-arp-scan/DSM_6.2/iperf_avoton-6.2_3.7-1.spk
Install that spk, then ssh to your nas, then just run iperf3 -s
Then hit your nas IP from linux or windows client also running iperf.. I compile iperf3 for windows myself..
There was a slight hint in the post
-
oh if your on say 18 of ubuntu, yeah that looks like only 3.1.3.. 21 currently has 3.9
but shouldn't matter 3.1 to 3.7 would be ok, same port used.
-
@johnpoz said in Very Poor Performance on VLAN Routing:
hehee - Yeah something getting missed. I clearly gave instructions in my post
And I also uploaded the lastest 3.9 version for windows.. not sure why you would be running some old 3.1 version ;) Are you on linux - if so I Know they have later than 3.1 available
@bingo600 said in Very Poor Performance on VLAN Routing:
@KDB9000
@johnpoz said in Very Poor Performance on VLAN Routing:@bingo600 said in Very Poor Performance on VLAN Routing:
DS1817+ Intel Atom C2538 Quad Core 4 Yes Avoton DDR3 2/8 GB
http://www.jadahl.com/iperf-arp-scan/DSM_6.2/iperf_avoton-6.2_3.7-1.spk
Install that spk, then ssh to your nas, then just run iperf3 -s
Then hit your nas IP from linux or windows client also running iperf.. I compile iperf3 for windows myself..
There was a slight hint in the post
Actually, nothing was missed. That part does not show on my side.
-
The system has been auto updating the posts as they come in. I am guessing you edited it after the initial post and the system did not update it. When I refresh the whole thread, it now shows up (the screen shot was before I refresh the whole thread).
If you look at my post after yours with the Quote, it still shows the initial post quote that I responded to (which doesn't have the included instructions).
-
@kdb9000
Don't waste time on "instructions"We WANNA see the iperf3 results
-
System A > Synology (Same Network)
>iperf3.exe -c 192.168.10.246 Connecting to host 192.168.10.246, port 5201 [ 4] local 192.168.10.60 port 58904 connected to 192.168.10.246 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 74.8 MBytes 627 Mbits/sec [ 4] 1.00-2.00 sec 74.8 MBytes 627 Mbits/sec [ 4] 2.00-3.00 sec 77.8 MBytes 652 Mbits/sec [ 4] 3.00-4.00 sec 78.1 MBytes 655 Mbits/sec [ 4] 4.00-5.00 sec 63.5 MBytes 533 Mbits/sec [ 4] 5.00-6.00 sec 73.6 MBytes 618 Mbits/sec [ 4] 6.00-7.00 sec 74.2 MBytes 623 Mbits/sec [ 4] 7.00-8.00 sec 76.2 MBytes 640 Mbits/sec [ 4] 8.00-9.00 sec 74.4 MBytes 623 Mbits/sec [ 4] 9.00-10.00 sec 75.9 MBytes 637 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 743 MBytes 623 Mbits/sec sender [ 4] 0.00-10.00 sec 743 MBytes 623 Mbits/sec receiver iperf Done.
System B > Synology (passing through pfSense)
>iperf3.exe -c 192.168.10.246 Connecting to host 192.168.10.246, port 5201 [ 4] local 192.168.13.235 port 49405 connected to 192.168.10.246 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 78.0 MBytes 654 Mbits/sec [ 4] 1.00-2.00 sec 78.2 MBytes 656 Mbits/sec [ 4] 2.00-3.00 sec 78.0 MBytes 655 Mbits/sec [ 4] 3.00-4.00 sec 78.1 MBytes 655 Mbits/sec [ 4] 4.00-5.00 sec 78.1 MBytes 655 Mbits/sec [ 4] 5.00-6.00 sec 77.8 MBytes 652 Mbits/sec [ 4] 6.00-7.00 sec 77.9 MBytes 654 Mbits/sec [ 4] 7.00-8.00 sec 78.1 MBytes 655 Mbits/sec [ 4] 8.00-9.00 sec 78.2 MBytes 656 Mbits/sec [ 4] 9.00-10.00 sec 78.2 MBytes 656 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 781 MBytes 655 Mbits/sec sender [ 4] 0.00-10.00 sec 781 MBytes 655 Mbits/sec receiver iperf Done.
Drivers on all of the systems are up to date and the correct drives. The Synology is on the latest (and I do mean latest, I had to manually upload it from Synology) version of software. While the iperf seems to indicate there isn't a network issue, it doesn't explain why I have such a difference in speed in Layer 3 to Layer 2 (more then just SMB).
Before I moved the Synology to the same network as System A I would get the following (I don't have actual numbers, so it is from memory at this point) when running a backup:System A (Main VLAN) > pfSense > Synology (Server VLAN) with speeds in the KB range and failure to complete
System A (Main VLAN) > Synology (Main VLAN) with speeds in the MB range and completed without an issueWhen this test was done, I had two VLANs setup on the Synology (one on the Server VLAN and one on the Main VLAN). The backup is setup in the job to go to the Server VLAN IP. I told it to do a standalone backup (which is a full) to the Main VLAN IP of the Synology and got better results without failure of the backup. I also had issues uploading things to the Synology using OwnCloud when it was on the Server VLAN IP (no issues when it was on the Main VLAN IP, and this isn't SMB at this point). And those files were KB in size when it gave me issues. After the move to the Main VLAN, it didn't have an issue with those same files.