[SOLVED] File transfer slow across other subnets, works fine on same subnet



  • Having this odd issue where I'm almost unable to transfer files from subnet A to subnet B as the speeds are less than 1 MB/s. In this scenario, subnet A is my LAN (10.1.1.x) and subnet B is my WiFi (10.1.2.x). My file server resides on subnet A and works perfectly fine when other systems on the same subnet transfer files to it. This network is fully gigabit and I'm getting an average of 100 MB/s on file copy. When I attempt to copy files from any WiFi systems on subnet B to my file server on subnet A, the speeds drop drastically to KB/s and barely hit 1 MB/s. The AP for this network is a TRENDnet TEW-638PAP 300Mbps AP connecting to an Intel 4 port gigabit NIC on my pfsense box.

    So far I've tried setting the duplex to full but that did nothing. Not really sure what else to try. I'm open to any other suggestions other than having my WiFi reside on subnet A. Also, not sure if this is related or has any bearing but I'm unable to ping or connect to systems on a another subnet such as the example here. I can connect by IP without any problems but hostnames will not work. The only time I can connect to a file share by hostname or ping it is if the system is on the same subnet which in my case is subnet A.

    Could this possibly be on the Windows side? The file sever is FreeBSD but I'm using Windows clients to access it.

    Appreciate any help I can receive on the first part of this post and please let me know if this question belongs in a different thread.

    Thanks


  • Netgate

    So far I've tried setting the duplex to full but that did nothing.

    You should be set to autonegotiate on both sides.

    Check for interface errors on both subnets in both directions.

    Your name resolution/discovery issues are a different problem.


  • Rebel Alliance Global Moderator

    That AP is quite old.. It only has 10/100 nic.  So as typical its 300 is marketing nonsense.  There is no way it could ever do that sort of speed.  You might be able to connect PHY at that speed.. But if your at 10/100 on the physical the MAX you could ever ever see would be at best if all the stars are aligned 90ish mbps..

    While this should give you closer to 10MBps – what other devices do you have on the wifi, what is the connection speed that AP is actually making - maybe is only wired at 10mbps.. Which would put your right around that 1MBps limit your seeing.

    Since you say you have it directly connected to pfsense interface - what does pfsense show for that interface speed when its set to auto?

    I would suggest it might be time to upgrade to a current AP that support gig on its wire and AC for wifi..



  • @johnpoz:

    That AP is quite old.. It only has 10/100 nic.  So as typical its 300 is marketing nonsense.  There is no way it could ever do that sort of speed.

    Completely agree, and I'm not expecting 100 MB/s internally over wireless but I'd like to think this AP can do better than 1 MB/s. I've thought about upgrading but I haven't had a reason to until now as my WiFi has been working fine and download speeds on the internet haven't raised any issues. There aren't many wireless devices connecting on this AP either, maybe 3 android phones setup for WiFi and a couple laptops. Nothing massive that would cause these speeds to come to a crippling halt when transferring files to the NAS.

    I'm going to do a bit more digging, I noticed that when I looked at the status of my interfaces I was getting 2654/0 in/out errors for this wifi interface. Decided to do a netstat - i and the "Ierrs" column has a value of 2871. Could possibly be a hardware issue with the AP itself. I've checked all the cables and the Intec NIC port the AP is connected on doesn't look to be an issue. I think I'm going to do a bit more testing then maybe just invest in another AP and see if that resolves my issues. Either way, I'll update this post with my results. Please feel free to comment with any other ideas that could possible help me track the bottleneck here.

    Thanks



  • @Derelict:

    So far I've tried setting the duplex to full but that did nothing.

    You should be set to autonegotiate on both sides.

    Check for interface errors on both subnets in both directions.

    Your name resolution/discovery issues are a different problem.

    I ran a netstat -i and noticed some Ierrs for that interface. When looking at the status of the interface I'm seeing 2654/0 - "in/out errors". Also, the media for my LAN and WiFi look to be set correctly according to the hardware they have. My LAN is set to 1000baseT <full-duplex>and my WiFi is set to 100baseTX <full-duplex>as this is only a 10/100mb AP. Each interface is set to default-auto which looks to be working as expected. I wouldn't think 2654/0 is a lot to be concerned with but when I reviewed the same info for my LAN and DMZ they are both clean at 0/0 in/out errors.</full-duplex></full-duplex>


  • Rebel Alliance Global Moderator

    So what speeds do you get to the internet?  What is your max internet speed that you pay for?  What speed to the internet do you get from your wired devices, and what speed of internet do you get from wifi?

    If your saying you get more than significantly more than 1MB per second to the internet from wifi, then there is something else going on between the vlans.

    Where exactly are you doing the netstat -i and seeing these errors?  Show errors is a sign of something wrong.. those numbers should really be zero or really close to it..

    All the interfaces on my pfsense show 0 for Ierrors, Drop, Oerrors and Coll



  • On my WiFi I get the following according to speedtest.net:

    7 ms ping - 15.82 MB/s Down - 17.30 MB/s Up

    LAN:

    5 ms ping - 53.49 MB/s Down - 63.55 MB/s up

    I have Verizon Fios and I pay for 50 up/50 down MB/s internet. I executed the netstat -i from the command prompt within pfsense to gather those results related to the errors after I noticed the in/out errors from the status of the interface.

    I plan on connecting a physical cable from my laptop to the wifi port and attempting a file copy to the NAS recording what speeds I'm getting across the different subnets. In theory, I should get the same results or close to it as I would from a copy just on subnet A. I'll post my results from that test shortly. I should also mention that I'm not using any vlan tagging/trunking nor am I utilizing any form of traffic shaping on any of these subnets. Thanks again for your responses!



  • Took my laptop and plugged it directly into the WiFi port on my NIC. My results on speedtest were 6 ms ping - 54.82 MB/s Down - 52.27 MB/s Up. My file transfer rate to the NAS was basically non existent it sat there "calculating" the time remaining while only displaying under 300 KB/s transfer. To me, this rules out the AP being the single point of the bottleneck. Is there a monitoring tool within pfsense that may help me diagnose this a bit more to figure out why I'm having such performance issues between subnets? I noticed there's a packet capture I could try but I'm not sure I'll get anything useful from that but I'm open to any suggestions.

    Thanks


  • Rebel Alliance Global Moderator

    Yeah you have something else going on it would seem.  What are you rules on your wifi lan interface on pfsense.  Your not trying to nat between them are you?  You don't have any gateways set on these lan side pfsense interfaces?



  • My WiFi firewall rules are vanilla. Protocol IPv4*, Source WiFi net, Port *, Destination *, Port *, Gateway *, Queue "None". I configured this rule so the WiFi would have access to all interfaces (including the WAN for internet access). Should I configure this in another way?

    I hope I'm not running NAT internally… My background is more in systems rather than networking but from what I can see under Firewall - NAT, my 1:1 is empty (no configurations made), my Outbound settings are checked for Manual Outbound NAT rule generation. (AON - Advanced Outbound NAT) with auto created rules for each subnet to have access to the WAN and my "NPt" has no configurations.

    Under "System - Routing - Gateways" I only have one entry configured for my WAN_DHCP (default). There's currently nothing configured in my "Static Routes" or "Gateway Groups". When I configured each individual interface, I created them with their own /24 IPv4 Address, but I didn't create any other gateways as this setup appeared to work for my needs.

    System - Advanced - Firewall & NAT rules are as follows:

    NAT Reflection mode for port forwards is set to "Pure NAT"
    Enable automatic outbound NAT for 1:1 NAT is checked for Automatic creation of additional NAT redirect rules from within the internal networks
    Enable automatic outbound NAT for Reflection is checked for Automatic create outbound NAT rules that direct traffic back out to the same subnet it originated from

    Maybe there's something here within my NAT settings that I should adjust? I should also mention that this is regarding pfsense version 2.3.2. I've also tested file copies using Windows, Mac, and Ubuntu 12.04. Neither operating system provides a difference in speed and they all are mostly identical transferring data in KB/s.

    I greatly appreciate the help around this. Let me know if there's any other settings/details I should provide. If I somehow come across a eureka moment I'll be sure to post my findings here to hopefully help the next padawon in his pfsense journey  :D.


  • Rebel Alliance Global Moderator

    Out of the box no pfsense would not nat between lan side segments, unless it thought it was a wan because you put a gateway on it.  You can see real easy by just looking at your outbound nats.  But even a nat should not cause a problem.

    What I would do is take a packet capture/sniff using pfsense packet capture on the interface of your wifi lan using your wired box that your seeing the really bad speed on do your test for a file transfer.. Are you seeing loads of errors? retrans etc….

    While there is going to be some hit on routing/firewalling traffic across pfsense it should not be anywhere close to such a hit.  And your hit to your wan should be the most because now its doing firewall/route and nat.. between your lans your only doing route/firewall rules.

    I move traffic between my segments all the time - and while I do not get full gig wire speeds.. I do see 400mbps or so - and then my pfsense is a VM on a OLD box running esxi..



  • Do you think it could be my firewall rules aren't setup correctly between the two interfaces? Even though I have it open on both interfaces is there somewhere else I should check so the traffic is reaching the destination? Seems like I'm able to connect to the file share by IP without issue but once I initiate a file copy it comes to a complete crawl/halt and I wonder if I have some loop in my network or causing lag somehow. Just trying to brainstorm anything else this could be and settings I should check.

    I tried doing a packet capture while setting the results to "full" so I could get a detailed report while a file copy is occurring. There's a bunch of info thrown at me but nothing screams an error of any sort unfortunately.



  • Good news, I figured out the problem. Long story short after much testing, I came to the conclusion that the NIC port on my 4-port Intel card was faulty for the WiFi interface. Luckily, I had a spare 4-port spare card kicking around that I was able to replace in my router and verify this was the case. Definitely appreciate all the help I got on this thread from johnpoz and although my issue didn't immediately scream it was a hardware issue, hopefully this thread helps someone else down the line if they experience these same symptoms.

    Thanks again all!


  • Rebel Alliance Global Moderator

    What is odd that is hardware is that your saying you were getting full speed to the internet using the same port..  That doesn't seem to make a lot of sense..



  • I was getting roughly 15 MB/s down and thought that was about normal for WiFi. However, I performed another speedtest with the new NIC card and was getting 50 MB/s (what I pay for essentially) on the WiFi. Kind of tricked me into thinking the interface port was working as it should have been for a 10/100 AP…

    Either way, I'm just glad I had another card kicking around to quickly test this and verify a solution even though the hardware issue wasn't as prevalent as most others are.


  • Rebel Alliance Global Moderator

    "Took my laptop and plugged it directly into the WiFi port on my NIC. My results on speedtest were 6 ms ping - 54.82 MB/s Down - 52.27 MB/s Up. "

    How does this show something wrong with the nic??  There your getting full speed of your internet are you not?



  • Hence why this wasn't obvious  :D

    Below are my results of the original NIC card compared to the identical spare card:

    Original Card Results

    • Speedtest while plugged directly into pfsense router for WiFi: 50 MB/s up/down
    • Speed of file transfers between subnet A to subnet B plugged into interface: 100-300 KB/s
    • Speedtest over WiFi (TrendNet AP): 15 MB/s Down - 17 MB/s Up
    • Speed of file transfers between subnet A to subnet B over WiFi: 100-300 KB/s

    Spare Card Results

    • Speedtest while plugged directly into pfsense router for WiFi: 50 MB/s up/down
    • Speed of file transfers between subnet A to subnet B plugged into interface: 70-80 MB/s
    • Speedtest over WiFi (TrendNet AP): 50 MB/s Down - 50 MB/s up/down
    • Speed of file transfers between subnet A to subnet B over WiFi: 60-70 MB/s

    No other settings were changed on pfsense as this was simply a card swap. It doesn't make much sense to me either why I was getting full speeds on that NIC port to the internet when plugged directly into the interface but for some reason it was having a difficult time sending/receiving traffic from two segmented subnets and the WiFi wasn't nearly as fast even for this old AP. Once I recorded my results with the spare card I chalked it up as a faulty NIC port. Maybe some engineer can come on here and give me an explanation why I saw such a drastic difference between the two identical cards but I'm happy it's all set now.

    Hope that helps give you some clarification into my troubleshooting johnpoz