[SOLVED] Large file transfers between interfaces dropping
-
Problem solved by adding static routes to VLANs that weren't in the same network as FreeNAS. I assume it had something to do with asymmetric routing and with FreeNAS not setting gateways on VLAN.
-
If you felt it was condescending I apologize.. I tend to write from the hip and the heart and just let it flow..
I am all for helping people learn what I love, etc.. But knowing this stuff that I do it can be frustrating watch someone butcher your passion ;)
Its like a chef watching someone over salt a dish or, or putting ketchup on a $50 steak ;)
I am more than willing to help you straighten out the mess it seems you have created from your description.. But lets start with a drawing of what you have - and what you want to accomplish and we can work out the best way to do that.. Throwing in another router/firewall when you already have one that your happy with and working is not always the best option.
While I think you could replace that usg and be much happier.. If that is what you have to work with - lets work out if makes sense to even use pfsense at all.. I have a usg on my shelf - it works, and not a bad price point for what it can do.. But to be honest - its not the most friendly interface to work with.. And is way more difficult to do even the basic stuff that is simple and straight forward with pfsense. As soon as the pfsense box I got was off back ordered and delivered I could not get that usg off my network fast enough ;) Even though I love their AP - their usg is like having a chef being forced to use a plastic knife and toothpicks as their only tools.
You can for sure use a downstream router and firewall in your network - but to be honest at your scale its more than likely just over complicating it all.
If you could take the time to draw up what you currently have and describe what you want to accomplish from a filter standpoint - this can talk to that, but only on ports xyz, etc. We can work out the most efficient and simplest way to do that.
-
I appreciate your offer to help but I'm actually leaving for a longer business trip and I was making sure the network was stable. OP wasn't a problem for my workflow since I don't access FreeNAS across VLANs, it was something I noticed accidentally that I couldn't explain and I wrongly assumed it was a pfsense issue.
I wouldn't have bought the USG if not for the abovementioned lockups. When the SG-2220 "broke" I went for "it just works" and since I already have some Unifi equipment the USG was a logical alternative. Now, the setup isn't optimal but it's better from a wife perspective. I can homelab and break things while netflix and the internet still works for the rest of the house. I'm running about 15 VMs (some of which are internet facing), LACP, proxies, VPNs and I'm more comfortable doing these in pfsense since I've been using it for about 4 years.
-
@netnewb2 said in [SOLVED] Large file transfers between interfaces dropping:
Unifi equipment the USG was a logical alternative.
You would think huh ;) I had gotten the usg for sim reasoning... I had bumped my connection from 100/10 to 500/50 and my pfsense on old N40L as vm could just not do that speed..
I knew I needed something quick that could push that speed and was "cheap".. Until I could figure out the direction I wanted to go. So I had gotten a usg3p for like $100.. And yeah I could get my 500/50 without too much issue.. As long as didn't turn off offload, like if wanted to play with their dpi stuff.. Then it was prob worse than my VM of pfsense..
After just a few minutes with it trying to setup just basic firewall rules, I could tell yeah not going to be using this.. So had to decide did I build a new VM host that could run pfsense and handle my speed - or did I go with actual hardware for pfsense.. Ended up with a sg4860.. And life was good again.. My usg sits there collecting dust - guess its a spare router/firewall... I can not even find a buyer for it for like $75 ;) You want to buy mine? I just turned it on the other day to update its firmware to current.. Which was fun since the current controller couldn't even adopt it with the firmware it had on it, etc.. I keep trying to come up with how I could actually use it.. And just can not come up with anything.. If they would enable it to just be say a monitor for dpi via simple bridge mode then that might be something I could do with - but yeah without some major playing in the cli, doesn't seem possible at all.
-
@johnpoz said in [SOLVED] Large file transfers between interfaces dropping:
to play with their dpi stuff.. Then it was prob worse than my VM of pfsense..
Oh yeah, I didn't expect doing anything advanced with the USG since they have this "Warning: Enabling IDS/IPS will affect the device maximum throughput. USG: 85 Mbps, USG-Pro: 250 Mbps, USG-XG-8: 1 Gbps." . I've only blocked outgoing traffic for some IoT devices and forwarded ports to pfsense.
I'm getting a free sg-2220 and sg-2440 from work soon (they switched to fortigate) and was thinking what to do with them... maybe HA between the 2220s or using just the 2440.
-
@johnpoz I came back to say you were right. I've used 2 gateways and while it worked, it was also tiresome since I had to babysit NAT, multiple firewall rules and gateways. It's also annoying to downgrade to something as basic and featureless as the USG.
I've setup CARP with 2 virtualized pFs about a month ago and I'm pretty happy with the setup and now I'm trying to decide whether to keep pF virtualized or switch to 2 x netgate SG-2220 which I already own. Maybe you can give some advice.
So, I have a "compute" unit with Proxmox, a FreeNAS "storage" unit and an Intel NUC with proxmox which hosts the failover pfsense. "Compute" has a 4 x Intel network card (which are setup in one LAG) and 2 x Intel onboard ports - it's a Supermicro MB. FreeNAS also has 2 ports setup in a LAG.
Most of the traffic happens between the VMs on Proxmox and with FreeNAS. The pFsense VM is using the 4 port LAG. I've assumed that this setup should work faster than a hardware pFsense with only 1 LAN port, especially for traffic between the VMs (pFs is routing between VLANs).
-
@netnewb2 You might not be able to CARP the SG-2220 boxes, since they have only 1 WAN and 1 LAN port. I might be wrong, however...
But, I thought there had to be a dedicated sync interface port between the 2 units for the boxes to keep updated with each other.
Jeff
-
@akuma1x it's recommended to have a dedicated sync interface and it can be done via VLAN as well. That's how I've set it up for now.
edit: strongly recommended
-
@netnewb2 said in [SOLVED] Large file transfers between interfaces dropping:
@akuma1x it's recommended to have a dedicated sync interface and it can be done via VLAN as well. That's how I've set it up for now.
That's what I was going to come back and add, that it might be able to sync over a VLAN. You beat me to it... Thanks!
Jeff
-
@netnewb2 said in [SOLVED] Large file transfers between interfaces dropping:
Problem solved by adding static routes to VLANs that weren't in the same network as FreeNAS. I assume it had something to do with asymmetric routing and with FreeNAS not setting gateways on VLAN.
Currently dealing with a similar scenario myself-- where transferring large files inter-VLAN between FreeNAS and a client crashes the network.
Can you explain the process of "adding static routes to VLANs that weren't in the same network as FreeNAS"? Thanks!
-
@dumdedumda said in [SOLVED] Large file transfers between interfaces dropping:
@netnewb2 said in [SOLVED] Large file transfers between interfaces dropping:
Problem solved by adding static routes to VLANs that weren't in the same network as FreeNAS. I assume it had something to do with asymmetric routing and with FreeNAS not setting gateways on VLAN.
Currently dealing with a similar scenario myself-- where transferring large files inter-VLAN between FreeNAS and a client crashes the network.
Can you explain the process of "adding static routes to VLANs that weren't in the same network as FreeNAS"? Thanks!
Well, it’s been a while since the initial post and right now even I don’t understand what I was trying to do. Looks needlessly complicated.
AFAIR, it has something to do with FreeNAS on multiple VLANs and asymmetric routing. Example:
PC on 192.168.1.5
FreeNAS on 192.168.1.100 and 192.168.100.100PC tries to access FreeNAS on 192.168.100.100. Works initially but after a while FreeNAS will try to respond via 192.168.1.100, as in, from the same VLAN as the PC.
Or another issue when FReenas tries to answer back from 100.100 but doesn’t have a gateway set on that interface. The solution was to add a gateway on 192.168.100.0/24 but that wasn’t an option (in the freenas gui). So I had to set a static route from Freenas something like, 192.168.1.0/24 via 192.168.100.1 (router interface on that VLAN that can talk between VLANs).
Tbh I can’t remember details and since then, I’ve moved on from freenas and pfsense
