Intervlan performance slow on my C2758 atom 8 core.
-
About 7GB. Within same VLAN ID, the switch will handle the transfer and not send the packets to the trunk LAGG on pfSense. Switch again is L2 and not routing, pfSense is doing routing between VLANs.
Why should a 7 GB file running through the firewall? And then you might be thinking about the performance
or think the LAG is miss matching? I don´t think so.Yup everything is setup except PowerD, 1000000 mbufs and trim enabled SSD.
Ok that would be fine then.
I'm leaning towards the LAGG setup, maybe LACP isn't good but had no issues prior on my old setup with zeroshell. no 10GbE on my switch, just a base catalyst 2960x model.
If have only a smaller and very cheap switch with 2 SFP ports one is connected to the NAS and and one to a
server and the pfSense firewall will be "only" connected to a 1 GBit/s port, but must on the other side also
and only routing the WAN - LAN traffic and the Switch is doing the entire LAN routing. If the firewall fails
at some time the entire LAN traffic will flow without a break.The hardware is exactly the same as I linked above with several improvements which shouldn't
impact NIC performance between VLANs.OK
What exactly are they doing that is fine tuning that everyone else can't get?
Because they know the hardware that is coming with the pre-installed version of pfSense
and so they can do some tunings that matches exactly this hardware, to unleash the full
power the hardware. And yes it is the same version like we both are using, but with some
tunings because if they sell the hardware they know what is exactly sold. By the community
version for everyone, no tuning can be done, because the developers are not knowing what
kind of hardware we are all using or we will use!Settings should be able to use for anyone.
Yes, for sure they are, but I really don´t thing that we all have so much wisdom and deeper knowledge
about pfSense as the developers will own! And if they know the hardware because they are selling it self
they can do some more things as we will be able to do. Or how many about such things you will know?This is open source right?
Yes OpenSource for sure, but if you are offering the software only without knowing what kind of hardware
will be in usage at the endpoint or on the customer site, what you will tune of pre-tune? But if you are selling
the hardware and the software together and also pre-installed, you will be exactly knowing the hardware basis
and would be able to pre-install and tune the absolutely identically pfSense community version that we are all
using, but with the deeper knowledge from the developer site that we all never will have. Not more but also
not less.But back to your problem, what kind of settings you where using in ZeroShell?
Are these the same one like now? And again a LAG is more for the use case that
many clients will connect to one other device likes your NAS. Because they will
be able to render one line completely and the next one will be in usage then. -
HiAdaptive did not do anything :(
Zeroshell was the same as what pfSense is doing, LACP with 6 ports.
I had no issues with zeroshell, ran about 850mbps throughput. And yes LAG is primarily for multiple users connected to my NAS to transfer files.
-
I think you'd be wise to do a wireshark capture of an FTP session to look for things like retransmissions or tcp zero windows. You might be able to tweak your systems' tcp parameters to get better throughput.
Ordinarily, I recommend against using jumbo frames on gigabit (and even on 10gb except for iSCSI), but in your case reducing the number of packets that pfsense has to look at might boost your performance.
Lastly, you might want to consider installing ESX or HyperV (ESX probably wouldn't have drivers for your supermicro NIC) and use pfsense for firewall, and something like zeroshell for intervlan routing.
Or, buy a layer three switch.
-
HiAdaptive did not do anything
Oh really sad, in normal it does the following, if the machine gets stressed it uses the full 2,4GHz and
if less power is used it saves electric power by running the CPU only a sometimes like 60MHz or 800MHz
like it is needed, and so if this is not enabled it can be that the cpu frequency is only and static running
at 600MHz or 800MHz and this will then not really unleash or delivers the performance and on top the
needed throughput, that you will need from time to time!Zeroshell was the same as what pfSense is doing, LACP with 6 ports.
Did you remember the settings like "active - active" or anything else, that you are not really
using or configuring this time together with pfSense? -
@BlueKobold:
HiAdaptive did not do anything
Oh really sad, in normal it does the following, if the machine gets stressed it uses the full 2,4GHz and
if less power is used it saves electric power by running the CPU only a sometimes like 60MHz or 800MHz
like it is needed, and so if this is not enabled it can be that the cpu frequency is only and static running
at 600MHz or 800MHz and this will then not really unleash or delivers the performance and on top the
needed throughput, that you will need from time to time!Zeroshell was the same as what pfSense is doing, LACP with 6 ports.
Did you remember the settings like "active - active" or anything else, that you are not really
using or configuring this time together with pfSense?It was just configuring in the interfaces file like any linux distro.
auto bond0 iface bond0 inet static address 192.168.1.10 gateway 192.168.1.1 netmask 255.255.255.0 bond-mode 4 bond-miimon 100 bond-slaves none
Something like that, there's nothing really to state active or passive.
I do have a layer 3 switch., the Cisco C2960X-48TS-L is not a full L3 switch but has routing capabilities and ACL.
I just don't know if I can define all the same rules in the switch and also allow certain hosts/networks outbound on pfSense to different gateways (OpenVPN clients)
From what I've read, you're supposed to use a transit network from the switch to pfsense so pfsense doesn't really know the internal vlans of the switch. In this case I don't think I can selectively route traffic outbound to different OpenVPN gateways.
-
From what I've read, you're supposed to use a transit network from the switch to pfsense so pfsense doesn't really know the internal vlans of the switch. In this case I don't think I can selectively route traffic outbound to different OpenVPN gateways.
You will be able to create a VLAN50 as an example and the Gateway of this VLAN50 will be then the IP address
from the pfSense box! So you could set up routes to any other VLANs and all would be fine. Thats it.Ok perhaps you wont to walk on this way but it is a really fine solution to get all LAN traffic fast routet
nearly wire speed pending on the power of your switch and the entire LAN will be also alive if the pfSense
box gets rebooted or is failing. -
Your C2960X-48TS-L is not a layer 3 switch, it runs the LANBase feature set. No routing possible.
If your switch is a C2960XR switch, then you are in tall cotton - by all means use it for your inter-VLAN routing and inter-VLAN access control lists.
Set up your VLANs on your switch and use it to route between them. Create a private IP network between the switch and the pfsense box and make the switch's default route the IP address of pfsense.
I don't think the 2960XR will originate any routing protocols, so you'll have to create routes on the pfsense box that route your VLAN subnets back to the switch.
You will be much happier with this setup.
-
I wish the netgate guys would chime in on threads like this.
-
Sorry didn't reply back for a while on this thread.
I think I've figured out the issue, maybe not, who knows.
But basically the LAGG algorithm is sending/receiving the file transfer on the same port on pfSense, so it's doing full duplex transfer. Now theoretically, the gigabit ethernet can handle 2000mbps total. But I ran iperf between 2 machines using the simultaneous option, and the max I was about to get was about 450mbps both ways the same time. So not sure why? Anyhow when I transfer a file the other direction, the algorithm uses 2 ports on pfSense, so then I'm getting closer to 1Gb in that direction.
Either way, I think I will upgrade to 10Gbe with the Chelsio card, that should solve any Gb bottlenecks.
-
But basically the LAGG algorithm is sending/receiving the file transfer on the same port on pfSense, so it's doing full duplex transfer.
I am not really sure but all depends on the configuration you made! You can also configure that one
LAN port is "doing" RX and the other is "doing" the TX part! And then you will be getting out;- 1 GBit/s > TX
- 1 GBit/s > RX
And this might be then even 1 GBit/s and not 2 GBit/s! But for sure the entire LAG (LACP) is building
a aggregated 2 GBit/s fat pipe!Now theoretically, the gigabit ethernet can handle 2000mbps total.
That is the exactly point where you are failing or made a so called thinking false in my eyes!
1 GBit/s line (cable) is able to send and receive 1 GBit/s over 4 adders of the cable in each direction
and this is then 1 GBit/s in each direction and not 2 GBit/s in one direction.But I ran iperf between 2 machines using the simultaneous option, and the max I was about to get was about 450mbps both ways the same time. So not sure why?
If the technical and theoretical max throughput of a 1 GBit/s line is 125 MBit/s and with your LAG (LACP)
you will get out then in normal and as a max. 500 MBit/s (4 x 125 MBit/s) but you got 450 MBit/s + the
TCP/IP overhead that must be count on this on top you will be getting also nearly the macimum, or am I
wrong with this?Anyhow when I transfer a file the other direction, the algorithm uses 2 ports on pfSense, so then I'm getting closer to 1Gb in that direction.
Then perhaps the network load you were producing with iPerf was not high enough perhaps I mean?
Either way, I think I will upgrade to 10Gbe with the Chelsio card, that should solve any Gb bottlenecks.
It is the best option as today in my eyes!!! The Chelsio card is fully offloading tasks such as VLANs based
on using an ASIC/FPGA on its NIC and it is better driver supported in pfSense! So you will be able to
fully unload from your pfSense box many TCP/IP based tasks and on top you will saving ports and
getting more throughput then now.