3 Wan load Balance - Large throughput needed - what hardware?

  • I have setup a 3 WAN Load Balancer with pfSense.
    The firewall and such are wide open and I'm just using it to balance traffic only.
    Each WAN link is 7MB dl / 1MB ul

    I originally had this setup on:

    AMD 1.4GHz
    Tyan S2390 mobo
    768MB ECC SDRam 133MHz
    4 x 10/100 NICs
    older 10GB hard drive

    This setup proved to be lacking for the bandwidth, so I changed it out to this:

    AMD Athlon XP 2400+
    MSI KT4 Ultra mobo
    512MB DDR 333MHz
    4 x 10/100 NICs (same cards as above)
    40GB Maxtor 5400rpm hard drive

    This seems to handle to bandwidth ok for now with good throughput and ~25% proc usage.

    I need to know if pfSense supports AMD 64 and/or X2, Intel Core Duo, DDR2, etc.  I will be adding more lines and more customers as time goes on and will need pfSense to accomodate this.

    How much beefiness can I run with pfSense and still be increasing performance?  At what point does the proc, mobo, and ram no longer increase performance?  Obviously I will need to change the NICs out to Intel PRO or something (from what I've been reading).

    Does anyone know anything about how this card would perform?  www.titanwirelessonline.com Router Board 44G?  Is is a 4 port 10/100/1000 NIC which recognizes as 4 independent ethernet ports.

    I would like to setup QoS for raising things like VoiP, but am told that this doesn't work well when load balancing.
    Can the Traffic Shaper be setup using a pool instead of an interface?  Any ideas?

    I'd also like to speed limit per IP address, but again am told this doesn't work well with load balancing.
    I don't mind too much as I'll just setup a Linux box to speed limit download and upload.

    Any other suggestions for more throughput capabilities?

  • what do you mean with "Large throughput needed"?
    are you talking about Gbit speed?
    if yesi think you will soon reach the maximum availlable bandwith of your PCI bus.
    you could move on to a Server-Class Mainboard with PCI 64


    Bus		|   Width (bits) | Bus Speed (MHz) | Bus Bandwidth MBytes/sec  | Mbit/s
    PCI		|   32		 |   33		   |		      127.2    | 1017.6
    64-bit PCI 2.1	|   64		 |   66		   |		      508.6    | 4068.8

  • PCI express should be able to move approx 250 MB/s, per lane. So a PCIe 4x card would be twice as fast as PCI64 (In theory). I would be looking at multiple GB cards off a PCIe bus.

    edit: Of course, I can't find any PCIe 4x cards on the market, all those available look to be 1x.
    I'll have to try and see what the motherboard embedded chips run at… Anyway, the 1x cards should  be twice as fast as PCI. I thought PCI 64 was twice PCI, approx. 266 MB/s (or 254.4, if you please), perhaps there are several flavors of 64 bit PCI... Anyway, running 10/100 PCI NICs is going to be a bottleneck and you should look at a faster bus...

    edit: Actually, Intel does have PCIe 4x cards available.

  • As stated earlier, PCIe has a dedicated bandwidth of 250MB/s per lane. Therefore, a PCI-Express (PCIe) 1x lane slot would be adequate for a single port or even dual port Gigabit NIC. you should only need a 4x card if you are getting a NIC card that has more than 2 interfaces (i.e. a 4 port card).

    Good, modern mainboards use onboard NIC chips that make use of PCIe – the onboard Ethernet if connected via PCIe would be just as good as an add in board. Verify the chip used is connected vis PCIe and not the (old, parallel) PCI. Google the chip name, e.g. Intel 82573L and look at the datasheet.

    Note that for the Original Poster, throughput in the <100Megabit range would be just fine on a (parallel) PCI slot -- it's only Gigabit that needs 64 bit PCI, PCIe or PCI-X to not lose some theoretical capacity.

    The original poster seems to have CPU constraints vs LAN. Using good NICs will decrease the CPU load. The CPU usage should scale through -- if you have 25% CPU usage today, you can have about 3X more throughput before you hit a CPU wall. Obviously changing the load via other means (e.g. using QoS) will change this.

  • Thanks for the insight.  Any other comments about the 64 bit and the dual core stuff etc?

    In case anyone cares, I did setup a Linux box (2.6 kernel) to handle the speed limiting per IP address.  It doesn't seem to have as much demand on CPU and RAM etc.  This combonation works quite well.

    Thanks pfSense.  I'll keep testing to see if this will handle my needs.  If so, expect a donation.

Log in to reply