Building 10G Pfsense router with SFP module - recommended modules and setup



  • Hello !

    As this is my 1st post in this forum - please excuse me if i entered litte info for my question or if i am missing something :)

    HARDWARE:

    DELL Power Edge T20 - / 16GB ram ; Xeon 1230v2 3.3 Ghz 4 core 8 hyperthreading /
    HDD - 1TB Western Digital WD1003FBYX - Black edition ( which is for storage mainly but i'm sacrificing it for this purpose )
    I will specify the motherboard later from now on.

    fiber module i thought of is : AOC-STGM-12s

    SOFTWARE:

    Pfsense 2.1.3 , Captive Portal with traffic shaper and local authentication of Pfsense 2.1.3 (custom page with "Continue" only button)

    SWITCHING:

    The server will be connected trough 1 of it's ports to a SFP module on a Cisco SG-500x24p switch .

    ALL CONNECTIONS WILL BE 10G!

    Little info on usage of the network below:

    Estimated client capacity : 2000
    Type of network clients : Wireless
    Shaper set in CaptivePortal : 2Mbps

    So i have two question's :

    1st - Do you think that with that setup i can achieve speed of 3 Gbps (the overall speed of the  WAN will be 5Gbps )

    2nd - Can you please recommend me some compatible fiber module's to do this setup or any kind of advices

    Thanks in advance !

    P.S.
    I'm using Pfsense since  ver. 1.2.3  as my primer router distro for my network and i'm very pleased with it for all this time :)



  • It might work and it might not.  The hardware is similar to mine, just a bin slower on the CPU.  I'm seeing a pretty hard wall at 2.2Gbit/s but that's without the captive portal.  The upgrade to pfSense 2.2 should increase throughput as we'll have multithreaded pf and other enhancements.

    At least one user here has seen higher throughput when running vSphere on the bare box and running pfSense in a VM on top.  You might give that a try.



  • We use similar hardware/scheme and we do't have problems with speeds ( 40% LA about 3Gb/s )
    I think you must use fast RAM and CPU for heavy CP usage.
    Maybe raid 1 with fastest SSD disks is better :)
    You may get more users per one router if disable NAT and use public IPs for clients.
    Do not use SMB switches,go to managed cheap Dell Powerconnect 5448  / Force10 !

    My HW is :
    CPU  -  E3 1240
    RAM -  4x4GB  1333 MHz Kingston
    MB  -  Supermicro X9SCI-LN4F
    AOC-STGM-12s - 2x10 Gb/s uplinks
    2x10Gb/s uplinks
    4x1Gb/s in lacp lagg downlinks
    Switch - Juniper  EX2200-48T-4G  ( second router use EX4200 )



  • Hello and thank you for your replies :)

    to:Jason Litka

    If i understand correctly , your concern is about me using the "Captive Portal" function which in your opinion could slow the overall throughput of the server ?

    If that is the case , what advice can you give me so i can achieve my goal ?

    Does the "Captive Portal" slow's down the server because of the many users trying to authenticate/re-authenticate or …. (think)

    to:savago

    I was wondering about using SSD disks :)
    What is your concern about using the SG-500X switch ( i checked the throughput of the switch you suggested )
    I will have no more than 10 ports connected which is almost a half of the ports i could use .

    EDIT:

    I red about pf's problem with Multicore processors - is this the reason you are reffering ? Jason Litka



  • @hybrid_867:

    Hello and thank you for your replies :)

    to:Jason Litka

    If i understand correctly , your concern is about me using the "Captive Portal" function which in your opinion could slow the overall throughput of the server ?

    If that is the case , what advice can you give me so i can achieve my goal ?

    Does the "Captive Portal" slow's down the server because of the many users trying to authenticate/re-authenticate or …. (think)

    to:savago

    I was wondering about using SSD disks :)
    What is your concern about using the SG-500X switch ( i checked the throughput of the switch you suggested )
    I will have no more than 10 ports connected which is almost a half of the ports i could use .

    EDIT:

    I red about pf's problem with Multicore processors - is this the reason you are reffering ? Jason Litka

    Captive portal adds additional load to the system.  How much?  I don't know, I don't use it.  You might want to ask a question about portal performance in the dedicated section.

    SSDs are fine, though unless you're using Squid, they're not going to do much aside from, if you use a decent one, enhance reliability by eliminating a movable part.

    I'm guessing the comment about your choice of switch was relating to the intended usage of that hardware.  You've selected a client access switch with 10Gbe uplinks, not a 10Gbe switch, and you've only picked one of them.  If reliability is critical then you should be looking at a pair of stackable switches with a LAG group across them so that if one switch fails your entire network doesn't.

    In FreeBSD 8.3 (which pfSense 2.1 is based on) pf is single-threaded.  This can cause bottlenecks on high-throughput systems.  In FreeBSD 10 (which pfSense 2.2 is based on) pf is multi-threaded and should eliminate the filtering bottleneck. That is not to say though that other services might not also hold you back.



  • Hello !
    I am glad to tell you that with my setup i achieved everything i wanted :)

    The only problem with the setup is indeed the "Captive Portal" function .
    When we were having about 1800 clients connected to the network , we had problems with pre-authentication URL redirection - it passes (the Auth) but you get a timeout error on some clients , but when you try again to open some website everything works OK .

    Thank you all for your help - it was needed and highly appreciated :)