Fresh install WebConfigurator hangs



  • Just setup pfSense 2.1.1 on a Supermicro 5018A-FTN4 SuperServer:
    http://www.supermicro.com/products/system/1U/5018/SYS-5018A-FTN4.cfm

    The NICS are I354 which 2.1.1 lists as supported in the changelog.

    Install went without a hitch (although for those with this machine boot from the black USB ports, not the blue ones).

    Terminal config also went fine but when trying to access the WebConfigurator it works for a while but saving any settings causes the WebConfigurator to hang. The box is still functional via Keyboard/Mouse but it requires rebooting to see the admin pages again.

    Any ideas?



  • Just noticed MBUF usage is 100%. The box only has a LAN interface configured at the moment…



  • Hi dogbait,

    have the same problem with the MBUF which is at 100%.
    And the other thing is that igb1 does not come up with the message:
      "igb1: Could not setup receive structures"

    I already had a look on a BSD-forum entry from 2010 (four years old)
      http://lists.freebsd.org/pipermail/freebsd-stable/2010-October/059534.html

    which says that I could modify the value for num_queues in if_igb.c which is really weired.

    Is there no solution for this? This makes the whole thing really unusable!

    PLEASE HELP me!

    Thanx!



  • The fastest solution for me was to enter the BIOS and
      LIMIT the CPU kernels to 4!

    WTH is this? It works fine now …
    leaving me with 4 unused cores!


  • Moderator

    Hi Hans,

    as you have igb interfaces, can you try this:

    login to pfSense with SSH and edit the file (via vi) "/boot/loader.conf.local" (or open the file via Diagnostic menu). Then give it the following content:

    
    kern.ipc.nmbclusters="655356"
    machdep.hyperthreading_allowed="1"
    hw.igb.num_queues="4"
    
    

    and try to reboot. Does it help?


  • Netgate Administrator


  • Moderator

    Because they work for us. ;)

    We had initial issues even installing pfSense 2.0.x on our brand new ibm hardware servers and intel/broadcast nics. We couldn't even get it to install nor to recognize the nics in the system because the cpu (8cores) had HT and the number of cores available to the system drew the mbufs into crazyness :)

    Those recommendations came from an older redmine bug. See: https://redmine.pfsense.org/issues/1221#note-8

    Greets


  • Netgate Administrator

    Ah, yes. However note that in the bug report that setting was tested with a uniprocessor kernel and hyperthreading disabled. The current kernels are all SMP.
    If it works for you though then that's really what matters. So you tested these settings with 16 logical CPUs?

    Steve



  • OK…so what's the consensus. Our pfSense box has 8 CPU cores, one onboard i354 Intel Network Controller with 4 ports and one PCIe Intel VT Quad Network Controller with 4 ports...

    Do we disable 4 cores AND set hw.igb.num_queues to 4 or 1?


  • Netgate Administrator

    You shouldn't need to disable any cores though 8 cores probably won't give you much advantage over 4 in 2.1.X. The new pf in 2.2 will use multiple cores much better.
    I would go with JeGr's suggested settings since they have been tested to work with similar cores/ports.

    Steve


  • Moderator

    As we re-used some similar hardware, we switched two cpus to other cluster members. So ATM our pair of firewalls are only running a single cpu with 4 cores and HT. So 8 visible to the system at the moment (as PF isn't using SMB now but hopefully will scale much better with FreeBSD 10.x underneath).  But our settings are still the same.

    So yes we tested originally with 16 visible cores, now have 8 and the system is running good so far. MBUF usage is about 5% (31638/655356) and all of the 8 igb seem fine wether they run single (as igbX interface) or in a LAGG group.

    Greets
    Jens



  • Thanks guys, appreciate the info.

    Just to add, I reduced the available cores in the BIOS from 8 to 4 and the MBUF utilization went from 100% to 50%.