X11SBA-LN4F vs A1SRi-2558F



  • Hint: Read the contents of the NIC-EEproms

    My Guess: Changing of ASPM-Parameters (I still can't fathom who invented that atrocity and also who decided to enable it on servers boards :))

    I have/had 3 of X11SBA-LN4F in production and they decided to fail this weekend.  :o

    Now testing: ASPM Disabled, MSI-X Disabled.



  • @Engineer:

    The board, since repair (modification?), has been running for 120 days now.  No issues.

    Much, much better.

    @OLBaID - the watchdog timeouts are a hardware issue.  SM made an unknown modification on my board to eliminate the watchdog timeouts.  Seems related to the PCIe switching chip (the first Ethernet port is attached directly to the PCIe bus of the N3700 while the other ports go through a port switching chip.  Those three ports all have watchdog timeout issues).

    There's quite a bit of information in this thread including the contact (Ken Huang IIRC) that has experience in this issue.

    Engineer

    Thanks so much i actually did reference this post and have talked to Ken and several others at SM and the board is back for RMA already. I also mentioned the eeprom updated listed in IDean's post.

    I will update this thread once it is returned to me but please note this as well:

    https://redmine.pfsense.org/issues/6296

    as it is relevant (same issue) but with many hardware configurations.

    Appreciate everyone whom added to this thread, the community is great!



  • @OLBaID:

    but please note this as well:

    https://redmine.pfsense.org/issues/6296

    as it is relevant (same issue) but with many hardware configurations.

    Appreciate everyone whom added to this thread, the community is great!

    That's something new with 2.3 it seems.  I'm running 2.2.5 (had the watchdog issues with 2.2.4 and 2.2.5 with the board before repair).  I tried as many option items as possible and even figured out how to compile Intel's latest driver for the I210 chip.  It ran but with the same watchdog timeouts on ports 2,3 and 4 (1, which is directly to the N3700 PCIe lane, always worked just fine).

    Keep us updated and good luck!



  • @Engineer:

    The board, since repair (modification?), has been running for 120 days now.  No issues.

    Much, much better.

    @OLBaID - the watchdog timeouts are a hardware issue.  SM made an unknown modification on my board to eliminate the watchdog timeouts.  Seems related to the PCIe switching chip (the first Ethernet port is attached directly to the PCIe bus of the N3700 while the other ports go through a port switching chip.  Those three ports all have watchdog timeout issues).

    There's quite a bit of information in this thread including the contact (Ken Huang IIRC) that has experience in this issue.

    I posted this information on another thread, but I thought putting it here
    might save somebody some time.

    Engineer seems to have figured out thought a lot of hard work that  an RMA
    to encorporate ECO 18137  is what is required to make the
    Supermicro X11SBA-LN4F-O N3700 stable.

    AND

    From what I understand from reading the form, it seems to be very low power, is
    easy to keep cool, and has decent performance for it's class.

    This motivated me to  follow up with SM Tech Support, and I got
    the following back:

    –----------------------------------------------------------------------------------------

    -------- Forwarded Message --------
    Subject:    RE: X11SBA-LN4F-O - Pre-Sales Enquiry [WT]
    Date:    Tue, 10 May 2016 00:02:16 +0000
    From:    Technical Support support@supermicro.comTo:    –---@---.ca>, Technical Support support@supermicro.comHello

    After further investigation,  the ECOs has been implemented onto
    PCB 1.02 for the aforementioned issues.  When you place an order
    with your distributor, please ensure to specify a PCB 1.02 to be
    shipped to you.

    Regards,

    Technical Support


    If I understand what I've read, the X11SBA-LN4F-O N3700 PCB 1.02
    should make a decent pfSense platform – or am I missing something?

    Clearly it's no A1SRi-2558F, but in Canada the difference between the
    two boards is $141 CDN based on the best prices I could find today.
    Unless broadband costs drop a lot, I can't see outgrowing it for 4 or 5
    years (Minimum), and by that time I'll likely have a cap dry out and have
    to replace whatever I buy anyway, so I might as well put the $141 toward
    a case and memory, or am I missing something?/support@supermicro.com/support@supermicro.com



  • @guardian:

    If I understand what I've read, the X11SBA-LN4F-O N3700 PCB 1.02
    should make a decent pfSense platform – or am I missing something?

    Clearly it's no A1SRi-2558F, but in Canada the difference between the
    two boards is $141 CDN based on the best prices I could find today.
    Unless broadband costs drop a lot, I can't see outgrowing it for 4 or 5
    years (Minimum), and by that time I'll likely have a cap dry out and have
    to replace whatever I buy anyway, so I might as well put the $141 toward
    a case and memory, or am I missing something?

    First, thanks for the update on PCB 1.02.  No, I don't think you're missing a thing.  Thing has been rock solid since the change (running 2.2.5).  Very low power and excellent IPMI setup (in my opinion).  More processing power than I need right now and hope it lasts for a long time.  10-11 watts for entire system @ wall with no fans:

    SM board
    Antec ITX-110 with 90W supply
    2 x 4GB DDRL DDR3 modules
    120GB Sandisk Plus SSD

    Quite happy with mine as of right now.  TWC is supposed to bumping speeds soon so I'll give it a whirl once that happens and report back.



  • @Engineer:

    2 x 4GB DDRL DDR3 modules

    Thanks for the input.

    Can you please give me a part number for RAM modules?

    Also, what is you CPU temp like?



  • @guardian:

    @Engineer:

    2 x 4GB DDRL DDR3 modules

    Thanks for the input.

    Can you please give me a part number for RAM modules?

    Also, what is you CPU temp like?

    Supermicro is selling this board together with a case from them too! This combination is then called superserver,
    and can be often bought ready assembled, only the RAM and SSD will be needed on top of this.
    Supermicro SuperServer E200-9B Server SYS-E200-9B



  • @guardian:

    @Engineer:

    2 x 4GB DDRL DDR3 modules

    Thanks for the input.

    Can you please give me a part number for RAM modules?

    Also, what is you CPU temp like?

    CPU temps run from 45C to 52C generally (the four cores).  I'll dig up the ram soon.  It's Samsung brand, IIRC.



  • "Supermicro is selling this board together with a case from them too! This combination is then called superserver,
    and can be often bought ready assembled, only the RAM and SSD will be needed on top of this.
    Supermicro SuperServer E200-9B Server SYS-E200-9B"

    This is the system i purchased, on the asset label with the serial and MAC addresses, was revision 1.00. I am going to request them provide me any updates done to my RMA and or new revisions if exchanged so you can know the proper revision without the issue. FYI I just purchased 1333 Crucial 4GB modules for this setup and a Kingston 120gb SSD that worked with no issue. (This system does leverage the Pericom 608GP controller mentioned previously in this thread).

    This system also has small 1 small case fan (just a heatsink for the cpu) included and hovered around 38c with pfsense.



  • For anyone interested, I monitored the CPU load while downloading at my new ISP speed (59 Mbps) and it never topped 9%.  That's with PowerD on.  For some reason, the CPU load shows higher with PowerD turned on than off (I'll test it off later).  The CPU hangs between 3% and 5% when idle so added 4% @ 59Mbps download.  Temperature was at 55C for one core and 48-49 for the other three cores.



  • @Engineer:

    For some reason, the CPU load shows higher with PowerD turned on than off (I'll test it off later).

    Because the cores are clocked down by PowerD when the load is low, hence, the percentage figure is higher - i.e. If the load needs 200MHz per core avg, then at 2GHz that's 10% but when the cores are clocked down to 1GHz, the load will show as 20%.



  • Running a A1SRi-2758f passively in the Supermicro CSE-504-203B chassis without any chassis fans.

    System idles at around 56'C (in ~28'C environment) and hasn't overheated till now (running SNORT on a 100M/100M line). I might consider adding a chassis fan once I get my servers and storage in - when the unit will have to be routing multi-gigabit inter-vlans.

    I'm still waiting for my custom patch cord order from AMP and my new APC Netshelter (Dell forgot to order the rack). So far, I just have the bare minimum connected to run the office.



  • There are cases at supermicro offering front-located connection ports, fitting these motherboards.  8)



  • @robi:

    There are cases at supermicro offering front-located connection ports, fitting these motherboards.  8)

    Yes, I just didn't want them because it looks ugly in the rack. This would be part of a rack and stack showcase so it has to look nice. My Netshelter will be arriving in a day's time - Dell decided to expedit my order after screwing up.

    On hindsight, I should have gone with the A1SRM-2758F and the SC510-203B. Would have had enough space to stick a sticker from the pfSense merchandise store on that chassis.


  • LAYER 8 Netgate

    Umm. I guess beauty is in the eye of the beholder.



  • Just wanted to reply to the thread, got my RMA back and updated and have not had the issue for over 24 hours. wanted to share the notes so anyone that has a supermicro has some help if needed. Thanks again for the contacts and help in this thread.

    https://dl.dropboxusercontent.com/u/42296/SuperMicro%20RMA%20notes.PDF

    And just to point to the solution that was most likely the issue:

    https://redmine.pfsense.org/issues/6296



  • @Engineer:

    @guardian:

    If I understand what I've read, the X11SBA-LN4F-O N3700 PCB 1.02
    should make a decent pfSense platform – or am I missing something?

    Clearly it's no A1SRi-2558F, but in Canada the difference between the
    two boards is $141 CDN based on the best prices I could find today.
    Unless broadband costs drop a lot, I can't see outgrowing it for 4 or 5
    years (Minimum), and by that time I'll likely have a cap dry out and have
    to replace whatever I buy anyway, so I might as well put the $141 toward
    a case and memory, or am I missing something?

    First, thanks for the update on PCB 1.02.  No, I don't think you're missing a thing.  Thing has been rock solid since the change (running 2.2.5).  Very low power and excellent IPMI setup (in my opinion).  More processing power than I need right now and hope it lasts for a long time.  10-11 watts for entire system @ wall with no fans:

    SM board
    Antec ITX-110 with 90W supply
    2 x 4GB DDRL DDR3 modules
    120GB Sandisk Plus SSD

    Quite happy with mine as of right now.  TWC is supposed to bumping speeds soon so I'll give it a whirl once that happens and report back.

    Hi Engineer & All..

    Just wondering if you have any updates, or have things still been working?

    BTW: What throughput are you getting/What is your connection?



  • @guardian:

    @Engineer:

    @guardian:

    If I understand what I've read, the X11SBA-LN4F-O N3700 PCB 1.02
    should make a decent pfSense platform – or am I missing something?

    Clearly it's no A1SRi-2558F, but in Canada the difference between the
    two boards is $141 CDN based on the best prices I could find today.
    Unless broadband costs drop a lot, I can't see outgrowing it for 4 or 5
    years (Minimum), and by that time I'll likely have a cap dry out and have
    to replace whatever I buy anyway, so I might as well put the $141 toward
    a case and memory, or am I missing something?

    First, thanks for the update on PCB 1.02.  No, I don't think you're missing a thing.  Thing has been rock solid since the change (running 2.2.5).  Very low power and excellent IPMI setup (in my opinion).  More processing power than I need right now and hope it lasts for a long time.  10-11 watts for entire system @ wall with no fans:

    SM board
    Antec ITX-110 with 90W supply
    2 x 4GB DDRL DDR3 modules
    120GB Sandisk Plus SSD

    Quite happy with mine as of right now.  TWC is supposed to bumping speeds soon so I'll give it a whirl once that happens and report back.

    Hi Engineer & All..

    Just wondering if you have any updates, or have things still been working?

    BTW: What throughput are you getting/What is your connection?

    Been up 150+ days with no issues.  My connection is only 60Mbps down / 5Mbps up.  No issues at all.  Max 9% CPU but that's with PowerD turned on, which was explained to me above to show a higher percentage because the CPU is running at a lower frequency.  Once frequency jumps, the max CPU % should drop (which is why I a lower CPU % when PowerD is turned off).



  • I'm wondering if anyone is running this rig with anything faster?

    I'm going to be going to Rogers 250/20… plan on doing a lot of filtering/running Snort or something similar... still have to figure that out....

    I hope the X11SBA-LN4F will do the job.



  • Here are a couple of reviews that I though the community might appreciate seeing.

    They provide some good overall info on the design.

    SUPERMICRO X11SBA-LN4F REVIEW – SWEET!
    http://www.servethehome.com/supermicro-x11sba-ln4f-review/

    SUPERMICRO A1SRI-2758F REVIEW – HELLO RANGELEY
    http://www.servethehome.com/supermicro-a1sri-2758f-review-rangeley/



  • Just to give some more feedback: I've been running pfSense on the Supermicro board (rev. 1.02) for a few months now and it has been rock solid. It can easily handle the 150Mbps that I get from Comcast without breaking a sweat (including some Snort rules). Temperatures are usually in the 50-55 degree Celsius range (at Californian summer room temperatures  ;)), using an Antec ISK110 case. I'm using two 4GB SO-DIMMs and an mSATA SSD on the board.

    Love the 4 Intel NICs and the IPMI interface (tip: you can access the BIOS over SSH via SOL by logging into IPMI and running the commands "cd system1/sol1; start" in the SMASH CLI). I have not been able to redirect the pfSense serial console though, since SOL uses COM2 and pfSense seems to only support COM1 for console output.



  • Where and when did you purchase your board?



  • @Jailer:

    Where and when did you purchase your board?

    I bought it from Newegg, I think in February or March.



  • My repaired board has been up 192 days with no issues now! :)



  • @enodeb:

    @Jailer:

    Where and when did you purchase your board?

    I bought it from Newegg, I think in February or March.

    Did it work out-of-the-box for you? I'm currently looking at deploying this board in home lab/home office environment and was about to pull the trigger until I came across this thread. If I can avoid RMAing just to get it to work for pfSense I would be so happy.



  • @nomofica:

    Did it work out-of-the-box for you? I'm currently looking at deploying this board in home lab/home office environment and was about to pull the trigger until I came across this thread. If I can avoid RMAing just to get it to work for pfSense I would be so happy.

    If it helps; I bought an E200-9B from superbiiz.com last week. I received rev 1.02 hardware and haven't had any issues with it so far.



  • Hi

    @mr2dave:

    If it helps; I bought an E200-9B from superbiiz.com last week. I received rev 1.02 hardware and haven't had any issues with it so far.

    I ordered this from a UK supplier which ordered it, I was told, from Supermicro as required.  My revision was 1.01.  I emailed Supermicro tech support and was told the board had been revised with the following:

    1. Solve only 2 LAN show up to BIOS menu for LN4F SKU
    2. Solve LAN EEPROM show up PXE-E05 error message issue
    3. Add rework instruction

    I think the first are likely a BIOS update only as the board came with the latest version, and the rework instruction is the fix applied.

    You can check for this rework yourself on a version 1.01 of the board, it is pretty obvious once you have located the component which is U35, to see it has been changed and hand soldered, see the attached image taken from Newegg's detailed images of the board.  My rework looked a bit neater but it was still obvious looking at the solder and comparing to other components it had been reworked.

    Hope that helps anyone else purchasing one.

    Regards

    Phil




  • Holy crab, don't they have someone who can actually solder SMD devices?
    If one of my guys would call such a job complete he'd have a hard time…



  • Hi

    Just seen on the X11SBA-LN4F that despite having 1600 DDR3L memory, which is reported in the BIOS at 1600MHz, the memory is reported as actually running at 1066MHz, this is seen in the IPMI system hardware list and if doing a #dmidecode in pfSense shell and viewing the memory info.

    Memory is Crucial CT51264BF160B 2 x 4GB.  Everything is running good but it's a bit annoying if memory is clocked slower than it needs to be.  Unfortunately there are no manual memory settings in the BIOS to force anything to run any differently.  It could just be a reporting error from the board perhaps.

    Anyone else seeing this.

    Regards

    Phil



  • @Phil_D,

    Mine is the same.  I'm not going into BIOS right now to snoop (it's running OK and has a 250 day uptime, lol).  Should it ever need a reboot, I'll investigate further.



  • Hi

    @Engineer:

    @Phil_D,

    Mine is the same.  I'm not going into BIOS right now to snoop (it's running OK and has a 250 day uptime, lol).  Should it ever need a reboot, I'll investigate further.

    Many thanks for checking.  The BIOS will probably report 1600MHz as mine does, so that is either the operating speed and the reported 1066MHz is wrong, or the BIOS is just reporting what the memory is saying it will support and is clocked slower at 1066MHz.

    I agree overall it probably doesn't make too much difference to performance, but annoying to not have it running at it's designed speed.  I suspect Supermicro only support 1600MHz on it's tested memory, which for this board is limited to just one option which I couldn't find available to buy.

    I will drop their tech support an email.

    Regards

    Phil



  • Hi

    Supermicro support have said that only memory they have tested will run at 1600MHz on this board, it seems anything else defaults to 1066MHz with no way of overriding that in the BIOS.  The number of tested memory SODIMMs that will run at 1600MHz is just one, and good luck sourcing it.

    Regards

    Phil



  • Thank you for reporting this.
    The manual though does say in general that 1600 Mhz memory clocks at 1600 Mhz and that only 1333 Mhz memory clocks down to 1066 Mhz. Nothing about tested memory. And I even bought Class 9 memory.




  • Hi

    Exactly, there is nothing about tested memory only being supported in the specifications, which I have raised them.

    Is your memory running at 1600MHz or also reported as running at 1066MHz?

    Their tested memory is rated at CL=11, which is the same as the memory I'm using, so it's not as though their tested memory is anything special, and the Intel CPU/SoC should be fine supporting anything with the correct rating.

    Of course it could be their support is being lazy and just saying they don't want to know as the memory isn't in their tested list (a list of one), and for all I know even if I had the their tested memory I may still see it running at 1066MHz as it may be a bug in the BIOS.

    Regards

    Phil



  • @Phil_D:

    Is your memory running at 1600MHz or also reported as running at 1066MHz?

    My memory is Kingston HyperX Impact Black Series - DDR3L 8 GB : 2 x 4 GB - SO DIMM 204-PIN - 1600 MHz / PC3L-12800 - CL9 and running at 1066 Mhz.



  • Hi

    @gogol:

    My memory is Kingston HyperX Impact Black Series - DDR3L 8 GB : 2 x 4 GB - SO DIMM 204-PIN - 1600 MHz / PC3L-12800 - CL9 and running at 1066 Mhz.

    I recommend yourself and anyone else with this problem getting in touch with Supermicro support on their website, the more people that complain the more likely they will fix it and unlock the BIOS to more memory.

    Regards

    Phil



  • I have been researching this MBD-X11SBA-LN4F-O board, I saw all this talk here before ordering and decided to give it a shot.

    FYI, for anyone interested in buying this board. I ordered from Newegg and received the 1.02 hardware version.

    I lost access to the web console twice, so I was a little worried I might have the same problem that Engineer described. However both times I was messing with the firewall so I am sort of thinking that had something to do with it. Anyway I digress…. I am testing it my work and it's been up three days and I have had a constant web video stream going 24/7 (part of the time through two of the LAN interfaces at the same time.) No crashes or anything weird so far since I was messing with the firewall.

    Engineer do you have any suggestions I could try to really test it out before I put it into "production?" This first build will just go into my apartment with a 300 mbps fiber connection. I just don't want to piss my roommates off and have it crash so I want to test it for a week or so. Just wondering what people would suggest specifically for this board relating to the earlier problems with the timeouts.

    P.S. Thanks for all the info, this was all very interesting and helped me decide to give this board a shot. I have been running a Gigabyte embedded 1007u board on PFSENSE for quite awhile with zero issues. It's been rock solid, but I wanted to lower the power draw, and get AES, and Intel NICS etc. This SM board seems to hit all the points for my situation. Anyway, just curious what people (esp Engineer) would suggest to put it through its paces.



  • I got an X11SBA-LN4f with the 1.02 pcb revision and still encountered the watchdog issue on igb1(LAN). Even after upgrading the bios to 1.0b. It took about 43 days of uptime before it triggered. I RMA'd it and got an even more defective unit.

    The X11SBA-LN4F is a good system on paper but these problems make it too unreliable for a router. I would advise new buyers to avoid it.



  • Very good thread, was just looking at these 2 SM boards and found this using google. I do have a few questions;

    1. Is this problem only on the X11SBA-LN4f board or does it also affect the A1SRi-2558F?

    2. are there any reported problems with the  A1SRi-2558F board pfsense? Planning on using OpenVPN.



  • 1. Is this problem only on the X11SBA-LN4f board or does it also affect the A1SRi-2558F?

    X11SBA-LN4f only.

    2. are there any reported problems with the  A1SRi-2558F board pfsense? Planning on using OpenVPN.

    No.