X11SBA-LN4F vs A1SRi-2558F



  • @Engineer:

    @Derelict:

    @Engineer:

    Oh, and here's the $216 link since you seem so hellbent against anything to do with me or this board:

    http://www.compsource.com/ttechnote.asp?part_no=MBDX11SBALN4FO&vid=428&src=F

    Looks like a $228 link.

    The search through Google for "Supermicro N3700" and then click on the Shopping link.

    You must have a coupon code or some other promotional offer saved. There is a small "11% off" tag that displays in the screenshot you posted that is not there when others (myself included) click on the posted link.

    Care to share the discount code?  ;D



  • @jwt:

    @Engineer:

    My board cost $216 shipped from Comp Source.

    Seems to be $228.33 here:

    http://www.compsource.com/pn/MBDX11SBALN4FO/Supermicro-428/X11sbaLn4f-N3700-Fcbga-Pentium–Max8gb-Ddr3-MiniItx-MBDX11SBALN4FO-MBDX11SBALN4FO/

    @Engineer:

    I paid less than about $305 for everything in it.

    So you've got $77 to spend for ram, disk, etc.

    Now, what is your time worth?

    Moreover, what am I to do with the negative reputation you're heaping on pfSense because you're attempting to run it on inferior platforms?

    Somebody definitely pissed in your Cheerios bowl yesterday…

    You should applaud the time and effort Engineer put into making this board work, yet all you can come up with is some undeserved negativism.



  • @Engineer:

    To anyone who is interested in this board and the progress on it, please PM me.  I'll keep you updated via PM.  It's obvious that some of the people, including the powers to be, have an issue with it and/or this posting.  I'll no longer update the status in this thead

    Please don't let some smart-ass response leave you with such negativity. I've been following this thread very closely, since i was on a quest to build my own pfsense box, and i think you are doing great in contributing to this community with your effort to resolve this issue. Thanks for that.



  • I've decided to update this thread because of lots of interest in this board.

    After sending the board out via RMA to SuperMicro, they kept it for about 3 weeks or so and ran the tests that I had used to trigger the watchdog timeout.  The engineering team came back and said that they had found the issue and had made a hardware change to my board.  I have tried to get a better explanation as to whether this was a defect on my board only or ALL of these board but have received nothing back yet.

    With that in mind, I tested the board for four days using Ubuntu 15.04 (was still on my drive).  Previously, it would not run for more than 30 hours with going down.  After the four day test, I installed pfsense and ran the same test.  It has been running for two days with no issue so far.  I'm not running a WAN but that doesn't matter as I was able to get it to crash previously without a WAN port simply by bombarding it with LAN traffic.

    If anyone has any specific questions, I'll try to answer.  If the board will run for seven (7) days without crashing the ports, I'll move it back into full WAN service.



  • Much appreciated update.

    On a side note, I cannot stop myself wondering what https://forum.pfsense.org/index.php?action=profile;u=273195 is going to say about this.



  • Thanks for the update!  ;)


  • Netgate

    @jjduru:

    On a side note, I cannot stop myself wondering what https://forum.pfsense.org/index.php?action=profile;u=273195 is going to say about this.

    Me, either.



  • Have just moved this back into full service (WAN) and am running off of it since it made it 8 days (4 on Ubuntu and 4 on pfsense 2.2.5/FreeBSD) without a network going down issue.  I'll keep people informed and answer questions that I can.  Also enabled TRIM for the first time (finally).

    Thanks for the help everyone and Merry Christmas! :)



  • Currently at 4+ days on pfsense 2.2.5 / FreeBSD 10.1 without the LAN going down / Watchdog.  I think it would be fair to say that this issue has been solved for my board.  Now whether SuperMicro has to change other boards or BIOS is anyone's guess right now as I have no feedback from them yet.



  • Thanks for all the info!

    I was wondering how it performs your board when under heavy usage and adding snort, ntopng,etc. I'm currently using a Mac Mini (late 2012) with pfSense and when using all my internet (around 200/20) the CPU usage gets around 40%). My CPU is a i5-3210M, looking at cpubenchmarks website it seems it got around the double of performance of yours… so that would mean (more or less) that when getting 200mbps from the router you should see an usage of around 80%.

    Do you have any numbers on this? thanks!



  • bluepr0, sorry, I don't.  I looked at your specs and your processor is a dual core with hyperthreading while this one is a slower quad core processor.  Your CPU load number seem higher than I would have expected thinking that your processor should handle higher speeds at lower CPU load.  I'm not experienced enough yet to say that though…just a guess on my part.

    Are both of your cores pulling 40% at that time or is only one core pulling 40%?  When simply pinging in and out, the average was 6Mbps in / out and I never saw any core above 4% load with all three other cores running 100% idle (using command top -S -H from the shell or from the command prompt).  But, I have not other packages running and am running basic, slow (compared to yours) internet.  Was trying to build a low power, somewhat future proof unit that could handle up to 1Gbps including encryption (AES-NI) in the future.  Your numbers make me second guess my choice and think that I may have overestimated this board.  sigh

    Edit:  After reading a bit, it seems it doesn't take much (relatively) to run a 1Gbps plain connection but adding SNORT, etc. is where much more processor power is needed to pump data through.  SNORT is single threaded so it would eat up much more of a single CPU core (I did read you can run multiple copies of SNORT and load balance it out but it was much more difficult to do so).



  • I ran again the tests, using top command to see the CPU usage (it seems the CPU usage on web interface is also accurate, making an average of all the cores or am I wrong?)

    Here's the screenshots

    1. This is using all my download bandwidth from the internet http://d.pr/i/1h6X9/3UOn6rvy
    CPU usage is around 30% here

    2. This is using all my download bandwidth from internet + iperf (maxing out gigabit) http://d.pr/i/1bL1T/5PWn64bR
    CPU usage is around 84% here

    What do you think?

    Thanks!



  • Looks like Squid is what is eating up a lot of your resources @bluepr0.

    Engineer, thanks for the update. I was going to ping you and see how things were going so your response was perfect timing.



  • Yep! stopping snort, ntop reduces quite a bit the CPU usage. See http://d.pr/i/1eYwd/NinxWFhR (usage is around 50%)



  • @Jailer:

    Looks like Squid is what is eating up a lot of your resources @bluepr0.

    Engineer, thanks for the update. I was going to ping you and see how things were going so your response was perfect timing.

    Right at 7 days now.  Never made it above 4.5 or so previously.  :)



  • great to know! I'm wondering what they change on the board? does the other boards out there will only need a BIOS update or it's a hardware problem?



  • @bluepr0:

    great to know! I'm wondering what they change on the board? does the other boards out there will only need a BIOS update or it's a hardware problem?

    @bluepr0, I honestly don't know.  SuperMicro said it was a "hardware modification" but they will not disclose what was changed and I cannot see any physical changes to my board.  When I asked whether other boards are affected or only mine, I was told that if there are any other boards out there with this problem, they will fix them.  Sorry guys, I wish I could give you more but SuperMicro won't let it out.  I have not seen a new BIOS other than a new IPMI firmware.



  • So i decided to go with the A1SRi-2558F board (and ECC RAM). Connected LAN1 to WAN (ISP router in bridge mode, connected through surge-protected RJ45 ports on the UPS), connected other 3 LAN ports and IPMI to my (managed) switch.
    Installing pfSense onto the SSD through IPMI mounted ISO… Sudden reboot... 2nd try... Install went fine, but reboot after a few minutes. My first thought is was something with the pfSense install. But....
    Went into BIOS settings and before i could even make any changes... Reboot. So it's NOT a pfSense issue.
    Disconnected all network cables except IPMI, restored default BIOS settings, did 10 cycles of memtest86 (took almost 3 days)... No errors and no reboots.
    Then I connected LAN1 to my network, configured it as the WAN port and connected an AP to LAN2 (configured as LAN). Booted pfSense, did nothing for about an hour and no reboots.

    So, I am now thinking it's a problem with 1 or more of the LAN ports. I want to do some testing, maybe anyone has some idea on how to do effective / efficiënt tests on the LAN ports?



  • Do you use STP or UTP cables?



  • @jahonix:

    Do you use STP or UTP cables?

    I use all CAT6 U/UTP cables. For the whole network, including test setup.



  • This eliminates at least a possible hum/ground loop, well known to audio guys (me) and lesser known in the network world.
    Just for the record: I do not say to only use UTP cables. It just rules out an idea I had in this case.



  • @jahonix:

    This eliminates at least a possible hum/ground loop, well known to audio guys (me) and lesser known in the network world.
    Just for the record: I do not say to only use UTP cables. It just rules out an idea I had in this case.

    Yeah, know what that is. In my setup i don't really need shielded cables (and i also dont want to go through the hassle of properly grounding the whole thing).
    But i'm still hoping for an idea to properly test the LAN ports…



  • Yeah, know what that is. In my setup i don't really need shielded cables (and i also dont want to go through the hassle of properly grounding the whole thing).

    I really don´t know from where you all are and what you have to pay for network cables, but here in Germany
    I have to pay the following money for each;

    • 1 meter patch cable CAT.5e UTP 1,80 €
    • 1 meter patch cable CAT.6a S/STP (PIMF) 2,10 €

    So the difference was so small that I was changing all my patch cables to CAT.6a S/STP (PIMF)
    perhaps not from a premium cable vendor but better then the lazy UTP ones.

    But i'm still hoping for an idea to properly test the LAN ports…

    • Be sure they are not on "auto" to surround a miss match
    • use iPerf or NetIO from one to another PC (client & server)
    • Use proper shielded cables and or a "LAN tester" for testing out also the cables.


  • Just got a question related with the Atom C2758 so I thought about using this thread instead of starting a new one

    These Atom Rangeley are starting to get a bit "old" so I was looking for other ideas or newer hardware. Found out that the new Xeon E3-1240Lv5 has a TDP of only 25w (5w more than the C2758). Also, it seems that on benchmarks doubles in performance the Atom (of course, not networking related task but it might help to get an overall idea).

    What do the experts think about going with a "normal" server board (not so expensive, 1 IPMI, 2 LAN Intel) + this E3-1240LV5?

    Prices in Spain:

    • Supermicro A1SRi-2558F = 440€
      Total = 440€

    • Xeon E3-1240LV5 = 330€ (It has AES, couldn't find if it has QuickAssist)

    • Server board (Asrock, Gigabyte) = 250€
      Total = 580€

    It's a bit more expensive but also more powerful with only 5w more of TDP.

    I'm genuinely asking your opinion, there's probably a lot that I'm missing. Also on a normal server board you would get "normal" server grade LAN, like the i211 or i210… while on the Supermicro you get the i350 (wondering how much of a difference this makes in practice)

    Thanks!


  • Netgate

    The Xeon E3-1240LVS does not have QuickAssist, but you can add it via PCIe. Example: http://store.netgate.com/ADI/QuickAssist8955.aspx

    I think you want to compare apples to apples, so you'll need 4 x 1GbE on-board.

    Supermicro X11SSH-CTF will run over $400, but has dual 10G on-board.
    Supermicro X11SSH-LN4F appears to be around $220 online, and has 4 x i210.
    I'm seeing E3-1240LV5 at between $290 and $320 online.  Call it $300 to split the difference.

    So that's $520, plus ram and an enclosure for 4 x i210 and your CPU (that you probably don't need).

    I don't think the Rangeley is getting a bit old.  I think we've only begun to explore the acceleration potential in the SoC.

    The i350 has more queues (8 per port) than the i211 (up to 2) or i210 (up to 4).  We don't do a lot with RSS (yet), but it's high on the list now, and when we do, you're going to want a queue per core.



  • Hey Engineer,
    First just wanted to say thanks for the time/effort you've put into getting this system working.  We are also in the process of rolling out this system as a gateway (although not using pfsense), and also experienced the same issue with watchdog timeouts occurring on the LAN ports (the ones that go through the Pericom608GP chip).  We have 2 of them in production (both exhibit this behavior) and 10 on backorder right now, so we have a vested interest in finding out/resolving what's causing this issue.  I have been in contact with supermicro as well, and they asked me to RMA my board.  I would like to be able to reference the case you opened with them in an effort to determine what fixes were done on your board that resolved the problem for you without a giant duplication of effort, and obviously we don't want to have to send every board we purchase to SM for repair  :).  Do you have a case number available?

    Also, what firmware version is running on the board you got back from SM?  I saw they have recently released revision 1.0a (no date mentioned but it wasn't there previously the last time I checked).  I wonder if that's related or not.  I installed it on the box that I have in my lab, and haven't been able to replicate the issue, but this box hadn't experienced the issue previously yet (maybe because it's not actually in production), and as you found, the issue seems to be rather sporadic and could be a few days before it occurs.

    Thanks again!!



  • Case# SM1511127317,

    Hi, I saw the firmware last night but have not updated.  No change log either.  Ken Huang is the guy who handled my case.  Keep me up to date if you don't mind!  Thanks😊



  • @Engineer:

    Case# SM1511127317,

    Hi, I saw the firmware last night but have not updated.  No change log either.  Ken Huang is the guy who handled my case.  Keep me up to date if you don't mind!  Thanks😊

    Thanks!!  Ken is handling my case as well.  I pointed him to this thread, and summarized, but he is still requesting I RMA one of the boards.  I'm going through that process now, hopefully we can determine the root cause and best way to proceed going forward!



  • Just a small update –
    our plan was to swap a new box in for the one we are experiencing the most issues with, and RMA that one.  We swapped the hardware last night and the new box seems to be even worse.  We saw about 4 watchdog timeouts last night, and today, the NIC stopped functioning, but did not trigger a timeout.  So it just remained indefinitely in a non-operational state, and even ifconfig igb3 down/up, and service netif restart igb3 had no effect.  A reboot was the only way to restore connectivity to our LAN.  During this time, both the server and the switch reported an active physical link.  And this server is running the 1.0a firmware, so that obviously didn't fix anything.  We might have to revert to a different device until we get this straightened out with supermicro.



  • @Idean,

    Thanks for the update.  I suspect that SM will issue a hardware revision on this board very soon.  What Hardware Version is on the replacement board that you received? 1.01 (which is what my board is)?



  • I haven't received a replacement from SM yet, I'm just shipping out one of the bad boards this morning.  The boards I have are revision 1.01 already.



  • @ldean:

    I haven't received a replacement from SM yet, I'm just shipping out one of the bad boards this morning.  The boards I have are revision 1.01 already.

    Good luck and hope it's faster than my case.  Took almost four weeks to turn it around for my board.  Just glad it's fixed and hopefully, will help in your case.  SM really needs to issue a revision quickly and get this under control.



  • Hello everyone, new to this forum, new to pfSense and FreeBSD.

    I wanted to thank the Engineer for this thread which just saved me a lot of nerves because I was considering buying this:
    http://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm

    I have two questions:

    1. Does anyone have a final answer whether all X11 boards are defective and require RMA?
    2. Could anyone share their CPU load results and their usage scenarios (what packages are used, download/upload bandwidth, number of devices on LAN, VPN, etc)?

    I have some contacts in Supermicro and I just asked them for information about this case #, I hope I can dig something out.

    If I can't, I will consider A1SRi.



  • @levicki:

    Hello everyone, new to this forum, new to pfSense and FreeBSD.

    I wanted to thank the Engineer for this thread which just saved me a lot of nerves because I was considering buying this:
    http://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm

    I have two questions:

    1. Does anyone have a final answer whether all X11 boards are defective and require RMA?
    2. Could anyone share their CPU load results and their usage scenarios (what packages are used, download/upload bandwidth, number of devices on LAN, VPN, etc)?

    I have some contacts in Supermicro and I just asked them for information about this case #, I hope I can dig something out.

    If I can't, I will consider A1SRi.

    1.  Don't think anyone knows yet but the fact that ldean has multiple boards doing this, I think there is a design flaw on the boards and that a revision will make it's way to fix it.  I would venture to say that the boards are defective at this point (again, just a guess).

    2.  I don't really have any packages and not the fastest speed (17Meg down/1Meg up).  I have been working on an IPsec VPN but haven't got it fully functional yet (it's enabled but not quite working).  Without the VPN enabled, I rarely saw the CPU % from the dashboard go above 3%.  With the VPN enabled, it varies from 3 to 9%.  Haven't investigate what can make it better until I get the VPN working.  This is with PowerD set to Hiadaptive

    Hope this helps.



  • After watching this thread closely for a while and now multiple reports of issues with this board I've decided to change my plans up a bit. I'm going to go with the X11SBA-F-O 8GB of memory and a mSATA for my build. It's just got 2 i210 LAN ports but that's not an issue for me as I don't need more than 2. If the need arises in the future a managed switch and vlan's can solve that problem.

    Hopefully this board with just the 2 LAN ports won't experience the issues the LN4F model has. Should be going forth with the build in the next couple of months.



  • I wanted to thank the Engineer for this thread which just saved me a lot of nerves because I was considering buying this:
    http://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm

    It looks great but if this Bare bone will be coming besides with the same failures or issues
    likes explained in this thread here, I would be really careful thinking about it using it together with pfSense.

    Alternatively to this box, you could try out a Axiomtek NA342 sorted with an Intel J1900 or an
    Intel Atom E825 CPU. It is running pfSense without any issues and looks like the named above
    Supermicro bare bone. This is not the newest board or CPU, but here in Germany able to get directly
    from Axiomtek for 229 € (J1900) or 289 € (E3825). Surely not the same as the both boards named in
    this thread but also perhaps an alternative to this. Likes the Axiomtek NA 361 and NA 361R based on
    the Intel Atom SoC C2358 or C2558 or C2758.

    I would venture to say that the boards are defective at this point (again, just a guess).

    Me too after all reading this stuff about this board.

    2.  I don't really have any packages and not the fastest speed (17Meg down/1Meg up).

    Perhaps you will be not able to fully saturate the line? Could you using one time iPerf or NetIO to measure
    it from PC to PC through the pfSense box?

    I have been working on an IPsec VPN but haven't got it fully functional yet (it's enabled but not quite working).  Without the VPN enabled, I rarely saw the CPU % from the dashboard go above 3%.  With the VPN enabled, it varies from 3 to 9%.

    Are you using IPSec together with AES-GCM? AES-GCM over IPSec is being benefiting from the
    AES-NI enormously as I am right informed.

    Haven't investigate what can make it better until I get the VPN working.  This is with PowerD set to Hiadaptive

    Would be the best for any CPU that is coming with TurboBoost or able to run on many different CPU
    frequencies to sort the pfSense box even with the right power that is needed at any point and in any
    situation. Likes this CPU from 1,16GHz to 2,4GHz.



  • @Jailer:

    After watching this thread closely for a while and now multiple reports of issues with this board I've decided to change my plans up a bit. I'm going to go with the X11SBA-F-O 8GB of memory and a mSATA for my build. It's just got 2 i210 LAN ports but that's not an issue for me as I don't need more than 2. If the need arises in the future a managed switch and vlan's can solve that problem.

    Hopefully this board with just the 2 LAN ports won't experience the issues the LN4F model has. Should be going forth with the build in the next couple of months.

    I have decided not to bother with Pentium N3700 based boards (no QuickAssist), and I am even less interested in J1900 (they even don't have AES-NI support). It looks like Atom C2558 (Rangeley) is much better suited for pfSense from all low power CPUs and boards I examined. As for two versus four LAN ports, I am not sure that would avoid the issue given that Engineer said not even first LAN (the one not behind the switch) wasn't working.

    I have found another board that looks interesting to me – A1SRM-2558F:
    http://www.supermicro.com/products/motherboard/Atom/X10/A1SRM-2558F.cfm

    It is mATX (so a bit bigger but I don't mind) and it is a cheaper build overall -- you can put regular DDR3 (cheaper than SO-DIMM) and it is few EUR cheaper (it's around 275 EUR here).

    There is also this nice looking case which can be used (around 45 EUR):
    http://www.lc-power.com/en/product/gehaeuse/mini-itx/lc-1410mi/

    Another option would be to make a desktop based mATX build -- H81 based board and Celeron CPU plus dedicated network adapters which would probably turn out cheaper and more powerful.



  • @levicki:

    As for two versus four LAN ports, I am not sure that would avoid the issue given that Engineer said not even first LAN (the one not behind the switch) wasn't working.

    Just a correction:  First LAN port worked fine out of the box.  It was ports 2-4 that were behind the PCIe switching chip that had the issue.  Regardless, this is a huge issue for people wanting to use this as a firewall.

    Whatever SM did, the board now seems fine and has been up 24+ days with no watchdog timeouts.  If I had to do over again knowing this, I would have chosen some other board.  I may change my opinion if SM gets this under control without needing to send boards in to be corrected.



  • I would agree that a 2558 based system would be a much better choice for someone with the bandwidth requirements that need it. My ISP is very limited so the quick assist means nothing to me but to others it may. Like I said the X1SBA-F-O is a good choice for me.

    It surprises me a bit that the 2558 options from Supermicro are less expensive than the 2358 boards since both are available with similar options. I guess the only reason to go with the 2358 is if power consumption is a priority.



  • @Jailer:

    I would agree that a 2558 based system would be a much better choice for someone with the bandwidth requirements that need it. My ISP is very limited so the quick assist means nothing to me but to others it may. Like I said the X1SBA-F-O is a good choice for me.

    It surprises me a bit that the 2558 options from Supermicro are less expensive than the 2358 boards since both are available with similar options. I guess the only reason to go with the 2358 is if power consumption is a priority.

    Have you bought the 2 port version yet?  Can you keep us (or me) updated as to whether you have issues with any of the LAN ports on the board?  Also, if you don't mind, can you post your board revision and BIOS version?

    Thanks in advance! :)

    Edit:  Darnit, sorry about not reading enough.  I see that you're going forward in the next few months.  Sorry about that.