SuperMicro A1SAI-2750F / C2000 SoC / C2000 Network Controller



  • I've seen a few threads floating around having to do with the Avoton Atom line, but from what I've been able to tell, all issues that had been seen have since been worked out. That being said, I'm seeing random hard reboots on the build I just finished.

    pfSense 2.1.5 x64 on a 120gig SSD. IPMI System log doesn't report hardware issues from what I can see.

    Any thoughts or suggestions on what I can do trouble shoot further? The reboots happen anywhere from 2-10 minutes after boot finishes. CPU temp is ambient at all times.



  • Did you adjust your MBUFs for the Intel NICs?


  • Netgate Administrator

    Hmm, hard reboots are usually a hardware issue. Often cooling or PSU issues. Can you try a different psu? Have you run any other OS on the board?

    Steve



  • @baggar11:

    Did you adjust your MBUFs for the Intel NICs?

    How would I do that in pfSense?

    @stephenw10:

    Hmm, hard reboots are usually a hardware issue. Often cooling or PSU issues. Can you try a different psu? Have you run any other OS on the board?

    Steve

    Got a second PSU already in there - same problems. It was purchased of Amazon, so RMA shouldn't be too big an issue



  • @geudrik:

    @baggar11:

    Did you adjust your MBUFs for the Intel NICs?

    How would I do that in pfSense?

    Check https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards



  • @Wolf666:

    @geudrik:

    @baggar11:

    Did you adjust your MBUFs for the Intel NICs?

    How would I do that in pfSense?

    Check https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

    I don't see anything specific for the C2000 series controllers (SoC). Further, the NICs actually seem to work/are all detected. I'm going to install Ubuntu for grins here in a minute and see if the problem persists.

    Update: Bad board, it has to be. It reboots under Ubuntu, too



  • If you get reboots, can also be bad ram, bad capacitors for the ram or a weak capacitor anywhere in the mobo…  Power flickers also.

    MBUF can do it sometimes, but never did with mine.  Just sort of made things not work well when MBUF became exhausted.

    Seems like maybe a hardware problem or perhaps some BIOS issue.

    Try running a RAM test.

    Or boot into linux on that machine and run prime95...

    Maybe its something as simple as low volt setting for memory on the board?



  • @kejianshi:

    If you get reboots, can also be bad ram, bad capacitors for the ram or a weak capacitor anywhere in the mobo…  Power flickers also.

    MBUF can do it sometimes, but never did with mine.  Just sort of made things not work well when MBUF became exhausted.

    Seems like maybe a hardware problem or perhaps some BIOS issue.

    Try running a RAM test.

    Or boot into linux on that machine and run prime95...

    Maybe its something as simple as low volt setting for memory on the board?

    It reboots idling in Ubuntu booted off USB. I'm going to try MemTest86+ and see if that has the same issues. If it does, perhaps ram, otherwise, I suspect the mobo itself. To be fair, I couldn't get the board to post using the DIMMs closest to the CPU. I had to use the DIMMs furthest away (won on each side)



  • Probably a bad memory slot or a bad memory capacitor…

    I've had exactly the same problem as you before.

    With a completely non-working slot, I think simply bad slot.

    With intermittent failure I suspect capacitor.

    Either way, fix is same.  Swap out the board.

    P.S.  Its a damn nice board.  I'm envious.



  • I know this probably needs not be said, but when testing RAM, if possible, use only 1 stick.  Test it in each slot.

    Then the other stick.  Test it in each slot.

    Then together.

    Otherwise you might chase your tail for hours or days.



  • @kejianshi:

    P.S.  Its a damn nice board.  I'm envious.

    You didn't like Atom boards. Nice to see you changed your mind :-)

    Cheers.



  • Well - I do actually like atom boards when they are used within reason.  All atoms are not created equal.

    I have 1 in a netbook that has 2 cores but I'm not running vmware and 6 VM servers on it.

    I use a 4 core atom as hardware pfsense server.

    This board here is a beast compared to old atoms.  8 cores and on par with my old Intel quad core extreme.

    So, yeah - now you have to qualify statements about atoms.

    I don't care what name they gave that cpu or board of yours.  Its clearly much better spec than an n2800.

    The guys here has an n2700 netbook and 1GB of ram, but seems like he intends to use it the same way I use my 8 core AMDs….

    https://forum.pfsense.org/index.php?topic=81954.0;topicseen

    When I'm talking about getting carried away with what can be done with an atom reliably, it this sort of thing I'm talking about.

    Yours isn't even in the same league - I want one.



  • @kejianshi:

    I know this probably needs not be said, but when testing RAM, if possible, use only 1 stick.  Test it in each slot.

    Then the other stick.  Test it in each slot.

    Then together.

    Otherwise you might chase your tail for hours or days.

    Yep, appreciate the responses! I've RMAd the board and have a new one coming from Amazon. I was thoroughly surprised when I stumbled across that board - "Atom" I feel, is now an inaccurate (based on its history) name for these new generation CPUs :P

    FWIW, this is preparation for an upcoming move / upgrade in home lab space. This will become my dedicated GW and Suricata appliance. On a 150x150 connection, I needed a little more oomph than my Atom D525.



  • I have a few VMs running on a machine thats chewing 125 watts all the time…

    Its a super reliable machine with a heat sink the size of my thigh.

    I'd gladly replace it with that board you have there when that old one fails.

    That might be a very long time though - Its all solid capacitors, top of the line board and PSU.

    But if its struck by lightening (it happens) I will be buying me one like yours.

    Let me know how well it manages temperature please?

    I'd love a powerful fanless solution that didn't run hot or throttle CPU to manage heat.



  • @kejianshi:

    I have a few VMs running on a machine thats chewing 125 watts all the time…

    Its a super reliable machine with a heat sink the size of my thigh.

    I'd gladly replace it with that board you have there when that old one fails.

    That might be a very long time though - Its all solid capacitors, top of the line board and PSU.

    But if its struck by lightening (it happens) I will be buying me one like yours.

    Let me know how well it manages temperature please?

    I'd love a powerful fanless solution that didn't run hot or throttle CPU to manage heat.

    Here's what I was running until the powerstrip I plugged into my PDU blew up and fried it.

    Supermicro SuperChassis CSE-503L-200B 1U
    Supermicro X7SPA-HF-D525 Motherboard
    4GB 800MHz DDR3 Non-ECC CL6 SODIMM (2x 2GB Twigs)

    Optional:
    PCI-E Express Riser Card with Flexible Cable
    PCI-E 1, 2, or 4 port NIC (I had a 2 port INTEL PRO sitting around)
    Price Tag: ~$400 shipped, ~$450 shipped with extra nic and riser
    Note: At least that was the pricetag about two years ago, no idea what it is now (this was copy-pasted from a writeup I did at the time of build)

    It sucked about 20 watts, was silent, and temps were about +10F above ambient at the CPU. The D525 isn't all that splended a processor though… it's pretty weeny. Currently I'm running a Dell PowerEdge R210 (i3 550 w/ 4gigs ram) but for sitting in my living room, it's too loud. It draws significantly more power than the Atom board :P

    I'll be out of town the next few days, but the replacement from Amazon should be here Monday. Once I get home and get it up and running, I'll let you know temps and performance. I run 5 IPSec tunnels that I can shove data down, too, so we can actually give a little work to the SoC ;P

    On an unrelated note, my R610 is awesome as a Virt server. It's completely reasonable in terms of decibels and you can pick them up fairly cheap on ebay. Note: It's a full-depth 1u, so don't expect it to rack properly (see: at all) in anything other than a full-depth rack (eg: at least a half-height). You can hear it, but it's .. pretty damn quiet, especially for the power that it touts. With the majority of my VMs running, I'm seeing about 200 watts being pulled (CPU Usage averaging 30%) at the plug.

    To be fair, I keep my house around 70F. Grew up in the north - the cold is in my veins :P



  • If it was up to me, every motherboard manufacturer would be crossed off my list and only supermicro would be left. Quality aside, supermicro is the supplier for ixsystems. Look that name up if you are interested in what they do (hint: extremely involved with *bsd).

    That being said, supermicros are notoriously "stubborn" when it comes to memory. Make sure you only use memory that is on the tested RAM list. Never had a problem with a supermicro board, that wasn't traced back to a memory issue. I've had one particular scenario where a C2SEA (typed that without even looking it up, that's how much I like supermicro boards) simply refused to use any other memory, except a specific model on their tested list. Even when the untested memory used the same chips as a tested module. Tested the "faulty" modules numerous times in other systems (not supermicro), all of them are fine. I'm even still using a pair of the "faulty" modules  it in a different system for the past 2 years.



  • @jflsakfja:

    If it was up to me, every motherboard manufacturer would be crossed off my list and only supermicro would be left. Quality aside, supermicro is the supplier for ixsystems. Look that name up if you are interested in what they do (hint: extremely involved with *bsd).

    "every"???  You don't know what you're talking about.

    @jflsakfja:

    That being said, supermicros are notoriously "stubborn" when it comes to memory. Make sure you only use memory that is on the tested RAM list.

    OP probably used non-ECC DDR3 SO-DIMMs in an effort to save money.

    Oh, and iXsystems uses the ASRock mobo for their miniNAS product.

    If you don't know enough to go off the "recommended/tested" memory list, then don't.  If you do, it's all on you anyway.



  • @gonzopancho:

    "every"???  You don't know what you're talking about.

    Please enlighten me.



  • Can't we all just get along….  (reaching for vodka)

    Would using the wrong spec ram work at all?

    Hmmm.  Maybe he will tell us what he used.

    I'll wait and see.



  • @jflsakfja:

    @gonzopancho:

    "every"???  You don't know what you're talking about.

    Please enlighten me.

    Supermicro and others all build based on "reference designs" produced by Intel / AMD / …

    They modify these designs, some more than others, in an effort to gain advantage in the market (features / price).  Most low-end PC mobos then go through a process where components (Rs & Cs) are shaved off the design until design will no longer pass end-of-line 'margin' tests.

    This is one of the many reasons that cheap Chinese manufacturers that some in the community cheer so loudly produce hardware that sucks so badly.  "Dude, I built a pfSense box for $99!" is so 1996.

    Supermicro stays on a somewhat more conservative path with this, but there are others.  This is especially true if you start dealing directly with the firms that do the original design engineering for Intel / AMD / ...

    You'll never guess who did the original design work for "Mohan Peak" (the original Avoton/Rangeley design).  There is a reason I've been so solid on the C2000 series SoCs here for the past year.

    RCC-VE is expressly a cost-reduction effort on Mohan Peak.

    Edit:  iXsystems is a good company, and knows a lot about FreeBSD.  I have many friends there.  Supermicro is not their sole supplier.



  • @kejianshi:

    Can't we all just get along….  (reaching for vodka)

    Would using the wrong spec ram work at all?

    Hmmm.  Maybe he will tell us what he used.

    I'll wait and see.

    It could have been a bad board, too.

    Some C2000 designs have been modified to use non-ECC ram.  It's not really a requirement unless you're doing storage.



  • So we shouldn't choose supermicro boards, because they are based on reference designs. The fact that supermicro supplies each and every one of the ixsystems' systems is irrelevant. And ixsystem's involvement with freebsd (and PC-BSD and FreeNAS) doesn't add any weight to the argument as to why go with a supermicro board instead of a different vendor.

    OK, got it. Next time I'll go with a mobo manufacturer that does absolutely no testing with freebsd and cross my fingers hoping it will work.



  • I've never touched one of these 8 core boards.  I like what I see though.

    Which ever one of my machines dies first will get replaced with something like this.



  • My perhaps silly impression is that time, more than anything determines compatibility of boards and features with BSD and linux.

    Not so much who is making the board.

    Thats why I usually opt for older designs these days if I'm straight hardware with no hypervisor.

    Who makes the board more affects quality, durability and features.

    Seems to me anyway.



  • @jflsakfja:

    So we shouldn't choose supermicro boards, because they are based on reference designs. The fact that supermicro supplies each and every one of the ixsystems' systems is irrelevant. And ixsystem's involvement with freebsd (and PC-BSD and FreeNAS) doesn't add any weight to the argument as to why go with a supermicro board instead of a different vendor.

    OK, got it. Next time I'll go with a mobo manufacturer that does absolutely no testing with freebsd and cross my fingers hoping it will work.

    What?  Not what I said at all.

    You said, "If it was up to me, every motherboard manufacturer would be crossed off my list and only supermicro would be left."

    I'm expressly stating that there are others.

    We carry Supermicro boards.



  • @kejianshi:

    My perhaps silly impression is that time, more than anything determines compatibility of boards and features with BSD and linux.

    Not so much who is making the board.

    Thats why I usually opt for older designs these days if I'm straight hardware with no hypervisor.

    Who makes the board more affects quality, durability and features.

    Seems to me anyway.

    The situation is not as bad as it was.

    these days it's mostly about laptops and esoteric hardware.
    FreeBSD finally has a current, 100% supported out of the box laptop.

    Some SoC vendors won't support FreeBSD.



  • Got a link?



  • to?



  • The BSD laptop?  Got a link?  I'd like to take a look.



  • @kejianshi:

    The BSD laptop?  Got a link?  I'd like to take a look.

    System76 makes *nix tested/compatible laptops. Not going to post a link, duckduckgo it  ;)



  • @gonzopancho:

    OP probably used non-ECC DDR3 SO-DIMMs in an effort to save money.

    I spend close to 400$ on a mobo and you think I'm skimping on my ram? I'm insulted, sir. :P

    @kejianshi:

    Would using the wrong spec ram work at all?

    Wrong type? No. It wouldn't even fit in the slots. Wrong speed/voltage? Maybe. Depends how the BIOS sees the hardware. I'm not familliar with SM subsystems, so I can't speak to how proficient they are at auto-detecting and behaving nicely with parts not listed on the QVL.

    FWIW: I'm using ram that is not on the QVL. If it wasn't going to work, it wouldn't have worked at all (to your point).

    This is a solved problem - I had a bad board. I swapped the board, and it's all working great now (except my expected membuf issue).

    kejianshi: I'm going to push this a bit and run a bunch of stuff through suricata and report back. I'll try to keep the OP updated with my results as my experiment (playtime) progresses. Appreciate your feedback :)



  • Thanks - I'd appreciate that.  I'm thinking to get something like this soon.  Nice to know what to expect.

    Glad you are up and running (-:



  • @geudrik:

    @gonzopancho:

    OP probably used non-ECC DDR3 SO-DIMMs in an effort to save money.

    I spend close to 400$ on a mobo and you think I'm skimping on my ram? I'm insulted, sir. :P

    @kejianshi:

    Would using the wrong spec ram work at all?

    Wrong type? No. It wouldn't even fit in the slots. Wrong speed/voltage? Maybe. Depends how the BIOS sees the hardware. I'm not familliar with SM subsystems, so I can't speak to how proficient they are at auto-detecting and behaving nicely with parts not listed on the QVL.

    The SoC can only do so much.  The CPU must be fed, and ram must be refreshed.

    @geudrik:

    FWIW: I'm using ram that is not on the QVL. If it wasn't going to work, it wouldn't have worked at all (to your point).

    No, it could just be flakey.

    Good to hear your problem is resolved.  We ship <a lot=""></a><a lot="">of these, and have yet to see a board failure.</a>



  • Hey gonzo, when will the C2XXX retail box be ready? I want to give Netgate more money. :)



  • I've been running a SuperMicro Rangeley SoC for I guess about a year now?  I'd have to look… anyways it's been running without any issues on ESX5.5 since I first set it up... I have a basic Intel 330 SSD and 2x sodimm 8gb Kingston ECC valueram.  Way overkill for a handful of VPNs and 150Mb WAN link... but it's nice to mess around with.  CPU/RAM utilization is around like 8%. I plan to install without Vmware on next release.



  • @docwho76:

    Hey gonzo, when will the C2XXX retail box be ready? I want to give Netgate more money. :)

    early 2015