X11SBA-LN4F vs A1SRi-2558F

rexki

Engineer: Doing some double checking on SuperMicro web site:

1. Still no BIOS update
2. Certified 4GB ram is only one item listed: 4GB – Hynix - MEM-DR340L-HL02-SO16 - HMT451S6BFR8A-PB - H5TC4G83BFR

Specs above further detailed:
• 4GB Memory Module
• DDR3-1600MHz
• PC3L-12800
• Non-ECC
• 1.35v Low Voltage LV
• SODIMM
• 204-pin
• Cas Latency 11
• 1Rx8 (H5TC4G83BFR - 512MB chips each)

3. Certified 8GB stick has the same exact component chip as the 4GB - H5TC4G83BFR - 512MB chips each

Your spec from page one listed as follows:
2 x 4GB Samsung PC1600 DDR3L: $36 shipped from eBay

Found Samsung 4gb pc1600 sodimm items to get comparatives:
1. Samsung Part number M471B5273DH0-CK0 - 1.50v
2. Samsung Part number M471B5173DBO-YK0 - 1.35v - CL11 - ?Single sided 1Rx4?
3. Samsung Part number M471B5173BHO-YK0 - 1.35v - CL11 - ?Single sided 1Rx4?
4. Samsung MV-3T4G3D/US - 1.35v - CL11 - 2Rx4 Double sided - 8 each

My questions:
1. Do these certified specs match exactly to yours especially 1Rx8 with 512 and CAS 11?
2. Have you run with just one chip of ram at 4GB - number one cycles solo and then number two cycled solo as well to isolate behavior?
3. Have you swapped memory places of the existing Samsung 2 *4GB sticks?

Engineer

I have re-seated the ram but not swapped it or run it in single channel (one stick). Once I completed the memtest+ test, I had assumed all was good. I suppose it could be ram but I tend to think that takes the whole ship down including the console (aka - one big crash). I suppose it might be ram. The ram is DDR3L 1.35V (I did make sure of that when I purchased it).

I'll see about removing a stick later. As for the certified memory stuff, I understand the testing and stuff but there's no real reason for most of the good stuff not to run. Just hasn't been certified….YET. When I bought the board and ram, there were NO certified sticks listed on the SuperMicro site. ;)

Oh, and for now, I've turned the Asus RT-AC68U back from AP to Router while trying to sort this out. Busy lately so I'll have to put it on hold a little. Hard to even do anything as people scream faster about the Internet being down than they do if we were being robbed, lol.

rexki

This is extremely interesting to me - the LAN1 LAN2 LAN3 and LAN4 ports on the back are not the same!!

Manual from SuperMicro - page 17 block diagram - shows:

A. 1x intel I210 is a single off the SoC on PCIe(1) and
B. the other three are 2 stepped off PCIe(2) then to a Pericom 608GP “PI7C9X2G608GP” (PCIe2 6-Port/ 8-Lane Packet Switch, GreenPacketTM Family) – add 150ns latency to stream per spec.

My thoughts would be to move the LAN port if on the three port Pericom switch to PCIe(1) directly off the SoC. Guessing this is lower left corner LAN1 while looking from back side or page 26 of the manual.

rexki

Going through the BIOS option in the manual some points worth noting to the problemamtic I210:

1. ASPM "Active State Power Management" - Auto default to disable (Applicable to the PCI-Express buses - ala LAN1,2,3,4
2. PCIe Speed from Auto to Gen2 since this is the spec for the I210 on PCIe(1) and (2).
3. Onboard LAN Option ROM Type - make sure Legacy which is default.
4. Onboard LAN1 option ROM from PXE - to Disabled - ironically the rest are off by default - my guess because of the Pericom.
5. Network Stack from Enabled to Disabled - no PXE!
6. Kill the AST2400 serial port since it will not be used?
7. last resort play with turning off and on WatchDog in BIOS

Engineer

@rexki:

This is extremely interesting to me - the LAN1 LAN2 LAN3 and LAN4 ports on the back are not the same!!

Manual from SuperMicro - page 17 block diagram - shows:

A. 1x intel I210 is a single off the SoC on PCIe(1) and
B. the other three are 2 stepped off PCIe(2) then to a Pericom 608GP “PI7C9X2G608GP” (PCIe2 6-Port/ 8-Lane Packet Switch, GreenPacketTM Family) – add 150ns latency to stream per spec.

My thoughts would be to move the LAN port if on the three port Pericom switch to PCIe(1) directly off the SoC. Guessing this is lower left corner LAN1 while looking from back side or page 26 of the manual.

1st - I've never been able to get a manual as it has not been available. Will check that out.

2nd - Very interesting indeed. Maybe that's why the WAN port stays up and the LAN ports go down.

Think I'm going to switch the LAN and WAN ports for giggles and see if the LAN port goes down on igb0.

Edit: Downloaded the manual (first time that it's been available online - thanks for the heads up) and switched the LAN and WAN ports. If it stays up, then I'm not sure where to go. It could be a hardware issue (with the PCIe lane switch on ports 2, 3 and 4) or it could be a software issue (not certified on FreeBSD 10 according to SuperMicro documents on page). Could also be a BIOS issue. If it has something to do with the three (3) ports and the internal PCIe switch, I would think the WAN port would start dropping and the LAN would stay up. I'll report back as time goes on.

Engineer

Well, I swapped the ports and the WAN went down tonight. Watchdog Timeout.

I think rexki might have found something in the PCIe switch on ports 2-4 (igb1-3). Port 1, not on the PCIe internal motherboard switch, functions normally. The others drop with timeouts.

Maybe it's a driver or FreeBSD issue, but since this board isn't FreeBSD certified by SuperMicro, I'm not sure they will even take a look at it.

Edit: I've opened a technical support case with SuperMicro to see if they can come up with something on this. Would be nice to have someone else with this board and confirm (or deny) if they have the same issues. If not, could see what they have different (power supply, cooling (or lack of as I have now), etc). I might put a USB powered fan blowing on the board to see if it makes a difference. The PCIe switch chip is only rated at 1.2W max and has no heatsink or other cooling.

Edit #2: SuperMicro came back with the suggestion to install a certain version of Ubantu on it and try?!?! WTF kind of suggestion is that, lol? I told him that was a no go unless it's just to run some sort of test program. Starting to wonder if the chip is either defective or getting hot (since no cooling - passive). I have lots of small video card heatsinks and a roll of very good heatsink tape to use if necessary. And the quest goes on….

Engineer

Guys and gals, I'm giving up for now. Went down again and I don't have the time or energy right now to try to fight this. Can't get much from SuperMicro and don't know if it's a hardware issue with the board or if it's a FreeBSD issue (since it does it on both pfsense and opnsense). If anyone else has this board and gets better results, let me know so I can then say it's hardware.

Jailer

I'd love to pick this board up and give it a try but I'm holding back based on your experience with it. If something doesn't change in the near future or someone else comes along and has success I'll likely be getting the X10SBA-L for my build.

Engineer

@Jailer:

I'd love to pick this board up and give it a try but I'm holding back based on your experience with it. If something doesn't change in the near future or someone else comes along and has success I'll likely be getting the X10SBA-L for my build.

Really sorry Jailer. I've left the LAN port connected to my network (change IP and turned off DHCP) and am hammering it with ping requests right now. I don't recall it going down when I was doing a burn in test for 8 days with NO WAN connection (i.e. no data flowing through the system). Maybe just needs a system driver update in FreeBSD since this is a VERY new system. I hate to spend the time/money to RMA it only to find out it's the OS/driver and not the hardware. I do have a five year warranty though so … :)

jwt

The i210 NICs only have 4 rx/tx queues, which is fine for the 4 core SoC (http://ark.intel.com/products/87261/Intel-Pentium-Processor-N3700-2M-Cache-up-to-2_40-GHz), but you'll find that future versions of pfSense have a minimum 4 core requirement (I might make it 8, I've not decided.)

As documented here: http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf , there are only 4 tx and 4 rx queues on an i210.

The SoC is significantly slower than a 4 core Rangeley (1.6GHz on the N3700, 2.4Ghz on the C2558), and this will translate into real-world performance differences. Someone pointed out 6W .vs 15W, and this is why.

Rangeley also has better (i350 .vs i210) NICs. https://twitter.com/gonzopancho/statuses/643443335114424320

I also don't believe in integrated graphics on a standalone networking device.

Engineer

@jwt:

The i210 NICs only have 4 rx/tx queues, which is fine for the 4 core SoC (http://ark.intel.com/products/87261/Intel-Pentium-Processor-N3700-2M-Cache-up-to-2_40-GHz), but you'll find that future versions of pfSense have a minimum 4 core requirement (I might make it 8, I've not decided.)

As documented here: http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf , there are only 4 tx and 4 rx queues on an i210.

The SoC is significantly slower than a 4 core Rangeley (1.6GHz on the N3700, 2.4Ghz on the C2558), and this will translate into real-world performance differences. Someone pointed out 6W .vs 15W, and this is why.

Rangeley also has better (i350 .vs i210) NICs. https://twitter.com/gonzopancho/statuses/643443335114424320

I also don't believe in integrated graphics on a standalone networking device.

Does the N3700's Turbo of 2.4GHz help much as the C2558 does not have it?

Of course, this is all to late for me as I am already stuck with it but I appreciate the breakdown.

Engineer

Even with no WAN connection, it went down by running a large ping (from shell). Was also pinging back to it via another PC. Traffic showed about 650Kbits/sec. Took less than 12 hours to kill it. :(

Edit: Going to install FreeBSD 11 and see if I can figure out some way to test this (not sure how to track watchdog errors with no log, lol). Want to know if this is hardware or some weird driver/OS error.

Edit #2: Have FreeBSD 11 installed and am pinging the living hell out of LAN port 2. Pinging from FreeBSD out to another device and have multiple PC's with multiple command windows open pinging 15,000 byte pings at LAN port 2. Is there a better way to test this? If so, please give detailed instructions as I'm a FreeBSD noob!

Edit #3: One full days of pinging the living hell out of (and in) the board. NO drops. The goal is to go seven days without a drop. If that happens, I'm going to try FreeBSD 10.1 (no pfsense). That will help narrow this thing down a tad more. Of course, if it fails on FreeBSD 11…..(might have to try Ubantu that's certified with this board).

David_W

If you want to stress test network interfaces, check out iperf. I'm sure Google will dig up some useful reading material.

Engineer

Well, if it's a FreeBSD problem, it is still in version 11 (or the NIC driver). Went down while pinging.

I might try Ubantu next to see if it does it there.

Edit: Running the 64 bit Ubuntu 15.04 test right now. Interesting that the LAN ports are detected as EM0-EM3 ports instead of IGB0-IGB3. I guess that's just a difference between Linux and FreeBSD.

Edit: Ubuntu also went down and reset. Time to tell SuperMicro to send a new one and if it does it again, they need to fix the issue (whether BIOS or board revision and send new ones)! >:(

jwt

@Engineer:

@jwt:

The i210 NICs only have 4 rx/tx queues, which is fine for the 4 core SoC (http://ark.intel.com/products/87261/Intel-Pentium-Processor-N3700-2M-Cache-up-to-2_40-GHz), but you'll find that future versions of pfSense have a minimum 4 core requirement (I might make it 8, I've not decided.)

As documented here: http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf , there are only 4 tx and 4 rx queues on an i210.

The SoC is significantly slower than a 4 core Rangeley (1.6GHz on the N3700, 2.4Ghz on the C2558), and this will translate into real-world performance differences. Someone pointed out 6W .vs 15W, and this is why.

Rangeley also has better (i350 .vs i210) NICs. https://twitter.com/gonzopancho/statuses/643443335114424320

I also don't believe in integrated graphics on a standalone networking device.

Does the N3700's Turbo of 2.4GHz help much as the C2558 does not have it?

Of course, this is all to late for me as I am already stuck with it but I appreciate the breakdown.

Base frequency on the 2558 is 2.4GHz. Base frequency on the N3700 is 1.6GHz. You'll find the CPU doesn't spend very long in 'Turbo' (and then only one one core).

jwt

@David_W:

If you want to stress test network interfaces, check out iperf. I'm sure Google will dig up some useful reading material.

iperf will not "stress test" your network. It's useful as a bandwidth / throughput test, and that's about all.

If iperf stresses your network, then you need to think about an upgrade.

https://2015.asiabsdcon.org/timetable.html.en#P10A

jwt

@Engineer:

I'm a FreeBSD noob!

this much is obvious.

Engineer

@jwt:

@Engineer:

I'm a FreeBSD noob!

this much is obvious.

Everyone has to start somewhere and I'm posting my results in efforts trying to help others. Your smartass comment wasn't necessary. At least I'm willing to take the effort to search before asking and trying to figure it out based on previous posters have done.

Adrocrag

@Engineer:

@jwt:

@Engineer:

I'm a FreeBSD noob!

this much is obvious.

Everyone has to start somewhere and I'm posting my results in efforts trying to help others. Your smartass comment wasn't necessary. At least I'm willing to take the effort to search before asking and trying to figure it out based on previous posters have done.

I just read through this whole thread, as I am very interested in this board. Despite your troubles, I think I'm going to attempt to put my pfSense build (currently on CentOS VM) on it directly. If the install does awry as it has with you, I will try to install it as a VM on top of CentOS like my lab build.

I probably won't get to buying the hardware for another month, but I'll keep watching this thread.

Thanks for all your updates and time spent working on this project.

Engineer

After sending screen shots of the watchdog timeouts (FreeBSD 10.1, FreeBSD 11 and now Ubuntu 15.04) to SuperMicro, they have finally decided that I need to RMA the board. I really hoping I just got a bad board and not a bad design. Too much money in this thing to have a bad design.