X11SBA-LN4F vs A1SRi-2558F
-
This is extremely interesting to me - the LAN1 LAN2 LAN3 and LAN4 ports on the back are not the same!!
Manual from SuperMicro - page 17 block diagram - shows:
A. 1x intel I210 is a single off the SoC on PCIe(1) and
B. the other three are 2 stepped off PCIe(2) then to a Pericom 608GP “PI7C9X2G608GP” (PCIe2 6-Port/ 8-Lane Packet Switch, GreenPacketTM Family) – add 150ns latency to stream per spec.My thoughts would be to move the LAN port if on the three port Pericom switch to PCIe(1) directly off the SoC. Guessing this is lower left corner LAN1 while looking from back side or page 26 of the manual.
-
Going through the BIOS option in the manual some points worth noting to the problemamtic I210:
1. ASPM "Active State Power Management" - Auto default to disable (Applicable to the PCI-Express buses - ala LAN1,2,3,4
2. PCIe Speed from Auto to Gen2 since this is the spec for the I210 on PCIe(1) and (2).
3. Onboard LAN Option ROM Type - make sure Legacy which is default.
4. Onboard LAN1 option ROM from PXE - to Disabled - ironically the rest are off by default - my guess because of the Pericom.
5. Network Stack from Enabled to Disabled - no PXE!
6. Kill the AST2400 serial port since it will not be used?
7. last resort play with turning off and on WatchDog in BIOS -
This is extremely interesting to me - the LAN1 LAN2 LAN3 and LAN4 ports on the back are not the same!!
Manual from SuperMicro - page 17 block diagram - shows:
A. 1x intel I210 is a single off the SoC on PCIe(1) and
B. the other three are 2 stepped off PCIe(2) then to a Pericom 608GP “PI7C9X2G608GP” (PCIe2 6-Port/ 8-Lane Packet Switch, GreenPacketTM Family) – add 150ns latency to stream per spec.My thoughts would be to move the LAN port if on the three port Pericom switch to PCIe(1) directly off the SoC. Guessing this is lower left corner LAN1 while looking from back side or page 26 of the manual.
1st - I've never been able to get a manual as it has not been available. Will check that out.
2nd - Very interesting indeed. Maybe that's why the WAN port stays up and the LAN ports go down.
Think I'm going to switch the LAN and WAN ports for giggles and see if the LAN port goes down on igb0.
Edit: Downloaded the manual (first time that it's been available online - thanks for the heads up) and switched the LAN and WAN ports. If it stays up, then I'm not sure where to go. It could be a hardware issue (with the PCIe lane switch on ports 2, 3 and 4) or it could be a software issue (not certified on FreeBSD 10 according to SuperMicro documents on page). Could also be a BIOS issue. If it has something to do with the three (3) ports and the internal PCIe switch, I would think the WAN port would start dropping and the LAN would stay up. I'll report back as time goes on.
-
Well, I swapped the ports and the WAN went down tonight. Watchdog Timeout.
I think rexki might have found something in the PCIe switch on ports 2-4 (igb1-3). Port 1, not on the PCIe internal motherboard switch, functions normally. The others drop with timeouts.
Maybe it's a driver or FreeBSD issue, but since this board isn't FreeBSD certified by SuperMicro, I'm not sure they will even take a look at it.
Edit: I've opened a technical support case with SuperMicro to see if they can come up with something on this. Would be nice to have someone else with this board and confirm (or deny) if they have the same issues. If not, could see what they have different (power supply, cooling (or lack of as I have now), etc). I might put a USB powered fan blowing on the board to see if it makes a difference. The PCIe switch chip is only rated at 1.2W max and has no heatsink or other cooling.
Edit #2: SuperMicro came back with the suggestion to install a certain version of Ubantu on it and try?!?! WTF kind of suggestion is that, lol? I told him that was a no go unless it's just to run some sort of test program. Starting to wonder if the chip is either defective or getting hot (since no cooling - passive). I have lots of small video card heatsinks and a roll of very good heatsink tape to use if necessary. And the quest goes on….
-
Guys and gals, I'm giving up for now. Went down again and I don't have the time or energy right now to try to fight this. Can't get much from SuperMicro and don't know if it's a hardware issue with the board or if it's a FreeBSD issue (since it does it on both pfsense and opnsense). If anyone else has this board and gets better results, let me know so I can then say it's hardware.
-
I'd love to pick this board up and give it a try but I'm holding back based on your experience with it. If something doesn't change in the near future or someone else comes along and has success I'll likely be getting the X10SBA-L for my build.
-
I'd love to pick this board up and give it a try but I'm holding back based on your experience with it. If something doesn't change in the near future or someone else comes along and has success I'll likely be getting the X10SBA-L for my build.
Really sorry Jailer. I've left the LAN port connected to my network (change IP and turned off DHCP) and am hammering it with ping requests right now. I don't recall it going down when I was doing a burn in test for 8 days with NO WAN connection (i.e. no data flowing through the system). Maybe just needs a system driver update in FreeBSD since this is a VERY new system. I hate to spend the time/money to RMA it only to find out it's the OS/driver and not the hardware. I do have a five year warranty though so … :)
-
The i210 NICs only have 4 rx/tx queues, which is fine for the 4 core SoC (http://ark.intel.com/products/87261/Intel-Pentium-Processor-N3700-2M-Cache-up-to-2_40-GHz), but you'll find that future versions of pfSense have a minimum 4 core requirement (I might make it 8, I've not decided.)
As documented here: http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf , there are only 4 tx and 4 rx queues on an i210.
The SoC is significantly slower than a 4 core Rangeley (1.6GHz on the N3700, 2.4Ghz on the C2558), and this will translate into real-world performance differences. Someone pointed out 6W .vs 15W, and this is why.
Rangeley also has better (i350 .vs i210) NICs. https://twitter.com/gonzopancho/statuses/643443335114424320
I also don't believe in integrated graphics on a standalone networking device.
-
@jwt:
The i210 NICs only have 4 rx/tx queues, which is fine for the 4 core SoC (http://ark.intel.com/products/87261/Intel-Pentium-Processor-N3700-2M-Cache-up-to-2_40-GHz), but you'll find that future versions of pfSense have a minimum 4 core requirement (I might make it 8, I've not decided.)
As documented here: http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf , there are only 4 tx and 4 rx queues on an i210.
The SoC is significantly slower than a 4 core Rangeley (1.6GHz on the N3700, 2.4Ghz on the C2558), and this will translate into real-world performance differences. Someone pointed out 6W .vs 15W, and this is why.
Rangeley also has better (i350 .vs i210) NICs. https://twitter.com/gonzopancho/statuses/643443335114424320
I also don't believe in integrated graphics on a standalone networking device.
Does the N3700's Turbo of 2.4GHz help much as the C2558 does not have it?
Of course, this is all to late for me as I am already stuck with it but I appreciate the breakdown.
-
Even with no WAN connection, it went down by running a large ping (from shell). Was also pinging back to it via another PC. Traffic showed about 650Kbits/sec. Took less than 12 hours to kill it. :(
Edit: Going to install FreeBSD 11 and see if I can figure out some way to test this (not sure how to track watchdog errors with no log, lol). Want to know if this is hardware or some weird driver/OS error.
Edit #2: Have FreeBSD 11 installed and am pinging the living hell out of LAN port 2. Pinging from FreeBSD out to another device and have multiple PC's with multiple command windows open pinging 15,000 byte pings at LAN port 2. Is there a better way to test this? If so, please give detailed instructions as I'm a FreeBSD noob!
Edit #3: One full days of pinging the living hell out of (and in) the board. NO drops. The goal is to go seven days without a drop. If that happens, I'm going to try FreeBSD 10.1 (no pfsense). That will help narrow this thing down a tad more. Of course, if it fails on FreeBSD 11…..(might have to try Ubantu that's certified with this board).
-
If you want to stress test network interfaces, check out iperf. I'm sure Google will dig up some useful reading material.
-
Well, if it's a FreeBSD problem, it is still in version 11 (or the NIC driver). Went down while pinging.
I might try Ubantu next to see if it does it there.
Edit: Running the 64 bit Ubuntu 15.04 test right now. Interesting that the LAN ports are detected as EM0-EM3 ports instead of IGB0-IGB3. I guess that's just a difference between Linux and FreeBSD.
Edit: Ubuntu also went down and reset. Time to tell SuperMicro to send a new one and if it does it again, they need to fix the issue (whether BIOS or board revision and send new ones)! >:(
-
@jwt:
The i210 NICs only have 4 rx/tx queues, which is fine for the 4 core SoC (http://ark.intel.com/products/87261/Intel-Pentium-Processor-N3700-2M-Cache-up-to-2_40-GHz), but you'll find that future versions of pfSense have a minimum 4 core requirement (I might make it 8, I've not decided.)
As documented here: http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf , there are only 4 tx and 4 rx queues on an i210.
The SoC is significantly slower than a 4 core Rangeley (1.6GHz on the N3700, 2.4Ghz on the C2558), and this will translate into real-world performance differences. Someone pointed out 6W .vs 15W, and this is why.
Rangeley also has better (i350 .vs i210) NICs. https://twitter.com/gonzopancho/statuses/643443335114424320
I also don't believe in integrated graphics on a standalone networking device.
Does the N3700's Turbo of 2.4GHz help much as the C2558 does not have it?
Of course, this is all to late for me as I am already stuck with it but I appreciate the breakdown.
Base frequency on the 2558 is 2.4GHz. Base frequency on the N3700 is 1.6GHz. You'll find the CPU doesn't spend very long in 'Turbo' (and then only one one core).
-
If you want to stress test network interfaces, check out iperf. I'm sure Google will dig up some useful reading material.
iperf will not "stress test" your network. It's useful as a bandwidth / throughput test, and that's about all.
If iperf stresses your network, then you need to think about an upgrade.
https://2015.asiabsdcon.org/timetable.html.en#P10A
-
-
@jwt:
I'm a FreeBSD noob!
this much is obvious.
Everyone has to start somewhere and I'm posting my results in efforts trying to help others. Your smartass comment wasn't necessary. At least I'm willing to take the effort to search before asking and trying to figure it out based on previous posters have done.
-
@jwt:
I'm a FreeBSD noob!
this much is obvious.
Everyone has to start somewhere and I'm posting my results in efforts trying to help others. Your smartass comment wasn't necessary. At least I'm willing to take the effort to search before asking and trying to figure it out based on previous posters have done.
I just read through this whole thread, as I am very interested in this board. Despite your troubles, I think I'm going to attempt to put my pfSense build (currently on CentOS VM) on it directly. If the install does awry as it has with you, I will try to install it as a VM on top of CentOS like my lab build.
I probably won't get to buying the hardware for another month, but I'll keep watching this thread.
Thanks for all your updates and time spent working on this project.
-
After sending screen shots of the watchdog timeouts (FreeBSD 10.1, FreeBSD 11 and now Ubuntu 15.04) to SuperMicro, they have finally decided that I need to RMA the board. I really hoping I just got a bad board and not a bad design. Too much money in this thing to have a bad design.
-
Your board is $300 on Amazon.
This:
http://store.netgate.com/ADI/RCC-VE-4860-board.aspx and is priced at $406.
but it has:
More clockspeed (2.5GHz .vs 1.6Ghz on your board)
8GB ram (your board has 0)
8GB (at least, new ones will be more) storage (emmc) on-board (your board has 0)
More Ethernets (6 .vs 4)
Better Ethernet (4 are i350, 2 are i210, your board has 4 i210)
You'll need an ATX power supply to power your board. The 4860 runs on a simple 12VDC barrel connector. -
@jwt:
Your board is $300 on Amazon.
This:
http://store.netgate.com/ADI/RCC-VE-4860-board.aspx and is priced at $406.
but it has:
More clockspeed (2.5GHz .vs 1.6Ghz on your board)
8GB ram (your board has 0)
8GB (at least, new ones will be more) storage (emmc) on-board (your board has 0)
More Ethernets (6 .vs 4)
Better Ethernet (4 are i350, 2 are i210, your board has 4 i210)
You'll need an ATX power supply to power your board. The 4860 runs on a simple 12VDC barrel connector.And on top it comes together with 1x mSATA and 2x miniPCIe + SIM slots and yours only with 1x Mini-PCIe slot.
