X11SBA-LN4F vs A1SRi-2558F

jwt

The Xeon E3-1240LVS does not have QuickAssist, but you can add it via PCIe. Example: http://store.netgate.com/ADI/QuickAssist8955.aspx

I think you want to compare apples to apples, so you'll need 4 x 1GbE on-board.

Supermicro X11SSH-CTF will run over $400, but has dual 10G on-board.
Supermicro X11SSH-LN4F appears to be around $220 online, and has 4 x i210.
I'm seeing E3-1240LV5 at between $290 and $320 online. Call it $300 to split the difference.

So that's $520, plus ram and an enclosure for 4 x i210 and your CPU (that you probably don't need).

I don't think the Rangeley is getting a bit old. I think we've only begun to explore the acceleration potential in the SoC.

The i350 has more queues (8 per port) than the i211 (up to 2) or i210 (up to 4). We don't do a lot with RSS (yet), but it's high on the list now, and when we do, you're going to want a queue per core.

ldean

Hey Engineer,
First just wanted to say thanks for the time/effort you've put into getting this system working. We are also in the process of rolling out this system as a gateway (although not using pfsense), and also experienced the same issue with watchdog timeouts occurring on the LAN ports (the ones that go through the Pericom608GP chip). We have 2 of them in production (both exhibit this behavior) and 10 on backorder right now, so we have a vested interest in finding out/resolving what's causing this issue. I have been in contact with supermicro as well, and they asked me to RMA my board. I would like to be able to reference the case you opened with them in an effort to determine what fixes were done on your board that resolved the problem for you without a giant duplication of effort, and obviously we don't want to have to send every board we purchase to SM for repair :). Do you have a case number available?

Also, what firmware version is running on the board you got back from SM? I saw they have recently released revision 1.0a (no date mentioned but it wasn't there previously the last time I checked). I wonder if that's related or not. I installed it on the box that I have in my lab, and haven't been able to replicate the issue, but this box hadn't experienced the issue previously yet (maybe because it's not actually in production), and as you found, the issue seems to be rather sporadic and could be a few days before it occurs.

Thanks again!!

Engineer

Case# SM1511127317,

Hi, I saw the firmware last night but have not updated. No change log either. Ken Huang is the guy who handled my case. Keep me up to date if you don't mind! Thanks

ldean

@Engineer:

Case# SM1511127317,

Hi, I saw the firmware last night but have not updated. No change log either. Ken Huang is the guy who handled my case. Keep me up to date if you don't mind! Thanks

Thanks!! Ken is handling my case as well. I pointed him to this thread, and summarized, but he is still requesting I RMA one of the boards. I'm going through that process now, hopefully we can determine the root cause and best way to proceed going forward!

ldean

Just a small update –
our plan was to swap a new box in for the one we are experiencing the most issues with, and RMA that one. We swapped the hardware last night and the new box seems to be even worse. We saw about 4 watchdog timeouts last night, and today, the NIC stopped functioning, but did not trigger a timeout. So it just remained indefinitely in a non-operational state, and even ifconfig igb3 down/up, and service netif restart igb3 had no effect. A reboot was the only way to restore connectivity to our LAN. During this time, both the server and the switch reported an active physical link. And this server is running the 1.0a firmware, so that obviously didn't fix anything. We might have to revert to a different device until we get this straightened out with supermicro.

Engineer

@Idean,

Thanks for the update. I suspect that SM will issue a hardware revision on this board very soon. What Hardware Version is on the replacement board that you received? 1.01 (which is what my board is)?

ldean

I haven't received a replacement from SM yet, I'm just shipping out one of the bad boards this morning. The boards I have are revision 1.01 already.

Engineer

@ldean:

I haven't received a replacement from SM yet, I'm just shipping out one of the bad boards this morning. The boards I have are revision 1.01 already.

Good luck and hope it's faster than my case. Took almost four weeks to turn it around for my board. Just glad it's fixed and hopefully, will help in your case. SM really needs to issue a revision quickly and get this under control.

levicki

Hello everyone, new to this forum, new to pfSense and FreeBSD.

I wanted to thank the Engineer for this thread which just saved me a lot of nerves because I was considering buying this:
http://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm

I have two questions:

1. Does anyone have a final answer whether all X11 boards are defective and require RMA?
2. Could anyone share their CPU load results and their usage scenarios (what packages are used, download/upload bandwidth, number of devices on LAN, VPN, etc)?

I have some contacts in Supermicro and I just asked them for information about this case #, I hope I can dig something out.

If I can't, I will consider A1SRi.

Engineer

@levicki:

Hello everyone, new to this forum, new to pfSense and FreeBSD.

I wanted to thank the Engineer for this thread which just saved me a lot of nerves because I was considering buying this:
http://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm

I have two questions:

1. Does anyone have a final answer whether all X11 boards are defective and require RMA?
2. Could anyone share their CPU load results and their usage scenarios (what packages are used, download/upload bandwidth, number of devices on LAN, VPN, etc)?

I have some contacts in Supermicro and I just asked them for information about this case #, I hope I can dig something out.

If I can't, I will consider A1SRi.

1. Don't think anyone knows yet but the fact that ldean has multiple boards doing this, I think there is a design flaw on the boards and that a revision will make it's way to fix it. I would venture to say that the boards are defective at this point (again, just a guess).

2. I don't really have any packages and not the fastest speed (17Meg down/1Meg up). I have been working on an IPsec VPN but haven't got it fully functional yet (it's enabled but not quite working). Without the VPN enabled, I rarely saw the CPU % from the dashboard go above 3%. With the VPN enabled, it varies from 3 to 9%. Haven't investigate what can make it better until I get the VPN working. This is with PowerD set to Hiadaptive

Hope this helps.

Jailer

After watching this thread closely for a while and now multiple reports of issues with this board I've decided to change my plans up a bit. I'm going to go with the X11SBA-F-O 8GB of memory and a mSATA for my build. It's just got 2 i210 LAN ports but that's not an issue for me as I don't need more than 2. If the need arises in the future a managed switch and vlan's can solve that problem.

Hopefully this board with just the 2 LAN ports won't experience the issues the LN4F model has. Should be going forth with the build in the next couple of months.

I wanted to thank the Engineer for this thread which just saved me a lot of nerves because I was considering buying this:
http://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm

It looks great but if this Bare bone will be coming besides with the same failures or issues
likes explained in this thread here, I would be really careful thinking about it using it together with pfSense.

Alternatively to this box, you could try out a Axiomtek NA342 sorted with an Intel J1900 or an
Intel Atom E825 CPU. It is running pfSense without any issues and looks like the named above
Supermicro bare bone. This is not the newest board or CPU, but here in Germany able to get directly
from Axiomtek for 229 € (J1900) or 289 € (E3825). Surely not the same as the both boards named in
this thread but also perhaps an alternative to this. Likes the Axiomtek NA 361 and NA 361R based on
the Intel Atom SoC C2358 or C2558 or C2758.

I would venture to say that the boards are defective at this point (again, just a guess).

Me too after all reading this stuff about this board.

2. I don't really have any packages and not the fastest speed (17Meg down/1Meg up).

Perhaps you will be not able to fully saturate the line? Could you using one time iPerf or NetIO to measure
it from PC to PC through the pfSense box?

I have been working on an IPsec VPN but haven't got it fully functional yet (it's enabled but not quite working). Without the VPN enabled, I rarely saw the CPU % from the dashboard go above 3%. With the VPN enabled, it varies from 3 to 9%.

Are you using IPSec together with AES-GCM? AES-GCM over IPSec is being benefiting from the
AES-NI enormously as I am right informed.

Haven't investigate what can make it better until I get the VPN working. This is with PowerD set to Hiadaptive

Would be the best for any CPU that is coming with TurboBoost or able to run on many different CPU
frequencies to sort the pfSense box even with the right power that is needed at any point and in any
situation. Likes this CPU from 1,16GHz to 2,4GHz.

levicki

@Jailer:

After watching this thread closely for a while and now multiple reports of issues with this board I've decided to change my plans up a bit. I'm going to go with the X11SBA-F-O 8GB of memory and a mSATA for my build. It's just got 2 i210 LAN ports but that's not an issue for me as I don't need more than 2. If the need arises in the future a managed switch and vlan's can solve that problem.

Hopefully this board with just the 2 LAN ports won't experience the issues the LN4F model has. Should be going forth with the build in the next couple of months.

I have decided not to bother with Pentium N3700 based boards (no QuickAssist), and I am even less interested in J1900 (they even don't have AES-NI support). It looks like Atom C2558 (Rangeley) is much better suited for pfSense from all low power CPUs and boards I examined. As for two versus four LAN ports, I am not sure that would avoid the issue given that Engineer said not even first LAN (the one not behind the switch) wasn't working.

I have found another board that looks interesting to me – A1SRM-2558F:
http://www.supermicro.com/products/motherboard/Atom/X10/A1SRM-2558F.cfm

It is mATX (so a bit bigger but I don't mind) and it is a cheaper build overall -- you can put regular DDR3 (cheaper than SO-DIMM) and it is few EUR cheaper (it's around 275 EUR here).

There is also this nice looking case which can be used (around 45 EUR):
http://www.lc-power.com/en/product/gehaeuse/mini-itx/lc-1410mi/

Another option would be to make a desktop based mATX build -- H81 based board and Celeron CPU plus dedicated network adapters which would probably turn out cheaper and more powerful.

Engineer

@levicki:

As for two versus four LAN ports, I am not sure that would avoid the issue given that Engineer said not even first LAN (the one not behind the switch) wasn't working.

Just a correction: First LAN port worked fine out of the box. It was ports 2-4 that were behind the PCIe switching chip that had the issue. Regardless, this is a huge issue for people wanting to use this as a firewall.

Whatever SM did, the board now seems fine and has been up 24+ days with no watchdog timeouts. If I had to do over again knowing this, I would have chosen some other board. I may change my opinion if SM gets this under control without needing to send boards in to be corrected.

Jailer

I would agree that a 2558 based system would be a much better choice for someone with the bandwidth requirements that need it. My ISP is very limited so the quick assist means nothing to me but to others it may. Like I said the X1SBA-F-O is a good choice for me.

It surprises me a bit that the 2558 options from Supermicro are less expensive than the 2358 boards since both are available with similar options. I guess the only reason to go with the 2358 is if power consumption is a priority.

Engineer

@Jailer:

I would agree that a 2558 based system would be a much better choice for someone with the bandwidth requirements that need it. My ISP is very limited so the quick assist means nothing to me but to others it may. Like I said the X1SBA-F-O is a good choice for me.

It surprises me a bit that the 2558 options from Supermicro are less expensive than the 2358 boards since both are available with similar options. I guess the only reason to go with the 2358 is if power consumption is a priority.

Have you bought the 2 port version yet? Can you keep us (or me) updated as to whether you have issues with any of the LAN ports on the board? Also, if you don't mind, can you post your board revision and BIOS version?

Thanks in advance! :)

Edit: Darnit, sorry about not reading enough. I see that you're going forward in the next few months. Sorry about that.

Jailer

No problem. :) I'll be sure to post results when I do order and get it up and running.

levicki

@Jailer:

I would agree that a 2558 based system would be a much better choice for someone with the bandwidth requirements that need it. My ISP is very limited so the quick assist means nothing to me but to others it may. Like I said the X1SBA-F-O is a good choice for me.

I understand.

However, given the cost, this is something I expect to buy/upgrade once in a few years (I hope it will last at least five years).

My bandwidth is currently 70Mbps/4Mbps which is really slow so C2558 might even be an overkill, but I plan to run a few packages, not to mention that the internet speeds can only go up in the future so I consider this future-proofing.

ldean

Just wanted to update the thread. We received our box back from supermicro yesterday and will be installing it into production tomorrow. The repair report is somewhat vague about what they changed, but maybe it makes more sense to someone else:

Customer Reported Symptoms:
Watchdog timeout on ethernet ports.  Per TS, need ECO 18137

Test result notes and repair:
REPORTED PROBLEM FOUND.  WATCHDOG TIMEOUTS ON ETHERNET PORTS.  M/B HW ECO COMPLETED BY REWORK.  M/B BIOS, IPMI FW UPDATED TO CURRENT REVISION DONE.  CPU, DIMM SLOT DETECTION VERIFIED.  NIC PORT, USB PORT, IPMI CONNECTION TEST PASS.  NIC PORT LAN EEPROM FW UPDATED TO CURRENT REVISION COMPLETE.  NIC PORT PASSED OVERNIGHT PING TEST.  COM PORT CONNECTION VERIFIED.  SYSTEM HARDWARE FUNCTIONAL TEST PASS.  ECO VERIFIED.  ALL M/B SCREWS CHECKED.  TEST PASSED.

I'm not too sure what the ECO refers to. Anyone have an idea?

timotl

I just got the 2 port version and have the same issue with the 2nd port.
I put both ports in a LAGG and see the 2nd port stop passing traffic. I don't see watchdog timeouts in the log though.
The rest of the machine keeps running fine on only 1 port.

ECO means Engineering Change Order (ECO)
Definitely sounds like a hardware change, looks like I'll have to send mine back too.
I was hoping it would be resolved by a BIOS update.