X11SBA-LN4F vs A1SRi-2558F

Guest

Any thoughts as to where to start?

A proper WAN connection should be up and running. And why not connecting the pfSense to
the Internet, for sure for playing around it is not really necessary and urgent but for the fully
functionality of the pfSense box it should be the best option for you.

Activate TRIM Support in pfSense for the SSD if the SSD is capable of the TRIM support
Activate PowerD (hi adaptive) to get the right CPU frequency and proper TurboBoost options as well*
Perhaps and if needed you could be also high up the mbuf size if needed and enough RAM is available
Create a /boot/loader.conf.local file for making all the custom set ups and tweaks persistent against upgrades

Not really a must be, but it could be helping out in some situations

Engineer

@BlueKobold:

Any thoughts as to where to start?

A proper WAN connection should be up and running. And why not connecting the pfSense to
the Internet, for sure for playing around it is not really necessary and urgent but for the fully
functionality of the pfSense box it should be the best option for you.

Activate TRIM Support in pfSense for the SSD if the SSD is capable of the TRIM support.

Activate PowerD (hi adaptive) to get the right CPU frequency and proper TurboBoost options as well.

Perhaps and if needed you could be also high up the mbuf size if needed and enough RAM is available

Create a /boot/loader.conf.local file for making all the custom set ups and tweaks persistent against upgrades

Not really a must be, but it could be helping out in some situations

It is currently on the Internet. I was testing hardware for a few days to make sure that nothing failed. In my long experience with electronics, they tend to fail within the first few weeks. Just wanted to get that out of the way.

I'll look into TRIM. Didn't know that option was available. However, I don't think that this would be causing the issue at hand.
Had already activated PowerD (hi adaptive). Have also tried other options with no effects (other than a slower GUI.
Had already changed mbuf size to the recommended size for Intel NIC's per a guide (found via Google several days ago). No change.
Will create the loader.conf.local file for what you recommend. If I downgrade (to try 2.1.5 for example), will this file hold persistent settings or will it only work for upgrades?

I might try a 2.2.5 snapshot for kicks. I really need stability and if I can't get it on the current version, I'll switch it up a bit. If I can't get it on those, I'll go back to regular router for now until I work it out. Wife hates interruptions in Internet and home television (Windows Media Center, Xbox 360 extenders and two HDHomerun primes). I have also turned off AES-NI for my N3700 for now to see if that has any effect.

The system seems to run for anywhere from an hour to 2 days before locking (again, console still works - I can remote into it via IPMI and it's responsive. I usually have to reboot from there).

One more thing: Apologies up front as I'm completely new to FreeBSD so I'll have to learn how to do things (like create files, edit files, etc). Looks like I can shell into the system and use 'ee' as an editor to create / edit files. Once created, I can use the built in editor of pfsense to edit them (or maybe I can create and edit them from pfsense). That's just stuff that I'll learn off to the side so please be patient as I may not know how to do some of these things but I'll google it to death to learn! :)

Jailer

@Engineer What are you using for a power supply?

Engineer

@Jailer…the built in 90W supply (new) for the Antec ISK-110. The Kill-a-Watt shows 11 watts pulled at the wall.

It could be power supply but since the console doesn't drop (nor did memtest or the system when I had a USB DVD burner attached), I would think the power seems OK.

I don't know if the SuperMicro logs power issues or not. IPMI page shows everything very good (very close to rated).

rexki

I have had a SuperMicro and Asus servers lock up on the IPMI access and the best help has always been to check for BIOS updates. Other firmwares on the system board from big iron True Raid controllers and the BMI firmware for the IPMI can fight the system board bios and cause this inability to access - It should not be related to pfSense as long as the cord is connected and you have a client on the same subnet with the BMI and client sharing the same gateway IP.

Also Java (Remote redirected video console app) can get really screwed up in a way I cannot decipher and reboots fix that usually.

Glad you confirmed 11 watts - that is an awesome number for a Quad core 2MB cache ITS board with plenty of horsepower - I think this N3700 has the same power as the C2558 and J1900 but with better power efficiency. I do now worry about ECC memory requirements since it not a File Server - its really an appliance.

Jailer

@Engineer:

@Jailer…the built in 90W supply (new) for the Antec ISK-110. The Kill-a-Watt shows 11 watts pulled at the wall.

It could be power supply but since the console doesn't drop (nor did memtest or the system when I had a USB DVD burner attached), I would think the power seems OK.

I don't know if the SuperMicro logs power issues or not. IPMI page shows everything very good (very close to rated).

Well my first guess would have been instability due to poor power supply but I can't find anything on the one used in that case. If it were me I'd still try to find another suitable power supply to try out and see if that helps.

Such are the pitfalls of blazing trails with new hardware…......

Jailer

@Engineer:

I don't know if the SuperMicro logs power issues or not. IPMI page shows everything very good (very close to rated).

Check the event log in the IPMI page and see if anything shows up there. It's under Server Health -> Event Log

Edit: Just noticed from your other post your mention of a usb DVD burner. No need for it, you can mount an ISO file in the KVM viewer and install from there. IPMI is awesome.

Engineer

@Jailer:

@Engineer:

I don't know if the SuperMicro logs power issues or not. IPMI page shows everything very good (very close to rated).

Check the event log in the IPMI page and see if anything shows up there. It's under Server Health -> Event Log

Edit: Just noticed from your other post your mention of a usb DVD burner. No need for it, you can mount an ISO file in the KVM viewer and install from there. IPMI is awesome.

Yea, I found that after the initial setup using the DVD burner. Having never used IPMI before, I didn't even have it connected to the network until after the first crash. I saw the virtual drive (using an image) in the viewer. I agree that IPMI is awesome so far.

I see nothing under the Event Log on the IPMI KVM viewer.

This is the first board that I've ever had that has an IPMI port….really do like this feature!!!!!

jjduru

Were you able to stabilize it by any chance? If yes, what did you do and with what pfsense version?

At this point I am debating between Supermicro X10SBA-L and Supermicro X11SBA-LN4F. I was dead set to get the first one, until I noticed that the Intel N3700 version has AES new instructions set.

So I am kind of waiting on your thread to see if it's yay or nay. Surprisingly, your thread is almost the only useful thread on the X11SBA-LN4F mainboard so you're opening new paths here.

Thanks,

Engineer

@jjduru:

Were you able to stabilize it by any chance? If yes, what did you and with what pfsense version?

At this point I am debating between Supermicro X10SBA-L and Supermicro X11SBA-LN4F. I was dead set to get the first one, until I noticed that the Intel N3700 version has AES new instructions set.

So I am kind of waiting on your thread to see if it's yay or nay. Surprisingly, your thread is almost the only useful thread on the X11SBA-LN4F mainboard so you're opening new paths here.

Thanks,

I'm tinkering with disabling hardware offload settings right now to see if it stabilizes. Takes quite a while to test because it has gone days between crashing before. Oh, and for the record, I had actually bought the X10SBA-L but cancelled the order because of the lower power and AES-NI instructions of the X11SBA-LN4F. It was $80 more but for future proofing (and a test showing the difference AES-NI adds to encrypted throughput), I chose the X11 board.

I'll keep this updated. I'm still on 2.2.4 for now.

jjduru

Is your mainboard having the latest BIOS version by any chance?

Engineer

@jjduru:

Is your mainboard having the latest BIOS version by any chance?

I went to the SuperMicro site the day I received the board. They didn't have ANY BIOS posted…for the main board nor the IPMI BIOS. I'll check again but don't think there is anything new there.

Edit: Nothing shows up (yet)....not even the first release BIOS or IPMI firmware.

jjduru

@Engineer,

I empathize with you for the troubles you're going through with that motherboard model, but in the end I guess I'll cough up the money to go for a Supermicro A1SRi-2558F. It's on long term support from Supermicro, FreeBSD 10 certified, AES-NI + QuickAssist. As you, I probably don't need headaches but reliability in the long shot.

Engineer

@jjduru:

@Engineer,

I empathize with you for the troubles you're going through with that motherboard model, but in the end I guess I'll cough up the money to go for a Supermicro A1SRi-2558F. It's on long term support from Supermicro, FreeBSD 10 certified, AES-NI + QuickAssist. As you, I probably don't need headaches but reliability in the long shot.

Well, it's been up longer than it has been so far (over two days now) and I suspect that it will get better as time goes on (and the software bugs get squashed). If the bug I posted about in another thread (quoted below) is the true culprit, it would seem that anything Intel related (at least with same driver) could be effected by this. Since FreeBSD development has recent posts about this, I would guess it would be solved fairly soon. It wouldn't be so bad to troubleshoot if the family (wife and kids) didn't come flying to me within 20 seconds of the Internet going down (even at 2:00am in the morning).

@Engineer:

Reading some more, I found this thread (https://forum.pfsense.org/index.php?topic=96325.0) which is Intel related but I'm not so sure it doesn't go beyond that. Seem FreeBSD 10.1 has an issue that needs to be corrected at the OS level that 'could' be causing this.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199174

States "In Progress" under status.

Not sure if this effects Realtek cards or not but reading though the freebsd.org link, the symptoms sound exactly like mine and yours too.

rexki

The previous customer build I did was on the SuperMicro A1SRi-2558F and it too would lock up in a similar manner on 2.2.4 - tried all the normal things except TRIM - it had an SSD like almost all of my pfsense builds. Also ECC memory - which could have been more of the culprit than a friend?

The iPMI would usually get locked up as well when it failed and the pfSense web GUI was a gonner until a hard power slap. The longest it ran was probably a week and it appeared the biggest culprits of crazy DNS_error_lookups / "lockups" were pfBlocker and Snort. If these were off it would go longer on its good runs.

This appliance is now in a box awaiting a customers Comcast second cable internet install - that may take months to years? It was a full SuperMicro solution hand built from component sku's to save money. SuperMicro nickel and dimes you on the component parts like a pci riser bracket to get this into their 1U case.

Eagerly awaiting stability notes on this X11SBA-LN4F - This unit would also make a great Camera IP recorder box since the Intel i210AT can be teamed and operating system support is the biggest range of any box I have ever seen - Windows XP to Windows 10 and FreeBSD 10.

Engineer

@rexki:

Eagerly awaiting stability notes on this X11SBA-LN4F - This unit would also make a great Camera IP recorder box since the Intel i210AT can be teamed and operating system support is the biggest range of any box I have ever seen - Windows XP to Windows 10 and FreeBSD 10.

I looked at trying to update the FreeBSD driver for the Intel I210AT and noticed that the driver download page for this chipset is very large (included all of the OS that you mentioned). However, it looks like the FreeBSD version is something that has to be compiled in and not simply installed over (from what little I looked at it). I have limited (and I mean limited) knowledge of FreeBSD and building an image is not a remote possibility right now.

rexki

My Self included in drivers for Linux/BSD

noted from the freebsg bug 199174 the board was a SuperMicro X7SPA-HF with 2x GigE Intel 82574L chipset - em(4) drivers?
The BMC chip is Winbond WPCM450

The boards corresponding sub components related to this post are:
1. GigE - Intel I210-AT on the Braswell and I354 on the C2558 - seems like em(4) was the driver for both and maybe the Intel 82574L as well?
2. BMC/iPMI - Aspeed AST2400

Also read this bug reports a BMC and Ethernet lockup at the same time as I have experienced??

It looks like they have it down to a TSO setting??

Background Research since I was not sure what TSO meant -:
In computer networking, large segment offload (LSO) is a technique for increasing outbound throughput of high-bandwidth network connections by reducing CPU overhead. It works by queuing up large buffers and letting the network interface card (NIC) split them into separate packets. The technique is also called TCP segmentation offload (TSO) when applied to TCP, or generic segmentation offload (GSO).

The inbound counterpart of large segment offload is large receive offload (LRO).

pfSense - System - Advanced - Networking (tab) - Network Interfaces (section):
1. Hardware TCP Segmentation offloading (TSOx) - TX mode - disabled checked
2. Hardware Large Receive Offloading (LRO) - RX mode - disabled checked

Engineer

It's interesting to note that I had both of those checked when it crashed the first two times (maybe 3rd too…don't remember). However, I have unchecked EVERYTHING in that section and it has run for nearly 3 days without a watchdog timeout on the LAN ports. I'll keep chugging along and see what happens.

On the IPMI note, it's separate and I don't think pfsense/FreeBSD has anything to do with it on my board, correct? I've had no issues getting to the console (even during the LAN crash / watchdog timeout) using the IPMI port.

rexki

"On the IPMI note, it's separate and I don't think pfsense/FreeBSD has anything to do with it on my board, correct? I've had no issues getting to the console (even during the LAN crash / watchdog timeout) using the IPMI port."

I am not sure since the bug report you mentioned as well as my experience with all SuperMicro boards (Q2 mentioned) all have done simultaneous lockups on LANS and BMC. You are correct in that I erroneously thought your BMC locked up simultaneously - my error - sorry about the rant on BMC.

How do you get Watch Dog to tell you the LAN has frozen?

I went to my "Service WatchDog" on pfSense and only have "ipferf" and that seems to be recommended only for internal trusted LAN ports?

Engineer

I don't know how to change the watchdog, it simply popped up on my console stating watchdog timeout on igb1, which is my LAN port. Also, I couldn't get to the GUI, DHCP stopped working as well as ANYTHING Internet. However, the IPMI port, separate from the 4 regular ports, was working fine so I logged into the console and noted it was completely responsive. After the watchdog message popped up, everything reset and started working again except IPV6 sites. A watchdog error for igb1 was also logged in the system log.

I suppose IPV6 would have eventually start working but I turned it off. I've since turned it back on.