Netgate 6100 Max unresponsive

[LoGman]

Hi,

This evening, I was unable to connect to the Netgate 6100 Max firewall using the IP address or a USB cable.

I saw the following messages in the Putty COM-port.

Checking media [Fail]
>>Start PXE over IPv4, Press [ESC] to EXIT...
Loading Network Lens...nd
Already started Lens...
Installing Network Lens...
Installing Network Lens...
EFI Network 1 for IPv4 (90-EC-77-**-**-**)
EFI Network 2 for IPv4 (90-EC-77-**-**-**)
      FS3: Alias(s):F0:
          Fv(E35C4A77-3D00-4337-A625-B980A9E00F6C)
      FS5: Alias(s):F0:
          Fv(F9657F27-CAE7-4951-A406-D286157B63EF)
      FS4: Alias(s):F0:
          Fv(E7082AE5-3036-407A-8060-FDF027F74632)
      FS0: Alias(s):F0:
          Fv(1643501D-ABD6-4ECC-85DE-3D95D23423FE)
      FS1: Alias(s):F0:
          Fv(50E81C5A-EAB2-495C-871E-EF46044C8959)
      FS2: Alias(s):F0:
          Fv(D6549619-FD0F-4464-B889-05D0A5D45666)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1C,0x0)/Msg(29,00)/Ctrl(0x0)
Loading Network Lens...
Already started
Installing Network Lens...

EFI Network 3 for IPv4 (90-EC-77-**-**-**)

The device does not respond to ping, and the internet connection is not working. The blue light is steadily on, meaning it is not blinking.

This was the first time this happened, and I had to unplug the power cable from the device. After the reboot, it started responding, and the internet connection was restored.

In the log: "Checking dump device /dev/label/swap0 for crash dumps ... no crash dumps on /dev/label/swap0."

=== START OF INFORMATION SECTION ===
Model Number:                       M.2 (P80) 3TE6
Serial Number:                      BC****
Firmware Version:                   V20B09
PCI Vendor/Subsystem ID:            0x1bc0
IEEE OUI Identifier:                0x24693e
Controller ID:                      0
NVMe Version:                       1.3
Number of Namespaces:               1
Namespace 1 Size/Capacity:          120,034,123,776 [120 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            24693e 66b902e7e5
Local Time is:                      Wed Sep 18 22:09:02 2024 EEST
Firmware Updates (0x02):            1 Slot
Optional Admin Commands (0x0006):   Format Frmw_DL
Optional NVM Commands (0x000f):     Comp Wr_Unc DS_Mngmt Wr_Zero
Log Page Attributes (0x03):         S/H_per_NS Cmd_Eff_Lg
Maximum Data Transfer Size:         256 Pages
Warning  Comp. Temp. Threshold:     70 Celsius
Critical Comp. Temp. Threshold:     85 Celsius

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        54 Celsius
Available Spare:                    100%
Available Spare Threshold:          1%
Percentage Used:                    3%
Data Units Read:                    34,896 [17.8 GB]
Data Units Written:                 10,712,446 [5.48 TB]
Host Read Commands:                 500,449
Host Write Commands:                161,962,603
Controller Busy Time:               2,005
Power Cycles:                       55
Power On Hours:                     6,553
Unsafe Shutdowns:                   38
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               80 Celsius
Temperature Sensor 2:               54 Celsius
Temperature Sensor 3:               56 Celsius
Temperature Sensor 4:               56 Celsius
Thermal Temp. 1 Transition Count:   7
Thermal Temp. 1 Total Time:         82082

The device is running the latest version (Netgate pfSense Plus 24.03-RELEASE (Patch 1) amd64 20240513-1217).

However, I am concerned about what caused this error and whether a component of the device (the disk?) might be failing?
If this is a disk failure, where can I purchase this specific disk (M.2 B+M Key) in the EU, as local online stores do not sell it?

[keyser]

@LoGman The console reflects a box that has restarted and was unable to find a bootdevice (It’s hanging in the “bios” boot procedure).

Likely your power off/on event made the SSD respond again, and thus booted succesfully.
Your SMART info does not seem to indicate an SSD that has been “worn” to death.

So likely it has either been a “fluke” which does happen with cheap consumer grade SSDs, or you have a DUD.

I would await to see if the issue happens again before I went out and purchased a new SSD. Although - they will likely only get even more troublesome to find with time (given the special key’ing), so that could be a factor to consider.

[LoGman]

@keyser

The M.2 drive in the device came with it when I purchased it.
Do you know if a similar drive can be purchased directly through sales@netgate.com?

The pfSense 6100 Max is still available.

[stephenw10]

We may have spare drives, drop an email to sales.

However any correctly keyed m.2 NVMe drive should work there.

You can also install to eMMC on the 6100 also as a backup.

[LoGman]

I was thinking of buying a backup disk for the device in case it breaks.

Will this work:
Link

SQF-C8BV2-128GDEDM
Category	Attribute	Value
SQFlash  	Capacity	128 GB
          	Connector	M.2 B+M Key
          	Flash Type	3D TLC BiCS5
          	Interface	PCIe / NVMe
          	Operational Temperature	-20 ~ 85℃ (Filter)
          	Shock	1,500G, Peak / 0.5 ms
          	SQFlash Utility	Support
          	Transfer Protocol	PCIe Gen. 3 x2
          	Vibration	20G, Peak / 80 ~ 2,000 Hz

[stephenw10]

I would expect it to. I've never tested that particular device though.