2.4.4 SMART nvme drive detection seems to fail

q54e3w

Testing a 2.4.4 install running on a x11sdv-8c with pair of nvme drives doesnt show up in SMART tests or devices although smartctl does see them from the command line.

nvme0: <Generic NVMe Device> port 0xb000-0xb0ff mem 0xe0e20000-0xe0e23fff at device 0.0 numa-domain 0 on pci6
pcib7: <ACPI PCI-PCI bridge> at device 1.0 numa-domain 0 on pci5
pci7: <ACPI PCI bus> numa-domain 0 on pcib7
nvme1: <Generic NVMe Device> port 0xa000-0xa0ff mem 0xe0d20000-0xe0d23fff at device 0.0 numa-domain 0 on pci7
pcib8: <ACPI PCI-PCI bridge> at device 3.0 numa-domain 0 on pci5
pci8: <ACPI PCI bus> numa-domain 0 on pcib8

nvme0@pci0:101:0:0:	class=0x010802 card=0xa801144d chip=0xa802144d rev=0x01 hdr=0x00
    vendor     = 'Samsung Electronics Co Ltd'
    device     = 'NVMe SSD Controller SM951/PM951'
    class      = mass storage
    subclass   = NVM
    bar   [10] = type Memory, range 64, base 0xe0e20000, size 16384, enabled
    bar   [18] = type I/O Port, range 32, base 0xb000, size 256, enabled
nvme1@pci0:102:0:0:	class=0x010802 card=0xa801144d chip=0xa802144d rev=0x01 hdr=0x00
    vendor     = 'Samsung Electronics Co Ltd'
    device     = 'NVMe SSD Controller SM951/PM951'
    class      = mass storage
    subclass   = NVM
    bar   [10] = type Memory, range 64, base 0xe0d20000, size 16384, enabled
    bar   [18] = type I/O Port, range 32, base 0xa000, size 256, enabled

smartctl -i /dev/nvme0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZ1WV480HCGL-000MV
Serial Number:                      S1Y0NYAGxxxxx
Firmware Version:                   BXU85M1Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Controller ID:                      5197
Number of Namespaces:               1
Namespace 1 Size/Capacity:          480,103,981,056 [480 GB]
Namespace 1 Utilization:            4,931,944,448 [4.93 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            382500 2045404200
Local Time is:                      Sat Oct 13 13:44:20 2018 PDT

pool: zroot
 state: ONLINE
  scan: scrub repaired 0 in 0h0m with 0 errors on Mon Oct  1 01:10:00 2018
config:

	NAME        STATE     READ WRITE CKSUM
	zroot       ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    nvd0p4  ONLINE       0     0     0
	    nvd1p4  ONLINE       0     0     0

stephenw10

Looks like it's only looking for ad, ada or da devices. nvme still relatively new:

https://github.com/pfsense/pfsense/blob/c0787ee92aeaa51cee0c03806405cfe8790ec2ca/src/etc/inc/util.inc#L2605

Looks like a very easy fix though. I opened a ticket to track it:
https://redmine.pfsense.org/issues/9042

Steve

jimp

I don't have anything here with nvme/nvd but the man page isn't quite clear how those will show up. It seems to have both nvdX and nvmeX interfaces, and it says the nvd devices are the GEOMs used for partitioning.

Do both show up for you? ls -l /dev/nvd* /dev/nvme*
Can you run smartctl against both?

ZFS is using the nvd version which agrees with the man page, so that seems to be what we should be showing on the page, but your example smartctl command disagrees.

q54e3w

@jimp said in 2.4.4 SMART nvme drive detection seems to fail:

ls -l /dev/nvd* /dev/nvme*

ls -l /dev/nvd* /dev/nvme*
crw-r-----  1 root  operator  0x4e Oct 13 10:04 /dev/nvd0
crw-r-----  1 root  operator  0x4f Oct 13 10:04 /dev/nvd0p1
crw-r-----  1 root  operator  0x50 Oct 13 10:04 /dev/nvd0p2
crw-r-----  1 root  operator  0x51 Oct 13 10:04 /dev/nvd0p3
crw-r-----  1 root  operator  0x52 Oct 13 10:04 /dev/nvd0p4
crw-r-----  1 root  operator  0x59 Oct 13 10:04 /dev/nvd1
crw-r-----  1 root  operator  0x5a Oct 13 10:04 /dev/nvd1p1
crw-r-----  1 root  operator  0x5b Oct 13 10:04 /dev/nvd1p2
crw-r-----  1 root  operator  0x5c Oct 13 10:04 /dev/nvd1p3
crw-r-----  1 root  operator  0x5d Oct 13 10:04 /dev/nvd1p4
crw-------  1 root  wheel     0x2e Oct 13 10:04 /dev/nvme0
crw-------  1 root  wheel     0x4c Oct 13 10:04 /dev/nvme0ns1
crw-------  1 root  wheel     0x2f Oct 13 10:04 /dev/nvme1
crw-------  1 root  wheel     0x58 Oct 13 10:04 /dev/nvme1ns1

Is this what you were after - let me know if you need more specific commands running.

smartctl /dev/nvd0
...
/dev/nvd0: Unable to detect device type
Please specify device type with the -d option.

smartctl /dev/nvd0p1
.../dev/nvd0p1: Unable to detect device type
Please specify device type with the -d option.

smartctl /dev/nvme0
NVMe device successfully opened

smartctl /dev/nvme0ns1
NVMe device successfully opened

jimp

Curious. I guess the page should just show the nvme devices then and not nvd. I'll push a change for that shortly.

q54e3w

I tried to force the type on 'nvd0' but no joy either.

smartctl -i -d nvme /dev/nvd0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

Smartctl open device: /dev/nvd0 failed: NVMe controller controller/namespace ids must begin with '/dev/nvme'

jimp

I pushed the change but after doing so I had a second thought. The smart disk list is checking the contents of kern.disks and I bet that only shows the nvd devices.

What does the output of sysctl kern.disks show?

q54e3w

I was just googling reading about this.... your thinking is correct.

sysctl kern.disks
kern.disks: nvd1 nvd0

jimp

OK, I found a better way to do it. Watch the redmine issue for a new patch shortly.

q54e3w

Thats fixed it, thanks Jim.