Ad0 errors with CompactFlash Sandisk cards and Soekris net5501

fabix · Sep 23, 2011, 1:26 PM

Hello,

On a pfsense 2.0, I have a lot of I/O errors:
ad0: FAILURE - READ status=51 <ready,dsc,error>error=10 <nid_not_found>LBA=7813119
ad0: FAILURE - READ status=51 <ready,dsc,error>error=10 <nid_not_found>LBA=7813119
…

$ /sbin/atacontrol list
ATA channel 0:
Master: ad0 <sandisk compactflash="" 200x="" hdx="" 6.03="">ATA/ATAPI revision 0
Slave: no device present
ATA channel 1:
Master: no device present
Slave: no device present

I tried several cards of this type, the same errors appear. pfSense seems to work normally.

The host is a Soekris net5501-60. I don'k know what is responsible of these errors and if this is a serious problem. Has anyone an idea to solve it ?

Regards.

Other informations:

The system has been installed with a dd of the file pfSense-2.0-RELEASE-2g-i386-nanobsd.img.gz ungzipped on the CF card:
2.0-RELEASE (i386) built on Tue Sep 13 18:02:53 EDT 2011

=== START OF INFORMATION SECTION ===
Device Model: SanDisk CompactFlash 200x
Serial Number: CEZ040811192649
Firmware Version: HDX 6.03
User Capacity: 4,000,317,440 bytes [4.00 GB]
Sector Size: 512 bytes logical/physical
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: [No Information Found]
ATA Standard is: [No Information Found]
Local Time is: Fri Sep 16 22:28:21 2011 CEST

$ /sbin/atacontrol mode ad0
current mode = PIO4

$ grep dma /boot/loader.conf
hw.ata.atapi_dma="0"
hw.ata.ata_dma="0"</sandisk></nid_not_found></ready,dsc,error></nid_not_found></ready,dsc,error>

BrianX · Oct 1, 2011, 7:07 PM

i am getting the same errors. tried two different cards. they are transcend industrial CF cards in an ide carrier in a dual atom d525 system. (hacom mars)

dmesg | grep ad0
ad0: 1943MB <transcend 20070418="">at ata0-master PIO4
ad0: FAILURE - READ status=ff <busy,ready,dma_ready,dsc,drq,correctable,index,error>error=0 LBA=532431
ad0: FAILURE - WRITE status=ff <busy,ready,dma_ready,dsc,drq,correctable,index,error>error=58 <uncorrectable,nid_not_found,media_change_reqest>LBA=88375
ad0: TIMEOUT - WRITE retrying (1 retry left) LBA=88631
ad0: FAILURE - WRITE status=ff <busy,ready,dma_ready,dsc,drq,correctable,index,error>error=58 <uncorrectable,nid_not_found,media_change_reqest>LBA=88887
ad0: TIMEOUT - WRITE retrying (0 retries left) LBA=88631
ad0: FAILURE - READ status=ff <busy,ready,dma_ready,dsc,drq,correctable,index,error>error=0 LBA=233943

/sbin/atacontrol list
ATA channel 0:
Master: ad0 <transcend 20070418="">ATA/ATAPI revision 4
Slave: no device present
ATA channel 2:
Master: no device present
Slave: no device present
ATA channel 3:
Master: no device present
Slave: no device present

The system has been installed with a dd of the file pfSense-2.0-RELEASE-2g-i386-nanobsd.img.gz ungzipped on the CF card:
2.0-RELEASE (i386) built on Tue Sep 13 18:02:53 EDT 2011

=== START OF INFORMATION SECTION ===
Model Family: Transcend CompactFlash Cards
Device Model: TRANSCEND
Serial Number: 20090309 00001386
Firmware Version: 20070418
User Capacity: 2,038,063,104 bytes [2.03 GB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: 4
ATA Standard is: Exact ATA specification draft version not indicated
Local Time is: Sat Oct 1 14:55:49 2011 EDT

/sbin/atacontrol mode ad0
current mode = PIO4

grep dma /boot/loader.conf
hw.ata.atapi_dma="0"
hw.ata.ata_dma="0"</transcend></busy,ready,dma_ready,dsc,drq,correctable,index,error></uncorrectable,nid_not_found,media_change_reqest></busy,ready,dma_ready,dsc,drq,correctable,index,error></uncorrectable,nid_not_found,media_change_reqest></busy,ready,dma_ready,dsc,drq,correctable,index,error></busy,ready,dma_ready,dsc,drq,correctable,index,error></transcend>

jimp · Oct 24, 2011, 1:45 PM

Can you find the real size of the available space on those cards?

We have been shrinking the image sizes more and more lately because the cards have been shrinking. The images for 2.0.1+ are smaller than ever, because Sandisk has been shipping cards with sizes smaller than those being shipped in current hardware from Netgate and such. We have given the folks at Netgate a new set of images that have slightly smaller sizes, which is likely what you need here.

I don't know if Netgate has posted theirs anywhere yet. We'll be releasing 2.0.1 in the very near future, so it'd be worth trying once we get a new set of images out.

fabix · Oct 24, 2011, 2:00 PM

I don't know how to get more details. Even if the card is smaller than 4GB, I used a 2GB image to be sure it can be contained on the card.

Wim Vandeputte confirmed me last SanDisk cards are smaller than 4GB. He adviced me to install a smaller image of pfSense. But that doesn't solve the problem.

Actually, I installed recently a new firewall (same hardware) and I don't have these errors. So, maybe it's a problem of card…

jimp · Oct 24, 2011, 2:24 PM

That may be, then… But it did look like the same sort of error (can't read repeatedly from the same high LBA) that I've seen when someone has written an image too large to the card.

nexus_VI · Oct 31, 2011, 1:10 PM

I'm seeing this same error here using a SanDisk card with the 4G image, namely:

Master: ad0 <sandisk sdcfh-004g="" hdx="" 6.03="">ATA/ATAPI revision 0

The main question is, will I face problems when operating the system like this, should I wait for the 2.0.1 images?</sandisk>

jimp · Oct 31, 2011, 1:15 PM

If you are running the 4G version and getting those errors, then yeah you'll want to wait if you're seeing those errors.

If you are running a 1G/2G version and getting errors, then you may have some other issue with the card not related to the size.

wm408 · Nov 1, 2011, 8:00 PM

In a net6501, I am having sort of similar errors on a 4G mSATA card that I purchased from Soekris, its a Transcend 4g MSA500 (I think?, something like this). I tested 2G and 4G nanobsd images on it with the same result.

Here is the link to my post: http://forum.pfsense.org/index.php/topic,42028.msg219929.html#msg219929

And here is some of the output without having to go to the link

_ad4: 3775MB <ts4gmsa500 20101212="">at ata2-master PIO4 SATA 3Gb/s
uhub6: 1 port with 1 removable, self powered
uhub3: 3 ports with 3 removable, self powered
uhub7: 3 ports with 3 removable, self powered
ad4: TIMEOUT - READ retrying (1 retry left) LBA=3861855
Trying to mount root from ufs:/dev/ufs/pfsense0
ad4: TIMEOUT - READ retrying (1 retry left) LBA=3452247
Configuring crash dumps…
Mounting filesystems...
Setting up embedded specific environment...ad4: TIMEOUT - READ retrying (1 retry left) LBA=3450831
ad4: TIMEOUT - READ retrying (0 retries left) LBA=3450831
ad4: FAILURE - READ timed out LBA=3450831
g_vfs_done():ufs/pfsense0[READ(offset=1766785024, length=14336)]error = 5
vnode_pager_getpages: I/O read error
ad4: TIMEOUT - READ retrying (1 retry left) LBA=1514407
ad4: TIMEOUT - READ retrying (0 retries left) LBA=1514407
ad4: FAILURE - READ timed out LBA=1514407
g_vfs_done():ufs/pfsense0[READ(offset=775335936, length=2048)]error = 5
/etc/rc.embedded: mdmfs: Input/output error
ad4: TIMEOUT - READ retrying (1 retry left) LBA=3450831
ad4: TIMEOUT - READ retrying (0 retries left) LBA=3450831
ad4: FAILURE - READ timed out LBA=3450831
g_vfs_done():ufs/pfsense0[READ(offset=1766785024, length=14336)]error = 5
vnode_pager_getpages: I/O read error
ad4: TIMEOUT - READ retrying (1 retry left) LBA=1514407
/etc/rc.embedded: mdmfs: Input/output error
ad4: TIMEOUT - READ retrying (1 retry left) LBA=1866375
done.
ad4: TIMEOUT - READ retrying (1 retry left) LBA=311183

___
/ f
/ p _/ Sense
_/
__/

Welcome to pfSense 2.0-RELEASE …

ad4: TIMEOUT - READ retrying (1 retry left) LBA=3447463
ad4: TIMEOUT - READ retrying (1 retry left) LBA=1814527
Creating symlinks...g_vfs_done():ufs/cf[READ(offset=13193216, length=4096)]error = 5
g_vfs_done():ufs/cf[READ(offset=13193216, length=4096)]error = 5
mkdir: /cf/conf: Input/output error
.g_vfs_done():ufs/cf[READ(offset=13193216, length=4096)]error = 5
cat: /cf/conf/config.xml: Input/output error
ad4: TIMEOUT - READ retrying (1 retry left) LBA=1764455
g_vfs_done():ufs/cf[READ(offset=13193216, length=4096)]error = 5
cat: /cf/conf/config.xml: Input/output error
ad4: TIMEOUT - READ retrying (1 retry left) LBA=1503551
..done.
ad4: TIMEOUT - READ retrying (1 retry left) LBA=327975
ad4: TIMEOUT - READ retrying (0 retries left) LBA=327975
ad4: FAILURE - READ timed out LBA=327975
g_vfs_done():ufs/pfsense0[READ(offset=167882752, length=6144)]error = 5
vnode_pager_getpages: I/O read error
/etc/rc: /usr/bin/cap_mkdb: Input/output error</ts4gmsa500>_

Honeybadger · Oct 31, 2011, 11:11 PM

I was getting these errors until I put my OS on a 2g (2g image) Sandisk Extreme IV card.

It is designed for heavy writing in warm environments (SLR cameras) and my problem went away.

Hope your problem does the same…

wm408 · Nov 1, 2011, 5:19 PM

the net6501's don't support CF cards. mSATA is an option which seems fine. I have tried 4G and 2G on the 4G card, no luck.

I am going to try a different mSATA card.

@Honeybadger:

I was getting these errors until I put my OS on a 2g (2g image) Sandisk Extreme IV card.

It is designed for heavy writing in warm environments (SLR cameras) and my problem went away.

Hope your problem does the same…

jimp · Nov 1, 2011, 5:31 PM

One thing I'd be wary of in the net6501 case is putting an mSATA card in the slot by the CPU. With no active cooling in the case, the CPU gets quite hot and the card in the slot next to the CPU will get a bit of heat from that as well. In the wrong environment I could see that killing an mSATA drive (or at least harming its performance in some way)

nexus_VI · Nov 1, 2011, 7:11 PM

I tried with the 2G image, sadly the results are the same:

ad0: FAILURE - READ status=51 <ready,dsc,error>error=10 <nid_not_found>LBA=7813119
ad0: FAILURE - READ status=51 <ready,dsc,error>error=10 <nid_not_found>LBA=7813119
ad0: FAILURE - READ status=51 <ready,dsc,error>error=10 <nid_not_found>LBA=7813119

Is buying a new CF card my best bet, or even my only choice?</nid_not_found></ready,dsc,error></nid_not_found></ready,dsc,error></nid_not_found></ready,dsc,error>

wm408 · Nov 1, 2011, 8:05 PM

Thx Jimp. Yeah slot 00 is the farthest from, 01 being closer. The run time on this mSATA card is so little that I don't think that's a factor yet for me. But I will follow that rule for sure.

Hey Jimp with this card, I am not sure if its relevant but I found a thread with sort of similar problems to mine here: http://freebsd.1045724.n5.nabble.com/HDD-READ-Timeout-messages-after-8-2-Upgrade-td3733475.html

Adding ahci_load="YES" to loader.conf solved the problem

Do you think its relevant to the problem I am having?

Right now I am trying to mount the *.img file to edit the file via:

mount -t auto -o loop,offset=$((63*512)) /media/B4E1-5C16/pfSense-2.0-RELEASE-2g-i386-20110913-1802-nanobsd.img /mnt/test

If you have thoughts let me know!

@jimp:

One thing I'd be wary of in the net6501 case is putting an mSATA card in the slot by the CPU. With no active cooling in the case, the CPU gets quite hot and the card in the slot next to the CPU will get a bit of heat from that as well. In the wrong environment I could see that killing an mSATA drive (or at least harming its performance in some way)

depiman · Nov 2, 2011, 11:04 AM

I have exactly the same problem on a Soekris net4801. A 4 GB image on a new 4 GB SanDisk Ultra card gives a write error at the end. The same image on an older SanDisk Ultra II card goes well. The newer card boots and runs with the 4 GB image, but is unstable. On the old card it's stable. The 2 GB image on the newer card runs, but still to early to say if it's stable. Fact is that it's not relyable for my production environment (…) Big question: where's the problem? On the image, the CF card or the Soekris?

Metu69salemi · Nov 2, 2011, 11:18 AM

@depiman:

Big question: where's the problem? On the image, the CF card or the Soekris?

Shaking from the sleeve(meaning this more like i feel than i know): CF card

depiman · Nov 2, 2011, 12:26 PM

@Metu69salemi:

Shaking from the sleeve(meaning this more like i feel than i know): CF card

I think so to. And that looks the same as Wim van der Putten his opinion: newer SanDisk has less bytes then before. Or something else is wrong. So we can say newer doesn't have to be better ;-) Sh*t, I bought ten of those cards two months ago! :-(

Metu69salemi · Nov 2, 2011, 12:32 PM

Use 2GB images, and your problem is solved

wm408 · Nov 2, 2011, 10:39 PM

Tried that, no luck.

@Metu69salemi:

Use 2GB images, and your problem is solved

Metu69salemi · Nov 2, 2011, 10:43 PM

Okay then it has to be something else on that card. Can you use it with your camera or windows machine without errors?

wm408 · Nov 2, 2011, 10:46 PM

I haven't tried moving files in and out of it on Windows, but it seems to format it just fine and then detect it as a drive. Also in Linux if I format, seems like I can access it thereafter.

@Metu69salemi:

Okay then it has to be something else on that card. Can you use it with your camera or windows machine without errors?