Skip to contentSkip to conversation

nvme nvme0: I/O 0 (I/O Cmd) QID 1 timeout, aborting, source drive corruption observed

DISCUSSION J. Hart 1 participant 1 message Dec 17, 2022 NVMe

lore
Share & tools
lore

J. Hart

author
#0 lore raw

I have done an fsck check on the /dev/nvme0n1p3 file system after the rsync invocation referenced earlier. In the first run I found errors which fsck should have repaired if I understand correctly. In repeating the fsck invocation immediately afterwards, I found errors again each time. This was done using the replacement Samsung nvme ssd (Samsung 970 EVO Plus 500G).

Here is the output :

-bash-3.2# fsck -f -C /dev/nvme0n1p3
fsck from util-linux 2.34
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure

Pass 3: Checking directory connectivity

Pass 4: Checking reference counts
Pass 5: Checking group summary information

Block bitmap differences: -87163220 -87163384 -87187960 -122079572 -122079736 Fix<y>? yes

/dev/nvme0n1p3: ***** FILE SYSTEM WAS MODIFIED *****
/dev/nvme0n1p3: 699563/30523392 files (0.1% non-contiguous), 13841458/122079744 blocks -bash-3.2# fsck -f -C /dev/nvme0n1p3
fsck from util-linux 2.34
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure

Pass 3: Checking directory connectivity

Pass 4: Checking reference counts

Pass 5: Checking group summary information

Block bitmap differences: -87163220 -87163384 -122079736

Fix<y>? yes

/dev/nvme0n1p3: ***** FILE SYSTEM WAS MODIFIED *****
/dev/nvme0n1p3: 699563/30523392 files (0.1% non-contiguous), 13841458/122079744 blocks -bash-3.2# fsck -f -C /dev/nvme0n1p3
fsck from util-linux 2.34
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure

Pass 3: Checking directory connectivity

Pass 4: Checking reference counts
Pass 5: Checking group summary information

Block bitmap differences: -87163220 -87163384 -122079572 -122079736

Fix<y>? yes

/dev/nvme0n1p3: ***** FILE SYSTEM WAS MODIFIED *****
/dev/nvme0n1p3: 699563/30523392 files (0.1% non-contiguous), 13841458/122079744 blocks

I also tried loading a smaller partition on the Samsung card at /dev/nvme0n1p3. The copy stopped with a "no space left on device" error, which should not have been possible as the source device is a 32MB partition, and the destination partition on the nvme ssd is a 64MB partition. The two files to be transferred were very small and could not have accounted for this as they totaled less than 5MB. I found file system damage on the nvme destination partition in this case as well. It also occurred repeatedly. I am still investigating this last case.

In no instance did I note any otherwise unusual log messages or errors from the nvme driver.

I do not yet know if there has been any damage to any other filesystems, but I will check.

J. Hart