Skip to contentSkip to conversation

block device write verify test 2nd attempt

PATCH In review Harry Butterworth 2 participants 9 messages May 23, 2006 – May 24, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 block device write verify test 2nd attempt

The only difference from the last version of the patch is that the minor version number in configure.ac is incremented.

Quoted 1 line

From the patch:

+# This test imports a ram disk device as a physical device into a domU. +# The domU initialises the ram disk with data from /dev/urandom and calculates
+# the md5 checksum of the data (using tee as it is written so as to avoid
+# reading it back from the device which might potentially mask problems).
+# The domU is stopped and the md5 checksum of the data on the device is +# calculated by dom0. The test succeeds if the checksums match, indicating
+# that all the data written by domU was sucessfully committed to the device.
+

This patch also enables tee and fancy head in busybox on the ramdisk. I have tested the patch with both make existing' where the tests run but the new test fails because the ramdisk is missing tee and fancy head and make` where the test passes successfully.

The patch also installs the block-integrity tests in the default test set so they get executed.

tools/xm-test/configure.ac +1 -1
diff -r 14717dedba02 -r 6394ff30dfc0 tools/xm-test/configure.ac
@@ -1,7 +1,7 @@
 # xm-test configure.ac input script
 
 # Basic header information
-AC_INIT([xm-test], [0.7.1])
+AC_INIT([xm-test], [0.7.2])
 AM_INIT_AUTOMAKE([1.7 foreign])
 
 # Check for dependencies
tools/xm-test/grouptest/default +1 -0
diff -r 14717dedba02 -r 6394ff30dfc0 tools/xm-test/grouptest/default
@@ -1,6 +1,7 @@
 block-create
 block-destroy
 block-list
+block-integrity
 console
 create
 destroy
tools/xm-test/ramdisk/configs/busybox +3 -3
diff -r 14717dedba02 -r 6394ff30dfc0 tools/xm-test/ramdisk/configs/busybox
@@ -92,7 +92,7 @@
 CONFIG_FALSE=y
 # CONFIG_FOLD is not set
 CONFIG_HEAD=y
-# CONFIG_FEATURE_FANCY_HEAD is not set
+CONFIG_FEATURE_FANCY_HEAD=y
 CONFIG_HOSTID=y
 CONFIG_ID=y
 CONFIG_INSTALL=y
@@ -127,8 +127,8 @@
 CONFIG_SYNC=y
 CONFIG_TAIL=y
 CONFIG_FEATURE_FANCY_TAIL=n
-CONFIG_TEE=n
-CONFIG_FEATURE_TEE_USE_BLOCK_IO=n
+CONFIG_TEE=y
+CONFIG_FEATURE_TEE_USE_BLOCK_IO=y
 CONFIG_TEST=y
 
 #
tools/xm-test/tests/block-integrity/Makefile.am +2 -1
diff -r 14717dedba02 -r 6394ff30dfc0 tools/xm-test/tests/block-integrity/Makefile.am
@@ -1,7 +1,8 @@
 
 SUBDIRS =
 
-TESTS = 01_block_device_read_verify.test
+TESTS = 01_block_device_read_verify.test \
+        02_block_device_write_verify.test
 
 XFAIL_TESTS = 
 
tools/xm-test/tests/block-integrity/02_block_device_write_verify.py new file +63 -0

63 lines not on this page · raw message

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw
Quoted 19 lines · Harry Butterworth

The only difference from the last version of the patch is that the minor version number in configure.ac is incremented.

Quoted 1 line

From the patch:

(quoted diff) +2 -0
+# This test imports a ram disk device as a physical device into a domU.
+# The domU initialises the ram disk with data from /dev/urandom and

calculates
+# the md5 checksum of the data (using tee as it is written so as to avoid
+# reading it back from the device which might potentially mask problems).

(quoted diff) +2 -0
+# The domU is stopped and the md5 checksum of the data on the device is
+# calculated by dom0.  The test succeeds if the checksums match,

indicating
+# that all the data written by domU was sucessfully committed to the device.
+

Quoted 4 lines

This patch also enables tee and fancy head in busybox on the ramdisk. I have tested the patch with both make existing' where the tests run but the new test fails because the ramdisk is missing tee and fancy head and make` where the test passes successfully.

Why don't you use dd instead of head -c?

Why don't you just fix the size of the datablock that you write to the ramdisk, instead of determining the current size of the ramdisk with cat /dev/hda1 | wc -c?

Ewan.

#2 lore raw
Quoted 21 lines · Ewan Mellor
Quoted 19 lines · Harry Butterworth

The only difference from the last version of the patch is that the minor version number in configure.ac is incremented.

Quoted 1 line

From the patch:

(quoted diff) +2 -0
+# This test imports a ram disk device as a physical device into a domU.
+# The domU initialises the ram disk with data from /dev/urandom and

calculates
+# the md5 checksum of the data (using tee as it is written so as to avoid
+# reading it back from the device which might potentially mask problems).

(quoted diff) +2 -0
+# The domU is stopped and the md5 checksum of the data on the device is
+# calculated by dom0.  The test succeeds if the checksums match,

indicating
+# that all the data written by domU was sucessfully committed to the device.
+

Quoted 6 lines
Quoted 4 lines

This patch also enables tee and fancy head in busybox on the ramdisk. I have tested the patch with both make existing' where the tests run but the new test fails because the ramdisk is missing tee and fancy head and make` where the test passes successfully.

Why don't you use dd instead of head -c?

I tried using dd with a block size of 1 and a count of the right number of bytes but the test was very slow. I didn't want to assume a 512b block size and I'm not very good at shell script so didn't manage to work out how to do it better.

Quoted 3 lines

Why don't you just fix the size of the datablock that you write to the ramdisk, instead of determining the current size of the ramdisk with cat /dev/hda1 | wc -c?

I wanted to test writing at the device limits. Sometimes there are off by one errors that mean you can't write the last sector of a block device.

cat | wc -c was the most robust way I could think of for getting the size.

Quoted 3 lines

Ewan.

#3 lore raw
Quoted 29 lines · Harry Butterworth
Quoted 21 lines · Ewan Mellor
Quoted 19 lines · Harry Butterworth

The only difference from the last version of the patch is that the minor version number in configure.ac is incremented.

Quoted 1 line

From the patch:

(quoted diff) +2 -0
+# This test imports a ram disk device as a physical device into a domU.
+# The domU initialises the ram disk with data from /dev/urandom and

calculates
+# the md5 checksum of the data (using tee as it is written so as to avoid
+# reading it back from the device which might potentially mask problems).

(quoted diff) +2 -0
+# The domU is stopped and the md5 checksum of the data on the device is
+# calculated by dom0.  The test succeeds if the checksums match,

indicating
+# that all the data written by domU was sucessfully committed to the device.
+

Quoted 6 lines
Quoted 4 lines

This patch also enables tee and fancy head in busybox on the ramdisk. I have tested the patch with both make existing' where the tests run but the new test fails because the ramdisk is missing tee and fancy head and make` where the test passes successfully.

Why don't you use dd instead of head -c?

Quoted 4 lines

I tried using dd with a block size of 1 and a count of the right number of bytes but the test was very slow. I didn't want to assume a 512b block size and I'm not very good at shell script so didn't manage to work out how to do it better.

dd bs=<number> count=1 should do just fine.

Quoted 10 lines
Quoted 3 lines

Why don't you just fix the size of the datablock that you write to the ramdisk, instead of determining the current size of the ramdisk with cat /dev/hda1 | wc -c?

I wanted to test writing at the device limits. Sometimes there are off by one errors that mean you can't write the last sector of a block device.

cat | wc -c was the most robust way I could think of for getting the size.

How about blockdev --getsize64 /dev/ram1?

Ewan.

#4 lore raw
Quoted 31 lines · Ewan Mellor
Quoted 29 lines · Harry Butterworth
Quoted 27 lines · Ewan Mellor
Quoted 19 lines · Harry Butterworth

The only difference from the last version of the patch is that the minor version number in configure.ac is incremented.

Quoted 1 line

From the patch:

(quoted diff) +2 -0
+# This test imports a ram disk device as a physical device into a domU.
+# The domU initialises the ram disk with data from /dev/urandom and

calculates
+# the md5 checksum of the data (using tee as it is written so as to avoid
+# reading it back from the device which might potentially mask problems).

(quoted diff) +2 -0
+# The domU is stopped and the md5 checksum of the data on the device is
+# calculated by dom0.  The test succeeds if the checksums match,

indicating
+# that all the data written by domU was sucessfully committed to the device.
+

Quoted 4 lines

This patch also enables tee and fancy head in busybox on the ramdisk. I have tested the patch with both make existing' where the tests run but the new test fails because the ramdisk is missing tee and fancy head and make` where the test passes successfully.

Why don't you use dd instead of head -c?

Quoted 6 lines
Quoted 4 lines

I tried using dd with a block size of 1 and a count of the right number of bytes but the test was very slow. I didn't want to assume a 512b block size and I'm not very good at shell script so didn't manage to work out how to do it better.

dd bs=<number> count=1 should do just fine.

The ram disk is 64MB, I thought that this would require a 64MB buffer in domU which I thought was too big. It would be better to use dd with the correct block size and count of blocks.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

Why don't you just fix the size of the datablock that you write to the ramdisk, instead of determining the current size of the ramdisk with cat /dev/hda1 | wc -c?

Quoted 8 lines
Quoted 6 lines

I wanted to test writing at the device limits. Sometimes there are off by one errors that mean you can't write the last sector of a block device.

cat | wc -c was the most robust way I could think of for getting the size.

How about blockdev --getsize64 /dev/ram1?

blockdev and stat are not available in the ramdisk. I could use blockdev to get the blocksize and number of blocks in dom0 and then pass the values to domU but it would be better to obtain all the information in domU since that would test whether the size of the device is correctly passed from the back to the front by the blk driver.

The test as it stands does check that aspect of the blk driver operation and is only a factor of 2 slower than the limit and the python code is trivial so I thought it was reasonable.

Can you think of any way to get the block size and number of blocks using tools available on the ramdisk?

Quoted 8 lines

Ewan.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#5 lore raw
Quoted 19 lines · Harry Butterworth
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Why don't you just fix the size of the datablock that you write to the ramdisk, instead of determining the current size of the ramdisk with cat /dev/hda1 | wc -c?

Quoted 8 lines
Quoted 6 lines

I wanted to test writing at the device limits. Sometimes there are off by one errors that mean you can't write the last sector of a block device.

cat | wc -c was the most robust way I could think of for getting the size.

How about blockdev --getsize64 /dev/ram1?

blockdev and stat are not available in the ramdisk. I could use blockdev to get the blocksize and number of blocks in dom0 and then pass the values to domU but it would be better to obtain all the information in domU since that would test whether the size of the device is correctly passed from the back to the front by the blk driver.

Quoted 6 lines

The test as it stands does check that aspect of the blk driver operation and is only a factor of 2 slower than the limit and the python code is trivial so I thought it was reasonable.

Can you think of any way to get the block size and number of blocks using tools available on the ramdisk?

dd if=/dev/urandom of=/dev/ram1 bs=512 count=cat /sys/block/ram1/size

The value in /sys/block/ram1/size is defined to be in multiples of 512 bytes, as rd.c shows:

		/* rd_size is given in kB */
                ...
		set_capacity(disk, rd_size * 2);

Catting the whole device to determine its size is vile, and would slow down the regression suite unnecessarily. Modifying the ramdisk is not verboten, but it is a right pain in the backside for everyone, particularly me because I need to build a new one and upload it and all of its source (because of the GPL) to xm-test.xensource.com.

Ewan.

#6 lore raw
Quoted 28 lines · Ewan Mellor
Quoted 19 lines · Harry Butterworth
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Why don't you just fix the size of the datablock that you write to the ramdisk, instead of determining the current size of the ramdisk with cat /dev/hda1 | wc -c?

Quoted 8 lines
Quoted 6 lines

I wanted to test writing at the device limits. Sometimes there are off by one errors that mean you can't write the last sector of a block device.

cat | wc -c was the most robust way I could think of for getting the size.

How about blockdev --getsize64 /dev/ram1?

blockdev and stat are not available in the ramdisk. I could use blockdev to get the blocksize and number of blocks in dom0 and then pass the values to domU but it would be better to obtain all the information in domU since that would test whether the size of the device is correctly passed from the back to the front by the blk driver.

Quoted 6 lines

The test as it stands does check that aspect of the blk driver operation and is only a factor of 2 slower than the limit and the python code is trivial so I thought it was reasonable.

Can you think of any way to get the block size and number of blocks using tools available on the ramdisk?

Quoted 8 lines

dd if=/dev/urandom of=/dev/ram1 bs=512 count=cat /sys/block/ram1/size

The value in /sys/block/ram1/size is defined to be in multiples of 512 bytes, as rd.c shows:

		/* rd_size is given in kB */
                ...
		set_capacity(disk, rd_size * 2);

OK, thanks I'll use that.

Quoted 4 lines

Modifying the ramdisk is not verboten,
but it is a right pain in the backside for everyone, particularly me because I need to build a new one and upload it and all of its source (because of the GPL) to xm-test.xensource.com.

The current test is modifying the ramdisk to add tee (as well as fancy head which we don't need anymore).

Are you going to accept the patch if it requires the addition of tee to the ramdisk? Or should I change the test to read the data back in domU---which I was specifically trying to avoid.

The problem with reading the data back in domU after having written it is that it might mask bugs in the block driver. Reading the data back might cause it to be committed to the device where otherwise a bug might cause it to be lost.

Harry.

#7 lore raw
Quoted 8 lines · Harry Butterworth
Quoted 4 lines

Modifying the ramdisk is not verboten,
but it is a right pain in the backside for everyone, particularly me because I need to build a new one and upload it and all of its source (because of the GPL) to xm-test.xensource.com.

The current test is modifying the ramdisk to add tee (as well as fancy head which we don't need anymore).

Quoted 8 lines

Are you going to accept the patch if it requires the addition of tee to the ramdisk? Or should I change the test to read the data back in domU---which I was specifically trying to avoid.

The problem with reading the data back in domU after having written it is that it might mask bugs in the block driver. Reading the data back might cause it to be committed to the device where otherwise a bug might cause it to be lost.

I think that's a fair point, so yes, we'll have to rev the ramdisk anyway.

Ewan.

#8 lore raw
In reply to Harry Butterworth
Quoted 30 lines · Harry Butterworth

The only difference from the last version of the patch is that the minor version number in configure.ac is incremented.

Quoted 1 line

From the patch:

(quoted diff) +2 -0
+# This test imports a ram disk device as a physical device into a domU.
+# The domU initialises the ram disk with data from /dev/urandom and

calculates
+# the md5 checksum of the data (using tee as it is written so as to avoid
+# reading it back from the device which might potentially mask problems).

(quoted diff) +2 -0
+# The domU is stopped and the md5 checksum of the data on the device is
+# calculated by dom0.  The test succeeds if the checksums match,

indicating
+# that all the data written by domU was sucessfully committed to the device.
+

This patch also enables tee and fancy head in busybox on the ramdisk. I have tested the patch with both make existing' where the tests run but the new test fails because the ramdisk is missing tee and fancy head and make` where the test passes successfully.

The patch also installs the block-integrity tests in the default test set so they get executed.
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel