Skip to contentSkip to conversation

test/py: gpt: update test_gpt

PATCH New Patrick Delaunay 3 participants 8 messages Oct 9, 2017 – Oct 16, 2017

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 test/py: gpt: update test_gpt

- copy the reference gpt binary file as it is modified during the test to avoid issue if the test fail: the test always restart with clean file
- update tests to highlight detected issues on gpt swap command (offset and size of partition are modified)
- add test for gpt verfiy, gpt read and gpt write

---

 test/py/tests/test_gpt.py | 79 +++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 66 insertions(+), 13 deletions(-)
test/py/tests/test_gpt.py +66 -13
index ec25fbb..1db65f7 100644
@@ -11,7 +11,8 @@ import u_boot_utils
 
 """
 These tests rely on a 4 MB disk image, which is automatically created by
-the test.
+the test, but as we expect specific content and it is modified by the gpt
+commands executed during the test, it is not safe be reused it directly
 """
 
 class GptTestDiskImage(object):
@@ -28,26 +29,31 @@ class GptTestDiskImage(object):
         """
 
         filename = 'test_gpt_disk_image.bin'
+
         self.path = u_boot_console.config.persistent_data_dir + '/' + filename
+        ref_path = u_boot_console.config.persistent_data_dir + '/ref_' + filename
 
-        if os.path.exists(self.path):
-            u_boot_console.log.action('Disk image file ' + self.path +
+        if os.path.exists(ref_path):
+            u_boot_console.log.action('Disk image file ' + ref_path +
                 ' already exists')
         else:
-            u_boot_console.log.action('Generating ' + self.path)
-            fd = os.open(self.path, os.O_RDWR | os.O_CREAT)
+            u_boot_console.log.action('Generating ' + ref_path)
+            fd = os.open(ref_path, os.O_RDWR | os.O_CREAT)
             os.ftruncate(fd, 4194304)
             os.close(fd)
             cmd = ('sgdisk', '-U', '375a56f7-d6c9-4e81-b5f0-09d41ca89efe',
-                self.path)
+                ref_path)
             u_boot_utils.run_and_log(u_boot_console, cmd)
-            cmd = ('sgdisk', '--new=1:2048:2560', self.path)
+            cmd = ('sgdisk', '--new=1:2048:2560', ref_path)
             u_boot_utils.run_and_log(u_boot_console, cmd)
-            cmd = ('sgdisk', '--new=2:4096:4608', self.path)
+            cmd = ('sgdisk', '--new=2:4096:4608', ref_path)
             u_boot_utils.run_and_log(u_boot_console, cmd)
-            cmd = ('sgdisk', '-l', self.path)
+            cmd = ('sgdisk', '-l', ref_path)
             u_boot_utils.run_and_log(u_boot_console, cmd)
 
+        cmd = ('cp', ref_path, self.path)
+        u_boot_utils.run_and_log(u_boot_console, cmd)
+
 gtdi = None
 @pytest.fixture(scope='function')
 def state_disk_image(u_boot_console):
@@ -64,6 +70,30 @@ def state_disk_image(u_boot_console):
 @pytest.mark.boardspec('sandbox')
 @pytest.mark.buildconfigspec('cmd_gpt')
 @pytest.mark.requiredtool('sgdisk')
+def test_gpt_read(state_disk_image, u_boot_console):
+    """Test the gpt read command."""
+
+    u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
+    output = u_boot_console.run_command('gpt read host 0')
+    assert 'Start 1MiB, size 0MiB' in output
+    assert 'Start 2MiB, size 0MiB' in output
+    output = u_boot_console.run_command('part list host 0')
+    assert '0x00000800	0x00000a00	""' in output
+    assert '0x00001000	0x00001200	""' in output
+
+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_gpt')
+ at pytest.mark.requiredtool('sgdisk')
+def test_gpt_verify(state_disk_image, u_boot_console):
+    """Test the gpt verify command."""
+
+    u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
+    output = u_boot_console.run_command('gpt verify host 0')
+    assert 'Verify GPT: success!' in output
+
+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_gpt')
+ at pytest.mark.requiredtool('sgdisk')
 def test_gpt_guid(state_disk_image, u_boot_console):
     """Test the gpt guid command."""
 
@@ -98,6 +128,9 @@ def test_gpt_rename_partition(state_disk_image, u_boot_console):
     u_boot_console.run_command('gpt rename host 0 2 second')
     output = u_boot_console.run_command('gpt read host 0')
     assert 'name second' in output
+    output = u_boot_console.run_command('part list host 0')
+    assert '0x00000800	0x00000a00	"first"' in output
+    assert '0x00001000	0x00001200	"second"' in output
 
 @pytest.mark.boardspec('sandbox')
 @pytest.mark.buildconfigspec('cmd_gpt')
@@ -109,9 +142,29 @@ def test_gpt_swap_partitions(state_disk_image, u_boot_console):
 
     u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
     output = u_boot_console.run_command('part list host 0')
-    assert '0x000007ff	"first"' in output
-    assert '0x000017ff	"second"' in output
+    assert '0x00000800	0x00000a00	"first"' in output
+    assert '0x00001000	0x00001200	"second"' in output
     u_boot_console.run_command('gpt swap host 0 first second')
     output = u_boot_console.run_command('part list host 0')
-    assert '0x000007ff	"second"' in output
-    assert '0x000017ff	"first"' in output
+    assert '0x00000800	0x00000a00	"second"' in output
+    assert '0x00001000	0x00001200	"first"' in output
+
+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_gpt')
+ at pytest.mark.buildconfigspec('cmd_part')
+ at pytest.mark.requiredtool('sgdisk')
+def test_gpt_write(state_disk_image, u_boot_console):
+    """Test the gpt write command."""
+
+    u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
+    output = u_boot_console.run_command('gpt write host 0 "name=all,size=0"')
+    assert 'Writing GPT: success!' in output
+    output = u_boot_console.run_command('part list host 0')
+    assert '0x00000022	0x00001fde	"all"' in output
+    output = u_boot_console.run_command('gpt write host 0 "uuid_disk=375a56f7-d6c9-4e81-b5f0-09d41ca89efe;name=first,start=0x100000,size=0x40200;name=second,start=0x200000,size=0x40200;"')
+    assert 'Writing GPT: success!' in output
+    output = u_boot_console.run_command('part list host 0')
+    assert '0x00000800	0x00000a00	"first"' in output
+    assert '0x00001000	0x00001200	"second"' in output
+    output = u_boot_console.run_command('gpt guid host 0')
+    assert '375a56f7-d6c9-4e81-b5f0-09d41ca89efe' in output
Signature · 1 line
2.7.4
#1 lore raw

PATCH 2/3 disk: efi: correct the overlap check on GPT header and PTE

the partition starting at 0x4400 is refused with overlap error:
$> gpt write mmc 0 "name=test,start=0x4400,size=0" Writing GPT: Partition overlap
error!

even if the 0x4400 is the first available offset for LBA35 with default value:
- MBR=LBA1
- GPT header=LBA2
- PTE= 32 LBAs (128 entry), 3 to 34

And the command to have one partition for all the disk failed also :
$> gpt write mmc 0 "name=test,size=0"

After the patch :

$> gpt write mmc 0 "name=test,size=0"
Writing GPT: success!
$> part list mmc 0

Partition Map for MMC device 0 -- Partition Type: EFI

Part Start LBA End LBA Name
Attributes
Type GUID
Partition GUID
1 0x00000022 0x01ce9fde "test"

	attrs:	0x0000000000000000
	type:	ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
	type:	data
	guid:	b4b84b8a-04e3-4000-0036-aff5c9c495b1

And 0x22 = 34 LBA => offset = 0x4400 is accepted as expected

---

 disk/part_efi.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
disk/part_efi.c +2 -2
index 0abf487..2992d9e 100644
@@ -469,8 +469,8 @@ int gpt_fill_pte(struct blk_desc *dev_desc,
 		 * If our partition overlaps with either the GPT
 		 * header, or the partition entry, reject it.
 		 */
-		if (((start <= hdr_end && hdr_start <= (start + size)) ||
-		     (start <= pte_end && pte_start <= (start + size)))) {
+		if (((start < hdr_end && hdr_start < (start + size)) ||
+		     (start < pte_end && pte_start < (start + size)))) {
 			printf("Partition overlap\n");
 			return -1;
 		}
Signature · 1 line
2.7.4
#2 lore raw

PATCH 2/3 disk: efi: correct the overlap check on GPT header and PTE

Quoted 32 lines · Patrick Delaunay

the partition starting at 0x4400 is refused with overlap error:
$> gpt write mmc 0 "name=test,start=0x4400,size=0" Writing GPT: Partition overlap
error!

even if the 0x4400 is the first available offset for LBA35 with default value:
- MBR=LBA1
- GPT header=LBA2
- PTE= 32 LBAs (128 entry), 3 to 34

And the command to have one partition for all the disk failed also :
$> gpt write mmc 0 "name=test,size=0"

After the patch :

$> gpt write mmc 0 "name=test,size=0"
Writing GPT: success!
$> part list mmc 0

   Partition Map for MMC device 0  --   Partition Type: EFI

   Part	Start LBA	End LBA		Name
	Attributes
	Type GUID
	Partition GUID
   1	0x00000022	0x01ce9fde	"test"
	attrs:	0x0000000000000000
	type:	ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
	type:	data
	guid:	b4b84b8a-04e3-4000-0036-aff5c9c495b1
Quoted 2 lines

And 0x22 = 34 LBA => offset = 0x4400 is accepted as expected

Quoted 22 lines

---

  disk/part_efi.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
disk/part_efi.c +2 -2
index 0abf487..2992d9e 100644
@@ -469,8 +469,8 @@ int gpt_fill_pte(struct blk_desc *dev_desc,
  		 * If our partition overlaps with either the GPT
  		 * header, or the partition entry, reject it.
  		 */
-		if (((start <= hdr_end && hdr_start <= (start + size)) ||
-		     (start <= pte_end && pte_start <= (start + size)))) {
+		if (((start < hdr_end && hdr_start < (start + size)) ||
+		     (start < pte_end && pte_start < (start + size)))) {
  			printf("Partition overlap\n");
  			return -1;
  		}
Signature · 9 lines
Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
#3 lore raw

PATCH 3/3 cmd: gpt: solve issue for swap

In reply to Patrick Delaunay

don't use prettyprint_part_size() in create_gpt_partitions_list() that avoid to align offset and size to 1 MiB and increase precision for start and size
This patch avoid the risk to change partition and lost data during swap

---

 cmd/gpt.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
cmd/gpt.c +6 -6
index d4406e3..38d09c6 100644
@@ -282,14 +282,14 @@ static int create_gpt_partitions_list(int numparts, const char *guid,
 		strcat(partitions_list, "name=");
 		strncat(partitions_list, (const char *)curr->gpt_part_info.name,
 			PART_NAME_LEN + 1);
-		strcat(partitions_list, ",start=");
-		prettyprint_part_size(partstr, (unsigned long)curr->gpt_part_info.start,
-				      (unsigned long) curr->gpt_part_info.blksz);
+		sprintf(partstr, ",start=0x%llx",
+			(unsigned long long)curr->gpt_part_info.start *
+					    curr->gpt_part_info.blksz);
 		/* one extra byte for NULL */
 		strncat(partitions_list, partstr, PART_NAME_LEN + 1);
-		strcat(partitions_list, ",size=");
-		prettyprint_part_size(partstr, curr->gpt_part_info.size,
-				      curr->gpt_part_info.blksz);
+		sprintf(partstr, ",size=0x%llx",
+			(unsigned long long)curr->gpt_part_info.size *
+					    curr->gpt_part_info.blksz);
 		strncat(partitions_list, partstr, PART_NAME_LEN + 1);
 
 		strcat(partitions_list, ",uuid=");
Signature · 1 line
2.7.4
#4 lore raw

PATCH 1/3 test/py: gpt: update test_gpt

In reply to Patrick Delaunay
Quoted 5 lines · Patrick Delaunay

- copy the reference gpt binary file as it is modified during the test to avoid issue if the test fail: the test always restart with clean file
- update tests to highlight detected issues on gpt swap command (offset and size of partition are modified)
- add test for gpt verfiy, gpt read and gpt write

It might be nice to split this into separate patches for separate logical changes.

test/py/tests/test_gpt.py import u_boot_utils · line 15

Quoted 6 lines
test/py/tests/test_gpt.py +2 -1
 
  These tests rely on a 4 MB disk image, which is automatically created by
-the test.
+the test, but as we expect specific content and it is modified by the gpt
+commands executed during the test, it is not safe be reused it directly

That's a bit of an internal detail. I'm not sure it's necessary to spell it out in the documentation?

test/py/tests/test_gpt.py class GptTestDiskImage(object): · line 34

Quoted 8 lines

class GptTestDiskImage(object):

test/py/tests/test_gpt.py +2 -0
@@ -28,26 +29,31 @@ class GptTestDiskImage(object):
          """
 
          filename = 'test_gpt_disk_image.bin'
+
          self.path = u_boot_console.config.persistent_data_dir + '/' + filename
+        ref_path = u_boot_console.config.persistent_data_dir + '/ref_' + filename

Can we put self.path somewhere other than persistent_data_dir? Since it gets re-created every time, it's not persistent. Other tests use u_boot_console.config.result_dir for temporary files.

test/py/tests/test_gpt.py def state_disk_image(u_boot_console): · line 80

Quoted 12 lines
test/py/tests/test_gpt.py +8 -0
@@ -64,6 +70,30 @@ def state_disk_image(u_boot_console):
  @pytest.mark.boardspec('sandbox')
  @pytest.mark.buildconfigspec('cmd_gpt')
  @pytest.mark.requiredtool('sgdisk')
+def test_gpt_read(state_disk_image, u_boot_console):
+    """Test the gpt read command."""
+
+    u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
+    output = u_boot_console.run_command('gpt read host 0')
+    assert 'Start 1MiB, size 0MiB' in output
+    assert 'Start 2MiB, size 0MiB' in output
+    output = u_boot_console.run_command('part list host 0')

I think this test should also be:
@pytest.mark.buildconfigspec('cmd_part')

Some of the other diffs in this path add use of the part command for the first time, so I think you need to add that decorator to a few other functions too.

test/py/tests/test_gpt.py def test_gpt_swap_partitions(state_disk_image, u_boot_console): · line 146

Quoted 8 lines
test/py/tests/test_gpt.py +2 -2
@@ -109,9 +142,29 @@ def test_gpt_swap_partitions(state_disk_image, u_boot_console):
 
      u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
      output = u_boot_console.run_command('part list host 0')
-    assert '0x000007ff	"first"' in output
-    assert '0x000017ff	"second"' in output
+    assert '0x00000800	0x00000a00	"first"' in output
+    assert '0x00001000	0x00001200	"second"' in output

I'm not sure why this change is required. I can see two separate changes here:

1) Verifies the value of 3 columns of data not just 2. That seems fine.

2) Changes the expected value in the column before the partition name. I'm not sure about this; shouldn't the value stay the same, or is the test failing right now?

#5 lore raw

PATCH 1/3 test/py: gpt: update test_gpt

Hi Stephen,

Quoted 4 lines

-----Original Message-----
From: Stephen Warren [mailto:swarren at wwwdotorg.org]

On 10/09/2017 01:47 AM, Patrick Delaunay wrote:

Quoted 8 lines
Quoted 6 lines

- copy the reference gpt binary file as it is modified during the test to avoid issue if the test fail: the test always restart with clean
file
- update tests to highlight detected issues on gpt swap command (offset and size of partition are modified)
- add test for gpt verfiy, gpt read and gpt write

It might be nice to split this into separate patches for separate logical changes.

Ok I will split this patch in 3 parts in v2

Quoted 11 lines

test/py/tests/test_gpt.py

Quoted 8 lines
test/py/tests/test_gpt.py +0 -0
 
  These tests rely on a 4 MB disk image, which is automatically

created by -the test.

test/py/tests/test_gpt.py +3 -0
+the test, but as we expect specific content and it is modified by the
+gpt commands executed during the test, it is not safe be reused it
+directly

That's a bit of an internal detail. I'm not sure it's necessary to spell it out in the documentation?

Yes, I agree.
It is the first time I modify tets/py and I don't see the fact that these lines are not just comment. I will remove it in v2

Quoted 5 lines
Quoted 5 lines

class GptTestDiskImage(object):

test/py/tests/test_gpt.py +0 -0
@@ -28,26 +29,31 @@ class GptTestDiskImage(object):
          """
 
Quoted 10 lines

test/py/tests/test_gpt.py class GptTestDiskImage(object): · line 36

Quoted 6 lines
test/py/tests/test_gpt.py +4 -0
          filename = 'test_gpt_disk_image.bin'
+
          self.path = u_boot_console.config.persistent_data_dir + '/'
+ filename
+        ref_path = u_boot_console.config.persistent_data_dir +
+ '/ref_' + filename

Can we put self.path somewhere other than persistent_data_dir? Since it gets re-created every time, it's not persistent. Other tests use u_boot_console.config.result_dir for temporary files.

Agree, I search a location for temporary file, but I don't found a good solution... So I will use u_boot_console.config.result_dir in v2

Quoted 17 lines

test/py/tests/test_gpt.py def state_disk_image(u_boot_console): · line 80

Quoted 13 lines
test/py/tests/test_gpt.py +8 -0
@@ -64,6 +70,30 @@ def state_disk_image(u_boot_console):
  @pytest.mark.boardspec('sandbox')
  @pytest.mark.buildconfigspec('cmd_gpt')
  @pytest.mark.requiredtool('sgdisk')
+def test_gpt_read(state_disk_image, u_boot_console):
+    """Test the gpt read command."""
+
+    u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
+    output = u_boot_console.run_command('gpt read host 0')
+    assert 'Start 1MiB, size 0MiB' in output
+    assert 'Start 2MiB, size 0MiB' in output
+    output = u_boot_console.run_command('part list host 0')

I think this test should also be:
@pytest.mark.buildconfigspec('cmd_part')

Quoted 2 lines

Some of the other diffs in this path add use of the part command for the first time, so I think you need to add that decorator to a few other functions too.

Ok

Quoted 11 lines

test/py/tests/test_gpt.py def test_gpt_swap_partitions(state_disk_image,

Quoted 2 lines
test/py/tests/test_gpt.py +0 -0
@@ -109,9 +142,29 @@ def test_gpt_swap_partitions(state_disk_image,

u_boot_console):

test/py/tests/test_gpt.py def test_gpt_swap_partitions(state_disk_image, u_boot_console): · line 146

Quoted 7 lines
test/py/tests/test_gpt.py +2 -2
      u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
      output = u_boot_console.run_command('part list host 0')
-    assert '0x000007ff	"first"' in output
-    assert '0x000017ff	"second"' in output
+    assert '0x00000800	0x00000a00	"first"' in output
+    assert '0x00001000	0x00001200	"second"' in output
Quoted 8 lines

I'm not sure why this change is required. I can see two separate changes here:

1) Verifies the value of 3 columns of data not just 2. That seems fine.

2) Changes the expected value in the column before the partition name. I'm not sure about this; shouldn't the value stay the same, or is the test failing right now?

in fact it the purpose of my modification in patch 3/3 : cmd: gpt: solve issue for swap

The command gpt swap (or gpt remane) don't expect change the offset or the size of the partition but it is the case today , it is an error in code and it is not detected with the current test

In the modified test, I test the expected correct values (3 columns): without modification of the offset and size.

=> the test if failing for master branch
=> the test is OK only after " patch 3/3 : cmd: gpt: solve issue for swap"

But I don't sure of the correct patchset order to avoid issue during merge in master branch

First Test update => test Failed
Then patch the code => test OK

Or

First patch the code => test Failed
Then patch the test => test OK

Patrick

#6 lore raw

PATCH 1/3 test/py: gpt: update test_gpt

Quoted 43 lines · Patrick DELAUNAY

Hi Stephen,

Quoted 23 lines

-----Original Message-----
From: Stephen Warren [mailto:swarren at wwwdotorg.org]

Quoted 8 lines · Patrick Delaunay

- copy the reference gpt binary file as it is modified during the test to avoid issue if the test fail: the test always restart with clean
file
- update tests to highlight detected issues on gpt swap command (offset and size of partition are modified)
- add test for gpt verfiy, gpt read and gpt write

test/py/tests/test_gpt.py def test_gpt_swap_partitions(state_disk_image, u_boot_console): · line 146

Quoted 7 lines
test/py/tests/test_gpt.py +2 -2
@@ -109,9 +142,29 @@ def test_gpt_swap_partitions(state_disk_image, u_boot_console):
 
       u_boot_console.run_command('host bind 0 ' + state_disk_image.path)
       output = u_boot_console.run_command('part list host 0')
-    assert '0x000007ff	"first"' in output
-    assert '0x000017ff	"second"' in output
+    assert '0x00000800	0x00000a00	"first"' in output
+    assert '0x00001000	0x00001200	"second"' in output

I'm not sure why this change is required. I can see two separate changes here:

Quoted 5 lines

1) Verifies the value of 3 columns of data not just 2. That seems fine.

2) Changes the expected value in the column before the partition name. I'm not sure about this; shouldn't the value stay the same, or is the test failing right now?

in fact it the purpose of my modification in patch 3/3 : cmd: gpt: solve issue for swap

The command gpt swap (or gpt remane) don't expect change the offset or the size of the partition but it is the case today , it is an error in code and it is not detected with the current test

In the modified test, I test the expected correct values (3 columns): without modification of the offset and size.

=> the test if failing for master branch
=> the test is OK only after " patch 3/3 : cmd: gpt: solve issue for swap"

But I don't sure of the correct patchset order to avoid issue during merge in master branch

Quoted 7 lines

First Test update => test Failed
Then patch the code => test OK

Or

First patch the code => test Failed
Then patch the test => test OK

I'd suggest something like:

Patch 1: Modify the test to check the value for all 3 columns in a way that passes with the current implementation of the command. Perhaps add a comment that there's a bug in the code, and remove it in the next patch. The test should pass both before and after this patch.

Patch 2: Fix the bug in the GPT command, and in the same patch, change the values that test_gpt expects so that it passes again. The test should pass both before and after this patch.

#7 lore raw

PATCH 1/3 test/py: gpt: update test_gpt

Hi Stephen

Quoted 23 lines

From: Stephen Warren [mailto:swarren at wwwdotorg.org]

Quoted 10 lines · Patrick DELAUNAY

Hi Stephen,

Quoted 4 lines

-----Original Message-----
From: Stephen Warren [mailto:swarren at wwwdotorg.org]

On 10/09/2017 01:47 AM, Patrick Delaunay wrote:

But I don't sure of the correct patchset order to avoid issue during merge in master branch

Quoted 7 lines

First Test update => test Failed
Then patch the code => test OK

Or

First patch the code => test Failed
Then patch the test => test OK

I'd suggest something like:

Quoted 8 lines

Patch 1: Modify the test to check the value for all 3 columns in a way that passes with the current implementation of the command. Perhaps add a comment that there's a bug in the code, and remove it in the next patch. The test should pass both before and after this patch.

Patch 2: Fix the bug in the GPT command, and in the same patch, change the values that test_gpt expects so that it passes again. The test should pass both before and after this patch.

I will do it as proposed in v2 patch with the test update split

Patrick