Skip to contentSkip to conversation

Reply [PATCH] dm-verity: unnecessary data blocks that need not read hash blocks

DISCUSSION xianrong.zhou(周先荣) 2 participants 3 messages Dec 16, 2019 – Jan 3, 2020 Device mapper

lore
Share & tools
lore
#0 lore raw

hey Eric:

Quoted 37 lines · zhou xianrong

From: "xianrong.zhou" <[email protected]>

If check_at_most_once enabled, just like verity work the prefetching work should check for data block bitmap firstly before reading hash block as well. Skip bit-set data blocks from both ends of data block range by testing the validated bitmap. This can reduce the amounts of data blocks which need to read hash blocks.

Launching 91 apps every 15s and repeat 21 rounds on Android Q. In prefetching work we can let only 2602/360312 = 0.72% data blocks really need to read hash blocks.

But the reduced data blocks range would be enlarged again by dm_verity_prefetch_cluster later.

---
 drivers/md/dm-verity-target.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
(quoted diff) +0 -0

b/drivers/md/dm-verity-target.c index 4fb33e7562c5..7b8eb754c0b6 100644

drivers/md/dm-verity-target.c +6 -0
@@ -581,6 +581,22 @@ static void verity_prefetch_io(struct work_struct *work)
 	struct dm_verity *v = pw->v;
 	int i;
 
+	if (v->validated_blocks) {
+		while (pw->n_blocks) {
+			if (unlikely(!test_bit(pw->block, v->validated_blocks)))
+				break;
+			pw->block++;
+			pw->n_blocks--;

drivers/md/dm-verity-target.c static void verity_prefetch_io(struct work_struct *work) · line 599

Quoted 10 lines
drivers/md/dm-verity-target.c +10 -0
+		}
+		while (pw->n_blocks) {
+			if (unlikely(!test_bit(pw->block + pw->n_blocks - 1,
+				v->validated_blocks)))
+				break;
+			pw->n_blocks--;
+		}
+		if (!pw->n_blocks)
+			return;
+	}

This is a good idea, but shouldn't this logic go in verity_submit_prefetch() prior to the struct dm_verity_prefetch_work being allocated? Then if no prefeching is needed, allocating and scheduling the work object can be skipped.

Eric, Do you mean it is more suitable in dm_bufio_prefetch which is called on different paths even though prefeching is disabled ?

Also note that you're currently leaking the work object with the early return.

Right! I leaked this. always so. Thanks!!!

- Eric

#1 lore raw

Re: [PATCH] dm-verity: unnecessary data blocks that need not read hash blocks

Quoted 56 lines · xianrong.zhou(周先荣)

hey Eric:

Quoted 37 lines · zhou xianrong

From: "xianrong.zhou" <[email protected]>

If check_at_most_once enabled, just like verity work the prefetching work should check for data block bitmap firstly before reading hash block as well. Skip bit-set data blocks from both ends of data block range by testing the validated bitmap. This can reduce the amounts of data blocks which need to read hash blocks.

Launching 91 apps every 15s and repeat 21 rounds on Android Q. In prefetching work we can let only 2602/360312 = 0.72% data blocks really need to read hash blocks.

But the reduced data blocks range would be enlarged again by dm_verity_prefetch_cluster later.

---
 drivers/md/dm-verity-target.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
(quoted diff) +0 -0

b/drivers/md/dm-verity-target.c index 4fb33e7562c5..7b8eb754c0b6 100644

drivers/md/dm-verity-target.c +6 -0
@@ -581,6 +581,22 @@ static void verity_prefetch_io(struct work_struct *work)
 	struct dm_verity *v = pw->v;
 	int i;
 
+	if (v->validated_blocks) {
+		while (pw->n_blocks) {
+			if (unlikely(!test_bit(pw->block, v->validated_blocks)))
+				break;
+			pw->block++;
+			pw->n_blocks--;

drivers/md/dm-verity-target.c static void verity_prefetch_io(struct work_struct *work) · line 599

Quoted 10 lines
drivers/md/dm-verity-target.c +10 -0
+		}
+		while (pw->n_blocks) {
+			if (unlikely(!test_bit(pw->block + pw->n_blocks - 1,
+				v->validated_blocks)))
+				break;
+			pw->n_blocks--;
+		}
+		if (!pw->n_blocks)
+			return;
+	}

This is a good idea, but shouldn't this logic go in verity_submit_prefetch() prior to the struct dm_verity_prefetch_work being allocated? Then if no prefeching is needed, allocating and scheduling the work object can be skipped.

Quoted 3 lines

Eric, Do you mean it is more suitable in dm_bufio_prefetch which is called on different paths even though prefeching is disabled ?

No, I'm talking about verity_submit_prefetch(). verity_submit_prefetch() allocates and schedules a work object, which executes verity_prefetch_io(). If all data blocks in the I/O request were already validated, there's no need to allocate and schedule the prefetch work.

- Eric

#2 lore raw

Re: [dm-devel] [PATCH] dm-verity: unnecessary data blocks that need not read hash blocks

Quoted 61 lines · Eric Biggers
Quoted 56 lines · xianrong.zhou(周先荣)

hey Eric:

Quoted 37 lines · zhou xianrong

From: "xianrong.zhou" <[email protected]>

If check_at_most_once enabled, just like verity work the prefetching work should check for data block bitmap firstly before reading hash block as well. Skip bit-set data blocks from both ends of data block range by testing the validated bitmap. This can reduce the amounts of data blocks which need to read hash blocks.

Launching 91 apps every 15s and repeat 21 rounds on Android Q. In prefetching work we can let only 2602/360312 = 0.72% data blocks really need to read hash blocks.

But the reduced data blocks range would be enlarged again by dm_verity_prefetch_cluster later.

---
 drivers/md/dm-verity-target.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
(quoted diff) +0 -0

b/drivers/md/dm-verity-target.c index 4fb33e7562c5..7b8eb754c0b6 100644

drivers/md/dm-verity-target.c +6 -0
@@ -581,6 +581,22 @@ static void verity_prefetch_io(struct work_struct *work)
 	struct dm_verity *v = pw->v;
 	int i;
 
+	if (v->validated_blocks) {
+		while (pw->n_blocks) {
+			if (unlikely(!test_bit(pw->block, v->validated_blocks)))
+				break;
+			pw->block++;
+			pw->n_blocks--;

drivers/md/dm-verity-target.c static void verity_prefetch_io(struct work_struct *work) · line 599

Quoted 10 lines
drivers/md/dm-verity-target.c +10 -0
+		}
+		while (pw->n_blocks) {
+			if (unlikely(!test_bit(pw->block + pw->n_blocks - 1,
+				v->validated_blocks)))
+				break;
+			pw->n_blocks--;
+		}
+		if (!pw->n_blocks)
+			return;
+	}

This is a good idea, but shouldn't this logic go in verity_submit_prefetch() prior to the struct dm_verity_prefetch_work being allocated? Then if no prefeching is needed, allocating and scheduling the work object can be skipped.

Quoted 3 lines

Eric, Do you mean it is more suitable in dm_bufio_prefetch which is called on different paths even though prefeching is disabled ?

Quoted 7 lines

No, I'm talking about verity_submit_prefetch(). verity_submit_prefetch() allocates and schedules a work object, which executes verity_prefetch_io(). If all data blocks in the I/O request were already validated, there's no need to allocate and schedule the prefetch work.

- Eric

Are you planning to send a new version of this patch?

- Eric