Skip to contentSkip to conversation

scsi: sd: Contribute to randomness when running rotational device

PATCH Applied Xuewei Zhang 6 participants 8 messages Sep 6, 2018 – Sep 17, 2018 SCSI

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 scsi: sd: Contribute to randomness when running rotational device

Currently a scsi device won't contribute to kernel randomness when it uses blk-mq. Since we commonly use scsi on rotational device with blk-mq, it make sense to keep contributing to kernel randomness in these cases. This is especially important for virtual machines.

commit b5b6e8c8d3b4 ("scsi: virtio_scsi: fix IO hang caused by automatic irq vector affinity") made all virtio-scsi device to use blk-mq, which does not contribute to randomness today. So for a virtual machine only having virtio-scsi disk (which is common), it will simple stop getting randomness from its disks in today's implementation.

With this patch, if the above VM has rotational virtio-scsi device, then it can still benefit from the entropy generated from the disk.

---
 drivers/scsi/sd.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/scsi/sd.c +3 -0
index b79b366a94f7..5e4f10d28065 100644
@@ -2959,6 +2959,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 	if (rot == 1) {
 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
+	} else {
+		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
 	}
 
 	if (sdkp->device->type == TYPE_ZBC) {
Signature · 1 line
2.19.0.rc2.392.g5ba43deb5a-goog
#1 lore raw
Quoted 24 lines · Xuewei Zhang

Currently a scsi device won't contribute to kernel randomness when it uses blk-mq. Since we commonly use scsi on rotational device with blk-mq, it make sense to keep contributing to kernel randomness in these cases. This is especially important for virtual machines.

commit b5b6e8c8d3b4 ("scsi: virtio_scsi: fix IO hang caused by automatic irq vector affinity" made all virtio-scsi device to use blk-mq, which does not contribute to randomness today. So for a virtual machine only having virtio-scsi disk (which is common), it will simple stop getting randomness from its disks in today's implementation.

With this patch, if the above VM has rotational virtio-scsi device, then it can still benefit from the entropy generated from the disk.

drivers/scsi/sd.c |3 +-+-+-

1 file changed, 3 insertions(+-)
drivers/scsi/sd.c +1 -1
index b79b366a94f7..5e4f10d28065 100644
--- a/drivers/scsi/sd.c
+-+-+- b/drivers/scsi/sd.c

@@-2959,6 +-2959,9 @@static void sd_read_block_characteristics(struct scsi_disk *sdkp)

Quoted 2 lines
	if (rot ==1) {> 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);> 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);>+-	}else {>+-		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);>+-		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);> 	}> 
	if (sdkp->device->type ==TYPE_ZBC) {

Although this patch looks fine to me, seeing this patch makes me wonder whether the default should be changed (QUEUE_FLAG_MQ_DEFAULT) instead of modifying the sd driver. Can anyone remind me why QUEUE_FLAG_MQ_DEFAULT does not include QUEUE_FLAG_ADD_RANDOM?

Thanks,

Bart.

#2 lore raw
Quoted 5 lines · Bart Van Assche

Although this patch looks fine to me, seeing this patch makes me wonder whether the default should be changed (QUEUE_FLAG_MQ_DEFAULT) instead of modifying the sd driver. Can anyone remind me why QUEUE_FLAG_MQ_DEFAULT does not include QUEUE_FLAG_ADD_RANDOM?

There was a discussion about a number of *years* ago; blk-mq has been baking for a very long time. In the early days of block_mq, the overwhelming percentage of the users of blk-mq where those who were using PCIe attached flash. So when, I raised this question, the argument was that SSD users have no entropy. Which I agree with; but now that blk-mq is the default, and hard drives are using blk-mq, it's time for a patch like Xuewei's.

Cheers,

- Ted

#3 lore raw
Quoted 7 lines · Theodore Y. Ts'o
Quoted 5 lines · Bart Van Assche

Although this patch looks fine to me, seeing this patch makes me wonder whether the default should be changed (QUEUE_FLAG_MQ_DEFAULT) instead of modifying the sd driver. Can anyone remind me why QUEUE_FLAG_MQ_DEFAULT does not include QUEUE_FLAG_ADD_RANDOM?

Besides Ted's point of "SSD users have no entropy", I think there are two more reasons:
1. setting QUEUE_FLAG_ADD_RANDOM has a more visible performance hit on SSD disks than rotational disks.
2. SSD disks provide less entropy than rotational disks.
I actually experimented on Container-Optimized OS, running on Google Compute Engine with QUEUE_FLAG_ADD_RANDOM set.
Turns out the VM will have ~800 bit of entropy provided on boot on rotational disk;
and will only have ~70 bit of entropy if running on SSD (and remember there are ~50 bit contributed from other sources).
(in the above experiment, both disks were virtualized disks)

Quoted 12 lines

There was a discussion about a number of *years* ago; blk-mq has been baking for a very long time. In the early days of block_mq, the overwhelming percentage of the users of blk-mq where those who were using PCIe attached flash. So when, I raised this question, the argument was that SSD users have no entropy. Which I agree with; but now that blk-mq is the default, and hard drives are using blk-mq, it's time for a patch like Xuewei's.

Cheers,

- Ted

Reviewed-by
#4 lore raw
Quoted 9 lines · Xuewei Zhang
Quoted 7 lines · Theodore Y. Ts'o

There was a discussion about a number of *years* ago; blk-mq has been baking for a very long time. In the early days of block_mq, the overwhelming percentage of the users of blk-mq where those who were using PCIe attached flash. So when, I raised this question, the argument was that SSD users have no entropy. Which I agree with; but now that blk-mq is the default, and hard drives are using blk-mq, it's time for a patch like Xuewei's.

Quoted 12 lines

Besides Ted's point of "SSD users have no entropy", I think there are two more reasons:
1. setting QUEUE_FLAG_ADD_RANDOM has a more visible performance hit on SSD disks than rotational disks.
2. SSD disks provide less entropy than rotational disks.
I actually experimented on Container-Optimized OS, running on Google Compute Engine with QUEUE_FLAG_ADD_RANDOM set.
Turns out the VM will have ~800 bit of entropy provided on boot on rotational disk;
and will only have ~70 bit of entropy if running on SSD (and remember there are ~50 bit contributed from other sources).
(in the above experiment, both disks were virtualized disks)

All of the above makes sense to me and is much less risky than changing QUEUE_FLAG_MQ_DEFAULT. Hence:

Reviewed-by on v1 1/1
#5 lore raw
In reply to Xuewei Zhang
Quoted 37 lines · Xuewei Zhang

Currently a scsi device won't contribute to kernel randomness when it uses blk-mq. Since we commonly use scsi on rotational device with blk-mq, it make sense to keep contributing to kernel randomness in these cases. This is especially important for virtual machines.

commit b5b6e8c8d3b4 ("scsi: virtio_scsi: fix IO hang caused by automatic irq vector affinity") made all virtio-scsi device to use blk-mq, which does not contribute to randomness today. So for a virtual machine only having virtio-scsi disk (which is common), it will simple stop getting randomness from its disks in today's implementation.

With this patch, if the above VM has rotational virtio-scsi device, then it can still benefit from the entropy generated from the disk.

---
 drivers/scsi/sd.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/scsi/sd.c +3 -0
index b79b366a94f7..5e4f10d28065 100644
@@ -2959,6 +2959,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 	if (rot == 1) {
 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
+	} else {
+		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
 	}
 
 	if (sdkp->device->type == TYPE_ZBC) {
Signature · 1 line
2.19.0.rc2.392.g5ba43deb5a-goog

Look reasonable, especially the disk randomness is added by SCSI itself.

Thanks,
Ming

#6 lore raw
Quoted 41 lines · Ming Lei
Quoted 35 lines · Xuewei Zhang

Currently a scsi device won't contribute to kernel randomness when it uses blk-mq. Since we commonly use scsi on rotational device with blk-mq, it make sense to keep contributing to kernel randomness in these cases. This is especially important for virtual machines.

commit b5b6e8c8d3b4 ("scsi: virtio_scsi: fix IO hang caused by automatic irq vector affinity") made all virtio-scsi device to use blk-mq, which does not contribute to randomness today. So for a virtual machine only having virtio-scsi disk (which is common), it will simple stop getting randomness from its disks in today's implementation.

With this patch, if the above VM has rotational virtio-scsi device, then it can still benefit from the entropy generated from the disk.

---
 drivers/scsi/sd.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/scsi/sd.c +3 -1
index b79b366a94f7..5e4f10d28065 100644
@@ -2959,6 +2959,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
      if (rot == 1) {
              blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
              blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
+     } else {
+             blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+             blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
      }
 
      if (sdkp->device->type == TYPE_ZBC) {
--
Quoted 2 lines

2.19.0.rc2.392.g5ba43deb5a-goog

Look reasonable, especially the disk randomness is added by SCSI itself.

Quoted 4 lines

Thanks,
Ming

Also, see: https://bugzilla.redhat.com/show_bug.cgi?id=1572944
where we're having randomness starvation problems on FC28 running 4.18.5 due to lack of virtio-rng device in VM. (VM boot takes 9+ hours or 2 haswell VMs)

I'd kindly request we get this not only into 4.19 but also stable trees. (along with Ted's other randomization fixes)

#7 lore raw
In reply to Xuewei Zhang

Xuewei,

Quoted 4 lines

Currently a scsi device won't contribute to kernel randomness when it uses blk-mq. Since we commonly use scsi on rotational device with blk-mq, it make sense to keep contributing to kernel randomness in these cases. This is especially important for virtual machines.

Applied to 4.19/scsi-fixes, thank you!

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering