Skip to contentSkip to conversation

virtio_blk: Dont waste major numbers

PATCH In review Christian Borntraeger 7 participants 12 messages Jan 31, 2008 – Mar 20, 2008 4 messages not in the archive

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 virtio_blk: Dont waste major numbers

Rusty,

currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations.

This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks.

---
 drivers/block/virtio_blk.c |   28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)
drivers/block/virtio_blk.c +16 -12
Index: kvm/drivers/block/virtio_blk.c
===================================================================
@@ -7,10 +7,13 @@
 #include <linux/scatterlist.h>
 
 #define VIRTIO_MAX_SG	(3+MAX_PHYS_SEGMENTS)
+#define PART_BITS 4
 
 MODULE_LICENSE("GPL");
 
 static unsigned char virtblk_index = 'a';
+static int major, minor;
+
 struct virtio_blk
 {
 	spinlock_t lock;
@@ -173,10 +176,13 @@ static struct block_device_operations vi
 static int virtblk_probe(struct virtio_device *vdev)
 {
 	struct virtio_blk *vblk;
-	int err, major;
+	int err;
 	u64 cap;
 	u32 v;
 
+	if (minor >= 1 << MINORBITS)
+		return -ENOSPC;
+
 	vdev->priv = vblk = kmalloc(sizeof(*vblk), GFP_KERNEL);
 	if (!vblk) {
 		err = -ENOMEM;
@@ -200,17 +206,11 @@ static int virtblk_probe(struct virtio_d
 		goto out_free_vq;
 	}
 
-	major = register_blkdev(0, "virtblk");
-	if (major < 0) {
-		err = major;
-		goto out_mempool;
-	}
-
 	/* FIXME: How many partitions?  How long is a piece of string? */
-	vblk->disk = alloc_disk(1 << 4);
+	vblk->disk = alloc_disk(1 << PART_BITS);
 	if (!vblk->disk) {
 		err = -ENOMEM;
-		goto out_unregister_blkdev;
+		goto out_mempool;
 	}
 
 	vblk->disk->queue = blk_init_queue(do_virtblk_request, &vblk->lock);
@@ -221,10 +221,12 @@ static int virtblk_probe(struct virtio_d
 
 	sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);
 	vblk->disk->major = major;
-	vblk->disk->first_minor = 0;
+	vblk->disk->first_minor = minor;
 	vblk->disk->private_data = vblk;
 	vblk->disk->fops = &virtblk_fops;
 
+	minor += 1 << PART_BITS;
+
 	/* If barriers are supported, tell block layer that queue is ordered */
 	if (vdev->config->feature(vdev, VIRTIO_BLK_F_BARRIER))
 		blk_queue_ordered(vblk->disk->queue, QUEUE_ORDERED_TAG, NULL);
@@ -260,8 +262,6 @@ static int virtblk_probe(struct virtio_d
 
 out_put_disk:
 	put_disk(vblk->disk);
-out_unregister_blkdev:
-	unregister_blkdev(major, "virtblk");
 out_mempool:
 	mempool_destroy(vblk->pool);
 out_free_vq:
@@ -302,11 +302,15 @@ static struct virtio_driver virtio_blk =
 
 static int __init init(void)
 {
+	major = register_blkdev(0, "virtblk");
+	if (major < 0)
+		return major;
 	return register_virtio_driver(&virtio_blk);
 }
 
 static void __exit fini(void)
 {
+	unregister_blkdev(major, "virtblk");
 	unregister_virtio_driver(&virtio_blk);
 }
 module_init(init);

A message that is not in the archive (sent off-list or not yet received): 200801311553.53653.borntraeger__25311.9055800655$1201791369$gmane$org-tA70FqPdS9bQT0dZR+AlfA@public.gmane.org

#2 lore raw
Quoted 9 lines · Christian Borntraeger

Rusty,

currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations.

This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks.

There's are some other limitations to the number of virtio block devices. For instances...

Quoted 1 line

sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);

This gets bogus after 64 disks. We also have a hard limit for virtio-pci based on the number of PCI slots available. One thing I was considering was whether we should try to support multiple disks per virtio device.

Otherwise, this patch looks good to me.

Regards,

Anthony Liguori

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Am Donnerstag, 31. Januar 2008 schrieb Anthony Liguori:

Quoted 6 lines

There's are some other limitations to the number of virtio block devices. For instances...

Quoted 1 line

sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);

This gets bogus after 64 disks.

Right. I will fix that with an additional patch.

Quoted 4 lines

We also have a hard limit for
virtio-pci based on the number of PCI slots available. One thing I was considering was whether we should try to support multiple disks per virtio device.

What is the hard limit in the PCI name space?

Christian

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

A message that is not in the archive (sent off-list or not yet received): 200802010833.51809.borntraeger-tA70FqPdS9bQT0dZR+AlfA@public.gmane.org

#6 lore raw

PATCH 1/1 virtio_blk: implement naming for vda-vdz, vdaa-vdzz, vdaaa-vdzzz

Am Freitag, 1. Februar 2008 schrieb Christian Borntraeger:

Quoted 1 line

Right. I will fix that with an additional patch.

This patch goes on top of the minor number patch. Please let me know if you want a merged patch:

Currently virtio_blk creates the disk name combinging "vd" with 'a'++. This will give strange names after vdz. I have implemented names up to vdzzz - inspired by the sd.c code. That should be sufficient for now.

There is one driver in the kernel (driver/s390/block/dasd_genhd.c) that implements names from dasda-dasdzzzz allowing even more disks. Maybe a janitor can come up with a common implementation usable for all kind of block device drivers.

I have tested this patch with 100 disks - seems to work.

---
 drivers/block/virtio_blk.c |   29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)
drivers/block/virtio_blk.c +22 -7
Index: kvm/drivers/block/virtio_blk.c
===================================================================
@@ -11,8 +11,7 @@
 
 MODULE_LICENSE("GPL");
 
-static unsigned char virtblk_index = 'a';
-static int major, minor;
+static int major, index;
 
 struct virtio_blk
 {
@@ -173,6 +172,11 @@ static struct block_device_operations vi
 	.getgeo = virtblk_getgeo,
 };
 
+static int index_to_minor(int index)
+{
+	return index << PART_BITS;
+}
+
 static int virtblk_probe(struct virtio_device *vdev)
 {
 	struct virtio_blk *vblk;
@@ -180,7 +184,7 @@ static int virtblk_probe(struct virtio_d
 	u64 cap;
 	u32 v;
 
-	if (minor >= 1 << MINORBITS)
+	if (index_to_minor(index) >= 1 << MINORBITS)
 		return -ENOSPC;
 
 	vdev->priv = vblk = kmalloc(sizeof(*vblk), GFP_KERNEL);
@@ -219,13 +223,24 @@ static int virtblk_probe(struct virtio_d
 		goto out_put_disk;
 	}
 
-	sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);
+	if (index < 26) {
+		sprintf(vblk->disk->disk_name, "vd%c", 'a' + index % 26);
+	} else if (index < (26 + 1) * 26) {
+		sprintf(vblk->disk->disk_name, "vd%c%c",
+			'a' + index / 26 - 1, 'a' + index % 26);
+	} else {
+		const unsigned int m1 = (index / 26 - 1) / 26 - 1;
+		const unsigned int m2 = (index / 26 - 1) % 26;
+		const unsigned int m3 =  index % 26;
+		sprintf(vblk->disk->disk_name, "vd%c%c%c",
+			'a' + m1, 'a' + m2, 'a' + m3);
+	}
+
 	vblk->disk->major = major;
-	vblk->disk->first_minor = minor;
+	vblk->disk->first_minor = index_to_minor(index);
 	vblk->disk->private_data = vblk;
 	vblk->disk->fops = &virtblk_fops;
-
-	minor += 1 << PART_BITS;
+	index++;
 
 	/* If barriers are supported, tell block layer that queue is ordered */
 	if (vdev->config->feature(vdev, VIRTIO_BLK_F_BARRIER))

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

A message that is not in the archive (sent off-list or not yet received): 200802010905.00336.borntraeger-tA70FqPdS9bQT0dZR+AlfA@public.gmane.org

#8 lore raw

Re: [Patch] virtio_blk: implement naming for vda-vdz, vdaa-vdzz, vdaaa-vdzzz

Quoted 9 lines · On Friday 01 February 2008 19:05:00 Christian Borntraeger wrote:

Am Freitag, 1. Februar 2008 schrieb Christian Borntraeger:

Quoted 1 line

Right. I will fix that with an additional patch.

This patch goes on top of the minor number patch. Please let me know if you want a merged patch:

Currently virtio_blk creates the disk name combinging "vd" with 'a'++. This will give strange names after vdz. I have implemented names up to vdzzz - inspired by the sd.c code. That should be sufficient for now.

Thanks, both applied.

Cheers,
Rusty.
PS. Am on vacation now, for 4 weeks. Will check mail only occasionally.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

#9 lore raw

Re: [Patch] virtio_blk: implement naming for vda-vdz, vdaa-vdzz, vdaaa-vdzzz

In reply to Christian Borntraeger
Quoted 9 lines · On Friday 01 February 2008 19:05:00 Christian Borntraeger wrote:

Am Freitag, 1. Februar 2008 schrieb Christian Borntraeger:

Quoted 1 line

Right. I will fix that with an additional patch.

This patch goes on top of the minor number patch. Please let me know if you want a merged patch:

Currently virtio_blk creates the disk name combinging "vd" with 'a'++. This will give strange names after vdz. I have implemented names up to vdzzz - inspired by the sd.c code. That should be sufficient for now.

Thanks, both applied.

Cheers,
Rusty.
PS. Am on vacation now, for 4 weeks. Will check mail only occasionally.

#10 lore raw

PATCH 1/1 virtio_blk: implement naming for vda-vdz, vdaa-vdzz, vdaaa-vdzzz

In reply to Christian Borntraeger

Am Freitag, 1. Februar 2008 schrieb Christian Borntraeger:

Quoted 1 line

Right. I will fix that with an additional patch.

This patch goes on top of the minor number patch. Please let me know if you want a merged patch:

Currently virtio_blk creates the disk name combinging "vd" with 'a'++. This will give strange names after vdz. I have implemented names up to vdzzz - inspired by the sd.c code. That should be sufficient for now.

There is one driver in the kernel (driver/s390/block/dasd_genhd.c) that implements names from dasda-dasdzzzz allowing even more disks. Maybe a janitor can come up with a common implementation usable for all kind of block device drivers.

I have tested this patch with 100 disks - seems to work.

---
 drivers/block/virtio_blk.c |   29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)
drivers/block/virtio_blk.c +22 -7
Index: kvm/drivers/block/virtio_blk.c
===================================================================
@@ -11,8 +11,7 @@
 
 MODULE_LICENSE("GPL");
 
-static unsigned char virtblk_index = 'a';
-static int major, minor;
+static int major, index;
 
 struct virtio_blk
 {
@@ -173,6 +172,11 @@ static struct block_device_operations vi
 	.getgeo = virtblk_getgeo,
 };
 
+static int index_to_minor(int index)
+{
+	return index << PART_BITS;
+}
+
 static int virtblk_probe(struct virtio_device *vdev)
 {
 	struct virtio_blk *vblk;
@@ -180,7 +184,7 @@ static int virtblk_probe(struct virtio_d
 	u64 cap;
 	u32 v;
 
-	if (minor >= 1 << MINORBITS)
+	if (index_to_minor(index) >= 1 << MINORBITS)
 		return -ENOSPC;
 
 	vdev->priv = vblk = kmalloc(sizeof(*vblk), GFP_KERNEL);
@@ -219,13 +223,24 @@ static int virtblk_probe(struct virtio_d
 		goto out_put_disk;
 	}
 
-	sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);
+	if (index < 26) {
+		sprintf(vblk->disk->disk_name, "vd%c", 'a' + index % 26);
+	} else if (index < (26 + 1) * 26) {
+		sprintf(vblk->disk->disk_name, "vd%c%c",
+			'a' + index / 26 - 1, 'a' + index % 26);
+	} else {
+		const unsigned int m1 = (index / 26 - 1) / 26 - 1;
+		const unsigned int m2 = (index / 26 - 1) % 26;
+		const unsigned int m3 =  index % 26;
+		sprintf(vblk->disk->disk_name, "vd%c%c%c",
+			'a' + m1, 'a' + m2, 'a' + m3);
+	}
+
 	vblk->disk->major = major;
-	vblk->disk->first_minor = minor;
+	vblk->disk->first_minor = index_to_minor(index);
 	vblk->disk->private_data = vblk;
 	vblk->disk->fops = &virtblk_fops;
-
-	minor += 1 << PART_BITS;
+	index++;
 
 	/* If barriers are supported, tell block layer that queue is ordered */
 	if (vdev->config->feature(vdev, VIRTIO_BLK_F_BARRIER))
#11 lore raw
In reply to Anthony Liguori

Am Donnerstag, 31. Januar 2008 schrieb Anthony Liguori:

Quoted 6 lines

There's are some other limitations to the number of virtio block devices. For instances...

Quoted 1 line

sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);

This gets bogus after 64 disks.

Right. I will fix that with an additional patch.

Quoted 4 lines

We also have a hard limit for
virtio-pci based on the number of PCI slots available. One thing I was considering was whether we should try to support multiple disks per virtio device.

What is the hard limit in the PCI name space?

Christian

#12 lore raw
In reply to Anthony Liguori
Quoted 14 lines · Anthony Liguori
Quoted 9 lines · Christian Borntraeger

Rusty,

currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations.

This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks.

There's are some other limitations to the number of virtio block devices. For instances...

Quoted 7 lines
Quoted 1 line
     sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);

This gets bogus after 64 disks. We also have a hard limit for virtio-pci based on the number of PCI slots available. One thing I was considering was whether we should try to support multiple disks per virtio device.

I would much rather prefer a /dev/vd/dXpY naming scheme, similar to cciss and other large disk installations.

Unfortunately yet another side effect of people not habitually registering major numbers is that the namespace is not as well maintained.

-hpa

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

#13 lore raw
In reply to Anthony Liguori
Quoted 14 lines · Anthony Liguori
Quoted 9 lines · Christian Borntraeger

Rusty,

currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations.

This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks.

There's are some other limitations to the number of virtio block devices. For instances...

Quoted 7 lines
Quoted 1 line
     sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);

This gets bogus after 64 disks. We also have a hard limit for virtio-pci based on the number of PCI slots available. One thing I was considering was whether we should try to support multiple disks per virtio device.

I would much rather prefer a /dev/vd/dXpY naming scheme, similar to cciss and other large disk installations.

Unfortunately yet another side effect of people not habitually registering major numbers is that the namespace is not as well maintained.

-hpa

#14 lore raw
In reply to Christian Borntraeger
Quoted 9 lines · Christian Borntraeger

Rusty,

currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations.

This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks.

There's are some other limitations to the number of virtio block devices. For instances...

Quoted 1 line

sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);

This gets bogus after 64 disks. We also have a hard limit for virtio-pci based on the number of PCI slots available. One thing I was considering was whether we should try to support multiple disks per virtio device.

Otherwise, this patch looks good to me.

Regards,

Anthony Liguori

#15 lore raw
In reply to Christian Borntraeger
Quoted 9 lines · Christian Borntraeger

Rusty,

currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations.

This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks.

There's are some other limitations to the number of virtio block devices. For instances...

Quoted 1 line

sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);

This gets bogus after 64 disks. We also have a hard limit for virtio-pci based on the number of PCI slots available. One thing I was considering was whether we should try to support multiple disks per virtio device.

Otherwise, this patch looks good to me.

Regards,

Anthony Liguori