block/mq: blk map queues by core id
PATCH 1/1 block/mq: blk map queues by core id
under virtual machine environment, cpu topology may differ from normal physical server.
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7
virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7
When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
block/blk-mq-cpumap.c +5 -4
@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/*
@@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
Signature · 1 line
2.14.1.40.g8e62ba1
Quoted 1 line · luferry
under virtual machine environment, cpu topology may differ from normal
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7 cpu_list nr_reserved_tags nr_tags
Quoted 58 lines
physical server.
for example (machine with 4 cores, 2 threads per core):normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
Quoted 2 lines · luferry
under virtual machine environment, cpu topology may differ from normal physical server.
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7 cpu_list nr_reserved_tags nr_tags
Thank you very much!
Dongli Zhang
Quoted 57 lines
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
I just bought one vm , so i cannot access to hypervisor. I will try to build the environment on my desktop. I'm sure about something.
The hypervisor is KVM and disk driver is virtio-blk.
[root@blk-mq ~]# dmesg |grep KVM
[ 0.000000] Hypervisor detected: KVM [ 0.186330] Booting paravirtualized kernel on KVM [ 0.279106] KVM setup async PF for cpu 0 [ 0.420819] KVM setup async PF for cpu 1 [ 0.421682] KVM setup async PF for cpu 2 [ 0.422113] KVM setup async PF for cpu 3 [ 0.422434] KVM setup async PF for cpu 4 [ 0.422434] KVM setup async PF for cpu 5 [ 0.423312] KVM setup async PF for cpu 6 [ 0.423394] KVM setup async PF for cpu 7 [ 0.424125] KVM setup async PF for cpu 8 [ 0.424414] KVM setup async PF for cpu 9 [ 0.424415] KVM setup async PF for cpu 10 [ 0.425329] KVM setup async PF for cpu 11 [ 0.425420] KVM setup async PF for cpu 12 [ 0.426156] KVM setup async PF for cpu 13 [ 0.426431] KVM setup async PF for cpu 14 [ 0.426431] KVM setup async PF for cpu 15
[root@blk-mq ~]# lspci |grep block
00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device
[root@blk-mq ~]# lscpu
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 2 Core(s) per socket: 8
[root@blk-mq ~]# ls /sys/block/vdb/mq/
0 1 2 3
[root@blk-mq ~]# cat /proc/cpuinfo |egrep 'processor|core id'
processor : 0
core id : 0
processor : 1
core id : 0
processor : 2
core id : 1
processor : 3
core id : 1
processor : 4
core id : 2
processor : 5
core id : 2
processor : 6
core id : 3
processor : 7
core id : 3
processor : 8
core id : 4
processor : 9
core id : 4
processor : 10
core id : 5
processor : 11
core id : 5
processor : 12
core id : 6
processor : 13
core id : 6
processor : 14
core id : 7
processor : 15
core id : 7
--before this patch--
[root@blk-mq ~]# cat /sys/block/vdb/mq/*/cpu_list
0, 4, 5, 8, 9, 12, 13
1
2, 6, 7, 10, 11, 14, 15
3
--after this patch--
[root@blk-mq ~]# cat /sys/block/vdb/mq/*/cpu_list
0, 4, 5, 12, 13
1, 6, 7, 14, 15
2, 8, 9
3, 10, 11
I add dump_stack insert blk_mq_map_queues.
[ 1.378680] Call Trace: [ 1.378690] dump_stack+0x5a/0x73 [ 1.378695] blk_mq_map_queues+0x29/0xb0 [ 1.378700] blk_mq_alloc_tag_set+0x1bd/0x2d0 [ 1.378705] virtblk_probe+0x1ae/0x8e4 [virtio_blk] [ 1.378709] virtio_dev_probe+0x18a/0x230 [virtio] [ 1.378713] really_probe+0x215/0x3f0 [ 1.378716] driver_probe_device+0x115/0x130 [ 1.378718] device_driver_attach+0x50/0x60 [ 1.378720] __driver_attach+0xbd/0x140 [ 1.378722] ? device_driver_attach+0x60/0x60 [ 1.378724] bus_for_each_dev+0x67/0xc0 [ 1.378727] ? klist_add_tail+0x57/0x70
Quoted 85 lines · Dongli Zhang
Quoted 2 lines · luferry
under virtual machine environment, cpu topology may differ from normal physical server.
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7cpu_listnr_reserved_tagsnr_tagsThank you very much!
Dongli Zhang
Quoted 57 lines
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
Quoted 90 lines · luferry
I just bought one vm , so i cannot access to hypervisor. I will try to build the environment on my desktop. I'm sure about something.
The hypervisor is KVM and disk driver is virtio-blk.
[root@blk-mq ~]# dmesg |grep KVM[ 0.000000] Hypervisor detected: KVM [ 0.186330] Booting paravirtualized kernel on KVM [ 0.279106] KVM setup async PF for cpu 0 [ 0.420819] KVM setup async PF for cpu 1 [ 0.421682] KVM setup async PF for cpu 2 [ 0.422113] KVM setup async PF for cpu 3 [ 0.422434] KVM setup async PF for cpu 4 [ 0.422434] KVM setup async PF for cpu 5 [ 0.423312] KVM setup async PF for cpu 6 [ 0.423394] KVM setup async PF for cpu 7 [ 0.424125] KVM setup async PF for cpu 8 [ 0.424414] KVM setup async PF for cpu 9 [ 0.424415] KVM setup async PF for cpu 10 [ 0.425329] KVM setup async PF for cpu 11 [ 0.425420] KVM setup async PF for cpu 12 [ 0.426156] KVM setup async PF for cpu 13 [ 0.426431] KVM setup async PF for cpu 14 [ 0.426431] KVM setup async PF for cpu 15[root@blk-mq ~]# lspci |grep block
00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device[root@blk-mq ~]# lscpu
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 2 Core(s) per socket: 8[root@blk-mq ~]# ls
/sys/block/vdb/mq/
0 1 2 3[root@blk-mq ~]# cat
/proc/cpuinfo|egrep 'processor|core id'
processor : 0
core id : 0
processor : 1
core id : 0
processor : 2
core id : 1
processor : 3
core id : 1
processor : 4
core id : 2
processor : 5
core id : 2
processor : 6
core id : 3
processor : 7
core id : 3
processor : 8
core id : 4
processor : 9
core id : 4
processor : 10
core id : 5
processor : 11
core id : 5
processor : 12
core id : 6
processor : 13
core id : 6
processor : 14
core id : 7
processor : 15
core id : 7--before this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 8, 9, 12, 13
1
2, 6, 7, 10, 11, 14, 15
3--after this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 12, 13
1, 6, 7, 14, 15
2, 8, 9
3, 10, 11I add
dump_stackinsertblk_mq_map_queues.
Quoted 13 lines
[ 1.378680] Call Trace: [ 1.378690] dump_stack+0x5a/0x73 [ 1.378695] blk_mq_map_queues+0x29/0xb0 [ 1.378700] blk_mq_alloc_tag_set+0x1bd/0x2d0 [ 1.378705] virtblk_probe+0x1ae/0x8e4 [virtio_blk] [ 1.378709] virtio_dev_probe+0x18a/0x230 [virtio] [ 1.378713] really_probe+0x215/0x3f0 [ 1.378716] driver_probe_device+0x115/0x130 [ 1.378718] device_driver_attach+0x50/0x60 [ 1.378720] __driver_attach+0xbd/0x140 [ 1.378722] ? device_driver_attach+0x60/0x60 [ 1.378724] bus_for_each_dev+0x67/0xc0 [ 1.378727] ? klist_add_tail+0x57/0x70
I am not able to reproduce above call stack when virtio-blk is assigned 4 queues while my qemu cmdline is "-smp 16,sockets=1,cores=8,threads=2".
# cat /sys/block/vda/mq/0/cpu_list 0, 1, 2, 3 # cat /sys/block/vda/mq/1/cpu_list 4, 5, 6, 7 # cat /sys/block/vda/mq/2/cpu_list 8, 9, 10, 11 # cat /sys/block/vda/mq/3/cpu_list 12, 13, 14, 15
I do agree in above case we would have issue if the mapping is established by blk_mq_map_queues().
However, I am just curious how we finally reach at blk_mq_map_queues() from blk_mq_alloc_tag_set()?
It should be something like:
blk_mq_alloc_tag_set()
-> blk_mq_update_queue_map()
-> if (set->ops->map_queues && !is_kdump_kernel())
return set->ops->map_queues(set);
-> else
return blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]);Wouldn't we always have set->ops->map_queues = virtblk_map_queues()?
Or the execution reach at:
virtblk_map_queues()
-> blk_mq_virtio_map_queues()
-> if (!vdev->config->get_vq_affinity)
return blk_mq_map_queues(qmap);
-> else
establish the mapping via get_vq_affinitybut vdev->config->get_vq_affinity == NULL?
For virtio pci, get_vq_affinity is always set. Seems virtio mmio would not set get_vq_affinity.
I used to play with firecracker (by amazon) and it is interesting firecracker uses mmio to setup virtio-blk.
Sorry for disturbing the review of this patch. I just would like to clarify in which scenario we would hit this issue, e.g., when virtio-blk is based on mmio?
Dongli Zhang
Quoted 88 lines
Quoted 85 lines · Dongli Zhang
Quoted 2 lines · luferry
under virtual machine environment, cpu topology may differ from normal physical server.
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7cpu_listnr_reserved_tagsnr_tagsThank you very much!
Dongli Zhang
Quoted 57 lines
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
After reading the code and compare pci device info.
I can reproduce this imbalance under KVM.
When enable virtio-blk with multi queues but with only 2 msix-vector. vp_dev->per_vq_vectors will be false, so blk_mq_virtio_map_queues will fallback to virtio_mq_map_queues
Since virtual machine users cannot change the vector numbers, I think virtio_mq_map_queues should be more friendly with different cpu topology.
448 const struct cpumask *vp_get_vq_affinity(struct virtio_device *vdev, int index) 449 {
450 struct virtio_pci_device *vp_dev = to_vp_device(vdev);
451
452 if (!vp_dev->per_vq_vectors ||
453 vp_dev->vqs[index]->msix_vector == VIRTIO_MSI_NO_VECTOR)
454 return NULL;
455
456 return pci_irq_get_affinity(vp_dev->pci_dev,
457 vp_dev->vqs[index]->msix_vector);
458 }
32 int blk_mq_virtio_map_queues(struct blk_mq_queue_map *qmap,
33 struct virtio_device *vdev, int first_vec)
34 {
35 const struct cpumask *mask;
36 unsigned int queue, cpu;
37
38 if (!vdev->config->get_vq_affinity)
39 goto fallback;
40
41 for (queue = 0; queue < qmap->nr_queues; queue++) {
42 mask = vdev->config->get_vq_affinity(vdev, first_vec + queue); //vp_get_vq_affinity
43 if (!mask)
44 goto fallback;
45
46 for_each_cpu(cpu, mask)
47 qmap->mq_map[cpu] = qmap->queue_offset + queue;
48 }
49
50 return 0;
51 fallback:
52 return blk_mq_map_queues(qmap);
53 }
Quoted 250 lines · Dongli Zhang
Quoted 90 lines · luferry
I just bought one vm , so i cannot access to hypervisor. I will try to build the environment on my desktop. I'm sure about something.
The hypervisor is KVM and disk driver is virtio-blk.
[root@blk-mq ~]# dmesg |grep KVM[ 0.000000] Hypervisor detected: KVM [ 0.186330] Booting paravirtualized kernel on KVM [ 0.279106] KVM setup async PF for cpu 0 [ 0.420819] KVM setup async PF for cpu 1 [ 0.421682] KVM setup async PF for cpu 2 [ 0.422113] KVM setup async PF for cpu 3 [ 0.422434] KVM setup async PF for cpu 4 [ 0.422434] KVM setup async PF for cpu 5 [ 0.423312] KVM setup async PF for cpu 6 [ 0.423394] KVM setup async PF for cpu 7 [ 0.424125] KVM setup async PF for cpu 8 [ 0.424414] KVM setup async PF for cpu 9 [ 0.424415] KVM setup async PF for cpu 10 [ 0.425329] KVM setup async PF for cpu 11 [ 0.425420] KVM setup async PF for cpu 12 [ 0.426156] KVM setup async PF for cpu 13 [ 0.426431] KVM setup async PF for cpu 14 [ 0.426431] KVM setup async PF for cpu 15[root@blk-mq ~]# lspci |grep block
00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device[root@blk-mq ~]# lscpu
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 2 Core(s) per socket: 8[root@blk-mq ~]# ls
/sys/block/vdb/mq/
0 1 2 3[root@blk-mq ~]# cat
/proc/cpuinfo|egrep 'processor|core id'
processor : 0
core id : 0
processor : 1
core id : 0
processor : 2
core id : 1
processor : 3
core id : 1
processor : 4
core id : 2
processor : 5
core id : 2
processor : 6
core id : 3
processor : 7
core id : 3
processor : 8
core id : 4
processor : 9
core id : 4
processor : 10
core id : 5
processor : 11
core id : 5
processor : 12
core id : 6
processor : 13
core id : 6
processor : 14
core id : 7
processor : 15
core id : 7--before this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 8, 9, 12, 13
1
2, 6, 7, 10, 11, 14, 15
3--after this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 12, 13
1, 6, 7, 14, 15
2, 8, 9
3, 10, 11I add
dump_stackinsertblk_mq_map_queues.Quoted 13 lines
[ 1.378680] Call Trace: [ 1.378690] dump_stack+0x5a/0x73 [ 1.378695] blk_mq_map_queues+0x29/0xb0 [ 1.378700] blk_mq_alloc_tag_set+0x1bd/0x2d0 [ 1.378705] virtblk_probe+0x1ae/0x8e4 [virtio_blk] [ 1.378709] virtio_dev_probe+0x18a/0x230 [virtio] [ 1.378713] really_probe+0x215/0x3f0 [ 1.378716] driver_probe_device+0x115/0x130 [ 1.378718] device_driver_attach+0x50/0x60 [ 1.378720] __driver_attach+0xbd/0x140 [ 1.378722] ? device_driver_attach+0x60/0x60 [ 1.378724] bus_for_each_dev+0x67/0xc0 [ 1.378727] ? klist_add_tail+0x57/0x70I am not able to reproduce above call stack when virtio-blk is assigned 4 queues while my qemu cmdline is "-smp 16,sockets=1,cores=8,threads=2".
# cat /sys/block/vda/mq/0/cpu_list 0, 1, 2, 3 # cat /sys/block/vda/mq/1/cpu_list 4, 5, 6, 7 # cat /sys/block/vda/mq/2/cpu_list 8, 9, 10, 11 # cat /sys/block/vda/mq/3/cpu_list 12, 13, 14, 15I do agree in above case we would have issue if the mapping is established by
blk_mq_map_queues().However, I am just curious how we finally reach at
blk_mq_map_queues()fromblk_mq_alloc_tag_set()?It should be something like:
blk_mq_alloc_tag_set()
->blk_mq_update_queue_map()-> if (set->ops->map_queues && !is_kdump_kernel()) return set->ops->map_queues(set); -> else return blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]);Wouldn't we always have
set->ops->map_queues=virtblk_map_queues()?Or the execution reach at:
virtblk_map_queues()
->blk_mq_virtio_map_queues()-> if (!vdev->config->get_vq_affinity) return blk_mq_map_queues(qmap); -> else establish the mapping via get_vq_affinitybut
vdev->config->get_vq_affinity== NULL?For virtio pci,
get_vq_affinityis always set. Seems virtio mmio would not setget_vq_affinity.I used to play with firecracker (by amazon) and it is interesting firecracker uses mmio to setup virtio-blk.
Sorry for disturbing the review of this patch. I just would like to clarify in which scenario we would hit this issue, e.g., when virtio-blk is based on mmio?
Dongli Zhang
Quoted 88 lines
Quoted 85 lines · Dongli Zhang
Quoted 2 lines · luferry
under virtual machine environment, cpu topology may differ from normal physical server.
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7cpu_listnr_reserved_tagsnr_tagsThank you very much!
Dongli Zhang
Quoted 57 lines
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
sorry, messed up some func name
When enable virtio-blk with multi queues but with only 2 msix-vector. vp_dev->per_vq_vectors will be false, so blk_mq_virtio_map_queues will fallback to blk_mq_map_queues
Since virtual machine users cannot change the vector numbers, I think blk_mq_map_queues should be more friendly with different cpu topology.
Quoted 301 lines · luferry
After reading the code and compare pci device info.
I can reproduce this imbalance under KVM.
When enable virtio-blk with multi queues but with only 2 msix-vector.vp_dev->per_vq_vectorswill be false, soblk_mq_virtio_map_queueswill fallback tovirtio_mq_map_queuesSince virtual machine users cannot change the vector numbers, I think
virtio_mq_map_queuesshould be more friendly with different cpu topology.448 const struct cpumask *vp_get_vq_affinity(struct
virtio_device*vdev, int index) 449 {
450 structvirtio_pci_device*vp_dev=to_vp_device(vdev);
451
452 if (!vp_dev->per_vq_vectors ||
453 vp_dev->vqs[index]->msix_vector ==VIRTIO_MSI_NO_VECTOR)
454 return NULL;
455
456 return pci_irq_get_affinity(vp_dev->pci_dev,
457 vp_dev->vqs[index]->msix_vector);
458 }32 int blk_mq_virtio_map_queues(struct
blk_mq_queue_map*qmap,
33 structvirtio_device*vdev, intfirst_vec)
34 {
35 const struct cpumask *mask;
36 unsigned int queue, cpu;
37
38 if (!vdev->config->get_vq_affinity)
39 goto fallback;
40
41 for (queue = 0; queue <qmap->nr_queues; queue++) {
42 mask =vdev->config->get_vq_affinity(vdev, first_vec + queue); //vp_get_vq_affinity
43 if (!mask)
44 goto fallback;
45
46for_each_cpu(cpu, mask)
47 qmap->mq_map[cpu] =qmap->queue_offset+ queue;
48 }
49
50 return 0;
51 fallback:
52 returnblk_mq_map_queues(qmap);
53 }Quoted 250 lines · Dongli Zhang
Quoted 90 lines · luferry
I just bought one vm , so i cannot access to hypervisor. I will try to build the environment on my desktop. I'm sure about something.
The hypervisor is KVM and disk driver is virtio-blk.
[root@blk-mq ~]# dmesg |grep KVM[ 0.000000] Hypervisor detected: KVM [ 0.186330] Booting paravirtualized kernel on KVM [ 0.279106] KVM setup async PF for cpu 0 [ 0.420819] KVM setup async PF for cpu 1 [ 0.421682] KVM setup async PF for cpu 2 [ 0.422113] KVM setup async PF for cpu 3 [ 0.422434] KVM setup async PF for cpu 4 [ 0.422434] KVM setup async PF for cpu 5 [ 0.423312] KVM setup async PF for cpu 6 [ 0.423394] KVM setup async PF for cpu 7 [ 0.424125] KVM setup async PF for cpu 8 [ 0.424414] KVM setup async PF for cpu 9 [ 0.424415] KVM setup async PF for cpu 10 [ 0.425329] KVM setup async PF for cpu 11 [ 0.425420] KVM setup async PF for cpu 12 [ 0.426156] KVM setup async PF for cpu 13 [ 0.426431] KVM setup async PF for cpu 14 [ 0.426431] KVM setup async PF for cpu 15[root@blk-mq ~]# lspci |grep block
00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device[root@blk-mq ~]# lscpu
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 2 Core(s) per socket: 8[root@blk-mq ~]# ls
/sys/block/vdb/mq/
0 1 2 3[root@blk-mq ~]# cat
/proc/cpuinfo|egrep 'processor|core id'
processor : 0
core id : 0
processor : 1
core id : 0
processor : 2
core id : 1
processor : 3
core id : 1
processor : 4
core id : 2
processor : 5
core id : 2
processor : 6
core id : 3
processor : 7
core id : 3
processor : 8
core id : 4
processor : 9
core id : 4
processor : 10
core id : 5
processor : 11
core id : 5
processor : 12
core id : 6
processor : 13
core id : 6
processor : 14
core id : 7
processor : 15
core id : 7--before this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 8, 9, 12, 13
1
2, 6, 7, 10, 11, 14, 15
3--after this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 12, 13
1, 6, 7, 14, 15
2, 8, 9
3, 10, 11I add
dump_stackinsertblk_mq_map_queues.Quoted 13 lines
[ 1.378680] Call Trace: [ 1.378690] dump_stack+0x5a/0x73 [ 1.378695] blk_mq_map_queues+0x29/0xb0 [ 1.378700] blk_mq_alloc_tag_set+0x1bd/0x2d0 [ 1.378705] virtblk_probe+0x1ae/0x8e4 [virtio_blk] [ 1.378709] virtio_dev_probe+0x18a/0x230 [virtio] [ 1.378713] really_probe+0x215/0x3f0 [ 1.378716] driver_probe_device+0x115/0x130 [ 1.378718] device_driver_attach+0x50/0x60 [ 1.378720] __driver_attach+0xbd/0x140 [ 1.378722] ? device_driver_attach+0x60/0x60 [ 1.378724] bus_for_each_dev+0x67/0xc0 [ 1.378727] ? klist_add_tail+0x57/0x70I am not able to reproduce above call stack when virtio-blk is assigned 4 queues while my qemu cmdline is "-smp 16,sockets=1,cores=8,threads=2".
# cat /sys/block/vda/mq/0/cpu_list 0, 1, 2, 3 # cat /sys/block/vda/mq/1/cpu_list 4, 5, 6, 7 # cat /sys/block/vda/mq/2/cpu_list 8, 9, 10, 11 # cat /sys/block/vda/mq/3/cpu_list 12, 13, 14, 15I do agree in above case we would have issue if the mapping is established by
blk_mq_map_queues().However, I am just curious how we finally reach at
blk_mq_map_queues()fromblk_mq_alloc_tag_set()?It should be something like:
blk_mq_alloc_tag_set()
->blk_mq_update_queue_map()-> if (set->ops->map_queues && !is_kdump_kernel()) return set->ops->map_queues(set); -> else return blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]);Wouldn't we always have
set->ops->map_queues=virtblk_map_queues()?Or the execution reach at:
virtblk_map_queues()
->blk_mq_virtio_map_queues()-> if (!vdev->config->get_vq_affinity) return blk_mq_map_queues(qmap); -> else establish the mapping via get_vq_affinitybut
vdev->config->get_vq_affinity== NULL?For virtio pci,
get_vq_affinityis always set. Seems virtio mmio would not setget_vq_affinity.I used to play with firecracker (by amazon) and it is interesting firecracker uses mmio to setup virtio-blk.
Sorry for disturbing the review of this patch. I just would like to clarify in which scenario we would hit this issue, e.g., when virtio-blk is based on mmio?
Dongli Zhang
Quoted 88 lines
Quoted 85 lines · Dongli Zhang
Quoted 2 lines · luferry
under virtual machine environment, cpu topology may differ from normal physical server.
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7cpu_listnr_reserved_tagsnr_tagsThank you very much!
Dongli Zhang
Quoted 57 lines
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
Quoted 7 lines · luferry
sorry, messed up some func name
When enable virtio-blk with multi queues but with only 2 msix-vector.
vp_dev->per_vq_vectorswill be false, soblk_mq_virtio_map_queueswill fallback toblk_mq_map_queuesSince virtual machine users cannot change the vector numbers, I think
blk_mq_map_queuesshould be more friendly with different cpu topology.
Yes, we will fallback to blk_mq_map_queues if there is lack of vectors.
Here is the qemu cmdline:
"-smp 8,sockets=1,cores=4,threads=2"
"-device virtio-blk-pci,drive=drive0,id=device0,num-queues=4,vectors=2"
# cat /sys/block/vda/mq/0/cpu_list 0, 4, 5 root@vm:/home/zhang# cat /sys/block/vda/mq/1/cpu_list 1 root@vm:/home/zhang# cat /sys/block/vda/mq/2/cpu_list 2, 6, 7 root@vm:/home/zhang# cat /sys/block/vda/mq/3/cpu_list 3
How about to put how to hit the issue in commit message so people would be able to know the scenario?
Dongli Zhang
Quoted 306 lines
Quoted 301 lines · luferry
After reading the code and compare pci device info.
I can reproduce this imbalance under KVM.
When enable virtio-blk with multi queues but with only 2 msix-vector.vp_dev->per_vq_vectorswill be false, soblk_mq_virtio_map_queueswill fallback tovirtio_mq_map_queuesSince virtual machine users cannot change the vector numbers, I think
virtio_mq_map_queuesshould be more friendly with different cpu topology.448 const struct cpumask *vp_get_vq_affinity(struct
virtio_device*vdev, int index) 449 {
450 structvirtio_pci_device*vp_dev=to_vp_device(vdev);
451
452 if (!vp_dev->per_vq_vectors ||
453 vp_dev->vqs[index]->msix_vector ==VIRTIO_MSI_NO_VECTOR)
454 return NULL;
455
456 return pci_irq_get_affinity(vp_dev->pci_dev,
457 vp_dev->vqs[index]->msix_vector);
458 }32 int blk_mq_virtio_map_queues(struct
blk_mq_queue_map*qmap,
33 structvirtio_device*vdev, intfirst_vec)
34 {
35 const struct cpumask *mask;
36 unsigned int queue, cpu;
37
38 if (!vdev->config->get_vq_affinity)
39 goto fallback;
40
41 for (queue = 0; queue <qmap->nr_queues; queue++) {
42 mask =vdev->config->get_vq_affinity(vdev, first_vec + queue); //vp_get_vq_affinity
43 if (!mask)
44 goto fallback;
45
46for_each_cpu(cpu, mask)
47 qmap->mq_map[cpu] =qmap->queue_offset+ queue;
48 }
49
50 return 0;
51 fallback:
52 returnblk_mq_map_queues(qmap);
53 }Quoted 250 lines · Dongli Zhang
Quoted 90 lines · luferry
I just bought one vm , so i cannot access to hypervisor. I will try to build the environment on my desktop. I'm sure about something.
The hypervisor is KVM and disk driver is virtio-blk.
[root@blk-mq ~]# dmesg |grep KVM[ 0.000000] Hypervisor detected: KVM [ 0.186330] Booting paravirtualized kernel on KVM [ 0.279106] KVM setup async PF for cpu 0 [ 0.420819] KVM setup async PF for cpu 1 [ 0.421682] KVM setup async PF for cpu 2 [ 0.422113] KVM setup async PF for cpu 3 [ 0.422434] KVM setup async PF for cpu 4 [ 0.422434] KVM setup async PF for cpu 5 [ 0.423312] KVM setup async PF for cpu 6 [ 0.423394] KVM setup async PF for cpu 7 [ 0.424125] KVM setup async PF for cpu 8 [ 0.424414] KVM setup async PF for cpu 9 [ 0.424415] KVM setup async PF for cpu 10 [ 0.425329] KVM setup async PF for cpu 11 [ 0.425420] KVM setup async PF for cpu 12 [ 0.426156] KVM setup async PF for cpu 13 [ 0.426431] KVM setup async PF for cpu 14 [ 0.426431] KVM setup async PF for cpu 15[root@blk-mq ~]# lspci |grep block
00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device[root@blk-mq ~]# lscpu
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 2 Core(s) per socket: 8[root@blk-mq ~]# ls
/sys/block/vdb/mq/
0 1 2 3[root@blk-mq ~]# cat
/proc/cpuinfo|egrep 'processor|core id'
processor : 0
core id : 0
processor : 1
core id : 0
processor : 2
core id : 1
processor : 3
core id : 1
processor : 4
core id : 2
processor : 5
core id : 2
processor : 6
core id : 3
processor : 7
core id : 3
processor : 8
core id : 4
processor : 9
core id : 4
processor : 10
core id : 5
processor : 11
core id : 5
processor : 12
core id : 6
processor : 13
core id : 6
processor : 14
core id : 7
processor : 15
core id : 7--before this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 8, 9, 12, 13
1
2, 6, 7, 10, 11, 14, 15
3--after this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 12, 13
1, 6, 7, 14, 15
2, 8, 9
3, 10, 11I add
dump_stackinsertblk_mq_map_queues.Quoted 13 lines
[ 1.378680] Call Trace: [ 1.378690] dump_stack+0x5a/0x73 [ 1.378695] blk_mq_map_queues+0x29/0xb0 [ 1.378700] blk_mq_alloc_tag_set+0x1bd/0x2d0 [ 1.378705] virtblk_probe+0x1ae/0x8e4 [virtio_blk] [ 1.378709] virtio_dev_probe+0x18a/0x230 [virtio] [ 1.378713] really_probe+0x215/0x3f0 [ 1.378716] driver_probe_device+0x115/0x130 [ 1.378718] device_driver_attach+0x50/0x60 [ 1.378720] __driver_attach+0xbd/0x140 [ 1.378722] ? device_driver_attach+0x60/0x60 [ 1.378724] bus_for_each_dev+0x67/0xc0 [ 1.378727] ? klist_add_tail+0x57/0x70I am not able to reproduce above call stack when virtio-blk is assigned 4 queues while my qemu cmdline is "-smp 16,sockets=1,cores=8,threads=2".
# cat /sys/block/vda/mq/0/cpu_list 0, 1, 2, 3 # cat /sys/block/vda/mq/1/cpu_list 4, 5, 6, 7 # cat /sys/block/vda/mq/2/cpu_list 8, 9, 10, 11 # cat /sys/block/vda/mq/3/cpu_list 12, 13, 14, 15I do agree in above case we would have issue if the mapping is established by
blk_mq_map_queues().However, I am just curious how we finally reach at
blk_mq_map_queues()fromblk_mq_alloc_tag_set()?It should be something like:
blk_mq_alloc_tag_set()
->blk_mq_update_queue_map()-> if (set->ops->map_queues && !is_kdump_kernel()) return set->ops->map_queues(set); -> else return blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]);Wouldn't we always have
set->ops->map_queues=virtblk_map_queues()?Or the execution reach at:
virtblk_map_queues()
->blk_mq_virtio_map_queues()-> if (!vdev->config->get_vq_affinity) return blk_mq_map_queues(qmap); -> else establish the mapping via get_vq_affinitybut
vdev->config->get_vq_affinity== NULL?For virtio pci,
get_vq_affinityis always set. Seems virtio mmio would not setget_vq_affinity.I used to play with firecracker (by amazon) and it is interesting firecracker uses mmio to setup virtio-blk.
Sorry for disturbing the review of this patch. I just would like to clarify in which scenario we would hit this issue, e.g., when virtio-blk is based on mmio?
Dongli Zhang
Quoted 88 lines
Quoted 85 lines · Dongli Zhang
Quoted 2 lines · luferry
under virtual machine environment, cpu topology may differ from normal physical server.
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7cpu_listnr_reserved_tagsnr_tagsThank you very much!
Dongli Zhang
Quoted 57 lines
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }
Thanks for your kindly suggestions, I will reformat this patch later!
Quoted 338 lines · Dongli Zhang
Quoted 7 lines · luferry
sorry, messed up some func name
When enable virtio-blk with multi queues but with only 2 msix-vector.
vp_dev->per_vq_vectorswill be false, soblk_mq_virtio_map_queueswill fallback toblk_mq_map_queuesSince virtual machine users cannot change the vector numbers, I think
blk_mq_map_queuesshould be more friendly with different cpu topology.Yes, we will fallback to
blk_mq_map_queuesif there is lack of vectors.Here is the qemu cmdline:
"-smp 8,sockets=1,cores=4,threads=2"
"-device virtio-blk-pci,drive=drive0,id=device0,num-queues=4,vectors=2"# cat /sys/block/vda/mq/0/cpu_list 0, 4, 5 root@vm:/home/zhang# cat /sys/block/vda/mq/1/cpu_list 1 root@vm:/home/zhang# cat /sys/block/vda/mq/2/cpu_list 2, 6, 7 root@vm:/home/zhang# cat /sys/block/vda/mq/3/cpu_list 3How about to put how to hit the issue in commit message so people would be able to know the scenario?
Dongli Zhang
Quoted 306 lines
Quoted 301 lines · luferry
After reading the code and compare pci device info.
I can reproduce this imbalance under KVM.
When enable virtio-blk with multi queues but with only 2 msix-vector.vp_dev->per_vq_vectorswill be false, soblk_mq_virtio_map_queueswill fallback tovirtio_mq_map_queuesSince virtual machine users cannot change the vector numbers, I think
virtio_mq_map_queuesshould be more friendly with different cpu topology.448 const struct cpumask *vp_get_vq_affinity(struct
virtio_device*vdev, int index) 449 {
450 structvirtio_pci_device*vp_dev=to_vp_device(vdev);
451
452 if (!vp_dev->per_vq_vectors ||
453 vp_dev->vqs[index]->msix_vector ==VIRTIO_MSI_NO_VECTOR)
454 return NULL;
455
456 return pci_irq_get_affinity(vp_dev->pci_dev,
457 vp_dev->vqs[index]->msix_vector);
458 }32 int blk_mq_virtio_map_queues(struct
blk_mq_queue_map*qmap,
33 structvirtio_device*vdev, intfirst_vec)
34 {
35 const struct cpumask *mask;
36 unsigned int queue, cpu;
37
38 if (!vdev->config->get_vq_affinity)
39 goto fallback;
40
41 for (queue = 0; queue <qmap->nr_queues; queue++) {
42 mask =vdev->config->get_vq_affinity(vdev, first_vec + queue); //vp_get_vq_affinity
43 if (!mask)
44 goto fallback;
45
46for_each_cpu(cpu, mask)
47 qmap->mq_map[cpu] =qmap->queue_offset+ queue;
48 }
49
50 return 0;
51 fallback:
52 returnblk_mq_map_queues(qmap);
53 }Quoted 250 lines · Dongli Zhang
Quoted 90 lines · luferry
I just bought one vm , so i cannot access to hypervisor. I will try to build the environment on my desktop. I'm sure about something.
The hypervisor is KVM and disk driver is virtio-blk.
[root@blk-mq ~]# dmesg |grep KVM[ 0.000000] Hypervisor detected: KVM [ 0.186330] Booting paravirtualized kernel on KVM [ 0.279106] KVM setup async PF for cpu 0 [ 0.420819] KVM setup async PF for cpu 1 [ 0.421682] KVM setup async PF for cpu 2 [ 0.422113] KVM setup async PF for cpu 3 [ 0.422434] KVM setup async PF for cpu 4 [ 0.422434] KVM setup async PF for cpu 5 [ 0.423312] KVM setup async PF for cpu 6 [ 0.423394] KVM setup async PF for cpu 7 [ 0.424125] KVM setup async PF for cpu 8 [ 0.424414] KVM setup async PF for cpu 9 [ 0.424415] KVM setup async PF for cpu 10 [ 0.425329] KVM setup async PF for cpu 11 [ 0.425420] KVM setup async PF for cpu 12 [ 0.426156] KVM setup async PF for cpu 13 [ 0.426431] KVM setup async PF for cpu 14 [ 0.426431] KVM setup async PF for cpu 15[root@blk-mq ~]# lspci |grep block
00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device[root@blk-mq ~]# lscpu
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 2 Core(s) per socket: 8[root@blk-mq ~]# ls
/sys/block/vdb/mq/
0 1 2 3[root@blk-mq ~]# cat
/proc/cpuinfo|egrep 'processor|core id'
processor : 0
core id : 0
processor : 1
core id : 0
processor : 2
core id : 1
processor : 3
core id : 1
processor : 4
core id : 2
processor : 5
core id : 2
processor : 6
core id : 3
processor : 7
core id : 3
processor : 8
core id : 4
processor : 9
core id : 4
processor : 10
core id : 5
processor : 11
core id : 5
processor : 12
core id : 6
processor : 13
core id : 6
processor : 14
core id : 7
processor : 15
core id : 7--before this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 8, 9, 12, 13
1
2, 6, 7, 10, 11, 14, 15
3--after this patch--
[root@blk-mq ~]# cat/sys/block/vdb/mq/*/cpu_list
0, 4, 5, 12, 13
1, 6, 7, 14, 15
2, 8, 9
3, 10, 11I add
dump_stackinsertblk_mq_map_queues.Quoted 13 lines
[ 1.378680] Call Trace: [ 1.378690] dump_stack+0x5a/0x73 [ 1.378695] blk_mq_map_queues+0x29/0xb0 [ 1.378700] blk_mq_alloc_tag_set+0x1bd/0x2d0 [ 1.378705] virtblk_probe+0x1ae/0x8e4 [virtio_blk] [ 1.378709] virtio_dev_probe+0x18a/0x230 [virtio] [ 1.378713] really_probe+0x215/0x3f0 [ 1.378716] driver_probe_device+0x115/0x130 [ 1.378718] device_driver_attach+0x50/0x60 [ 1.378720] __driver_attach+0xbd/0x140 [ 1.378722] ? device_driver_attach+0x60/0x60 [ 1.378724] bus_for_each_dev+0x67/0xc0 [ 1.378727] ? klist_add_tail+0x57/0x70I am not able to reproduce above call stack when virtio-blk is assigned 4 queues while my qemu cmdline is "-smp 16,sockets=1,cores=8,threads=2".
# cat /sys/block/vda/mq/0/cpu_list 0, 1, 2, 3 # cat /sys/block/vda/mq/1/cpu_list 4, 5, 6, 7 # cat /sys/block/vda/mq/2/cpu_list 8, 9, 10, 11 # cat /sys/block/vda/mq/3/cpu_list 12, 13, 14, 15I do agree in above case we would have issue if the mapping is established by
blk_mq_map_queues().However, I am just curious how we finally reach at
blk_mq_map_queues()fromblk_mq_alloc_tag_set()?It should be something like:
blk_mq_alloc_tag_set()
->blk_mq_update_queue_map()-> if (set->ops->map_queues && !is_kdump_kernel()) return set->ops->map_queues(set); -> else return blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]);Wouldn't we always have
set->ops->map_queues=virtblk_map_queues()?Or the execution reach at:
virtblk_map_queues()
->blk_mq_virtio_map_queues()-> if (!vdev->config->get_vq_affinity) return blk_mq_map_queues(qmap); -> else establish the mapping via get_vq_affinitybut
vdev->config->get_vq_affinity== NULL?For virtio pci,
get_vq_affinityis always set. Seems virtio mmio would not setget_vq_affinity.I used to play with firecracker (by amazon) and it is interesting firecracker uses mmio to setup virtio-blk.
Sorry for disturbing the review of this patch. I just would like to clarify in which scenario we would hit this issue, e.g., when virtio-blk is based on mmio?
Dongli Zhang
Quoted 88 lines
Quoted 85 lines · Dongli Zhang
Quoted 2 lines · luferry
under virtual machine environment, cpu topology may differ from normal physical server.
Would mind share the name of virtual machine monitor, the command line if available, and which device to reproduce.
For instance, I am not able to reproduce with qemu nvme or virtio-blk as I assume they use pci or virtio specific mapper to establish the mapping.
E.g., with qemu and nvme: -smp 8,sockets=1,cores=4,threads=2
Indeed I use three queues instead of twp as one is reserved for admin.
# ls /sys/block/nvme0n1/mq/* /sys/block/nvme0n1/mq/0: cpu0 cpu1 cpu2 cpu3 cpu_list nr_reserved_tags nr_tags
/sys/block/nvme0n1/mq/1:
cpu4 cpu5 cpu6 cpu7cpu_listnr_reserved_tagsnr_tagsThank you very much!
Dongli Zhang
Quoted 57 lines
for example (machine with 4 cores, 2 threads per core):
normal physical server:
core-id thread-0-id thread-1-id 0 0 4 1 1 5 2 2 6 3 3 7virtual machine:
core-id thread-0-id thread-1-id 0 0 1 1 2 3 2 4 5 3 6 7When attach disk with two queues, all the even numbered cpus will be mapped to queue 0. Under virtual machine, all the cpus is followed by its sibling cpu.Before this patch, all the odd numbered cpus will also be mapped to queue 0, can cause serious imbalance.this will lead to performance impact on system IO
So suggest to allocate cpu map by core id, this can be more currency
Signed-off-by luferry <[email protected]>--- block/blk-mq-cpumap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)block/blk-mq-cpumap.c +5
-4@@ -35,7 +35,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) {unsigned int *map = qmap->mq_map;unsigned int nr_queues = qmap->nr_queues; -unsigned int cpu, first_sibling; +unsigned int cpu, first_sibling, core =0 ;for_each_possible_cpu (cpu) {/* @@ -48,9 +48,10 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap) map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); }else { first_sibling =get_first_sibling (cpu); -if (first_sibling == cpu) - map[cpu] =cpu_to_queue_index (qmap, nr_queues, cpu); -else +if (first_sibling == cpu) { + map[cpu] =cpu_to_queue_index (qmap, nr_queues, core); + core++; + }else map[cpu] = map[first_sibling]; } }