nvme: fix out-of-bounds access during irq vectors allocation
PATCH 1/1 nvme: fix out-of-bounds access during irq vectors allocation
While reducing irq_queues in the do-while loop in nvme_setup_irqs(), the reduction of irq_sets[] is behind irq_queues. Below is an example.
On a 8 cpu platform, with default setting, nvme_setup_irqs() begin with irq_queues = 8 (but when allocating irq vectors it will become 9 due to the admin queue), affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8. If MSI-X resources are not enough, then the do-while loop will reduce irq vectors:
The 1st time call pci_alloc_irq_vectors_affinity(), irq_queues = 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time call pci_alloc_irq_vectors_affinity(), irq_queues = 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 3rd time call pci_alloc_irq_vectors_affinity(), irq_queues = 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7
However, this will cause an out of bounds access in __pci_enable_msix() --> ... --> irq_create_affinity_masks() --> irq_build_affinity_masks().
In the 2nd round of reduction, let's pay attention to the calling of irq_build_affinity_masks(affd, curvec, this_vecs, curvec, node_to_cpumask, masks):
The number of elements in masks is 8 (depends on nvecs which is equal to irq_queues), curvec is 1 (depends on affd.pre_vectors), then irq_build_affinity_masks() will access 8 elements in masks (depends on this_vecs which is equal to affd.sets[0]), and the last element is out of bounds.
So the root cause is affd.sets[] + affd.pre_vectors should not be larger than vectors to be allocated. In this patch we introduce alloc_queues to indicate how many queues to allocate (not reuse irq_queues), and so we can adjust affd.set[] correctly (depends on irq_queues) to avoid out of bounds access.
After this patch:
The 1st time call pci_alloc_irq_vectors_affinity(),irq_queues = 8, alloc_queues = 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time call pci_alloc_irq_vectors_affinity(),irq_queues = 7, alloc_queues = 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7 The 3rd time call pci_alloc_irq_vectors_affinity(),irq_queues = 6, alloc_queues = 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 6
--- drivers/nvme/host/pci.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
drivers/nvme/host/pci.c +7 -5
@@ -2084,7 +2084,7 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues) .sets = irq_sets, };int result =0 ; -unsigned int irq_queues, this_p_queues; +unsigned int irq_queues, this_p_queues, alloc_queues;/* * Poll queues don't need interrupts, but we need at least one IO
@@ -2116,11 +2116,13 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues)* 1 + 1 queues, just ask for a single vector. We'll share * that between the single IO queue and the admin queue. */ -if (result >=0 && irq_queues >1 ) - irq_queues = irq_sets[0 ] + irq_sets[1 ] +1 ; +if (irq_queues ==1 ) + alloc_queues =1 ; +else + alloc_queues = irq_sets[0 ] + irq_sets[1 ] +1 ; - result =pci_alloc_irq_vectors_affinity (pdev, irq_queues, - irq_queues, + result =pci_alloc_irq_vectors_affinity (pdev, alloc_queues, + alloc_queues,PCI_IRQ_ALL_TYPES |PCI_IRQ_AFFINITY , &affd);/*
Signature · 1 line
2.7.0
Jens Axboe
maintainerPATCH 1/1 nvme: fix out-of-bounds access during irq vectors allocation
Quoted 34 lines · Huacai Chen
While reducing
irq_queuesin the do-while loop innvme_setup_irqs(), the reduction of irq_sets[] is behindirq_queues. Below is an example.On a 8 cpu platform, with default setting,
nvme_setup_irqs()begin withirq_queues= 8 (but when allocating irq vectors it will become 9 due to the admin queue), affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8. If MSI-X resources are not enough, then the do-while loop will reduce irq vectors:The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7However, this will cause an out of bounds access in
__pci_enable_msix()--> ... -->irq_create_affinity_masks()-->irq_build_affinity_masks().In the 2nd round of reduction, let's pay attention to the calling of
irq_build_affinity_masks(affd, curvec, this_vecs, curvec, node_to_cpumask, masks):The number of elements in masks is 8 (depends on nvecs which is equal to
irq_queues), curvec is 1 (depends on affd.pre_vectors), thenirq_build_affinity_masks()will access 8 elements in masks (depends onthis_vecswhich is equal to affd.sets[0]), and the last element is out of bounds.So the root cause is affd.sets[] + affd.pre_vectors should not be larger than vectors to be allocated. In this patch we introduce
alloc_queuesto indicate how many queues to allocate (not reuseirq_queues), and so we can adjust affd.set[] correctly (depends onirq_queues) to avoid out of bounds access.
Quoted 8 lines
After this patch:
The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 8,alloc_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7,alloc_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 6,alloc_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 6
We currently have this one queued up:
can you check if it fixes the issue for you?
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerPATCH 1/1 nvme: fix out-of-bounds access during irq vectors allocation
Quoted 44 lines · Jens Axboe
Quoted 34 lines · Huacai Chen
While reducing
irq_queuesin the do-while loop innvme_setup_irqs(), the reduction of irq_sets[] is behindirq_queues. Below is an example.On a 8 cpu platform, with default setting,
nvme_setup_irqs()begin withirq_queues= 8 (but when allocating irq vectors it will become 9 due to the admin queue), affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8. If MSI-X resources are not enough, then the do-while loop will reduce irq vectors:The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7However, this will cause an out of bounds access in
__pci_enable_msix()--> ... -->irq_create_affinity_masks()-->irq_build_affinity_masks().In the 2nd round of reduction, let's pay attention to the calling of
irq_build_affinity_masks(affd, curvec, this_vecs, curvec, node_to_cpumask, masks):The number of elements in masks is 8 (depends on nvecs which is equal to
irq_queues), curvec is 1 (depends on affd.pre_vectors), thenirq_build_affinity_masks()will access 8 elements in masks (depends onthis_vecswhich is equal to affd.sets[0]), and the last element is out of bounds.So the root cause is affd.sets[] + affd.pre_vectors should not be larger than vectors to be allocated. In this patch we introduce
alloc_queuesto indicate how many queues to allocate (not reuseirq_queues), and so we can adjust affd.set[] correctly (depends onirq_queues) to avoid out of bounds access.Quoted 8 lines
After this patch:
The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 8,alloc_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7,alloc_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 6,alloc_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 6
Quoted 5 lines
We currently have this one queued up:
can you check if it fixes the issue for you?
Nevermind, took a closer look, and this looks like a different issue.
Signature · 1 line
Jens Axboe
PATCH 1/1 nvme: fix out-of-bounds access during irq vectors allocation
Quoted 45 lines · Jens Axboe
Quoted 44 lines · Jens Axboe
Quoted 42 lines · Huacai Chen
While reducing
irq_queuesin the do-while loop innvme_setup_irqs(), the reduction of irq_sets[] is behindirq_queues. Below is an example.On a 8 cpu platform, with default setting,
nvme_setup_irqs()begin withirq_queues= 8 (but when allocating irq vectors it will become 9 due to the admin queue), affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8. If MSI-X resources are not enough, then the do-while loop will reduce irq vectors:The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7However, this will cause an out of bounds access in
__pci_enable_msix()--> ... -->irq_create_affinity_masks()-->irq_build_affinity_masks().In the 2nd round of reduction, let's pay attention to the calling of
irq_build_affinity_masks(affd, curvec, this_vecs, curvec, node_to_cpumask, masks):The number of elements in masks is 8 (depends on nvecs which is equal to
irq_queues), curvec is 1 (depends on affd.pre_vectors), thenirq_build_affinity_masks()will access 8 elements in masks (depends onthis_vecswhich is equal to affd.sets[0]), and the last element is out of bounds.So the root cause is affd.sets[] + affd.pre_vectors should not be larger than vectors to be allocated. In this patch we introduce
alloc_queuesto indicate how many queues to allocate (not reuseirq_queues), and so we can adjust affd.set[] correctly (depends onirq_queues) to avoid out of bounds access.After this patch:
The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 8,alloc_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7,alloc_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 6,alloc_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 6
Quoted 7 lines
Quoted 5 lines
We currently have this one queued up:
can you check if it fixes the issue for you?
Nevermind, took a closer look, and this looks like a different issue.
The solutions look different, but I think they're both targeting the same problem, which is the older code had been accounting for vectors and queues differenting in the first iteration than subsequent ones. I think Ming's patch will probably fix the issue raised here and worth a shot at testing it.
Jens Axboe
maintainerPATCH 1/1 nvme: fix out-of-bounds access during irq vectors allocation
Quoted 52 lines · Keith Busch
Quoted 51 lines · Jens Axboe
Quoted 49 lines · Jens Axboe
Quoted 34 lines · Huacai Chen
While reducing
irq_queuesin the do-while loop innvme_setup_irqs(), the reduction of irq_sets[] is behindirq_queues. Below is an example.On a 8 cpu platform, with default setting,
nvme_setup_irqs()begin withirq_queues= 8 (but when allocating irq vectors it will become 9 due to the admin queue), affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8. If MSI-X resources are not enough, then the do-while loop will reduce irq vectors:The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7However, this will cause an out of bounds access in
__pci_enable_msix()--> ... -->irq_create_affinity_masks()-->irq_build_affinity_masks().In the 2nd round of reduction, let's pay attention to the calling of
irq_build_affinity_masks(affd, curvec, this_vecs, curvec, node_to_cpumask, masks):The number of elements in masks is 8 (depends on nvecs which is equal to
irq_queues), curvec is 1 (depends on affd.pre_vectors), thenirq_build_affinity_masks()will access 8 elements in masks (depends onthis_vecswhich is equal to affd.sets[0]), and the last element is out of bounds.So the root cause is affd.sets[] + affd.pre_vectors should not be larger than vectors to be allocated. In this patch we introduce
alloc_queuesto indicate how many queues to allocate (not reuseirq_queues), and so we can adjust affd.set[] correctly (depends onirq_queues) to avoid out of bounds access.Quoted 8 lines
After this patch:
The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 8,alloc_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7,alloc_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 6,alloc_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 6We currently have this one queued up:
can you check if it fixes the issue for you?
Quoted 7 lines
Quoted 1 line
Nevermind, took a closer look, and this looks like a different issue.
The solutions look different, but I think they're both targeting the same problem, which is the older code had been accounting for vectors and queues differenting in the first iteration than subsequent ones. I think Ming's patch will probably fix the issue raised here and worth a shot at testing it.
OK good, then I wasn't completely crazy after all :-)
Signature · 1 line
Jens Axboe
PATCH 1/1 nvme: fix out-of-bounds access during irq vectors allocation
Quoted 56 lines · Jens Axboe
Quoted 54 lines · Keith Busch
Quoted 52 lines · Jens Axboe
Quoted 44 lines · Jens Axboe
Quoted 34 lines · Huacai Chen
While reducing
irq_queuesin the do-while loop innvme_setup_irqs(), the reduction of irq_sets[] is behindirq_queues. Below is an example.On a 8 cpu platform, with default setting,
nvme_setup_irqs()begin withirq_queues= 8 (but when allocating irq vectors it will become 9 due to the admin queue), affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8. If MSI-X resources are not enough, then the do-while loop will reduce irq vectors:The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7However, this will cause an out of bounds access in
__pci_enable_msix()--> ... -->irq_create_affinity_masks()-->irq_build_affinity_masks().In the 2nd round of reduction, let's pay attention to the calling of
irq_build_affinity_masks(affd, curvec, this_vecs, curvec, node_to_cpumask, masks):The number of elements in masks is 8 (depends on nvecs which is equal to
irq_queues), curvec is 1 (depends on affd.pre_vectors), thenirq_build_affinity_masks()will access 8 elements in masks (depends onthis_vecswhich is equal to affd.sets[0]), and the last element is out of bounds.So the root cause is affd.sets[] + affd.pre_vectors should not be larger than vectors to be allocated. In this patch we introduce
alloc_queuesto indicate how many queues to allocate (not reuseirq_queues), and so we can adjust affd.set[] correctly (depends onirq_queues) to avoid out of bounds access.Quoted 8 lines
After this patch:
The 1st time call
pci_alloc_irq_vectors_affinity(),irq_queues= 8,alloc_queues= 9, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 8 The 2nd time callpci_alloc_irq_vectors_affinity(),irq_queues= 7,alloc_queues= 8, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 7 The 3rd time callpci_alloc_irq_vectors_affinity(),irq_queues= 6,alloc_queues= 7, affd.pre_vectors = 1, affd.nr_sets = 1, affd.sets[0] = 6Quoted 5 lines
We currently have this one queued up:
can you check if it fixes the issue for you?
Nevermind, took a closer look, and this looks like a different issue.
Quoted 7 lines
Quoted 5 lines
The solutions look different, but I think they're both targeting the same problem, which is the older code had been accounting for vectors and queues differenting in the first iteration than subsequent ones. I think Ming's patch will probably fix the issue raised here and worth a shot at testing it.
OK good, then I wasn't completely crazy after all :-)
Hi, all
I have tested, both patches have solve the same problem.
Huacai
Quoted 4 lines
--
Jens Axboe