scsi: sd_zbc: emulate ZONE_APPEND commands
Hello Johannes Thumshirn,
The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:
drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*
722 * There is nothing to do for regular disks, including host-aware disks
723 * that have partitions.
724 */
725 if (!blk_queue_is_zoned(q))
726 return 0;
727
728 /*
729 * Make sure revalidate zones are serialized to ensure exclusive
730 * updates of the scsi disk data.
731 */
732 mutex_lock(&sdkp->rev_mutex);
733
734 if (sdkp->zone_blocks == zone_blocks &&
735 sdkp->nr_zones == nr_zones &&
736 disk->queue->nr_zones == nr_zones)
737 goto unlock;
738
739 sdkp->zone_blocks = zone_blocks;
740 sdkp->nr_zones = nr_zones;
741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.
742 if (!sdkp->rev_wp_offset) {
743 ret = -ENOMEM;
744 goto unlock;
745 }
746
747 ret = blk_revalidate_disk_zones(disk, sd_zbc_revalidate_zones_cb);
748 regards,
dan carpenter
Quoted 31 lines · Dan Carpenter
Hello Johannes Thumshirn,
The patch
5795eb443060: "scsi:sd_zbc: emulateZONE_APPENDcommands" from May 12, 2020, leads to the following static checker warning:drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*722 * There is nothing to do for regular disks, including host-aware disks 723 * that have partitions. 724 */ 725 if (!blk_queue_is_zoned(q)) 726 return 0; 727 728 /* 729 * Make sure revalidate zones are serialized to ensure exclusive 730 * updates of the scsi disk data. 731 */ 732 mutex_lock(&sdkp->rev_mutex); 733 734 if (sdkp->zone_blocks == zone_blocks && 735 sdkp->nr_zones == nr_zones && 736 disk->queue->nr_zones == nr_zones) 737 goto unlock; 738 739 sdkp->zone_blocks = zone_blocks; 740 sdkp->nr_zones = nr_zones; 741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines
We're passing
GFP_NOIOhere so it just defaults tokcalloc()and will notvmalloc()the memory.
Indeed... And the allocation can get a little too big for kmalloc().
Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.
Quoted 12 lines
742 if (!sdkp->rev_wp_offset) { 743 ret = -ENOMEM; 744 goto unlock; 745 } 746 747 ret = blk_revalidate_disk_zones(disk, sd_zbc_revalidate_zones_cb); 748regards,
dan carpenter
Signature · 2 lines
Damien Le Moal Western Digital Research
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter
Hello Johannes Thumshirn,
The patch
5795eb443060: "scsi:sd_zbc: emulateZONE_APPENDcommands" from May 12, 2020, leads to the following static checker warning:drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*722 * There is nothing to do for regular disks, including host-aware disks 723 * that have partitions. 724 */ 725 if (!blk_queue_is_zoned(q)) 726 return 0; 727 728 /* 729 * Make sure revalidate zones are serialized to ensure exclusive 730 * updates of the scsi disk data. 731 */ 732 mutex_lock(&sdkp->rev_mutex); 733 734 if (sdkp->zone_blocks == zone_blocks && 735 sdkp->nr_zones == nr_zones && 736 disk->queue->nr_zones == nr_zones) 737 goto unlock; 738 739 sdkp->zone_blocks = zone_blocks; 740 sdkp->nr_zones = nr_zones; 741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Quoted 2 lines
We're passing
GFP_NOIOhere so it just defaults tokcalloc()and will notvmalloc()the memory.
Quoted 8 lines
Indeed... And the allocation can get a little too big for
kmalloc().Johannes, I think we need to move that allocation before the
rev_mutexlocking, using a local var for the allocated address, and then usingGFP_KERNELshould be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.
Would a kmem_cache for these revalidations help us in any way?
Quoted 46 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter
Hello Johannes Thumshirn,
The patch
5795eb443060: "scsi:sd_zbc: emulateZONE_APPENDcommands" from May 12, 2020, leads to the following static checker warning:drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*722 * There is nothing to do for regular disks, including host-aware disks 723 * that have partitions. 724 */ 725 if (!blk_queue_is_zoned(q)) 726 return 0; 727 728 /* 729 * Make sure revalidate zones are serialized to ensure exclusive 730 * updates of the scsi disk data. 731 */ 732 mutex_lock(&sdkp->rev_mutex); 733 734 if (sdkp->zone_blocks == zone_blocks && 735 sdkp->nr_zones == nr_zones && 736 disk->queue->nr_zones == nr_zones) 737 goto unlock; 738 739 sdkp->zone_blocks = zone_blocks; 740 sdkp->nr_zones = nr_zones; 741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Quoted 2 lines
We're passing
GFP_NOIOhere so it just defaults tokcalloc()and will notvmalloc()the memory.Quoted 8 lines
Indeed... And the allocation can get a little too big for
kmalloc().Johannes, I think we need to move that allocation before the
rev_mutexlocking, using a local var for the allocated address, and then usingGFP_KERNELshould be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.
Quoted 3 lines
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.
Yes, but blk_revalidate_disk_zones() only allocates the zone bitmaps and these are much smaller. So kmalloc is fine and GFP_NOIO is natural. For the wp array, I think we really need kvmalloc() due to the potential very large size (and growing with new drive models) and GFP_NOIO does not work for that. Not sure if memalloc_noio_{save,restore}() can change that in vmalloc context (I do not think so).
Quoted 1 line
Would a
kmem_cachefor these revalidations help us in any way?
I am not familiar with that... Would need to dig into it.
For this to be safe, we only need to guarantee forward progress, and in this case this means not causing problems if a GFP_KERNEL allocation causes us to reenter the scsi driver for I/Os. Since (I think) revalidation never happens in FS or I/O context, GFP_KERNEL allocation should be safe if done outside of the rev_mutex lock. Not 100% sure here, just a hunch... We may need to check the block layer level to check if there are any locks being held when revalidation triggers.
Signature · 2 lines
Damien Le Moal Western Digital Research
Quoted 46 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter
Hello Johannes Thumshirn,
The patch
5795eb443060: "scsi:sd_zbc: emulateZONE_APPENDcommands" from May 12, 2020, leads to the following static checker warning:drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*722 * There is nothing to do for regular disks, including host-aware disks 723 * that have partitions. 724 */ 725 if (!blk_queue_is_zoned(q)) 726 return 0; 727 728 /* 729 * Make sure revalidate zones are serialized to ensure exclusive 730 * updates of the scsi disk data. 731 */ 732 mutex_lock(&sdkp->rev_mutex); 733 734 if (sdkp->zone_blocks == zone_blocks && 735 sdkp->nr_zones == nr_zones && 736 disk->queue->nr_zones == nr_zones) 737 goto unlock; 738 739 sdkp->zone_blocks = zone_blocks; 740 sdkp->nr_zones = nr_zones; 741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Quoted 2 lines
We're passing
GFP_NOIOhere so it just defaults tokcalloc()and will notvmalloc()the memory.Quoted 8 lines
Indeed... And the allocation can get a little too big for
kmalloc().Johannes, I think we need to move that allocation before the
rev_mutexlocking, using a local var for the allocated address, and then usingGFP_KERNELshould be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.
Quoted 3 lines
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.
You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
Quoted 2 lines
Would a
kmem_cachefor these revalidations help us in any way?
I am not sure what you mean here.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 7 lines · Michal Hocko
Quoted 3 lines
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
That means all callers of blk_revalidate_disk_zones() should do memalloc_noio_{save,restore}? If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?
Quoted 5 lines
Quoted 2 lines
Would a
kmem_cachefor these revalidations help us in any way?I am not sure what you mean here.
Using a kmem_cache for the allocations passed into blk_revalidate_disk_zones(). I've looked into kmem_cache_alloc() and I couldn't find anything that speaks against it, but I'm not too familiar with the code.
Quoted 11 lines · Johannes Thumshirn
Quoted 7 lines · Michal Hocko
Quoted 3 lines
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
That means all callers of
blk_revalidate_disk_zones()should do memalloc_noio_{save,restore}?
I am not really familiar with the IO area to answer this. The base idea is to start the NOIO scope at the boundary which defines "unsafe to re-enter or cannot deal with a new IO" from the reclaim path.
Quoted 2 lines
If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?
Could you elaborate?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Would a
kmem_cachefor these revalidations help us in any way?I am not sure what you mean here.
Using a
kmem_cachefor the allocations passed intoblk_revalidate_disk_zones(). I've looked intokmem_cache_alloc()and I couldn't find anything that speaks against it, but I'm not too familiar with the code.
kmem_cache_alloc is only an extension to allow to allocate from a specific cache. I do not really see how it is going to help with larger allocation and my current understanding is that kvmalloc is used because the requested allocation size can be large.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 13 lines · Michal Hocko
Quoted 11 lines · Johannes Thumshirn
Quoted 7 lines · Michal Hocko
Quoted 3 lines
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
That means all callers of
blk_revalidate_disk_zones()should do memalloc_noio_{save,restore}?
Quoted 8 lines
I am not really familiar with the IO area to answer this. The base idea is to start the NOIO scope at the boundary which defines "unsafe to re-enter or cannot deal with a new IO" from the reclaim path.
Quoted 2 lines
If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?
Could you elaborate?
I though of lifting the noio scope into the callers of blk_revalidate_disk_zones() and then "check" in blk_revalidate_disk_zones() this has been done. But it looks like memalloc_noio_save() can handle nesting, so this is actually unneeded.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
Would a
kmem_cachefor these revalidations help us in any way?I am not sure what you mean here.
Quoted 9 lines
Quoted 3 lines
Using a
kmem_cachefor the allocations passed intoblk_revalidate_disk_zones(). I've looked intokmem_cache_alloc()and I couldn't find anything that speaks against it, but I'm not too familiar with the code.
kmem_cache_allocis only an extension to allow to allocate from a specific cache. I do not really see how it is going to help with larger allocation and my current understanding is that kvmalloc is used because the requested allocation size can be large.
Ah ok so we can't set aside a big enough pool to do allocations from there, this was a misunderstanding from my side.
Quoted 21 lines · Johannes Thumshirn
Quoted 20 lines · Michal Hocko
Quoted 11 lines · Johannes Thumshirn
Quoted 7 lines · Michal Hocko
Quoted 3 lines
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
That means all callers of
blk_revalidate_disk_zones()should do memalloc_noio_{save,restore}?I am not really familiar with the IO area to answer this. The base idea is to start the NOIO scope at the boundary which defines "unsafe to re-enter or cannot deal with a new IO" from the reclaim path.
Quoted 2 lines
If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?
Quoted 6 lines
Quoted 1 line
Could you elaborate?
I though of lifting the noio scope into the callers of
blk_revalidate_disk_zones()and then "check" inblk_revalidate_disk_zones()this has been done. But it looks likememalloc_noio_save()can handle nesting, so this is actually unneeded.
Yes, it can handle nesting. As to where to start the scope, this should be at the boundary of "do not reenter IO path from the reclaim". This can be a lock which is held for the allocation which will be needed for an IO induced by the reclaim or any other resouce that will block from the IO path. Is this more clear now?
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 47 lines · Michal Hocko
Quoted 46 lines · Johannes Thumshirn
Quoted 43 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter
Hello Johannes Thumshirn,
The patch
5795eb443060: "scsi:sd_zbc: emulateZONE_APPENDcommands" from May 12, 2020, leads to the following static checker warning:drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*722 * There is nothing to do for regular disks, including host-aware disks 723 * that have partitions. 724 */ 725 if (!blk_queue_is_zoned(q)) 726 return 0; 727 728 /* 729 * Make sure revalidate zones are serialized to ensure exclusive 730 * updates of the scsi disk data. 731 */ 732 mutex_lock(&sdkp->rev_mutex); 733 734 if (sdkp->zone_blocks == zone_blocks && 735 sdkp->nr_zones == nr_zones && 736 disk->queue->nr_zones == nr_zones) 737 goto unlock; 738 739 sdkp->zone_blocks = zone_blocks; 740 sdkp->nr_zones = nr_zones; 741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Quoted 2 lines
We're passing
GFP_NOIOhere so it just defaults tokcalloc()and will notvmalloc()the memory.Indeed... And the allocation can get a little too big for
kmalloc().Johannes, I think we need to move that allocation before the
rev_mutexlocking, using a local var for the allocated address, and then usingGFP_KERNELshould be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.
Quoted 7 lines
Quoted 3 lines
No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
But it does not look like __vmalloc_node() (fallback in kvmalloc_node() if kmalloc() fails) cares about the context allocation flags... I can't see if/where the context allocation flags are taken into account. It looks like only the gfp_mask argument is used. Am I missing something ?
Quoted 5 lines
Quoted 2 lines
Would a
kmem_cachefor these revalidations help us in any way?I am not sure what you mean here.
Signature · 2 lines
Damien Le Moal Western Digital Research
Quoted 52 lines · Damien Le Moal
Quoted 51 lines · Michal Hocko
Quoted 49 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter
Hello Johannes Thumshirn,
The patch
5795eb443060: "scsi:sd_zbc: emulateZONE_APPENDcommands" from May 12, 2020, leads to the following static checker warning:drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*722 * There is nothing to do for regular disks, including host-aware disks 723 * that have partitions. 724 */ 725 if (!blk_queue_is_zoned(q)) 726 return 0; 727 728 /* 729 * Make sure revalidate zones are serialized to ensure exclusive 730 * updates of the scsi disk data. 731 */ 732 mutex_lock(&sdkp->rev_mutex); 733 734 if (sdkp->zone_blocks == zone_blocks && 735 sdkp->nr_zones == nr_zones && 736 disk->queue->nr_zones == nr_zones) 737 goto unlock; 738 739 sdkp->zone_blocks = zone_blocks; 740 sdkp->nr_zones = nr_zones; 741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Quoted 2 lines
We're passing
GFP_NOIOhere so it just defaults tokcalloc()and will notvmalloc()the memory.Quoted 8 lines
Indeed... And the allocation can get a little too big for
kmalloc().Johannes, I think we need to move that allocation before the
rev_mutexlocking, using a local var for the allocated address, and then usingGFP_KERNELshould be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.
Quoted 8 lines
Quoted 3 lines
You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
But it does not look like
__vmalloc_node()(fallback inkvmalloc_node()ifkmalloc()fails) cares about the context allocation flags... I can't see if/where the context allocation flags are taken into account. It looks like only thegfp_maskargument is used. Am I missing something ?
It does here:
void *kvmalloc_node(size_t size, gfp_t flags, int node)
{
gfp_t kmalloc_flags = flags;
void *ret;
/*
* vmalloc uses GFP_KERNEL for some internal allocations (e.g page tables)
* so the given set of flags has to be compatible.
*/
if ((flags & GFP_KERNEL) != GFP_KERNEL)
return kmalloc_node(size, flags, node);
/* [...] */
return __vmalloc_node(size, 1, flags, node,
__builtin_return_address(0));
} EXPORT_SYMBOL(kvmalloc_node);
We're not reaching __vmalloc_node() if GFP_KERNEL isn't set. Also, from the, function's documentation:
* @flags: gfp mask for the allocation - must be compatible (superset) with GFP_KERNEL.
So yeah, we should've read that before calling kvcalloc(..., GFP_NOIO).
Quoted 3 lines · Johannes Thumshirn
* @flags: gfp mask for the allocation - must be compatible (superset) with
GFP_KERNEL.So yeah, we should've read that before calling
kvcalloc(..., GFP_NOIO).
This is a Smatch check now and checks are better than comments. :)
regards,
dan carpenter
Quoted 6 lines · Dan Carpenter
Quoted 3 lines · Johannes Thumshirn
* @flags: gfp mask for the allocation - must be compatible (superset) with
GFP_KERNEL.So yeah, we should've read that before calling
kvcalloc(..., GFP_NOIO).This is a Smatch check now and checks are better than comments. :)
Definitively
Quoted 52 lines · Damien Le Moal
Quoted 51 lines · Michal Hocko
Quoted 49 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter
Hello Johannes Thumshirn,
The patch
5795eb443060: "scsi:sd_zbc: emulateZONE_APPENDcommands" from May 12, 2020, leads to the following static checker warning:drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones() error: kvmalloc() only makes sense with GFP_KERNEL
drivers/scsi/sd_zbc.c
721 /*722 * There is nothing to do for regular disks, including host-aware disks 723 * that have partitions. 724 */ 725 if (!blk_queue_is_zoned(q)) 726 return 0; 727 728 /* 729 * Make sure revalidate zones are serialized to ensure exclusive 730 * updates of the scsi disk data. 731 */ 732 mutex_lock(&sdkp->rev_mutex); 733 734 if (sdkp->zone_blocks == zone_blocks && 735 sdkp->nr_zones == nr_zones && 736 disk->queue->nr_zones == nr_zones) 737 goto unlock; 738 739 sdkp->zone_blocks = zone_blocks; 740 sdkp->nr_zones = nr_zones; 741 sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Quoted 2 lines
We're passing
GFP_NOIOhere so it just defaults tokcalloc()and will notvmalloc()the memory.Quoted 8 lines
Indeed... And the allocation can get a little too big for
kmalloc().Johannes, I think we need to move that allocation before the
rev_mutexlocking, using a local var for the allocated address, and then usingGFP_KERNELshould be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use
GFP_KERNELas we're about to do I/O.blk_revalidate_disk_zones()called a few line below also does the memalloc_noio_{save,restore}() dance.
Quoted 8 lines
Quoted 3 lines
You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.
But it does not look like
__vmalloc_node()(fallback inkvmalloc_node()ifkmalloc()fails) cares about the context allocation flags... I can't see if/where the context allocation flags are taken into account. It looks like only thegfp_maskargument is used. Am I missing something ?
current_gfp_context in the page allocator. vmalloc doesn't do any reclaim on its own. It relies on the page allocator for that.
Signature · 2 lines
Michal Hocko SUSE Labs