Skip to contentSkip to conversation

scsi: sd_zbc: emulate ZONE_APPEND commands

bug report Dan Carpenter 4 participants 14 messages Feb 16, 2021 – Feb 17, 2021 SCSI

lore
Share & tools
lore
#0 lore raw

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

   742          if (!sdkp->rev_wp_offset) {
   743                  ret = -ENOMEM;
   744                  goto unlock;
   745          }
   746  
   747          ret = blk_revalidate_disk_zones(disk, sd_zbc_revalidate_zones_cb);
   748  

regards,
dan carpenter

#1 lore raw
Quoted 31 lines · Dan Carpenter

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

Indeed... And the allocation can get a little too big for kmalloc().

Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.

Quoted 12 lines
   742          if (!sdkp->rev_wp_offset) {
   743                  ret = -ENOMEM;
   744                  goto unlock;
   745          }
   746  
   747          ret = blk_revalidate_disk_zones(disk, sd_zbc_revalidate_zones_cb);
   748  

regards,
dan carpenter

Signature · 2 lines
Damien Le Moal
Western Digital Research
#2 lore raw
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

Quoted 8 lines

Indeed... And the allocation can get a little too big for kmalloc().

Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

Would a kmem_cache for these revalidations help us in any way?

#3 lore raw
Quoted 46 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

Quoted 8 lines

Indeed... And the allocation can get a little too big for kmalloc().

Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.

Quoted 3 lines

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

Yes, but blk_revalidate_disk_zones() only allocates the zone bitmaps and these are much smaller. So kmalloc is fine and GFP_NOIO is natural. For the wp array, I think we really need kvmalloc() due to the potential very large size (and growing with new drive models) and GFP_NOIO does not work for that. Not sure if memalloc_noio_{save,restore}() can change that in vmalloc context (I do not think so).

Quoted 1 line

Would a kmem_cache for these revalidations help us in any way?

I am not familiar with that... Would need to dig into it.

For this to be safe, we only need to guarantee forward progress, and in this case this means not causing problems if a GFP_KERNEL allocation causes us to reenter the scsi driver for I/Os. Since (I think) revalidation never happens in FS or I/O context, GFP_KERNEL allocation should be safe if done outside of the rev_mutex lock. Not 100% sure here, just a hunch... We may need to check the block layer level to check if there are any locks being held when revalidation triggers.

Signature · 2 lines
Damien Le Moal
Western Digital Research
#4 lore raw
In reply to Johannes Thumshirn
Quoted 46 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

Quoted 8 lines

Indeed... And the allocation can get a little too big for kmalloc().

Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.

Quoted 3 lines

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

Quoted 2 lines

Would a kmem_cache for these revalidations help us in any way?

I am not sure what you mean here.

Signature · 2 lines
Michal Hocko
SUSE Labs
#5 lore raw
Quoted 7 lines · Michal Hocko
Quoted 3 lines

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

That means all callers of blk_revalidate_disk_zones() should do memalloc_noio_{save,restore}? If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?

Quoted 5 lines
Quoted 2 lines

Would a kmem_cache for these revalidations help us in any way?

I am not sure what you mean here.

Using a kmem_cache for the allocations passed into blk_revalidate_disk_zones(). I've looked into kmem_cache_alloc() and I couldn't find anything that speaks against it, but I'm not too familiar with the code.

#6 lore raw
Quoted 11 lines · Johannes Thumshirn
Quoted 7 lines · Michal Hocko
Quoted 3 lines

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

That means all callers of blk_revalidate_disk_zones() should do memalloc_noio_{save,restore}?

I am not really familiar with the IO area to answer this. The base idea is to start the NOIO scope at the boundary which defines "unsafe to re-enter or cannot deal with a new IO" from the reclaim path.

Quoted 2 lines

If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?

Could you elaborate?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Would a kmem_cache for these revalidations help us in any way?

I am not sure what you mean here.

Using a kmem_cache for the allocations passed into blk_revalidate_disk_zones(). I've looked into kmem_cache_alloc() and I couldn't find anything that speaks against it, but I'm not too familiar with the code.

kmem_cache_alloc is only an extension to allow to allocate from a specific cache. I do not really see how it is going to help with larger allocation and my current understanding is that kvmalloc is used because the requested allocation size can be large.

Signature · 2 lines
Michal Hocko
SUSE Labs
#7 lore raw
Quoted 13 lines · Michal Hocko
Quoted 11 lines · Johannes Thumshirn
Quoted 7 lines · Michal Hocko
Quoted 3 lines

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

That means all callers of blk_revalidate_disk_zones() should do memalloc_noio_{save,restore}?

Quoted 8 lines

I am not really familiar with the IO area to answer this. The base idea is to start the NOIO scope at the boundary which defines "unsafe to re-enter or cannot deal with a new IO" from the reclaim path.

Quoted 2 lines

If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?

Could you elaborate?

I though of lifting the noio scope into the callers of blk_revalidate_disk_zones() and then "check" in blk_revalidate_disk_zones() this has been done. But it looks like memalloc_noio_save() can handle nesting, so this is actually unneeded.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Would a kmem_cache for these revalidations help us in any way?

I am not sure what you mean here.

Quoted 9 lines
Quoted 3 lines

Using a kmem_cache for the allocations passed into blk_revalidate_disk_zones(). I've looked into kmem_cache_alloc() and I couldn't find anything that speaks against it, but I'm not too familiar with the code.

kmem_cache_alloc is only an extension to allow to allocate from a specific cache. I do not really see how it is going to help with larger allocation and my current understanding is that kvmalloc is used because the requested allocation size can be large.

Ah ok so we can't set aside a big enough pool to do allocations from there, this was a misunderstanding from my side.

#8 lore raw
Quoted 21 lines · Johannes Thumshirn
Quoted 20 lines · Michal Hocko
Quoted 11 lines · Johannes Thumshirn
Quoted 7 lines · Michal Hocko
Quoted 3 lines

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

That means all callers of blk_revalidate_disk_zones() should do memalloc_noio_{save,restore}?

I am not really familiar with the IO area to answer this. The base idea is to start the NOIO scope at the boundary which defines "unsafe to re-enter or cannot deal with a new IO" from the reclaim path.

Quoted 2 lines

If yes, can we somehow runtime assert that this is done, so we don't end up with bad surprises?

Quoted 6 lines
Quoted 1 line

Could you elaborate?

I though of lifting the noio scope into the callers of blk_revalidate_disk_zones() and then "check" in blk_revalidate_disk_zones() this has been done. But it looks like memalloc_noio_save() can handle nesting, so this is actually unneeded.

Yes, it can handle nesting. As to where to start the scope, this should be at the boundary of "do not reenter IO path from the reclaim". This can be a lock which is held for the allocation which will be needed for an IO induced by the reclaim or any other resouce that will block from the IO path. Is this more clear now?

Signature · 2 lines
Michal Hocko
SUSE Labs
#9 lore raw
In reply to Michal Hocko
Quoted 47 lines · Michal Hocko
Quoted 46 lines · Johannes Thumshirn
Quoted 43 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

Indeed... And the allocation can get a little too big for kmalloc().

Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.

Quoted 7 lines
Quoted 3 lines

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

But it does not look like __vmalloc_node() (fallback in kvmalloc_node() if kmalloc() fails) cares about the context allocation flags... I can't see if/where the context allocation flags are taken into account. It looks like only the gfp_mask argument is used. Am I missing something ?

Quoted 5 lines
Quoted 2 lines

Would a kmem_cache for these revalidations help us in any way?

I am not sure what you mean here.

Signature · 2 lines
Damien Le Moal
Western Digital Research
#10 lore raw
Quoted 52 lines · Damien Le Moal
Quoted 51 lines · Michal Hocko
Quoted 49 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

Quoted 8 lines

Indeed... And the allocation can get a little too big for kmalloc().

Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

Quoted 8 lines
Quoted 3 lines

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

But it does not look like __vmalloc_node() (fallback in kvmalloc_node() if kmalloc() fails) cares about the context allocation flags... I can't see if/where the context allocation flags are taken into account. It looks like only the gfp_mask argument is used. Am I missing something ?

It does here:

void *kvmalloc_node(size_t size, gfp_t flags, int node)

{                                                                                                                                                                                    
        gfp_t kmalloc_flags = flags;                                                                                                                                                 
        void *ret;                                                                                                                                                                   
                                                                                                                                                                                     
        /*                                                                                                                                                                           
         * vmalloc uses GFP_KERNEL for some internal allocations (e.g page tables)                                                                                                   
         * so the given set of flags has to be compatible.                                                                                                                           
         */                                                                                                                                                                          
        if ((flags & GFP_KERNEL) != GFP_KERNEL)                                                                                                                                      
                return kmalloc_node(size, flags, node);

	/* [...] */

	return __vmalloc_node(size, 1, flags, node,                                                                                                                                  
                        __builtin_return_address(0));                                                                                                                                
}                                                                                                                                                                                    

EXPORT_SYMBOL(kvmalloc_node);

We're not reaching __vmalloc_node() if GFP_KERNEL isn't set. Also, from the, function's documentation:

* @flags: gfp mask for the allocation - must be compatible (superset) with GFP_KERNEL.

So yeah, we should've read that before calling kvcalloc(..., GFP_NOIO).

#11 lore raw
Quoted 3 lines · Johannes Thumshirn

* @flags: gfp mask for the allocation - must be compatible (superset) with GFP_KERNEL.

So yeah, we should've read that before calling kvcalloc(..., GFP_NOIO).

This is a Smatch check now and checks are better than comments. :)

regards,
dan carpenter

#12 lore raw
Quoted 6 lines · Dan Carpenter
Quoted 3 lines · Johannes Thumshirn

* @flags: gfp mask for the allocation - must be compatible (superset) with GFP_KERNEL.

So yeah, we should've read that before calling kvcalloc(..., GFP_NOIO).

This is a Smatch check now and checks are better than comments. :)

Definitively

#13 lore raw
In reply to Damien Le Moal
Quoted 52 lines · Damien Le Moal
Quoted 51 lines · Michal Hocko
Quoted 49 lines · Johannes Thumshirn
Quoted 35 lines · Damien Le Moal
Quoted 31 lines · Dan Carpenter

Hello Johannes Thumshirn,

The patch 5795eb443060: "scsi: sd_zbc: emulate ZONE_APPEND commands" from May 12, 2020, leads to the following static checker warning:

	drivers/scsi/sd_zbc.c:741 sd_zbc_revalidate_zones()
	error: kvmalloc() only makes sense with GFP_KERNEL

drivers/scsi/sd_zbc.c
721 /*

   722           * There is nothing to do for regular disks, including host-aware disks
   723           * that have partitions.
   724           */
   725          if (!blk_queue_is_zoned(q))
   726                  return 0;
   727  
   728          /*
   729           * Make sure revalidate zones are serialized to ensure exclusive
   730           * updates of the scsi disk data.
   731           */
   732          mutex_lock(&sdkp->rev_mutex);
   733  
   734          if (sdkp->zone_blocks == zone_blocks &&
   735              sdkp->nr_zones == nr_zones &&
   736              disk->queue->nr_zones == nr_zones)
   737                  goto unlock;
   738  
   739          sdkp->zone_blocks = zone_blocks;
   740          sdkp->nr_zones = nr_zones;
   741          sdkp->rev_wp_offset = kvcalloc(nr_zones, sizeof(u32), GFP_NOIO);
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

We're passing GFP_NOIO here so it just defaults to kcalloc() and will not vmalloc() the memory.

Quoted 8 lines

Indeed... And the allocation can get a little too big for kmalloc().

Johannes, I think we need to move that allocation before the rev_mutex locking, using a local var for the allocated address, and then using GFP_KERNEL should be safe... But not entirely sure. Using kmalloc would be simpler but on large SMR drives, that allocation will soon need to be 400K or so (i.e. 100,000 zones or even more), too large for kmalloc to succeed reliably.

No I don't think so. A mutex isn't a spinlock so we can sleep on the allocation. We can't use GFP_KERNEL as we're about to do I/O. blk_revalidate_disk_zones() called a few line below also does the memalloc_noio_{save,restore}() dance.

Quoted 8 lines
Quoted 3 lines

You should be extending noio scope then if this allocation falls into the same category. Ideally the scope should start at the recursion place and end where the scope really ened.

But it does not look like __vmalloc_node() (fallback in kvmalloc_node() if kmalloc() fails) cares about the context allocation flags... I can't see if/where the context allocation flags are taken into account. It looks like only the gfp_mask argument is used. Am I missing something ?

current_gfp_context in the page allocator. vmalloc doesn't do any reclaim on its own. It relies on the page allocator for that.

Signature · 2 lines
Michal Hocko
SUSE Labs