Skip to contentSkip to conversation

lib/sbitmap: allocate sb->map via kvzalloc_node

PATCH Applied Ming Lei 3 participants 5 messages Mar 16, 2022 – Mar 22, 2022

Subsystems (2)
lore
Share & tools
lore

Ming Lei

author
#0 lore raw

PATCH 1/1 lib/sbitmap: allocate sb->map via kvzalloc_node

sbitmap has been used in scsi for replacing atomic operations on sdev->device_busy, so IOPS on some fast scsi storage can be improved.

However, sdev->device_busy can be changed in fast path, so we have to allocate the sb->map statically. sdev->device_busy has been capped to 1024, but some drivers may configure the default depth as < 8, then cause each sbitmap word to hold only one bit. Finally 1024 * 128( sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5 allocation, sometimes it may fail.

Avoid the issue by using kvzalloc_node() for allocating sb->map.

Cc Ewan D. Milne <[email protected]>
---
 include/linux/sbitmap.h | 2 +-
 lib/sbitmap.c           | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
include/linux/sbitmap.h +1 -1
index dffeb8281c2d..8f5a86e210b9 100644
@@ -174,7 +174,7 @@ static inline unsigned int __map_depth(const struct sbitmap *sb, int index)
 static inline void sbitmap_free(struct sbitmap *sb)
 {
 	free_percpu(sb->alloc_hint);
-	kfree(sb->map);
+	kvfree(sb->map);
 	sb->map = NULL;
 }
 
lib/sbitmap.c +1 -1
index 2eb3de18ded3..ae4fd4de9ebe 100644
@@ -110,7 +110,7 @@ int sbitmap_init_node(struct sbitmap *sb, unsigned int depth, int shift,
 		sb->alloc_hint = NULL;
 	}
 
-	sb->map = kcalloc_node(sb->map_nr, sizeof(*sb->map), flags, node);
+	sb->map = kvzalloc_node(sb->map_nr * sizeof(*sb->map), flags, node);
 	if (!sb->map) {
 		free_percpu(sb->alloc_hint);
 		return -ENOMEM;
Signature · 1 line
2.31.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 10 lines · Ming Lei

sbitmap has been used in scsi for replacing atomic operations on sdev->device_busy, so IOPS on some fast scsi storage can be improved.

However, sdev->device_busy can be changed in fast path, so we have to allocate the sb->map statically. sdev->device_busy has been capped to 1024, but some drivers may configure the default depth as < 8, then cause each sbitmap word to hold only one bit. Finally 1024 * 128( sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5 allocation, sometimes it may fail.

Quoted 6 lines

Avoid the issue by using kvzalloc_node() for allocating sb->map.

Cc Ewan D. Milne <[email protected]>

---

maybe considering the total size before actual allocation that will not buy us anything ?

Looks good.

-ck

author
#2 lore raw
In reply to Ming Lei

Hello,

Quoted 10 lines · Ming Lei

sbitmap has been used in scsi for replacing atomic operations on sdev->device_busy, so IOPS on some fast scsi storage can be improved.

However, sdev->device_busy can be changed in fast path, so we have to allocate the sb->map statically. sdev->device_busy has been capped to 1024, but some drivers may configure the default depth as < 8, then cause each sbitmap word to hold only one bit. Finally 1024 * 128( sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5 allocation, sometimes it may fail.

Quoted 4 lines

Avoid the issue by using kvzalloc_node() for allocating sb->map.

Cc Ewan D. Milne <[email protected]>

Ping...

thanks,
Ming

maintainer
#3 lore raw
Quoted 13 lines · Ming Lei

Hello,

Quoted 10 lines · Ming Lei

sbitmap has been used in scsi for replacing atomic operations on sdev->device_busy, so IOPS on some fast scsi storage can be improved.

However, sdev->device_busy can be changed in fast path, so we have to allocate the sb->map statically. sdev->device_busy has been capped to 1024, but some drivers may configure the default depth as < 8, then cause each sbitmap word to hold only one bit. Finally 1024 * 128( sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5 allocation, sometimes it may fail.

Quoted 6 lines
Quoted 4 lines

Avoid the issue by using kvzalloc_node() for allocating sb->map.

Cc Ewan D. Milne <[email protected]>

Ping...

Was waiting on a response to the question on why we don't make this dependent on size, but upon look at kvzalloc_node(), this is already what it does. So I guess this looks fine to me too.

Signature · 1 line
Jens Axboe
maintainer Applied
#4 lore raw
In reply to Ming Lei
Quoted 11 lines · Ming Lei

sbitmap has been used in scsi for replacing atomic operations on sdev->device_busy, so IOPS on some fast scsi storage can be improved.

However, sdev->device_busy can be changed in fast path, so we have to allocate the sb->map statically. sdev->device_busy has been capped to 1024, but some drivers may configure the default depth as < 8, then cause each sbitmap word to hold only one bit. Finally 1024 * 128( sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5 allocation, sometimes it may fail.

[...]

Applied, thanks!

[1/1] lib/sbitmap: allocate sb->map via kvzalloc_node

      commit: 863a66cdb4df25fd146d9851c3289072298566d5

Best regards,

Signature · 1 line
Jens Axboe