Skip to contentSkip to conversation

fs/pipe: use kvcalloc to allocate a pipe_buffer array

PATCH Applied Andrei Vagin 2 participants 6 messages Jan 4, 2022 – Jan 25, 2022 VFS

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fs/pipe: use kvcalloc to allocate a pipe_buffer array

Right now, kcalloc is used to allocate a pipe_buffer array. The size of the pipe_buffer struct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal to PAGE_ALLOC_COSTLY_ORDER (3). It means that the maximum pipe size is 3.2MB in this case.

In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.

kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by PAGE_ALLOC_COSTLY_ORDER.

The maximum pipe size for non-root users is limited by
the /proc/sys/fs/pipe-max-size sysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.

Cc Dmitry Safonov <[email protected]>
---
 fs/pipe.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
fs/pipe.c +4 -5
index 6d4342bad9f1..45565773ec33 100644
@@ -802,7 +802,7 @@ struct pipe_inode_info *alloc_pipe_info(void)
 	if (too_many_pipe_buffers_hard(user_bufs) && pipe_is_unprivileged_user())
 		goto out_revert_acct;
 
-	pipe->bufs = kcalloc(pipe_bufs, sizeof(struct pipe_buffer),
+	pipe->bufs = kvcalloc(pipe_bufs, sizeof(struct pipe_buffer),
 			     GFP_KERNEL_ACCOUNT);
 
 	if (pipe->bufs) {
@@ -845,7 +845,7 @@ void free_pipe_info(struct pipe_inode_info *pipe)
 	}
 	if (pipe->tmp_page)
 		__free_page(pipe->tmp_page);
-	kfree(pipe->bufs);
+	kvfree(pipe->bufs);
 	kfree(pipe);
 }
 
@@ -1260,8 +1260,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots)
 	if (nr_slots < n)
 		return -EBUSY;
 
-	bufs = kcalloc(nr_slots, sizeof(*bufs),
-		       GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
+	bufs = kvcalloc(nr_slots, sizeof(*bufs), GFP_KERNEL_ACCOUNT);
 	if (unlikely(!bufs))
 		return -ENOMEM;
 
@@ -1288,7 +1287,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots)
 	head = n;
 	tail = 0;
 
-	kfree(pipe->bufs);
+	kvfree(pipe->bufs);
 	pipe->bufs = bufs;
 	pipe->ring_size = nr_slots;
 	if (pipe->max_usage > nr_slots)
Signature · 1 line
2.33.1
#1 lore raw
Quoted 14 lines · Andrei Vagin

Right now, kcalloc is used to allocate a pipe_buffer array. The size of the pipe_buffer struct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal to PAGE_ALLOC_COSTLY_ORDER (3). It means that the maximum pipe size is 3.2MB in this case.

In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.

kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by PAGE_ALLOC_COSTLY_ORDER.

Quoted 6 lines

The maximum pipe size for non-root users is limited by
the /proc/sys/fs/pipe-max-size sysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.

Cc Dmitry Safonov <[email protected]>

Good idea!

I wonder if you need to apply this on the top:

fs/pipe.c +0 -0
index 45565773ec33..b4ccafffa350 100644
@@ -605,7 +605,7 @@ pipe_write(struct kiocb *iocb, struct iov_iter *from)
 static long pipe_ioctl(struct file *filp, unsigned int cmd, unsigned

long arg)

 {
        struct pipe_inode_info *pipe = filp->private_data;

- int count, head, tail, mask;
+ unsigned int count, head, tail, mask;

        switch (cmd) {
        case FIONREAD:

@@ -827,7 +827,7 @@ struct pipe_inode_info *alloc_pipe_info(void)

void free_pipe_info(struct pipe_inode_info *pipe) {
- int i;
+ unsigned int i;

 #ifdef CONFIG_WATCH_QUEUE
        if (pipe->watch_queue) {

--->8---

Otherwise this loop in free_pipe_info() may become lockup on some ugly platforms with INTMAX allocation reachable, I think. I may be wrong :-)

Thanks,
Dmitry

#2 lore raw
Quoted 52 lines · Dmitry Safonov
Quoted 14 lines · Andrei Vagin

Right now, kcalloc is used to allocate a pipe_buffer array. The size of the pipe_buffer struct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal to PAGE_ALLOC_COSTLY_ORDER (3). It means that the maximum pipe size is 3.2MB in this case.

In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.

kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by PAGE_ALLOC_COSTLY_ORDER.

Quoted 6 lines

The maximum pipe size for non-root users is limited by
the /proc/sys/fs/pipe-max-size sysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.

Cc Dmitry Safonov <[email protected]>

Good idea!

I wonder if you need to apply this on the top:

fs/pipe.c +1 -1
index 45565773ec33..b4ccafffa350 100644
@@ -605,7 +605,7 @@ pipe_write(struct kiocb *iocb, struct iov_iter *from)
 static long pipe_ioctl(struct file *filp, unsigned int cmd, unsigned
 long arg)
 {
        struct pipe_inode_info *pipe = filp->private_data;
-       int count, head, tail, mask;
+       unsigned int count, head, tail, mask;
 
        switch (cmd) {

case FIONREAD:

fs/pipe.c +1 -2
@@ -827,7 +827,7 @@ struct pipe_inode_info *alloc_pipe_info(void)
 
 void free_pipe_info(struct pipe_inode_info *pipe)
 {
-       int i;
+       unsigned int i;
 
 #ifdef CONFIG_WATCH_QUEUE
        if (pipe->watch_queue) {
--->8---
Quoted 2 lines

Otherwise this loop in free_pipe_info() may become lockup on some ugly platforms with INTMAX allocation reachable, I think. I may be wrong :-)

This change looks reasonable, it makes types of local variables consistent with proper fields of pipe_inode_info. But right now, the maximum pipe size is limited by (1<<31) (look at round_pipe_size) and so we don't have a real issue here.

Thanks,
Andrei

#3 lore raw
Quoted 1 line · Andrei Vagin

On Tue, Jan 4, 2022 at 10:54 PM Dmitry Safonov <[email protected]> wrote:

[..]

Quoted 7 lines
Quoted 2 lines

Otherwise this loop in free_pipe_info() may become lockup on some ugly platforms with INTMAX allocation reachable, I think. I may be wrong :-)

This change looks reasonable, it makes types of local variables consistent with proper fields of pipe_inode_info. But right now, the maximum pipe size is limited by (1<<31) (look at round_pipe_size) and so we don't have a real issue here.

Right you are, I haven't noticed it.

Hmm,
: if (size > (1U << 31))
: return 0;

Isn't size == (1U << 31) a negative integer on 32-bit?

[probably, the question is not very related to your patch, though]

Thanks,
Dmitry

Reviewed-by on v1 1/1
#4 lore raw
In reply to Andrei Vagin
Quoted 14 lines · Andrei Vagin

Right now, kcalloc is used to allocate a pipe_buffer array. The size of the pipe_buffer struct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal to PAGE_ALLOC_COSTLY_ORDER (3). It means that the maximum pipe size is 3.2MB in this case.

In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.

kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by PAGE_ALLOC_COSTLY_ORDER.

Quoted 6 lines

The maximum pipe size for non-root users is limited by
the /proc/sys/fs/pipe-max-size sysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.

Cc Dmitry Safonov <[email protected]>
Quoted 37 lines
---
 fs/pipe.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
fs/pipe.c +3 -4
index 6d4342bad9f1..45565773ec33 100644
@@ -802,7 +802,7 @@ struct pipe_inode_info *alloc_pipe_info(void)
 	if (too_many_pipe_buffers_hard(user_bufs) && pipe_is_unprivileged_user())
 		goto out_revert_acct;
 
-	pipe->bufs = kcalloc(pipe_bufs, sizeof(struct pipe_buffer),
+	pipe->bufs = kvcalloc(pipe_bufs, sizeof(struct pipe_buffer),
 			     GFP_KERNEL_ACCOUNT);
 
 	if (pipe->bufs) {
@@ -845,7 +845,7 @@ void free_pipe_info(struct pipe_inode_info *pipe)
 	}
 	if (pipe->tmp_page)
 		__free_page(pipe->tmp_page);
-	kfree(pipe->bufs);
+	kvfree(pipe->bufs);
 	kfree(pipe);
 }
 
@@ -1260,8 +1260,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots)
 	if (nr_slots < n)
 		return -EBUSY;
 
-	bufs = kcalloc(nr_slots, sizeof(*bufs),
-		       GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
+	bufs = kvcalloc(nr_slots, sizeof(*bufs), GFP_KERNEL_ACCOUNT);
 	if (unlikely(!bufs))
 		return -ENOMEM;
 

fs/pipe.c int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots) · line 1293

Quoted 8 lines
fs/pipe.c +1 -1
@@ -1288,7 +1287,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots)
 	head = n;
 	tail = 0;
 
-	kfree(pipe->bufs);
+	kvfree(pipe->bufs);
 	pipe->bufs = bufs;
 	pipe->ring_size = nr_slots;
 	if (pipe->max_usage > nr_slots)

Thanks,
Dmitry

#5 lore raw
In reply to Andrei Vagin
Quoted 15 lines · Andrei Vagin

Right now, kcalloc is used to allocate a pipe_buffer array. The size of the pipe_buffer struct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal to PAGE_ALLOC_COSTLY_ORDER (3). It means that the maximum pipe size is 3.2MB in this case.

In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.

kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by PAGE_ALLOC_COSTLY_ORDER.

Quoted 6 lines

The maximum pipe size for non-root users is limited by
the /proc/sys/fs/pipe-max-size sysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.

Cc Dmitry Safonov <[email protected]>

Alexander and Andrew,

Do you have any objections? Could you merge this patch?

Dmitry's comments have been addressed in the separate patch:
https://lkml.org/lkml/2022/1/6/463

BTW: I found that fuse already uses kvmalloc to allocate pipe buffers (e. g. fuse_dev_splice_read).

Thanks,
Andrei