fs/pipe: use kvcalloc to allocate a pipe_buffer array
PATCH 1/1 fs/pipe: use kvcalloc to allocate a pipe_buffer array
Right now, kcalloc is used to allocate a pipe_buffer array. The size of the pipe_buffer struct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal to PAGE_ALLOC_COSTLY_ORDER (3). It means that the maximum pipe size is 3.2MB in this case.
In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.
kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by PAGE_ALLOC_COSTLY_ORDER.
The maximum pipe size for non-root users is limited by
the /proc/sys/fs/pipe-max-size sysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.
--- fs/pipe.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
fs/pipe.c +4 -5
@@ -802,7 +802,7 @@ struct pipe_inode_info *alloc_pipe_info(void)if (too_many_pipe_buffers_hard (user_bufs) &&pipe_is_unprivileged_user ()) goto out_revert_acct; - pipe->bufs =kcalloc (pipe_bufs,sizeof (struct pipe_buffer ), + pipe->bufs =kvcalloc (pipe_bufs,sizeof (struct pipe_buffer ),GFP_KERNEL_ACCOUNT );if (pipe->bufs) {
@@ -845,7 +845,7 @@ void free_pipe_info(struct pipe_inode_info *pipe) }if (pipe->tmp_page)__free_page (pipe->tmp_page); -kfree (pipe->bufs); +kvfree (pipe->bufs);kfree (pipe); }
@@ -1260,8 +1260,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots)if (nr_slots < n)return -EBUSY ; - bufs =kcalloc (nr_slots,sizeof (*bufs), -GFP_KERNEL_ACCOUNT | __GFP_NOWARN); + bufs =kvcalloc (nr_slots,sizeof (*bufs),GFP_KERNEL_ACCOUNT );if (unlikely (!bufs))return -ENOMEM ;
@@ -1288,7 +1287,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots) head = n; tail =0 ; -kfree (pipe->bufs); +kvfree (pipe->bufs); pipe->bufs = bufs; pipe->ring_size = nr_slots;if (pipe->max_usage > nr_slots)
Signature · 1 line
2.33.1
Quoted 14 lines · Andrei Vagin
Right now, kcalloc is used to allocate a
pipe_bufferarray. The size of thepipe_bufferstruct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal toPAGE_ALLOC_COSTLY_ORDER(3). It means that the maximum pipe size is 3.2MB in this case.In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.
kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by
PAGE_ALLOC_COSTLY_ORDER.
Quoted 6 lines
The maximum pipe size for non-root users is limited by
the/proc/sys/fs/pipe-max-sizesysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.Cc Dmitry Safonov <[email protected]>Signed-off-by Andrei Vagin <[email protected]>
Good idea!
I wonder if you need to apply this on the top:
fs/pipe.c +0 -0
@@ -605,7 +605,7 @@ pipe_write(struct kiocb *iocb, struct iov_iter *from) static long pipe_ioctl(struct file *filp, unsigned int cmd, unsigned
long arg)
{
struct pipe_inode_info *pipe = filp->private_data;- int count, head, tail, mask;
+ unsigned int count, head, tail, mask;
switch (cmd) {
case FIONREAD:@@ -827,7 +827,7 @@ struct pipe_inode_info *alloc_pipe_info(void)
void free_pipe_info(struct pipe_inode_info *pipe) {
- int i;
+ unsigned int i;
#ifdef CONFIG_WATCH_QUEUE
if (pipe->watch_queue) {--->8---
Otherwise this loop in free_pipe_info() may become lockup on some ugly platforms with INTMAX allocation reachable, I think. I may be wrong :-)
Thanks,
Dmitry
Quoted 52 lines · Dmitry Safonov
Quoted 14 lines · Andrei Vagin
Right now, kcalloc is used to allocate a
pipe_bufferarray. The size of thepipe_bufferstruct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal toPAGE_ALLOC_COSTLY_ORDER(3). It means that the maximum pipe size is 3.2MB in this case.In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.
kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by
PAGE_ALLOC_COSTLY_ORDER.Quoted 6 lines
The maximum pipe size for non-root users is limited by
the/proc/sys/fs/pipe-max-sizesysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.Cc Dmitry Safonov <[email protected]>Signed-off-by Andrei Vagin <[email protected]>Good idea!
I wonder if you need to apply this on the top:
fs/pipe.c +1
-1@@ -605,7 +605,7 @@ pipe_write(struct kiocb *iocb, struct iov_iter *from)static long pipe_ioctl (struct file *filp,unsigned int cmd,unsigned long arg) {struct pipe_inode_info *pipe = filp->private_data; -int count, head, tail, mask; +unsigned int count, head, tail, mask;switch (cmd) {case FIONREAD:
fs/pipe.c +1
-2@@ -827,7 +827,7 @@ struct pipe_inode_info *alloc_pipe_info(void)void free_pipe_info (struct pipe_inode_info *pipe) { -int i; +unsigned int i;#ifdef CONFIG_WATCH_QUEUE if (pipe->watch_queue) { --->8---
Quoted 2 lines
Otherwise this loop in
free_pipe_info()may become lockup on some ugly platforms with INTMAX allocation reachable, I think. I may be wrong :-)
This change looks reasonable, it makes types of local variables consistent with proper fields of pipe_inode_info. But right now, the maximum pipe size is limited by (1<<31) (look at round_pipe_size) and so we don't have a real issue here.
Thanks,
Andrei
Quoted 1 line · Andrei Vagin
On Tue, Jan 4, 2022 at 10:54 PM Dmitry Safonov <[email protected]> wrote:
[..]
Quoted 7 lines
Quoted 2 lines
Otherwise this loop in
free_pipe_info()may become lockup on some ugly platforms with INTMAX allocation reachable, I think. I may be wrong :-)This change looks reasonable, it makes types of local variables consistent with proper fields of
pipe_inode_info. But right now, the maximum pipe size is limited by (1<<31) (look atround_pipe_size) and so we don't have a real issue here.
Right you are, I haven't noticed it.
Hmm,
: if (size > (1U << 31))
: return 0;
Isn't size == (1U << 31) a negative integer on 32-bit?
[probably, the question is not very related to your patch, though]
Thanks,
Dmitry
Quoted 14 lines · Andrei Vagin
Right now, kcalloc is used to allocate a
pipe_bufferarray. The size of thepipe_bufferstruct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal toPAGE_ALLOC_COSTLY_ORDER(3). It means that the maximum pipe size is 3.2MB in this case.In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.
kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by
PAGE_ALLOC_COSTLY_ORDER.
Quoted 6 lines
The maximum pipe size for non-root users is limited by
the/proc/sys/fs/pipe-max-sizesysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.Cc Dmitry Safonov <[email protected]>Signed-off-by Andrei Vagin <[email protected]>
Quoted 37 lines
--- fs/pipe.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)fs/pipe.c +3
-4@@ -802,7 +802,7 @@ struct pipe_inode_info *alloc_pipe_info(void)if (too_many_pipe_buffers_hard (user_bufs) &&pipe_is_unprivileged_user ()) goto out_revert_acct; - pipe->bufs =kcalloc (pipe_bufs,sizeof (struct pipe_buffer ), + pipe->bufs =kvcalloc (pipe_bufs,sizeof (struct pipe_buffer ),GFP_KERNEL_ACCOUNT );if (pipe->bufs) {@@ -845,7 +845,7 @@ void free_pipe_info(struct pipe_inode_info *pipe) }if (pipe->tmp_page)__free_page (pipe->tmp_page); -kfree (pipe->bufs); +kvfree (pipe->bufs);kfree (pipe); }@@ -1260,8 +1260,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots)if (nr_slots < n)return -EBUSY ; - bufs =kcalloc (nr_slots,sizeof (*bufs), -GFP_KERNEL_ACCOUNT | __GFP_NOWARN); + bufs =kvcalloc (nr_slots,sizeof (*bufs),GFP_KERNEL_ACCOUNT );if (unlikely (!bufs))return -ENOMEM ;
fs/pipe.c int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots) · line 1293
Quoted 8 lines
fs/pipe.c +1
-1@@ -1288,7 +1287,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots) head = n; tail =0 ; -kfree (pipe->bufs); +kvfree (pipe->bufs); pipe->bufs = bufs; pipe->ring_size = nr_slots;if (pipe->max_usage > nr_slots)
Thanks,
Dmitry
Quoted 15 lines · Andrei Vagin
Right now, kcalloc is used to allocate a
pipe_bufferarray. The size of thepipe_bufferstruct is 40 bytes. kcalloc allows allocating reliably chunks with sizes less or equal toPAGE_ALLOC_COSTLY_ORDER(3). It means that the maximum pipe size is 3.2MB in this case.In CRIU, we use pipes to dump processes memory. CRIU freezes a target process, injects a parasite code into it and then this code splices memory into pipes. If a maximum pipe size is small, we need to do many iterations or create many pipes.
kvcalloc attempt to allocate physically contiguous memory, but upon failure, fall back to non-contiguous (vmalloc) allocation and so it isn't limited by
PAGE_ALLOC_COSTLY_ORDER.
Quoted 6 lines
The maximum pipe size for non-root users is limited by
the/proc/sys/fs/pipe-max-sizesysctl that is 1MB by default, so only the root user will be able to trigger vmalloc allocations.Cc Dmitry Safonov <[email protected]>Signed-off-by Andrei Vagin <[email protected]>
Alexander and Andrew,
Do you have any objections? Could you merge this patch?
Dmitry's comments have been addressed in the separate patch:
https://lkml.org/lkml/2022/1/6/463
BTW: I found that fuse already uses kvmalloc to allocate pipe buffers (e. g. fuse_dev_splice_read).
Thanks,
Andrei