splice call weird results
Hi! While been trying to run criu on linux-next (due to recent Kirill's patch related to /proc/pid/clear_refs with THP enabled) I noticed that it fails dumping programs when moves data from memory pages into an image file. So I wrote pretty idiotic test and run it on current fedora 20 kernel and then on linux-next.
fedora-20 output
----------------
[cyrill@moon criu] ~/pipe
Opened pipe-test.SQBcJa
vmspliced 8192 bytes
spliced 4096 bytes
linux-next
----------
[root@fc criu]# ~/pipe
Opened pipe-test.9nZSW7
vmspliced 8192 bytes
spliced 8192 bytes
In test I fill pipe with 8K data then splice 4K of it into a file (the test is below). Is it intended, or I did some silly mistake?
---
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <sys/uio.h>
#define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0]))
static char buf[8 << 20];
int main(int argc, char *argv[])
{
char *tmpname, t[64] = "pipe-test.XXXXXX";
int _pipe[2], fd, ret = 0;
struct iovec iov[] = {
[0] = {
.iov_base = &buf[0],
.iov_len = 4096,
},
[1] = {
.iov_base = &buf[4096],
.iov_len = 4096,
},
};
if (pipe(_pipe)) {
perror("Can't create pipe");
exit(1);
}
memset(buf, 0x1, sizeof(buf));
tmpname = mktemp(t);
fd = open(tmpname, O_RDWR | O_CREAT | O_TRUNC);
if (fd < 0) {
perror("Can't open temp file");
close(_pipe[0]);
close(_pipe[1]);
exit(1);
}
printf("Opened %s\n", tmpname);
ret = vmsplice(_pipe[1], iov, ARRAY_SIZE(iov), SPLICE_F_GIFT);
printf("vmspliced %li bytes\n", (long)ret);
ret = splice(_pipe[0], NULL, fd, NULL, iov[0].iov_len, SPLICE_F_MOVE);
printf("spliced %li bytes\n", (long)ret);out:
close(_pipe[0]); close(_pipe[1]); close(fd); unlink(tmpname); return ret; }
Quoted 5 lines · Cyrill Gorcunov
Hi! While been trying to run criu on linux-next (due to recent Kirill's patch related to
/proc/pid/clear_refswith THP enabled) I noticed that it fails dumping programs when moves data from memory pages into an image file. So I wrote pretty idiotic test and run it on current fedora 20 kernel and then on linux-next.
+Al. He reworked splice code in linux-next.
Quoted 78 lines
fedora-20 output
----------------
[cyrill@moon criu] ~/pipe
Opened pipe-test.SQBcJa
vmspliced 8192 bytes
spliced 4096 byteslinux-next
----------
[root@fc criu]# ~/pipe
Opened pipe-test.9nZSW7
vmspliced 8192 bytes
spliced 8192 bytesIn test I fill pipe with 8K data then splice 4K of it into a file (the test is below). Is it intended, or I did some silly mistake?
---
#define_GNU_SOURCE#include <
stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>#include <
sys/uio.h>#define
ARRAY_SIZE(x)(sizeof(x)/sizeof(x[0]))static char buf[8 << 20];
int
main(int argc, char *argv[]){ char *tmpname, t[64] = "pipe-test.XXXXXX"; int _pipe[2], fd, ret = 0; struct iovec iov[] = { [0] = { .iov_base = &buf[0], .iov_len = 4096, }, [1] = { .iov_base = &buf[4096], .iov_len = 4096, }, }; if (pipe(_pipe)) { perror("Can't create pipe"); exit(1); } memset(buf, 0x1, sizeof(buf)); tmpname = mktemp(t); fd = open(tmpname, O_RDWR | O_CREAT | O_TRUNC); if (fd < 0) { perror("Can't open temp file"); close(_pipe[0]); close(_pipe[1]); exit(1); } printf("Opened %s\n", tmpname); ret = vmsplice(_pipe[1], iov, ARRAY_SIZE(iov), SPLICE_F_GIFT); printf("vmspliced %li bytes\n", (long)ret); ret = splice(_pipe[0], NULL, fd, NULL, iov[0].iov_len, SPLICE_F_MOVE); printf("spliced %li bytes\n", (long)ret);out:
close(_pipe[0]); close(_pipe[1]); close(fd); unlink(tmpname); return ret; }
Signature · 1 line
Kirill A. Shutemov
Quoted 8 lines · Kirill A. Shutemov
Quoted 5 lines · Cyrill Gorcunov
Hi! While been trying to run criu on linux-next (due to recent Kirill's patch related to
/proc/pid/clear_refswith THP enabled) I noticed that it fails dumping programs when moves data from memory pages into an image file. So I wrote pretty idiotic test and run it on current fedora 20 kernel and then on linux-next.+Al. He reworked splice code in linux-next.
Thanks Kirill. Al, it looks like iter_file_splice_write no longer honor @len while building kiocb vector but pushes as much as it can.
Quoted 8 lines · Cyrill Gorcunov
Quoted 7 lines · Kirill A. Shutemov
Quoted 5 lines · Cyrill Gorcunov
Hi! While been trying to run criu on linux-next (due to recent Kirill's patch related to
/proc/pid/clear_refswith THP enabled) I noticed that it fails dumping programs when moves data from memory pages into an image file. So I wrote pretty idiotic test and run it on current fedora 20 kernel and then on linux-next.
Quoted 4 lines
Quoted 1 line
+Al. He reworked splice code in linux-next.
Thanks Kirill. Al, it looks like
iter_file_splice_writeno longer honor @len while building kiocb vector but pushes as much as it can.
I guess this should fix your problem:
https://lkml.org/lkml/2014/5/27/353
Quoted 9 lines · Cong Wang
Quoted 8 lines · Cyrill Gorcunov
Quoted 7 lines · Kirill A. Shutemov
Quoted 5 lines · Cyrill Gorcunov
Hi! While been trying to run criu on linux-next (due to recent Kirill's patch related to
/proc/pid/clear_refswith THP enabled) I noticed that it fails dumping programs when moves data from memory pages into an image file. So I wrote pretty idiotic test and run it on current fedora 20 kernel and then on linux-next.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
+Al. He reworked splice code in linux-next.
Thanks Kirill. Al, it looks like
iter_file_splice_writeno longer honor @len while building kiocb vector but pushes as much as it can.I guess this should fix your problem:
https://lkml.org/lkml/2014/5/27/353
Thanks, but i doubt, the problem I notice is rather in iter_file_splice_write.