viriofsd: do not fall back to buffer io when cache=auto
RFC PATCH 1/1 viriofsd: do not fall back to buffer io when cache=auto
Direct io flags will be tured off even if cache=auto, which seems a little bit strange. It's better to keep the open flags set by user. If I missed the discussion about this issue, please let me know, thanks.
--- contrib/virtiofsd/passthrough_ll.c | 4 ++++ 1 file changed, 4 insertions(+)
contrib/virtiofsd/passthrough_ll.c +4 -0
@@ -1687,6 +1687,8 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT); out:lo_inode_put (lo, &parent_inode);
@@ -1899,6 +1901,8 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);fuse_reply_open (req, fi); }
Ping, any comments on this patch?
Quoted 31 lines · piaojun
Direct io flags will be tured off even if cache=auto, which seems a little bit strange. It's better to keep the open flags set by user. If I missed the discussion about this issue, please let me know, thanks.
Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 4 ++++ 1 file changed, 4 insertions(+)contrib/virtiofsd/passthrough_ll.c +4
-0@@ -1687,6 +1687,8 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT); out:lo_inode_put (lo, &parent_inode);@@ -1899,6 +1901,8 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);fuse_reply_open (req, fi); }
Quoted 23 lines · piaojun
Direct io flags will be tured off even if cache=auto, which seems a little bit strange. It's better to keep the open flags set by user. If I missed the discussion about this issue, please let me know, thanks.
Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 4 ++++ 1 file changed, 4 insertions(+)contrib/virtiofsd/passthrough_ll.c +2
-0@@ -1687,6 +1687,8 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT); out:lo_inode_put (lo, &parent_inode);
contrib/virtiofsd/passthrough_ll.c static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) · line 1906
Quoted 6 lines
contrib/virtiofsd/passthrough_ll.c +2
-0@@ -1899,6 +1901,8 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);fuse_reply_open (req, fi);
IMHO, this is necessary, it seems that in the case cache=auto/always, virtiofsd already follows what guest requests, i.e. virtiofsd only does direct I/O when guest opens file with O_DIRECT.
If I understand the code correctly, fi->direct_io = 1 is used to force guest to use direct I/O even if guest opens file without O_DIRECT.
Thanks,
Eryu
Quoted 8 lines
contrib/virtiofsd/passthrough_ll.c +0
-0}Signature · 4 lines
_______________________________________________ Virtio-fs mailing list [email protected] https://www.redhat.com/mailman/listinfo/virtio-fs
Quoted 35 lines · Eryu Guan
Quoted 23 lines · piaojun
Direct io flags will be tured off even if cache=auto, which seems a little bit strange. It's better to keep the open flags set by user. If I missed the discussion about this issue, please let me know, thanks.
Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 4 ++++ 1 file changed, 4 insertions(+)contrib/virtiofsd/passthrough_ll.c +2
-0@@ -1687,6 +1687,8 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT); out:lo_inode_put (lo, &parent_inode);contrib/virtiofsd/passthrough_ll.c static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) · line 1906
Quoted 6 lines
contrib/virtiofsd/passthrough_ll.c +2
-0@@ -1899,6 +1901,8 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);fuse_reply_open (req, fi);IMHO, this is necessary, it seems that in the case cache=auto/always, virtiofsd already follows what guest requests, i.e. virtiofsd only does direct I/O when guest opens file with
O_DIRECT.
Quoted 1 line
From my testing, virtiofsd won't do direct io even if guest opens file
with O_DIRECT when cache=auto/alway. It's weird to convert direct io to buffer io at host side.
Quoted 3 lines
If I understand the code correctly,
fi->direct_io= 1 is used to force guest to use direct I/O even if guest opens file withoutO_DIRECT.
But in fact, direct I/O will be forced to buffer I/O even if with O_DIRECT. I will look into this problem by further testing, and very glad to share my test result.
Jun
Quoted 24 lines · piaojun
Quoted 21 lines · Eryu Guan
Quoted 20 lines · piaojun
Direct io flags will be tured off even if cache=auto, which seems a little bit strange. It's better to keep the open flags set by user. If I missed the discussion about this issue, please let me know, thanks.
Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 4 ++++ 1 file changed, 4 insertions(+)contrib/virtiofsd/passthrough_ll.c +2
-0@@ -1687,6 +1687,8 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);
Quoted 11 lines
Quoted 11 lines
contrib/virtiofsd/passthrough_ll.c static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) · line 1906
Quoted 9 lines
contrib/virtiofsd/passthrough_ll.c +2
-0out:lo_inode_put (lo, &parent_inode);@@ -1899,6 +1901,8 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);fuse_reply_open (req, fi);IMHO, this is necessary, it seems that in the case cache=auto/always,
Huge typo here, I meant "not necessary" here..
Quoted 6 lines
Quoted 2 lines
virtiofsd already follows what guest requests, i.e. virtiofsd only does direct I/O when guest opens file with
O_DIRECT.From my testing, virtiofsd won't do direct io even if guest opens file with
O_DIRECTwhen cache=auto/alway. It's weird to convert direct io to buffer io at host side.
That doesn't match what's in my mind, I think FUSE_OPEN will pass guest open flags to virtiofsd and virtiofsd saves the fd (which is opened with O_DIRECT) in fd map, then subsequent FUSE_WRITE does direct write to that fd.
Quoted 8 lines
Quoted 4 lines
If I understand the code correctly,
fi->direct_io= 1 is used to force guest to use direct I/O even if guest opens file withoutO_DIRECT.But in fact, direct I/O will be forced to buffer I/O even if with
O_DIRECT. I will look into this problem by further testing, and very glad to share my test result.
Thanks!
Eryu
Quoted 42 lines · Eryu Guan
Quoted 24 lines · piaojun
Quoted 21 lines · Eryu Guan
Quoted 20 lines · piaojun
Direct io flags will be tured off even if cache=auto, which seems a little bit strange. It's better to keep the open flags set by user. If I missed the discussion about this issue, please let me know, thanks.
Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 4 ++++ 1 file changed, 4 insertions(+)contrib/virtiofsd/passthrough_ll.c +2
-0@@ -1687,6 +1687,8 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);Quoted 11 lines
Quoted 11 lines
contrib/virtiofsd/passthrough_ll.c static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) · line 1906
Quoted 9 lines
contrib/virtiofsd/passthrough_ll.c +2
-0out:lo_inode_put (lo, &parent_inode);@@ -1899,6 +1901,8 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);fuse_reply_open (req, fi);IMHO, this is necessary, it seems that in the case cache=auto/always,
Huge typo here, I meant "not necessary" here..
Quoted 3 lines
Quoted 2 lines
virtiofsd already follows what guest requests, i.e. virtiofsd only does direct I/O when guest opens file with
O_DIRECT.
Quoted 8 lines
Quoted 3 lines
From my testing, virtiofsd won't do direct io even if guest opens file with
O_DIRECTwhen cache=auto/alway. It's weird to convert direct io to buffer io at host side.That doesn't match what's in my mind, I think
FUSE_OPENwill pass guest open flags to virtiofsd and virtiofsd saves the fd (which is opened withO_DIRECT) in fd map, then subsequentFUSE_WRITEdoes direct write to that fd.
Yes, I agree with this. And my comment was a little confuing, so I have to resend the patch later to make it more clear.
Jun
RFC PATCH 1/1 virtiofsd: do not fall back to buffer io when cache=auto/always
In reply to piaojunWhen O_DIRECT flags is set by Guest, virtiofsd will open file with O_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path. That causes inconsistency between Guest and Host.
The cache option in virtiofsd should not affect the file io mode, so set 'fi->direct_io' according to 'fi->flags'.
--- contrib/virtiofsd/passthrough_ll.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
contrib/virtiofsd/passthrough_ll.c +7 -2
@@ -1774,8 +1774,10 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;
@@ -1982,8 +1984,11 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) } fi->fh = fh; -if (lo->cache ==CACHE_NONE ) + +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;fuse_reply_open (req, fi);
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
Quoted 17 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path. That causes inconsistency between Guest and Host.The cache option in virtiofsd should not affect the file io mode, so set '
fi->direct_io' according to 'fi->flags'.Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)contrib/virtiofsd/passthrough_ll.c +0
-0
contrib/virtiofsd/passthrough_ll.c static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, · line 1777
Quoted 5 lines
contrib/virtiofsd/passthrough_ll.c +1
-1@@ -1774,8 +1774,10 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (fi->flags &O_DIRECT )
Looks like this changes cache=none behavior. Previously, we set direct_io when cache=none and guest will go to dio path even when the file is opened for buffer io.
But with this change, guest could do buffer io in cache=none case.
contrib/virtiofsd/passthrough_ll.c static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, · line 1780
Quoted 3 lines
contrib/virtiofsd/passthrough_ll.c +2
-0fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;
And this only causes guest to invalidate page cache at open(2) time, but still go buffer io path on cache=none.
contrib/virtiofsd/passthrough_ll.c static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, · line 1782
Quoted 2 lines
contrib/virtiofsd/passthrough_ll.c +0
-0else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;
I'm thinking about something like (totally untested):
contrib/virtiofsd/passthrough_ll.c +2 -2
@@ -1774,7 +1774,7 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (lo->cache ==CACHE_NONE || fi->flags &O_DIRECT ) fi->direct_io =1 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;
@@ -1982,7 +1982,7 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) } fi->fh = fh; -if (lo->cache ==CACHE_NONE ) +if (lo->cache ==CACHE_NONE || fi->flags &O_DIRECT ) fi->direct_io =1 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;
contrib/virtiofsd/passthrough_ll.c static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) · line 1993
Quoted 13 lines
contrib/virtiofsd/passthrough_ll.c +4
-1@@ -1982,8 +1984,11 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) } fi->fh = fh; -if (lo->cache ==CACHE_NONE ) + +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;
And this code snippet duplicates with lo_create(), maybe we could factor it into a new helper function.
Thanks,
Eryu
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
Quoted 28 lines · Eryu Guan
Quoted 17 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path. That causes inconsistency between Guest and Host.The cache option in virtiofsd should not affect the file io mode, so set '
fi->direct_io' according to 'fi->flags'.Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)contrib/virtiofsd/passthrough_ll.c +0
-0contrib/virtiofsd/passthrough_ll.c static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, · line 1777
Quoted 5 lines
contrib/virtiofsd/passthrough_ll.c +1
-1@@ -1774,8 +1774,10 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (fi->flags &O_DIRECT )Looks like this changes cache=none behavior. Previously, we set
direct_iowhen cache=none and guest will go to dio path even when the file is opened for buffer io.
Quoted 2 lines
But with this change, guest could do buffer io in cache=none case.
IMO, the cache option is only used for determining whether using the last inode's cache when do open(). The user could choose direct or buffer io no matter if the cache exists.
Moreover, the cache option only works for the first time buffer io, and has little effect on subsequent ones.
Quoted 6 lines
contrib/virtiofsd/passthrough_ll.c static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, · line 1780
Quoted 3 lines
contrib/virtiofsd/passthrough_ll.c +2
-0fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;And this only causes guest to invalidate page cache at open(2) time, but still go buffer io path on cache=none.
As my comments above, io path has nothing to do with inode's current page cache.
Quoted 18 lines
contrib/virtiofsd/passthrough_ll.c static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, · line 1782
Quoted 3 lines
contrib/virtiofsd/passthrough_ll.c +0
-0else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;I'm thinking about something like (totally untested):
contrib/virtiofsd/passthrough_ll.c +1
-1@@ -1774,7 +1774,7 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (lo->cache ==CACHE_NONE || fi->flags &O_DIRECT ) fi->direct_io =1 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;
contrib/virtiofsd/passthrough_ll.c static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) · line 1988
Quoted 8 lines
contrib/virtiofsd/passthrough_ll.c +1
-1@@ -1982,7 +1982,7 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) } fi->fh = fh; -if (lo->cache ==CACHE_NONE ) +if (lo->cache ==CACHE_NONE || fi->flags &O_DIRECT ) fi->direct_io =1 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;
This will mix cache option with io flags which seems weird.
Quoted 5 lines
Quoted 5 lines
contrib/virtiofsd/passthrough_ll.c +0
-0@@ -1982,8 +1984,11 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) }
Quoted 12 lines
contrib/virtiofsd/passthrough_ll.c static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, · line 1780
Quoted 9 lines
contrib/virtiofsd/passthrough_ll.c +4
-1fi->fh = fh; -if (lo->cache ==CACHE_NONE ) + +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;And this code snippet duplicates with
lo_create(), maybe we could factor it into a new helper function.
It makes sense and I will consider adding a helper.
Jun
Quoted 3 lines · piaojun
Direct io flags will be tured off even if cache=auto, which seems a little bit strange. It's better to keep the open flags set by user. If I missed the discussion about this issue, please let me know, thanks.
fi->direct_io and fi->keep_cache are controlling hints for guest kernel. So for cache=none, we want to enforce direct I/O in guest hence we set fi->direct_io (FOPEN_DIRECT_IO) in reply to guest.
For cache=always, we want to cache data and metadata hence we don't set fi->direct_io. Also, we don't want to invalidate current cache contents of file in guest hence we set fi->keep_cache (FOPEN_KEEP_CACHE)
For cache=auto, want guest kernel to cache data and metadata hence we don't set fi->direct_io. We want guest kernel to invalidate data on next open hence we don't set fi->keep_cache.
If guest is doing direct IO on file and fi->flags has O_DIRECT set, that should not do anything to fi->direct_io. That's a hint for guest kernel enforced by virtiofsd.
I am not sure what problem you are trying to solve with this patch.
Thanks
Vivek
Quoted 34 lines
Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 4 ++++ 1 file changed, 4 insertions(+)contrib/virtiofsd/passthrough_ll.c +4
-0@@ -1687,6 +1687,8 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT); out:lo_inode_put (lo, &parent_inode);@@ -1899,6 +1901,8 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fi->direct_io =1 ; else if (lo->cache == CACHE_ALWAYS) fi->keep_cache = 1; + else + fi->direct_io = !!(fi->flags & O_DIRECT);fuse_reply_open (req, fi); }Signature · 4 lines
_______________________________________________ Virtio-fs mailing list [email protected] https://www.redhat.com/mailman/listinfo/virtio-fs
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
In reply to piaojunQuoted 2 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path.
Where is virtiofsd unset fi->direct_io flag? All I can see is that it sets this flag if cache=none?
Thanks
Vivek
Quoted 45 lines
That causes inconsistency between Guest and Host.
The cache option in virtiofsd should not affect the file io mode, so set '
fi->direct_io' according to 'fi->flags'.Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)contrib/virtiofsd/passthrough_ll.c +7
-2@@ -1774,8 +1774,10 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;@@ -1982,8 +1984,11 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) } fi->fh = fh; -if (lo->cache ==CACHE_NONE ) + +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;fuse_reply_open (req, fi);Signature · 4 lines
_______________________________________________ Virtio-fs mailing list [email protected] https://www.redhat.com/mailman/listinfo/virtio-fs
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
Quoted 7 lines · Vivek Goyal
Quoted 2 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path.Where is virtiofsd unset
fi->direct_ioflag? All I can see is that it sets this flag if cache=none?
Ok, I looked at it little bit more closely and I don't think it is a problem. With cache=always/auto, we don't set fi->direct_io. That means server is not enforcing a direct I/O policy on client.
Now if client decides to direct I/O by opening file with O_DIRECT, it will still take direct I/O path and call fuse_direct_IO(). Look at fuse_file_aops->fuse_direct_IO().
Thanks
Vivek
Quoted 48 lines
Thanks
VivekQuoted 45 lines
That causes inconsistency between Guest and Host.
The cache option in virtiofsd should not affect the file io mode, so set '
fi->direct_io' according to 'fi->flags'.Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)contrib/virtiofsd/passthrough_ll.c +7
-2@@ -1774,8 +1774,10 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;@@ -1982,8 +1984,11 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) } fi->fh = fh; -if (lo->cache ==CACHE_NONE ) + +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;fuse_reply_open (req, fi);Signature · 4 lines
_______________________________________________ Virtio-fs mailing list [email protected] https://www.redhat.com/mailman/listinfo/virtio-fs
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
Quoted 9 lines · Vivek Goyal
Quoted 7 lines · Vivek Goyal
Quoted 2 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path.Where is virtiofsd unset
fi->direct_ioflag? All I can see is that it sets this flag if cache=none?
Quoted 7 lines
Ok, I looked at it little bit more closely and I don't think it is a problem. With cache=always/auto, we don't set
fi->direct_io. That means server is not enforcing a direct I/O policy on client.Now if client decides to direct I/O by opening file with
O_DIRECT, it will still take direct I/O path and callfuse_direct_IO(). Look atfuse_file_aops->fuse_direct_IO().
The problem is that when Guest open with O_DIRECT, the Host won't set open_flags with FOPEN_DIRECT_IO if CACHE_AUTO is set. It makes Guest goes into fuse_cache_write_iter() rather than fuse_direct_write_iter(). In other words, the Guest's I/O mode will be changed by virtiofsd.
You cleared my first doubt that 'fi->direct_io' and 'fi->keep_cache' are used to control Guest I/O mode together, and there may be different I/O modes between Guest and Host if I undersatnd it correctly. I admit this offers a more flexible config to user, but also introduces more complex option. Maybe we could just use *keep_cache* to handle all these options.
My second doubt is that *keep_cache* is used to control whether reusing the cache of last open(). IMO, it has nothing to do with the following I/O path.
Jun
Quoted 54 lines
Thanks
VivekQuoted 44 lines
Thanks
VivekQuoted 41 lines
That causes inconsistency between Guest and Host.
The cache option in virtiofsd should not affect the file io mode, so set '
fi->direct_io' according to 'fi->flags'.Signed-off-by Jun Piao <[email protected]>--- contrib/virtiofsd/passthrough_ll.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)contrib/virtiofsd/passthrough_ll.c +7
-2@@ -1774,8 +1774,10 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, fi->fh = fh; err =lo_do_lookup (req, parent, name, &e); } -if (lo->cache ==CACHE_NONE ) +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;@@ -1982,8 +1984,11 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) } fi->fh = fh; -if (lo->cache ==CACHE_NONE ) + +if (fi->flags &O_DIRECT ) fi->direct_io =1 ; +if (lo->cache ==CACHE_NONE ) + fi->keep_cache =0 ;else if (lo->cache ==CACHE_ALWAYS ) fi->keep_cache =1 ;fuse_reply_open (req, fi);Quoted 4 lines
Quoted 4 lines
_______________________________________________
Virtio-fs mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virtio-fs.
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
Quoted 16 lines · piaojun
Quoted 13 lines · Vivek Goyal
Quoted 7 lines · Vivek Goyal
Quoted 2 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path.Where is virtiofsd unset
fi->direct_ioflag? All I can see is that it sets this flag if cache=none?Ok, I looked at it little bit more closely and I don't think it is a problem. With cache=always/auto, we don't set
fi->direct_io. That means server is not enforcing a direct I/O policy on client.
Quoted 8 lines
Quoted 3 lines
Now if client decides to direct I/O by opening file with
O_DIRECT, it will still take direct I/O path and callfuse_direct_IO(). Look atfuse_file_aops->fuse_direct_IO().The problem is that when Guest open with
O_DIRECT, the Host won't setopen_flagswithFOPEN_DIRECT_IOifCACHE_AUTOis set. It makes Guest goes intofuse_cache_write_iter()rather thanfuse_direct_write_iter(). In other words, the Guest's I/O mode will be changed by virtiofsd.
I think there are two direct I/O paths. One is where server requests that direct I/O be done on the file and in that case FOPEN_DIRECT_IO is set.
Other case is where client initiates direct I/O and in that case FOPEN_DIRECT_IO is not set and we take regular path which is capable of doing both direct and buffered I/O. For example, for reads we do following.
fuse_file_read_iter()
fuse_cache_read_iter()
generic_file_read_iter() mapping->a_ops->direct_IO() <--- (If iocb->ki_flags & IOCB_DIRECT) fuse_direct_IO()
Similary for write path.
fuse_file_write_iter()
fuse_cache_write_iter()
generic_file_direct_write() <----- (If iocb->ki_flags & IOCB_DIRECT)
mapping->a_ops->direct_IO()
fuse_direct_IO()Can you give it a try. I don't think we have the problem what you are describing.
Quoted 11 lines
You cleared my first doubt that '
fi->direct_io' and 'fi->keep_cache' are used to control Guest I/O mode together, and there may be different I/O modes between Guest and Host if I undersatnd it correctly. I admit this offers a more flexible config to user, but also introduces more complex option. Maybe we could just use *keep_cache* to handle all these options.My second doubt is that *keep_cache* is used to control whether reusing the cache of last
open(). IMO, it has nothing to do with the following I/O path.
FOPEN_KEEP_CACHE is already part of fuse protocol. cache=always is just making use of that option so that cache contents are not discarded upon next open.
Thanks
Vivek
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
Quoted 43 lines · Vivek Goyal
Quoted 16 lines · piaojun
Quoted 13 lines · Vivek Goyal
Quoted 7 lines · Vivek Goyal
Quoted 2 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path.Where is virtiofsd unset
fi->direct_ioflag? All I can see is that it sets this flag if cache=none?Ok, I looked at it little bit more closely and I don't think it is a problem. With cache=always/auto, we don't set
fi->direct_io. That means server is not enforcing a direct I/O policy on client.Quoted 8 lines
Quoted 3 lines
Now if client decides to direct I/O by opening file with
O_DIRECT, it will still take direct I/O path and callfuse_direct_IO(). Look atfuse_file_aops->fuse_direct_IO().The problem is that when Guest open with
O_DIRECT, the Host won't setopen_flagswithFOPEN_DIRECT_IOifCACHE_AUTOis set. It makes Guest goes intofuse_cache_write_iter()rather thanfuse_direct_write_iter(). In other words, the Guest's I/O mode will be changed by virtiofsd.I think there are two direct I/O paths. One is where server requests that direct I/O be done on the file and in that case
FOPEN_DIRECT_IOis set.Other case is where client initiates direct I/O and in that case
FOPEN_DIRECT_IOis not set and we take regular path which is capable of doing both direct and buffered I/O. For example, for reads we do following.
fuse_file_read_iter()
fuse_cache_read_iter()generic_file_read_iter() mapping->a_ops->direct_IO() <--- (If iocb->ki_flags & IOCB_DIRECT) fuse_direct_IO()Similary for write path.
Quoted 8 lines
fuse_file_write_iter()
fuse_cache_write_iter()generic_file_direct_write() <----- (If iocb->ki_flags & IOCB_DIRECT) mapping->a_ops->direct_IO() fuse_direct_IO()Can you give it a try. I don't think we have the problem what you are describing.
Thanks for your detailed explanation, and perhaps I missed the regular path you mentioned above, so I will have a try to confirm this later.
Quoted 9 lines
Quoted 9 lines
You cleared my first doubt that '
fi->direct_io' and 'fi->keep_cache' are used to control Guest I/O mode together, and there may be different I/O modes between Guest and Host if I undersatnd it correctly. I admit this offers a more flexible config to user, but also introduces more complex option. Maybe we could just use *keep_cache* to handle all these options.
Quoted 7 lines
Quoted 3 lines
My second doubt is that *keep_cache* is used to control whether reusing the cache of last
open(). IMO, it has nothing to do with the following I/O path.
FOPEN_KEEP_CACHEis already part of fuse protocol. cache=always is just making use of that option so that cache contents are not discarded upon next open.
It still confused me that probably we needn't set fi->direct_io if CACHE_NONE is set to indicate Guest go direct I/O path. As Guest could also go direct I/O through regular path you have mentioned above.
Jun
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
On Wed, Aug 21, 2019 at 10:38:39PM +0800, piaojun wrote:
[..]
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
You cleared my first doubt that '
fi->direct_io' and 'fi->keep_cache' are used to control Guest I/O mode together, and there may be different I/O modes between Guest and Host if I undersatnd it correctly. I admit this offers a more flexible config to user, but also introduces more complex option. Maybe we could just use *keep_cache* to handle all these options.My second doubt is that *keep_cache* is used to control whether reusing the cache of last
open(). IMO, it has nothing to do with the following I/O path.
Quoted 7 lines
Quoted 3 lines
FOPEN_KEEP_CACHEis already part of fuse protocol. cache=always is just making use of that option so that cache contents are not discarded upon next open.It still confused me that probably we needn't set
fi->direct_ioifCACHE_NONEis set to indicate Guest go direct I/O path. As Guest could also go direct I/O through regular path you have mentioned above.
fi->direct_io is needed so that server can *enforce* direct I/O on guest. If guest applications are not doing direct I/O, without fi->direct_io, you can't force client to do direct I/O.
Vivek
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
Quoted 19 lines · Vivek Goyal
On Wed, Aug 21, 2019 at 10:38:39PM +0800, piaojun wrote:
[..]
Quoted 16 lines
Quoted 15 lines
Quoted 11 lines
You cleared my first doubt that '
fi->direct_io' and 'fi->keep_cache' are used to control Guest I/O mode together, and there may be different I/O modes between Guest and Host if I undersatnd it correctly. I admit this offers a more flexible config to user, but also introduces more complex option. Maybe we could just use *keep_cache* to handle all these options.My second doubt is that *keep_cache* is used to control whether reusing the cache of last
open(). IMO, it has nothing to do with the following I/O path.
FOPEN_KEEP_CACHEis already part of fuse protocol. cache=always is just making use of that option so that cache contents are not discarded upon next open.
Quoted 7 lines
Quoted 3 lines
It still confused me that probably we needn't set
fi->direct_ioifCACHE_NONEis set to indicate Guest go direct I/O path. As Guest could also go direct I/O through regular path you have mentioned above.
fi->direct_iois needed so that server can *enforce* direct I/O on guest. If guest applications are not doing direct I/O, withoutfi->direct_io, you can't force client to do direct I/O.
I get yout point, it seems nothing wrong with it, but it indeed made me puzzled once. Maybe we'd better wait for some response from real user in the future.
Thanks,
Jun
Quoted 3 lines
Vivek
Re: [Virtio-fs] [PATCH RESEND][RFC] virtiofsd: do not fall back to buffer io when cache=auto/always
In reply to Vivek GoyalQuoted 43 lines · Vivek Goyal
Quoted 16 lines · piaojun
Quoted 13 lines · Vivek Goyal
Quoted 7 lines · Vivek Goyal
Quoted 2 lines · piaojun
When
O_DIRECTflags is set by Guest, virtiofsd will open file withO_DIRECT, but unset 'fi->direct_io' which makes Guest go buffer io path.Where is virtiofsd unset
fi->direct_ioflag? All I can see is that it sets this flag if cache=none?Ok, I looked at it little bit more closely and I don't think it is a problem. With cache=always/auto, we don't set
fi->direct_io. That means server is not enforcing a direct I/O policy on client.Quoted 8 lines
Quoted 3 lines
Now if client decides to direct I/O by opening file with
O_DIRECT, it will still take direct I/O path and callfuse_direct_IO(). Look atfuse_file_aops->fuse_direct_IO().The problem is that when Guest open with
O_DIRECT, the Host won't setopen_flagswithFOPEN_DIRECT_IOifCACHE_AUTOis set. It makes Guest goes intofuse_cache_write_iter()rather thanfuse_direct_write_iter(). In other words, the Guest's I/O mode will be changed by virtiofsd.I think there are two direct I/O paths. One is where server requests that direct I/O be done on the file and in that case
FOPEN_DIRECT_IOis set.Other case is where client initiates direct I/O and in that case
FOPEN_DIRECT_IOis not set and we take regular path which is capable of doing both direct and buffered I/O. For example, for reads we do following.
fuse_file_read_iter()
fuse_cache_read_iter()generic_file_read_iter() mapping->a_ops->direct_IO() <--- (If iocb->ki_flags & IOCB_DIRECT) fuse_direct_IO()Similary for write path.
Quoted 9 lines
fuse_file_write_iter()
fuse_cache_write_iter()generic_file_direct_write() <----- (If iocb->ki_flags & IOCB_DIRECT) mapping->a_ops->direct_IO() fuse_direct_IO()Can you give it a try. I don't think we have the problem what you are describing.
I make a try as you said, and it indeed goes direct path in guest even if cache=always. So my doubt has been cleared.
Thanks,
Jun