9pfs: readdir optimization
v5 cover 9pfs: readdir optimization
As previously mentioned, I was investigating performance issues with 9pfs. Raw file read/write of 9pfs is actually quite good, provided that client picked a reasonable high msize (maximum message size). I would recommend to log a warning on 9p server side if a client attached with a small msize that would cause performance issues for that reason.
However there are other aspects where 9pfs currently performs suboptimally, especially readdir handling of 9pfs is extremely slow, a simple readdir request of a guest typically blocks for several hundred milliseconds or even several seconds, no matter how powerful the underlying hardware is. The reason for this performance issue: latency.
Currently 9pfs is heavily dispatching a T_readdir request numerous times between main I/O thread and a background I/O thread back and forth; in fact it is actually hopping between threads even multiple times for every single directory entry during T_readdir request handling which leads in total to huge latencies for a single T_readdir request.
This patch series aims to address this severe performance issue of 9pfs T_readdir request handling. The actual performance optimization is patch 5.
v4->v5:
* Rebased to master (SHA-1 762fa6d7).
* Dropped benchmark patches (see v4 if you want to run a benchmark on v5).
* Divided split-readdir test into 3 individual tests, which also fixes the previously discussed transport error [patch 1].
* Fixed English spelling for 'split' [patch 1].
* Rename max_count -> maxsize [NEW patch 2].
* Divided previous huge readdir optimization patch into individual patches
[patch 3], [patch 4], [patch 5].
* Added comment on v9fs_readdir_response_size() [patch 3].
* Renamed v9fs_co_readdir_lowlat() -> v9fs_co_readdir_many() [patch 4].
* Adjusted comment on v9fs_co_readdir_many() [patch 4].
* Added comment on v9fs_co_run_in_worker() [NEW patch 6].
* Adjusted commit log message of several patches.
Message-ID of previous version (v4):
[email protected]
Christian Schoenebeck (6):
tests/virtio-9p: added split readdir tests 9pfs readdir: rename max_count -> maxsize 9pfs: make v9fs_readdir_response_size() public 9pfs: add new function v9fs_co_readdir_many() 9pfs: T_readdir latency optimization 9pfs: clarify latency of v9fs_co_run_in_worker()
hw/9pfs/9p.c | 148 ++++++++++++++-------------- hw/9pfs/9p.h | 23 +++++ hw/9pfs/codir.c | 181 ++++++++++++++++++++++++++++++++--- hw/9pfs/coth.h | 15 ++- tests/qtest/virtio-9p-test.c | 108 +++++++++++++++++++++ 5 files changed, 386 insertions(+), 89 deletions(-)
Signature · 1 line
2.20.1
PATCH v5 1/6 tests/virtio-9p: added split readdir tests
The previous, already existing 'basic' readdir test simply used a 'count' parameter big enough to retrieve all directory entries with a single Treaddir request.
In the 3 new 'split' readdir tests added by this patch, directory entries are retrieved, split over several Treaddir requests by picking small 'count' parameters which force the server to truncate the response. So the test client sends as many Treaddir requests as necessary to get all directory entries.
The following 3 new tests are added (executed in this sequence):
1. Split readdir test with count=512
2. Split readdir test with count=256
3. Split readdir test with count=128
This test case sequence is chosen because the smaller the 'count' value, the higher the chance of errors in case of implementation bugs on server side.
--- tests/qtest/virtio-9p-test.c | 108 +++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+)
tests/qtest/virtio-9p-test.c +108 -0
@@ -578,6 +578,7 @@ static bool fs_dirents_contain_name(struct V9fsDirent *e, const char* name)return false; } +/* basic readdir test where reply fits into a single response message */ static void fs_readdir (void *obj,void *data,QGuestAllocator *t_alloc) {QVirtio9P *v9p = obj;
@@ -631,6 +632,89 @@ static void fs_readdir(void *obj, void *data, QGuestAllocator *t_alloc)g_free (wnames[0 ]); } +/* readdir test where overall request is split over several messages */ +static void fs_readdir_split (void *obj,void *data,QGuestAllocator *t_alloc, +uint32_t count) +{ +QVirtio9P *v9p = obj; + alloc = t_alloc; +char *const wnames[] = {g_strdup (QTEST_V9FS_SYNTH_READDIR_DIR ) }; +uint16_t nqid; +v9fs_qid qid; +uint32_t nentries, npartialentries; +struct V9fsDirent *entries, *tail, *partialentries; +P9Req *req; +int fid; +uint64_t offset; + +fs_attach (v9p,NULL , t_alloc); + + fid =1 ; + offset =0 ; + entries =NULL ; + nentries =0 ; + tail =NULL ; + + req =v9fs_twalk (v9p,0 , fid,1 , wnames,0 ); +v9fs_req_wait_for_reply (req,NULL ); +v9fs_rwalk (req, &nqid,NULL ); + g_assert_cmpint(nqid, ==, 1); + + req =v9fs_tlopen (v9p, fid,O_DIRECTORY ,0 ); +v9fs_req_wait_for_reply (req,NULL ); +v9fs_rlopen (req, &qid,NULL ); + +/* +* send as many Treaddir requests as required to get all directory +* entries +*/ +while (true) { + npartialentries =0 ; + partialentries =NULL ; + + req =v9fs_treaddir (v9p, fid, offset, count,0 ); +v9fs_req_wait_for_reply (req,NULL ); +v9fs_rreaddir (req, &count, &npartialentries, &partialentries); +if (npartialentries >0 && partialentries) { +if (!entries) { + entries = partialentries; + nentries = npartialentries; + tail = partialentries; + }else { + tail->next = partialentries; + nentries += npartialentries; + } +while (tail->next) { + tail = tail->next; + } + offset = tail->offset; + }else { +break ; + } + } + +g_assert_cmpint ( + nentries, ==, +QTEST_V9FS_SYNTH_READDIR_NFILES +2 /* "." and ".." */ + ); + +/* +* Check all file names exist in returned entries, ignore their order +* though. +*/ + g_assert_cmpint(fs_dirents_contain_name(entries, "."), ==, true); + g_assert_cmpint(fs_dirents_contain_name(entries, ".."), ==, true); +for (int i =0 ; i <QTEST_V9FS_SYNTH_READDIR_NFILES ; ++i) { +char *name =g_strdup_printf (QTEST_V9FS_SYNTH_READDIR_FILE , i); + g_assert_cmpint(fs_dirents_contain_name(entries, name), ==, true); +g_free (name); + } + +v9fs_free_dirents (entries); + +g_free (wnames[0 ]); +} +static void fs_walk_no_slash (void *obj,void *data,QGuestAllocator *t_alloc) {QVirtio9P *v9p = obj;
@@ -793,6 +877,24 @@ static void fs_flush_ignored(void *obj, void *data, QGuestAllocator *t_alloc)g_free (wnames[0 ]); } +static void fs_readdir_split_128 (void *obj,void *data, +QGuestAllocator *t_alloc) +{ +fs_readdir_split (obj, data, t_alloc,128 ); +} + +static void fs_readdir_split_256 (void *obj,void *data, +QGuestAllocator *t_alloc) +{ +fs_readdir_split (obj, data, t_alloc,256 ); +} + +static void fs_readdir_split_512 (void *obj,void *data, +QGuestAllocator *t_alloc) +{ +fs_readdir_split (obj, data, t_alloc,512 ); +} +static void register_virtio_9p_test (void ) {qos_add_test ("config" ,"virtio-9p" , pci_config,NULL );
@@ -810,6 +912,12 @@ static void register_virtio_9p_test(void)qos_add_test ("fs/flush/ignored" ,"virtio-9p" , fs_flush_ignored,NULL );qos_add_test ("fs/readdir/basic" ,"virtio-9p" , fs_readdir,NULL ); +qos_add_test ("fs/readdir/split_512" ,"virtio-9p" , + fs_readdir_split_512,NULL ); +qos_add_test ("fs/readdir/split_256" ,"virtio-9p" , + fs_readdir_split_256,NULL ); +qos_add_test ("fs/readdir/split_128" ,"virtio-9p" , + fs_readdir_split_128,NULL ); }libqos_init (register_virtio_9p_test);
Signature · 1 line
2.20.1
PATCH v5 2/6 9pfs readdir: rename max_count -> maxsize
In reply to Christian SchoenebeckAlthough the 9p protocol specs use the term 'max_count' as argument for Treaddir, let's rename our variables for that to 'maxsize' instead, because 'max_count' is semantically completely wrong. This variable does not count integral entries, it is rather a maximum size (in bytes) of the destination (response) buffer being filled.
Since this is just refactoring, hence this patch does not introduce any behaviour change at all.
--- hw/9pfs/9p.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
hw/9pfs/9p.c +10 -10
@@ -2162,7 +2162,7 @@ static int v9fs_xattr_read(V9fsState *s, V9fsPDU *pdu, V9fsFidState *fidp, static int coroutine_fn v9fs_do_readdir_with_stat(V9fsPDU *pdu,V9fsFidState *fidp, -uint32_t max_count) +uint32_t maxsize) {V9fsPath path;V9fsStat v9stat;
@@ -2199,7 +2199,7 @@ static int coroutine_fn v9fs_do_readdir_with_stat(V9fsPDU *pdu,if (err <0 ) {break ; } -if ((count + v9stat.size +2 ) > max_count) { +if ((count + v9stat.size +2 ) > maxsize) {v9fs_readdir_unlock (&fidp->fs.dir);/* Ran out of buffer. Set dir back to old position and return */
@@ -2332,7 +2332,7 @@ static size_t v9fs_readdir_data_size(V9fsString *name) } static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, -int32_t max_count) +int32_t maxsize) {size_t size;V9fsQID qid;
@@ -2357,7 +2357,7 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, }v9fs_string_init (&name);v9fs_string_sprintf (&name,"%s" , dent->d_name); -if ((count +v9fs_readdir_data_size (&name)) > max_count) { +if ((count +v9fs_readdir_data_size (&name)) > maxsize) {v9fs_readdir_unlock (&fidp->fs.dir);/* Ran out of buffer. Set dir back to old position and return */
@@ -2432,20 +2432,20 @@ static void coroutine_fn v9fs_readdir(void *opaque)size_t offset =7 ;uint64_t initial_offset;int32_t count; -uint32_t max_count; +uint32_t maxsize;V9fsPDU *pdu = opaque;V9fsState *s = pdu->s; retval =pdu_unmarshal (pdu, offset,"dqd" , &fid, - &initial_offset, &max_count); + &initial_offset, &maxsize);if (retval <0 ) { goto out_nofid; } -trace_v9fs_readdir (pdu->tag, pdu->id, fid, initial_offset, max_count); +trace_v9fs_readdir (pdu->tag, pdu->id, fid, initial_offset, maxsize);/* Enough space for a R_readdir header: size[4] Rreaddir tag[2] count[4] */ -if (max_count > s->msize -11 ) { - max_count = s->msize -11 ; +if (maxsize > s->msize -11 ) { + maxsize = s->msize -11 ;warn_report_once ("9p: bad client: T_readdir with count > msize - 11" );
@@ -2465,7 +2465,7 @@ static void coroutine_fn v9fs_readdir(void *opaque) }else {v9fs_co_seekdir (pdu, fidp, initial_offset); } - count =v9fs_do_readdir (pdu, fidp, max_count); + count =v9fs_do_readdir (pdu, fidp, maxsize);if (count <0 ) { retval = count; goto out;
Signature · 1 line
2.20.1
PATCH v5 3/6 9pfs: make v9fs_readdir_response_size() public
In reply to Christian SchoenebeckRename function v9fs_readdir_data_size() -> v9fs_readdir_response_size() and make it callable from other units. So far this function is only used by 9p.c, however subsequent patch requires the function to be callable from another 9pfs unit. And as we're at it; also make it clear for what this function is used for.
--- hw/9pfs/9p.c | 10 ++++++++-- hw/9pfs/9p.h | 1 + 2 files changed, 9 insertions(+), 2 deletions(-)
hw/9pfs/9p.c +8 -2
@@ -2322,7 +2322,13 @@ out_nofid:pdu_complete (pdu, err); } -static size_t v9fs_readdir_data_size (V9fsString *name) +/** +* Returns size required in Rreaddir response for the passed dirent @p name. +* +* @param name - directory entry's name (i.e. file name, directory name) +* @returns required size in bytes +*/ +size_t v9fs_readdir_response_size (V9fsString *name) {/* * Size of each dirent on the wire: size of qid (13) + size of offset (8)
@@ -2357,7 +2363,7 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, }v9fs_string_init (&name);v9fs_string_sprintf (&name,"%s" , dent->d_name); -if ((count +v9fs_readdir_data_size (&name)) > maxsize) { +if ((count +v9fs_readdir_response_size (&name)) > maxsize) {v9fs_readdir_unlock (&fidp->fs.dir);/* Ran out of buffer. Set dir back to old position and return */
hw/9pfs/9p.h +1 -0
@@ -419,6 +419,7 @@ void v9fs_path_init(V9fsPath *path);void v9fs_path_free (V9fsPath *path);void v9fs_path_sprintf (V9fsPath *path,const char *fmt, ...);void v9fs_path_copy (V9fsPath *dst,const V9fsPath *src); +size_t v9fs_readdir_response_size (V9fsString *name);int v9fs_name_to_path (V9fsState *s,V9fsPath *dirpath,const char *name,V9fsPath *path); int v9fs_device_realize_common(V9fsState *s, const V9fsTransport *t,
Signature · 1 line
2.20.1
PATCH v5 4/6 9pfs: add new function v9fs_co_readdir_many()
In reply to Christian SchoenebeckThe newly added function v9fs_co_readdir_many() retrieves multiple directory entries with a single fs driver request. It is intended to replace uses of v9fs_co_readdir(), the latter only retrives a single directory entry per fs driver request instead.
The reason for this planned replacement is that for every fs driver request the coroutine is dispatched from main I/O thread to a background I/O thread and eventually dispatched back to main I/O thread. Hopping between threads adds latency. So if a 9pfs Treaddir request reads a large amount of directory entries, this currently sums up to huge latencies of several hundred ms or even more. So using v9fs_co_readdir_many() instead of v9fs_co_readdir() will provide significant performance improvements.
--- hw/9pfs/9p.h | 22 ++++++ hw/9pfs/codir.c | 181 +++++++++++++++++++++++++++++++++++++++++++++--- hw/9pfs/coth.h | 3 + 3 files changed, 195 insertions(+), 11 deletions(-)
hw/9pfs/9p.h +22 -0
@@ -215,6 +215,28 @@ static inline void v9fs_readdir_init(V9fsDir *dir)qemu_mutex_init (&dir->readdir_mutex); } +/** +* Type for 9p fs drivers' (a.k.a. 9p backends) result of readdir requests, +* which is a chained list of directory entries. +*/ +typedef struct V9fsDirEnt { +/* mandatory (must not be NULL) information for all readdir requests */ +struct dirent *dent; +/* +* optional (may be NULL): A full stat of each directory entry is just +* done if explicitly told to fs driver. +*/ +struct stat *st; +/* +* instead of an array, directory entries are always returned as +* chained list, that's because the amount of entries retrieved by fs +* drivers is dependent on the individual entries' name (since response +* messages are size limited), so the final amount cannot be estimated +* before hand +*/ +struct V9fsDirEnt *next; +}V9fsDirEnt ; +/* * Filled by fs driver on open and other * calls.
hw/9pfs/codir.c +170 -11
198 lines not on this page · raw message
hw/9pfs/coth.h +3 -0
@@ -49,6 +49,9 @@void co_run_in_worker_bh (void *); int coroutine_fn v9fs_co_readlink(V9fsPDU *, V9fsPath *, V9fsString *); int coroutine_fn v9fs_co_readdir(V9fsPDU *, V9fsFidState *, struct dirent **); +int coroutine_fn v9fs_co_readdir_many(V9fsPDU *, V9fsFidState *, +struct V9fsDirEnt **, +int32_t ,bool ); off_t coroutine_fn v9fs_co_telldir(V9fsPDU *, V9fsFidState *); void coroutine_fn v9fs_co_seekdir(V9fsPDU *, V9fsFidState *, off_t); void coroutine_fn v9fs_co_rewinddir(V9fsPDU *, V9fsFidState *);
Signature · 1 line
2.20.1
PATCH v5 5/6 9pfs: T_readdir latency optimization
In reply to Christian SchoenebeckMake top half really top half and bottom half really bottom half:
Each T_readdir request handling is hopping between threads (main I/O thread and background I/O driver threads) several times for every individual directory entry, which sums up to huge latencies for handling just a single T_readdir request.
Instead of doing that, collect now all required directory entries (including all potentially required stat buffers for each entry) in one rush on a background I/O thread from fs driver by calling the previously added function v9fs_co_readdir_many() instead of v9fs_co_readdir(), then assemble the entire resulting network response message for the readdir request on main I/O thread. The fs driver is still aborting the directory entry retrieval loop (on the background I/O thread inside of v9fs_co_readdir_many()) as soon as it would exceed the client's requested maximum R_readdir response size. So this will not introduce a performance penalty on another end.
--- hw/9pfs/9p.c | 122 +++++++++++++++++++++++---------------------------- 1 file changed, 55 insertions(+), 67 deletions(-)
hw/9pfs/9p.c +55 -67
@@ -971,30 +971,6 @@ static int coroutine_fn fid_to_qid(V9fsPDU *pdu, V9fsFidState *fidp,return 0 ; } -static int coroutine_fn dirent_to_qid(V9fsPDU *pdu, V9fsFidState *fidp, -struct dirent *dent,V9fsQID *qidp) -{ -struct stat stbuf; -V9fsPath path; -int err; - -v9fs_path_init (&path); - - err =v9fs_co_name_to_path (pdu, &fidp->path, dent->d_name, &path); -if (err <0 ) { - goto out; - } - err =v9fs_co_lstat (pdu, &path, &stbuf); -if (err <0 ) { - goto out; - } - err =stat_to_qid (pdu, &stbuf, qidp); - -out: -v9fs_path_free (&path); -return err; -} -V9fsPDU *pdu_alloc (V9fsState *s) {V9fsPDU *pdu =NULL ;
@@ -2337,6 +2313,18 @@ size_t v9fs_readdir_response_size(V9fsString *name)return 24 +v9fs_string_size (name); } +static void v9fs_free_dirents (struct V9fsDirEnt *e) +{ +struct V9fsDirEnt *next =NULL ; + +for (; e; e = next) { + next = e->next; +g_free (e->dent); +g_free (e->st); +g_free (e); + } +} + static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp,int32_t maxsize) {
@@ -2345,54 +2333,53 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp,V9fsString name;int len, err =0 ;int32_t count =0 ; -off_t saved_dir_pos;struct dirent *dent; +struct stat *st; +struct V9fsDirEnt *entries =NULL ; -/* save the directory position */ - saved_dir_pos =v9fs_co_telldir (pdu, fidp); -if (saved_dir_pos <0 ) { -return saved_dir_pos; - } - -while (1 ) { -v9fs_readdir_lock (&fidp->fs.dir); +/* +* inode remapping requires the device id, which in turn might be +* different for different directory entries, so if inode remapping is +* enabled we have to make a full stat for each directory entry +*/ +const bool dostat = pdu->s->ctx.export_flags &V9FS_REMAP_INODES ; - err =v9fs_co_readdir (pdu, fidp, &dent); -if (err || !dent) { -break ; - } -v9fs_string_init (&name); -v9fs_string_sprintf (&name,"%s" , dent->d_name); -if ((count +v9fs_readdir_response_size (&name)) > maxsize) { -v9fs_readdir_unlock (&fidp->fs.dir); +/* +* Fetch all required directory entries altogether on a background IO +* thread from fs driver. We don't want to do that for each entry +* individually, because hopping between threads (this main IO thread +* and background IO driver thread) would sum up to huge latencies. +*/ + count =v9fs_co_readdir_many (pdu, fidp, &entries, maxsize, dostat); +if (count <0 ) { + err = count; + count =0 ; + goto out; + } + count =0 ; -/* Ran out of buffer. Set dir back to old position and return */ -v9fs_co_seekdir (pdu, fidp, saved_dir_pos); -v9fs_string_free (&name); -return count; - } +for (struct V9fsDirEnt *e = entries; e; e = e->next) { + dent = e->dent;if (pdu->s->ctx.export_flags &V9FS_REMAP_INODES ) { -/* -* dirent_to_qid() implies expensive stat call for each entry, -* we must do that here though since inode remapping requires -* the device id, which in turn might be different for -* different entries; we cannot make any assumption to avoid -* that here. -*/ - err =dirent_to_qid (pdu, fidp, dent, &qid); + st = e->st; +/* e->st should never be NULL, but just to be sure */ +if (!st) { + err =-1 ; +break ; + } + +/* remap inode */ + err =stat_to_qid (pdu, st, &qid);if (err <0 ) { -v9fs_readdir_unlock (&fidp->fs.dir); -v9fs_co_seekdir (pdu, fidp, saved_dir_pos); -v9fs_string_free (&name); -return err; +break ; } }else {/* * Fill up just the path field of qid because the client uses * only that. To fill the entire qid structure we will have * to stat each dirent found, which is expensive. For the -* latter reason we don't call dirent_to_qid() here. Only drawback +* latter reason we don't call stat_to_qid() here. Only drawback * is that no multi-device export detection of stat_to_qid() * would be done and provided as error to the user here. But * user would get that error anyway when accessing those
@@ -2405,25 +2392,26 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, qid.version =0 ; } +v9fs_string_init (&name); +v9fs_string_sprintf (&name,"%s" , dent->d_name); +/* 11 = 7 + 4 (7 = start offset, 4 = space for storing count) */ len =pdu_marshal (pdu,11 + count,"Qqbs" , &qid, dent->d_off, dent->d_type, &name); -v9fs_readdir_unlock (&fidp->fs.dir); +v9fs_string_free (&name);if (len <0 ) { -v9fs_co_seekdir (pdu, fidp, saved_dir_pos); -v9fs_string_free (&name); -return len; + err = len; +break ; } + count += len; -v9fs_string_free (&name); - saved_dir_pos = dent->d_off; } -v9fs_readdir_unlock (&fidp->fs.dir); - +out: +v9fs_free_dirents (entries);if (err <0 ) {return err; }
Signature · 1 line
2.20.1
PATCH v5 6/6 9pfs: clarify latency of v9fs_co_run_in_worker()
In reply to Christian SchoenebeckAs we just fixed a severe performance issue with Treaddir request handling, clarify this overall issue as a comment on v9fs_co_run_in_worker() with the intention to hopefully prevent such performance mistakes in future (and fixing other yet outstanding ones).
--- hw/9pfs/coth.h | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
hw/9pfs/coth.h +11 -1
@@ -19,7 +19,7 @@#include "qemu/coroutine.h" #include "9p.h" -/* +/** * we want to use bottom half because we want to make sure the below * sequence of events. *
@@ -28,6 +28,16 @@* 3. Enter the coroutine in the worker thread. * we cannot swap step 1 and 2, because that would imply worker thread * can enter coroutine while step1 is still running +* +* @b PERFORMANCE @b CONSIDERATIONS: As a rule of thumb, keep in mind +* that hopping between threads adds @b latency! So when handling a +* 9pfs request, avoid calling v9fs_co_run_in_worker() too often, because +* this might otherwise sum up to a significant, huge overall latency for +* providing the response for just a single request. For that reason it +* is highly recommended to fetch all data from fs driver with a single +* fs driver request on a background I/O thread (bottom half) in one rush +* first and then eventually assembling the final response from that data +* on main I/O thread (top half). */ #define v9fs_co_run_in_worker (code_block) \ do { \
Signature · 1 line
2.20.1