lustre: sync closely to 2.13.52
Subsystems (3)
PATCH 593/622 lnet: lnet response entries leak
In reply to James SimmonsFrom: Alexey Lyashkov <[email protected]>
LNetPut with ACK flag called, but LNetMDUnlink issued before ACK arrives. It can due timeout or it is application call (ldiskfs commit for difficult replies on MDT).
It freed an MD but rsp don't detached, as ACK don't hold an reference to the MD between request sends and ACK arrives.
monitor thread detect it situation and RSP entry moved into the zombie list, which don't freed as no msg processed due MD absence.
Let's remove a response tracking in case nobody want to have reply aka LNetMDUnlink called.
Cray-bug-id: LUS-8188
WC-bug-id: https://jira.whamcloud.com/browse/LU-12991
Lustre-commit: b7035222bd64 ("LU-12991 lnet: lnet response entries leak")
Reviewed-on: https://review.whamcloud.com/36896
--- include/linux/lnet/lib-lnet.h | 2 ++ net/lnet/lnet/lib-md.c | 3 +++ 2 files changed, 5 insertions(+)
include/linux/lnet/lib-lnet.h +2 -0
@@ -157,6 +157,8 @@ static inline int lnet_md_unlinkable(struct lnet_libmd *md) {unsigned int size; +LASSERTF (md->md_rspt_ptr ==NULL ,"md %p rsp %p\n" , md, md->md_rspt_ptr); +if ((md->md_options &LNET_MD_KIOV ) !=0 ) size = offsetof(struct lnet_libmd, md_iov.kiov[md->md_niov]);else
net/lnet/lnet/lib-md.c +3 -0
@@ -548,6 +548,9 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset)lnet_eq_enqueue_event (md->md_eq, &ev); } +if (md->md_rspt_ptr) +lnet_detach_rsp_tracker (md, cpt); +lnet_md_unlink (md);lnet_res_unlock (cpt);
Signature · 1 line
1.8.3.1
PATCH 594/622 lustre: lmv: disable statahead for remote objects
In reply to James SimmonsFrom: Vladimir Saveliev <[email protected]>
Statahead for remote objects is supposed to be disabled by LU-11681 lmv: disable remote file statahead.
However due to typo it is not and statahead for remote objects is accompanied by warnings like:
ll_set_inode()) Can not initialize inode .. without object type.. ll_prep_inode()) new_inode -fatal: rc -12
Fix the typo.
Test to illustrate the issue is added.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13099
Lustre-commit: 68330379b01c ("LU-13099 lmv: disable statahead for remote objects")
Cray-bug-id: LUS-8262
Reviewed-on: https://review.whamcloud.com/37089
--- fs/lustre/lmv/lmv_obd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/lustre/lmv/lmv_obd.c +1 -1
@@ -3369,7 +3369,7 @@ static int lmv_intent_getattr_async(struct obd_export *exp,if (IS_ERR (ptgt))return PTR_ERR (ptgt); - ctgt =lmv_fid2tgt (lmv, &op_data->op_fid1); + ctgt =lmv_fid2tgt (lmv, &op_data->op_fid2);if (IS_ERR (ctgt))return PTR_ERR (ctgt);
Signature · 1 line
1.8.3.1
PATCH 595/622 lustre: llite: eviction during ll_open_cleanup()
In reply to James SimmonsFrom: Andriy Skulysh <[email protected]>
On error ll_open_cleanup() is called while intent lock remains pinned. So eviction can happen while close request waits for a mod rpc slot.
Release intent lock before ll_open_cleanup()
Cray-bug-id: LUS-8055
WC-bug-id: https://jira.whamcloud.com/browse/LU-13101
Lustre-commit: 6d5d7c6bdb4f ("LU-13101 llite: eviction during ll_open_cleanup()")
Reviewed-on: https://review.whamcloud.com/37096
--- fs/lustre/llite/llite_lib.c | 4 +++- fs/lustre/llite/namei.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
fs/lustre/llite/llite_lib.c +3 -1
@@ -2507,8 +2507,10 @@ int ll_prep_inode(struct inode **inode, struct ptlrpc_request *req,/* cleanup will be done if necessary */ md_free_lustre_md (sbi->ll_md_exp, &md); -if (rc !=0 && it && it->it_op &IT_OPEN ) +if (rc !=0 && it && it->it_op &IT_OPEN ) { +ll_intent_drop_lock (it);ll_open_cleanup (sb ? sb : (*inode)->i_sb, req); + }return rc; }
fs/lustre/llite/namei.c +3 -1
@@ -709,8 +709,10 @@ static int ll_lookup_it_finish(struct ptlrpc_request *request, } out: -if (rc !=0 && it->it_op &IT_OPEN ) +if (rc !=0 && it->it_op &IT_OPEN ) { +ll_intent_drop_lock (it);ll_open_cleanup ((*de)->d_sb, request); + }return rc; }
Signature · 1 line
1.8.3.1
PATCH 596/622 lustre: ptlrpc: show target name in req_history
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
Currently the req_history tracing shows the "self" NID as the second field. However, this is not very useful since there may be a number of different targets on the same server, and since the logs are all collected directly on the server we already know the local NID.
Instead of printing the "self" NID, store the target name as the second field, if that is available, so that we can determine which target the RPC was intended for. This makes it easier to debug problems with bad clients and isolate traffic for a specific target.
WC-bug-id: https://jira.whamcloud.com/browse/LU-11644
Lustre-commit: 83b6c6608e94 ("LU-11644 ptlrpc: show target name in req_history")
Reviewed-on: https://review.whamcloud.com/37193
--- fs/lustre/ptlrpc/lproc_ptlrpc.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
fs/lustre/ptlrpc/lproc_ptlrpc.c +7 -3
@@ -956,7 +956,6 @@ static int ptlrpc_lprocfs_svc_req_history_show(struct seq_file *s, void *iter) req = srhi->srhi_req; -libcfs_nid2str_r (req->rq_self, nidstr,sizeof (nidstr)); arrival.tv_sec = req->rq_arrival_time.tv_sec; arrival.tv_nsec = req->rq_arrival_time.tv_nsec; sent.tv_sec = req->rq_sent;
@@ -970,8 +969,13 @@ static int ptlrpc_lprocfs_svc_req_history_show(struct seq_file *s, void *iter)* parser. Currently I only print stuff here I know is OK * to look at coz it was set up in request_in_callback()!!! */ - seq_printf(s, "%lld:%s:%s:x%llu:%d:%s:%lld.%06lld:%lld.%06llds(%+lld.0s) ", - req->rq_history_seq, nidstr, + seq_printf(s, + "%lld:%s:%s:x%llu:%d:%s:%lld.%06lld:%lld.%06llds(%+lld.0s) ", + req->rq_history_seq, + req->rq_export && req->rq_export->exp_obd ? + req->rq_export->exp_obd->obd_name : + libcfs_nid2str_r(req->rq_self, nidstr, + sizeof(nidstr)), libcfs_id2str(req->rq_peer), req->rq_xid, req->rq_reqlen, ptlrpc_rqphase2str(req), (s64)req->rq_arrival_time.tv_sec,
Signature · 1 line
1.8.3.1
PATCH 597/622 lustre: dom: check read-on-open buffer presents in reply
In reply to James SimmonsFrom: Mikhail Pershin <[email protected]>
The ll_dom_finish_open() uses req_capsule_has_field() wronly, it check only format but not buffer presence in reply, that causes unneeded console errors about missing buffer later in req_capsule_server_get()
Patch replaces that with req_capsule_field_present() to check if server pack that field in reply or not and properly skip responses from an old server.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13136
Lustre-commit: 58bea527100b ("LU-13136 dom: check read-on-open buffer presents in reply")
Reviewed-on: https://review.whamcloud.com/37249
--- fs/lustre/include/lustre_req_layout.h | 3 +++ fs/lustre/llite/file.c | 4 ++-- fs/lustre/ptlrpc/layout.c | 7 ++++--- 3 files changed, 9 insertions(+), 5 deletions(-)
fs/lustre/include/lustre_req_layout.h +3 -0
@@ -112,6 +112,9 @@ u32 req_capsule_fmt_size(u32 magic, const struct req_format *fmt,int req_capsule_has_field (const struct req_capsule *pill,const struct req_msg_field *field,enum req_location loc); +int req_capsule_field_present (const struct req_capsule *pill, +const struct req_msg_field *field, +enum req_location loc);void req_capsule_shrink (struct req_capsule *pill,const struct req_msg_field *field,u32 newlen,enum req_location loc);
fs/lustre/llite/file.c +2 -2
@@ -459,8 +459,8 @@ void ll_dom_finish_open(struct inode *inode, struct ptlrpc_request *req,if (!obj)return ; -if (!req_capsule_has_field (&req->rq_pill, &RMF_NIOBUF_INLINE , -RCL_SERVER )) +if (!req_capsule_field_present (&req->rq_pill, &RMF_NIOBUF_INLINE , +RCL_SERVER ))return ; rnb =req_capsule_server_get (&req->rq_pill, &RMF_NIOBUF_INLINE );
fs/lustre/ptlrpc/layout.c +4 -3
@@ -2268,9 +2268,9 @@ int req_capsule_has_field(const struct req_capsule *pill,* Returns a non-zero value if the given @field is present in the given * @pill's PTLRPC request or reply (@loc), else it returns 0. */ -static int req_capsule_field_present (const struct req_capsule *pill, -const struct req_msg_field *field, -enum req_location loc) +int req_capsule_field_present (const struct req_capsule *pill, +const struct req_msg_field *field, +enum req_location loc) {u32 offset;
@@ -2280,6 +2280,7 @@ static int req_capsule_field_present(const struct req_capsule *pill, offset =__req_capsule_offset (pill, field, loc);return lustre_msg_bufcount (__req_msg (pill, loc)) > offset; } +EXPORT_SYMBOL (req_capsule_field_present);/** * This function shrinks the size of the _buffer_ of the @pill's PTLRPC
Signature · 1 line
1.8.3.1
PATCH 598/622 lustre: llite: proper names/types for offset/pages
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
Use loff_t for file offsets and pgoff_t for page index values instead of unsigned long, so that it is possible to distinguish what type of value is being used in the byte-granular readahead code. Otherwise, it is difficult to determine what units "start" or "end" in a given function are in.
Rename variables that reference page index values with an "_idx" suffix to make this clear when reading the code. Similarly, use "bytes" or "pages" for variable names instead of "count" or "len".
Fix stride_page_count() to properly use loff_t for the byte_count, which might otherwise overflow for large strides.
Cast pgoff_t vars to loff_t before PAGE_SIZE shift to avoid overflow. Use shift and mask with PAGE_SIZE and PAGE_MASK instead of mod/div.
Use proper 64-bit division functions for the loff_t types when calculating stride, since they are not guaranteed to be within 4GB.
Remove unused "remainder" argument from ras_align() function.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12518
Lustre-commit: 83d8dd1d7c30 ("LU-12518 llite: proper names/types for offset/pages")
Reviewed-on: https://review.whamcloud.com/37248
--- fs/lustre/include/cl_object.h | 10 +- fs/lustre/llite/file.c | 6 +- fs/lustre/llite/llite_internal.h | 49 +++-- fs/lustre/llite/rw.c | 455 ++++++++++++++++++++------------------- fs/lustre/llite/vvp_internal.h | 4 +- fs/lustre/llite/vvp_io.c | 18 +- fs/lustre/lov/lov_io.c | 21 +- fs/lustre/mdc/mdc_dev.c | 4 +- fs/lustre/obdclass/integrity.c | 2 +- fs/lustre/osc/osc_cache.c | 2 +- fs/lustre/osc/osc_io.c | 8 +- 11 files changed, 294 insertions(+), 285 deletions(-)
fs/lustre/include/cl_object.h +5 -5
@@ -1464,14 +1464,14 @@ struct cl_read_ahead {* This is determined DLM lock coverage, RPC and stripe boundary. * cra_end is included. */ -pgoff_t cra_end; +pgoff_t cra_end_idx;/* optimal RPC size for this read, by pages */ -unsigned long cra_rpc_size; -/* -* Release callback. If readahead holds resources underneath, this +unsigned long cra_rpc_pages; +/* Release callback. If readahead holds resources underneath, this * function should be called to release it. */ -void (*cra_release)(const struct lu_env *env,void *cbdata); +void (*cra_release)(const struct lu_env *env, +void *cbdata);/* Callback data for cra_release routine */ void *cra_cbdata;/* whether lock is in contention */
fs/lustre/llite/file.c +3 -3
@@ -472,7 +472,7 @@ void ll_dom_finish_open(struct inode *inode, struct ptlrpc_request *req,* client PAGE_SIZE to be used on that client, if server's PAGE_SIZE is * smaller then offset may be not aligned and that data is just ignored. */ -if (rnb->rnb_offset %PAGE_SIZE ) +if (rnb->rnb_offset & ~PAGE_MASK )return ;/* Server returns whole file or just file tail if it fills in reply
@@ -492,9 +492,9 @@ void ll_dom_finish_open(struct inode *inode, struct ptlrpc_request *req, data = (char *)rnb +sizeof (*rnb); lnb.lnb_file_offset = rnb->rnb_offset; - start = lnb.lnb_file_offset /PAGE_SIZE ; + start = lnb.lnb_file_offset >>PAGE_SHIFT ; index =0 ; -LASSERT (lnb.lnb_file_offset %PAGE_SIZE ==0 ); +LASSERT ((lnb.lnb_file_offset & ~PAGE_MASK ) ==0 ); lnb.lnb_page_offset =0 ; do { lnb.lnb_data = data + (index << PAGE_SHIFT);
fs/lustre/llite/llite_internal.h +24 -25
114 lines not on this page · raw message
fs/lustre/llite/rw.c +232 -223
921 lines not on this page · raw message
fs/lustre/llite/vvp_internal.h +2 -2
@@ -103,8 +103,8 @@ struct vvp_io {struct kiocb *vui_iocb;/* Readahead state. */ -pgoff_t vui_ra_start; -pgoff_t vui_ra_count; +pgoff_t vui_ra_start_idx; +pgoff_t vui_ra_pages;/* Set when vui_ra_{start,count} have been initialized. */ bool vui_ra_valid; };
fs/lustre/llite/vvp_io.c +9 -9
@@ -739,8 +739,8 @@ static int vvp_io_read_start(const struct lu_env *env,struct file *file = vio->vui_fd->fd_file;int result;loff_t pos = io->u.ci_rd.rd.crw_pos; -long cnt = io->u.ci_rd.rd.crw_count; -long tot = vio->vui_tot_count; +size_t cnt = io->u.ci_rd.rd.crw_count; +size_t tot = vio->vui_tot_count;int exceed =0 ;CLOBINVRNT (env, obj,vvp_object_invariant (obj));
@@ -776,16 +776,16 @@ static int vvp_io_read_start(const struct lu_env *env,/* initialize read-ahead window once per syscall */ if (!vio->vui_ra_valid) { vio->vui_ra_valid = true; - vio->vui_ra_start =cl_index (obj, pos); - vio->vui_ra_count =cl_index (obj, tot +PAGE_SIZE -1 ); + vio->vui_ra_start_idx =cl_index (obj, pos); + vio->vui_ra_pages =cl_index (obj, tot +PAGE_SIZE -1 );/* If both start and end are unaligned, we read one more page * than the index math suggests. */ -if (pos %PAGE_SIZE !=0 && (pos + tot) %PAGE_SIZE !=0 ) - vio->vui_ra_count++; +if ((pos & ~PAGE_MASK ) !=0 && ((pos + tot) & ~PAGE_MASK ) !=0 ) + vio->vui_ra_pages++; -CDEBUG (D_READA ,"tot %ld, ra_start %lu, ra_count %lu\n" , tot, - vio->vui_ra_start, vio->vui_ra_count); +CDEBUG (D_READA ,"tot %zu, ra_start %lu, ra_count %lu\n" , + tot, vio->vui_ra_start_idx, vio->vui_ra_pages); }/* BUG: 5972 */
@@ -1424,7 +1424,7 @@ static int vvp_io_read_ahead(const struct lu_env *env,struct vvp_io *vio =cl2vvp_io (env, ios);if (unlikely (vio->vui_fd->fd_flags &LL_FILE_GROUP_LOCKED )) { - ra->cra_end =CL_PAGE_EOF ; + ra->cra_end_idx =CL_PAGE_EOF ; result =1 ;/* no need to call down */ } }
fs/lustre/lov/lov_io.c +11 -10
@@ -1014,7 +1014,8 @@ static int lov_io_read_ahead(const struct lu_env *env, ra);CDEBUG (D_READA ,DFID " cra_end = %lu, stripes = %d, rc = %d\n" , -PFID (lu_object_fid (lov2lu (loo))), ra->cra_end, r0->lo_nr, rc); +PFID (lu_object_fid (lov2lu (loo))), ra->cra_end_idx, + r0->lo_nr, rc);if (rc)return rc;
@@ -1027,15 +1028,15 @@ static int lov_io_read_ahead(const struct lu_env *env,*/ /* cra_end is stripe level, convert it into file level */ - ra_end = ra->cra_end; + ra_end = ra->cra_end_idx;if (ra_end !=CL_PAGE_EOF ) - ra->cra_end =lov_stripe_pgoff (loo->lo_lsm, index, - ra_end, stripe); + ra->cra_end_idx =lov_stripe_pgoff (loo->lo_lsm, index, + ra_end, stripe);/* boundary of current component */ ra_end =cl_index (obj, (loff_t )lov_io_extent (lio, index)->e_end); -if (ra_end !=CL_PAGE_EOF && ra->cra_end >= ra_end) - ra->cra_end = ra_end -1 ; +if (ra_end !=CL_PAGE_EOF && ra->cra_end_idx >= ra_end) + ra->cra_end_idx = ra_end -1 ;if (r0->lo_nr ==1 )/* single stripe file */ return 0 ;
@@ -1043,13 +1044,13 @@ static int lov_io_read_ahead(const struct lu_env *env, pps =lov_lse (loo, index)->lsme_stripe_size >>PAGE_SHIFT ;CDEBUG (D_READA , -DFID " max_index = %lu, pps = %u, index = %u, stripe_size = %u, stripe no = %u, start index = %lu\n" , -PFID (lu_object_fid (lov2lu (loo))), ra->cra_end, pps, index, +DFID " max_index = %lu, pps = %u, index = %d, stripe_size = %u, stripe no = %u, start index = %lu\n" , +PFID (lu_object_fid (lov2lu (loo))), ra->cra_end_idx, pps, index,lov_lse (loo, index)->lsme_stripe_size, stripe, start);/* never exceed the end of the stripe */ - ra->cra_end =min_t (pgoff_t, - ra->cra_end, start + pps - start % pps -1 ); + ra->cra_end_idx =min_t (pgoff_t, ra->cra_end_idx, + start + pps - start % pps -1 );return 0 ; }
fs/lustre/mdc/mdc_dev.c +2 -2
@@ -1099,8 +1099,8 @@ static int mdc_io_read_ahead(const struct lu_env *env,ldlm_lock_decref (&lockh, dlmlock->l_req_mode); } - ra->cra_rpc_size =osc_cli (osc)->cl_max_pages_per_rpc; - ra->cra_end =CL_PAGE_EOF ; + ra->cra_rpc_pages =osc_cli (osc)->cl_max_pages_per_rpc; + ra->cra_end_idx =CL_PAGE_EOF ; ra->cra_release = osc_read_ahead_release; ra->cra_cbdata = dlmlock;
fs/lustre/obdclass/integrity.c +1 -1
@@ -229,7 +229,7 @@ static void obd_t10_performance_test(const char *obd_name,for (start = jiffies, end = start +HZ /4 , bcount =0 ;time_before (jiffies, end) && rc ==0 ; bcount++) { rc =__obd_t10_performance_test (obd_name, cksum_type, page, - buf_len /PAGE_SIZE ); + buf_len >>PAGE_SHIFT );if (rc)break ; }
fs/lustre/osc/osc_cache.c +1 -1
@@ -1349,7 +1349,7 @@ static int osc_refresh_count(const struct lu_env *env,return 0 ; else if (cl_offset(obj, index + 1) > kms) /* catch sub-page write at end of file */ - return kms % PAGE_SIZE; + return kms & ~PAGE_MASK; else return PAGE_SIZE; }
fs/lustre/osc/osc_io.c +4 -4
@@ -88,12 +88,12 @@ static int osc_io_read_ahead(const struct lu_env *env,ldlm_lock_decref (&lockh, dlmlock->l_req_mode); } - ra->cra_rpc_size =osc_cli (osc)->cl_max_pages_per_rpc; - ra->cra_end =cl_index (osc2cl (osc), - dlmlock->l_policy_data.l_extent.end); + ra->cra_rpc_pages =osc_cli (osc)->cl_max_pages_per_rpc; + ra->cra_end_idx =cl_index (osc2cl (osc), + dlmlock->l_policy_data.l_extent.end); ra->cra_release = osc_read_ahead_release; ra->cra_cbdata = dlmlock; -if (ra->cra_end !=CL_PAGE_EOF ) +if (ra->cra_end_idx !=CL_PAGE_EOF ) ra->cra_contention = true; result =0 ; }
Signature · 1 line
1.8.3.1
PATCH 599/622 lustre: llite: Accept EBUSY for page unaligned read
In reply to James SimmonsFrom: Patrick Farrell <[email protected]>
When doing unaligned strided reads, it's possible for the first and last page of a stride to be read by another thread on the same node, resulting in EBUSY.
Also this could potentially happen for sequential read, for example, several MPI split one large file with unaligned page size, sequential read happen with each MPI program.
We shouldn't stop readahead in these cases.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12518
Lustre-commit: b9c155065d2c ("LU-12518 llite: Accept EBUSY for page unaligned read")
Reviewed-on: https://review.whamcloud.com/35457
--- fs/lustre/llite/rw.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
fs/lustre/llite/rw.c +17 -2
@@ -360,7 +360,8 @@ static bool ras_inside_ra_window(pgoff_t idx, struct ra_io_arg *ria) {struct cl_read_ahead ra = {0 };pgoff_t page_idx; -int count =0 ; +/* busy page count is per stride */ +int count =0 , busy_page_count =0 ;int rc;LASSERT (ria);
@@ -416,8 +417,21 @@ static bool ras_inside_ra_window(pgoff_t idx, struct ra_io_arg *ria)/* If the page is inside the read-ahead window */ rc =ll_read_ahead_page (env, io, queue, page_idx); -if (rc <0 ) +if (rc <0 && rc != -EBUSY )break ; +if (rc == -EBUSY ) { + busy_page_count++; +CDEBUG (D_READA , +"skip busy page: %lu\n" , page_idx); +/* For page unaligned readahead the first +* last pages of each region can be read by +* another reader on the same node, and so +* may be busy. So only stop for > 2 busy +* pages. +*/ +if (busy_page_count >2 ) +break ; + } *ra_end = page_idx;/* Only subtract from reserve & count the page if we
@@ -441,6 +455,7 @@ static bool ras_inside_ra_window(pgoff_t idx, struct ra_io_arg *ria) pos += (ria->ria_length - offset);if ((pos >>PAGE_SHIFT ) >= page_idx +1 ) page_idx = (pos >>PAGE_SHIFT ) -1 ; + busy_page_count =0 ;CDEBUG (D_READA ,"Stride: jump %llu pages to %lu\n" , ria->ria_length - offset, page_idx);
Signature · 1 line
1.8.3.1
PATCH 600/622 lustre: handle: remove locking from class_handle2object()
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
There is limited value in this locking and test on h_in.
If the lookup could have run in parallel with class_handle_unhash_nolock() and seen "h_in == 0", then it could equally well have run moments earlier and not seen it - no locking would prevent that, so the caller much be prepared to have an object returned which has already been unhashed by the time it sees the object.
In other words, any interlock between unhash and lookup must be provided at a higher level than where this code is trying to handle it.
The locking *does* prevent the refcount from being incremented if the object has already been removed from the list. As the final reference is always dropped after that removal, it indirectly stops the refcount from being incremented after the final reference is dropped. This can be more directly achieved by using refcount_inc_not_zero().
So remove the locking, and replace it with refcount_inc_not_zero().
WC-bug-id: https://jira.whamcloud.com/browse/LU-12542
Lustre-commit: e2458a94a6a2 ("LU-12542 handle: remove locking from class_handle2object()")
Reviewed-on: https://review.whamcloud.com/35861
--- fs/lustre/obdclass/lustre_handles.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
fs/lustre/obdclass/lustre_handles.c +1 -4
@@ -149,15 +149,12 @@ void *class_handle2object(u64 cookie, const char *owner)if (h->h_cookie != cookie || h->h_owner != owner)continue ; -spin_lock (&h->h_lock); -if (likely (h->h_in !=0 )) { -refcount_inc (&h->h_ref); +if (refcount_inc_not_zero (&h->h_ref)) {CDEBUG (D_INFO ,"GET %s %p refcount=%d\n" , h->h_owner, h,refcount_read (&h->h_ref)); retval = h; } -spin_unlock (&h->h_lock);break ; }rcu_read_unlock ();
Signature · 1 line
1.8.3.1
PATCH 601/622 lustre: handle: use hlist for hash lists.
In reply to James SimmonsFrom: NeilBrown <[email protected]>
hlist_head/hlist_node is the preferred data structure for hash tables. Not only does it make the 'head' smaller, but is also provides hlist_unhashed() which can be used to check if an object is in the list. This means that we don't need h_in any more.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12542
Lustre-commit: 9c9ea6584cfb ("LU-12542 handle: use hlist for hash lists.")
Reviewed-on: https://review.whamcloud.com/35862
--- fs/lustre/include/lustre_handles.h | 3 +-- fs/lustre/ldlm/ldlm_lock.c | 2 +- fs/lustre/obdclass/genops.c | 2 +- fs/lustre/obdclass/lustre_handles.c | 20 +++++++++----------- 4 files changed, 12 insertions(+), 15 deletions(-)
fs/lustre/include/lustre_handles.h +1 -2
@@ -58,7 +58,7 @@* to compute the start of the structure based on the handle field. */ struct portals_handle { -struct list_head h_link; +struct hlist_node h_link;u64 h_cookie;const char *h_owner;refcount_t h_ref;
@@ -66,7 +66,6 @@ struct portals_handle {/* newly added fields to handle the RCU issue. -jxiong */ struct rcu_head h_rcu;spinlock_t h_lock; - unsigned int h_in:1; };/* handles.c */
fs/lustre/ldlm/ldlm_lock.c +1 -1
@@ -404,7 +404,7 @@ static struct ldlm_lock *ldlm_lock_new(struct ldlm_resource *resource)lprocfs_counter_incr (ldlm_res_to_ns (resource)->ns_stats,LDLM_NSS_LOCKS ); -INIT_LIST_HEAD (&lock->l_handle.h_link); +INIT_HLIST_NODE (&lock->l_handle.h_link);class_handle_hash (&lock->l_handle, lock_handle_owner);lu_ref_init (&lock->l_reference);
fs/lustre/obdclass/genops.c +1 -1
@@ -813,7 +813,7 @@ static struct obd_export *__class_new_export(struct obd_device *obd,spin_lock_init (&export->exp_uncommitted_replies_lock);INIT_LIST_HEAD (&export->exp_uncommitted_replies);INIT_LIST_HEAD (&export->exp_req_replay_queue); -INIT_LIST_HEAD_RCU (&export->exp_handle.h_link); +INIT_HLIST_NODE (&export->exp_handle.h_link);INIT_LIST_HEAD (&export->exp_hp_rpcs);class_handle_hash (&export->exp_handle, export_handle_owner);spin_lock_init (&export->exp_lock);
fs/lustre/obdclass/lustre_handles.c +9 -11
@@ -48,7 +48,7 @@static struct handle_bucket {spinlock_t lock; -struct list_head head; +struct hlist_head head; } *handle_hash;#define HANDLE_HASH_SIZE (1 << 16)
@@ -63,7 +63,7 @@ void class_handle_hash(struct portals_handle *h, const char *owner)struct handle_bucket *bucket;LASSERT (h); -LASSERT (list_empty (&h->h_link)); +LASSERT (hlist_unhashed (&h->h_link));/* * This is fast, but simplistic cookie generation algorithm, it will
@@ -89,8 +89,7 @@ void class_handle_hash(struct portals_handle *h, const char *owner) bucket = &handle_hash[h->h_cookie &HANDLE_HASH_MASK ];spin_lock (&bucket->lock); -list_add_rcu (&h->h_link, &bucket->head); - h->h_in =1 ; +hlist_add_head_rcu (&h->h_link, &bucket->head);spin_unlock (&bucket->lock); CDEBUG(D_INFO, "added object %p with handle %#llx to hash\n",
@@ -100,7 +99,7 @@ void class_handle_hash(struct portals_handle *h, const char *owner)static void class_handle_unhash_nolock (struct portals_handle *h) { -if (list_empty (&h->h_link)) { +if (hlist_unhashed (&h->h_link)) {CERROR ("removing an already-removed handle (%#llx)\n" , h->h_cookie);return ;
@@ -110,13 +109,12 @@ static void class_handle_unhash_nolock(struct portals_handle *h) h, h->h_cookie);spin_lock (&h->h_lock); -if (h->h_in ==0 ) { +if (hlist_unhashed (&h->h_link)) {spin_unlock (&h->h_lock);return ; } - h->h_in =0 ; +hlist_del_init_rcu (&h->h_link);spin_unlock (&h->h_lock); -list_del_rcu (&h->h_link); }void class_handle_unhash (struct portals_handle *h)
@@ -145,7 +143,7 @@ void *class_handle2object(u64 cookie, const char *owner) bucket = handle_hash + (cookie &HANDLE_HASH_MASK );rcu_read_lock (); -list_for_each_entry_rcu (h, &bucket->head, h_link) { +hlist_for_each_entry_rcu (h, &bucket->head, h_link) {if (h->h_cookie != cookie || h->h_owner != owner)continue ;
@@ -177,7 +175,7 @@ int class_handle_init(void)spin_lock_init (&handle_base_lock);for (bucket = handle_hash +HANDLE_HASH_SIZE -1 ; bucket >= handle_hash; bucket--) { -INIT_LIST_HEAD (&bucket->head); +INIT_HLIST_HEAD (&bucket->head);spin_lock_init (&bucket->lock); }
@@ -196,7 +194,7 @@ static int cleanup_all_handles(void)struct portals_handle *h;spin_lock (&handle_hash[i].lock); -list_for_each_entry_rcu (h, &handle_hash[i].head, h_link) { +hlist_for_each_entry_rcu (h, &handle_hash[i].head, h_link) {CERROR ("force clean handle %#llx addr %p owner %p\n" , h->h_cookie, h, h->h_owner);
Signature · 1 line
1.8.3.1
PATCH 602/622 lustre: obdclass: convert waiting in cl_sync_io_wait().
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
This function will *always* wait until ->csi_sync_nr reaches zero. The effect of the timeout is:
1/ to report an error if the count doesn't reach zero in the given
time
2/ to return -ETIMEDOUt instead of csi_sync_rc if the timeout was
exceeded.
So we rearrange the code to make that more obvious.
A small exrta change is that we now call wait_event_idle() again even if there was a timeout and the first wait succeeded. This will simply test csi_sync_nr again and not actually wait. We could protected it with 'rc != 0 || timeout == 0' but there seems no point.
WC-bug-id: https://jira.whamcloud.com/browse/LU-10467
Lustre-commit: d6ce546eb7e2 ("LU-10467 obdclass: convert waiting in cl_sync_io_wait().")
Reviewed-on: https://review.whamcloud.com/36102
--- fs/lustre/obdclass/cl_io.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)
fs/lustre/obdclass/cl_io.c +10 -13
@@ -1054,27 +1054,24 @@ void cl_sync_io_init_notify(struct cl_sync_io *anchor, int nr,int cl_sync_io_wait (const struct lu_env *env,struct cl_sync_io *anchor,long timeout) { -int rc =1 ; +int rc =0 ;LASSERT (timeout >=0 ); -if (timeout ==0 ) -wait_event_idle (anchor->csi_waitq, -atomic_read (&anchor->csi_sync_nr) ==0 ); -else - rc =wait_event_idle_timeout (anchor->csi_waitq, -atomic_read (&anchor->csi_sync_nr) ==0 , - timeout *HZ ); -if (rc ==0 ) { +if (timeout >0 && +wait_event_idle_timeout (anchor->csi_waitq, +atomic_read (&anchor->csi_sync_nr) ==0 , + timeout *HZ ) ==0 ) { rc = -ETIMEDOUT ;CERROR ("IO failed: %d, still wait for %d remaining entries\n" , rc,atomic_read (&anchor->csi_sync_nr)); + } -wait_event_idle (anchor->csi_waitq, -atomic_read (&anchor->csi_sync_nr) ==0 ); - }else { +wait_event_idle (anchor->csi_waitq, +atomic_read (&anchor->csi_sync_nr) ==0 ); +if (!rc) rc = anchor->csi_sync_rc; - } +/* We take the lock to ensure that cl_sync_io_note() has finished */ spin_lock (&anchor->csi_waitq.lock);LASSERT (atomic_read (&anchor->csi_sync_nr) ==0 );
Signature · 1 line
1.8.3.1
PATCH 603/622 lnet: modules: use list_move were appropriate.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
Rather than
list_del(&foo);
list_add(&foo, &bar);
use
list_move(&foo, &bar);
Similarly for list_add_tail and list_move_tail.
In lnet_attach_rsp_tracker, local_rspt already has a suitably initialised ->rspt_on_list, so the new_entry variable can be discarded.
WC-bug-id: https://jira.whamcloud.com/browse/LU-9679
Lustre-commit: 7525fd36a266 ("LU-9679 modules: use list_move were appropriate.")
Reviewed-on: https://review.whamcloud.com/36670
--- net/lnet/lnet/lib-move.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
net/lnet/lnet/lib-move.c +1 -5
@@ -4374,7 +4374,6 @@ void lnet_monitor_thr_stop(void) struct lnet_libmd *md, struct lnet_handle_md mdh) {s64 timeout_ns; -bool new_entry = true;struct lnet_rsp_tracker *local_rspt;/* MD has a refcount taken by message so it's not going away.
@@ -4391,7 +4390,6 @@ void lnet_monitor_thr_stop(void)* update the deadline on that one. */ lnet_rspt_free (rspt, cpt); - new_entry = false; }else {/* new md */ rspt->rspt_mdh = mdh;
@@ -4406,9 +4404,7 @@ void lnet_monitor_thr_stop(void)* list in order to expire all the older entries first. */ lnet_net_lock (cpt); -if (!new_entry && !list_empty (&local_rspt->rspt_on_list)) -list_del_init (&local_rspt->rspt_on_list); -list_add_tail (&local_rspt->rspt_on_list, the_lnet.ln_mt_rstq[cpt]); +list_move_tail (&local_rspt->rspt_on_list, the_lnet.ln_mt_rstq[cpt]);lnet_net_unlock (cpt);lnet_res_unlock (cpt); }
Signature · 1 line
1.8.3.1
PATCH 604/622 lnet: fix small race in unloading klnd modules.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
Reference counting of klnd modules is handled by the module itself. Currently, it is possible for a module to be completely unloaded between the time when the module called module_put(), and when it subsequently returns from the function that makes that call. During this time there may be one or two instructions to execute, and if the module is unmapped before they are executed, an exception will result.
The module unload will call lnet_unregister_lnd() which takes the_lnet.ln_lnd_mutex, so module unload cannot complete while that is held. lnd_startup is called with this mutex held to avoid any races, but lnd_shutdown is not. Adding that protection will close the race.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12678
Lustre-commit: c087091cd901 ("LU-12678 lnet: fix small race in unloading klnd modules.")
Reviewed-on: https://review.whamcloud.com/36853
--- net/lnet/lnet/api-ni.c | 7 +++++++ 1 file changed, 7 insertions(+)
net/lnet/lnet/api-ni.c +7 -0
@@ -1983,7 +1983,14 @@ static void lnet_push_target_fini(void) islo = ni->ni_net->net_lnd->lnd_type ==LOLND ;LASSERT (!in_interrupt ()); +/* Holding the mutex makes it safe for lnd_shutdown +* to call module_put(). Module unload cannot finish +* until lnet_unregister_lnd() completes, and that +* requires the mutex. +*/ +mutex_lock (&the_lnet.ln_lnd_mutex); net->net_lnd->lnd_shutdown (ni); +mutex_unlock (&the_lnet.ln_lnd_mutex);if (!islo) CDEBUG(D_LNI, "Removed LNI %s\n",
Signature · 1 line
1.8.3.1
PATCH 605/622 lnet: me: discard struct lnet_handle_me
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
The Portals API uses a cookie 'handle' to identify an ME. This is appropriate for a user-space API for objects maintained by the kernel, but it brings no value when the API client and implementation are both in the kernel, as is the case with Lustre and LNet.
Instead of using a 'handle', a pointer to the 'struct lnet_me' can be used. This object is not reference counted and is always freed correctly, so there can be no case where the cookie becomes invalid while it is still held - as can be seen by the fact that the return value from LNetMEUnlink() is never used except to assert that it is zero.
So use 'struct lnet_me *' directly instead of having indirection through a 'struct lnet_handle_me'.
Also:
- change LNetMEUnlink() to return void as it cannot fail now.
- have LNetMEAttach() return the pointer, using ERR_PTR() to return errors.
- discard ln_me_containers and don't store the me there-in.
- store an explicit 'cpt' in each me, we no longer store one implicitly via the cookie.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12678
Lustre-commit: ceeeae4271fd ("LU-12678 lnet: me: discard struct lnet_handle_me")
Reviewed-on: https://review.whamcloud.com/36859
--- fs/lustre/ptlrpc/niobuf.c | 45 ++++++++++++++++----------------- include/linux/lnet/api.h | 20 +++++++-------- include/linux/lnet/lib-lnet.h | 22 ----------------- include/linux/lnet/lib-types.h | 4 +-- include/uapi/linux/lnet/lnet-types.h | 4 --- net/lnet/lnet/api-ni.c | 46 ++++++++++++---------------------- net/lnet/lnet/lib-md.c | 16 +++++------- net/lnet/lnet/lib-me.c | 48 +++++++++++------------------------- net/lnet/selftest/rpc.c | 14 +++++------ 9 files changed, 76 insertions(+), 143 deletions(-)
fs/lustre/ptlrpc/niobuf.c +21 -24
@@ -118,11 +118,10 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req)struct ptlrpc_bulk_desc *desc = req->rq_bulk;struct lnet_process_id peer;int rc =0 ; -int rc2;int posted_md;int total_md;u64 mbits; -struct lnet_handle_me me_h; +struct lnet_me *me;struct lnet_md md;if (OBD_FAIL_CHECK (OBD_FAIL_PTLRPC_BULK_GET_NET ))
@@ -183,8 +182,9 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_BULK_ATTACH)) { rc = -ENOMEM; } else { - rc =LNetMEAttach (desc->bd_portal, peer, mbits,0 , -LNET_UNLINK ,LNET_INS_AFTER , &me_h); + me =LNetMEAttach (desc->bd_portal, peer, mbits,0 , +LNET_UNLINK ,LNET_INS_AFTER ); + rc =PTR_ERR_OR_ZERO (me); }if (rc !=0 ) { CERROR("%s: LNetMEAttach failed x%llu/%d: rc = %d\n",
@@ -194,14 +194,13 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) }/* About to let the network at it... */ - rc =LNetMDAttach (me_h, md,LNET_UNLINK , + rc =LNetMDAttach (me, md,LNET_UNLINK , &desc->bd_mds[posted_md]);if (rc !=0 ) {CERROR ("%s: LNetMDAttach failed x%llu/%d: rc = %d\n" , desc->bd_import->imp_obd->obd_name, mbits, posted_md, rc); - rc2 =LNetMEUnlink (me_h); -LASSERT (rc2 ==0 ); +LNetMEUnlink (me);break ; } }
@@ -479,11 +478,10 @@ int ptlrpc_error(struct ptlrpc_request *req)int ptl_send_rpc (struct ptlrpc_request *request,int noreply) {int rc; -int rc2;unsigned int mpflag =0 ;struct lnet_handle_md bulk_cookie;struct ptlrpc_connection *connection; -struct lnet_handle_me reply_me_h; +struct lnet_me *reply_me;struct lnet_md reply_md;struct obd_import *imp = request->rq_import;struct obd_device *obd = imp->imp_obd;
@@ -611,10 +609,11 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) request->rq_repmsg =NULL ; } - rc =LNetMEAttach (request->rq_reply_portal,/*XXX FIXME bug 249*/ - connection->c_peer, request->rq_xid,0 , -LNET_UNLINK ,LNET_INS_AFTER , &reply_me_h); -if (rc !=0 ) { + reply_me =LNetMEAttach (request->rq_reply_portal, + connection->c_peer, request->rq_xid,0 , +LNET_UNLINK ,LNET_INS_AFTER ); +if (IS_ERR (reply_me)) { + rc =PTR_ERR (reply_me);CERROR ("LNetMEAttach failed: %d\n" , rc);LASSERT (rc == -ENOMEM ); rc = -ENOMEM ;
@@ -652,7 +651,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply)/* We must see the unlink callback to set rq_reply_unlinked, * so we can't auto-unlink */ - rc =LNetMDAttach (reply_me_h, reply_md,LNET_RETAIN , + rc =LNetMDAttach (reply_me, reply_md,LNET_RETAIN , &request->rq_reply_md_h);if (rc !=0 ) {CERROR ("LNetMDAttach failed: %d\n" , rc);
@@ -710,8 +709,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply)* nobody apart from the PUT's target has the right nid+XID to * access the reply buffer. */ - rc2 =LNetMEUnlink (reply_me_h); -LASSERT (rc2 ==0 ); +LNetMEUnlink (reply_me);/* UNLINKED callback called synchronously */ LASSERT (!request->rq_receiving_reply);
@@ -750,7 +748,7 @@ int ptlrpc_register_rqbd(struct ptlrpc_request_buffer_desc *rqbd) };int rc;struct lnet_md md; -struct lnet_handle_me me_h; +struct lnet_me *me;CDEBUG (D_NET ,"LNetMEAttach: portal %d\n" , service->srv_req_portal);
@@ -762,12 +760,12 @@ int ptlrpc_register_rqbd(struct ptlrpc_request_buffer_desc *rqbd)* which means buffer can only be attached on local CPT, and LND * threads can find it by grabbing a local lock */ - rc =LNetMEAttach (service->srv_req_portal, + me =LNetMEAttach (service->srv_req_portal, match_id,0 , ~0 ,LNET_UNLINK , rqbd->rqbd_svcpt->scp_cpt >=0 ? -LNET_INS_LOCAL :LNET_INS_AFTER , &me_h); -if (rc !=0 ) { -CERROR ("LNetMEAttach failed: %d\n" , rc); +LNET_INS_LOCAL :LNET_INS_AFTER ); +if (IS_ERR (me)) { +CERROR ("LNetMEAttach failed: %ld\n" ,PTR_ERR (me));return -ENOMEM ; }
@@ -782,14 +780,13 @@ int ptlrpc_register_rqbd(struct ptlrpc_request_buffer_desc *rqbd) md.user_ptr = &rqbd->rqbd_cbid; md.eq_handle = ptlrpc_eq_h; - rc =LNetMDAttach (me_h, md,LNET_UNLINK , &rqbd->rqbd_md_h); + rc =LNetMDAttach (me, md,LNET_UNLINK , &rqbd->rqbd_md_h);if (rc ==0 )return 0 ;CERROR ("LNetMDAttach failed: %d;\n" , rc);LASSERT (rc == -ENOMEM ); - rc =LNetMEUnlink (me_h); -LASSERT (rc ==0 ); +LNetMEUnlink (me); rqbd->rqbd_refcount =0 ;return -ENOMEM ;
include/linux/lnet/api.h +10 -10
@@ -94,15 +94,15 @@* and removed from its list by LNetMEUnlink(). * @{ */ -int LNetMEAttach (unsigned int portal, -struct lnet_process_id match_id_in, -u64 match_bits_in, -u64 ignore_bits_in, -enum lnet_unlink unlink_in, -enum lnet_ins_pos pos_in, -struct lnet_handle_me *handle_out); - -int LNetMEUnlink (struct lnet_handle_me current_in); +struct lnet_me * +LNetMEAttach (unsigned int portal, +struct lnet_process_id match_id_in, +u64 match_bits_in, +u64 ignore_bits_in, +enum lnet_unlink unlink_in, +enum lnet_ins_pos pos_in); + +void LNetMEUnlink (struct lnet_me *current_in);/** @} lnet_me */ /** \defgroup lnet_md Memory descriptors
@@ -118,7 +118,7 @@ int LNetMEAttach(unsigned int portal,* associated with a MD: LNetMDUnlink(). * @{ */ -int LNetMDAttach (struct lnet_handle_me current_in, +int LNetMDAttach (struct lnet_me *current_in,struct lnet_md md_in,enum lnet_unlink unlink_in,struct lnet_handle_md *md_handle_out);
include/linux/lnet/lib-lnet.h +0 -22
@@ -326,28 +326,6 @@ void lnet_res_lh_initialize(struct lnet_res_container *rec, }static inline void -lnet_me2handle (struct lnet_handle_me *handle,struct lnet_me *me) -{ - handle->cookie = me->me_lh.lh_cookie; -} - -static inline struct lnet_me * -lnet_handle2me (struct lnet_handle_me *handle) -{ -/* ALWAYS called with resource lock held */ -struct lnet_libhandle *lh; -int cpt; - - cpt =lnet_cpt_of_cookie (handle->cookie); - lh =lnet_res_lh_lookup (the_lnet.ln_me_containers[cpt], - handle->cookie); -if (!lh) -return NULL ; - - return lh_entry(lh, struct lnet_me, me_lh); -} - -static inline void lnet_peer_net_addref_locked (struct lnet_peer_net *lpn) {atomic_inc (&lpn->lpn_refcount);
include/linux/lnet/lib-types.h +1 -3
@@ -192,7 +192,7 @@ struct lnet_eq {struct lnet_me {struct list_head me_list; -struct lnet_libhandle me_lh; +int me_cpt;struct lnet_process_id me_match_id;unsigned int me_portal;unsigned int me_pos;/* hash offset in mt_hash */
@@ -1027,8 +1027,6 @@ struct lnet {int ln_nportals;/* the vector of portals */ struct lnet_portal **ln_portals; -/* percpt ME containers */ -struct lnet_res_container **ln_me_containers;/* percpt MD container */ struct lnet_res_container **ln_md_containers;
include/uapi/linux/lnet/lnet-types.h +0 -4
@@ -374,10 +374,6 @@ static inline int LNetMDHandleIsInvalid(struct lnet_handle_md h)return (LNET_WIRE_HANDLE_COOKIE_NONE == h.cookie); } -struct lnet_handle_me { -u64 cookie; -}; -/** * Global process ID. */
net/lnet/lnet/api-ni.c +16 -30
110 lines not on this page · raw message
net/lnet/lnet/lib-md.c +6 -10
@@ -337,7 +337,7 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset)/** * Create a memory descriptor and attach it to a ME * -* @meh A handle for a ME to associate the new MD with. +* @me An ME to associate the new MD with. * @umd Provides initial values for the user-visible parts of a MD. * Other than its use for initialization, there is no linkage * between this structure and the MD maintained by the LNet.
@@ -354,19 +354,18 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset)* Return: 0 on success. * -EINVAL If @umd is not valid. * -ENOMEM If new MD cannot be allocated. -* -ENOENT Either @meh or @umd.eq_handle does not point to a +* -ENOENT Either @me or @umd.eq_handle does not point to a * valid object. Note that it's OK to supply a NULL @umd.eq_handle * by calling LNetInvalidateHandle() on it. -* -EBUSY if the ME pointed to by @meh is already associated with +* -EBUSY if the ME pointed to by @me is already associated with * a MD. */ int -LNetMDAttach (struct lnet_handle_me meh,struct lnet_md umd, +LNetMDAttach (struct lnet_me *me,struct lnet_md umd,enum lnet_unlink unlink,struct lnet_handle_md *handle) {LIST_HEAD (matches);LIST_HEAD (drops); -struct lnet_me *me;struct lnet_libmd *md;int cpt;int rc;
@@ -389,14 +388,11 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset)if (rc) goto out_free; - cpt =lnet_cpt_of_cookie (meh.cookie); + cpt = me->me_cpt;lnet_res_lock (cpt); - me =lnet_handle2me (&meh); -if (!me) - rc = -ENOENT ; -else if (me->me_md) +if (me->me_md) rc = -EBUSY ;else rc =lnet_md_link (md, umd.eq_handle, cpt);
net/lnet/lnet/lib-me.c +15 -33
114 lines not on this page · raw message
net/lnet/selftest/rpc.c +7 -7
@@ -360,11 +360,12 @@ struct srpc_bulk * {int rc;struct lnet_md md; -struct lnet_handle_me meh; +struct lnet_me *me; - rc =LNetMEAttach (portal, peer, matchbits,0 ,LNET_UNLINK , - local ?LNET_INS_LOCAL :LNET_INS_AFTER , &meh); -if (rc) { + me =LNetMEAttach (portal, peer, matchbits,0 ,LNET_UNLINK , + local ?LNET_INS_LOCAL :LNET_INS_AFTER ); +if (IS_ERR (me)) { + rc =PTR_ERR (me);CERROR ("LNetMEAttach failed: %d\n" , rc);LASSERT (rc == -ENOMEM );return -ENOMEM ;
@@ -377,13 +378,12 @@ struct srpc_bulk * md.options = options; md.eq_handle = srpc_data.rpc_lnet_eq; - rc =LNetMDAttach (meh, md,LNET_UNLINK , mdh); + rc =LNetMDAttach (me, md,LNET_UNLINK , mdh);if (rc) {CERROR ("LNetMDAttach failed: %d\n" , rc);LASSERT (rc == -ENOMEM ); - rc =LNetMEUnlink (meh); -LASSERT (!rc); +LNetMEUnlink (me);return -ENOMEM ; }
Signature · 1 line
1.8.3.1
PATCH 606/622 lnet: avoid extra memory consumption
In reply to James SimmonsFrom: Alexey Lyashkov <[email protected]>
use slab allocation for the rsp_tracker and lnet_message structs to avoid memory fragmnetation.
Cray-bug-id: LUS-8190
WC-bug-id: https://jira.whamcloud.com/browse/LU-13036
Lustre-commit: a3ce59ae2c62 ("LU-13036 lnet: avoid extra memory consumption")
Reviewed-on: https://review.whamcloud.com/36897
--- include/linux/lnet/lib-lnet.h | 13 ++++++++++--- net/lnet/lnet/api-ni.c | 28 ++++++++++++++++++++++++---- net/lnet/lnet/lib-move.c | 11 ++++++----- 3 files changed, 40 insertions(+), 12 deletions(-)
include/linux/lnet/lib-lnet.h +10 -3
@@ -83,13 +83,17 @@/* default timeout */ #define DEFAULT_PEER_TIMEOUT 180 +#define LNET_LND_DEFAULT_TIMEOUT 5 + +bool lnet_is_route_alive (struct lnet_route *route);#define LNET_SMALL_MD_SIZE offsetof(struct lnet_libmd, md_iov.iov[1])extern struct kmem_cache *lnet_mes_cachep;/* MEs kmem_cache */ extern struct kmem_cache *lnet_small_mds_cachep;/* <= LNET_SMALL_MD_SIZE bytes * MDs kmem_cache */ -#define LNET_LND_DEFAULT_TIMEOUT 5 +extern struct kmem_cache *lnet_rspt_cachep; +extern struct kmem_cache *lnet_msg_cachep;bool lnet_is_route_alive (struct lnet_route *route);bool lnet_is_gateway_alive (struct lnet_peer *gw);
@@ -417,19 +421,22 @@ void lnet_res_lh_initialize(struct lnet_res_container *rec, {struct lnet_rsp_tracker *rspt; - rspt =kzalloc (sizeof (*rspt),GFP_NOFS ); + rspt =kmem_cache_zalloc (lnet_rspt_cachep,GFP_NOFS );if (rspt) {lnet_net_lock (cpt); the_lnet.ln_counters[cpt]->lct_health.lch_rst_alloc++;lnet_net_unlock (cpt); } +CDEBUG (D_MALLOC ,"rspt alloc %p\n" , rspt);return rspt; }static inline void lnet_rspt_free (struct lnet_rsp_tracker *rspt,int cpt) { -kfree (rspt); +CDEBUG (D_MALLOC ,"rspt free %p\n" , rspt); + +kmem_cache_free (lnet_rspt_cachep, rspt);lnet_net_lock (cpt); the_lnet.ln_counters[cpt]->lct_health.lch_rst_alloc--;lnet_net_unlock (cpt);
net/lnet/lnet/api-ni.c +24 -4
@@ -494,9 +494,11 @@ static int lnet_discover(struct lnet_process_id id, u32 force,struct kmem_cache *lnet_small_mds_cachep;/* <= LNET_SMALL_MD_SIZE bytes * MDs kmem_cache */ +struct kmem_cache *lnet_rspt_cachep;/* response tracker cache */ +struct kmem_cache *lnet_msg_cachep;static int -lnet_descriptor_setup (void ) +lnet_slab_setup (void ) {/* create specific kmem_cache for MEs and small MDs (i.e., originally * allocated in <size-xxx> kmem_cache).
@@ -512,12 +514,30 @@ static int lnet_discover(struct lnet_process_id id, u32 force,if (!lnet_small_mds_cachep)return -ENOMEM ; + lnet_rspt_cachep =kmem_cache_create ("lnet_rspt" , +sizeof (struct lnet_rsp_tracker ), +0 ,0 ,NULL ); +if (!lnet_rspt_cachep) +return -ENOMEM ; + + lnet_msg_cachep =kmem_cache_create ("lnet_msg" , +sizeof (struct lnet_msg ), +0 ,0 ,NULL ); +if (!lnet_msg_cachep) +return -ENOMEM ; +return 0 ; }static void -lnet_descriptor_cleanup (void ) +lnet_slab_cleanup (void ) { +kmem_cache_destroy (lnet_msg_cachep); + lnet_msg_cachep =NULL ; + +kmem_cache_destroy (lnet_rspt_cachep); + lnet_rspt_cachep =NULL ; +kmem_cache_destroy (lnet_small_mds_cachep); lnet_small_mds_cachep =NULL ;
@@ -1081,7 +1101,7 @@ struct list_head **LNetInvalidateEQHandle (&the_lnet.ln_mt_eqh);init_completion (&the_lnet.ln_started); - rc =lnet_descriptor_setup (); + rc =lnet_slab_setup ();if (rc !=0 ) goto failed;
@@ -1188,7 +1208,7 @@ struct list_head ** the_lnet.ln_counters =NULL ; }lnet_destroy_remote_nets_table (); -lnet_descriptor_cleanup (); +lnet_slab_cleanup ();return 0 ; }
net/lnet/lnet/lib-move.c +6 -5
@@ -4186,7 +4186,7 @@ void lnet_monitor_thr_stop(void) } } - msg =kzalloc (sizeof (*msg),GFP_NOFS ); + msg =kmem_cache_zalloc (lnet_msg_cachep,GFP_NOFS );if (!msg) { CERROR("%s, src %s: Dropping %s (out of memory)\n", libcfs_nid2str(from_nid), libcfs_nid2str(src_nid),
@@ -4194,7 +4194,7 @@ void lnet_monitor_thr_stop(void) goto drop; } -/* msg zeroed by kzalloc() +/* msg zeroed by kmem_cache_zalloc(). * i.e. flags all clear, pointers NULL etc */ msg->msg_type = type;
@@ -4475,7 +4475,7 @@ void lnet_monitor_thr_stop(void)return -EIO ; } - msg =kzalloc (sizeof (*msg),GFP_NOFS ); + msg =kmem_cache_zalloc (lnet_msg_cachep,GFP_NOFS );if (!msg) {CERROR ("Dropping PUT to %s: ENOMEM on struct lnet_msg\n" ,libcfs_id2str (target));
@@ -4571,7 +4571,7 @@ struct lnet_msg ** CAVEAT EMPTOR: 'getmsg' is the original GET, which is freed when * lnet_finalize() is called on it, so the LND must call this first */ -struct lnet_msg *msg =kzalloc (sizeof (*msg),GFP_NOFS ); +struct lnet_msg *msg;struct lnet_libmd *getmd = getmsg->msg_md;struct lnet_process_id peer_id = getmsg->msg_target;int cpt;
@@ -4579,6 +4579,7 @@ struct lnet_msg *LASSERT (!getmsg->msg_target_is_router);LASSERT (!getmsg->msg_routing); + msg =kmem_cache_zalloc (lnet_msg_cachep,GFP_NOFS );if (!msg) {CERROR ("%s: Dropping REPLY from %s: can't allocate msg\n" ,libcfs_nid2str (ni->ni_nid),libcfs_id2str (peer_id));
@@ -4708,7 +4709,7 @@ struct lnet_msg *return -EIO ; } - msg =kzalloc (sizeof (*msg),GFP_NOFS ); + msg =kmem_cache_zalloc (lnet_msg_cachep,GFP_NOFS );if (!msg) {CERROR ("Dropping GET to %s: ENOMEM on struct lnet_msg\n" ,libcfs_id2str (target));
Signature · 1 line
1.8.3.1
PATCH 607/622 lustre: uapi: remove unused LUSTRE_DIRECTIO_FL
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
The LUSTRE_DIRECTIO_FL was added based on the upstream FS_DIRECTIO_FL flag in the hopes that it might be useful, but it has since been removed from the upstream in kernel commit v4.4-rc4-22-g68ce7bfcd995 and replaced by FS_VERITY_FL using the same value in kernel commit v5.3-rc2-4-gfe9918d3b228, which we are much more likely to use.
Since LUSTRE_DIRECTIO_FL was unused, there is no risk to remove it.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13164
Lustre-commit: ff168481a1b2 ("LU-13164 uapi: remove unused LUSTRE_DIRECTIO_FL")
Reviewed-on: https://review.whamcloud.com/37295
--- fs/lustre/ptlrpc/wiretest.c | 2 -- include/uapi/linux/lustre/lustre_idl.h | 1 - 2 files changed, 3 deletions(-)
fs/lustre/ptlrpc/wiretest.c +0 -2
@@ -2176,8 +2176,6 @@ void lustre_assert_wire_constants(void) LUSTRE_DIRSYNC_FL);LASSERTF (LUSTRE_TOPDIR_FL ==0x00020000 ,"found 0x%.8x\n" ,LUSTRE_TOPDIR_FL ); -LASSERTF (LUSTRE_DIRECTIO_FL ==0x00100000 ,"found 0x%.8x\n" , -LUSTRE_DIRECTIO_FL );LASSERTF (LUSTRE_INLINE_DATA_FL ==0x10000000 ,"found 0x%.8x\n" ,LUSTRE_INLINE_DATA_FL ); LASSERTF(MDS_INODELOCK_LOOKUP == 0x00000001UL, "found 0x%.8x\n",
include/uapi/linux/lustre/lustre_idl.h +0 -1
@@ -1543,7 +1543,6 @@ enum {#define LUSTRE_INDEX_FL 0x00001000/* hash-indexed directory */ #define LUSTRE_DIRSYNC_FL 0x00010000/* dirsync behaviour (dir only) */ #define LUSTRE_TOPDIR_FL 0x00020000/* Top of directory hierarchies*/ -#define LUSTRE_DIRECTIO_FL 0x00100000/* Use direct i/o */ #define LUSTRE_INLINE_DATA_FL 0x10000000/* Inode has inline data. */ #define LUSTRE_PROJINHERIT_FL 0x20000000/* Create with parents projid */
Signature · 1 line
1.8.3.1
PATCH 608/622 lustre: lustre: Reserve OST_FALLOCATE(fallocate) opcode
In reply to James SimmonsFrom: Swapnil Pimpale <[email protected]>
A new RPC, OST_FALLOCATE has been added for space preallocation. This patch reserves OST_FALLOCATE opcode for fallocate syscall. Reserving opcode upfront would ensure consistency and would avoid protocol interoperability issues in the future.
WC-bug-id: https://jira.whamcloud.com/browse/LU-3606
Lustre-commit: 46a11df089c9 ("LU-3606 lustre: Reserve OST_FALLOCATE(fallocate) opcode")
Reviewed-on: https://review.whamcloud.com/37277
--- fs/lustre/ptlrpc/lproc_ptlrpc.c | 3 ++- fs/lustre/ptlrpc/wiretest.c | 4 +++- include/uapi/linux/lustre/lustre_idl.h | 2 ++ 3 files changed, 7 insertions(+), 2 deletions(-)
fs/lustre/ptlrpc/lproc_ptlrpc.c +2 -1
@@ -67,6 +67,7 @@ { OST_QUOTACTL, "ost_quotactl" }, { OST_QUOTA_ADJUST_QUNIT, "ost_quota_adjust_qunit" }, { OST_LADVISE, "ost_ladvise" }, + { OST_FALLOCATE, "ost_fallocate"}, { MDS_GETATTR, "mds_getattr" }, { MDS_GETATTR_NAME, "mds_getattr_lock" }, { MDS_CLOSE, "mds_close" },
@@ -115,7 +116,7 @@ { 401, /* was OBD_LOG_CANCEL */ "llog_cancel" }, { 402, /* was OBD_QC_CALLBACK */ "obd_quota_callback" }, { OBD_IDX_READ, "dt_index_read" }, - { LLOG_ORIGIN_HANDLE_CREATE, "llog_origin_handle_open" }, + { LLOG_ORIGIN_HANDLE_CREATE, "llog_origin_handle_open" }, { LLOG_ORIGIN_HANDLE_NEXT_BLOCK, "llog_origin_handle_next_block" }, { LLOG_ORIGIN_HANDLE_READ_HEADER, "llog_origin_handle_read_header" }, { 504, /*LLOG_ORIGIN_HANDLE_WRITE_REC*/ "llog_origin_handle_write_rec" },
fs/lustre/ptlrpc/wiretest.c +3 -1
@@ -106,7 +106,9 @@ void lustre_assert_wire_constants(void) (long long)OST_QUOTA_ADJUST_QUNIT);LASSERTF (OST_LADVISE ==21 ,"found %lld\n" , (long long )OST_LADVISE ); -LASSERTF (OST_LAST_OPC ==22 ,"found %lld\n" , +LASSERTF (OST_FALLOCATE ==22 ,"found %lld\n" , + (long long )OST_FALLOCATE ); +LASSERTF (OST_LAST_OPC ==23 ,"found %lld\n" , (long long )OST_LAST_OPC );LASSERTF (OBD_OBJECT_EOF ==0xffffffffffffffffULL ,"found 0x%.16llxULL\n" ,OBD_OBJECT_EOF );
include/uapi/linux/lustre/lustre_idl.h +2 -0
@@ -956,6 +956,7 @@ enum ost_cmd {OST_QUOTACTL =19 ,OST_QUOTA_ADJUST_QUNIT =20 ,/* not used since 2.4 */ OST_LADVISE =21 , +OST_FALLOCATE =22 , OST_LAST_OPC /* must be < 33 to avoid MDS_GETATTR */ };#define OST_FIRST_OPC OST_REPLY
@@ -2789,6 +2790,7 @@ struct obdo {#define o_dropped o_misc#define o_cksum o_nlink#define o_grant_used o_data_version +#define o_falloc_mode o_nlink/* request structure for OST's */ struct ost_body {
Signature · 1 line
1.8.3.1
PATCH 609/622 lnet: libcfs: Cleanup use of bare printk
In reply to James SimmonsFrom: Shaun Tancheff <[email protected]>
Some users of printk(<LEVEL> "fmt" can be converted to pr_level("fmt" equivalents
WC-bug-id: https://jira.whamcloud.com/browse/LU-12861
Lustre-commit: b4c8a5180dec ("LU-12861 libcfs: Cleanup use of bare printk")
Reviewed-on: https://review.whamcloud.com/37046
--- net/lnet/libcfs/debug.c | 2 +- net/lnet/libcfs/module.c | 4 ++-- net/lnet/libcfs/tracefile.c | 43 +++++++++++++++++++++++++------------------ 3 files changed, 28 insertions(+), 21 deletions(-)
net/lnet/libcfs/debug.c +1 -1
@@ -418,7 +418,7 @@ void libcfs_debug_dumplog(void) "libcfs_debug_dumper");set_current_state (TASK_INTERRUPTIBLE );if (IS_ERR (dumper)) -pr_err ("LustreError: cannot start log dump thread: %ld\n" , +pr_err ("LustreError: cannot start log dump thread: rc = %ld\n" ,PTR_ERR (dumper));else schedule ();
net/lnet/libcfs/module.c +2 -2
@@ -720,7 +720,7 @@ int libcfs_setup(void) rc =libcfs_debug_init (5 *1024 *1024 );if (rc <0 ) { -pr_err ("LustreError: libcfs_debug_init: %d\n" , rc); +pr_err ("LustreError: libcfs_debug_init: rc = %d\n" , rc); goto err; }
@@ -794,7 +794,7 @@ static void libcfs_exit(void)/* the below message is checked in test-framework.sh check_mem_leak() */ rc =libcfs_debug_cleanup ();if (rc) -pr_err ("LustreError: libcfs_debug_cleanup: %d\n" , rc); +pr_err ("LustreError: libcfs_debug_cleanup: rc = %d\n" , rc); }MODULE_AUTHOR ("OpenSFS, Inc. <http://www.lustre.org/>" );
net/lnet/libcfs/tracefile.c +25 -18
110 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 610/622 lnet: Do not assume peers are MR capable
In reply to James SimmonsFrom: Chris Horn <[email protected]>
If a peer has discovery disabled then it will not consolidate peer NI information. This means we need to use a consistent source NI when sending to it just like we do for non-MR peers.
A comment in lnet_discovery_event_reply() indicates that this was a known issue, but the situation is not handled properly.
Do not assume peers are multi-rail capable when peer objects are allocated and initialized.
Do not mark a peer as multi-rail capable unless all of the following conditions are satisified:
1. The peer has the MR feature flag set
2. The peer has discovery enabled.
3. We have discovery enabled locally
Note: 1, 2, and 3 above are implemented in the code for lnet_discovery_event_reply(), but code earlier in the function breaks this behavior. Remove the offending code.
Update sanity-lnet tests 100 and 101 to reflect the fact that peers added via the traffic path no longer have multi-rail by default.
Cray-bug-id: LUS-7918
WC-bug-id: https://jira.whamcloud.com/browse/LU-12889
Lustre-commit: 3c580c93b8d3 ("LU-12889 lnet: Do not assume peers are MR capable")
Reviewed-on: https://review.whamcloud.com/36512
--- net/lnet/lnet/peer.c | 45 ++++++++++++++++----------------------------- 1 file changed, 16 insertions(+), 29 deletions(-)
net/lnet/lnet/peer.c +16 -29
@@ -1520,10 +1520,7 @@ struct lnet_peer_net *struct lnet_peer *lp;struct lnet_peer_net *lpn;struct lnet_peer_ni *lpni; -/* Assume peer is Multi-Rail capable and let discovery find out -* otherwise. -*/ -unsigned int flags =LNET_PEER_MULTI_RAIL ; +unsigned int flags =0 ;int rc =0 ;if (nid ==LNET_NID_ANY ) {
@@ -2298,20 +2295,7 @@ static void lnet_peer_clear_discovery_error(struct lnet_peer *lp) }/* -* Only enable the multi-rail feature on the peer if both sides of -* the connection have discovery on -*/ -if (pbuf->pb_info.pi_features &LNET_PING_FEAT_MULTI_RAIL ) { -CDEBUG (D_NET ,"Peer %s has Multi-Rail feature enabled\n" , -libcfs_nid2str (lp->lp_primary_nid)); - lp->lp_state |=LNET_PEER_MULTI_RAIL ; - }else { -CDEBUG (D_NET ,"Peer %s has Multi-Rail feature disabled\n" , -libcfs_nid2str (lp->lp_primary_nid)); - lp->lp_state &= ~LNET_PEER_MULTI_RAIL ; - } - -/* The peer may have discovery disabled at its end. Set +* The peer may have discovery disabled at its end. Set * NO_DISCOVERY as appropriate. */ if ((pbuf->pb_info.pi_features & LNET_PING_FEAT_DISCOVERY) &&
@@ -2332,21 +2316,24 @@ static void lnet_peer_clear_discovery_error(struct lnet_peer *lp)*/ if (pbuf->pb_info.pi_features &LNET_PING_FEAT_MULTI_RAIL ) {if (lp->lp_state &LNET_PEER_MULTI_RAIL ) { -/* Everything's fine */ +CDEBUG (D_NET ,"peer %s(%p) is MR\n" , +libcfs_nid2str (lp->lp_primary_nid), lp); }else if (lp->lp_state &LNET_PEER_CONFIGURED ) {CWARN ("Reply says %s is Multi-Rail, DLC says not\n" ,libcfs_nid2str (lp->lp_primary_nid)); + }else if (lnet_peer_discovery_disabled) { +CDEBUG (D_NET , +"peer %s(%p) not MR: DD disabled locally\n" , +libcfs_nid2str (lp->lp_primary_nid), lp); + }else if (lp->lp_state &LNET_PEER_NO_DISCOVERY ) { +CDEBUG (D_NET , +"peer %s(%p) not MR: DD disabled remotely\n" , +libcfs_nid2str (lp->lp_primary_nid), lp); }else { -/* if discovery is disabled then we don't want to -* update the state of the peer. All we'll do is -* update the peer_nis which were reported back in -* the initial ping -*/ - -if (!lnet_is_discovery_disabled_locked (lp)) { - lp->lp_state |=LNET_PEER_MULTI_RAIL ; -lnet_peer_clr_non_mr_pref_nids (lp); - } +CDEBUG (D_NET ,"peer %s(%p) is MR capable\n" , +libcfs_nid2str (lp->lp_primary_nid), lp); + lp->lp_state |=LNET_PEER_MULTI_RAIL ; +lnet_peer_clr_non_mr_pref_nids (lp); } }else if (lp->lp_state &LNET_PEER_MULTI_RAIL ) {if (lp->lp_state &LNET_PEER_CONFIGURED ) {
Signature · 1 line
1.8.3.1
PATCH 611/622 lnet: socklnd: convert peers hash table to hashtable.h
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
Using a hashtable.h hashtable, rather than bespoke code, has several advantages:
- the table is comprised of hlist_head, rather than list_head, so it consumes less memory (though we need to make it a little bigger as it must be a power-of-2)
- there are existing macros for easily walking the whole table
- it uses a "real" hash function rather than "mod a prime number".
In some ways, rhashtable might be even better, but it can change the ordering of objects in the table at arbitrary moments, and that could hurt the user-space API. It also does not support the partitioned walking that ksocknal_check_peer_timeouts() depends on.
Note that new peers are inserted at the top of a hash chain, rather than appended at the end. I don't think that should be a problem.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12678
Lustre-commit: dbbcf61d2bdc ("LU-12678 socklnd: convert peers hash table to hashtable.h")
Reviewed-on: https://review.whamcloud.com/36837
--- net/lnet/klnds/socklnd/socklnd.c | 299 ++++++++++++++++-------------------- net/lnet/klnds/socklnd/socklnd.h | 18 +-- net/lnet/klnds/socklnd/socklnd_cb.c | 8 +- 3 files changed, 140 insertions(+), 185 deletions(-)
net/lnet/klnds/socklnd/socklnd.c +131 -168
@@ -167,10 +167,10 @@struct ksock_peer_ni *ksocknal_find_peer_locked (struct lnet_ni *ni,struct lnet_process_id id) { -struct list_head *peer_list =ksocknal_nid2peerlist (id.nid);struct ksock_peer_ni *peer_ni; -list_for_each_entry (peer_ni, peer_list, ksnp_list) { +hash_for_each_possible (ksocknal_data.ksnd_peers, peer_ni, + ksnp_list, id.nid) {LASSERT (!peer_ni->ksnp_closing);if (peer_ni->ksnp_ni != ni)
@@ -229,7 +229,7 @@ struct ksock_peer_ni *LASSERT (list_empty (&peer_ni->ksnp_routes));LASSERT (!peer_ni->ksnp_closing); peer_ni->ksnp_closing =1 ; -list_del (&peer_ni->ksnp_list); +hlist_del (&peer_ni->ksnp_list);/* lose peerlist's ref */ ksocknal_peer_decref (peer_ni); }
@@ -247,55 +247,52 @@ struct ksock_peer_ni *read_lock (&ksocknal_data.ksnd_global_lock); -for (i =0 ; i < ksocknal_data.ksnd_peer_hash_size; i++) { -list_for_each_entry (peer_ni, &ksocknal_data.ksnd_peers[i], - ksnp_list) { -if (peer_ni->ksnp_ni != ni) -continue ; +hash_for_each (ksocknal_data.ksnd_peers, i, peer_ni, ksnp_list) { +if (peer_ni->ksnp_ni != ni) +continue ; -if (!peer_ni->ksnp_n_passive_ips && -list_empty (&peer_ni->ksnp_routes)) { -if (index-- >0 ) -continue ; +if (!peer_ni->ksnp_n_passive_ips && +list_empty (&peer_ni->ksnp_routes)) { +if (index-- >0 ) +continue ; - *id = peer_ni->ksnp_id; - *myip =0 ; - *peer_ip =0 ; - *port =0 ; - *conn_count =0 ; - *share_count =0 ; - rc =0 ; - goto out; - } + *id = peer_ni->ksnp_id; + *myip =0 ; + *peer_ip =0 ; + *port =0 ; + *conn_count =0 ; + *share_count =0 ; + rc =0 ; + goto out; + } -for (j =0 ; j < peer_ni->ksnp_n_passive_ips; j++) { -if (index-- >0 ) -continue ; +for (j =0 ; j < peer_ni->ksnp_n_passive_ips; j++) { +if (index-- >0 ) +continue ; - *id = peer_ni->ksnp_id; - *myip = peer_ni->ksnp_passive_ips[j]; - *peer_ip =0 ; - *port =0 ; - *conn_count =0 ; - *share_count =0 ; - rc =0 ; - goto out; - } + *id = peer_ni->ksnp_id; + *myip = peer_ni->ksnp_passive_ips[j]; + *peer_ip =0 ; + *port =0 ; + *conn_count =0 ; + *share_count =0 ; + rc =0 ; + goto out; + } -list_for_each_entry (route, &peer_ni->ksnp_routes, - ksnr_list) { -if (index-- >0 ) -continue ; +list_for_each_entry (route, &peer_ni->ksnp_routes, + ksnr_list) { +if (index-- >0 ) +continue ; - *id = peer_ni->ksnp_id; - *myip = route->ksnr_myipaddr; - *peer_ip = route->ksnr_ipaddr; - *port = route->ksnr_port; - *conn_count = route->ksnr_conn_count; - *share_count = route->ksnr_share_count; - rc =0 ; - goto out; - } + *id = peer_ni->ksnp_id; + *myip = route->ksnr_myipaddr; + *peer_ip = route->ksnr_ipaddr; + *port = route->ksnr_port; + *conn_count = route->ksnr_conn_count; + *share_count = route->ksnr_share_count; + rc =0 ; + goto out; } } out:
@@ -463,8 +460,7 @@ struct ksock_peer_ni * peer_ni = peer2; }else {/* peer_ni table takes my ref on peer_ni */ -list_add_tail (&peer_ni->ksnp_list, -ksocknal_nid2peerlist (id.nid)); +hash_add (ksocknal_data.ksnd_peers, &peer_ni->ksnp_list, id.nid); }list_for_each_entry (route2, &peer_ni->ksnp_routes, ksnr_list) {
@@ -544,7 +540,7 @@ struct ksock_peer_ni * ksocknal_del_peer(struct lnet_ni *ni, struct lnet_process_id id, u32 ip) {LIST_HEAD (zombies); -struct ksock_peer_ni *pnxt; +struct hlist_node *pnxt;struct ksock_peer_ni *peer_ni;int lo;int hi;
@@ -554,17 +550,17 @@ struct ksock_peer_ni *write_lock_bh (&ksocknal_data.ksnd_global_lock);if (id.nid !=LNET_NID_ANY ) { - lo = (int )(ksocknal_nid2peerlist (id.nid) - ksocknal_data.ksnd_peers); - hi = (int )(ksocknal_nid2peerlist (id.nid) - ksocknal_data.ksnd_peers); + lo =hash_min (id.nid,HASH_BITS (ksocknal_data.ksnd_peers)); + hi = lo; }else { lo =0 ; - hi = ksocknal_data.ksnd_peer_hash_size -1 ; + hi =HASH_SIZE (ksocknal_data.ksnd_peers) -1 ; }for (i = lo; i <= hi; i++) { -list_for_each_entry_safe (peer_ni, pnxt, - &ksocknal_data.ksnd_peers[i], - ksnp_list) { +hlist_for_each_entry_safe (peer_ni, pnxt, + &ksocknal_data.ksnd_peers[i], + ksnp_list) {if (peer_ni->ksnp_ni != ni)continue ;
@@ -609,23 +605,20 @@ struct ksock_peer_ni *read_lock (&ksocknal_data.ksnd_global_lock); -for (i =0 ; i < ksocknal_data.ksnd_peer_hash_size; i++) { -list_for_each_entry (peer_ni, &ksocknal_data.ksnd_peers[i], - ksnp_list) { -LASSERT (!peer_ni->ksnp_closing); +hash_for_each (ksocknal_data.ksnd_peers, i, peer_ni, ksnp_list) { +LASSERT (!peer_ni->ksnp_closing); + +if (peer_ni->ksnp_ni != ni) +continue ; -if (peer_ni->ksnp_ni != ni) +list_for_each_entry (conn, &peer_ni->ksnp_conns, + ksnc_list) { +if (index-- >0 )continue ; -list_for_each_entry (conn, &peer_ni->ksnp_conns, - ksnc_list) { -if (index-- >0 ) -continue ; - -ksocknal_conn_addref (conn); -read_unlock (&ksocknal_data.ksnd_global_lock); -return conn; - } +ksocknal_conn_addref (conn); +read_unlock (&ksocknal_data.ksnd_global_lock); +return conn; } }
@@ -1119,8 +1112,8 @@ struct ksock_peer_ni ** NB this puts an "empty" peer_ni in the peer * table (which takes my ref) */ -list_add_tail (&peer_ni->ksnp_list, -ksocknal_nid2peerlist (peerid.nid)); +hash_add (ksocknal_data.ksnd_peers, + &peer_ni->ksnp_list, peerid.nid); }else {ksocknal_peer_decref (peer_ni); peer_ni = peer2;
@@ -1732,7 +1725,7 @@ struct ksock_peer_ni * ksocknal_close_matching_conns(struct lnet_process_id id, u32 ipaddr) {struct ksock_peer_ni *peer_ni; -struct ksock_peer_ni *pnxt; +struct hlist_node *pnxt;int lo;int hi;int i;
@@ -1741,17 +1734,17 @@ struct ksock_peer_ni *write_lock_bh (&ksocknal_data.ksnd_global_lock);if (id.nid !=LNET_NID_ANY ) { - lo = (int )(ksocknal_nid2peerlist (id.nid) - ksocknal_data.ksnd_peers); - hi = (int )(ksocknal_nid2peerlist (id.nid) - ksocknal_data.ksnd_peers); + lo =hash_min (id.nid,HASH_BITS (ksocknal_data.ksnd_peers)); + hi = lo; }else { lo =0 ; - hi = ksocknal_data.ksnd_peer_hash_size -1 ; + hi =HASH_SIZE (ksocknal_data.ksnd_peers) -1 ; }for (i = lo; i <= hi; i++) { -list_for_each_entry_safe (peer_ni, pnxt, - &ksocknal_data.ksnd_peers[i], - ksnp_list) { +hlist_for_each_entry_safe (peer_ni, pnxt, + &ksocknal_data.ksnd_peers[i], + ksnp_list) { if (!((id.nid == LNET_NID_ANY || id.nid == peer_ni->ksnp_id.nid) && (id.pid == LNET_PID_ANY ||
@@ -1769,10 +1762,7 @@ struct ksock_peer_ni *if (id.nid ==LNET_NID_ANY || id.pid ==LNET_PID_ANY || !ipaddr)return 0 ; -if (!count) -return -ENOENT ; -else -return 0 ; +return count ?0 : -ENOENT ; }void
@@ -1892,21 +1882,20 @@ struct ksock_peer_ni *static int ksocknal_push (struct lnet_ni *ni,struct lnet_process_id id) { -struct list_head *start; -struct list_head *end; -struct list_head *tmp; +int lo; +int hi; +int bkt;int rc = -ENOENT ; -unsigned int hsize = ksocknal_data.ksnd_peer_hash_size; -if (id.nid ==LNET_NID_ANY ) { - start = &ksocknal_data.ksnd_peers[0 ]; - end = &ksocknal_data.ksnd_peers[hsize -1 ]; +if (id.nid !=LNET_NID_ANY ) { + lo =hash_min (id.nid,HASH_BITS (ksocknal_data.ksnd_peers)); + hi = lo; }else { - start =ksocknal_nid2peerlist (id.nid); - end =ksocknal_nid2peerlist (id.nid); + lo =0 ; + hi =HASH_SIZE (ksocknal_data.ksnd_peers) -1 ; } -for (tmp = start; tmp <= end; tmp++) { +for (bkt = lo; bkt <= hi; bkt++) {int peer_off;/* searching offset in peer_ni hash table */ for (peer_off =0 ; ; peer_off++) {
@@ -1914,7 +1903,9 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)int i =0 ;read_lock (&ksocknal_data.ksnd_global_lock); -list_for_each_entry (peer_ni, tmp, ksnp_list) { +hlist_for_each_entry (peer_ni, + &ksocknal_data.ksnd_peers[bkt], + ksnp_list) { if (!((id.nid == LNET_NID_ANY || id.nid == peer_ni->ksnp_id.nid) && (id.pid == LNET_PID_ANY ||
@@ -1969,24 +1960,15 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) iface->ksni_nroutes =0 ; iface->ksni_npeers =0 ; -for (i =0 ; i < ksocknal_data.ksnd_peer_hash_size; i++) { -list_for_each_entry (peer_ni, - &ksocknal_data.ksnd_peers[i], - ksnp_list) { - -for (j =0 ; - j < peer_ni->ksnp_n_passive_ips; - j++) -if (peer_ni->ksnp_passive_ips[j] == - ipaddress) - iface->ksni_npeers++; - -list_for_each_entry (route, - &peer_ni->ksnp_routes, - ksnr_list) { -if (route->ksnr_myipaddr == ipaddress) - iface->ksni_nroutes++; - } +hash_for_each (ksocknal_data.ksnd_peers, i, peer_ni, ksnp_list) { +for (j =0 ; j < peer_ni->ksnp_n_passive_ips; j++) +if (peer_ni->ksnp_passive_ips[j] == ipaddress) + iface->ksni_npeers++; + +list_for_each_entry (route, &peer_ni->ksnp_routes, + ksnr_list) { +if (route->ksnr_myipaddr == ipaddress) + iface->ksni_nroutes++; } }
@@ -2048,7 +2030,7 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) {struct ksock_net *net = ni->ni_data;int rc = -ENOENT ; -struct ksock_peer_ni *nxt; +struct hlist_node *nxt;struct ksock_peer_ni *peer_ni;u32 this_ip;int i;
@@ -2070,16 +2052,12 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) net->ksnn_ninterfaces--; -for (j =0 ; j < ksocknal_data.ksnd_peer_hash_size; j++) { -list_for_each_entry_safe (peer_ni, nxt, - &ksocknal_data.ksnd_peers[j], - ksnp_list) { -if (peer_ni->ksnp_ni != ni) -continue ; +hash_for_each_safe (ksocknal_data.ksnd_peers, j, + nxt, peer_ni, ksnp_list) { +if (peer_ni->ksnp_ni != ni) +continue ; -ksocknal_peer_del_interface_locked (peer_ni, - this_ip); - } +ksocknal_peer_del_interface_locked (peer_ni, this_ip); } }
@@ -2224,8 +2202,6 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)if (ksocknal_data.ksnd_schedulers)cfs_percpt_free (ksocknal_data.ksnd_schedulers); -kvfree (ksocknal_data.ksnd_peers); -spin_lock (&ksocknal_data.ksnd_tx_lock);if (!list_empty (&ksocknal_data.ksnd_idle_noop_txs)) {
@@ -2250,6 +2226,7 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)ksocknal_base_shutdown (void) {struct ksock_sched *sched; +struct ksock_peer_ni *peer_ni;int i;LASSERT (!ksocknal_data.ksnd_nnets);
@@ -2260,9 +2237,8 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)/* fall through */ case SOCKNAL_INIT_ALL :case SOCKNAL_INIT_DATA : -LASSERT (ksocknal_data.ksnd_peers); -for (i =0 ; i < ksocknal_data.ksnd_peer_hash_size; i++) -LASSERT (list_empty (&ksocknal_data.ksnd_peers[i])); +hash_for_each (ksocknal_data.ksnd_peers, i, peer_ni, ksnp_list) +LASSERT (0 );LASSERT (list_empty (&ksocknal_data.ksnd_nets));LASSERT (list_empty (&ksocknal_data.ksnd_enomem_conns));
@@ -2326,15 +2302,7 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)memset (&ksocknal_data,0 ,sizeof (ksocknal_data));/* zero pointers */ - ksocknal_data.ksnd_peer_hash_size =SOCKNAL_PEER_HASH_SIZE ; - ksocknal_data.ksnd_peers =kvmalloc_array (ksocknal_data.ksnd_peer_hash_size, -sizeof (struct list_head ), -GFP_KERNEL ); -if (!ksocknal_data.ksnd_peers) -return -ENOMEM ; - -for (i =0 ; i < ksocknal_data.ksnd_peer_hash_size; i++) -INIT_LIST_HEAD (&ksocknal_data.ksnd_peers[i]); +hash_init (ksocknal_data.ksnd_peers);rwlock_init (&ksocknal_data.ksnd_global_lock);INIT_LIST_HEAD (&ksocknal_data.ksnd_nets);
@@ -2452,43 +2420,38 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)read_lock (&ksocknal_data.ksnd_global_lock); -for (i =0 ; i < ksocknal_data.ksnd_peer_hash_size; i++) { -list_for_each_entry (peer_ni, &ksocknal_data.ksnd_peers[i], - ksnp_list) { -struct ksock_route *route; -struct ksock_conn *conn; - -if (peer_ni->ksnp_ni != ni) -continue ; +hash_for_each (ksocknal_data.ksnd_peers, i, peer_ni, ksnp_list) { +struct ksock_route *route; +struct ksock_conn *conn; -CWARN ("Active peer_ni on shutdown: %s, ref %d, closing %d, accepting %d, err %d, zcookie %llu, txq %d, zc_req %d\n" , -libcfs_id2str (peer_ni->ksnp_id), -atomic_read (&peer_ni->ksnp_refcount), - peer_ni->ksnp_closing, - peer_ni->ksnp_accepting, peer_ni->ksnp_error, - peer_ni->ksnp_zc_next_cookie, - !list_empty (&peer_ni->ksnp_tx_queue), - !list_empty (&peer_ni->ksnp_zc_req_list)); +if (peer_ni->ksnp_ni != ni) +continue ; -list_for_each_entry (route, &peer_ni->ksnp_routes, - ksnr_list) { -CWARN ("Route: ref %d, schd %d, conn %d, cnted %d, del %d\n" , -atomic_read (&route->ksnr_refcount), - route->ksnr_scheduled, - route->ksnr_connecting, - route->ksnr_connected, - route->ksnr_deleted); - } +CWARN ("Active peer_ni on shutdown: %s, ref %d, closing %d, accepting %d, err %d, zcookie %llu, txq %d, zc_req %d\n" , +libcfs_id2str (peer_ni->ksnp_id), +atomic_read (&peer_ni->ksnp_refcount), + peer_ni->ksnp_closing, + peer_ni->ksnp_accepting, peer_ni->ksnp_error, + peer_ni->ksnp_zc_next_cookie, + !list_empty (&peer_ni->ksnp_tx_queue), + !list_empty (&peer_ni->ksnp_zc_req_list)); + +list_for_each_entry (route, &peer_ni->ksnp_routes, ksnr_list) { +CWARN ("Route: ref %d, schd %d, conn %d, cnted %d, del %d\n" , +atomic_read (&route->ksnr_refcount), + route->ksnr_scheduled, + route->ksnr_connecting, + route->ksnr_connected, + route->ksnr_deleted); + } -list_for_each_entry (conn, &peer_ni->ksnp_conns, - ksnc_list) { -CWARN ("Conn: ref %d, sref %d, t %d, c %d\n" , -atomic_read (&conn->ksnc_conn_refcount), -atomic_read (&conn->ksnc_sock_refcount), - conn->ksnc_type, conn->ksnc_closing); - } - goto done; +list_for_each_entry (conn, &peer_ni->ksnp_conns, ksnc_list) { +CWARN ("Conn: ref %d, sref %d, t %d, c %d\n" , +atomic_read (&conn->ksnc_conn_refcount), +atomic_read (&conn->ksnc_sock_refcount), + conn->ksnc_type, conn->ksnc_closing); } + goto done; } done:read_unlock (&ksocknal_data.ksnd_global_lock);
net/lnet/klnds/socklnd/socklnd.h +5 -13
@@ -43,7 +43,7 @@#include <linux/sysctl.h> #include <linux/uio.h> #include <linux/unistd.h> -#include <asm/irq.h> +#include <linux/hashtable.h> #include <net/sock.h> #include <net/tcp.h>
@@ -54,7 +54,7 @@#define SOCKNAL_NSCHEDS 3#define SOCKNAL_NSCHEDS_HIGH (SOCKNAL_NSCHEDS << 1) -#define SOCKNAL_PEER_HASH_SIZE 101/* # peer_ni lists */ +#define SOCKNAL_PEER_HASH_BITS 7/* # log2 of # of peer_ni lists */ #define SOCKNAL_RESCHED 100/* # scheduler loops before reschedule */ #define SOCKNAL_INSANITY_RECONN 5000/* connd is trying on reconn infinitely */ #define SOCKNAL_ENOMEM_RETRY 1/* seconds between retries */
@@ -190,10 +190,10 @@ struct ksock_nal_data {rwlock_t ksnd_global_lock;/* stabilize * peer_ni/conn ops */ -struct list_head *ksnd_peers;/* hash table of all my +DECLARE_HASHTABLE (ksnd_peers,SOCKNAL_PEER_HASH_BITS ); +/* hash table of all my * known peers */ -int ksnd_peer_hash_size;/* size of ksnd_peers */ int ksnd_nthreads;/* # live threads */ int ksnd_shuttingdown;/* tell threads to exit
@@ -411,7 +411,7 @@ struct ksock_route {#define SOCKNAL_KEEPALIVE_PING 1/* cookie for keepalive ping */ struct ksock_peer_ni { -struct list_head ksnp_list;/* stash on global peer_ni list */ +struct hlist_node ksnp_list;/* on global peer_nis hash table */ time64_t ksnp_last_alive;/* when (in seconds) I was last * alive */
@@ -519,14 +519,6 @@ struct ksock_proto { (1 << SOCKLND_CONN_BULK_OUT)); } -static inline struct list_head * -ksocknal_nid2peerlist (lnet_nid_t nid) -{ -unsigned int hash = ((unsigned int )nid) % ksocknal_data.ksnd_peer_hash_size; - -return &ksocknal_data.ksnd_peers[hash]; -} -static inline void ksocknal_conn_addref (struct ksock_conn *conn) {
net/lnet/klnds/socklnd/socklnd_cb.c +4 -4
@@ -2386,7 +2386,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)static void ksocknal_check_peer_timeouts (int idx) { -struct list_head *peers = &ksocknal_data.ksnd_peers[idx]; +struct hlist_head *peers = &ksocknal_data.ksnd_peers[idx];struct ksock_peer_ni *peer_ni;struct ksock_conn *conn;struct ksock_tx *tx;
@@ -2399,7 +2399,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)*/ read_lock (&ksocknal_data.ksnd_global_lock); -list_for_each_entry (peer_ni, peers, ksnp_list) { +hlist_for_each_entry (peer_ni, peers, ksnp_list) {struct ksock_tx *tx_stale;time64_t deadline =0 ;int resid =0 ;
@@ -2564,7 +2564,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)while ((timeout = deadline -ktime_get_seconds ()) <=0 ) {const int n =4 ;const int p =1 ; -int chunk = ksocknal_data.ksnd_peer_hash_size; +int chunk =HASH_SIZE (ksocknal_data.ksnd_peers);unsigned int lnd_timeout;/*
@@ -2585,7 +2585,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)for (i =0 ; i < chunk; i++) {ksocknal_check_peer_timeouts (peer_index); peer_index = (peer_index +1 ) % - ksocknal_data.ksnd_peer_hash_size; +HASH_SIZE (ksocknal_data.ksnd_peers); } deadline += p;
Signature · 1 line
1.8.3.1
PATCH 612/622 lustre: llite: Update mdc and lite stats on open|creat
In reply to James SimmonsFrom: Olaf Faaland <[email protected]>
Increment "create" counter in mdc/<instance>/md_stats, and "mknod" counter in llite/<instance>stats when an open with the CREAT flag results in a newly created file.
The mknod counter is chosen for consistency with
patch http://review.whamcloud.com/20246
"LU-8150 mdt: Track open+create as mknod"
but the mdc counter set does not include mknod.
WC-bug-id: https://jira.whamcloud.com/browse/LU-11114
Lustre-commit: 4b8518ee4fa5 ("LU-11114 llite: Update mdc and lite stats on open|creat")
Reviewed-on: https://review.whamcloud.com/36948
--- fs/lustre/llite/namei.c | 12 ++++++++++-- fs/lustre/mdc/mdc_locks.c | 6 ++++++ 2 files changed, 16 insertions(+), 2 deletions(-)
fs/lustre/llite/namei.c +10 -2
@@ -605,7 +605,8 @@ struct dentry *ll_splice_alias(struct inode *inode, struct dentry *de)static int ll_lookup_it_finish (struct ptlrpc_request *request,struct lookup_intent *it,struct inode *parent,struct dentry **de, -void *secctx,u32 secctxlen) +void *secctx,u32 secctxlen, +ktime_t kstart) {struct inode *inode =NULL ;u64 bits =0 ;
@@ -708,6 +709,11 @@ static int ll_lookup_it_finish(struct ptlrpc_request *request, } } +if (it_disposition (it,DISP_OPEN_CREATE )) { +ll_stats_ops_tally (ll_i2sbi (parent),LPROC_LL_MKNOD , +ktime_us_delta (ktime_get (), kstart)); + } + out:if (rc !=0 && it->it_op &IT_OPEN ) {ll_intent_drop_lock (it);
@@ -722,6 +728,7 @@ static struct dentry *ll_lookup_it(struct inode *parent, struct dentry *dentry,u32 *secctxlen, struct pcc_create_attach *pca) { + ktime_t kstart = ktime_get();struct lookup_intent lookup_it = { .it_op =IT_LOOKUP };struct dentry *save = dentry, *retval;struct ptlrpc_request *req =NULL ;
@@ -887,7 +894,8 @@ static struct dentry *ll_lookup_it(struct inode *parent, struct dentry *dentry,ll_unlock_md_op_lsm (op_data); rc =ll_lookup_it_finish (req, it, parent, &dentry, secctx ? *secctx :NULL , - secctxlen ? *secctxlen :0 ); + secctxlen ? *secctxlen :0 , + kstart);if (rc !=0 ) {ll_intent_release (it); retval =ERR_PTR (rc);
fs/lustre/mdc/mdc_locks.c +6 -0
@@ -733,6 +733,12 @@ static int mdc_finish_enqueue(struct obd_export *exp,mdc_set_open_replay_data (NULL ,NULL , it); } +if (it_disposition (it,DISP_OPEN_CREATE ) && + !it_open_error (DISP_OPEN_CREATE , it)) { +lprocfs_counter_incr (exp->exp_obd->obd_md_stats, +LPROC_MD_CREATE ); + } +if (body->mbo_valid & (OBD_MD_FLDIREA |OBD_MD_FLEASIZE )) {void *eadata;
Signature · 1 line
1.8.3.1
PATCH 613/622 lustre: osc: glimpse and lock cancel race
In reply to James SimmonsFrom: Alexander Zarochentsev <[email protected]>
osc_dlm_blocking_ast0 clears l_ast_data before writing file data to OST and opens a race window. Neither a glimpse AST nor ldlm_cb_interpret can find correct file attributes at that moment.
Cray-bug-id: LUS-8344
WC-bug-id: https://jira.whamcloud.com/browse/LU-13128
Lustre-commit: 7c99f67d9d39 ("LU-13128 osc: glimpse and lock cancel race")
Reviewed-on: https://review.whamcloud.com/37215
--- fs/lustre/include/obd_support.h | 1 + fs/lustre/mdc/mdc_dev.c | 2 +- fs/lustre/osc/osc_lock.c | 8 ++++++-- 3 files changed, 8 insertions(+), 3 deletions(-)
fs/lustre/include/obd_support.h +1 -0
@@ -332,6 +332,7 @@#define OBD_FAIL_OSC_CONNECT_GRANT_PARAM 0x413#define OBD_FAIL_OSC_DELAY_IO 0x414#define OBD_FAIL_OSC_NO_SIZE_DATA 0x415 +#define OBD_FAIL_OSC_DELAY_CANCEL 0x416#define OBD_FAIL_PTLRPC 0x500#define OBD_FAIL_PTLRPC_ACK 0x501
fs/lustre/mdc/mdc_dev.c +1 -1
@@ -313,7 +313,6 @@ static int mdc_dlm_blocking_ast0(const struct lu_env *env,if (dlmlock->l_ast_data) { obj =osc2cl (dlmlock->l_ast_data); - dlmlock->l_ast_data =NULL ;cl_object_get (obj); }unlock_res_and_lock (dlmlock);
@@ -332,6 +331,7 @@ static int mdc_dlm_blocking_ast0(const struct lu_env *env,*/ /* losing a lock, update kms */ lock_res_and_lock (dlmlock); + dlmlock->l_ast_data =NULL ;cl_object_attr_lock (obj); attr->cat_kms =0 ;cl_object_attr_update (env, obj, attr,CAT_KMS );
fs/lustre/osc/osc_lock.c +6 -2
@@ -419,13 +419,13 @@ static int __osc_dlm_blocking_ast(const struct lu_env *env,if (dlmlock->l_ast_data) { obj =osc2cl (dlmlock->l_ast_data); - dlmlock->l_ast_data =NULL ; -cl_object_get (obj); }unlock_res_and_lock (dlmlock); +OBD_FAIL_TIMEOUT (OBD_FAIL_OSC_DELAY_CANCEL ,5 ); +/* if l_ast_data is NULL, the dlmlock was enqueued by AGL or * the object has been destroyed. */
@@ -442,6 +442,10 @@ static int __osc_dlm_blocking_ast(const struct lu_env *env,/* losing a lock, update kms */ lock_res_and_lock (dlmlock); +/* clearing l_ast_data after flushing data, +* to let glimpse ast find the lock and the object +*/ + dlmlock->l_ast_data =NULL ;cl_object_attr_lock (obj);/* Must get the value under the lock to avoid race. */ old_kms =cl2osc (obj)->oo_oinfo->loi_kms;
Signature · 1 line
1.8.3.1
PATCH 614/622 lustre: llog: keep llog handle alive until last reference
In reply to James SimmonsFrom: Mikhail Pershin <[email protected]>
Llog handle keeps related dt_object pinned until llog_close() call, meanwhile llog handle can still have other users which took llog handle via llog_cat_id2handle()
Patch changes llog_handle_put() to call lop_close() upon last reference drop. So llog_osd_close() will put dt_object only when llog_handle has no more references.
The llog_handle_get() checks and reports if llog_handle has zero reference.
Also patch modifies checks for destroyed llogs, llog handle has new lgh_destroyed flag which is set when llog is destroyed, llog_osd_exist() checks dt_object_exist() and lgh_destroyed flag, so destroyed llogs are considered as non-existent too. Previously it uses lu_object_is_dying() check which is not reliable because means only that object is not to be kept in cache.
WC-bug-id: https://jira.whamcloud.com/browse/LU-10198
Lustre-commit: d6bd5e9cc49b ("LU-10198 llog: keep llog handle alive until last reference")
Reviewed-on: https://review.whamcloud.com/37367
--- fs/lustre/include/lustre_log.h | 3 ++- fs/lustre/obdclass/llog.c | 49 +++++++++++++++++++------------------- fs/lustre/obdclass/llog_cat.c | 19 +++++++++------ fs/lustre/obdclass/llog_internal.h | 4 ++-- 4 files changed, 40 insertions(+), 35 deletions(-)
fs/lustre/include/lustre_log.h +2 -1
@@ -226,7 +226,8 @@ struct llog_handle {char *lgh_name;void *private_data;struct llog_operations *lgh_logops; -struct kref lgh_refcount; +refcount_t lgh_refcount; +bool lgh_destroyed; };#define LLOG_CTXT_FLAG_UNINITIALIZED 0x00000001
fs/lustre/obdclass/llog.c +24 -25
83 lines not on this page · raw message
fs/lustre/obdclass/llog_cat.c +12 -7
@@ -85,10 +85,16 @@ static int llog_cat_id2handle(const struct lu_env *env, cgl->lgl_ogen, logid->lgl_ogen);continue ; } + *res =llog_handle_get (loghandle); +if (!*res) { +CERROR ("%s: log " DFID " refcount is zero!\n" , + loghandle->lgh_ctxt->loc_obd->obd_name, +PFID (&logid->lgl_oi.oi_fid)); +continue ; + } loghandle->u.phd.phd_cat_handle = cathandle;up_write (&cathandle->lgh_lock); - rc =0 ; - goto out; +return rc; } }up_write (&cathandle->lgh_lock);
@@ -105,10 +111,12 @@ static int llog_cat_id2handle(const struct lu_env *env, rc =llog_init_handle (env, loghandle, fmt |LLOG_F_IS_PLAIN ,NULL );if (rc <0 ) {llog_close (env, loghandle); - loghandle =NULL ; + *res =NULL ;return rc; } + *res =llog_handle_get (loghandle); +LASSERT (*res);down_write (&cathandle->lgh_lock);list_add_tail (&loghandle->u.phd.phd_entry, &cathandle->u.chd.chd_head);up_write (&cathandle->lgh_lock);
@@ -117,9 +125,6 @@ static int llog_cat_id2handle(const struct lu_env *env, loghandle->u.phd.phd_cookie.lgc_lgl = cathandle->lgh_id; loghandle->u.phd.phd_cookie.lgc_index = loghandle->lgh_hdr->llh_cat_idx; -out: -llog_handle_get (loghandle); - *res = loghandle;return 0 ; }
@@ -204,7 +209,7 @@ static int llog_cat_process_cb(const struct lu_env *env, } out: -llog_handle_put (llh); +llog_handle_put (env, llh);return rc; }
fs/lustre/obdclass/llog_internal.h +2 -2
@@ -61,8 +61,8 @@ struct llog_thread_info {int llog_info_init (void );void llog_info_fini (void ); -void llog_handle_get (struct llog_handle *loghandle); -void llog_handle_put (struct llog_handle *loghandle); +struct llog_handle *llog_handle_get (struct llog_handle *loghandle); +int llog_handle_put (const struct lu_env *env,struct llog_handle *loghandle);int class_config_dump_handler (const struct lu_env *env,struct llog_handle *handle,struct llog_rec_hdr *rec,void *data);
Signature · 1 line
1.8.3.1
PATCH 615/622 lnet: handling device failure by IB event handler
In reply to James SimmonsFrom: Tatsushi Takamura <[email protected]>
The following IB events cannot be handled by QP event handler
- IB_EVENT_DEVICE_FATAL
- IB_EVENT_PORT_ERR
- IB_EVENT_PORT_ACTIVE
IB event handler handles device errors such as hardware errors and link down.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12287
Lustre-commit: c6e4c21c4f8b ("LU-12287 lnet: handling device failure by IB event handler")
Reviewed-on: https://review.whamcloud.com/35037
--- net/lnet/klnds/o2iblnd/o2iblnd.c | 100 +++++++++++++++++++++++++++++++++++++++ net/lnet/klnds/o2iblnd/o2iblnd.h | 8 ++++ 2 files changed, 108 insertions(+)
net/lnet/klnds/o2iblnd/o2iblnd.c +100 -0
@@ -2306,9 +2306,93 @@ static int kiblnd_net_init_pools(struct kib_net *net, struct lnet_ni *ni,return rc; } +static int kiblnd_port_get_attr (struct kib_hca_dev *hdev) +{ +struct ib_port_attr *port_attr; +int rc; +unsigned long flags; +rwlock_t *g_lock = &kiblnd_data.kib_global_lock; + + port_attr =kzalloc (sizeof (*port_attr),GFP_NOFS ); +if (!port_attr) { +CDEBUG (D_NETERROR ,"Out of memory\n" ); +return -ENOMEM ; + } + + rc =ib_query_port (hdev->ibh_ibdev, hdev->ibh_port, port_attr); + +write_lock_irqsave (g_lock, flags); + +if (rc ==0 ) + hdev->ibh_state = port_attr->state ==IB_PORT_ACTIVE + ?IBLND_DEV_PORT_ACTIVE + :IBLND_DEV_PORT_DOWN ; + +write_unlock_irqrestore (g_lock, flags); +kfree (port_attr); + +if (rc !=0 ) { +CDEBUG (D_NETERROR ,"Failed to query IB port: %d\n" , rc); +return rc; + } +return 0 ; +} + +static inline void +kiblnd_set_ni_fatal_on (struct kib_hca_dev *hdev,int val) +{ +struct kib_net *net; + +/* for health check */ +list_for_each_entry (net, &hdev->ibh_dev->ibd_nets, ibn_list) { +if (val) +CDEBUG (D_NETERROR ,"Fatal device error for NI %s\n" , +libcfs_nid2str (net->ibn_ni->ni_nid)); +atomic_set (&net->ibn_ni->ni_fatal_error_on, val); + } +} + +void +kiblnd_event_handler (struct ib_event_handler *handler,struct ib_event *event) +{ +rwlock_t *g_lock = &kiblnd_data.kib_global_lock; +struct kib_hca_dev *hdev; +unsigned long flags; + + hdev = container_of(handler, struct kib_hca_dev, ibh_event_handler); + +write_lock_irqsave (g_lock, flags); + +switch (event->event) { +case IB_EVENT_DEVICE_FATAL : +CDEBUG (D_NET ,"IB device fatal\n" ); + hdev->ibh_state =IBLND_DEV_FATAL ; +kiblnd_set_ni_fatal_on (hdev,1 ); +break ; +case IB_EVENT_PORT_ACTIVE : +CDEBUG (D_NET ,"IB port active\n" ); +if (event->element.port_num == hdev->ibh_port) { + hdev->ibh_state =IBLND_DEV_PORT_ACTIVE ; +kiblnd_set_ni_fatal_on (hdev,0 ); + } +break ; +case IB_EVENT_PORT_ERR : +CDEBUG (D_NET ,"IB port err\n" ); +if (event->element.port_num == hdev->ibh_port) { + hdev->ibh_state =IBLND_DEV_PORT_DOWN ; +kiblnd_set_ni_fatal_on (hdev,1 ); + } +break ; +default : +break ; + } +write_unlock_irqrestore (g_lock, flags); +} +static int kiblnd_hdev_get_attr (struct kib_hca_dev *hdev) {struct ib_device_attr *dev_attr = &hdev->ibh_ibdev->attrs; +int rc2 =0 ;/* * It's safe to assume a HCA can handle a page size
@@ -2338,12 +2422,19 @@ static int kiblnd_hdev_get_attr(struct kib_hca_dev *hdev) hdev->ibh_mr_size = dev_attr->max_mr_size; hdev->ibh_max_qp_wr = dev_attr->max_qp_wr; + rc2 =kiblnd_port_get_attr (hdev); +if (rc2 !=0 ) +return rc2; +CERROR ("Invalid mr size: %#llx\n" , hdev->ibh_mr_size);return -EINVAL ; }void kiblnd_hdev_destroy (struct kib_hca_dev *hdev) { +if (hdev->ibh_event_handler.device) +ib_unregister_event_handler (&hdev->ibh_event_handler); +if (hdev->ibh_pd)ib_dealloc_pd (hdev->ibh_pd);
@@ -2491,6 +2582,7 @@ int kiblnd_dev_failover(struct kib_dev *dev, struct net *ns) hdev->ibh_dev = dev; hdev->ibh_cmid = cmid; hdev->ibh_ibdev = cmid->device; + hdev->ibh_port = cmid->port_num; pd =ib_alloc_pd (cmid->device,0 );if (IS_ERR (pd)) {
@@ -2513,6 +2605,10 @@ int kiblnd_dev_failover(struct kib_dev *dev, struct net *ns) goto out; } +INIT_IB_EVENT_HANDLER (&hdev->ibh_event_handler, + hdev->ibh_ibdev, kiblnd_event_handler); +ib_register_event_handler (&hdev->ibh_event_handler); +write_lock_irqsave (&kiblnd_data.kib_global_lock, flags);swap (dev->ibd_hdev, hdev);/* take over the refcount */
@@ -2907,6 +3003,7 @@ static int kiblnd_startup(struct lnet_ni *ni) goto net_failed; } + net->ibn_ni = ni; net->ibn_incarnation =ktime_get_real_ns () /NSEC_PER_USEC ; rc =kiblnd_tunables_setup (ni);
@@ -3000,6 +3097,9 @@ static int kiblnd_startup(struct lnet_ni *ni)write_lock_irqsave (&kiblnd_data.kib_global_lock, flags); ibdev->ibd_nnets++;list_add_tail (&net->ibn_list, &ibdev->ibd_nets); +/* for health check */ +if (ibdev->ibd_hdev->ibh_state ==IBLND_DEV_PORT_DOWN ) +kiblnd_set_ni_fatal_on (ibdev->ibd_hdev,1 );write_unlock_irqrestore (&kiblnd_data.kib_global_lock, flags); net->ibn_init =IBLND_INIT_ALL ;
net/lnet/klnds/o2iblnd/o2iblnd.h +8 -0
@@ -180,6 +180,13 @@ struct kib_hca_dev {u64 ibh_mr_size;/* size of MR */ int ibh_max_qp_wr;/* maximum work requests size */ struct ib_pd *ibh_pd;/* PD */ +u8 ibh_port;/* port number */ +struct ib_event_handler + ibh_event_handler;/* IB event handler */ +int ibh_state;/* device status */ +#define IBLND_DEV_PORT_DOWN 0 +#define IBLND_DEV_PORT_ACTIVE 1 +#define IBLND_DEV_FATAL 2struct kib_dev *ibh_dev;/* owner */ atomic_t ibh_ref;/* refcount */ };
@@ -309,6 +316,7 @@ struct kib_net {struct kib_fmr_poolset **ibn_fmr_ps;/* fmr pool-set */ struct kib_dev *ibn_dev;/* underlying IB device */ +struct lnet_ni *ibn_ni;/* LNet interface */ };#define KIB_THREAD_SHIFT 16
Signature · 1 line
1.8.3.1
PATCH 616/622 lustre: ptlrpc: simplify wait_event handling in unregister functions
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
We can simplify the wait_event_idle_timeout() handling in both ptlrpc_unregister_bulk() and ptlrpc_unregister_reply() by changing the timeout to a countdown. Less variables are needed on the stack.
WC-bug-id: https://jira.whamcloud.com/browse/LU-10467
Lustre-commit: 5e30a2c06176f50f ("LU-10467 lustre: convert most users of LWI_TIMEOUT_INTERVAL()")
Reviewed-on: https://review.whamcloud.com/35973
--- fs/lustre/ptlrpc/client.c | 31 ++++++++++--------------------- fs/lustre/ptlrpc/niobuf.c | 30 ++++++++++++++---------------- 2 files changed, 24 insertions(+), 37 deletions(-)
fs/lustre/ptlrpc/client.c +10 -21
@@ -2570,9 +2570,6 @@ u64 ptlrpc_req_xid(struct ptlrpc_request *request)*/ static int ptlrpc_unregister_reply (struct ptlrpc_request *request,int async) { -int rc; -wait_queue_head_t *wq; -/* Might sleep. */ LASSERT (!in_interrupt ());
@@ -2599,29 +2596,21 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async)if (async)return 0 ; -/* -* We have to wait_event_idle_timeout() whatever the result, to get -* a chance to run reply_in_callback(), and to make sure we've -* unlinked before returning a req to the pool. -*/ -if (request->rq_set) - wq = &request->rq_set->set_waitq; -else - wq = &request->rq_reply_waitq; -for (;;) { +wait_queue_head_t *wq = (request->rq_set) ? + &request->rq_set->set_waitq : + &request->rq_reply_waitq; +int seconds =LONG_UNLINK ;/* * Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish NALs */ -int cnt =0 ; - -while (cnt <LONG_UNLINK && - (rc =wait_event_idle_timeout (*wq, - !ptlrpc_client_recv_or_unlink (request), -HZ )) ==0 ) - cnt +=1 ; -if (rc >0 ) { +while (seconds >LONG_UNLINK && + (wait_event_idle_timeout (*wq, + !ptlrpc_client_recv_or_unlink (request), +HZ )) ==0 ) + seconds -=1 ; +if (seconds >0 ) {ptlrpc_rqphase_move (request, request->rq_next_phase);return 1 ; }
fs/lustre/ptlrpc/niobuf.c +14 -16
@@ -244,8 +244,6 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req)int ptlrpc_unregister_bulk (struct ptlrpc_request *req,int async) {struct ptlrpc_bulk_desc *desc = req->rq_bulk; -wait_queue_head_t *wq; -int rc;LASSERT (!in_interrupt ());/* might sleep */
@@ -276,23 +274,23 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async)if (async)return 0 ; -if (req->rq_set) - wq = &req->rq_set->set_waitq; -else - wq = &req->rq_reply_waitq; -for (;;) { -/* Network access will complete in finite time but the HUGE +/* The wq argument is ignored by user-space wait_event macros */ +wait_queue_head_t *wq = (req->rq_set !=NULL ) ? + &req->rq_set->set_waitq : + &req->rq_reply_waitq; +/* +* Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish LNDs */ -int cnt =0 ; - -while (cnt <LONG_UNLINK && - (rc =wait_event_idle_timeout (*wq, - !ptlrpc_client_bulk_active (req), -HZ )) ==0 ) - cnt +=1 ; -if (rc >0 ) { +int seconds =LONG_UNLINK ; + +while (seconds >LONG_UNLINK && +wait_event_idle_timeout (*wq, + !ptlrpc_client_bulk_active (req), +HZ ) ==0 ) + seconds -=1 ; +if (seconds >0 ) {ptlrpc_rqphase_move (req, req->rq_next_phase);return 1 ; }
Signature · 1 line
1.8.3.1
PATCH 617/622 lustre: ptlrpc: use l_wait_event_abortable in ptlrpcd_add_reg()
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
Using wait_event_idle() will ignore signals which is not what we want in ptlrpcd_add_req(). Change it to l_wait_event_abortable().
WC-bug-id: https://jira.whamcloud.com/browse/LU-10467
Lustre-commit: ca6c35cab141 ("LU-10467 lustre: convert users of back_to_sleep()")
Reviewed-on: https://review.whamcloud.com/35980
--- fs/lustre/ptlrpc/ptlrpcd.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/lustre/ptlrpc/ptlrpcd.c +2 -2
@@ -235,8 +235,8 @@ void ptlrpcd_add_req(struct ptlrpc_request *req)if (wait_event_idle_timeout (req->rq_set_waitq, !req->rq_set,5 *HZ ) ==0 ) -wait_event_idle (req->rq_set_waitq, - !req->rq_set); +l_wait_event_abortable (req->rq_set_waitq, + !req->rq_set); }else if (req->rq_set) {/* * If we have a valid "rq_set", just reuse it to avoid double
Signature · 1 line
1.8.3.1
PATCH 618/622 lnet: use LIST_HEAD() for local lists.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
When declaring a local list head, instead of
struct list_head list; INIT_LIST_HEAD(&list);
use
LIST_HEAD(list);
which does both steps.
WC-bug-id: https://jira.whamcloud.com/browse/LU-9679
Lustre-commit: 135b5c0009e5 ("LU-9679 lnet: use LIST_HEAD() for local lists.")
Reviewed-on: https://review.whamcloud.com/36954
--- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 7 ++----- net/lnet/klnds/socklnd/socklnd_cb.c | 3 +-- net/lnet/lnet/api-ni.c | 20 +++++--------------- net/lnet/lnet/config.c | 28 ++++++++-------------------- net/lnet/lnet/lib-move.c | 30 ++++++++---------------------- net/lnet/lnet/net_fault.c | 14 ++++---------- net/lnet/lnet/peer.c | 8 ++------ net/lnet/lnet/router.c | 15 ++++----------- net/lnet/selftest/console.c | 4 +--- 9 files changed, 35 insertions(+), 94 deletions(-)
net/lnet/klnds/o2iblnd/o2iblnd_cb.c +2 -5
@@ -1384,11 +1384,9 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, {rwlock_t *glock = &kiblnd_data.kib_global_lock;char *reason =NULL ; -struct list_head txs; +LIST_HEAD (txs);unsigned long flags; -INIT_LIST_HEAD (&txs); -write_lock_irqsave (glock, flags);if (!peer_ni->ibp_reconnecting) {if (peer_ni->ibp_accepting)
@@ -2218,7 +2216,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, {struct kib_peer_ni *peer_ni = conn->ibc_peer;struct kib_tx *tx; -struct list_head txs; +LIST_HEAD (txs);unsigned long flags;int active;
@@ -2277,7 +2275,6 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, }/* grab pending txs while I have the lock */ -INIT_LIST_HEAD (&txs);list_splice_init (&peer_ni->ibp_tx_queue, &txs); if (!kiblnd_peer_active(peer_ni) || /* peer_ni has been deleted */
net/lnet/klnds/socklnd/socklnd_cb.c +1 -2
@@ -2491,14 +2491,13 @@ void ksocknal_write_callback(struct ksock_conn *conn)wait_queue_entry_t wait;struct ksock_conn *conn;struct ksock_sched *sched; -struct list_head enomem_conns; +LIST_HEAD (enomem_conns);int nenomem_conns;time64_t timeout;int i;int peer_index =0 ;time64_t deadline =ktime_get_seconds (); -INIT_LIST_HEAD (&enomem_conns);init_waitqueue_entry (&wait, current);spin_lock_bh (&ksocknal_data.ksnd_reaper_lock);
net/lnet/lnet/api-ni.c +5 -15
@@ -2062,11 +2062,9 @@ static void lnet_push_target_fini(void)lnet_shutdown_lndnets (void) {struct lnet_net *net; -struct list_head resend; +LIST_HEAD (resend);struct lnet_msg *msg, *tmp; -INIT_LIST_HEAD (&resend); -/* NB called holding the global mutex */ /* All quiet on the API front */
@@ -2202,7 +2200,7 @@ static void lnet_push_target_fini(void) {struct lnet_ni *ni;struct lnet_net *net_l =NULL ; -struct list_head local_ni_list; +LIST_HEAD (local_ni_list);int ni_count =0 ;u32 lnd_type;struct lnet_lnd *lnd;
@@ -2214,8 +2212,6 @@ static void lnet_push_target_fini(void)int peerrtrcredits = net->net_tunables.lct_peer_rtr_credits; -INIT_LIST_HEAD (&local_ni_list); -/* * make sure that this net is unique. If it isn't then * we are adding interfaces to an already existing network, and
@@ -2509,11 +2505,9 @@ void lnet_lib_exit(void)int ni_count;struct lnet_ping_buffer *pbuf;struct lnet_handle_md ping_mdh; -struct list_head net_head; +LIST_HEAD (net_head);struct lnet_net *net; -INIT_LIST_HEAD (&net_head); -mutex_lock (&the_lnet.ln_api_mutex);CDEBUG (D_OTHER ,"refs %d\n" , the_lnet.ln_refcount);
@@ -3098,9 +3092,7 @@ static int lnet_handle_legacy_ip2nets(char *ip2nets,struct lnet_net *net;char *nets;int rc; -struct list_head net_head; - -INIT_LIST_HEAD (&net_head); +LIST_HEAD (net_head); rc =lnet_parse_ip2nets (&nets, ip2nets);if (rc <0 )
@@ -3282,13 +3274,11 @@ int lnet_dyn_del_ni(struct lnet_ioctl_config_ni *conf) lnet_dyn_add_net(struct lnet_ioctl_config_data *conf) {struct lnet_net *net; -struct list_head net_head; +LIST_HEAD (net_head);int rc;struct lnet_ioctl_config_lnd_tunables tun;char *nets = conf->cfg_config_u.cfg_net.net_intf; -INIT_LIST_HEAD (&net_head); -/* Create a net/ni structures for the network string */ rc =lnet_parse_networks (&net_head, nets, use_tcp_bonding);if (rc <=0 )
net/lnet/lnet/config.c +8 -20
@@ -889,14 +889,12 @@ struct lnet_ni *static int lnet_str2tbs_sep (struct list_head *tbs,char *str) { -struct list_head pending; +LIST_HEAD (pending);char *sep;int nob;int i;struct lnet_text_buf *ltb; -INIT_LIST_HEAD (&pending); -/* Split 'str' into separate commands */ for (;;) {/* skip leading whitespace */
@@ -973,7 +971,7 @@ struct lnet_ni * lnet_str2tbs_expand(struct list_head *tbs, char *str) { char num[16]; -struct list_head pending; +LIST_HEAD (pending);char *sep;char *sep2;char *parsed;
@@ -985,8 +983,6 @@ struct lnet_ni *int nob;int scanned; -INIT_LIST_HEAD (&pending); - sep =strchr (str,'[' );if (!sep)/* nothing to expand */ return 0 ;
@@ -1097,8 +1093,8 @@ struct lnet_ni * {/* static scratch buffer OK (single threaded) */ static char cmd[LNET_SINGLE_TEXTBUF_NOB ]; -struct list_head nets; -struct list_head gateways; +LIST_HEAD (nets); +LIST_HEAD (gateways);struct list_head *tmp1;struct list_head *tmp2;u32 net;
@@ -1114,9 +1110,6 @@ struct lnet_ni *int got_hops =0 ;unsigned int priority =0 ; -INIT_LIST_HEAD (&gateways); -INIT_LIST_HEAD (&nets); -/* save a copy of the string for error messages */ strncpy (cmd, str,sizeof (cmd)); cmd[sizeof (cmd) -1 ] ='\0' ;
@@ -1260,13 +1253,11 @@ struct lnet_ni *int lnet_parse_routes (char *routes,int *im_a_router) { -struct list_head tbs; +LIST_HEAD (tbs);int rc =0 ; *im_a_router =0 ; -INIT_LIST_HEAD (&tbs); -if (lnet_str2tbs_sep (&tbs, routes) <0 ) {CERROR ("Error parsing routes\n" ); rc = -EINVAL ;
@@ -1453,9 +1444,9 @@ struct lnet_ni * {static char networks[LNET_SINGLE_TEXTBUF_NOB ];static char source[LNET_SINGLE_TEXTBUF_NOB ]; -struct list_head raw_entries; -struct list_head matched_nets; -struct list_head current_nets; +LIST_HEAD (raw_entries); +LIST_HEAD (matched_nets); +LIST_HEAD (current_nets);struct list_head *t;struct list_head *t2;struct lnet_text_buf *tb;
@@ -1467,15 +1458,12 @@ struct lnet_ni *int dup;int rc; -INIT_LIST_HEAD (&raw_entries);if (lnet_str2tbs_sep (&raw_entries, ip2nets) <0 ) {CERROR ("Error parsing ip2nets\n" );LASSERT (!lnet_tbnob);return -EINVAL ; } -INIT_LIST_HEAD (&matched_nets); -INIT_LIST_HEAD (¤t_nets); networks[0 ] =0 ; count =0 ; len =0 ;
net/lnet/lnet/lib-move.c +8 -22
@@ -166,7 +166,7 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,struct lnet_test_peer *tp;struct list_head *el;struct list_head *next; -struct list_head cull; +LIST_HEAD (cull);/* NB: use lnet_net_lock(0) to serialize operations on test peers */ if (threshold) {
@@ -184,9 +184,6 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,return 0 ; } -/* removing entries */ -INIT_LIST_HEAD (&cull); -lnet_net_lock (0 );list_for_each_safe (el, next, &the_lnet.ln_test_peers) {
@@ -216,11 +213,9 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,struct lnet_test_peer *tp;struct list_head *el;struct list_head *next; -struct list_head cull; +LIST_HEAD (cull);int fail =0 ; -INIT_LIST_HEAD (&cull); -/* NB: use lnet_net_lock(0) to serialize operations on test peers */ lnet_net_lock (0 );
@@ -2620,7 +2615,6 @@ struct lnet_mt_event_info {lnet_finalize_expired_responses (void) {struct lnet_libmd *md; -struct list_head local_queue;struct lnet_rsp_tracker *rspt, *tmp;ktime_t now;int i;
@@ -2629,7 +2623,7 @@ struct lnet_mt_event_info {return ;cfs_cpt_for_each (i,lnet_cpt_table ()) { -INIT_LIST_HEAD (&local_queue); +LIST_HEAD (local_queue);lnet_net_lock (i);if (!the_lnet.ln_mt_rstq[i]) {
@@ -2856,8 +2850,8 @@ struct lnet_mt_event_info {lnet_recover_local_nis (void) {struct lnet_mt_event_info *ev_info; -struct list_head processed_list; -struct list_head local_queue; +LIST_HEAD (processed_list); +LIST_HEAD (local_queue);struct lnet_handle_md mdh;struct lnet_ni *tmp;struct lnet_ni *ni;
@@ -2865,9 +2859,6 @@ struct lnet_mt_event_info {int healthv;int rc; -INIT_LIST_HEAD (&local_queue); -INIT_LIST_HEAD (&processed_list); -/* splice the recovery queue on a local queue. We will iterate * through the local queue and update it as needed. Once we're * done with the traversal, we'll splice the local queue back on
@@ -3091,11 +3082,9 @@ struct lnet_mt_event_info {lnet_clean_resendqs (void) {struct lnet_msg *msg, *tmp; -struct list_head msgs; +LIST_HEAD (msgs);int i; -INIT_LIST_HEAD (&msgs); -cfs_cpt_for_each (i,lnet_cpt_table ()) {lnet_net_lock (i);list_splice_init (the_lnet.ln_mt_resendqs[i], &msgs);
@@ -3114,8 +3103,8 @@ struct lnet_mt_event_info {lnet_recover_peer_nis (void) {struct lnet_mt_event_info *ev_info; -struct list_head processed_list; -struct list_head local_queue; +LIST_HEAD (processed_list); +LIST_HEAD (local_queue);struct lnet_handle_md mdh;struct lnet_peer_ni *lpni;struct lnet_peer_ni *tmp;
@@ -3123,9 +3112,6 @@ struct lnet_mt_event_info {int healthv;int rc; -INIT_LIST_HEAD (&local_queue); -INIT_LIST_HEAD (&processed_list); -/* Always use cpt 0 for locking across all interactions with * ln_mt_peerNIRecovq */
net/lnet/lnet/net_fault.c +4 -10
@@ -201,11 +201,9 @@ struct lnet_drop_rule { {struct lnet_drop_rule *rule;struct lnet_drop_rule *tmp; -struct list_head zombies; +LIST_HEAD (zombies);int n =0 ; -INIT_LIST_HEAD (&zombies); -lnet_net_lock (LNET_LOCK_EX );list_for_each_entry_safe (rule, tmp, &the_lnet.ln_drop_rules, dr_link) {if (rule->dr_attr.fa_src != src && src)
@@ -725,9 +723,8 @@ struct delay_daemon_data {lnet_delay_rule_check (void) {struct lnet_delay_rule *rule; -struct list_head msgs; +LIST_HEAD (msgs); -INIT_LIST_HEAD (&msgs);while (1 ) {if (list_empty (&delay_dd.dd_sched_rules))break ;
@@ -886,14 +883,11 @@ struct delay_daemon_data { {struct lnet_delay_rule *rule;struct lnet_delay_rule *tmp; -struct list_head rule_list; -struct list_head msg_list; +LIST_HEAD (rule_list); +LIST_HEAD (msg_list);int n =0 ;bool cleanup; -INIT_LIST_HEAD (&rule_list); -INIT_LIST_HEAD (&msg_list); -if (shutdown) { src =0 ; dst =0 ;
net/lnet/lnet/peer.c +2 -6
@@ -1912,9 +1912,7 @@ static void lnet_peer_discovery_complete(struct lnet_peer *lp) {struct lnet_msg *msg, *tmp;int rc =0 ; -struct list_head pending_msgs; - -INIT_LIST_HEAD (&pending_msgs); +LIST_HEAD (pending_msgs);CDEBUG (D_NET ,"Discovery complete. Dequeue peer %s\n" ,libcfs_nid2str (lp->lp_primary_nid));
@@ -3238,11 +3236,9 @@ static int lnet_peer_discovery_wait_for_work(void)static void lnet_resend_msgs (void ) {struct lnet_msg *msg, *tmp; -struct list_head resend; +LIST_HEAD (resend);int rc; -INIT_LIST_HEAD (&resend); -spin_lock (&the_lnet.ln_msg_resend_lock);list_splice (&the_lnet.ln_msg_resend, &resend);spin_unlock (&the_lnet.ln_msg_resend_lock);
net/lnet/lnet/router.c +4 -11
@@ -717,19 +717,16 @@ static void lnet_shuffle_seed(void)int lnet_del_route (u32 net,lnet_nid_t gw_nid) { -struct list_head rnet_zombies; +LIST_HEAD (rnet_zombies);struct lnet_remotenet *rnet;struct lnet_remotenet *tmp;struct list_head *rn_list;struct lnet_peer_ni *lpni;struct lnet_route *route; -struct list_head zombies; +LIST_HEAD (zombies);struct lnet_peer *lp =NULL ;int i =0 ; -INIT_LIST_HEAD (&rnet_zombies); -INIT_LIST_HEAD (&zombies); -CDEBUG (D_NET ,"Del route: net %s : gw %s\n" ,libcfs_net2str (net),libcfs_nid2str (gw_nid));
@@ -1152,14 +1149,12 @@ bool lnet_router_checker_active(void) lnet_rtrpool_free_bufs(struct lnet_rtrbufpool *rbp, int cpt) { int npages = rbp->rbp_npages; -struct list_head tmp; +LIST_HEAD (tmp);struct lnet_rtrbuf *rb;if (!rbp->rbp_nbuffers)/* not initialized or already freed */ return ; -INIT_LIST_HEAD (&tmp); -lnet_net_lock (cpt);list_splice_init (&rbp->rbp_msgs, &tmp);lnet_drop_routed_msgs_locked (&tmp, cpt);
@@ -1181,7 +1176,7 @@ bool lnet_router_checker_active(void)static int lnet_rtrpool_adjust_bufs (struct lnet_rtrbufpool *rbp,int nbufs,int cpt) { -struct list_head rb_list; +LIST_HEAD (rb_list);struct lnet_rtrbuf *rb;int num_rb;int num_buffers =0 ;
@@ -1213,8 +1208,6 @@ bool lnet_router_checker_active(void) rbp->rbp_req_nbuffers = nbufs;lnet_net_unlock (cpt); -INIT_LIST_HEAD (&rb_list); -/* * allocate the buffers on a local list first. If all buffers are * allocated successfully then join this list to the rbp buffer
net/lnet/selftest/console.c +1 -3
@@ -1484,12 +1484,10 @@ static void lstcon_group_ndlink_release(struct lstcon_group *, lstcon_ndlist_stat(struct list_head *ndlist, int timeout, struct list_head __user *result_up) { - struct list_head head; +LIST_HEAD (head);struct lstcon_rpc_trans *trans;int rc; -INIT_LIST_HEAD (&head); - rc =lstcon_rpc_trans_ndlist (ndlist, &head,LST_TRANS_STATQRY ,NULL ,NULL , &trans);if (rc) {
Signature · 1 line
1.8.3.1
PATCH 619/622 lustre: lustre: use LIST_HEAD() for local lists.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
When declaring a local list head, instead of
struct list_head list; INIT_LIST_HEAD(&list);
use
LIST_HEAD(list);
which does both steps.
WC-bug-id: https://jira.whamcloud.com/browse/LU-9679
Lustre-commit: 0098396983e1 ("LU-9679 lustre: use LIST_HEAD() for local lists.")
Reviewed-on: https://review.whamcloud.com/36955
--- fs/lustre/obdclass/lprocfs_status.c | 3 +-- fs/lustre/obdclass/lu_object.c | 6 ++---- fs/lustre/obdclass/obd_mount.c | 3 +-- fs/lustre/ptlrpc/client.c | 3 +-- fs/lustre/ptlrpc/service.c | 3 +-- 5 files changed, 6 insertions(+), 12 deletions(-)
fs/lustre/obdclass/lprocfs_status.c +1 -2
@@ -1885,7 +1885,7 @@ int lprocfs_wr_nosquash_nids(const char __user *buffer, unsigned long count, struct root_squash_info *squash, char *name) { char *kernbuf = NULL, *errmsg; -struct list_head tmp; +LIST_HEAD (tmp);int len = count;int rc;
@@ -1924,7 +1924,6 @@ int lprocfs_wr_nosquash_nids(const char __user *buffer, unsigned long count,return count; } -INIT_LIST_HEAD (&tmp);if (cfs_parse_nidlist (kernbuf, count, &tmp) <=0 ) { errmsg ="can't parse" ; rc = -EINVAL ;
fs/lustre/obdclass/lu_object.c +2 -4
@@ -361,7 +361,7 @@ static void lu_object_free(const struct lu_env *env, struct lu_object *o)struct lu_site *site;struct lu_object *scan;struct list_head *layers; -struct list_head splice; +LIST_HEAD (splice); site = o->lo_dev->ld_site; layers = &o->lo_header->loh_layers;
@@ -380,7 +380,6 @@ static void lu_object_free(const struct lu_env *env, struct lu_object *o)* necessary, because lu_object_header is freed together with the * top-level slice. */ -INIT_LIST_HEAD (&splice);list_splice_init (layers, &splice);while (!list_empty (&splice)) {/*
@@ -408,7 +407,7 @@ int lu_site_purge_objects(const struct lu_env *env, struct lu_site *s,struct lu_object_header *h;struct lu_object_header *temp;struct lu_site_bkt_data *bkt; -struct list_head dispose; +LIST_HEAD (dispose);int did_sth;unsigned int start =0 ;int count;
@@ -418,7 +417,6 @@ int lu_site_purge_objects(const struct lu_env *env, struct lu_site *s,if (OBD_FAIL_CHECK (OBD_FAIL_OBD_NO_LRU ))return 0 ; -INIT_LIST_HEAD (&dispose);/* * Under LRU list lock, scan LRU list and move unreferenced objects to * the dispose list, removing them from LRU and hash table.
fs/lustre/obdclass/obd_mount.c +1 -2
@@ -982,7 +982,7 @@ static bool lmd_find_delimiter(char *buf, char **endh)*/ static int lmd_parse_nidlist (char *buf,char **endh) { -struct list_head nidlist; +LIST_HEAD (nidlist);char *endp = buf;int rc =0 ;char tmp;
@@ -1000,7 +1000,6 @@ static int lmd_parse_nidlist(char *buf, char **endh) tmp = *endp; *endp ='\0' ; -INIT_LIST_HEAD (&nidlist);if (cfs_parse_nidlist (buf,strlen (buf), &nidlist) <=0 ) rc =1 ;cfs_free_nidlist (&nidlist);
fs/lustre/ptlrpc/client.c +1 -2
@@ -1715,13 +1715,12 @@ static inline int ptlrpc_set_producer(struct ptlrpc_request_set *set)int ptlrpc_check_set (const struct lu_env *env,struct ptlrpc_request_set *set) {struct ptlrpc_request *req, *next; -struct list_head comp_reqs; +LIST_HEAD (comp_reqs);int force_timer_recalc =0 ;if (atomic_read (&set->set_remaining) ==0 )return 1 ; -INIT_LIST_HEAD (&comp_reqs);list_for_each_entry_safe (req, next, &set->set_requests, rq_set_chain) {struct obd_import *imp = req->rq_import;int unregistered =0 ;
fs/lustre/ptlrpc/service.c +1 -2
@@ -1211,7 +1211,7 @@ static void ptlrpc_at_check_timed(struct ptlrpc_service_part *svcpt) {struct ptlrpc_at_array *array = &svcpt->scp_at_array;struct ptlrpc_request *rq, *n; -struct list_head work_list; +LIST_HEAD (work_list);u32 index, count;time64_t deadline;time64_t now =ktime_get_real_seconds ();
@@ -1244,7 +1244,6 @@ static void ptlrpc_at_check_timed(struct ptlrpc_service_part *svcpt)* We're close to a timeout, and we don't know how much longer the * server will take. Send early replies to everyone expiring soon. */ -INIT_LIST_HEAD (&work_list); deadline =-1 ;div_u64_rem (array->paa_deadline, array->paa_size, &index); count = array->paa_count;
Signature · 1 line
1.8.3.1
PATCH 620/622 lustre: handle: discard h_lock.
In reply to James SimmonsFrom: NeilBrown <[email protected]>
The h_lock spinlock is now only taken while bucket->lock is held. As a handle is associated with precisely one bucket, this means that h_lock can never be contended, so it isn't needed.
So discard h_lock.
Also discard an increasingly irrelevant comment in the declaration of struct portals_handle.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12542
Lustre-commit: 6acafe7ac4ef ("LU-12542 handle: discard h_lock.")
Reviewed-on: https://review.whamcloud.com/35863
--- fs/lustre/include/lustre_handles.h | 3 --- fs/lustre/obdclass/lustre_handles.c | 7 ------- 2 files changed, 10 deletions(-)
fs/lustre/include/lustre_handles.h +0 -3
@@ -62,10 +62,7 @@ struct portals_handle {u64 h_cookie;const char *h_owner;refcount_t h_ref; - -/* newly added fields to handle the RCU issue. -jxiong */ struct rcu_head h_rcu; -spinlock_t h_lock; };/* handles.c */
fs/lustre/obdclass/lustre_handles.c +0 -7
@@ -85,7 +85,6 @@ void class_handle_hash(struct portals_handle *h, const char *owner)spin_unlock (&handle_base_lock); h->h_owner = owner; -spin_lock_init (&h->h_lock); bucket = &handle_hash[h->h_cookie &HANDLE_HASH_MASK ];spin_lock (&bucket->lock);
@@ -108,13 +107,7 @@ static void class_handle_unhash_nolock(struct portals_handle *h)CDEBUG (D_INFO ,"removing object %p with handle %#llx from hash\n" , h, h->h_cookie); -spin_lock (&h->h_lock); -if (hlist_unhashed (&h->h_link)) { -spin_unlock (&h->h_lock); -return ; - }hlist_del_init_rcu (&h->h_link); -spin_unlock (&h->h_lock); }void class_handle_unhash (struct portals_handle *h)
Signature · 1 line
1.8.3.1
PATCH 621/622 lnet: remove lnd_query interface.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
The ->lnd_query interface is completely unused, and has been since commit 8e498d3f23ea ("LU-11300 lnet: peer aliveness")
So remove all mention of it.
WC-bug-id: https://jira.whamcloud.com/browse/LU-11300
Lustre-commit: 0d816af574b7 ("LU-11300 lnet: remove lnd_query interface.")
Reviewed-on: https://review.whamcloud.com/37337
--- include/linux/lnet/lib-types.h | 3 -- net/lnet/klnds/o2iblnd/o2iblnd.c | 32 ------------------- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 2 +- net/lnet/klnds/socklnd/socklnd.c | 62 ------------------------------------- net/lnet/lnet/api-ni.c | 3 -- 5 files changed, 1 insertion(+), 101 deletions(-)
include/linux/lnet/lib-types.h +0 -3
@@ -298,9 +298,6 @@ struct lnet_lnd {/* notification of peer down */ void (*lnd_notify_peer_down)(lnet_nid_t peer); -/* query of peer aliveness */ -void (*lnd_query)(struct lnet_ni *ni,lnet_nid_t peer,time64_t *when); -/* accept a new connection */ int (*lnd_accept)(struct lnet_ni *ni,struct socket *sock); };
net/lnet/klnds/o2iblnd/o2iblnd.c +0 -32
@@ -1128,37 +1128,6 @@ static int kiblnd_ctl(struct lnet_ni *ni, unsigned int cmd, void *arg)return rc; } -static void kiblnd_query (struct lnet_ni *ni,lnet_nid_t nid,time64_t *when) -{ -time64_t last_alive =0 ; -time64_t now =ktime_get_seconds (); -rwlock_t *glock = &kiblnd_data.kib_global_lock; -struct kib_peer_ni *peer_ni; -unsigned long flags; - -read_lock_irqsave (glock, flags); - - peer_ni =kiblnd_find_peer_locked (ni, nid); -if (peer_ni) - last_alive = peer_ni->ibp_last_alive; - -read_unlock_irqrestore (glock, flags); - -if (last_alive) - *when = last_alive; - -/* -* peer_ni is not persistent in hash, trigger peer_ni creation -* and connection establishment with a NULL tx -*/ -if (!peer_ni) -kiblnd_launch_tx (ni,NULL , nid); - -CDEBUG (D_NET ,"peer_ni %s %p, alive %lld secs ago\n" , -libcfs_nid2str (nid), peer_ni, - last_alive ? now - last_alive :-1 ); -} -static void kiblnd_free_pages (struct kib_pages *p) {int npages = p->ibp_npages;
@@ -3125,7 +3094,6 @@ static int kiblnd_startup(struct lnet_ni *ni) .lnd_startup = kiblnd_startup, .lnd_shutdown = kiblnd_shutdown, .lnd_ctl = kiblnd_ctl, - .lnd_query = kiblnd_query, .lnd_send = kiblnd_send, .lnd_recv = kiblnd_recv, };
net/lnet/klnds/o2iblnd/o2iblnd_cb.c +1 -1
@@ -2684,7 +2684,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,* attempts (active or passive) are in progress * NB: reconnect is still needed even when ibp_tx_queue is * empty if ibp_version != version because reconnect may be -* initiated by kiblnd_query() +* initiated. */ reconnect = (!list_empty (&peer_ni->ibp_tx_queue) || peer_ni->ibp_version != version) &&
net/lnet/klnds/socklnd/socklnd.c +0 -62
74 lines not on this page · raw message
net/lnet/lnet/api-ni.c +0 -3
@@ -2304,9 +2304,6 @@ static void lnet_push_target_fini(void)if (rc <0 ) goto failed1; -LASSERT (ni->ni_net->net_tunables.lct_peer_timeout <=0 || - ni->ni_net->net_lnd->lnd_query); -lnet_ni_addref (ni);list_add_tail (&ni->ni_netlist, &local_ni_list);
Signature · 1 line
1.8.3.1
PATCH 622/622 lnet: use conservative health timeouts
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
Use more conservative lnet_transaction_timeout and lnet_retry_count values by default. Currently with timeout=10 and retry=3 there is only a 3s window for the RPC to be sent before it is timed out. This has caused fault injection rather than fault tolerance. Increase the default timeout to 50s with retry=2, which is hopefully long enough to cover virtually all uses, but still allows LNet Health to be enabled by default and resend before Lustre times out itself.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13145
Lustre-commit: 361e9eaef13c ("LU-13145 lnet: use conservative health timeouts")
Reviewed-on: https://review.whamcloud.com/37430
--- net/lnet/lnet/api-ni.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
net/lnet/lnet/api-ni.c +2 -2
@@ -141,7 +141,7 @@ static int recovery_interval_set(const char *val, "Set to 1 to drop asymmetrical route messages.");#define LNET_TRANSACTION_TIMEOUT_NO_HEALTH_DEFAULT 50 -#define LNET_TRANSACTION_TIMEOUT_HEALTH_DEFAULT 10 +#define LNET_TRANSACTION_TIMEOUT_HEALTH_DEFAULT 50unsigned int lnet_transaction_timeout =LNET_TRANSACTION_TIMEOUT_HEALTH_DEFAULT ;static int transaction_to_set (const char *val,const struct kernel_param *kp);
@@ -156,7 +156,7 @@ static int recovery_interval_set(const char *val,MODULE_PARM_DESC (lnet_transaction_timeout,"Maximum number of seconds to wait for a peer response." ); -#define LNET_RETRY_COUNT_HEALTH_DEFAULT 3 +#define LNET_RETRY_COUNT_HEALTH_DEFAULT 2unsigned int lnet_retry_count =LNET_RETRY_COUNT_HEALTH_DEFAULT ;static int retry_count_set (const char *val,const struct kernel_param *kp);static struct kernel_param_ops param_ops_retry_count = {
Signature · 1 line
1.8.3.1
cover lustre: sync closely to 2.13.52
In reply to James SimmonsQuoted 2 lines · James Simmons
These patches need to be applied to the lustre-backport branch starting at commit
a436653f641e4b3e2841f38113620535e918dd3f.
Hi James et al,
I applied these patches on top of a43.... then added the other patches I had an looked for difference to my previous tree.
I found a bunch of improvements you had made, plus some errors that came through your patches, plus some other problems that already existed...
I've then added patches from OpenSFS master to get uptodate. The result is not on my github tree in the lustre/lustre branch. It has 226 patches on top of the set you posted.
git log 823691f8d49c~3..823691f8d49c
will show you some error I fixed, in case you are interested.
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20200424/d1f49362/attachment.sig>
cover lustre: sync closely to 2.13.52
Quoted 11 lines
Quoted 3 lines
These patches need to be applied to the lustre-backport branch starting at commit
a436653f641e4b3e2841f38113620535e918dd3f.Hi James et al,
I applied these patches on top of a43.... then added the other patches I had an looked for difference to my previous tree.
I found a bunch of improvements you had made, plus some errors that came through your patches, plus some other problems that already existed...
Quoted 7 lines
I've then added patches from OpenSFS master to get uptodate. The result is not on my github tree in the lustre/lustre branch. It has 226 patches on top of the set you posted.
git log 823691f8d49c~3..823691f8d49c
will show you some error I fixed, in case you are interested.
I grabbed those fixes and applied them to my tree. Its nice to see my tree, your tree, and the OpenSFS branch starting to merge. I updated my tree to the same OpenSFS commit as your tree and have started to look at difference. I noticed the sync with my tree didn't exactly match up. You lost a few changes. Some of the major changes missing are
1) LU-9679 llite: Discard LUSTRE_FPRIVATE() OpenSFS hash : 9e5cb57addbb5d7bc1596096821ad8dcac7a939b
2) LU-13274 uapi: make lnet UAPI headers C99 compliant OpenSFS hash: 742897a967cff5be53c447d14b17ae405c2b31f2
Also their are some kmem_cache bugs that causes several of the sanity test to crash the client. I need to do more comparing of our tree and sort out the changes that haven't landed or been pushed to the OpenSFS branch to figure out these regressions. I can easily see in the 4 to 8 weeks all the patches in your tree to land to OpenSFS as well as my tree. So we could sort everything out once everything has landed. I need to pull a few changes you have as well to my tree for cleanups like for the fid layer that seem to have gotten lost. Mostly likely from people doing cleanups while in staging.
Now for some important info. I noticed sanity test failing recently which I tracked down to me not updating my lustre utilities. This is going to be a problem so to make it painless I created a few patches to enable building the lustre utilies only against the linux client. If you apply the following patches:
https://review.whamcloud.com/#/c/38369
https://review.whamcloud.com/#/c/38370
https://review.whamcloud.com/#/c/38105
https://review.whamcloud.com/#/c/36603
https://review.whamcloud.com/#/c/34954
Now for 33603 patch it exposes a regression with LSOM and open_by_handle_at(). This patch removes the need for dot_lustre which is broken anyways for submounts (filesets). The 'dot_lustre' in the UAPI header is used for general utilites as well as the kernel server code. Since it had no use in the linux client it was nuked :-) Instead of restoring it we can resolve the LSOM issues for 33603 patch.
Once you apply the above patches just run:
sh autogen.sh
./configure --disable-modules --disable-server
make rpms
and you will have proper rpms to use with the linux client. I recommend after updating the linux client to update your utilities as well everytime.
cover lustre: sync closely to 2.13.52
Quoted 18 lines · James Simmons
Quoted 10 lines
Quoted 2 lines
These patches need to be applied to the lustre-backport branch starting at commit
a436653f641e4b3e2841f38113620535e918dd3f.Hi James et al,
I applied these patches on top of a43.... then added the other patches I had an looked for difference to my previous tree.
I found a bunch of improvements you had made, plus some errors that came through your patches, plus some other problems that already existed...Quoted 7 lines
I've then added patches from OpenSFS master to get uptodate. The result is not on my github tree in the lustre/lustre branch. It has 226 patches on top of the set you posted.
git log 823691f8d49c~3..823691f8d49c
will show you some error I fixed, in case you are interested.
Quoted 8 lines
I grabbed those fixes and applied them to my tree. Its nice to see my tree, your tree, and the OpenSFS branch starting to merge. I updated my tree to the same OpenSFS commit as your tree and have started to look at difference. I noticed the sync with my tree didn't exactly match up. You lost a few changes. Some of the major changes missing are
1) LU-9679 llite: Discard
LUSTRE_FPRIVATE()OpenSFS hash :9e5cb57addbb5d7bc1596096821ad8dcac7a939b
Thanks. I've also added
2bea4a7a3706 LU-5432 fld: don't loop forever on bogus FID sequences 2f66b4903516 LU-7768 fld: Do not retry fld request 8d4ef45e0780 LU-7524 fld: fld_clientlookup retries next target
which I had found by comparison myself but not applied yet.
Quoted 2 lines
2) LU-13274 uapi: make lnet UAPI headers C99 compliant OpenSFS hash:
742897a967cff5be53c447d14b17ae405c2b31f2
I have that...
Commit 619f523e5c36 ("lustre: uapi: make lnet UAPI headers C99 compliant")
What did I miss?
Quoted 11 lines
Also their are some
kmem_cachebugs that causes several of the sanity test to crash the client. I need to do more comparing of our tree and sort out the changes that haven't landed or been pushed to the OpenSFS branch to figure out these regressions. I can easily see in the 4 to 8 weeks all the patches in your tree to land to OpenSFS as well as my tree. So we could sort everything out once everything has landed. I need to pull a few changes you have as well to my tree for cleanups like for the fid layer that seem to have gotten lost. Mostly likely from people doing cleanups while in staging.
Quoted 8 lines
Now for some important info. I noticed sanity test failing recently which I tracked down to me not updating my lustre utilities. This is going to be a problem so to make it painless I created a few patches to enable building the lustre utilies only against the linux client. If you apply the following patches:
https://review.whamcloud.com/#/c/38369
https://review.whamcloud.com/#/c/38370
This one:
LU-12511 build: don't use OpenSFS UAPI headers with --disable-modules
doesn't seem like a good thing. It implies that the UAPI headers in OpenSFS give different results than the ones in Linux.
If this is true - we need to fix it. The UAPI needs to be stable.
Do you know what the important differences are.
Quoted 13 lines
https://review.whamcloud.com/#/c/38105
https://review.whamcloud.com/#/c/36603
https://review.whamcloud.com/#/c/34954Now for 33603 patch it exposes a regression with LSOM and
open_by_handle_at(). This patch removes the need fordot_lustrewhich is broken anyways for submounts (filesets). The 'dot_lustre' in the UAPI header is used for general utilites as well as the kernel server code. Since it had no use in the linux client it was nuked :-) Instead of restoring it we can resolve the LSOM issues for 33603 patch.Once you apply the above patches just run:
Quoted 7 lines
sh
autogen.sh
./configure --disable-modules --disable-server
make rpmsand you will have proper rpms to use with the linux client. I recommend after updating the linux client to update your utilities as well everytime.
Thanks from the thorough response. I'll keep this in mind when I next refresh my linux test setup.
Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20200429/cb1f1b9a/attachment.sig>