staging/lustre: patch bomb 3
PATCH 1/26 staging/lustre/hsm: handle file ownership and timestamps
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
In hsm_init_ucred() set the capability mask to CFS_CAP_FS_MASK thereby allowing the coordinator to swap layouts of files not owned by root. In mdt_orphan_open() raise CFS_CAP_DAC_OVERRIDE before creating the orphan filee in the MDT's local root. In mdt_hsm_release() set inode size and timestamps before getting the LOV xattr and ensure that the orphan file is created with the same UID and GID of the original. In lhsmtool_posix.c similalrly ensure that the volatile file used for restore has the same ownership, access and modification timestamps as the original.
Strengthen sanity-hsm test 24 to check that none of archive, release, or restore will change the access or modified timestamps. Add 24b to check that root can archive, release, and restore an ordinary user's files and that the user will be able to read a released file even if the parent directory is unwritable.
Lustre-change: http://review.whamcloud.com/7461
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3811
--- .../lustre/lustre/include/lustre/lustreapi.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/lustre/lustre/include/lustre/lustreapi.h +2 -2
@@ -313,8 +313,8 @@ extern int llapi_hsm_action_begin(struct hsm_copyaction_private **phcp, int restore_mdt_index, int restore_open_flags, bool is_error);extern int llapi_hsm_action_end (struct hsm_copyaction_private **phcp, -const struct hsm_extent *he,int flags, -int errval); +const struct hsm_extent *he, +int hp_flags,int errval);extern int llapi_hsm_action_progress (struct hsm_copyaction_private *hcp,const struct hsm_extent *he,int hp_flags); extern int llapi_hsm_action_get_dfid(const struct hsm_copyaction_private *hcp,
Signature · 1 line
1.7.9.5
cover staging/lustre: patch bomb 3
Hi Greg,
This is the last patch bomb for now. All 26 patches are ported from Lustre tree. And it syncs kernel Lustre client with Lustre tree up to commit (Revert "LU-1346 libcfs: replace CFS_CAP_XXX with kernel definition").
Now kernel client is still a few dozens patches away. I will close the gap in the next few days. My estimation is there will be another 20 patches or so and then we are in sync.
Thanks,
Tao
Andreas Dilger (1):
staging/lustre/seq: make seq_proc_write_common() safer
Andrew Perepechko (2):
staging/lustre/llite: speedup in unlink/rmdir staging/lustre/lprocfs: implement log2 using bitops
Andriy Skulysh (1):
staging/lustre/ldlm: Fix flock detection for different mounts
Bobi Jam (2):
staging/lustre/ldlm: MDT mount fails on MDS w/o MGS on it staging/lustre/lov: avoid subobj's coh_parent race
Bruno Faccini (2):
staging/lustre/ldlm: Fix a race during FLock handling staging/lustre/ldlm: refine LU-2665 patch for POSIX compliance
Christopher J. Morrone (1):
staging/lustre/changelogs: Correct KUC code max changelog msg size
Fan Yong (1):
staging/lustre/scrub: support dryrun mode OI scrub
James Simmons (2):
staging/lustre/autoconf: remove quota_on/quota_off checks staging/lustre/autoconf: remove LC_BI_HW_SEGMENTS test
Jinshan Xiong (3):
staging/lustre/llite: error setting max_cache_mb at mount time staging/lustre/llite: Truncate to restore file staging/lustre/osc: osc_extent_wait() shouldn't be interruptible
John L. Hammond (2):
staging/lustre/hsm: handle file ownership and timestamps staging/lustre/hsm: reprocess LDLM resource in mdt_hsm_release()
Keith Mannthey (1):
staging/lustre/ptlrpc: skip rpcs that fail ptl_send_rpc
Nikitas Angelinas (1):
staging/lustre/nrs: Fix a race condition in the ORR policy
Niu Yawei (1):
staging/lustre/clio: incorrect assertions in 'enable-invariants'
Oleg Drokin (1):
staging/lustre/clio: Do not shrink sublock at cancel
Sebastien Buisson (2):
staging/lustre/build: fix race issues thanks to oap_lock staging/lustre/build: fix 'NULL pointer dereference' errors
Swapnil Pimpale (2):
staging/lustre/dcache: Unsafe error handling arnd ll_splice_alias staging/lustre/ptlrpc: Return a meaningful status from ptlrpcd_init()
wang di (1):
staging/lustre/mdt: return EXDEV for cross-MDT rename
.../include/linux/libcfs/libcfs_kernelcomm.h | 2 + .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 2 +- drivers/staging/lustre/lnet/lnet/lib-move.c | 2 + drivers/staging/lustre/lnet/selftest/conctl.c | 51 +++++++++-------- drivers/staging/lustre/lustre/fid/lproc_fid.c | 14 +++-- drivers/staging/lustre/lustre/include/dt_object.h | 4 ++ .../lustre/lustre/include/linux/lustre_compat25.h | 47 --------------- .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- .../lustre/lustre/include/lustre/lustre_user.h | 3 + .../lustre/lustre/include/lustre/lustreapi.h | 4 +- drivers/staging/lustre/lustre/include/lustre_fid.h | 5 ++ drivers/staging/lustre/lustre/include/lustre_net.h | 4 +- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 60 +++++++++++++++++--- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 1 + drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 14 ++++- drivers/staging/lustre/lustre/llite/file.c | 5 +- .../staging/lustre/lustre/llite/llite_internal.h | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 57 +++++++++---------- drivers/staging/lustre/lustre/llite/lproc_llite.c | 4 +- drivers/staging/lustre/lustre/llite/namei.c | 51 +++++++---------- drivers/staging/lustre/lustre/llite/statahead.c | 27 +++++---- drivers/staging/lustre/lustre/llite/vvp_io.c | 5 +- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 2 +- drivers/staging/lustre/lustre/lov/lov_object.c | 12 ++-- drivers/staging/lustre/lustre/lov/lov_request.c | 2 +- drivers/staging/lustre/lustre/lov/lovsub_dev.c | 4 ++ drivers/staging/lustre/lustre/lov/lovsub_lock.c | 5 +- drivers/staging/lustre/lustre/mdc/mdc_lib.c | 38 ++++++------- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 4 +- drivers/staging/lustre/lustre/mdc/mdc_request.c | 21 ++++--- drivers/staging/lustre/lustre/mgc/libmgc.c | 4 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 18 ++++-- drivers/staging/lustre/lustre/obdclass/cl_page.c | 7 ++- .../lustre/lustre/obdclass/lprocfs_status.c | 30 +++++----- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 38 +++++++++---- drivers/staging/lustre/lustre/osc/osc_cache.c | 8 ++- drivers/staging/lustre/lustre/osc/osc_io.c | 2 + drivers/staging/lustre/lustre/ptlrpc/client.c | 1 + drivers/staging/lustre/lustre/ptlrpc/layout.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 13 +++-- drivers/staging/lustre/lustre/ptlrpc/sec_config.c | 10 ++-- 41 files changed, 335 insertions(+), 252 deletions(-)
Signature · 1 line
1.7.9.5
PATCH 3/26 staging/lustre/clio: incorrect assertions in 'enable-invariants'
From: Niu Yawei <[email protected]>
Fixed several incorrect assumptions in 'enable-invariants'.
Lustre-change: http://review.whamcloud.com/6832
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3521
--- drivers/staging/lustre/lustre/obdclass/cl_page.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/staging/lustre/lustre/obdclass/cl_page.c +4 -3
@@ -481,7 +481,7 @@ static inline int cl_page_invariant(const struct cl_page *pg) child = pg->cp_child; owner = pg->cp_owner; -return cl_page_in_use (pg) && +return cl_page_in_use_noref (pg) &&ergo (parent !=NULL , parent->cp_child == pg) &&ergo (child !=NULL , child->cp_parent == pg) && ergo(child != NULL, pg->cp_obj != child->cp_obj) &&
@@ -814,7 +814,7 @@ void cl_page_disown0(const struct lu_env *env, state = pg->cp_state;PINVRNT (env, pg, state ==CPS_OWNED || state ==CPS_FREEING ); -PINVRNT (env, pg,cl_page_invariant (pg)); +PINVRNT (env, pg,cl_page_invariant (pg) || state ==CPS_FREEING );cl_page_owner_clear (pg);if (state ==CPS_OWNED )
@@ -989,7 +989,8 @@ EXPORT_SYMBOL(cl_page_unassume);void cl_page_disown (const struct lu_env *env,struct cl_io *io,struct cl_page *pg) { -PINVRNT (env, pg,cl_page_is_owned (pg, io)); +PINVRNT (env, pg,cl_page_is_owned (pg, io) || + pg->cp_state ==CPS_FREEING ); pg =cl_page_top (pg); io =cl_io_top (io);
Signature · 1 line
1.7.9.5
PATCH 7/26 staging/lustre/ldlm: refine LU-2665 patch for POSIX compliance
In reply to Peng TaoFrom: Bruno Faccini <[email protected]>
Follow-on to patch introduced to fix LU-2665 ticket (Gerrit Change at http://review.whamcloud.com/6415 with Change-Id:
I8faa331712abeadee46eabe111ee1c23a05840d5).
Original patch introduced regressions against POSIX test suite (fcntl.18/fcntl.35 tests in LSB-VSX POSIX test suite at
http://www.opengroup.org/testing/linux-test/lsb-vsx.html),
so the idea is to only resend F_UNLCKs to have both LU-2665 bug and POSIX test suite happy.
Lustre-change: http://review.whamcloud.com/7453
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3701
--- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/staging/lustre/lustre/mdc/mdc_locks.c +3 -1
@@ -882,7 +882,9 @@ resend:can not rely on caller and this mainly for F_UNLCKs (explicits or automatically generated by Kernel to clean current FLocks upon exit) that can't be trashed */ -if ((rc == -EINTR ) || (rc == -ETIMEDOUT )) +if (((rc == -EINTR ) || (rc == -ETIMEDOUT )) && + (einfo->ei_type ==LDLM_FLOCK ) && + (einfo->ei_mode ==LCK_NL )) goto resend;return rc; }
Signature · 1 line
1.7.9.5
PATCH 6/26 staging/lustre/build: fix 'NULL pointer dereference' errors
In reply to Peng TaoFrom: Sebastien Buisson <[email protected]>
Fix 'NULL pointer dereference' defects found by Coverity version 6.5.3:
Dereference after null check (FORWARD_NULL)
For instance, Passing null pointer to a function which dereferences it.
Dereference before null check (REVERSE_INULL) Null-checking variable suggests that it may be null, but it has already been dereferenced on all paths leading to the check. Dereference null return value (NULL_RETURNS)
Lustre-change: http://review.whamcloud.com/4720
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2217
--- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 2 +- drivers/staging/lustre/lnet/lnet/lib-move.c | 2 + drivers/staging/lustre/lnet/selftest/conctl.c | 51 ++++++++++---------- .../lustre/lustre/include/lustre/lustre_user.h | 3 ++ drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 14 +++++- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 2 +- drivers/staging/lustre/lustre/lov/lov_request.c | 2 +- drivers/staging/lustre/lustre/mdc/mdc_lib.c | 38 +++++++-------- drivers/staging/lustre/lustre/mgc/mgc_request.c | 10 +++- .../lustre/lustre/obdclass/lprocfs_status.c | 24 ++++----- drivers/staging/lustre/lustre/ptlrpc/layout.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/sec_config.c | 10 ++-- 12 files changed, 94 insertions(+), 66 deletions(-)
drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +1 -1
@@ -3217,7 +3217,7 @@ kiblnd_startup (lnet_ni_t *ni)return 0 ; failed: -if (net->ibn_dev ==NULL && ibdev !=NULL ) +if (net !=NULL && net->ibn_dev ==NULL && ibdev !=NULL )kiblnd_destroy_dev (ibdev);kiblnd_shutdown (ni);
drivers/staging/lustre/lnet/lnet/lib-move.c +2 -0
@@ -162,6 +162,7 @@ lnet_iov_nob(unsigned int niov, struct iovec *iov) {unsigned int nob =0 ; +LASSERT (niov ==0 || iov !=NULL );while (niov-- >0 ) nob += (iov++)->iov_len;
@@ -281,6 +282,7 @@ lnet_kiov_nob(unsigned int niov, lnet_kiov_t *kiov) {unsigned int nob =0 ; +LASSERT (niov ==0 || kiov !=NULL );while (niov-- >0 ) nob += (kiov++)->kiov_len;
drivers/staging/lustre/lnet/selftest/conctl.c +26 -25
71 lines not on this page · raw message
drivers/staging/lustre/lustre/include/lustre/lustre_user.h +3 -0
@@ -451,6 +451,9 @@ static inline void obd_str2uuid(struct obd_uuid *uuid, const char *tmp)/* For printf's only, make sure uuid is terminated */ static inline char *obd_uuid2str (const struct obd_uuid *uuid) { +if (uuid ==NULL ) +return NULL ; +if (uuid->uuid[sizeof (*uuid) -1 ] !='\0' ) {/* Obviously not safe, but for printfs, no real harm done... we're always null-terminated, even in a race. */
drivers/staging/lustre/lustre/ldlm/ldlm_request.c +12 -2
@@ -439,8 +439,13 @@ int ldlm_cli_enqueue_local(struct ldlm_namespace *ns, lock->l_policy_data = *policy;if (client_cookie !=NULL ) lock->l_client_cookie = *client_cookie; -if (type ==LDLM_EXTENT ) +if (type ==LDLM_EXTENT ) { +/* extent lock without policy is a bug */ +if (policy ==NULL ) +LBUG (); + lock->l_req_extent = policy->l_extent; + } err =ldlm_lock_enqueue (ns, &lock, policy, flags);if (unlikely (err !=ELDLM_OK ))
@@ -892,8 +897,13 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, lock->l_policy_data = *policy; } -if (einfo->ei_type ==LDLM_EXTENT ) +if (einfo->ei_type ==LDLM_EXTENT ) { +/* extent lock without policy is a bug */ +if (policy ==NULL ) +LBUG (); + lock->l_req_extent = policy->l_extent; + }LDLM_DEBUG (lock,"client-side enqueue START, flags %llx\n" , *flags); }
drivers/staging/lustre/lustre/lmv/lmv_obd.c +1 -1
@@ -244,7 +244,7 @@ static int lmv_connect(const struct lu_env *env,* and MDC stuff will be called directly, for instance while reading * ../mdc/../kbytesfree procfs file, etc. */ -if (data->ocd_connect_flags &OBD_CONNECT_REAL ) +if (data !=NULL && (data->ocd_connect_flags &OBD_CONNECT_REAL )) rc =lmv_check_connect (obd);if (rc && lmv_proc_dir) {
drivers/staging/lustre/lustre/lov/lov_request.c +1 -1
@@ -185,7 +185,7 @@ int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) cfs_time_seconds(1), NULL, NULL); rc =l_wait_event (waitq,lov_check_set (lov, ost_idx), &lwi); -if (tgt !=NULL && tgt->ltd_active) +if (tgt->ltd_active)return 1 ;return 0 ;
drivers/staging/lustre/lustre/mdc/mdc_lib.c +19 -19
@@ -223,29 +223,29 @@ void mdc_open_pack(struct ptlrpc_request *req, struct md_op_data *op_data, rec->cr_fsuid =from_kuid (&init_user_ns,current_fsuid ()); rec->cr_fsgid =from_kgid (&init_user_ns,current_fsgid ()); rec->cr_cap =cfs_curproc_cap_pack (); -if (op_data !=NULL ) { - rec->cr_fid1 = op_data->op_fid1; - rec->cr_fid2 = op_data->op_fid2; - } rec->cr_mode = mode; cr_flags =mds_pack_open_flags (flags, mode); rec->cr_rdev = rdev; - rec->cr_time = op_data->op_mod_time; - rec->cr_suppgid1 = op_data->op_suppgids[0 ]; - rec->cr_suppgid2 = op_data->op_suppgids[1 ]; - rec->cr_bias = op_data->op_bias; rec->cr_umask =current_umask (); - rec->cr_old_handle = op_data->op_handle; - -mdc_pack_capa (req, &RMF_CAPA1 , op_data->op_capa1); -/* the next buffer is child capa, which is used for replay, -* will be packed from the data in reply message. */ - -if (op_data->op_name) { - tmp =req_capsule_client_get (&req->rq_pill, &RMF_NAME ); -LOGL0 (op_data->op_name, op_data->op_namelen, tmp); -if (op_data->op_bias &MDS_CREATE_VOLATILE ) - cr_flags |=MDS_OPEN_VOLATILE ; +if (op_data !=NULL ) { + rec->cr_fid1 = op_data->op_fid1; + rec->cr_fid2 = op_data->op_fid2; + rec->cr_time = op_data->op_mod_time; + rec->cr_suppgid1 = op_data->op_suppgids[0 ]; + rec->cr_suppgid2 = op_data->op_suppgids[1 ]; + rec->cr_bias = op_data->op_bias; + rec->cr_old_handle = op_data->op_handle; + +mdc_pack_capa (req, &RMF_CAPA1 , op_data->op_capa1); +/* the next buffer is child capa, which is used for replay, +* will be packed from the data in reply message. */ + +if (op_data->op_name) { + tmp =req_capsule_client_get (&req->rq_pill, &RMF_NAME ); +LOGL0 (op_data->op_name, op_data->op_namelen, tmp); +if (op_data->op_bias &MDS_CREATE_VOLATILE ) + cr_flags |=MDS_OPEN_VOLATILE ; + } }if (lmm) {
drivers/staging/lustre/lustre/mgc/mgc_request.c +9 -1
@@ -352,7 +352,15 @@ static int config_log_add(struct obd_device *obd, char *logname,LASSERT (lsi->lsi_lmd);if (!(lsi->lsi_lmd->lmd_flags &LMD_FLG_NOIR )) {struct config_llog_data *recover_cld; - *strrchr (seclogname,'-' ) =0 ; + ptr =strrchr (seclogname,'-' ); +if (ptr !=NULL ) { + *ptr =0 ; + } +else { +CERROR ("sptlrpc log name not correct: %s" , seclogname); +config_log_put (cld); +return -EINVAL ; + } recover_cld =config_recover_log_add (obd, seclogname, cfg, sb);if (IS_ERR (recover_cld))GOTO (out_err3, rc =PTR_ERR (recover_cld));
drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +13 -11
@@ -1847,17 +1847,19 @@ int lprocfs_write_frac_u64_helper(const char *buffer, unsigned long count, } units =1 ; -switch (*end) { -case 'p' :case 'P' : - units <<=10 ; -case 't' :case 'T' : - units <<=10 ; -case 'g' :case 'G' : - units <<=10 ; -case 'm' :case 'M' : - units <<=10 ; -case 'k' :case 'K' : - units <<=10 ; +if (end !=NULL ) { +switch (*end) { +case 'p' :case 'P' : + units <<=10 ; +case 't' :case 'T' : + units <<=10 ; +case 'g' :case 'G' : + units <<=10 ; +case 'm' :case 'M' : + units <<=10 ; +case 'k' :case 'K' : + units <<=10 ; + } }/* Specified units override the multiplier */ if (units)
drivers/staging/lustre/lustre/ptlrpc/layout.c +1 -1
@@ -1893,7 +1893,7 @@ swabber_dumper_helper(struct req_capsule *pill,return ;swabber (value);ptlrpc_buf_set_swabbed (pill->rc_req, inout, offset); -if (dump) { +if (dump && field->rmf_dumper) {CDEBUG (D_RPCTRACE ,"Dump of swabbed field %s " "follows\n" , field->rmf_name); field->rmf_dumper (value);
drivers/staging/lustre/lustre/ptlrpc/sec_config.c +6 -4
@@ -745,11 +745,13 @@ void sptlrpc_conf_log_update_begin(const char *logname)mutex_lock (&sptlrpc_conf_lock); conf =sptlrpc_conf_get (fsname,0 ); -if (conf && conf->sc_local) { -LASSERT (conf->sc_updated ==0 ); -sptlrpc_conf_free_rsets (conf); +if (conf) { +if (conf->sc_local) { +LASSERT (conf->sc_updated ==0 ); +sptlrpc_conf_free_rsets (conf); + } + conf->sc_modified =0 ; } - conf->sc_modified =0 ;mutex_unlock (&sptlrpc_conf_lock); }
Signature · 1 line
1.7.9.5
PATCH 12/26 staging/lustre/ldlm: Fix flock detection for different mounts
In reply to Peng TaoFrom: Andriy Skulysh <[email protected]>
Deadlock can happen when 2 processes take concurrent locks on files situated in different mountpoints.
Modify flock detection algorithm to distinguish process by pair PID+NID instead of PID+export.
It is done by searching for a blocking owner in all OBD's exports with the same NID.
Lustre-change: http://review.whamcloud.com/3276
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1601
--- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 45 +++++++++++++++++++++-- 1 file changed, 41 insertions(+), 4 deletions(-)
drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +41 -4
@@ -161,6 +161,31 @@ ldlm_flock_destroy(struct ldlm_lock *lock, ldlm_mode_t mode, __u64 flags)* one client holds a lock on something and want a lock on something * else and at the same time another client has the opposite situation). */ + +struct ldlm_flock_lookup_cb_data { +__u64 *bl_owner; +struct ldlm_lock *lock; +struct obd_export *exp; +}; + +static int ldlm_flock_lookup_cb (struct cfs_hash *hs,struct cfs_hash_bd *bd, +struct hlist_node *hnode,void *data) +{ +struct ldlm_flock_lookup_cb_data *cb_data = data; +struct obd_export *exp =cfs_hash_object (hs, hnode); +struct ldlm_lock *lock; + + lock =cfs_hash_lookup (exp->exp_flock_hash, cb_data->bl_owner); +if (lock ==NULL ) +return 0 ; + +/* Stop on first found lock. Same process can't sleep twice */ + cb_data->lock = lock; + cb_data->exp =class_export_get (exp); + +return 1 ; +} +static int ldlm_flock_deadlock (struct ldlm_lock *req,struct ldlm_lock *bl_lock) {
@@ -175,16 +200,26 @@ ldlm_flock_deadlock(struct ldlm_lock *req, struct ldlm_lock *bl_lock)class_export_get (bl_exp);while (1 ) { +struct ldlm_flock_lookup_cb_data cb_data = { + .bl_owner = &bl_owner, + .lock =NULL , + .exp =NULL };struct obd_export *bl_exp_new;struct ldlm_lock *lock =NULL ;struct ldlm_flock *flock; -if (bl_exp->exp_flock_hash !=NULL ) - lock =cfs_hash_lookup (bl_exp->exp_flock_hash, - &bl_owner); +if (bl_exp->exp_flock_hash !=NULL ) { +cfs_hash_for_each_key (bl_exp->exp_obd->obd_nid_hash, + &bl_exp->exp_connection->c_peer.nid, + ldlm_flock_lookup_cb, &cb_data); + lock = cb_data.lock; + }if (lock ==NULL )break ; +class_export_put (bl_exp); + bl_exp = cb_data.exp; +LASSERT (req != lock); flock = &lock->l_policy_data.l_flock;LASSERT (flock->owner == bl_owner);
@@ -198,7 +233,9 @@ ldlm_flock_deadlock(struct ldlm_lock *req, struct ldlm_lock *bl_lock)if (bl_exp->exp_failed)break ; -if (bl_owner == req_owner && bl_exp == req_exp) { +if (bl_owner == req_owner && + (bl_exp->exp_connection->c_peer.nid == + req_exp->exp_connection->c_peer.nid)) {class_export_put (bl_exp);return 1 ; }
Signature · 1 line
1.7.9.5
PATCH 13/26 staging/lustre/nrs: Fix a race condition in the ORR policy
In reply to Peng TaoFrom: Nikitas Angelinas <[email protected]>
Checking the atomic oo_ref in nrs_orr_hop_put_free() and then taking the bucket lock leaves a race window open, by which a second thread can attempt to free an already-freed nrs_orr_object. Fix this, and change the hash bucket lock type from an rwlock to a spinlock, as there are now as many calls on the write path, as there are on the read path. Rehashing is not used on the hashes, so libcfs_hash API usage can also be simplified.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3430
Lustre-change: http://review.whamcloud.com/7623
--- drivers/staging/lustre/lustre/include/lustre_net.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/lustre/lustre/include/lustre_net.h +2 -2
@@ -1569,7 +1569,7 @@ struct nrs_orr_data {struct ptlrpc_nrs_resource od_res;cfs_binheap_t *od_binheap;struct cfs_hash *od_obj_hash; -struct kmem_cache *od_cache; +struct kmem_cache *od_cache;/** * Used when a new scheduling round commences, in order to synchronize * all object or OST batches with the new round number.
@@ -1626,7 +1626,7 @@ struct nrs_orr_object {* scheduling RPCs */ struct nrs_orr_key oo_key; -atomic_t oo_ref; +long oo_ref;/** * Round Robin quantum; the maximum number of RPCs that are allowed to * be scheduled for the object or OST in a single batch of each round.
Signature · 1 line
1.7.9.5
PATCH 11/26 staging/lustre/ptlrpc: Return a meaningful status from ptlrpcd_init()
In reply to Peng TaoFrom: Swapnil Pimpale <[email protected]>
This patch has the following:
1) Fix for the return value from ptlrpcd_init(). It will now return a correct status instead of returning zero always.
2) ptlrpcd_addref() should not increment ptlrpcd_users on error.
3) Added code in a mdc_setup() and mgc_setup() to test the return value of ptlrpcd_addref() and return on error.
Lustre-change: http://review.whamcloud.com/7522
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3808
--- drivers/staging/lustre/lustre/mdc/mdc_request.c | 15 +++++++++------ drivers/staging/lustre/lustre/mgc/libmgc.c | 4 +++- drivers/staging/lustre/lustre/mgc/mgc_request.c | 8 +++++--- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 13 ++++++++----- 4 files changed, 25 insertions(+), 15 deletions(-)
drivers/staging/lustre/lustre/mdc/mdc_request.c +9 -6
@@ -2364,20 +2364,22 @@ struct ldlm_valblock_ops inode_lvbo = {static int mdc_setup (struct obd_device *obd,struct lustre_cfg *cfg) { -struct client_obd *cli = &obd->u.cli; -struct lprocfs_static_vars lvars = {0 }; -int rc; +struct client_obd *cli = &obd->u.cli; +struct lprocfs_static_vars lvars = {0 }; +int rc;OBD_ALLOC (cli->cl_rpc_lock,sizeof (*cli->cl_rpc_lock));if (!cli->cl_rpc_lock)return -ENOMEM ;mdc_init_rpc_lock (cli->cl_rpc_lock); -ptlrpcd_addref (); + rc =ptlrpcd_addref (); +if (rc <0 ) +GOTO (err_rpc_lock, rc);OBD_ALLOC (cli->cl_close_lock,sizeof (*cli->cl_close_lock));if (!cli->cl_close_lock) -GOTO (err_rpc_lock, rc = -ENOMEM ); +GOTO (err_ptlrpcd_decref, rc = -ENOMEM );mdc_init_rpc_lock (cli->cl_close_lock); rc =client_obd_setup (obd, cfg);
@@ -2403,9 +2405,10 @@ static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) err_close_lock:OBD_FREE (cli->cl_close_lock,sizeof (*cli->cl_close_lock)); +err_ptlrpcd_decref: +ptlrpcd_decref (); err_rpc_lock:OBD_FREE (cli->cl_rpc_lock,sizeof (*cli->cl_rpc_lock)); -ptlrpcd_decref ();return rc; }
drivers/staging/lustre/lustre/mgc/libmgc.c +3 -1
@@ -57,7 +57,9 @@ static int mgc_setup(struct obd_device *obd, struct lustre_cfg *lcfg) {int rc; -ptlrpcd_addref (); + rc =ptlrpcd_addref (); +if (rc <0 ) +return rc; rc =client_obd_setup (obd, lcfg);if (rc)
drivers/staging/lustre/lustre/mgc/mgc_request.c +5 -3
@@ -852,10 +852,12 @@ static int mgc_cleanup(struct obd_device *obd)static int mgc_setup (struct obd_device *obd,struct lustre_cfg *lcfg) { -struct lprocfs_static_vars lvars; -int rc; +struct lprocfs_static_vars lvars; +int rc; -ptlrpcd_addref (); + rc =ptlrpcd_addref (); +if (rc <0 ) +return rc; rc =client_obd_setup (obd, lcfg);if (rc)
drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +8 -5
@@ -727,9 +727,9 @@ static void ptlrpcd_fini(void)static int ptlrpcd_init (void ) { -int nthreads =num_online_cpus (); -char name[16 ]; -int size, i =-1 , j, rc =0 ; +int nthreads =num_online_cpus (); +char name[16 ]; +int size, i =-1 , j, rc =0 ;if (max_ptlrpcds >0 && max_ptlrpcds < nthreads) nthreads = max_ptlrpcds;
@@ -786,7 +786,7 @@ out: ptlrpcds =NULL ; } -return 0 ; +return rc; }int ptlrpcd_addref (void )
@@ -794,8 +794,11 @@ int ptlrpcd_addref(void)int rc =0 ;mutex_lock (&ptlrpcd_mutex); -if (++ptlrpcd_users ==1 ) +if (++ptlrpcd_users ==1 ) { rc =ptlrpcd_init (); +if (rc <0 ) + ptlrpcd_users--; + }mutex_unlock (&ptlrpcd_mutex);return rc; }
Signature · 1 line
1.7.9.5
PATCH 17/26 staging/lustre/changelogs: Correct KUC code max changelog msg size
In reply to Peng TaoFrom: "Christopher J. Morrone" <[email protected]>
The kernel to userspace communication routines (KUC) allocate and limit the maximum cs_buf size to CR_MAXSIZE. However this fails to account for the fact that the buffer is assumed to begin with a struct kuc_hdr. To allocate and account for that space, we introduce a new define, KUC_CHANGELOG_MSG_MAXSIZE.
Lustre-change: http://review.whamcloud.com/7406
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3587
--- .../include/linux/libcfs/libcfs_kernelcomm.h | 2 ++ drivers/staging/lustre/lustre/mdc/mdc_request.c | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-)
drivers/staging/lustre/include/linux/libcfs/libcfs_kernelcomm.h +2 -0
@@ -61,6 +61,8 @@ struct kuc_hdr {__u16 kuc_msglen;/* Including header */ } __attribute__((aligned(sizeof(__u64)))); +#define KUC_CHANGELOG_MSG_MAXSIZE (sizeof(struct kuc_hdr)+CR_MAXSIZE) +#define KUC_MAGIC 0x191C/*Lustre9etLinC */ #define KUC_FL_BLOCK 0x01/* Wait for send */
drivers/staging/lustre/lustre/mdc/mdc_request.c +3 -3
@@ -1452,7 +1452,7 @@ static struct kuc_hdr *changelog_kuc_hdr(char *buf, int len, int flags) {struct kuc_hdr *lh = (struct kuc_hdr *)buf; -LASSERT (len <=CR_MAXSIZE ); +LASSERT (len <=KUC_CHANGELOG_MSG_MAXSIZE ); lh->kuc_magic =KUC_MAGIC ; lh->kuc_transport =KUC_TRANSPORT_CHANGELOG ;
@@ -1525,7 +1525,7 @@ static int mdc_changelog_send_thread(void *csdata)CDEBUG (D_CHANGELOG ,"changelog to fp=%p start " LPU64 "\n" , cs->cs_fp, cs->cs_startrec); -OBD_ALLOC (cs->cs_buf,CR_MAXSIZE ); +OBD_ALLOC (cs->cs_buf,KUC_CHANGELOG_MSG_MAXSIZE );if (cs->cs_buf ==NULL )GOTO (out, rc = -ENOMEM );
@@ -1562,7 +1562,7 @@ out:if (ctxt)llog_ctxt_put (ctxt);if (cs->cs_buf) -OBD_FREE (cs->cs_buf,CR_MAXSIZE ); +OBD_FREE (cs->cs_buf,KUC_CHANGELOG_MSG_MAXSIZE );OBD_FREE_PTR (cs);return rc; }
Signature · 1 line
1.7.9.5
PATCH 22/26 staging/lustre/osc: osc_extent_wait() shouldn't be interruptible
In reply to Peng TaoFrom: Jinshan Xiong <[email protected]>
Otherwise it will hit the assertion at cl_lock.c:
cl_lock.c:1967:discard_cb())
ASSERTION( (!(page->cp_type == CPT_CACHEABLE) || (!PageWriteback(cl_page_vmpage(env, page)))) ) failed:
This is because in osc_lock_flush() we have to make sure the IO is finished before discarding the pages.
Lustre-change: http://review.whamcloud.com/5419
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2779
--- drivers/staging/lustre/lustre/osc/osc_cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/lustre/lustre/osc/osc_cache.c +1 -1
@@ -891,7 +891,7 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext,"%s: wait ext to %d timedout, recovery in progress?\n" , osc_export(obj)->exp_obd->obd_name, state); - lwi =LWI_INTR (LWI_ON_SIGNAL_NOOP ,NULL ); + lwi =LWI_INTR (NULL ,NULL ); rc =l_wait_event (ext->oe_waitq,extent_wait_cb (ext, state), &lwi); }
Signature · 1 line
1.7.9.5
PATCH 21/26 staging/lustre/clio: Do not shrink sublock at cancel
In reply to Peng TaoFrom: Oleg Drokin <[email protected]>
Shrinking sublock at ldlm lock cancel time means whoever happened to attach to this lock just before will reenqueue the wrong lock.
Lustre-change: http://review.whamcloud.com/7569
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3027
--- drivers/staging/lustre/lustre/lov/lovsub_lock.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/staging/lustre/lustre/lov/lovsub_lock.c +4 -1
@@ -208,7 +208,10 @@ int lov_sublock_modify(const struct lu_env *env, struct lov_lock *lov, pd->cld_mode = parent_descr->cld_mode; pd->cld_gid = parent_descr->cld_gid;lovsub_lock_descr_map (d, subobj->lso_super, subobj->lso_index, pd); - lov->lls_sub[idx].sub_got = *d; + +/* LU-3027: only update extent of lock */ + lov->lls_sub[idx].sub_got.cld_start = d->cld_start; + lov->lls_sub[idx].sub_got.cld_end = d->cld_end;/* * Notify top-lock about modification, if lock description changes * materially.
Signature · 1 line
1.7.9.5
PATCH 26/26 staging/lustre/autoconf: remove LC_BI_HW_SEGMENTS test
In reply to Peng TaoFrom: James Simmons <[email protected]>
bio.bi_hw_segments has been removed from the upstream kernel since v2.6.27
We can remove the test and dead code.
Lustre-change: http://review.whamcloud.com/5370
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2800
--- .../lustre/lustre/include/linux/lustre_compat25.h | 4 ---- 1 file changed, 4 deletions(-)
drivers/staging/lustre/lustre/include/linux/lustre_compat25.h +0 -4
@@ -117,10 +117,6 @@ static inline void ll_set_fs_pwd(struct fs_struct *fs, struct vfsmount *mnt,#define ll_d_hlist_for_each_entry (dentry, i_dentry, alias) \ hlist_for_each_entry(dentry, i_dentry, alias) - -#define bio_hw_segments (q, bio) 0 - -#define ll_pagevec_init (pv, cold) do {} while (0)#define ll_pagevec_add (pv, pg) (0)#define ll_pagevec_lru_add_file (pv) do {} while (0)
Signature · 1 line
1.7.9.5
PATCH 25/26 staging/lustre/autoconf: remove quota_on/quota_off checks
In reply to Peng TaoFrom: James Simmons <[email protected]>
With the rewrite of the Lustre quota code we no longer need ll_quota_[on/off]. Those obsolete methods touched linux kernel internals that have changed over time. With ll_quota_[on/off[ gone we can remove all the quota autoconf that was required.
Lustre-change: http://review.whamcloud.com/5367
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2800
--- .../lustre/lustre/include/linux/lustre_compat25.h | 43 -------------------- 1 file changed, 43 deletions(-)
drivers/staging/lustre/lustre/include/linux/lustre_compat25.h +0 -43
@@ -106,49 +106,6 @@ static inline void ll_set_fs_pwd(struct fs_struct *fs, struct vfsmount *mnt,#define SLAB_DESTROY_BY_RCU 0#endif - - -static inline int -ll_quota_on (struct super_block *sb,int off,int ver,char *name,int remount) -{ -int rc; - -if (sb->s_qcop->quota_on) { -struct path path; - - rc =kern_path (name,LOOKUP_FOLLOW , &path); -if (!rc) -return rc; - rc = sb->s_qcop->quota_on (sb, off, ver - , &path - ); -path_put (&path); -return rc; - } -else -return -ENOSYS ; -} - -static inline int ll_quota_off (struct super_block *sb,int off,int remount) -{ -if (sb->s_qcop->quota_off) { -return sb->s_qcop->quota_off (sb, off - ); - } -else -return -ENOSYS ; -} - - -# define ll_vfs_dq_init dquot_initialize -# define ll_vfs_dq_drop dquot_drop -# define ll_vfs_dq_transfer dquot_transfer -# define ll_vfs_dq_off (sb, remount) dquot_suspend(sb, -1) - - - - -#define queue_max_phys_segments (rq) queue_max_segments(rq)#define queue_max_hw_segments (rq) queue_max_segments(rq)
Signature · 1 line
1.7.9.5
PATCH 24/26 staging/lustre/lprocfs: implement log2 using bitops
In reply to Peng TaoFrom: Andrew Perepechko <[email protected]>
This patch implements log2 using fls.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3496
Lustre-change: http://review.whamcloud.com/6757
--- .../lustre/lustre/obdclass/lprocfs_status.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +3 -3
@@ -1961,10 +1961,10 @@ EXPORT_SYMBOL(lprocfs_oh_tally);void lprocfs_oh_tally_log2 (struct obd_histogram *oh,unsigned int value) { -unsigned int val; +unsigned int val =0 ; -for (val =0 ; ((1 << val) < value) && (val <=OBD_HIST_MAX ); val++) - ; +if (likely (value !=0 )) + val =min (fls (value -1 ),OBD_HIST_MAX );lprocfs_oh_tally (oh, val); }
Signature · 1 line
1.7.9.5
PATCH 23/26 staging/lustre/seq: make seq_proc_write_common() safer
In reply to Peng TaoFrom: Andreas Dilger <[email protected]>
Don't allow seq_proc_write_common() to specify arbitrary ranges, since this can permanently corrupt the sequence controller and/or sequnece server. That would allow duplicate FID allocation, or possibly prevent any new files to be created or servers to be added to the filesystem.
Instead, limit the sequence range that can be written via /proc to a subset of the sequence range currently allocated to that node. Add the "clear" keyword to allow dropping the entire local sequence and force a new one to be fetched from the sequence server.
Lustre-change: http://review.whamcloud.com/7123
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3642
--- drivers/staging/lustre/lustre/fid/lproc_fid.c | 14 ++++++++++---- .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- 2 files changed, 11 insertions(+), 5 deletions(-)
drivers/staging/lustre/lustre/fid/lproc_fid.c +10 -4
@@ -56,22 +56,28 @@#ifdef LPROCFS /* -* Note: this function is only used for testing, it is no safe for production -* use. +* Reduce the SEQ range allocated to a node to a strict subset of the range +* currently-allocated SEQ range. If the specified range is "clear", then +* drop all allocated sequences and request a new one from the master. +* +* Note: this function should only be used for testing, it is not necessarily +* safe for production use. */ static int lprocfs_fid_write_common (const char *buffer,unsigned long count,struct lu_seq_range *range) { -struct lu_seq_range tmp; +struct lu_seq_range tmp = {0 , };int rc;LASSERT (range !=NULL ); +/* of the form "[0x0000000240000400 - 0x000000028000400]" */ rc =sscanf (buffer,"[%llx - %llx]\n" , (long long unsigned *)&tmp.lsr_start, (long long unsigned *)&tmp.lsr_end); -if (rc !=2 || !range_is_sane (&tmp) ||range_is_zero (&tmp)) +if (!range_is_sane (&tmp) ||range_is_zero (&tmp) || + tmp.lsr_start < range->lsr_start || tmp.lsr_end > range->lsr_end)return -EINVAL ; *range = tmp;return 0 ;
drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +1 -1
@@ -264,7 +264,7 @@ static inline __u64 range_space(const struct lu_seq_range *range)static inline void range_init (struct lu_seq_range *range) { - range->lsr_start = range->lsr_end = range->lsr_index =0 ; +memset (range,0 ,sizeof (*range)); }/**
Signature · 1 line
1.7.9.5
PATCH 20/26 staging/lustre/hsm: reprocess LDLM resource in mdt_hsm_release()
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
In mdt_hsm_release() manually reprocess the LDLM resource (the file being released) after cancelling the lease (exclusive open lock on the same file).
Lustre-change: http://review.whamcloud.com/7621
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3931
--- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 1 + 1 file changed, 1 insertion(+)
drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +1 -0
@@ -2091,6 +2091,7 @@ void ldlm_cancel_locks_for_export(struct obd_export *exp)cfs_hash_for_each_empty (exp->exp_lock_hash, ldlm_cancel_locks_for_export_cb, &ecl); } +EXPORT_SYMBOL (ldlm_reprocess_all);/** * Downgrade an exclusive lock.
Signature · 1 line
1.7.9.5
PATCH 19/26 staging/lustre/mdt: return EXDEV for cross-MDT rename
In reply to Peng TaoFrom: wang di <[email protected]>
If target does not exist, and source directory is remote directory, it should return -EXDEV.
Add test in test_24x to verify it.
Create local directory for test_214, since cross-MDT rename is not allowed yet.
Lustre-change: http://review.whamcloud.com/7579
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3894
--- drivers/staging/lustre/lustre/include/lustre_fid.h | 5 +++++ 1 file changed, 5 insertions(+)
drivers/staging/lustre/lustre/include/lustre_fid.h +5 -0
@@ -289,6 +289,11 @@ static inline int fid_is_quota(const struct lu_fid *fid)fid_seq (fid) ==FID_SEQ_QUOTA_GLB ; } +static inline int fid_is_name_llog (const struct lu_fid *fid) +{ +return fid_seq (fid) ==FID_SEQ_LLOG_NAME ; +} +static inline int fid_is_namespace_visible (const struct lu_fid *fid) {const __u64 seq =fid_seq (fid);
Signature · 1 line
1.7.9.5
PATCH 18/26 staging/lustre/scrub: support dryrun mode OI scrub
In reply to Peng TaoFrom: Fan Yong <[email protected]>
The administrator can detect whether there are any inconsistent OI mappings but without repairing them via the command:
lctl lfsck_start --dryrun on -M <$device>
Under default mode, dryrun is "off". When the device is mounted, the initial OI scrub will detect whether there are upgrade case, or file-level backup/resotre case, or OI file(s) recreated case. If found some inconsistency, it will trigger non-dryrun mode OI scrub automatically. If the administrator does NOT want to such auto non-dryrun OI scrub to avoid some unexpected repairing, he (or she) can specify "-o noscrub" when mount the device.
Lustre-change: http://review.whamcloud.com/7720
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3935
--- drivers/staging/lustre/lustre/include/dt_object.h | 4 ++++ 1 file changed, 4 insertions(+)
drivers/staging/lustre/lustre/include/dt_object.h +4 -0
@@ -622,6 +622,7 @@ struct dt_index_operations {enum dt_otable_it_valid {DOIV_ERROR_HANDLE =0x0001 , +DOIV_DRYRUN =0x0002 , };enum dt_otable_it_flags {
@@ -633,6 +634,9 @@ enum dt_otable_it_flags {/* There is up layer component uses the iteration. */ DOIF_OUTUSED =0x0004 , + +/* Check only without repairing. */ + DOIF_DRYRUN = 0x0008, };/* otable based iteration needs to use the common DT interation APIs.
Signature · 1 line
1.7.9.5
PATCH 16/26 staging/lustre/lov: avoid subobj's coh_parent race
In reply to Peng TaoFrom: Bobi Jam <[email protected]>
* during a file lov object initialization, we need protect the access and change of its subobj->coh_parent, since it could be another layout change race there, which makes an unreferenced lovsub obj in the site object hash table.
* dump lovsub objects in the site if the lovsub device reference > 0 during its finalization phase.
Lustre-change: http://review.whamcloud.com/6105
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1480
--- drivers/staging/lustre/lustre/lov/lov_object.c | 12 ++++++++---- drivers/staging/lustre/lustre/lov/lovsub_dev.c | 4 ++++ 2 files changed, 12 insertions(+), 4 deletions(-)
drivers/staging/lustre/lustre/lov/lov_object.c +8 -4
@@ -122,13 +122,13 @@ static struct cl_object *lov_sub_find(const struct lu_env *env, }static int lov_init_sub (const struct lu_env *env,struct lov_object *lov, -struct cl_object *stripe, -struct lov_layout_raid0 *r0,int idx) +struct cl_object *stripe,struct lov_layout_raid0 *r0, +int idx) {struct cl_object_header *hdr;struct cl_object_header *subhdr;struct cl_object_header *parent; -struct lov_oinfo *oinfo; +struct lov_oinfo *oinfo;int result;if (OBD_FAIL_CHECK (OBD_FAIL_LOV_INIT )) {
@@ -144,7 +144,6 @@ static int lov_init_sub(const struct lu_env *env, struct lov_object *lov, hdr =cl_object_header (lov2cl (lov)); subhdr =cl_object_header (stripe); - parent = subhdr->coh_parent; oinfo = lov->lo_lsm->lsm_oinfo[idx]; CDEBUG(D_INODE, DFID"@%p[%d] -> "DFID"@%p: ostid: "DOSTID
@@ -153,8 +152,12 @@ static int lov_init_sub(const struct lu_env *env, struct lov_object *lov,PFID (&hdr->coh_lu.loh_fid), hdr,POSTID (&oinfo->loi_oi), oinfo->loi_ost_idx, oinfo->loi_ost_gen); +/* reuse ->coh_attr_guard to protect coh_parent change */ +spin_lock (&subhdr->coh_attr_guard); + parent = subhdr->coh_parent;if (parent ==NULL ) { subhdr->coh_parent = hdr; +spin_unlock (&subhdr->coh_attr_guard); subhdr->coh_nesting = hdr->coh_nesting +1 ;lu_object_ref_add (&stripe->co_lu,"lov-parent" , lov); r0->lo_sub[idx] =cl2lovsub (stripe);
@@ -166,6 +169,7 @@ static int lov_init_sub(const struct lu_env *env, struct lov_object *lov,struct lov_object *old_lov;unsigned int mask =D_INODE ; +spin_unlock (&subhdr->coh_attr_guard); old_obj =lu_object_locate (&parent->coh_lu, &lov_device_type);LASSERT (old_obj !=NULL ); old_lov =cl2lov (lu2cl (old_obj));
drivers/staging/lustre/lustre/lov/lovsub_dev.c +4 -0
@@ -131,6 +131,10 @@ static struct lu_device *lovsub_device_free(const struct lu_env *env,struct lovsub_device *lsd =lu2lovsub_dev (d);struct lu_device *next =cl2lu_dev (lsd->acid_next); +if (atomic_read (&d->ld_ref) && d->ld_site) { +LIBCFS_DEBUG_MSG_DATA_DECL (msgdata,D_ERROR ,NULL ); +lu_site_print (env, d->ld_site, &msgdata, lu_cdebug_printer); + }cl_device_fini (lu2cl_dev (d));OBD_FREE_PTR (lsd);return next;
Signature · 1 line
1.7.9.5
PATCH 15/26 staging/lustre/llite: Truncate to restore file
In reply to Peng TaoFrom: Jinshan Xiong <[email protected]>
Truncate up is safe so it won't trigger restore.
Copy optimization for truncate down - only copy the part under truncate length. If a file is truncated to zero usually it'll be followed by write so I choose to restore the file and set correct stripe information.
Lustre-change: http://review.whamcloud.com/7505
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3817
--- drivers/staging/lustre/lustre/llite/file.c | 5 +- .../staging/lustre/lustre/llite/llite_internal.h | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 57 ++++++++++---------- drivers/staging/lustre/lustre/llite/vvp_io.c | 5 +- 4 files changed, 34 insertions(+), 35 deletions(-)
drivers/staging/lustre/lustre/llite/file.c +3 -2
@@ -3661,7 +3661,7 @@ again:/** * This function send a restore request to the MDT */ -int ll_layout_restore (struct inode *inode) +int ll_layout_restore (struct inode *inode,loff_t offset,__u64 length) {struct hsm_user_request *hur;int len, rc;
@@ -3677,7 +3677,8 @@ int ll_layout_restore(struct inode *inode) hur->hur_request.hr_flags =0 ;memcpy (&hur->hur_user_item[0 ].hui_fid, &ll_i2info (inode)->lli_fid,sizeof (hur->hur_user_item[0 ].hui_fid)); - hur->hur_user_item[0 ].hui_extent.length =-1 ; + hur->hur_user_item[0 ].hui_extent.offset = offset; + hur->hur_user_item[0 ].hui_extent.length = length; hur->hur_request.hr_itemcount =1 ; rc =obd_iocontrol (LL_IOC_HSM_REQUEST ,cl_i2sbi (inode)->ll_md_exp, len, hur,NULL );
drivers/staging/lustre/lustre/llite/llite_internal.h +1 -1
@@ -1615,7 +1615,7 @@ enum {int ll_layout_conf (struct inode *inode,const struct cl_object_conf *conf);int ll_layout_refresh (struct inode *inode,__u32 *gen); -int ll_layout_restore (struct inode *inode); +int ll_layout_restore (struct inode *inode,loff_t start,__u64 length);int ll_xattr_init (void );void ll_xattr_fini (void );
drivers/staging/lustre/lustre/llite/llite_lib.c +27 -30
94 lines not on this page · raw message
drivers/staging/lustre/lustre/llite/vvp_io.c +3 -2
@@ -118,6 +118,7 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios)struct cl_io *io = ios->cis_io;struct cl_object *obj = io->ci_obj;struct ccc_io *cio =cl2ccc_io (env, ios); +struct inode *inode =ccc_object_inode (obj);CLOBINVRNT (env, obj,ccc_object_invariant (obj));
@@ -133,7 +134,7 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios)/* file was detected release, we need to restore it * before finishing the io */ - rc =ll_layout_restore (ccc_object_inode (obj)); + rc =ll_layout_restore (inode,0 ,OBD_OBJECT_EOF );/* if restore registration failed, no restart, * we will return -ENODATA */ /* The layout will change after restore, so we need to
@@ -158,7 +159,7 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios)__u32 gen =0 ;/* check layout version */ -ll_layout_refresh (ccc_object_inode (obj), &gen); +ll_layout_refresh (inode, &gen); io->ci_need_restart = cio->cui_layout_gen != gen;if (io->ci_need_restart) { CDEBUG(D_VFSTRACE,
Signature · 1 line
1.7.9.5
PATCH 14/26 staging/lustre/ptlrpc: skip rpcs that fail ptl_send_rpc
In reply to Peng TaoFrom: Keith Mannthey <[email protected]>
ptl_send_rpc is not dealing with -ENOMEM in some situations. When the ptl_send_rpc fails we need set error and skip further processing or trigger and LBUG
Original patch from Peng Tao.
Lustre-change: http://review.whamcloud.com/7411
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3698
--- drivers/staging/lustre/lustre/ptlrpc/client.c | 1 + 1 file changed, 1 insertion(+)
drivers/staging/lustre/lustre/ptlrpc/client.c +1 -0
@@ -1715,6 +1715,7 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set)spin_lock (&req->rq_lock); req->rq_net_err =1 ;spin_unlock (&req->rq_lock); +continue ; }/* need to reset the timeout */ force_timer_recalc =1 ;
Signature · 1 line
1.7.9.5
PATCH 10/26 staging/lustre/ldlm: MDT mount fails on MDS w/o MGS on it
In reply to Peng TaoFrom: Bobi Jam <[email protected]>
If we specify multiple --mgsnode for a MDT, when we start MDS upon it while MGS is no the other node, the MGC import connection will always select the local nid (which is one of the candidate mgsnode) since it think its the closest connection.
This patch treats further --mgsnode nids as failover nids, so that multiple import connections are added for the MGC import.
Lustre-change: http://review.whamcloud.com/7509
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3829
--- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 38 +++++++++++++------- 1 file changed, 26 insertions(+), 12 deletions(-)
drivers/staging/lustre/lustre/obdclass/obd_mount.c +26 -12
@@ -332,12 +332,13 @@ int lustre_start_mgc(struct super_block *sb)sprintf (niduuid,"%s_%x" , mgcname, i);if (IS_SERVER (lsi)) { ptr = lsi->lsi_lmd->lmd_mgs; +CDEBUG (D_MOUNT ,"mgs nids %s.\n" , ptr);if (IS_MGS (lsi)) {/* Use local nids (including LO) */ lnet_process_id_t id;while ((rc =LNetGetId (i++, &id)) != -ENOENT ) { - rc =do_lcfg (mgcname, id.nid, -LCFG_ADD_UUID , niduuid,0 ,0 ,0 ); + rc =do_lcfg (mgcname, id.nid,LCFG_ADD_UUID , + niduuid,0 ,0 ,0 ); } }else {/* Use mgsnode= nids */
@@ -349,19 +350,30 @@ int lustre_start_mgc(struct super_block *sb)CERROR ("No MGS nids given.\n" );GOTO (out_free, rc = -EINVAL ); } +/* +* LU-3829. +* Here we only take the first mgsnid as its primary +* serving mgs node, the rest mgsnid will be taken as +* failover mgs node, otherwise they would be takens +* as multiple nids of a single mgs node. +*/ while (class_parse_nid (ptr, &nid, &ptr) ==0 ) { - rc =do_lcfg (mgcname, nid, -LCFG_ADD_UUID , niduuid,0 ,0 ,0 ); - i++; + rc =do_lcfg (mgcname, nid,LCFG_ADD_UUID , + niduuid,0 ,0 ,0 ); +if (rc ==0 ) { + i =1 ; +break ; + } } } }else {/* client */ /* Use nids from mount line: uml1,1@elan:uml2,2@elan:/lustre */ ptr = lsi->lsi_lmd->lmd_dev;while (class_parse_nid (ptr, &nid, &ptr) ==0 ) { - rc =do_lcfg (mgcname, nid, -LCFG_ADD_UUID , niduuid,0 ,0 ,0 ); - i++; + rc =do_lcfg (mgcname, nid,LCFG_ADD_UUID , + niduuid,0 ,0 ,0 ); +if (rc ==0 ) + ++i;/* Stop at the first failover nid */ if (*ptr ==':' )break ;
@@ -394,16 +406,18 @@ int lustre_start_mgc(struct super_block *sb)sprintf (niduuid,"%s_%x" , mgcname, i); j =0 ;while (class_parse_nid_quiet (ptr, &nid, &ptr) ==0 ) { - j++; - rc =do_lcfg (mgcname, nid, -LCFG_ADD_UUID , niduuid,0 ,0 ,0 ); + rc =do_lcfg (mgcname, nid,LCFG_ADD_UUID , + niduuid,0 ,0 ,0 ); +if (rc ==0 ) + ++j;if (*ptr ==':' )break ; }if (j >0 ) { rc =do_lcfg (mgcname,0 ,LCFG_ADD_CONN , niduuid,0 ,0 ,0 ); - i++; +if (rc ==0 ) + ++i; }else {/* at ":/fsname" */ break ;
Signature · 1 line
1.7.9.5
PATCH 9/26 staging/lustre/llite: error setting max_cache_mb at mount time
In reply to Peng TaoFrom: Jinshan Xiong <[email protected]>
The root cause is that when max_cache_mb conf parameter is applied, the client isn't connected to the OST yet so that sbi->ll_dt_exp is NULL. However, it's not necessary to shrink the cache memory in this case so success should be returned.
Lustre-change: http://review.whamcloud.com/7194
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3676
--- drivers/staging/lustre/lustre/llite/lproc_llite.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/lustre/lustre/llite/lproc_llite.c +2 -2
@@ -393,8 +393,8 @@ static ssize_t ll_max_cached_mb_seq_write(struct file *file, const char *buffer,return -ERANGE ; } -if (sbi->ll_dt_exp ==NULL ) -return -ENODEV ; +if (sbi->ll_dt_exp ==NULL )/* being initialized */ +GOTO (out, rc =0 );spin_lock (&sbi->ll_lock); diff = pages_number - cache->ccc_lru_max;
Signature · 1 line
1.7.9.5
PATCH 8/26 staging/lustre/llite: speedup in unlink/rmdir
In reply to Peng TaoFrom: Andrew Perepechko <[email protected]>
Assume dchild argument is fully initialized in ->unlink and ->rmdir callbacks, so additional lookup for ELC is not needed.
Lustre-change: http://review.whamcloud.com/6648
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3473
--- drivers/staging/lustre/lustre/llite/namei.c | 30 +++++++++------------------ 1 file changed, 10 insertions(+), 20 deletions(-)
drivers/staging/lustre/lustre/llite/namei.c +10 -20
@@ -942,22 +942,6 @@ static int ll_mkdir_generic(struct inode *dir, struct qstr *name,return err; } -/* Try to find the child dentry by its name. -If found, put the result fid into @fid. */ -static void ll_get_child_fid (struct inode * dir,struct qstr *name, -struct lu_fid *fid) -{ -struct dentry *parent, *child; - - parent = ll_d_hlist_entry(dir->i_dentry, struct dentry, d_alias); - child =d_lookup (parent, name); -if (child) { -if (child->d_inode) - *fid = *ll_inode2fid (child->d_inode); -dput (child); - } -} -static int ll_rmdir_generic (struct inode *dir,struct dentry *dparent,struct dentry *dchild,struct qstr *name) {
@@ -976,7 +960,8 @@ static int ll_rmdir_generic(struct inode *dir, struct dentry *dparent,if (IS_ERR (op_data))return PTR_ERR (op_data); -ll_get_child_fid (dir, name, &op_data->op_fid3); +if (dchild !=NULL && dchild->d_inode !=NULL ) + op_data->op_fid3 = *ll_inode2fid (dchild->d_inode); op_data->op_fid2 = op_data->op_fid3; rc =md_unlink (ll_i2sbi (dir)->ll_md_exp, op_data, &request);ll_finish_md_op_data (op_data);
@@ -1117,7 +1102,9 @@ static int ll_unlink_generic(struct inode *dir, struct dentry *dparent,if (IS_ERR (op_data))return PTR_ERR (op_data); -ll_get_child_fid (dir, name, &op_data->op_fid3); +if (dchild !=NULL && dchild->d_inode !=NULL ) + op_data->op_fid3 = *ll_inode2fid (dchild->d_inode); + op_data->op_fid2 = op_data->op_fid3; rc =md_unlink (ll_i2sbi (dir)->ll_md_exp, op_data, &request);ll_finish_md_op_data (op_data);
@@ -1157,8 +1144,11 @@ static int ll_rename_generic(struct inode *src, struct dentry *src_dparent,if (IS_ERR (op_data))return PTR_ERR (op_data); -ll_get_child_fid (src, src_name, &op_data->op_fid3); -ll_get_child_fid (tgt, tgt_name, &op_data->op_fid4); +if (src_dchild !=NULL && src_dchild->d_inode !=NULL ) + op_data->op_fid3 = *ll_inode2fid (src_dchild->d_inode); +if (tgt_dchild !=NULL && tgt_dchild->d_inode !=NULL ) + op_data->op_fid4 = *ll_inode2fid (tgt_dchild->d_inode); + err =md_rename (sbi->ll_md_exp, op_data, src_name->name, src_name->len, tgt_name->name, tgt_name->len, &request);
Signature · 1 line
1.7.9.5
PATCH 5/26 staging/lustre/dcache: Unsafe error handling arnd ll_splice_alias
In reply to Peng TaoFrom: Swapnil Pimpale <[email protected]>
Callers of ll_splice_alias() should not assign the returned pointer to the dentry since it can be an err pointer. Fixed the above bug using a temporary dentry pointer. This temporary pointer is assigned to dentry only if ll_splice_alias has not returned an err pointer.
Lustre-change: http://review.whamcloud.com/7460
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3807
--- drivers/staging/lustre/lustre/llite/namei.c | 21 ++++++++++-------- drivers/staging/lustre/lustre/llite/statahead.c | 27 +++++++++++++---------- 2 files changed, 27 insertions(+), 21 deletions(-)
drivers/staging/lustre/lustre/llite/namei.c +12 -9
@@ -428,12 +428,12 @@ struct dentry *ll_splice_alias(struct inode *inode, struct dentry *de)int ll_lookup_it_finish (struct ptlrpc_request *request,struct lookup_intent *it,void *data) { -struct it_cb_data *icbd = data; -struct dentry **de = icbd->icbd_childp; -struct inode *parent = icbd->icbd_parent; -struct inode *inode =NULL ; -__u64 bits =0 ; -int rc; +struct it_cb_data *icbd = data; +struct dentry **de = icbd->icbd_childp; +struct inode *parent = icbd->icbd_parent; +struct inode *inode =NULL ; +__u64 bits =0 ; +int rc;/* NB 1 request reference will be taken away by ll_intent_lock() * when I return */
@@ -460,9 +460,12 @@ int ll_lookup_it_finish(struct ptlrpc_request *request,* Atoimc_open may passin hashed dentries for open. */ if (d_unhashed (*de)) { - *de =ll_splice_alias (inode, *de); -if (IS_ERR (*de)) -return PTR_ERR (*de); +struct dentry *alias; + + alias =ll_splice_alias (inode, *de); +if (IS_ERR (alias)) +return PTR_ERR (alias); + *de = alias; }if (!it_disposition (it,DISP_LOOKUP_NEG )) {
drivers/staging/lustre/lustre/llite/statahead.c +15 -12
@@ -1500,14 +1500,14 @@ ll_sai_unplug(struct ll_statahead_info *sai, struct ll_sa_entry *entry)int do_statahead_enter (struct inode *dir,struct dentry **dentryp,int only_unplug) { -struct ll_inode_info *lli =ll_i2info (dir); -struct ll_statahead_info *sai = lli->lli_sai; -struct dentry *parent; -struct ll_sa_entry *entry; -struct ptlrpc_thread *thread; -struct l_wait_info lwi = {0 }; -int rc =0 ; -struct ll_inode_info *plli; +struct ll_inode_info *lli =ll_i2info (dir); +struct ll_statahead_info *sai = lli->lli_sai; +struct dentry *parent; +struct ll_sa_entry *entry; +struct ptlrpc_thread *thread; +struct l_wait_info lwi = {0 }; +int rc =0 ; +struct ll_inode_info *plli;LASSERT (lli->lli_opendir_pid ==current_pid ());
@@ -1585,12 +1585,15 @@ int do_statahead_enter(struct inode *dir, struct dentry **dentryp, ll_inode2fid(inode), &bits);if (rc ==1 ) {if ((*dentryp)->d_inode ==NULL ) { - *dentryp =ll_splice_alias (inode, - *dentryp); -if (IS_ERR (*dentryp)) { +struct dentry *alias; + + alias =ll_splice_alias (inode, + *dentryp); +if (IS_ERR (alias)) {ll_sai_unplug (sai, entry); -return PTR_ERR (*dentryp); +return PTR_ERR (alias); } + *dentryp = alias; }else if ((*dentryp)->d_inode != inode) {/* revalidate, but inode is recreated */ CDEBUG(D_READA,
Signature · 1 line
1.7.9.5
PATCH 4/26 staging/lustre/ldlm: Fix a race during FLock handling
In reply to Peng TaoFrom: Bruno Faccini <[email protected]>
Protect against race where lock could have been just destroyed due to overlap, in ldlm_process_flock_lock().
Easy reproducer is BULL's NFS Locktests in pthread mode.
(http://nfsv4.bullopensource.org/tools/tests/locktest.php)
Lustre-change: http://review.whamcloud.com/7134
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1126
--- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +10 -5
@@ -653,11 +653,6 @@ ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) granted:OBD_FAIL_TIMEOUT (OBD_FAIL_LDLM_CP_CB_WAIT ,10 ); -if (lock->l_flags &LDLM_FL_DESTROYED ) { -LDLM_DEBUG (lock,"client-side enqueue waking up: destroyed" ); -return 0 ; - } -if (lock->l_flags &LDLM_FL_FAILED ) {LDLM_DEBUG (lock,"client-side enqueue waking up: failed" );return -EIO ;
@@ -673,6 +668,16 @@ granted:lock_res_and_lock (lock); + +/* Protect against race where lock could have been just destroyed +* due to overlap in ldlm_process_flock_lock(). +*/ +if (lock->l_flags &LDLM_FL_DESTROYED ) { +unlock_res_and_lock (lock); +LDLM_DEBUG (lock,"client-side enqueue waking up: destroyed" ); +return 0 ; + } +/* take lock off the deadlock detection hash list. */ ldlm_flock_blocking_unlink (lock);
Signature · 1 line
1.7.9.5
PATCH 2/26 staging/lustre/build: fix race issues thanks to oap_lock
In reply to Peng TaoFrom: Sebastien Buisson <[email protected]>
Fix 'data race condition' defects found by Coverity version 6.5.0:
Data race condition (MISSING_LOCK)
Accessing variable without holding lock. Elsewhere, this variable is accessed with lock held.
This patch is dedicated to code fragments involving oap_lock.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2744
Lustre-change: http://review.whamcloud.com/6572
--- drivers/staging/lustre/lustre/osc/osc_cache.c | 6 ++++++ drivers/staging/lustre/lustre/osc/osc_io.c | 2 ++ 2 files changed, 8 insertions(+)
drivers/staging/lustre/lustre/osc/osc_cache.c +6 -0
@@ -1071,7 +1071,9 @@ static int osc_extent_make_ready(const struct lu_env *env, last->oap_count =osc_refresh_count (env, last,OBD_BRW_WRITE );LASSERT (last->oap_count >0 );LASSERT (last->oap_page_off + last->oap_count <=PAGE_CACHE_SIZE ); +spin_lock (&last->oap_lock); last->oap_async_flags |=ASYNC_COUNT_STABLE ; +spin_unlock (&last->oap_lock); }/* for the rest of pages, we don't need to call osf_refresh_count()
@@ -1079,7 +1081,9 @@ static int osc_extent_make_ready(const struct lu_env *env,list_for_each_entry (oap, &ext->oe_pages, oap_pending_item) {if (!(oap->oap_async_flags &ASYNC_COUNT_STABLE )) { oap->oap_count =PAGE_CACHE_SIZE - oap->oap_page_off; +spin_lock (&oap->oap_lock); oap->oap_async_flags |=ASYNC_COUNT_STABLE ; +spin_unlock (&oap->oap_lock); } }
@@ -2238,6 +2242,8 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io, oap->oap_cmd = cmd; oap->oap_page_off = ops->ops_from; oap->oap_count = ops->ops_to - ops->ops_from; +/* No need to hold a lock here, +* since this page is not in any list yet. */ oap->oap_async_flags =0 ; oap->oap_brw_flags = brw_flags;
drivers/staging/lustre/lustre/osc/osc_io.c +2 -0
@@ -164,8 +164,10 @@ static int osc_io_submit(const struct lu_env *env, }cl_page_list_move (qout, qin, page); +spin_lock (&oap->oap_lock); oap->oap_async_flags =ASYNC_URGENT |ASYNC_READY ; oap->oap_async_flags |=ASYNC_COUNT_STABLE ; +spin_unlock (&oap->oap_lock);osc_page_submit (env, opg, crt, brw_flags);list_add_tail (&oap->oap_pending_item, &list);
Signature · 1 line
1.7.9.5