lustre: sync to OpenSFS as of Aug 22, 2021
cover lustre: sync to OpenSFS as of Aug 22, 2021
Update to latest OpenSFS work which is at 2.14.54
Alex Zhuravlev (1):
lustre: mgc: rework mgc_apply_recover_logs() for gcc10
Amir Shehata (3):
lnet: keep in insync to change due to GPU Direct Support
lustre: osc: Support RDMA only pages
lnet: peer state to lock primary nid
Chris Horn (3):
lnet: Reflect ni_fatal in NI status
lnet: Provide kernel API for adding peers
lustre: obdclass: Add peer/peer NI when processing llog
James Simmons (1):
lustre: obdclass: reintroduce lu_ref
Lai Siyao (1):
lustre: uapi: support fixed directory layout
Mikhail Pershin (1):
lustre: mdt: implement fallocate in MDC/MDT
Oleg Drokin (1):
lustre: update version to 2.14.54
Qian Yingjin (1):
lustre: pcc: add LCM_FL_PCC_RDONLY layout flag
Serguei Smirnov (2):
lnet: socklnd: allow dynamic setting of conns_per_peer
lnet: socklnd: set conns_per_peer based on link speed
Shaun Tancheff (1):
lustre: llite: Proved an abstraction for AS_EXITING
fs/lustre/Kconfig | 9 + fs/lustre/include/lu_ref.h | 104 +++++++- fs/lustre/include/lustre_osc.h | 10 +- fs/lustre/ldlm/ldlm_lib.c | 3 +- fs/lustre/llite/llite_internal.h | 7 + fs/lustre/llite/vvp_object.c | 2 +- fs/lustre/mdc/mdc_dev.c | 29 ++- fs/lustre/mgc/mgc_request.c | 24 +- fs/lustre/obdclass/Makefile | 3 +- fs/lustre/obdclass/cl_io.c | 8 + fs/lustre/obdclass/lu_ref.c | 393 +++++++++++++++++++++++++++++ fs/lustre/obdclass/lustre_peer.c | 18 +- fs/lustre/osc/osc_io.c | 7 +- fs/lustre/osc/osc_request.c | 18 +- fs/lustre/ptlrpc/wiretest.c | 3 + include/linux/lnet/api.h | 1 + include/linux/lnet/lib-lnet.h | 19 +- include/linux/lnet/lib-types.h | 32 +++ include/uapi/linux/lnet/libcfs_ioctl.h | 3 +- include/uapi/linux/lnet/lnet-dlc.h | 14 + include/uapi/linux/lustre/lustre_user.h | 18 +- include/uapi/linux/lustre/lustre_ver.h | 4 +- net/lnet/klnds/o2iblnd/o2iblnd.c | 1 + net/lnet/klnds/o2iblnd/o2iblnd.h | 9 +- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 16 +- net/lnet/klnds/socklnd/socklnd.c | 70 ++--- net/lnet/klnds/socklnd/socklnd.h | 4 + net/lnet/klnds/socklnd/socklnd_modparams.c | 126 ++++++++- net/lnet/lnet/api-ni.c | 59 ++++- net/lnet/lnet/lib-move.c | 62 ++++- net/lnet/lnet/peer.c | 60 ++++- net/lnet/lnet/router_proc.c | 2 +- 32 files changed, 1007 insertions(+), 131 deletions(-)
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 1/15 lustre: uapi: support fixed directory layout
From: Lai Siyao <[email protected]>
User may not want directories split automatically in some cases:
*.directory migrated.
* directory restriped.
To support this, an LMV flag LMV_HASH_FLAG_FIXED is added, and it will be set on migrated/restriped directories. NB, if directory is migrated or restriped to a one-stripe directory, it won't be transformed into a plain directory, because this flag needs to be kept.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14459
Lustre-commit: 4c2514f483280137 ("LU-14459 mdt: support fixed directory layout")
Reviewed-on: https://review.whamcloud.com/43291
--- fs/lustre/ptlrpc/wiretest.c | 1 + include/uapi/linux/lustre/lustre_user.h | 5 +++++ 2 files changed, 6 insertions(+)
fs/lustre/ptlrpc/wiretest.c +1 -0
@@ -1778,6 +1778,7 @@ void lustre_assert_wire_constants(void)BUILD_BUG_ON (LMV_MAGIC_V1 !=0x0CD20CD0 );BUILD_BUG_ON (LMV_MAGIC_STRIPE !=0x0CD40CD0 );BUILD_BUG_ON (LMV_HASH_TYPE_MASK !=0x0000ffff ); +BUILD_BUG_ON (LMV_HASH_FLAG_FIXED !=0x02000000 );BUILD_BUG_ON (LMV_HASH_FLAG_MERGE !=0x04000000 );BUILD_BUG_ON (LMV_HASH_FLAG_SPLIT !=0x08000000 );BUILD_BUG_ON (LMV_HASH_FLAG_LOST_LMV !=0x10000000 );
include/uapi/linux/lustre/lustre_user.h +5 -0
@@ -718,6 +718,9 @@ static inline bool lmv_is_known_hash_type(__u32 type) (type &LMV_HASH_TYPE_MASK ) ==LMV_HASH_TYPE_CRUSH ; } +/* fixed layout, such directories won't split automatically */ +/* NB, update LMV_HASH_FLAG_KNOWN when adding new flag */ +#define LMV_HASH_FLAG_FIXED 0x02000000#define LMV_HASH_FLAG_MERGE 0x04000000#define LMV_HASH_FLAG_SPLIT 0x08000000
@@ -733,6 +736,8 @@ static inline bool lmv_is_known_hash_type(__u32 type)#define LMV_HASH_FLAG_LAYOUT_CHANGE \ (LMV_HASH_FLAG_MIGRATION | LMV_HASH_FLAG_SPLIT | LMV_HASH_FLAG_MERGE) +#define LMV_HASH_FLAG_KNOWN 0xfe000000 +/* both SPLIT and MIGRATION are set for directory split */ static inline bool lmv_hash_is_splitting (__u32 hash) {
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 2/15 lustre: pcc: add LCM_FL_PCC_RDONLY layout flag
In reply to James SimmonsFrom: Qian Yingjin <[email protected]>
The upcoming new feature PCC-RO is combined with FLR and extend the on-disk data strucutre 'enum lov_comp_md_flags' for layout components. It adds a new layout flag: LCM_FL_PCC_RDONLY.
enum lov_comp_md_flags {
LCM_FL_NONE = 0x0,
LCM_FL_RDONLY = 0x1,
LCM_FL_WRITE_PENDING = 0x2,
LCM_FL_SYNC_PENDING = 0x3,
LCM_FL_PCC_RDONLY = 0x8,
LCM_FL_FLR_MASK = 0xB,
};The LCM_FL_PCC_RDONLY flag, which is dedicated for PCC-RO, is different from LCM_FL_RDONLY.
A PCC-RO cached file could be in the state:
- LCM_FL_PCC_RDONLY | LCM_FL_RDONLY: it means that all FLR components are synced and in up-to-date state. The replicated file is on read-only state. And then one client attaches the file into the PCC backend with PCC-RO mode.
- LCM_FL_PCC_RDONLY | LCM_FL_WRITE_PENDING: it means the file was once modified, the data content of layout components are not synced. MDT has already picked a promary replica and marked other components as STALE. At this time, a client can still PCC-RO attach the file. On this client, the primary component and the PCC copy are both in up-to-date state.
As a new LCM_FL_PCC_RDONLY flag is added, the old client may not understand this new FLR layout flag, and may result in inconsistent data access.
This patch adds this new flag for the purpose of compatibility and interoperability.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13602
Lustre-commit: adc1bbbf20e0a8a5 ("LU-13602 pcc: add LCM_FL_PCC_RDONLY layout flag")
Reviewed-on: https://review.whamcloud.com/40813
--- fs/lustre/ptlrpc/wiretest.c | 2 ++ include/uapi/linux/lustre/lustre_user.h | 13 +++++++------ 2 files changed, 9 insertions(+), 6 deletions(-)
fs/lustre/ptlrpc/wiretest.c +2 -0
@@ -1727,6 +1727,8 @@ void lustre_assert_wire_constants(void) (long long)LCM_FL_WRITE_PENDING);LASSERTF (LCM_FL_SYNC_PENDING ==3 ,"found %lld\n" , (long long )LCM_FL_SYNC_PENDING ); +LASSERTF (LCM_FL_PCC_RDONLY ==8 ,"found %lld\n" , + (long long )LCM_FL_PCC_RDONLY );/* Checks for struct lmv_mds_md_v1 */ LASSERTF((int)sizeof(struct lmv_mds_md_v1) == 56, "found %lld\n",
include/uapi/linux/lustre/lustre_user.h +7 -6
@@ -622,12 +622,13 @@ static inline __u16 mirror_id_of(__u32 id)* on-disk data for lcm_flags. Valid if lcm_magic is LOV_MAGIC_COMP_V1. */ enum lov_comp_md_flags { -/* the least 2 bits are used by FLR to record file state */ -LCM_FL_NONE =0 , -LCM_FL_RDONLY =1 , -LCM_FL_WRITE_PENDING =2 , -LCM_FL_SYNC_PENDING =3 , -LCM_FL_FLR_MASK =0x3 , +/* the least 4 bits are used by FLR to record file state */ +LCM_FL_NONE =0x0 , +LCM_FL_RDONLY =0x1 , +LCM_FL_WRITE_PENDING =0x2 , +LCM_FL_SYNC_PENDING =0x3 , +LCM_FL_PCC_RDONLY =0x8 , +LCM_FL_FLR_MASK =0xB , };struct lov_comp_md_v1 {
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 3/15 lustre: mdt: implement fallocate in MDC/MDT
In reply to James SimmonsFrom: Mikhail Pershin <[email protected]>
- add CLIO fallocate() handling in MDC
- implement FALLOCATE RPC handling at MDT side
- update test group 150 in sanity to work with
sanity-dom.sh test
WC-bug-id: https://jira.whamcloud.com/browse/LU-14382
Lustre-commit: 163870abfb7c3fe3 ("LU-14382 mdt: implement fallocate in MDC/MDT")
Reviewed-on: https://review.whamcloud.com/41418
--- fs/lustre/include/lustre_osc.h | 4 ++++ fs/lustre/mdc/mdc_dev.c | 29 ++++++++++++++++++++--------- fs/lustre/osc/osc_io.c | 5 +++-- fs/lustre/osc/osc_request.c | 3 ++- 4 files changed, 29 insertions(+), 12 deletions(-)
fs/lustre/include/lustre_osc.h +4 -0
@@ -678,6 +678,8 @@ int osc_reconnect(const struct lu_env *env, struct obd_export *exp,int osc_disconnect (struct obd_export *exp);int osc_punch_send (struct obd_export *exp,struct obdo *oa,obd_enqueue_update_f upcall,void *cookie); +int osc_fallocate_base (struct obd_export *exp,struct obdo *oa, +obd_enqueue_update_f upcall,void *cookie,int mode);/* osc_io.c */ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios,
@@ -712,6 +714,8 @@ void osc_io_lseek_end(const struct lu_env *env, const struct cl_io_slice *slice);int osc_io_lru_reserve (const struct lu_env *env,const struct cl_io_slice *ios,loff_t pos,size_t count); +int osc_punch_start (const struct lu_env *env,struct cl_io *io, +struct cl_object *obj);/* osc_lock.c */ void osc_lock_to_lockless(const struct lu_env *env, struct osc_lock *ols,
fs/lustre/mdc/mdc_dev.c +20 -9
@@ -35,6 +35,7 @@#include <obd_class.h> #include <lustre_osc.h> +#include <linux/falloc.h> #include <uapi/linux/lustre/lustre_param.h> #include "mdc_internal.h"
@@ -1035,11 +1036,13 @@ static int mdc_io_setattr_start(const struct lu_env *env, &oio->oi_trunc);if (rc <0 )return rc; + }else if (cl_io_is_fallocate (io) && + io->u.ci_setattr.sa_falloc_mode &FALLOC_FL_PUNCH_HOLE ) { + rc =osc_punch_start (env, io, obj); +if (rc <0 ) +return rc; } -if (cl_io_is_fallocate (io)) -return -EOPNOTSUPP ; -if (oio->oi_lockless ==0 ) {cl_object_attr_lock (obj); rc =cl_object_attr_get (env, obj, attr);
@@ -1070,7 +1073,7 @@ static int mdc_io_setattr_start(const struct lu_env *env,return rc; } -if (!(ia_valid &ATTR_SIZE )) +if (!(ia_valid &ATTR_SIZE ) && !cl_io_is_fallocate (io))return 0 ;memset (oa,0 ,sizeof (*oa));
@@ -1078,12 +1081,10 @@ static int mdc_io_setattr_start(const struct lu_env *env, oa->o_mtime = attr->cat_mtime; oa->o_atime = attr->cat_atime; oa->o_ctime = attr->cat_ctime; - - oa->o_size = size; - oa->o_blocks =OBD_OBJECT_EOF ; oa->o_valid =OBD_MD_FLID |OBD_MD_FLGROUP |OBD_MD_FLATIME |OBD_MD_FLCTIME |OBD_MD_FLMTIME |OBD_MD_FLSIZE |OBD_MD_FLBLOCKS ; +if (oio->oi_lockless) { oa->o_flags =OBD_FL_SRVLOCK ; oa->o_valid |=OBD_MD_FLFLAGS ;
@@ -1095,9 +1096,19 @@ static int mdc_io_setattr_start(const struct lu_env *env, }init_completion (&cbargs->opc_sync); +if (cl_io_is_fallocate (io)) { +int falloc_mode = io->u.ci_setattr.sa_falloc_mode; - rc =osc_punch_send (osc_export (cl2osc (obj)), oa, - mdc_async_upcall, cbargs); + oa->o_size = io->u.ci_setattr.sa_falloc_offset; + oa->o_blocks = io->u.ci_setattr.sa_falloc_end; + rc =osc_fallocate_base (osc_export (cl2osc (obj)), oa, + mdc_async_upcall, cbargs, falloc_mode); + }else { + oa->o_size = size; + oa->o_blocks =OBD_OBJECT_EOF ; + rc =osc_punch_send (osc_export (cl2osc (obj)), oa, + mdc_async_upcall, cbargs); + } cbargs->opc_rpc_sent = rc ==0 ;return rc; }
fs/lustre/osc/osc_io.c +3 -2
@@ -548,8 +548,8 @@ static void osc_trunc_check(const struct lu_env *env, struct cl_io *io,* if server doesn't support fallocate punch, we also need these data to be * flushed first to prevent re-ordering with the punch */ -static int osc_punch_start (const struct lu_env *env,struct cl_io *io, -struct cl_object *obj) +int osc_punch_start (const struct lu_env *env,struct cl_io *io, +struct cl_object *obj) {struct osc_object *osc =cl2osc (obj);pgoff_t pg_start =cl_index (obj, io->u.ci_setattr.sa_falloc_offset);
@@ -564,6 +564,7 @@ static int osc_punch_start(const struct lu_env *env, struct cl_io *io, osc);return 0 ; } +EXPORT_SYMBOL (osc_punch_start);static int osc_io_setattr_start (const struct lu_env *env,const struct cl_io_slice *slice)
fs/lustre/osc/osc_request.c +2 -1
@@ -472,7 +472,7 @@ int osc_fallocate_base(struct obd_export *exp, struct obdo *oa,ptlrpc_request_set_replen (req); - req->rq_interpret_reply = (ptlrpc_interpterer_t )osc_setattr_interpret; + req->rq_interpret_reply = osc_setattr_interpret;BUILD_BUG_ON (sizeof (*sa) >sizeof (req->rq_async_args)); sa =ptlrpc_req_async_args (sa, req); sa->sa_oa = oa;
@@ -482,6 +482,7 @@ int osc_fallocate_base(struct obd_export *exp, struct obdo *oa,ptlrpcd_add_req (req);return 0 ; } +EXPORT_SYMBOL (osc_fallocate_base); static int osc_sync_interpret(const struct lu_env *env, struct ptlrpc_request *req,
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 4/15 lnet: Reflect ni_fatal in NI status
In reply to James SimmonsFrom: Chris Horn <[email protected]>
If the ni_fatal_error_on flag is set on an NI then that NI should be considered down.
HPE-bug-id: LUS-10167
WC-bug-id: https://jira.whamcloud.com/browse/LU-14790
Lustre-commit: d77e95cc6d4e947b ("LU-14790 lnet: Reflect ni_fatal in NI status")
Reviewed-on: https://review.whamcloud.com/44072
--- include/linux/lnet/lib-lnet.h | 14 ++++++++++++++ net/lnet/lnet/api-ni.c | 14 +++----------- net/lnet/lnet/router_proc.c | 2 +- 3 files changed, 18 insertions(+), 12 deletions(-)
include/linux/lnet/lib-lnet.h +14 -0
@@ -116,6 +116,20 @@return update; } +static inline unsigned int +lnet_ni_get_status_locked (struct lnet_ni *ni) +__must_hold(&ni->ni_lock) +{ +if (ni->ni_nid ==LNET_NID_LO_0 ) +return LNET_NI_STATUS_UP ; +else if (atomic_read (&ni->ni_fatal_error_on)) +return LNET_NI_STATUS_DOWN ; +else if (ni->ni_status) +return ni->ni_status->ns_status; +else +return LNET_NI_STATUS_UP ; +} +static inline bool lnet_ni_set_status (struct lnet_ni *ni,u32 status) {
net/lnet/lnet/api-ni.c +3 -11
@@ -1829,9 +1829,7 @@ struct lnet_ping_buffer * ns->ns_nid = ni->ni_nid;lnet_ni_lock (ni); - ns->ns_status = ni->ni_status ? - ni->ni_status->ns_status : -LNET_NI_STATUS_UP ; + ns->ns_status =lnet_ni_get_status_locked (ni); ni->ni_status = ns;lnet_ni_unlock (ni);
@@ -2936,10 +2934,7 @@ void lnet_lib_exit(void) } cfg_ni->lic_nid = ni->ni_nid; -if (ni->ni_nid ==LNET_NID_LO_0 ) - cfg_ni->lic_status =LNET_NI_STATUS_UP ; -else - cfg_ni->lic_status = ni->ni_status->ns_status; + cfg_ni->lic_status =lnet_ni_get_status_locked (ni); cfg_ni->lic_dev_cpt = ni->ni_dev_cpt;memcpy (&tun->lt_cmn, &ni->ni_net->net_tunables,sizeof (tun->lt_cmn));
@@ -3022,10 +3017,7 @@ void lnet_lib_exit(void) config->cfg_config_u.cfg_net.net_peer_rtr_credits = ni->ni_net->net_tunables.lct_peer_rtr_credits; -if (ni->ni_nid ==LNET_NID_LO_0 ) - net_config->ni_status =LNET_NI_STATUS_UP ; -else - net_config->ni_status = ni->ni_status->ns_status; + net_config->ni_status =lnet_ni_get_status_locked (ni);if (ni->ni_cpts) {int num_cpts =min (ni->ni_ncpts,LNET_MAX_SHOW_NUM_CPT );
net/lnet/lnet/router_proc.c +1 -1
@@ -673,7 +673,7 @@ static int proc_lnet_nis(struct ctl_table *table, int write,lnet_ni_lock (ni);LASSERT (ni->ni_status); - stat = (ni->ni_status->ns_status == + stat = (lnet_ni_get_status_locked (ni) ==LNET_NI_STATUS_UP ) ?"up" :"down" ;lnet_ni_unlock (ni);
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 5/15 lustre: obdclass: reintroduce lu_ref
In reply to James SimmonsPreviously lu_ref was removed due to the lack of testing. Intel brought this back to life so reintroduce this debugging feature.
WC-bug-id: https://jira.whamcloud.com/browse/LU-6142
Lustre-commit: 5c98de856618f30 ("LU-6142 obdclass: resolve lu_ref checkpatch issues")
Reviewed-on: https://review.whamcloud.com/44088
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Lustre-commit: 6b319185659104b ("LU-8066 obdclass: move lu_ref to debugfs")
Reviewed-on: https://review.whamcloud.com/44311
--- fs/lustre/Kconfig | 9 + fs/lustre/include/lu_ref.h | 104 ++++++++++-- fs/lustre/obdclass/Makefile | 3 +- fs/lustre/obdclass/cl_io.c | 8 + fs/lustre/obdclass/lu_ref.c | 393 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 503 insertions(+), 14 deletions(-)
fs/lustre/Kconfig +9 -0
@@ -61,6 +61,15 @@ config LUSTRE_DEBUG_EXPENSIVE_CHECK Use with caution. If unsure, say N. +config LUSTRE_DEBUG_LU_REF + bool "Enable Lustre lu_ref checks" + depends on LUSTRE_DEBUG_EXPENSIVE_CHECK + help + lu_ref gives the ability to track references to a given object. It is + quite cpu expensive so its disabled by default. + + Use with caution. If unsure, say N. + config LUSTRE_TRANSLATE_ERRNOS bool depends on LUSTRE_FS && !X86
fs/lustre/include/lu_ref.h +91 -13
127 lines not on this page · raw message
fs/lustre/obdclass/Makefile +2 -1
@@ -6,7 +6,8 @@ obj-$(CONFIG_LUSTRE_FS) += obdclass.o obdclass-y := llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o \ genops.o obd_sysfs.o lprocfs_status.o lprocfs_counters.o \ lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \ - obdo.o obd_config.o obd_mount.o lu_object.o lu_ref.o \ + obdo.o obd_config.o obd_mount.o lu_object.o \ cl_object.o cl_page.o cl_lock.o cl_io.o kernelcomm.o \ jobid.o integrity.o obd_cksum.o range_lock.o \ lu_tgt_descs.o lu_tgt_pool.o +obdclass-$(CONFIG_LUSTRE_DEBUG_LU_REF) += lu_ref.o
fs/lustre/obdclass/cl_io.c +8 -0
@@ -895,6 +895,14 @@ void cl_page_list_move_head(struct cl_page_list *dst, struct cl_page_list *src,*/ void cl_page_list_splice (struct cl_page_list *src,struct cl_page_list *dst) { +#ifdef CONFIG_LUSTRE_DEBUG_LU_REF +struct cl_page *page; +struct cl_page *tmp; + +cl_page_list_for_each_safe (page, tmp, src) +lu_ref_set_at (&page->cp_reference, &page->cp_queue_ref, +"queue" , src, dst); +#endif dst->pl_nr += src->pl_nr; src->pl_nr =0 ;list_splice_tail_init (&src->pl_pages, &dst->pl_pages);
fs/lustre/obdclass/lu_ref.c +393 -0
396 lines not on this page · raw message
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 6/15 lnet: keep in insync to change due to GPU Direct Support
In reply to James SimmonsFrom: Amir Shehata <[email protected]>
Since in the HPC community most people run 10+ year old kernels Nvidia created their own version of PCI peer2peer which sites want to use. The OpenSFS supports this special one off out of tree driver which impacts the LNet code. To keep in sync we port to the Linux proper tree these changes. This also allows the potential to support the support PCI peer2peer in the future. This initial abstract was poorly done so it will have to be revisted.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14798
Lustre-commit: a7a889f77cec3ad44 ("LU-14798 lnet: add LNet GPU Direct Support")
lustre-change: https://review.whamcloud.com/37368
Whamcloud-bug-id: EX-773
Reviewed-on: https://review.whamcloud.com/44110
--- include/linux/lnet/lib-types.h | 25 +++++++++++++++ net/lnet/klnds/o2iblnd/o2iblnd.c | 1 + net/lnet/klnds/o2iblnd/o2iblnd.h | 9 +++--- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 16 ++++++++-- net/lnet/lnet/lib-move.c | 62 ++++++++++++++++++++++++++++++------- 5 files changed, 95 insertions(+), 18 deletions(-)
include/linux/lnet/lib-types.h +25 -0
@@ -312,8 +312,33 @@ struct lnet_lnd {/* accept a new connection */ int (*lnd_accept)(struct lnet_ni *ni,struct socket *sock); + +/* get dma_dev priority */ +unsigned int (*lnd_get_dev_prio)(struct lnet_ni *ni, +unsigned int dev_idx); }; +/* FIXME !!!!! The abstract for GPU page support (PCI peer2peer) +* was done for only the external NVIDIA driver and done very +* poorly. Once DRI / TTM supports peer2peer we can redo this +* right. +*/ +static inline unsigned int lnet_get_dev_prio (struct device *dev, +unsigned int dev_idx) +{ +return UINT_MAX ; +} + +static inline bool lnet_is_rdma_only_page (struct page *page) +{ +return false; +} + +static inline unsigned int lnet_get_dev_idx (struct page *page) +{ +return false; +} +struct lnet_tx_queue {int tq_credits;/* # tx credits free */ int tq_credits_min;/* lowest it's been */
net/lnet/klnds/o2iblnd/o2iblnd.c +1 -0
@@ -2953,6 +2953,7 @@ static int kiblnd_startup(struct lnet_ni *ni) .lnd_ctl = kiblnd_ctl, .lnd_send = kiblnd_send, .lnd_recv = kiblnd_recv, + .lnd_get_dev_prio = kiblnd_get_dev_prio, };static void ko2inlnd_assert_wire_constants (void )
net/lnet/klnds/o2iblnd/o2iblnd.h +5 -4
@@ -858,18 +858,18 @@ static inline void kiblnd_dma_unmap_single(struct ib_device *dev,#define KIBLND_UNMAP_ADDR_SET (p, m, a) do {} while (0)#define KIBLND_UNMAP_ADDR (p, m, a) (a) -static inline int kiblnd_dma_map_sg (struct ib_device *dev, +static inline int kiblnd_dma_map_sg (struct kib_hca_dev *hdev,struct scatterlist *sg,int nents,enum dma_data_direction direction) { -return ib_dma_map_sg (dev, sg, nents, direction); +return ib_dma_map_sg (hdev->ibh_ibdev, sg, nents, direction); } -static inline void kiblnd_dma_unmap_sg (struct ib_device *dev, +static inline void kiblnd_dma_unmap_sg (struct kib_hca_dev *hdev,struct scatterlist *sg,int nents,enum dma_data_direction direction) { -ib_dma_unmap_sg (dev, sg, nents, direction); +ib_dma_unmap_sg (hdev->ibh_ibdev, sg, nents, direction); } static inline u64 kiblnd_sg_dma_address(struct ib_device *dev,
@@ -959,3 +959,4 @@ void kiblnd_pack_msg(struct lnet_ni *ni, struct kib_msg *msg, int version,int kiblnd_send (struct lnet_ni *ni,void *private,struct lnet_msg *lntmsg);int kiblnd_recv (struct lnet_ni *ni,void *private,struct lnet_msg *lntmsg,int delayed,struct iov_iter *to,unsigned int rlen); +unsigned int kiblnd_get_dev_prio (struct lnet_ni *ni,unsigned int dev_idx);
net/lnet/klnds/o2iblnd/o2iblnd_cb.c +14 -2
@@ -615,7 +615,7 @@ static void kiblnd_unmap_tx(struct kib_tx *tx)kiblnd_fmr_pool_unmap (&tx->tx_fmr, tx->tx_status);if (tx->tx_nfrags) { -kiblnd_dma_unmap_sg (tx->tx_pool->tpo_hdev->ibh_ibdev, +kiblnd_dma_unmap_sg (tx->tx_pool->tpo_hdev, tx->tx_frags, tx->tx_nfrags, tx->tx_dmadir); tx->tx_nfrags =0 ; }
@@ -636,7 +636,7 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, tx->tx_dmadir = (rd != tx->tx_rd) ?DMA_FROM_DEVICE :DMA_TO_DEVICE ; tx->tx_nfrags = nfrags; - rd->rd_nfrags =kiblnd_dma_map_sg (hdev->ibh_ibdev, tx->tx_frags, + rd->rd_nfrags =kiblnd_dma_map_sg (hdev, tx->tx_frags, tx->tx_nfrags, tx->tx_dmadir);for (i =0 , nob =0 ; i < rd->rd_nfrags; i++) {
@@ -1721,6 +1721,18 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx,lnet_finalize (lntmsg, -EIO ); } +unsigned int +kiblnd_get_dev_prio (struct lnet_ni *ni,unsigned int dev_idx) +{ +struct kib_net *net = ni->ni_data; +struct device *dev =NULL ; + +if (net) + dev = net->ibn_dev->ibd_hdev->ibh_ibdev->dma_device; + +return lnet_get_dev_prio (dev, dev_idx); +} +int kiblnd_recv (struct lnet_ni *ni,void *private,struct lnet_msg *lntmsg,int delayed,struct iov_iter *to,unsigned int rlen)
net/lnet/lnet/lib-move.c +50 -12
179 lines not on this page · raw message
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 7/15 lustre: osc: Support RDMA only pages
In reply to James SimmonsFrom: Amir Shehata <[email protected]>
Some memory architectures and CPU-offload cards with on-board memory do not map data pages into the CPU address space. Allow RDMA of data directly into those pages without accessing contents.
Therefore, made changes to prevent doing checksum on these type of pages.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14798
Lustre-commit: 29eabeb34c5ba2cffd ("LU-14798 lustre: Support RDMA only pages")
Lustre-change: https://review.whamcloud.com/37454
Whamcloud-bug-id: EX-773
Reviewed-on: https://review.whamcloud.com/44111
--- fs/lustre/include/lustre_osc.h | 6 ++++-- fs/lustre/osc/osc_io.c | 2 ++ fs/lustre/osc/osc_request.c | 15 +++++++++++---- 3 files changed, 17 insertions(+), 6 deletions(-)
fs/lustre/include/lustre_osc.h +4 -2
@@ -956,8 +956,10 @@ struct osc_extent { oe_ndelay:1, /* direct IO pages */ oe_dio:1, -/* this extent consists of RDMA only pages */ - oe_is_rdma_only; + /* this extent consists of pages that are not directly accessible + * from the CPU + */ + oe_is_rdma_only:1;/* how many grants allocated for this extent. * Grant allocated for this extent. There is no grant allocated * for reading extents and sync write extents.
fs/lustre/osc/osc_io.c +2 -0
@@ -153,6 +153,8 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, page =cl_page_list_first (qin);if (page->cp_type ==CPT_TRANSIENT ) brw_flags |=OBD_BRW_NOCACHE ; +if (lnet_is_rdma_only_page (page->cp_vmpage)) + brw_flags |=OBD_BRW_RDMA_ONLY ;/* * NOTE: here @page is a top-level page. This is done to avoid
fs/lustre/osc/osc_request.c +11 -4
@@ -1402,6 +1402,7 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli,const char *obd_name = cli->cl_import->imp_obd->obd_name;struct inode *inode =NULL ;bool directio = false; +bool enable_checksum = true;if (pga[0 ]->pg) { inode =page2inode (pga[0 ]->pg);
@@ -1545,6 +1546,11 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, } } +if (lnet_is_rdma_only_page (pga[0 ]->pg)) { + enable_checksum = false; + short_io_size =0 ; + } +/* Check if read/write is small enough to be a short io. */ if (short_io_size > cli->cl_max_short_io_bytes || niocount >1 || !imp_connect_shortio (cli->cl_import))
@@ -1700,10 +1706,12 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli,if (osc_should_shrink_grant (cli))osc_shrink_grant_local (cli, &body->oa); +if (!cli->cl_checksum ||sptlrpc_flavor_has_bulk (&req->rq_flvr)) + enable_checksum = false; +/* size[REQ_REC_OFF] still sizeof (*body) */ if (opc ==OST_WRITE ) { -if (cli->cl_checksum && - !sptlrpc_flavor_has_bulk (&req->rq_flvr)) { +if (enable_checksum) {/* store cl_cksum_type in a local variable since * it can be changed via lprocfs */
@@ -1743,8 +1751,7 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli,req_capsule_set_size (pill, &RMF_RCS ,RCL_SERVER ,sizeof (u32) * niocount); }else { -if (cli->cl_checksum && - !sptlrpc_flavor_has_bulk (&req->rq_flvr)) { +if (enable_checksum) {if ((body->oa.o_valid &OBD_MD_FLFLAGS ) ==0 ) body->oa.o_flags =0 ; body->oa.o_flags |= obd_cksum_type_pack(obd_name,
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 8/15 lustre: mgc: rework mgc_apply_recover_logs() for gcc10
In reply to James SimmonsFrom: Alex Zhuravlev <[email protected]>
rework mgc_apply_recover_logs() to use a separate buffer of appropriate size so that gcc10 doesn't complain:
mgc_request.c:1506:24: error: argument 4 may overlap destination
object [-Werror=restrict]
1506 | pos += sprintf(obdname + pos, "-%s-%s", cname, inst);
WC-bug-id: https://jira.whamcloud.com/browse/LU-14093
Lustre-commit: d13d8158e816b7ac ("LU-14093 mgc: rework mgc_apply_recover_logs() for gcc10")
Reviewed-on: https://review.whamcloud.com/40484
--- fs/lustre/mgc/mgc_request.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-)
fs/lustre/mgc/mgc_request.c +10 -14
@@ -1093,7 +1093,7 @@ static int mgc_apply_recover_logs(struct obd_device *mgc,struct lustre_cfg *lcfg;struct lustre_cfg_bufs bufs;u64 prev_version =0 ; -char *inst; +char inst[MTI_NAME_MAXLEN +1 ];char *buf;int bufsz;int pos;
@@ -1107,19 +1107,15 @@ static int mgc_apply_recover_logs(struct obd_device *mgc,/* get dynamic nids setting */ dynamic_nids = mgc->obd_dynamic_nids; - inst =kzalloc (PAGE_SIZE ,GFP_KERNEL ); -if (!inst) -return -ENOMEM ; - - pos =snprintf (inst,PAGE_SIZE ,"%px" , cfg->cfg_instance); -if (pos >=PAGE_SIZE ) { -kfree (inst); + pos =snprintf (inst,sizeof (inst),"%px" , cfg->cfg_instance); +if (pos >=sizeof (inst))return -E2BIG ; - } - ++pos; - buf = inst + pos; - bufsz =PAGE_SIZE - pos; + buf =kzalloc (PAGE_SIZE ,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + bufsz =PAGE_SIZE ; + pos =0 ;while (datalen >0 ) {int entry_len =sizeof (*entry);
@@ -1204,7 +1200,7 @@ static int mgc_apply_recover_logs(struct obd_device *mgc, is_ost ? "OST" : "MDT", entry->mne_index); cname = is_ost ?"osc" :"mdc" , - pos +=sprintf (obdname + pos,"-%s-%s" , cname, inst); + pos +=snprintf (obdname + pos, bufsz,"-%s-%s" , cname, inst);lustre_cfg_bufs_reset (&bufs, obdname);/* find the obd by obdname */
@@ -1308,7 +1304,7 @@ static int mgc_apply_recover_logs(struct obd_device *mgc,/* continue, even one with error */ } -kfree (inst); +kfree (buf);return rc; }
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 9/15 lnet: socklnd: allow dynamic setting of conns_per_peer
In reply to James SimmonsFrom: Serguei Smirnov <[email protected]>
Modify lnetctl and associated code to allow dynamic setting of conns_per_peer lnd parameter per ni.
The parameter can be set for a specific active nid:
lnetctl net set --nid 192.168.122.10@tcp --conns-per-peer=4
Or when adding a new net, taking effect on the new nid:
lnetctl net add --net tcp --if eth0 --conns-per-peer=1
By default, conns_per_peer value specified as the module parameter shall be used.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12815
Lustre-commit: a5cbe7883db6d77b ("LU-12815 socklnd: allow dynamic setting of conns_per_peer")
Reviewed-on: https://review.whamcloud.com/41463
--- include/linux/lnet/lib-lnet.h | 3 ++ include/uapi/linux/lnet/libcfs_ioctl.h | 3 +- include/uapi/linux/lnet/lnet-dlc.h | 14 ++++++ net/lnet/klnds/socklnd/socklnd.c | 70 +++++++++++++++--------------- net/lnet/klnds/socklnd/socklnd.h | 4 ++ net/lnet/klnds/socklnd/socklnd_modparams.c | 55 +++++++++++++++++++++-- net/lnet/lnet/api-ni.c | 43 ++++++++++++++++++ 7 files changed, 154 insertions(+), 38 deletions(-)
include/linux/lnet/lib-lnet.h +3 -0
@@ -86,6 +86,9 @@#define DEFAULT_PEER_CREDITS 8#define DEFAULT_CREDITS 256 +/* default number of connections per peer */ +#define DEFAULT_CONNS_PER_PEER 1 +int choose_ipv4_src (u32 *ret,int interface,u32 dst_ipaddr,struct net *ns);bool lnet_is_route_alive (struct lnet_route *route);
include/uapi/linux/lnet/libcfs_ioctl.h +2 -1
@@ -156,6 +156,7 @@ struct libcfs_ioctl_data {#define IOC_LIBCFS_GET_UDSP _IOWR(IOC_LIBCFS_TYPE, 108, IOCTL_CONFIG_SIZE)#define IOC_LIBCFS_GET_CONST_UDSP_INFO _IOWR(IOC_LIBCFS_TYPE, 109, IOCTL_CONFIG_SIZE)#define IOC_LIBCFS_RESET_LNET_STATS _IOWR(IOC_LIBCFS_TYPE, 110, IOCTL_CONFIG_SIZE) -#define IOC_LIBCFS_MAX_NR 110 +#define IOC_LIBCFS_SET_CONNS_PER_PEER _IOWR(IOC_LIBCFS_TYPE, 111, IOCTL_CONFIG_SIZE) +#define IOC_LIBCFS_MAX_NR 111#endif /* __LIBCFS_IOCTL_H__ */
include/uapi/linux/lnet/lnet-dlc.h +14 -0
@@ -81,9 +81,16 @@ struct lnet_ioctl_config_o2iblnd_tunables {__u16 lnd_ntx; }; +struct lnet_ioctl_config_socklnd_tunables { +__u32 lnd_version; +__u16 lnd_conns_per_peer; +__u16 lnd_pad; +}; +struct lnet_lnd_tunables {union {struct lnet_ioctl_config_o2iblnd_tunables lnd_o2ib; +struct lnet_ioctl_config_socklnd_tunables lnd_sock; } lnd_tun_u; };
@@ -280,6 +287,13 @@ struct lnet_ioctl_reset_health_cfg {lnet_nid_t rh_nid; }; +struct lnet_ioctl_reset_conns_per_peer_cfg { +struct libcfs_ioctl_hdr rcpp_hdr; +__u16 rcpp_all:1 ; +__s16 rcpp_value; +lnet_nid_t rcpp_nid; +}; +struct lnet_ioctl_recovery_list {struct libcfs_ioctl_hdr rlst_hdr;enum lnet_health_type rlst_type:32 ;
net/lnet/klnds/socklnd/socklnd.c +36 -34
138 lines not on this page · raw message
net/lnet/klnds/socklnd/socklnd.h +4 -0
@@ -403,6 +403,9 @@ struct ksock_conn_cb {int ksnr_conn_count;/* total # conns for * this cb */ +unsigned int ksnr_max_conns;/* conns_per_peer at +* peer creation +*/ };#define SOCKNAL_KEEPALIVE_PING 1/* cookie for keepalive ping */
@@ -696,6 +699,7 @@ int ksocknal_lib_get_conn_tunables(struct ksock_conn *conn, int *txmem,void ksocknal_write_callback (struct ksock_conn *conn);int ksocknal_tunables_init (void ); +void ksocknal_tunables_setup (struct lnet_ni *ni);void ksocknal_lib_csum_tx (struct ksock_tx *tx);
net/lnet/klnds/socklnd/socklnd_modparams.c +52 -3
79 lines not on this page · raw message
net/lnet/lnet/api-ni.c +43 -0
55 lines not on this page · raw message
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 10/15 lnet: Provide kernel API for adding peers
In reply to James SimmonsFrom: Chris Horn <[email protected]>
Implement LNetAddPeer() API to allow other kernel modules to add peers to LNet.
Peers created via this API are not marked as having been configured by DLC. As such, they can be overwritten by discovery.
HPE-bug-id: LUS-9293
WC-bug-id: https://jira.whamcloud.com/browse/LU-14661
Lustre-commit: ac201366ad5700ed ("LU-14661 lnet: Provide kernel API for adding peers")
Reviewed-on: https://review.whamcloud.com/43509
--- include/linux/lnet/api.h | 1 + include/linux/lnet/lib-lnet.h | 2 +- net/lnet/lnet/api-ni.c | 2 +- net/lnet/lnet/peer.c | 60 ++++++++++++++++++++++++++++++++++++++++--- 4 files changed, 59 insertions(+), 6 deletions(-)
include/linux/lnet/api.h +1 -0
@@ -164,6 +164,7 @@ int LNetGet(lnet_nid_t self,int LNetCtl (unsigned int cmd,void *arg);void LNetDebugPeer (struct lnet_process_id id);int LNetGetPeerDiscoveryStatus (void ); +int LNetAddPeer (lnet_nid_t *nids,u32 num_nids);/** @} lnet_misc */
include/linux/lnet/lib-lnet.h +1 -1
@@ -854,7 +854,7 @@ struct lnet_peer_net *lnet_peer_get_net_locked(struct lnet_peer *peer,void lnet_peer_clr_pref_rtrs (struct lnet_peer_ni *lpni);int lnet_peer_add_pref_rtr (struct lnet_peer_ni *lpni,lnet_nid_t nid);int lnet_peer_ni_set_non_mr_pref_nid (struct lnet_peer_ni *lpni,lnet_nid_t nid); -int lnet_add_peer_ni (lnet_nid_t key_nid,lnet_nid_t nid,bool mr); +int lnet_add_peer_ni (lnet_nid_t key_nid,lnet_nid_t nid,bool mr,bool temp);int lnet_del_peer_ni (lnet_nid_t key_nid,lnet_nid_t nid);int lnet_get_peer_info (struct lnet_ioctl_peer_cfg *cfg,void __user *bulk); int lnet_get_peer_ni_info(u32 peer_index, u64 *nid,
net/lnet/lnet/api-ni.c +1 -1
@@ -4015,7 +4015,7 @@ u32 lnet_get_dlc_seq_locked(void)mutex_lock (&the_lnet.ln_api_mutex); rc =lnet_add_peer_ni (cfg->prcfg_prim_nid, cfg->prcfg_cfg_nid, - cfg->prcfg_mr); + cfg->prcfg_mr, false);mutex_unlock (&the_lnet.ln_api_mutex);return rc; }
net/lnet/lnet/peer.c +56 -4
92 lines not on this page · raw message
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 11/15 lustre: obdclass: Add peer/peer NI when processing llog
In reply to James SimmonsFrom: Chris Horn <[email protected]>
Construct peers when processing the config log so that LNet has complete information about peer info stored in the config log.
These are "temporary" peers which can be overwritten by discovery.
In client_import_add_nids_to_conn(), we do not need to hold the import lock when adding NIDs to the obd_uuid, and LNet needs to take the LNet API mutex when adding/modifying peers. We don't want to take the mutex while a spin lock is already being held, so drop the spin lock prior to calling class_add_nids_to_uuid().
HPE-bug-id: LUS-9293
WC-bug-id: https://jira.whamcloud.com/browse/LU-14661
Lustre-commit: 16321de596f63951 ("LU-14661 obdclass: Add peer/peer NI when processing llog")
Reviewed-on: https://review.whamcloud.com/43510
--- fs/lustre/ldlm/ldlm_lib.c | 3 ++- fs/lustre/obdclass/lustre_peer.c | 18 +++++++++++++++++- 2 files changed, 19 insertions(+), 2 deletions(-)
fs/lustre/ldlm/ldlm_lib.c +2 -1
@@ -173,9 +173,10 @@ int client_import_add_nids_to_conn(struct obd_import *imp, lnet_nid_t *nids,list_for_each_entry (conn, &imp->imp_conn_list, oic_item) {if (class_check_uuid (&conn->oic_uuid, nids[0 ])) { *uuid = conn->oic_uuid; +spin_unlock (&imp->imp_lock); rc =class_add_nids_to_uuid (&conn->oic_uuid, nids, nid_count); -break ; +return rc; } }spin_unlock (&imp->imp_lock);
fs/lustre/obdclass/lustre_peer.c +17 -1
@@ -81,6 +81,7 @@ int class_add_uuid(const char *uuid, u64 nid) {struct uuid_nid_data *data, *entry;int found =0 ; +int rc;LASSERT (nid !=0 );/* valid newconfig NID is never zero */
@@ -119,9 +120,15 @@ int class_add_uuid(const char *uuid, u64 nid)if (found) {CDEBUG (D_INFO ,"found uuid %s %s cnt=%d\n" , uuid,libcfs_nid2str (nid), entry->un_nid_count); + rc =LNetAddPeer (entry->un_nids, entry->un_nid_count); +CDEBUG (D_INFO ,"Add peer %s rc = %d\n" , +libcfs_nid2str (data->un_nids[0 ]), rc);kfree (data); }else {CDEBUG (D_INFO ,"add uuid %s %s\n" , uuid,libcfs_nid2str (nid)); + rc =LNetAddPeer (data->un_nids, data->un_nid_count); +CDEBUG (D_INFO ,"Add peer %s rc = %d\n" , +libcfs_nid2str (data->un_nids[0 ]), rc); }return 0 ; }
@@ -173,7 +180,8 @@ int class_add_nids_to_uuid(struct obd_uuid *uuid, lnet_nid_t *nids, int nid_count) {struct uuid_nid_data *entry; -int i; +int i, rc; +bool matched = false;if (nid_count >=MTI_NIDS_MAX ) { CDEBUG(D_NET, "too many NIDs (%d) for UUID '%s'\n",
@@ -188,6 +196,8 @@ int class_add_nids_to_uuid(struct obd_uuid *uuid, lnet_nid_t *nids,if (!obd_uuid_equals (&entry->un_uuid, uuid))continue ; + + matched = true;CDEBUG (D_NET ,"Updating UUID '%s'\n" ,obd_uuid2str (uuid));for (i =0 ; i < nid_count; i++) entry->un_nids[i] = nids[i];
@@ -195,6 +205,12 @@ int class_add_nids_to_uuid(struct obd_uuid *uuid, lnet_nid_t *nids,break ; }spin_unlock (&g_uuid_lock); +if (matched) { + rc =LNetAddPeer (entry->un_nids, entry->un_nid_count); +CDEBUG (D_INFO ,"Add peer %s rc = %d\n" , +libcfs_nid2str (entry->un_nids[0 ]), rc); + } +return 0 ; }EXPORT_SYMBOL (class_add_nids_to_uuid);
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 12/15 lnet: peer state to lock primary nid
In reply to James SimmonsFrom: Amir Shehata <[email protected]>
Introduce the following two peer states:
LNET_PEER_LOCK_PRIMARY, set by Lustre to lock the primary NID of a peer to the NID Lustre is configured with
LNET_PEER_BAD_CONFIG, set by LNet if Lustre attempts to set a peer's Primary NID to a NID used as the primary NID of another peer
WC-bug-id: https://jira.whamcloud.com/browse/LU-14668
Lustre-commit: 684943e2d0c2ad09 ("LU-14668 lnet: peer state to lock primary nid")
Reviewed-on: https://review.whamcloud.com/43562
--- include/linux/lnet/lib-types.h | 7 +++++++ 1 file changed, 7 insertions(+)
include/linux/lnet/lib-types.h +7 -0
@@ -812,6 +812,13 @@ struct lnet_peer {#define LNET_PEER_MARK_DELETION BIT(18)/* lnet_peer_del()/lnet_peer_del_locked() has been called on the peer */ #define LNET_PEER_MARK_DELETED BIT(19) +/* lock primary NID to what's requested by ULP */ +#define LNET_PEER_LOCK_PRIMARY BIT(20) +/* this is for informational purposes only. It is set if a peer gets +* configured from Lustre with a primary NID which belongs to another peer +* which is also configured by Lustre as the primary NID. +*/ +#define LNET_PEER_BAD_CONFIG BIT(21)struct lnet_peer_net {/* chain on lp_peer_nets */
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 13/15 lustre: llite: Proved an abstraction for AS_EXITING
In reply to James SimmonsFrom: Shaun Tancheff <[email protected]>
Linux kernel v3.14-7405-g91b0abe36a7b added AS_EXITING flag AS_EXITING flag is set while address_space mapping is exiting.
Provide an abstraction mapping_clear_exiting() to clear the AS_EXITING flag. This balances the kernel mapping_set_existing().
HPE-bug-id: LUS-9977
WC-bug-id: https://jira.whamcloud.com/browse/LU-14787
Lustre-commit: e423a0bd7a4a59be ("LU-14787 libcfs: Proved an abstraction for AS_EXITING")
Reviewed-on: https://review.whamcloud.com/44070
--- fs/lustre/llite/llite_internal.h | 7 +++++++ fs/lustre/llite/vvp_object.c | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-)
fs/lustre/llite/llite_internal.h +7 -0
@@ -32,6 +32,8 @@#ifndef LLITE_INTERNAL_H #define LLITE_INTERNAL_H + +#include <linux/pagemap.h> #include <obd.h> #include <uapi/linux/lustre/lustre_ver.h> #include <lustre_disk.h> /* for s2sbi */
@@ -1110,6 +1112,11 @@ void ll_cl_add(struct file *file, const struct lu_env *env, struct cl_io *io,void ll_cl_remove (struct file *file,const struct lu_env *env);struct ll_cl_context *ll_cl_find (struct file *file); +static inline void mapping_clear_exiting (struct address_space *mapping) +{ +clear_bit (AS_EXITING , &mapping->flags); +} +extern const struct address_space_operations ll_aops;/* llite/file.c */
fs/lustre/llite/vvp_object.c +1 -1
@@ -164,7 +164,7 @@ static int vvp_prune(const struct lu_env *env, struct cl_object *obj) }ll_truncate_inode_pages_final (inode); -clear_bit (AS_EXITING , &inode->i_mapping->flags); +mapping_clear_exiting (inode->i_mapping);return 0 ; }
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 14/15 lnet: socklnd: set conns_per_peer based on link speed
In reply to James SimmonsFrom: Serguei Smirnov <[email protected]>
Specifying conns_per_peer=0 for a ni is now used to set the conns_per_peer as a function of the corresponding link speed as follows:
conns_per_peer = (ilog2(Gbps) / 2 + 1)
Listed below are the resulting defaults for common link speeds:
100Gbps, 200Gbps -> 4
50Gbps -> 3
5Gbps, 10Gbps -> 2
less than 4Gbps -> 1WC-bug-id: https://jira.whamcloud.com/browse/LU-12815
Lustre-commit: c44afcfb72a1c2fd ("LU-12815 socklnd: set conns_per_peer based on link speed")
Reviewed-on: https://review.whamcloud.com/44417
--- net/lnet/klnds/socklnd/socklnd_modparams.c | 75 +++++++++++++++++++++++++++++- 1 file changed, 73 insertions(+), 2 deletions(-)
net/lnet/klnds/socklnd/socklnd_modparams.c +73 -2
@@ -23,6 +23,8 @@#if defined(__x86_64__) || defined(__i386__)#include <asm/hypervisor.h> #endif +#include <linux/inetdevice.h> +#include <linux/ethtool.h> #define CURRENT_LND_VERSION 1
@@ -154,6 +156,75 @@struct ksock_tunables ksocknal_tunables;static struct lnet_ioctl_config_socklnd_tunables default_tunables; +static int ksocklnd_ni_get_eth_intf_speed (struct lnet_ni *ni) +{ +const struct in_ifaddr *ifa; +struct net_device *dev; +int intf_idx =-1 ; +int ret =-1 ; + +rtnl_lock (); +for_each_netdev (ni->ni_net_ns, dev) { +int flags =dev_get_flags (dev); +struct in_device *in_dev; + +if (flags &IFF_LOOPBACK )/* skip the loopback IF */ +continue ; + +if (!(flags &IFF_UP )) +continue ; + + in_dev =__in_dev_get_rcu (dev); +if (!in_dev) +continue ; + +in_dev_for_each_ifa_rcu (ifa, in_dev) { +if (strcmp (ifa->ifa_label, ni->ni_interface) ==0 ) + intf_idx = dev->ifindex; + } +if (intf_idx >=0 ) +break ; + } +if (intf_idx >=0 ) { +struct ethtool_link_ksettings cmd; +int ethtool_ret; + +/* Some devices may not be providing link settings */ + ethtool_ret =__ethtool_get_link_ksettings (dev, &cmd); +if (!ethtool_ret) + ret = cmd.base.speed; +else + ret = ethtool_ret; + } +rtnl_unlock (); + +return ret; +} + +static int ksocklnd_speed2cpp (int speed) +{ +/* Use the minimum of 1Gbps to avoid calling ilog2 with 0 */ +if (speed <1000 ) + speed =1000 ; + +/* Pick heuristically optimal conns_per_peer value +* for the specified ethernet interface speed (Mbps) +*/ +return ilog2 (speed /1000 ) /2 +1 ; +} + +static int ksocklnd_lookup_conns_per_peer (struct lnet_ni *ni) +{ +int cpp =DEFAULT_CONNS_PER_PEER ; +int speed =ksocklnd_ni_get_eth_intf_speed (ni); + +CDEBUG (D_NET ,"intf %s speed %d\n" , ni->ni_interface, speed); +if (speed >0 ) + cpp =ksocklnd_speed2cpp (speed); + +return cpp; +} +int ksocknal_tunables_init (void ) { default_tunables.lnd_version =CURRENT_LND_VERSION ;
@@ -248,6 +319,6 @@ void ksocknal_tunables_setup(struct lnet_ni *ni) *ksocknal_tunables.ksnd_peerrtrcredits;if (!tunables->lnd_conns_per_peer) - tunables->lnd_conns_per_peer = (conns_per_peer) ? - conns_per_peer :DEFAULT_CONNS_PER_PEER ; + tunables->lnd_conns_per_peer = +ksocklnd_lookup_conns_per_peer (ni); }
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
PATCH 15/15 lustre: update version to 2.14.54
In reply to James SimmonsFrom: Oleg Drokin <[email protected]>
New tag 2.14.54
--- include/uapi/linux/lustre/lustre_ver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/uapi/linux/lustre/lustre_ver.h +2 -2
@@ -3,9 +3,9 @@#define LUSTRE_MAJOR 2#define LUSTRE_MINOR 14 -#define LUSTRE_PATCH 53 +#define LUSTRE_PATCH 54#define LUSTRE_FIX 0 -#define LUSTRE_VERSION_STRING "2.14.53" +#define LUSTRE_VERSION_STRING "2.14.54"#define OBD_OCD_VERSION (major, minor, patch, fix) \ (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix))
Signature · 6 lines
1.8.3.1 _______________________________________________ lustre-devel mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org