pnfs_submit file layout client device management
Subsystems (2)
cover pnfs_submit file layout client device management
Apply against 2.6.34-rc3 pnfs branch. For pNFS client submission. Only touches the file layout driver.
This set of patches implements several design changes enmerated at
http://wiki.linux-nfs.org/wiki/index.php/Porposed_Device_ Manangement_Design.
- Only call GETDEVICEINFO from one place
- Only cache the first data server in the multipath_list4 array.
- Wait to connect to data server when needed for I/O
- Move data server cache to stand alone cache
Also some cleanup, and function interfaces are changed in preparation for the soon to come generic deviceid cache.
0001-SQUASHME-pnfs_submit-remove-unused-getdevicelist-fun.patch 0002-SQUASHME-pnfs-submit-call-get_device_info-from-filel.patch 0003-SQUASHME-pnfs_submit-remove-multilist4-caching.patch 0004-SQUASHME-pnfs_submit-wait-for-I-O-for-data-server-co.patch 0005-SQUASHME-pnfs_submit-remove-xdr-macros.patch 0006-SQUASHME-pnfs_submit-remove-unused-nfs4_pnfs_device_.patch 0007-SQUASHME-pnfs_submit-change-get_device_info-paramete.patch 0008-SQUASHME-pnfs_submit-change-decode_and_add_device-pa.patch 0009-SQUASHME-pnfs_submit-change-decode_device-parameter.patch 0010-SQUASHME-pnfs_submit-change-decode_and_add_ds-parame.patch 0011-SQUASHME-pnfs_submit-change-nfs4_pnfs_ds_add-paramet.patch 0012-SQUASHME-pnfs_submit-remove-filelayout-CB_NOTIFY_DEV.patch 0013-SQUASHME-pnfs_submit-stand-alone-data-server-cache.patch 0014-SQUASHME-pnfs_submit-remove-kfree-from-under-spinloc.patch
Testing:CONFIG_NFS_V4_1 set:
pNFS mount, NFSv4.1 mount, NFSv4.0 mount:
Connecatathon tests pass. pyNFS testclient tests pass.
CONFIG_NFS_V4_1 not set:
Connecatahon tests pass.
-->Andy
PATCH 1/14 SQUASHME: pnfs_submit: remove unused getdevicelist functions
From: Andy Adamson <[email protected]>
--- fs/nfs/nfs4filelayout.h | 4 ---- fs/nfs/nfs4filelayoutdev.c | 27 --------------------------- 2 files changed, 0 insertions(+), 31 deletions(-)
fs/nfs/nfs4filelayout.h +0 -4
@@ -106,10 +106,6 @@ int nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg,loff_t offset, size_t count, struct nfs4_pnfs_dserver *dserver); -int decode_and_add_devicelist (struct filelayout_mount_type *mt,struct pnfs_devicelist *devlist); -int process_deviceid_list (struct filelayout_mount_type *mt, -struct nfs_fh *fh, -struct pnfs_devicelist *devlist);u32 filelayout_dserver_get_index (loff_t offset,struct nfs4_file_layout_dsaddr *di,struct nfs4_filelayout_segment *layout);
fs/nfs/nfs4filelayoutdev.c +0 -27
@@ -662,33 +662,6 @@ decode_and_add_device(struct filelayout_mount_type *mt, struct pnfs_device *dev)return dsaddr; } -/* For each deviceid, if not already in the cache, -* call getdeviceinfo and add the devices associated with -* the deviceid to the list of available devices for this -* mount point. -* Must at some point be followed up with nfs4_pnfs_device_destroy. -*/ -int -process_deviceid_list (struct filelayout_mount_type *mt, -struct nfs_fh *fh, -struct pnfs_devicelist *devlist) -{ -int i; - -dprintk ("--> %s: num_devs=%d\n" , __func__, devlist->num_devs); - -for (i =0 ; i < devlist->num_devs; i++) { -if (!nfs4_file_layout_dsaddr_get (mt, &devlist->dev_id[i])) { -printk (KERN_WARNING -"<-- %s: Error retrieving device %d\n" , - __func__, i); -return 1 ; - } - } -dprintk ("<-- %s: success\n" , __func__); -return 0 ; -} -/* Retrieve the information for dev_id, add it to the list * of available devices, and return it. */
Signature · 1 line
1.6.6
PATCH 2/14 SQUASHME pnfs-submit: call get_device_info from filelayout_check
From: Andy Adamson <[email protected]>
filelayout_check() validates a LAYOUTGET return including the deviceid. If the deviceid is not cached, GETDEVICEINFO is called.
Once cached, the deviceid will only be uncached upon umount or a return of all refering layouts. So, there is no need for a get_device_info call in the I/O paths.
Remove nfs4_file_layout_dsaddr_get.
--- fs/nfs/nfs4filelayout.c | 11 +++++++---- fs/nfs/nfs4filelayout.h | 6 ++++-- fs/nfs/nfs4filelayoutdev.c | 27 +++------------------------ 3 files changed, 14 insertions(+), 30 deletions(-)
fs/nfs/nfs4filelayout.c +7 -4
@@ -413,12 +413,15 @@ filelayout_check_layout(struct pnfs_layout_type *lo,struct nfs_server *nfss =NFS_SERVER (PNFS_INODE (lo));dprintk ("--> %s\n" , __func__); - dsaddr =nfs4_file_layout_dsaddr_get (FILE_MT (PNFS_INODE (lo)), + dsaddr =nfs4_pnfs_device_item_find (FILE_MT (PNFS_INODE (lo))->hlist, &fl->dev_id);if (dsaddr ==NULL ) { -dprintk ("%s NO device for dev_id %s\n" , + dsaddr =get_device_info (FILE_MT (PNFS_INODE (lo)), &fl->dev_id); +if (dsaddr ==NULL ) { +dprintk ("%s NO device for dev_id %s\n" , __func__,deviceid_fmt (&fl->dev_id)); - goto out; + goto out; + } }if (fl->first_stripe_index <0 || fl->first_stripe_index > dsaddr->stripe_count) {
@@ -645,7 +648,7 @@ filelayout_commit(struct pnfs_layout_type *layoutid, int sync, stripesz =filelayout_get_stripesize (layoutid);dprintk ("%s stripesize %Zd\n" , __func__, stripesz); - dsaddr =nfs4_file_layout_dsaddr_get (FILE_MT (data->inode), + dsaddr =nfs4_pnfs_device_item_find (FILE_MT (data->inode)->hlist, &nfslay->dev_id);if (dsaddr ==NULL ) { data->pdata.pnfs_error = -EIO ;
fs/nfs/nfs4filelayout.h +4 -2
@@ -121,7 +121,9 @@ u32 filelayout_dserver_get_index(loff_t offset, } while (0) struct nfs4_file_layout_dsaddr * -nfs4_file_layout_dsaddr_get (struct filelayout_mount_type *, -struct pnfs_deviceid *); +nfs4_pnfs_device_item_find (struct nfs4_pnfs_dev_hlist *hlist, +struct pnfs_deviceid *dev_id); +struct nfs4_file_layout_dsaddr * +get_device_info (struct filelayout_mount_type *mt,struct pnfs_deviceid *dev_id);#endif /* FS_NFS_NFS4FILELAYOUT_H */
fs/nfs/nfs4filelayoutdev.c +3 -24
@@ -52,13 +52,6 @@#define NFSDBG_FACILITY NFSDBG_PNFS_LD -struct nfs4_file_layout_dsaddr *nfs4_file_layout_dsaddr_get ( -struct filelayout_mount_type *mt, -struct pnfs_deviceid *dev_id); -struct nfs4_file_layout_dsaddr *nfs4_pnfs_device_item_find ( -struct nfs4_pnfs_dev_hlist *hlist, -struct pnfs_deviceid *dev_id); -void print_ds_list (struct nfs4_multipath *multipath) {
@@ -665,7 +658,7 @@ decode_and_add_device(struct filelayout_mount_type *mt, struct pnfs_device *dev)/* Retrieve the information for dev_id, add it to the list * of available devices, and return it. */ -static struct nfs4_file_layout_dsaddr * +struct nfs4_file_layout_dsaddr *get_device_info (struct filelayout_mount_type *mt,struct pnfs_deviceid *dev_id) {
@@ -737,21 +730,6 @@ out_free: }struct nfs4_file_layout_dsaddr * -nfs4_file_layout_dsaddr_get (struct filelayout_mount_type *mt, -struct pnfs_deviceid *dev_id) -{ -struct nfs4_file_layout_dsaddr *dsaddr; - -read_lock (&mt->hlist->dev_lock); - dsaddr =_device_lookup (mt->hlist, dev_id); -read_unlock (&mt->hlist->dev_lock); - -if (dsaddr ==NULL ) - dsaddr =get_device_info (mt, dev_id); -return dsaddr; -} - -struct nfs4_file_layout_dsaddr *nfs4_pnfs_device_item_find (struct nfs4_pnfs_dev_hlist *hlist,struct pnfs_deviceid *dev_id) {
@@ -798,7 +776,8 @@ nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg,if (!layout)return 1 ; - dsaddr =nfs4_file_layout_dsaddr_get (FILE_MT (inode), &layout->dev_id); + dsaddr =nfs4_pnfs_device_item_find (FILE_MT (inode)->hlist, + &layout->dev_id);if (dsaddr ==NULL )return 1 ;
Signature · 1 line
1.6.6
PATCH 3/14 SQUASHME pnfs_submit: remove multilist4 caching
From: Andy Adamson <[email protected]>
We only support the use of the first multipath4 data server.
Remove the multipath structures from struct nfs4_file_layout_dsaddr and regroup it's allocation to fit the new strucuture.
Refactor the associated debug print routines.
--- fs/nfs/nfs4filelayout.c | 44 ++++--------- fs/nfs/nfs4filelayout.h | 12 +--- fs/nfs/nfs4filelayoutdev.c | 151 +++++++++++++++++++------------------------- 3 files changed, 81 insertions(+), 126 deletions(-)
fs/nfs/nfs4filelayout.c +13 -31
@@ -252,7 +252,6 @@ filelayout_read_pagelist(struct pnfs_layout_type *layoutid,struct inode *inode =PNFS_INODE (layoutid);struct nfs4_filelayout_segment *flseg;struct nfs4_pnfs_dserver dserver; -struct nfs4_pnfs_ds *ds;int status; dprintk("--> %s ino %lu nr_pages %d pgbase %u req %Zu@%llu\n",
@@ -271,14 +270,12 @@ filelayout_read_pagelist(struct pnfs_layout_type *layoutid,return PNFS_NOT_ATTEMPTED ; } - ds = dserver.multipath->ds_list[0 ]; -dprintk ("%s USE DS:ip %x %s\n" , __func__, -htonl (ds->ds_ip_addr), ds->r_addr); +htonl (dserver.ds->ds_ip_addr), dserver.ds->r_addr);/* just try the first data server for the index..*/ - data->fldata.pnfs_client = ds->ds_clp->cl_rpcclient; - data->fldata.ds_nfs_client = ds->ds_clp; + data->fldata.pnfs_client = dserver.ds->ds_clp->cl_rpcclient; + data->fldata.ds_nfs_client = dserver.ds->ds_clp; data->args.fh = dserver.fh;/* Now get the file offset on the dserver
@@ -300,16 +297,6 @@ filelayout_read_pagelist(struct pnfs_layout_type *layoutid,return PNFS_ATTEMPTED ; } -void -print_ds (struct nfs4_pnfs_ds *ds) -{ -dprintk (" ds->ds_ip_addr %x\n" ,htonl (ds->ds_ip_addr)); -dprintk (" ds->ds_port %hu\n" ,ntohs (ds->ds_port)); -dprintk (" ds->ds_clp %p\n" , ds->ds_clp); -dprintk (" ds->ds_count %d\n" ,atomic_read (&ds->ds_count)); -dprintk (" %s\n" , ds->r_addr); -} -/* Perform async writes. */ static enum pnfs_try_status filelayout_write_pagelist(struct pnfs_layout_type *layoutid,
@@ -324,7 +311,6 @@ filelayout_write_pagelist(struct pnfs_layout_type *layoutid,struct inode *inode =PNFS_INODE (layoutid);struct nfs4_filelayout_segment *flseg =LSEG_LD_DATA (data->pdata.lseg);struct nfs4_pnfs_dserver dserver; -struct nfs4_pnfs_ds *ds;int status; dprintk("--> %s ino %lu nr_pages %d pgbase %u req %Zu@%llu sync %d\n",
@@ -342,16 +328,13 @@ filelayout_write_pagelist(struct pnfs_layout_type *layoutid,return PNFS_NOT_ATTEMPTED ; } -/* use the first multipath data server */ - ds = dserver.multipath->ds_list[0 ]; -dprintk ("%s ino %lu %Zu@%llu DS:%x:%hu %s\n" , __func__, inode->i_ino, count, offset, -htonl (ds->ds_ip_addr),ntohs (ds->ds_port), - ds->r_addr); +htonl (dserver.ds->ds_ip_addr),ntohs (dserver.ds->ds_port), + dserver.ds->r_addr); - data->fldata.pnfs_client = ds->ds_clp->cl_rpcclient; - data->fldata.ds_nfs_client = ds->ds_clp; + data->fldata.pnfs_client = dserver.ds->ds_clp->cl_rpcclient; + data->fldata.ds_nfs_client = dserver.ds->ds_clp; data->args.fh = dserver.fh;/* Get the file offset on the dserver. Set the write offset to
@@ -396,8 +379,8 @@ filelayout_free_layout(void *layoutid)* Make sure layout segment parameters are sane WRT the device. * * Notes: -* 1) current code insists that # stripe index = # multipath devices which -* is wrong. +* 1) current code insists that # stripe index = # data servers in ds_list +* which is wrong. * 2) pattern_offset is ignored and must == 0 which is wrong; * 3) the pattern_offset needs to be a mutliple of the stripe unit. * 4) stripe unit is multiple of page size
@@ -443,9 +426,9 @@ filelayout_check_layout(struct pnfs_layout_type *lo, }/* XXX only support SPARSE packing. Don't support use MDS open fh */ -if (!(fl->num_fh ==1 || fl->num_fh == dsaddr->multipath_count)) { -dprintk ("%s num_fh %u not equal to 1 or multipath_count %u\n" , - __func__, fl->num_fh, dsaddr->multipath_count); +if (!(fl->num_fh ==1 || fl->num_fh == dsaddr->ds_num)) { +dprintk ("%s num_fh %u not equal to 1 or ds_num %u\n" , + __func__, fl->num_fh, dsaddr->ds_num); goto out; }
@@ -714,8 +697,7 @@ filelayout_commit(struct pnfs_layout_type *layoutid, int sync,list_add (&dsdata->pages, &head2);list_del_init (&head2); -/* Just try the first multipath data server */ - ds = dserver.multipath->ds_list[0 ]; + ds = dserver.ds; dsdata->fldata.pnfs_client = ds->ds_clp->cl_rpcclient; dsdata->fldata.ds_nfs_client = ds->ds_clp; dsdata->args.fh = dserver.fh;
fs/nfs/nfs4filelayout.h +4 -8
@@ -42,18 +42,13 @@ struct nfs4_pnfs_ds {char r_addr[29 ]; }; -struct nfs4_multipath { -int num_ds; -struct nfs4_pnfs_ds *ds_list[NFS4_PNFS_MAX_MULTI_DS ]; -}; -struct nfs4_file_layout_dsaddr {struct hlist_node hash_node;/* nfs4_pnfs_dev_hlist dev_list */ struct pnfs_deviceid dev_id;u32 stripe_count;u8 *stripe_indices; -u32 multipath_count; -struct nfs4_multipath *multipath_list; +u32 ds_num; +struct nfs4_pnfs_ds *ds_list[1 ]; };struct nfs4_pnfs_dev_hlist {
@@ -69,7 +64,7 @@ struct nfs4_pnfs_dev_hlist {struct nfs4_pnfs_dserver {struct nfs_fh *fh; -struct nfs4_multipath *multipath; +struct nfs4_pnfs_ds *ds; };struct nfs4_filelayout_segment {
@@ -97,6 +92,7 @@ struct filelayout_mount_type {extern struct pnfs_client_operations *pnfs_callback_ops; +extern void print_ds (struct nfs4_pnfs_ds *ds);char *deviceid_fmt (const struct pnfs_deviceid *dev_id);int nfs4_pnfs_devlist_init (struct nfs4_pnfs_dev_hlist *hlist);void nfs4_pnfs_devlist_destroy (struct nfs4_pnfs_dev_hlist *hlist);
fs/nfs/nfs4filelayoutdev.c +64 -87
265 lines not on this page · raw message
Signature · 1 line
1.6.6
PATCH 4/14 SQUASHME pnfs_submit: wait for I/O for data server connect
From: Andy Adamson <[email protected]>
Don't create (and maintain) a session to a data server until needed.
--- fs/nfs/nfs4filelayoutdev.c | 34 ++++++++++++++++------------------ 1 files changed, 16 insertions(+), 18 deletions(-)
fs/nfs/nfs4filelayoutdev.c +16 -18
@@ -458,10 +458,9 @@ nfs4_pnfs_ds_add(struct filelayout_mount_type *mt, struct nfs4_pnfs_ds **dsp,static struct nfs4_pnfs_ds *decode_and_add_ds (uint32_t **pp,struct filelayout_mount_type *mt) { -struct nfs_server *mds_srv =NFS_SB (mt->fl_sb);struct nfs4_pnfs_ds *ds =NULL ;char r_addr[29 ];/* max size of ip/port string */ -int len, err; +int len;u32 ip_addr, port;int tmp[6 ];uint32_t *p = *pp;
@@ -495,19 +494,6 @@ decode_and_add_ds(uint32_t **pp, struct filelayout_mount_type *mt)nfs4_pnfs_ds_add (mt, &ds, ip_addr, port, r_addr, len); -/* XXX: Don't connect to data servers here, because we -* don't want any un-used (never used!) connections. -* We should wait until I/O demands use of the data server. -*/ -if (!ds->ds_clp) { - err =nfs4_pnfs_ds_create (mds_srv, ds); -if (err) { -printk (KERN_ERR "%s nfs4_pnfs_ds_create error %d\n" , - __func__, err); - goto out_err; - } - } -dprintk ("%s: addr:port string = %s\n" , __func__, r_addr);return ds; out_err:
@@ -746,6 +732,7 @@ nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg, {struct nfs4_filelayout_segment *layout =LSEG_LD_DATA (lseg);struct inode *inode =PNFS_INODE (lseg->layout); +struct nfs_server *mds_srv =NFS_SERVER (inode);struct nfs4_file_layout_dsaddr *dsaddr;u64 tmp, tmp2;u32 stripe_idx, end_idx, ds_idx;
@@ -776,13 +763,24 @@ nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg,BUG_ON (stripe_idx >= dsaddr->stripe_count); ds_idx = dsaddr->stripe_indices[stripe_idx]; - dserver->ds = dsaddr->ds_list[ds_idx]; - -if (dserver->ds ==NULL ) { +if (dsaddr->ds_list[ds_idx] ==NULL ) {printk (KERN_ERR "%s: No data server for device id (%s)!! \n" , __func__,deviceid_fmt (&layout->dev_id));return 1 ; } + +if (!dsaddr->ds_list[ds_idx]->ds_clp) { +int err; + + err =nfs4_pnfs_ds_create (mds_srv, dsaddr->ds_list[ds_idx]); +if (err) { +printk (KERN_ERR "%s nfs4_pnfs_ds_create error %d\n" , + __func__, err); +return 1 ; + } + } + dserver->ds = dsaddr->ds_list[ds_idx]; +if (layout->num_fh ==1 ) dserver->fh = &layout->fh_array[0 ];else
Signature · 1 line
1.6.6
PATCH 5/14 SQUASHME pnfs_submit: remove xdr macros
From: Andy Adamson <[email protected]>
As per the rest of the nfs client, remove the old xdr macros
--- fs/nfs/nfs4filelayout.c | 16 +++++++++------- fs/nfs/nfs4filelayout.h | 11 ----------- fs/nfs/nfs4filelayoutdev.c | 25 +++++++++++++------------ 3 files changed, 22 insertions(+), 30 deletions(-)
fs/nfs/nfs4filelayout.c +9 -7
@@ -459,8 +459,9 @@ filelayout_set_layout(struct nfs4_filelayout *flo,dprintk ("%s: set_layout_map Begin\n" , __func__); -COPYMEM (&fl->dev_id,NFS4_PNFS_DEVICEID4_SIZE ); -READ32 (nfl_util); +memcpy (&fl->dev_id, p,NFS4_PNFS_DEVICEID4_SIZE ); + p +=XDR_QUADLEN (NFS4_PNFS_DEVICEID4_SIZE ); + nfl_util =be32_to_cpup (p++);if (nfl_util &NFL4_UFLG_COMMIT_THRU_MDS ) fl->commit_through_mds =1 ;if (nfl_util &NFL4_UFLG_DENSE )
@@ -477,9 +478,9 @@ filelayout_set_layout(struct nfs4_filelayout *flo, flo->stripe_unit = fl->stripe_unit; } -READ32 (fl->first_stripe_index); -READ64 (fl->pattern_offset); -READ32 (fl->num_fh); + fl->first_stripe_index =be32_to_cpup (p++); + p =xdr_decode_hyper (p, &fl->pattern_offset); + fl->num_fh =be32_to_cpup (p++); dprintk("%s: nfl_util 0x%X num_fh %u fsi %u po %llu dev_id %s\n", __func__, nfl_util, fl->num_fh, fl->first_stripe_index,
@@ -499,7 +500,7 @@ filelayout_set_layout(struct nfs4_filelayout *flo,for (i =0 ; i < fl->num_fh; i++) {/* fh */ -READ32 (fl->fh_array[i].size); + fl->fh_array[i].size =be32_to_cpup (p++);if (sizeof (struct nfs_fh ) < fl->fh_array[i].size) {printk (KERN_ERR "Too big fh %d received %d\n" , i, fl->fh_array[i].size);
@@ -509,7 +510,8 @@ filelayout_set_layout(struct nfs4_filelayout *flo, fl->num_fh =0 ;break ; } -COPYMEM (fl->fh_array[i].data, fl->fh_array[i].size); +memcpy (fl->fh_array[i].data, p, fl->fh_array[i].size); + p +=XDR_QUADLEN (fl->fh_array[i].size);dprintk ("DEBUG: %s: fh len %d\n" , __func__, fl->fh_array[i].size); }
fs/nfs/nfs4filelayout.h +0 -11
@@ -105,17 +105,6 @@ int nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg,u32 filelayout_dserver_get_index (loff_t offset,struct nfs4_file_layout_dsaddr *di,struct nfs4_filelayout_segment *layout); - -#define READ32 (x) (x) = ntohl(*p++) -#define READ64 (x) do { \ - (x) = (u64)ntohl(*p++) << 32; \ - (x) |= ntohl(*p++); \ -} while (0) -#define COPYMEM (x,nbytes) do { \ - memcpy((x), p, nbytes); \ - p += XDR_QUADLEN(nbytes); \ -} while (0) -struct nfs4_file_layout_dsaddr *nfs4_pnfs_device_item_find (struct nfs4_pnfs_dev_hlist *hlist,struct pnfs_deviceid *dev_id);
fs/nfs/nfs4filelayoutdev.c +13 -12
@@ -88,8 +88,8 @@ deviceid_fmt(const struct pnfs_deviceid *dev_id)uint32_t *p = (uint32_t *)dev_id->data;uint64_t major, minor; -READ64 (major); -READ64 (minor); + p =xdr_decode_hyper (p, &major); + p =xdr_decode_hyper (p, &minor);sprintf (buf,"%08llu %08llu" , major, minor);return buf;
@@ -467,7 +467,7 @@ decode_and_add_ds(uint32_t **pp, struct filelayout_mount_type *mt)dprintk ("%s enter\n" , __func__);/* check and skip r_netid */ -READ32 (len); + len =be32_to_cpup (p++);/* "tcp" */ if (len !=3 ) { printk("%s: ERROR: non TCP r_netid len %d\n",
@@ -476,15 +476,16 @@ decode_and_add_ds(uint32_t **pp, struct filelayout_mount_type *mt) }/* Read the bytes into a temporary buffer */ /* XXX: should probably sanity check them */ -READ32 (tmp[0 ]); + tmp[0 ] =be32_to_cpup (p++); -READ32 (len); + len =be32_to_cpup (p++);if (len >29 ) {printk ("%s: ERROR: Device ip/port too long (%d)\n" , __func__, len); goto out_err; } -COPYMEM (r_addr, len); +memcpy (r_addr, p, len); + p +=XDR_QUADLEN (len); *pp = p; r_addr[len] ='\0' ; sscanf(r_addr, "%d.%d.%d.%d.%d.%d", &tmp[0], &tmp[1],
@@ -513,7 +514,7 @@ decode_device(struct filelayout_mount_type *mt, struct pnfs_device *pdev)struct nfs4_file_layout_dsaddr *dsaddr;/* Get the stripe count (number of stripe index) */ -READ32 (cnt); + cnt =be32_to_cpup (p++);dprintk ("%s stripe count %d\n" , __func__, cnt);if (cnt >NFS4_PNFS_MAX_STRIPE_CNT ) { printk(KERN_WARNING "%s: stripe count %d greater than "
@@ -525,7 +526,7 @@ decode_device(struct filelayout_mount_type *mt, struct pnfs_device *pdev)/* Check the multipath list count */ indicesp = p; p +=XDR_QUADLEN (cnt <<2 ); -READ32 (num); + num =be32_to_cpup (p++);dprintk ("%s ds_num %u\n" , __func__, num);if (num >NFS4_PNFS_MAX_MULTI_CNT ) { printk(KERN_WARNING "%s: multipath count %d greater than "
@@ -552,7 +553,7 @@ decode_device(struct filelayout_mount_type *mt, struct pnfs_device *pdev) p = indicesp; indexp = &dsaddr->stripe_indices[0 ];for (i =0 ; i < dsaddr->stripe_count; i++) { -READ32 (dummy); + dummy =be32_to_cpup (p++); *indexp = dummy;/* bound by NFS4_PNFS_MAX_MULTI_CNT */ indexp++; }
@@ -562,7 +563,7 @@ decode_device(struct filelayout_mount_type *mt, struct pnfs_device *pdev)for (i =0 ; i < dsaddr->ds_num; i++) {int j; -READ32 (dummy);/* multipath count */ + dummy =be32_to_cpup (p++);/* multipath count */ if (dummy >1 ) {printk (KERN_WARNING "%s: Multipath count %d not supported, "
@@ -577,9 +578,9 @@ decode_device(struct filelayout_mount_type *mt, struct pnfs_device *pdev) }else {u32 len;/* skip extra multipath */ -READ32 (len); + len =be32_to_cpup (p++); p +=XDR_QUADLEN (len); -READ32 (len); + len =be32_to_cpup (p++); p +=XDR_QUADLEN (len);continue ; }
Signature · 1 line
1.6.6
PATCH 6/14 SQUASHME pnfs_submit: remove unused nfs4_pnfs_device_put
From: Andy Adamson <[email protected]>
--- fs/nfs/nfs4filelayoutdev.c | 21 --------------------- 1 files changed, 0 insertions(+), 21 deletions(-)
fs/nfs/nfs4filelayoutdev.c +0 -21
@@ -795,24 +795,3 @@ nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg,return 0 ; } - -/* Currently not used. -* I have disabled checking the device count until we can think of a good way -* to call nfs4_pnfs_device_put in a generic way from the pNFS client. -* The only way I think think of is to put the nfs4_file_layout_dsaddr directly -* in the nfs4_write/read_data structure, which breaks the clear line between -* the pNFS client and layout drivers. If I did do this, then I could call -* an ioctl on the NFSv4 file layout driver to decrement the device count. -*/ -#if 0 -static void -nfs4_pnfs_device_put (struct nfs_server *server, -struct nfs4_pnfs_dev_hlist *hlist, -struct nfs4_file_layout_dsaddr *dsaddr) -{ -dprintk ("nfs4_pnfs_device_put: dev_id=%u\n" , dsaddr->dev_id); -/* XXX Do we need to invoke this put_client? */ -/* server->rpc_ops->put_client(dsaddr->clp); */ -atomic_dec (&dsaddr->count); -} -#endif
Signature · 1 line
1.6.6
PATCH 7/14 SQUASHME pnfs_submit: change get_device_info parameter
From: Andy Adamson <[email protected]>
In preparation to use the generic deviceid cache.
--- fs/nfs/nfs4filelayout.c | 2 +- fs/nfs/nfs4filelayout.h | 2 +- fs/nfs/nfs4filelayoutdev.c | 9 ++++----- 3 files changed, 6 insertions(+), 7 deletions(-)
fs/nfs/nfs4filelayout.c +1 -1
@@ -399,7 +399,7 @@ filelayout_check_layout(struct pnfs_layout_type *lo, dsaddr =nfs4_pnfs_device_item_find (FILE_MT (PNFS_INODE (lo))->hlist, &fl->dev_id);if (dsaddr ==NULL ) { - dsaddr =get_device_info (FILE_MT (PNFS_INODE (lo)), &fl->dev_id); + dsaddr =get_device_info (PNFS_INODE (lo), &fl->dev_id);if (dsaddr ==NULL ) {dprintk ("%s NO device for dev_id %s\n" , __func__,deviceid_fmt (&fl->dev_id));
fs/nfs/nfs4filelayout.h +1 -1
@@ -109,6 +109,6 @@ struct nfs4_file_layout_dsaddr * nfs4_pnfs_device_item_find(struct nfs4_pnfs_dev_hlist *hlist, struct pnfs_deviceid *dev_id);struct nfs4_file_layout_dsaddr * -get_device_info (struct filelayout_mount_type *mt,struct pnfs_deviceid *dev_id); +get_device_info (struct inode *inode,struct pnfs_deviceid *dev_id);#endif /* FS_NFS_NFS4FILELAYOUT_H */
fs/nfs/nfs4filelayoutdev.c +4 -5
@@ -623,8 +623,7 @@ decode_and_add_device(struct filelayout_mount_type *mt, struct pnfs_device *dev)* of available devices, and return it. */ struct nfs4_file_layout_dsaddr * -get_device_info (struct filelayout_mount_type *mt, -struct pnfs_deviceid *dev_id) +get_device_info (struct inode *inode,struct pnfs_deviceid *dev_id) {struct pnfs_device *pdev =NULL ;int maxpages =NFS4_GETDEVINFO_MAXSIZE >>PAGE_SHIFT ;
@@ -632,7 +631,7 @@ get_device_info(struct filelayout_mount_type *mt,struct nfs4_file_layout_dsaddr *dsaddr =NULL ;int rc, i, j, minpages =1 ; -dprintk ("%s mt %p\n" , __func__, mt); +dprintk ("%s \n" , __func__); pdev =kzalloc (sizeof (struct pnfs_device ),GFP_KERNEL );if (pdev ==NULL )return NULL ;
@@ -664,7 +663,7 @@ retry_once:/* TODO: Update types when CB_NOTIFY_DEVICEID is available */ pdev->dev_notify_types =0 ; - rc = pnfs_callback_ops->nfs_getdeviceinfo (mt->fl_sb, pdev); + rc = pnfs_callback_ops->nfs_getdeviceinfo (inode->i_sb, pdev);/* Retry once with the returned mincount if a page was too small */ dprintk ("%s getdevice info returns %d minpages %d\n" , __func__, rc, minpages);
@@ -682,7 +681,7 @@ retry_once:/* Found new device, need to decode it and then add it to the * list of known devices for this mountpoint. */ - dsaddr =decode_and_add_device (mt, pdev); + dsaddr =decode_and_add_device (FILE_MT (inode), pdev); out_free:if (minpages >1 && pdev->area !=NULL )vunmap (pdev->area);
Signature · 1 line
1.6.6
PATCH 8/14 SQUASHME pnfs_submit: change decode_and_add_device parameter
From: Andy Adamson <[email protected]>
In preparation to use the generic deviceid cache.
--- fs/nfs/nfs4filelayoutdev.c | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-)
fs/nfs/nfs4filelayoutdev.c +5 -5
@@ -601,19 +601,19 @@ out_err:* Must at some point be followed up with nfs4_pnfs_device_destroy */ static struct nfs4_file_layout_dsaddr * -decode_and_add_device (struct filelayout_mount_type *mt,struct pnfs_device *dev) +decode_and_add_device (struct inode *inode,struct pnfs_device *dev) {struct nfs4_file_layout_dsaddr *dsaddr; - dsaddr =decode_device (mt, dev); + dsaddr =decode_device (FILE_MT (inode), dev);if (!dsaddr) {printk (KERN_WARNING "%s: Could not decode device\n" , __func__); -nfs4_pnfs_device_destroy (dsaddr, mt->hlist); +nfs4_pnfs_device_destroy (dsaddr,FILE_MT (inode)->hlist);return NULL ; } -if (nfs4_pnfs_device_add (mt, dsaddr)) +if (nfs4_pnfs_device_add (FILE_MT (inode), dsaddr))return NULL ;return dsaddr;
@@ -681,7 +681,7 @@ retry_once:/* Found new device, need to decode it and then add it to the * list of known devices for this mountpoint. */ - dsaddr =decode_and_add_device (FILE_MT (inode), pdev); + dsaddr =decode_and_add_device (inode, pdev); out_free:if (minpages >1 && pdev->area !=NULL )vunmap (pdev->area);
Signature · 1 line
1.6.6
PATCH 9/14 SQUASHME pnfs_submit: change decode_device parameter
From: Andy Adamson <[email protected]>
In preparation to use the generic deviceid cache
--- fs/nfs/nfs4filelayoutdev.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-)
fs/nfs/nfs4filelayoutdev.c +5 -4
@@ -505,7 +505,7 @@ out_err:/* Decode opaque device data and return the result */ static struct nfs4_file_layout_dsaddr * -decode_device (struct filelayout_mount_type *mt,struct pnfs_device *pdev) +decode_device (struct inode *ino,struct pnfs_device *pdev) {int i, dummy;u32 cnt, num;
@@ -572,7 +572,8 @@ decode_device(struct filelayout_mount_type *mt, struct pnfs_device *pdev) }for (j =0 ; j < dummy; j++) {if (j ==0 ) { - dsaddr->ds_list[i] =decode_and_add_ds (&p, mt); + dsaddr->ds_list[i] =decode_and_add_ds (&p, +FILE_MT (ino));if (dsaddr->ds_list[i] ==NULL ) goto out_err_free; }else {
@@ -589,7 +590,7 @@ decode_device(struct filelayout_mount_type *mt, struct pnfs_device *pdev)return dsaddr; out_err_free: -nfs4_pnfs_device_destroy (dsaddr, mt->hlist); +nfs4_pnfs_device_destroy (dsaddr,FILE_MT (ino)->hlist); out_err:dprintk ("%s ERROR: returning NULL\n" , __func__);return NULL ;
@@ -605,7 +606,7 @@ decode_and_add_device(struct inode *inode, struct pnfs_device *dev) {struct nfs4_file_layout_dsaddr *dsaddr; - dsaddr =decode_device (FILE_MT (inode), dev); + dsaddr =decode_device (inode, dev);if (!dsaddr) {printk (KERN_WARNING "%s: Could not decode device\n" , __func__);
Signature · 1 line
1.6.6
PATCH 10/14 SQUASHME pnfs_submit: change decode_and_add_ds parameter
From: Andy Adamson <[email protected]>
In preparation to use the generic deviceid cache
--- fs/nfs/nfs4filelayoutdev.c | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-)
fs/nfs/nfs4filelayoutdev.c +3 -4
@@ -456,7 +456,7 @@ nfs4_pnfs_ds_add(struct filelayout_mount_type *mt, struct nfs4_pnfs_ds **dsp, }static struct nfs4_pnfs_ds * -decode_and_add_ds (uint32_t **pp,struct filelayout_mount_type *mt) +decode_and_add_ds (uint32_t **pp,struct inode *inode) {struct nfs4_pnfs_ds *ds =NULL ;char r_addr[29 ];/* max size of ip/port string */
@@ -493,7 +493,7 @@ decode_and_add_ds(uint32_t **pp, struct filelayout_mount_type *mt) ip_addr =htonl ((tmp[0 ]<<24 ) | (tmp[1 ]<<16 ) | (tmp[2 ]<<8 ) | (tmp[3 ])); port =htons ((tmp[4 ] <<8 ) | (tmp[5 ])); -nfs4_pnfs_ds_add (mt, &ds, ip_addr, port, r_addr, len); +nfs4_pnfs_ds_add (FILE_MT (inode), &ds, ip_addr, port, r_addr, len);dprintk ("%s: addr:port string = %s\n" , __func__, r_addr);return ds;
@@ -572,8 +572,7 @@ decode_device(struct inode *ino, struct pnfs_device *pdev) }for (j =0 ; j < dummy; j++) {if (j ==0 ) { - dsaddr->ds_list[i] =decode_and_add_ds (&p, -FILE_MT (ino)); + dsaddr->ds_list[i] =decode_and_add_ds (&p,ino);if (dsaddr->ds_list[i] ==NULL ) goto out_err_free; }else {
Signature · 1 line
1.6.6
PATCH 11/14 SQUASHME pnfs_submit: change nfs4_pnfs_ds_add parameter
From: Andy Adamson <[email protected]>
In preparation to use the generic deviceid cache
--- fs/nfs/nfs4filelayoutdev.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-)
fs/nfs/nfs4filelayoutdev.c +3 -3
@@ -416,11 +416,11 @@ nfs4_pnfs_device_add(struct filelayout_mount_type *mt, }static void -nfs4_pnfs_ds_add (struct filelayout_mount_type *mt,struct nfs4_pnfs_ds **dsp, +nfs4_pnfs_ds_add (struct inode *inode,struct nfs4_pnfs_ds **dsp,u32 ip_addr,u32 port,char *r_addr,int len) {struct nfs4_pnfs_ds *tmp_ds, *ds; -struct nfs4_pnfs_dev_hlist *hlist = mt->hlist; +struct nfs4_pnfs_dev_hlist *hlist =FILE_MT (inode)->hlist; *dsp =NULL ;
@@ -493,7 +493,7 @@ decode_and_add_ds(uint32_t **pp, struct inode *inode) ip_addr =htonl ((tmp[0 ]<<24 ) | (tmp[1 ]<<16 ) | (tmp[2 ]<<8 ) | (tmp[3 ])); port =htons ((tmp[4 ] <<8 ) | (tmp[5 ])); -nfs4_pnfs_ds_add (FILE_MT (inode), &ds, ip_addr, port, r_addr, len); +nfs4_pnfs_ds_add (inode, &ds, ip_addr, port, r_addr, len);dprintk ("%s: addr:port string = %s\n" , __func__, r_addr);return ds;
Signature · 1 line
1.6.6
PATCH 12/14 SQUASHME pnfs_submit: remove filelayout CB_NOTIFY_DEVICE support
From: Andy Adamson <[email protected]>
We won't support CB_NOTIFY_DEVICE in the first submission stage. Remove filelayout_device_delete and associated functionality.
--- fs/nfs/nfs4filelayout.c | 16 ---------------- fs/nfs/nfs4filelayout.h | 2 -- fs/nfs/nfs4filelayoutdev.c | 18 ------------------ 3 files changed, 0 insertions(+), 36 deletions(-)
fs/nfs/nfs4filelayout.c +0 -16
@@ -135,21 +135,6 @@ filelayout_uninitialize_mountpoint(struct pnfs_mount_type *mountid)return 0 ; } -int -filelayout_device_delete (struct pnfs_mount_type *mountid, -struct pnfs_deviceid *dev_id) -{ -struct filelayout_mount_type *fl_mt =NULL ; - -if (mountid) { - fl_mt = (struct filelayout_mount_type *)mountid->mountid; - -if (fl_mt !=NULL ) -nfs4_pnfs_dev_destroy (fl_mt->hlist, dev_id); - } -return 0 ; -} -/* This function is used by the layout driver to calculate the * offset of the file on the dserver based on whether the * layout type is STRIPE_DENSE or STRIPE_SPARSE
@@ -797,7 +782,6 @@ struct layoutdriver_io_operations filelayout_io_operations = { .free_lseg = filelayout_free_lseg, .initialize_mountpoint = filelayout_initialize_mountpoint, .uninitialize_mountpoint = filelayout_uninitialize_mountpoint, - .device_delete = filelayout_device_delete, };struct layoutdriver_policy_operations filelayout_policy_operations = {
fs/nfs/nfs4filelayout.h +0 -2
@@ -96,8 +96,6 @@ extern void print_ds(struct nfs4_pnfs_ds *ds);char *deviceid_fmt (const struct pnfs_deviceid *dev_id);int nfs4_pnfs_devlist_init (struct nfs4_pnfs_dev_hlist *hlist);void nfs4_pnfs_devlist_destroy (struct nfs4_pnfs_dev_hlist *hlist); -void nfs4_pnfs_dev_destroy (struct nfs4_pnfs_dev_hlist *hlist, -struct pnfs_deviceid *dev_id); int nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg, loff_t offset, size_t count,
fs/nfs/nfs4filelayoutdev.c +0 -18
@@ -368,24 +368,6 @@ nfs4_pnfs_devlist_destroy(struct nfs4_pnfs_dev_hlist *hlist) } } -/* De-alloc a devices for a mount point. */ -void -nfs4_pnfs_dev_destroy (struct nfs4_pnfs_dev_hlist *hlist, -struct pnfs_deviceid *dev_id) -{ -struct nfs4_file_layout_dsaddr *dsaddr; - -if (hlist ==NULL ) -return ; - -dprintk ("%s: dev_id=%s\n" , __func__,deviceid_fmt (dev_id)); - - dsaddr =nfs4_pnfs_device_item_find (hlist, dev_id); -if (dsaddr) -/* nfs4_pnfs_device_destroy grabs hlist->dev_lock */ -nfs4_pnfs_device_destroy (dsaddr, hlist); -} -/* * Add the device to the list of available devices for this mount point. * The * rpc client is created during first I/O.
Signature · 1 line
1.6.6
PATCH 13/14 SQUASHME pnfs_submit: stand alone data server cache
From: Andy Adamson <[email protected]>
Data servers can be associated with any device ID from any server. Multiple file layout meta data servers can share a data server. Implement a stand alone data server cache to share data servers and their session between deviceid's and meta data servers.
The data server cache is only changed at GETDEVICEINFO return or umount. Once in the cache and referenced by a deviceid structure, the use of the data server in the I/O path is via deviceid struct array indexes. Therefore, nether an hlist nor rwspin locks are needed.
--- fs/nfs/nfs4filelayout.h | 3 +- fs/nfs/nfs4filelayoutdev.c | 57 ++++++++++++++++--------------------------- 2 files changed, 22 insertions(+), 38 deletions(-)
fs/nfs/nfs4filelayout.h +1 -2
@@ -34,7 +34,7 @@ enum stripetype4 {/* Individual ip address */ struct nfs4_pnfs_ds { -struct hlist_node ds_node;/* nfs4_pnfs_dev_hlist dev_dslist */ +struct list_head ds_node;/* nfs4_pnfs_dev_hlist dev_dslist */ u32 ds_ip_addr;u32 ds_port;struct nfs_client *ds_clp;
@@ -54,7 +54,6 @@ struct nfs4_file_layout_dsaddr {struct nfs4_pnfs_dev_hlist {rwlock_t dev_lock;struct hlist_head dev_list[NFS4_PNFS_DEV_HASH_SIZE ]; -struct hlist_head dev_dslist[NFS4_PNFS_DEV_HASH_SIZE ]; };/*
fs/nfs/nfs4filelayoutdev.c +21 -36
126 lines not on this page · raw message
Signature · 1 line
1.6.6
PATCH 14/14 SQUASHME pnfs_submit: remove kfree from under spinlock
From: Andy Adamson <[email protected]>
--- fs/nfs/nfs4filelayoutdev.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-)
fs/nfs/nfs4filelayoutdev.c +3 -3
@@ -410,9 +410,7 @@ nfs4_pnfs_ds_add(struct inode *inode, struct nfs4_pnfs_ds **dsp, ds->ds_ip_addr);list_add (&ds->ds_node, &nfs4_data_server_cache); *dsp = ds; - } -if (tmp_ds !=NULL ) { -destroy_ds (ds); + }else {atomic_inc (&tmp_ds->ds_count); dprintk("%s data server found ip 0x%x, inc'ed ds_count to %d\n", __func__, tmp_ds->ds_ip_addr,
@@ -420,6 +418,8 @@ nfs4_pnfs_ds_add(struct inode *inode, struct nfs4_pnfs_ds **dsp, *dsp = tmp_ds; }spin_unlock (&nfs4_ds_cache_lock); +if (tmp_ds !=NULL ) +destroy_ds (ds); } static struct nfs4_pnfs_ds *
Signature · 1 line
1.6.6
Re: [pnfs] [PATCH 12/14] SQUASHME pnfs_submit: remove filelayout CB_NOTIFY_DEVICE support
In reply to [email protected]OK. I'll keep a patch reverting this patch (reintroducing this functionality) in the pnfs branch, on top of the pnfs-submit branch.
Benny
Quoted 88 lines · [email protected] wrote:
From: Andy Adamson <[email protected]>
We won't support
CB_NOTIFY_DEVICEin the first submission stage. Removefilelayout_device_deleteand associated functionality.Signed-off-by Andy Adamson <[email protected]>--- fs/nfs/nfs4filelayout.c | 16 ---------------- fs/nfs/nfs4filelayout.h | 2 -- fs/nfs/nfs4filelayoutdev.c | 18 ------------------ 3 files changed, 0 insertions(+), 36 deletions(-)fs/nfs/nfs4filelayout.c +0
-16@@ -135,21 +135,6 @@ filelayout_uninitialize_mountpoint(struct pnfs_mount_type *mountid)return 0 ; } -int -filelayout_device_delete (struct pnfs_mount_type *mountid, -struct pnfs_deviceid *dev_id) -{ -struct filelayout_mount_type *fl_mt =NULL ; - -if (mountid) { - fl_mt = (struct filelayout_mount_type *)mountid->mountid; - -if (fl_mt !=NULL ) -nfs4_pnfs_dev_destroy (fl_mt->hlist, dev_id); - } -return 0 ; -} -/* This function is used by the layout driver to calculate the * offset of the file on the dserver based on whether the * layout type is STRIPE_DENSE or STRIPE_SPARSE @@ -797,7 +782,6 @@ struct layoutdriver_io_operations filelayout_io_operations = { .free_lseg = filelayout_free_lseg, .initialize_mountpoint = filelayout_initialize_mountpoint, .uninitialize_mountpoint = filelayout_uninitialize_mountpoint, - .device_delete = filelayout_device_delete, };struct layoutdriver_policy_operations filelayout_policy_operations = {fs/nfs/nfs4filelayout.h +0
-2@@ -96,8 +96,6 @@ extern void print_ds(struct nfs4_pnfs_ds *ds);char *deviceid_fmt (const struct pnfs_deviceid *dev_id);int nfs4_pnfs_devlist_init (struct nfs4_pnfs_dev_hlist *hlist);void nfs4_pnfs_devlist_destroy (struct nfs4_pnfs_dev_hlist *hlist); -void nfs4_pnfs_dev_destroy (struct nfs4_pnfs_dev_hlist *hlist, -struct pnfs_deviceid *dev_id); int nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg, loff_t offset, size_t count,fs/nfs/nfs4filelayoutdev.c +0
-18@@ -368,24 +368,6 @@ nfs4_pnfs_devlist_destroy(struct nfs4_pnfs_dev_hlist *hlist) } } -/* De-alloc a devices for a mount point. */ -void -nfs4_pnfs_dev_destroy (struct nfs4_pnfs_dev_hlist *hlist, -struct pnfs_deviceid *dev_id) -{ -struct nfs4_file_layout_dsaddr *dsaddr; - -if (hlist ==NULL ) -return ; - -dprintk ("%s: dev_id=%s\n" , __func__,deviceid_fmt (dev_id)); - - dsaddr =nfs4_pnfs_device_item_find (hlist, dev_id); -if (dsaddr) -/* nfs4_pnfs_device_destroy grabs hlist->dev_lock */ -nfs4_pnfs_device_destroy (dsaddr, hlist); -} -/* * Add the device to the list of available devices for this mount point. * The * rpc client is created during first I/O.
Re: [pnfs] [PATCH 07/14] SQUASHME pnfs_submit: change get_device_info parameter
In reply to [email protected]Quoted 51 lines · On Apr. 08, 2010, 22:54 +0300, [email protected] wrote:
From: Andy Adamson <[email protected]>
In preparation to use the generic deviceid cache.
Signed-off-by Andy Adamson <[email protected]>--- fs/nfs/nfs4filelayout.c | 2 +- fs/nfs/nfs4filelayout.h | 2 +- fs/nfs/nfs4filelayoutdev.c | 9 ++++----- 3 files changed, 6 insertions(+), 7 deletions(-)fs/nfs/nfs4filelayout.c +1
-1@@ -399,7 +399,7 @@ filelayout_check_layout(struct pnfs_layout_type *lo, dsaddr =nfs4_pnfs_device_item_find (FILE_MT (PNFS_INODE (lo))->hlist, &fl->dev_id);if (dsaddr ==NULL ) { - dsaddr =get_device_info (FILE_MT (PNFS_INODE (lo)), &fl->dev_id); + dsaddr =get_device_info (PNFS_INODE (lo), &fl->dev_id);if (dsaddr ==NULL ) {dprintk ("%s NO device for dev_id %s\n" , __func__,deviceid_fmt (&fl->dev_id));fs/nfs/nfs4filelayout.h +1
-1@@ -109,6 +109,6 @@ struct nfs4_file_layout_dsaddr * nfs4_pnfs_device_item_find(struct nfs4_pnfs_dev_hlist *hlist, struct pnfs_deviceid *dev_id);struct nfs4_file_layout_dsaddr * -get_device_info (struct filelayout_mount_type *mt,struct pnfs_deviceid *dev_id); +get_device_info (struct inode *inode,struct pnfs_deviceid *dev_id);#endif /* FS_NFS_NFS4FILELAYOUT_H */ fs/nfs/nfs4filelayoutdev.c +1
-2@@ -623,8 +623,7 @@ decode_and_add_device(struct filelayout_mount_type *mt, struct pnfs_device *dev)* of available devices, and return it. */ struct nfs4_file_layout_dsaddr * -get_device_info (struct filelayout_mount_type *mt, -struct pnfs_deviceid *dev_id) +get_device_info (struct inode *inode,struct pnfs_deviceid *dev_id) {struct pnfs_device *pdev =NULL ;int maxpages =NFS4_GETDEVINFO_MAXSIZE >>PAGE_SHIFT ;
fs/nfs/nfs4filelayoutdev.c get_device_info(struct filelayout_mount_type *mt, · line 634
Quoted 5 lines
fs/nfs/nfs4filelayoutdev.c +1
-1@@ -632,7 +631,7 @@ get_device_info(struct filelayout_mount_type *mt,struct nfs4_file_layout_dsaddr *dsaddr =NULL ;int rc, i, j, minpages =1 ; -dprintk ("%s mt %p\n" , __func__, mt); +dprintk ("%s \n" , __func__);
nit: checkpatch rightfully complains about the stray space character before the newline. I'll just remove that upon committing, but there are other places
like that we need to clean up before submitting the final version...
Benny
Quoted 21 lines
fs/nfs/nfs4filelayoutdev.c +2
-2pdev =kzalloc (sizeof (struct pnfs_device ),GFP_KERNEL );if (pdev ==NULL )return NULL ;@@ -664,7 +663,7 @@ retry_once:/* TODO: Update types when CB_NOTIFY_DEVICEID is available */ pdev->dev_notify_types =0 ; - rc = pnfs_callback_ops->nfs_getdeviceinfo (mt->fl_sb, pdev); + rc = pnfs_callback_ops->nfs_getdeviceinfo (inode->i_sb, pdev);/* Retry once with the returned mincount if a page was too small */ dprintk ("%s getdevice info returns %d minpages %d\n" , __func__, rc, minpages);@@ -682,7 +681,7 @@ retry_once:/* Found new device, need to decode it and then add it to the * list of known devices for this mountpoint. */ - dsaddr =decode_and_add_device (mt, pdev); + dsaddr =decode_and_add_device (FILE_MT (inode), pdev); out_free:if (minpages >1 && pdev->area !=NULL )vunmap (pdev->area);
Re: [pnfs] [PATCH 02/14] SQUASHME pnfs-submit: call get_device_info from filelayout_check
In reply to [email protected]Quoted 55 lines · On Apr. 08, 2010, 22:53 +0300, [email protected] wrote:
From: Andy Adamson <[email protected]>
filelayout_check()validates a LAYOUTGET return including the deviceid. If the deviceid is not cached, GETDEVICEINFO is called.Once cached, the deviceid will only be uncached upon umount or a return of all refering layouts. So, there is no need for a
get_device_infocall in the I/O paths.Remove
nfs4_file_layout_dsaddr_get.Signed-off-by Andy Adamson <[email protected]>--- fs/nfs/nfs4filelayout.c | 11 +++++++---- fs/nfs/nfs4filelayout.h | 6 ++++-- fs/nfs/nfs4filelayoutdev.c | 27 +++------------------------ 3 files changed, 14 insertions(+), 30 deletions(-)fs/nfs/nfs4filelayout.c +7
-4@@ -413,12 +413,15 @@ filelayout_check_layout(struct pnfs_layout_type *lo,struct nfs_server *nfss =NFS_SERVER (PNFS_INODE (lo));dprintk ("--> %s\n" , __func__); - dsaddr =nfs4_file_layout_dsaddr_get (FILE_MT (PNFS_INODE (lo)), + dsaddr =nfs4_pnfs_device_item_find (FILE_MT (PNFS_INODE (lo))->hlist, &fl->dev_id);if (dsaddr ==NULL ) { -dprintk ("%s NO device for dev_id %s\n" , + dsaddr =get_device_info (FILE_MT (PNFS_INODE (lo)), &fl->dev_id); +if (dsaddr ==NULL ) { +dprintk ("%s NO device for dev_id %s\n" , __func__,deviceid_fmt (&fl->dev_id)); - goto out; + goto out; + } }if (fl->first_stripe_index <0 || fl->first_stripe_index > dsaddr->stripe_count) {@@ -645,7 +648,7 @@ filelayout_commit(struct pnfs_layout_type *layoutid, int sync, stripesz =filelayout_get_stripesize (layoutid);dprintk ("%s stripesize %Zd\n" , __func__, stripesz); - dsaddr =nfs4_file_layout_dsaddr_get (FILE_MT (data->inode), + dsaddr =nfs4_pnfs_device_item_find (FILE_MT (data->inode)->hlist, &nfslay->dev_id);if (dsaddr ==NULL ) { data->pdata.pnfs_error = -EIO ;fs/nfs/nfs4filelayout.h +0
-0
fs/nfs/nfs4filelayout.h u32 filelayout_dserver_get_index(loff_t offset, · line 128
Quoted 10 lines
fs/nfs/nfs4filelayout.h +4
-2@@ -121,7 +121,9 @@ u32 filelayout_dserver_get_index(loff_t offset, }while (0 ) struct nfs4_file_layout_dsaddr * -nfs4_file_layout_dsaddr_get (struct filelayout_mount_type *, -struct pnfs_deviceid *); +nfs4_pnfs_device_item_find (struct nfs4_pnfs_dev_hlist *hlist, +struct pnfs_deviceid *dev_id); +struct nfs4_file_layout_dsaddr * +get_device_info (struct filelayout_mount_type *mt,struct pnfs_deviceid *dev_id);
Andy, this function name is too generic IMO.
better be something more specific to pnfs and file layout. (I'm committing this patchset as is so we can fix that later)
Benny
Quoted 60 lines
fs/nfs/nfs4filelayout.h +0
-0#endif /* FS_NFS_NFS4FILELAYOUT_H */ fs/nfs/nfs4filelayoutdev.c +3
-24@@ -52,13 +52,6 @@#define NFSDBG_FACILITY NFSDBG_PNFS_LD -struct nfs4_file_layout_dsaddr *nfs4_file_layout_dsaddr_get ( -struct filelayout_mount_type *mt, -struct pnfs_deviceid *dev_id); -struct nfs4_file_layout_dsaddr *nfs4_pnfs_device_item_find ( -struct nfs4_pnfs_dev_hlist *hlist, -struct pnfs_deviceid *dev_id); -void print_ds_list (struct nfs4_multipath *multipath) {@@ -665,7 +658,7 @@ decode_and_add_device(struct filelayout_mount_type *mt, struct pnfs_device *dev)/* Retrieve the information for dev_id, add it to the list * of available devices, and return it. */ -static struct nfs4_file_layout_dsaddr * +struct nfs4_file_layout_dsaddr *get_device_info (struct filelayout_mount_type *mt,struct pnfs_deviceid *dev_id) {@@ -737,21 +730,6 @@ out_free: }struct nfs4_file_layout_dsaddr * -nfs4_file_layout_dsaddr_get (struct filelayout_mount_type *mt, -struct pnfs_deviceid *dev_id) -{ -struct nfs4_file_layout_dsaddr *dsaddr; - -read_lock (&mt->hlist->dev_lock); - dsaddr =_device_lookup (mt->hlist, dev_id); -read_unlock (&mt->hlist->dev_lock); - -if (dsaddr ==NULL ) - dsaddr =get_device_info (mt, dev_id); -return dsaddr; -} - -struct nfs4_file_layout_dsaddr *nfs4_pnfs_device_item_find (struct nfs4_pnfs_dev_hlist *hlist,struct pnfs_deviceid *dev_id) {@@ -798,7 +776,8 @@ nfs4_pnfs_dserver_get(struct pnfs_layout_segment *lseg,if (!layout)return 1 ; - dsaddr =nfs4_file_layout_dsaddr_get (FILE_MT (inode), &layout->dev_id); + dsaddr =nfs4_pnfs_device_item_find (FILE_MT (inode)->hlist, + &layout->dev_id);if (dsaddr ==NULL )return 1 ;
Quoted 6 lines · On Thu, Apr 08, 2010 at 03:53:56PM -0400, [email protected] wrote:
Apply against 2.6.34-rc3 pnfs branch. For pNFS client submission. Only touches the file layout driver.
This set of patches implements several design changes enmerated at
http://wiki.linux-nfs.org/wiki/index.php/Porposed_Device_Manangement_Design.
(Should be:
http://wiki.linux-nfs.org/wiki/index.php/Proposed_Device_Management_Design
.)
Quoted 39 lines
- Only call GETDEVICEINFO from one place
- Only cache the first data server in themultipath_list4array.
- Wait to connect to data server when needed for I/O
- Move data server cache to stand alone cacheAlso some cleanup, and function interfaces are changed in preparation for the soon to come generic deviceid cache.
0001-SQUASHME-pnfs_submit-remove-unused-getdevicelist-fun.patch 0002-SQUASHME-pnfs-submit-call-get_device_info-from-filel.patch 0003-SQUASHME-pnfs_submit-remove-multilist4-caching.patch 0004-SQUASHME-pnfs_submit-wait-for-I-O-for-data-server-co.patch 0005-SQUASHME-pnfs_submit-remove-xdr-macros.patch 0006-SQUASHME-pnfs_submit-remove-unused-nfs4_pnfs_device_.patch 0007-SQUASHME-pnfs_submit-change-get_device_info-paramete.patch 0008-SQUASHME-pnfs_submit-change-decode_and_add_device-pa.patch 0009-SQUASHME-pnfs_submit-change-decode_device-parameter.patch 0010-SQUASHME-pnfs_submit-change-decode_and_add_ds-parame.patch 0011-SQUASHME-pnfs_submit-change-nfs4_pnfs_ds_add-paramet.patch 0012-SQUASHME-pnfs_submit-remove-filelayout-CB_NOTIFY_DEV.patch 0013-SQUASHME-pnfs_submit-stand-alone-data-server-cache.patch 0014-SQUASHME-pnfs_submit-remove-kfree-from-under-spinloc.patch
Testing:
CONFIG_NFS_V4_1set:
pNFS mount, NFSv4.1 mount, NFSv4.0 mount:
Connecatathon tests pass. pyNFS testclient tests pass.
CONFIG_NFS_V4_1not set:
Connecatahon tests pass.-->Andy
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
Committed to a new branch in the 2.6.34-rc3 stream: pnfs-submit
Thanks!
Benny
Quoted 45 lines · On Apr. 08, 2010, 22:53 +0300, [email protected] wrote:
Apply against 2.6.34-rc3 pnfs branch. For pNFS client submission. Only touches the file layout driver.
This set of patches implements several design changes enmerated at
http://wiki.linux-nfs.org/wiki/index.php/Porposed_Device_Manangement_Design.- Only call GETDEVICEINFO from one place
- Only cache the first data server in themultipath_list4array.
- Wait to connect to data server when needed for I/O
- Move data server cache to stand alone cacheAlso some cleanup, and function interfaces are changed in preparation for the soon to come generic deviceid cache.
0001-SQUASHME-pnfs_submit-remove-unused-getdevicelist-fun.patch 0002-SQUASHME-pnfs-submit-call-get_device_info-from-filel.patch 0003-SQUASHME-pnfs_submit-remove-multilist4-caching.patch 0004-SQUASHME-pnfs_submit-wait-for-I-O-for-data-server-co.patch 0005-SQUASHME-pnfs_submit-remove-xdr-macros.patch 0006-SQUASHME-pnfs_submit-remove-unused-nfs4_pnfs_device_.patch 0007-SQUASHME-pnfs_submit-change-get_device_info-paramete.patch 0008-SQUASHME-pnfs_submit-change-decode_and_add_device-pa.patch 0009-SQUASHME-pnfs_submit-change-decode_device-parameter.patch 0010-SQUASHME-pnfs_submit-change-decode_and_add_ds-parame.patch 0011-SQUASHME-pnfs_submit-change-nfs4_pnfs_ds_add-paramet.patch 0012-SQUASHME-pnfs_submit-remove-filelayout-CB_NOTIFY_DEV.patch 0013-SQUASHME-pnfs_submit-stand-alone-data-server-cache.patch 0014-SQUASHME-pnfs_submit-remove-kfree-from-under-spinloc.patch
Testing:
CONFIG_NFS_V4_1set:
pNFS mount, NFSv4.1 mount, NFSv4.0 mount:
Connecatathon tests pass. pyNFS testclient tests pass.
CONFIG_NFS_V4_1not set:
Connecatahon tests pass.-->Andy
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs