fs/ufs: Replace kmap() with kmap_local_page
v5 cover fs/ufs: Replace kmap() with kmap_local_page
kmap() is being deprecated in favor of kmap_local_page().
There are two main problems with kmap(): (1) It comes with an overhead as the mapping space is restricted and protected by a global lock for synchronization and (2) it also requires global TLB invalidation when the kmap’s pool wraps and it might block when the mapping space is fully utilized until a slot becomes available.
With kmap_local_page() the mappings are per thread, CPU local, can take page faults, and can be called from any context (including interrupts). It is faster than kmap() in kernels with HIGHMEM enabled. Furthermore, the tasks can be preempted and, when they are scheduled to run again, the kernel virtual addresses are restored and still valid.
Since its use in fs/ufs is safe everywhere, it should be preferred.
Therefore, replace kmap() with kmap_local_page() in fs/ufs. kunmap_local() requires the mapping address, so return that address from ufs_get_page() to be used in ufs_put_page().
This series could have not been ever made because nothing prevented the previous patch from working properly but Al Viro made a long series of very appreciated comments about how many unnecessary and redundant lines of code I could have removed. He could see things I was entirely unable to notice. Furthermore, he also provided solutions and details about how I could decompose a single patch into a small series of three independent units.[1][2][3]
I want to thank him so much for the patience, kindness and the time he decided to spend to provide those analysis and write three messages full of interesting insights.[1][2][3]
Changes from v1:
1/3: No changes.
2/3: Restore the return of "err" that was mistakenly deleted
together with the removal of the "out" label in ufs_add_link(). Thanks to Al Viro.[4]
Return the address of the kmap()'ed page instead of a pointer to a pointer to the mapped page; a page_address() had been overlooked in ufs_get_page(). Thanks to Al Viro.[5]
3/3: Return the kernel virtual address got from the call to
kmap_local_page() after conversion from kmap(). Again thanks to Al Viro.[6]
Changes from v2:
1/3: No changes.
2/3: Rework ufs_get_page() because the previous version had two
errors: (1) It could return an invalid pages with the out argument "page" and (2) it could return "page_address(page)" also in cases where read_mapping_page() returned an error and the page is never kmap()'ed. Thanks to Al Viro.[7]
3/3: Rework ufs_get_page() after conversion to
kmap_local_page(), in accordance to the last changes in 2/3.
Changes from v3:
1/3: No changes. 2/3: No changes. 3/3: Replace kunmap() with kunmap_local().
Changes from v4:
1/4: It was 1/3.
2/4: Move the declaration of a page into an inner loop. Add Ira
Weiny's "Reviewed-by" tag (thanks!).
3/4: Add this patch to use ufs_put_page() to replace three kunmap()
and put_page() in namei.c. Thanks to Ira Weiny who noticed that I had overlooked their presence.
4/4: Remove an unnecessary masking that is already carried out by
kunmap_local() via kunmap_local_indexed(). Add a comment to clarify that a ufs_dir_entry passed to ufs_delete_entry() points in the same page we need the address of. Suggested by Ira Weiny.
[1] https://lore.kernel.org/lkml/Y4E++JERgUMoqfjG@ZenIV/#t
[2] https://lore.kernel.org/lkml/Y4FG0O7VWTTng5yh@ZenIV/#t
[3] https://lore.kernel.org/lkml/Y4ONIFJatIGsVNpf@ZenIV/#t
[4] https://lore.kernel.org/lkml/Y5Zc0qZ3+zsI74OZ@ZenIV/#t
[5] https://lore.kernel.org/lkml/Y5ZZy23FFAnQDR3C@ZenIV/#t
[6] https://lore.kernel.org/lkml/Y5ZcMPzPG9h6C9eh@ZenIV/#t
[7] https://lore.kernel.org/lkml/Y5glgpD7fFifC4Fi@ZenIV/#t
The cover letter of the v1 series is at
https://lore.kernel.org/lkml/[email protected]/
The cover letter of the v2 series is at
https://lore.kernel.org/lkml/[email protected]/
The cover letter of the v3 series is at
https://lore.kernel.org/lkml/[email protected]/
The cover letter of the v4 series is at
https://lore.kernel.org/lkml/[email protected]/
Fabio M. De Francesco (4):
fs/ufs: Use the offset_in_page() helper
fs/ufs: Change the signature of ufs_get_page()
fs/ufs: Use ufs_put_page() in ufs_rename()
fs/ufs: Replace kmap() with kmap_local_page()
fs/ufs/dir.c | 131 +++++++++++++++++++++++++++---------------------- fs/ufs/namei.c | 11 ++--- fs/ufs/ufs.h | 1 + 3 files changed, 78 insertions(+), 65 deletions(-)
Signature · 1 line
2.39.0
PATCH v5 1/4 fs/ufs: Use the offset_in_page() helper
Use the offset_in_page() helper because it is more suitable than doing explicit subtractions between pointers to directory entries and kernel virtual addresses of mapped pages.
--- fs/ufs/dir.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
fs/ufs/dir.c +5 -7
@@ -87,8 +87,7 @@ void ufs_set_link(struct inode *dir, struct ufs_dir_entry *de, struct page *page, struct inode *inode, bool update_times) { - loff_t pos = page_offset(page) + - (char *) de - (char *)page_address (page); + loff_t pos = page_offset(page) + offset_in_page(de);unsigned len =fs16_to_cpu (dir->i_sb, de->d_reclen);int err;
@@ -371,8 +370,7 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode)return -EINVAL ; got_it: - pos =page_offset (page) + - (char *)de - (char *)page_address (page); + pos =page_offset (page) +offset_in_page (de); err =ufs_prepare_chunk (page, pos, rec_len);if (err) goto out_unlock;
@@ -497,8 +495,8 @@ int ufs_delete_entry(struct inode *inode, struct ufs_dir_entry *dir, {struct super_block *sb = inode->i_sb;char *kaddr =page_address (page); -unsigned from = ((char *)dir - kaddr) & ~(UFS_SB (sb)->s_uspi->s_dirblksize -1 ); -unsigned to = ((char *)dir - kaddr) +fs16_to_cpu (sb, dir->d_reclen); +unsigned int from =offset_in_page (dir) & ~(UFS_SB (sb)->s_uspi->s_dirblksize -1 ); +unsigned int to =offset_in_page (dir) +fs16_to_cpu (sb, dir->d_reclen);loff_t pos;struct ufs_dir_entry *pde =NULL ;struct ufs_dir_entry *de = (struct ufs_dir_entry *) (kaddr + from);
@@ -522,7 +520,7 @@ int ufs_delete_entry(struct inode *inode, struct ufs_dir_entry *dir, de =ufs_next_entry (sb, de); }if (pde) - from = (char *)pde - (char *)page_address (page); + from =offset_in_page (pde); pos =page_offset (page) + from;lock_page (page);
Signature · 1 line
2.39.0
Re: [PATCH v5 1/4] fs/ufs: Use the offset_in_page() helper
Quoted 5 lines · Fabio M. De Francesco
Use the
offset_in_page()helper because it is more suitable than doing explicit subtractions between pointers to directory entries and kernel virtual addresses of mapped pages.Cc Ira Weiny <[email protected]>
Quoted 53 lines
Suggested-by Al Viro <[email protected]>Signed-off-by Fabio M. De Francesco <[email protected]>--- fs/ufs/dir.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)fs/ufs/dir.c +5
-7@@ -87,8 +87,7 @@ void ufs_set_link(struct inode *dir, struct ufs_dir_entry *de, struct page *page, struct inode *inode, bool update_times) { - loff_t pos = page_offset(page) + - (char *) de - (char *)page_address (page); + loff_t pos = page_offset(page) + offset_in_page(de);unsigned len =fs16_to_cpu (dir->i_sb, de->d_reclen);int err;@@ -371,8 +370,7 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode)return -EINVAL ; got_it: - pos =page_offset (page) + - (char *)de - (char *)page_address (page); + pos =page_offset (page) +offset_in_page (de); err =ufs_prepare_chunk (page, pos, rec_len);if (err) goto out_unlock;@@ -497,8 +495,8 @@ int ufs_delete_entry(struct inode *inode, struct ufs_dir_entry *dir, {struct super_block *sb = inode->i_sb;char *kaddr =page_address (page); -unsigned from = ((char *)dir - kaddr) & ~(UFS_SB (sb)->s_uspi->s_dirblksize -1 ); -unsigned to = ((char *)dir - kaddr) +fs16_to_cpu (sb, dir->d_reclen); +unsigned int from =offset_in_page (dir) & ~(UFS_SB (sb)->s_uspi->s_dirblksize -1 ); +unsigned int to =offset_in_page (dir) +fs16_to_cpu (sb, dir->d_reclen);loff_t pos;struct ufs_dir_entry *pde =NULL ;struct ufs_dir_entry *de = (struct ufs_dir_entry *) (kaddr + from);@@ -522,7 +520,7 @@ int ufs_delete_entry(struct inode *inode, struct ufs_dir_entry *dir, de =ufs_next_entry (sb, de); }if (pde) - from = (char *)pde - (char *)page_address (page); + from =offset_in_page (pde); pos =page_offset (page) + from;lock_page (page);Signature · 1 line
2.39.0
PATCH v5 2/4 fs/ufs: Change the signature of ufs_get_page()
In reply to Fabio M. De FrancescoChange the signature of ufs_get_page() in order to prepare this function to the conversion to the use of kmap_local_page(). Change also those call sites which are required to conform its invocations to the new signature.
--- fs/ufs/dir.c | 49 +++++++++++++++++++++---------------------------- 1 file changed, 21 insertions(+), 28 deletions(-)
fs/ufs/dir.c +21 -28
@@ -185,7 +185,7 @@ static bool ufs_check_page(struct page *page)return false; } -static struct page *ufs_get_page (struct inode *dir,unsigned long n) +static void *ufs_get_page (struct inode *dir,unsigned long n,struct page **p) {struct address_space *mapping = dir->i_mapping;struct page *page =read_mapping_page (mapping, n,NULL );
@@ -195,8 +195,10 @@ static struct page *ufs_get_page(struct inode *dir, unsigned long n)if (!ufs_check_page (page)) goto fail; } + *p = page; +return page_address (page); } -return page; +return ERR_CAST (page); fail:ufs_put_page (page);
@@ -227,15 +229,12 @@ ufs_next_entry(struct super_block *sb, struct ufs_dir_entry *p)struct ufs_dir_entry *ufs_dotdot (struct inode *dir,struct page **p) { -struct page *page =ufs_get_page (dir,0 ); -struct ufs_dir_entry *de =NULL ; +struct ufs_dir_entry *de =ufs_get_page (dir,0 , p); -if (!IS_ERR (page)) { - de =ufs_next_entry (dir->i_sb, - (struct ufs_dir_entry *)page_address (page)); - *p = page; - } -return de; +if (!IS_ERR (de)) +return ufs_next_entry (dir->i_sb, de); +else +return NULL ; }/*
@@ -273,11 +272,10 @@ struct ufs_dir_entry *ufs_find_entry(struct inode *dir, const struct qstr *qstr, start =0 ; n = start; do { - char *kaddr; - page = ufs_get_page(dir, n); - if (!IS_ERR(page)) { - kaddr = page_address(page); - de = (struct ufs_dir_entry *) kaddr; + char *kaddr = ufs_get_page(dir, n, &page); + + if (!IS_ERR(kaddr)) { + de = (struct ufs_dir_entry *)kaddr; kaddr += ufs_last_byte(dir, n) - reclen; while ((char *) de <= kaddr) { if (ufs_match(sb, namelen, name, de))
@@ -328,12 +326,10 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode)for (n =0 ; n <= npages; n++) {char *dir_end; - page =ufs_get_page (dir, n); - err =PTR_ERR (page); -if (IS_ERR (page)) - goto out; + kaddr =ufs_get_page (dir, n, &page); +if (IS_ERR (kaddr)) +return PTR_ERR (kaddr);lock_page (page); - kaddr =page_address (page); dir_end = kaddr +ufs_last_byte (dir, n); de = (struct ufs_dir_entry *)kaddr; kaddr +=PAGE_SIZE - reclen;
@@ -395,7 +391,6 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode)/* OFFSET_CACHE */ out_put:ufs_put_page (page); -out:return err; out_unlock:unlock_page (page);
@@ -438,17 +433,16 @@ ufs_readdir(struct file *file, struct dir_context *ctx)for ( ; n < npages; n++, offset =0 ) {char *kaddr, *limit;struct ufs_dir_entry *de; +struct page *page; -struct page *page =ufs_get_page (inode, n); - -if (IS_ERR (page)) { + kaddr =ufs_get_page (inode, n, &page); +if (IS_ERR (kaddr)) {ufs_error (sb, __func__,"bad page in #%lu" , inode->i_ino); ctx->pos +=PAGE_SIZE - offset;return -EIO ; } - kaddr =page_address (page);if (unlikely (need_revalidate)) {if (offset) { offset =ufs_validate_entry (sb, kaddr, offset, chunk_mask);
@@ -595,12 +589,11 @@ int ufs_empty_dir(struct inode * inode)for (i =0 ; i < npages; i++) {char *kaddr;struct ufs_dir_entry *de; - page =ufs_get_page (inode, i); -if (IS_ERR (page)) + kaddr =ufs_get_page (inode, i, &page); +if (IS_ERR (kaddr))continue ; - kaddr =page_address (page); de = (struct ufs_dir_entry *)kaddr; kaddr +=ufs_last_byte (inode, i) -UFS_DIR_REC_LEN (1 );
Signature · 1 line
2.39.0
PATCH v5 3/4 fs/ufs: Use ufs_put_page() in ufs_rename()
In reply to Fabio M. De FrancescoUse the ufs_put_page() helper in ufs_rename() instead of open-coding three kunmap() + put_page().
--- fs/ufs/dir.c | 2 +- fs/ufs/namei.c | 9 +++------ fs/ufs/ufs.h | 1 + 3 files changed, 5 insertions(+), 7 deletions(-)
fs/ufs/dir.c +1 -1
@@ -61,7 +61,7 @@ static int ufs_commit_chunk(struct page *page, loff_t pos, unsigned len)return err; } -static inline void ufs_put_page (struct page *page) +inline void ufs_put_page (struct page *page) {kunmap (page);put_page (page);
fs/ufs/namei.c +3 -6
@@ -307,8 +307,7 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,if (old_dir != new_dir)ufs_set_link (old_inode, dir_de, dir_page, new_dir,0 );else { -kunmap (dir_page); -put_page (dir_page); +ufs_put_page (dir_page); }inode_dec_link_count (old_dir); }
@@ -317,12 +316,10 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir, out_dir:if (dir_de) { -kunmap (dir_page); -put_page (dir_page); +ufs_put_page (dir_page); } out_old: -kunmap (old_page); -put_page (old_page); +ufs_put_page (old_page); out:return err; }
fs/ufs/ufs.h +1 -0
@@ -98,6 +98,7 @@ extern struct ufs_cg_private_info * ufs_load_cylinder (struct super_block *, unsextern void ufs_put_cylinder (struct super_block *,unsigned );/* dir.c */ +extern void ufs_put_page (struct page *page);extern const struct inode_operations ufs_dir_inode_operations;extern int ufs_add_link (struct dentry *,struct inode *);extern ino_t ufs_inode_by_name (struct inode *,const struct qstr *);
Signature · 1 line
2.39.0
Re: [PATCH v5 3/4] fs/ufs: Use ufs_put_page() in ufs_rename()
Quoted 5 lines · Fabio M. De Francesco
Use the
ufs_put_page()helper inufs_rename()instead of open-coding threekunmap()+put_page().Cc Al Viro <[email protected]>Suggested-by Ira Weiny <[email protected]>
Quoted 64 lines
Signed-off-by Fabio M. De Francesco <[email protected]>--- fs/ufs/dir.c | 2 +- fs/ufs/namei.c | 9 +++------ fs/ufs/ufs.h | 1 + 3 files changed, 5 insertions(+), 7 deletions(-)fs/ufs/dir.c +1
-1@@ -61,7 +61,7 @@ static int ufs_commit_chunk(struct page *page, loff_t pos, unsigned len)return err; } -static inline void ufs_put_page (struct page *page) +inline void ufs_put_page (struct page *page) {kunmap (page);put_page (page);fs/ufs/namei.c +3
-6@@ -307,8 +307,7 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,if (old_dir != new_dir)ufs_set_link (old_inode, dir_de, dir_page, new_dir,0 );else { -kunmap (dir_page); -put_page (dir_page); +ufs_put_page (dir_page); }inode_dec_link_count (old_dir); }@@ -317,12 +316,10 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir, out_dir:if (dir_de) { -kunmap (dir_page); -put_page (dir_page); +ufs_put_page (dir_page); } out_old: -kunmap (old_page); -put_page (old_page); +ufs_put_page (old_page); out:return err; }fs/ufs/ufs.h +1
-0@@ -98,6 +98,7 @@ extern struct ufs_cg_private_info * ufs_load_cylinder (struct super_block *, unsextern void ufs_put_cylinder (struct super_block *,unsigned );/* dir.c */ +extern void ufs_put_page (struct page *page);extern const struct inode_operations ufs_dir_inode_operations;extern int ufs_add_link (struct dentry *,struct inode *);extern ino_t ufs_inode_by_name (struct inode *,const struct qstr *);Signature · 1 line
2.39.0
PATCH v5 4/4 fs/ufs: Replace kmap() with kmap_local_page()
In reply to Fabio M. De Francescokmap() is being deprecated in favor of kmap_local_page().
There are two main problems with kmap(): (1) It comes with an overhead as the mapping space is restricted and protected by a global lock for synchronization and (2) it also requires global TLB invalidation when the kmap’s pool wraps and it might block when the mapping space is fully utilized until a slot becomes available.
With kmap_local_page() the mappings are per thread, CPU local, can take page faults, and can be called from any context (including interrupts). It is faster than kmap() in kernels with HIGHMEM enabled. Furthermore, the tasks can be preempted and, when they are scheduled to run again, the kernel virtual addresses are restored and still valid.
The use of kmap_local_page() in fs/ufs is "safe" because (1) the kernel virtual addresses are exclusively re-used by the thread which established the mappings (i.e., thread locality is never violated) and (2) the nestings of mappings and un-mappings are always stack based (LIFO).
Therefore, replace kmap() with kmap_local_page() in fs/ufs. kunmap_local() requires the mapping address, so return that address from ufs_get_page() and use it as parameter for the second argument of ufs_put_page().
--- fs/ufs/dir.c | 72 +++++++++++++++++++++++++++++++++----------------- fs/ufs/namei.c | 8 +++--- fs/ufs/ufs.h | 2 +- 3 files changed, 53 insertions(+), 29 deletions(-)
fs/ufs/dir.c +48 -24
@@ -61,9 +61,9 @@ static int ufs_commit_chunk(struct page *page, loff_t pos, unsigned len)return err; } -inline void ufs_put_page (struct page *page) +inline void ufs_put_page (struct page *page,void *page_addr) { -kunmap (page); +kunmap_local (page_addr);put_page (page); }
@@ -76,7 +76,7 @@ ino_t ufs_inode_by_name(struct inode *dir, const struct qstr *qstr) de =ufs_find_entry (dir, qstr, &page);if (de) { res =fs32_to_cpu (dir->i_sb, de->d_ino); -ufs_put_page (page); +ufs_put_page (page, de); }return res; }
@@ -99,18 +99,17 @@ void ufs_set_link(struct inode *dir, struct ufs_dir_entry *de,ufs_set_de_type (dir->i_sb, de, inode->i_mode); err =ufs_commit_chunk (page, pos, len); -ufs_put_page (page); +ufs_put_page (page, de);if (update_times) dir->i_mtime = dir->i_ctime =current_time (dir);mark_inode_dirty (dir); } -static bool ufs_check_page (struct page *page) +static bool ufs_check_page (struct page *page,char *kaddr) {struct inode *dir = page->mapping->host;struct super_block *sb = dir->i_sb; -char *kaddr =page_address (page);unsigned offs, rec_len;unsigned limit =PAGE_SIZE ;const unsigned chunk_mask =UFS_SB (sb)->s_uspi->s_dirblksize -1 ;
@@ -185,23 +184,32 @@ static bool ufs_check_page(struct page *page)return false; } +/* +* Calls to ufs_get_page()/ufs_put_page() must be nested according to the +* rules documented in kmap_local_page()/kunmap_local(). +* +* NOTE: ufs_find_entry() and ufs_dotdot() act as calls to ufs_get_page() +* and must be treated accordingly for nesting purposes. +*/ static void *ufs_get_page (struct inode *dir,unsigned long n,struct page **p) { +char *kaddr; +struct address_space *mapping = dir->i_mapping;struct page *page =read_mapping_page (mapping, n,NULL );if (!IS_ERR (page)) { -kmap (page); + kaddr =kmap_local_page (page);if (unlikely (!PageChecked (page))) { -if (!ufs_check_page (page)) +if (!ufs_check_page (page, kaddr)) goto fail; } *p = page; -return page_address (page); +return kaddr; }return ERR_CAST (page); fail: -ufs_put_page (page); +ufs_put_page (page, kaddr);return ERR_PTR (-EIO ); }
@@ -227,6 +235,13 @@ ufs_next_entry(struct super_block *sb, struct ufs_dir_entry *p) fs16_to_cpu(sb, p->d_reclen)); } +/* +* Calls to ufs_get_page()/ufs_put_page() must be nested according to the +* rules documented in kmap_local_page()/kunmap_local(). +* +* ufs_dotdot() acts as a call to ufs_get_page() and must be treated +* accordingly for nesting purposes. +*/ struct ufs_dir_entry *ufs_dotdot (struct inode *dir,struct page **p) {struct ufs_dir_entry *de =ufs_get_page (dir,0 , p);
@@ -244,6 +259,11 @@ struct ufs_dir_entry *ufs_dotdot(struct inode *dir, struct page **p)* returns the page in which the entry was found, and the entry itself * (as a parameter - res_dir). Page is returned mapped and unlocked. * Entry is guaranteed to be valid. +* +* On Success ufs_put_page() should be called on *res_page. +* +* ufs_find_entry() acts as a call to ufs_get_page() and must be treated +* accordingly for nesting purposes. */ struct ufs_dir_entry *ufs_find_entry (struct inode *dir,const struct qstr *qstr,struct page **res_page)
@@ -282,7 +302,7 @@ struct ufs_dir_entry *ufs_find_entry(struct inode *dir, const struct qstr *qstr, goto found; de =ufs_next_entry (sb, de); } -ufs_put_page (page); +ufs_put_page (page, kaddr); }if (++n >= npages) n =0 ;
@@ -360,7 +380,7 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode) de = (struct ufs_dir_entry *) ((char *) de + rec_len); }unlock_page (page); -ufs_put_page (page); +ufs_put_page (page, kaddr); }BUG ();return -EINVAL ;
@@ -390,7 +410,7 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode)mark_inode_dirty (dir);/* OFFSET_CACHE */ out_put: -ufs_put_page (page); +ufs_put_page (page, kaddr);return err; out_unlock:unlock_page (page);
@@ -468,13 +488,13 @@ ufs_readdir(struct file *file, struct dir_context *ctx) ufs_get_de_namlen(sb, de), fs32_to_cpu(sb, de->d_ino), d_type)) { -ufs_put_page (page); +ufs_put_page (page, kaddr);return 0 ; } } ctx->pos +=fs16_to_cpu (sb, de->d_reclen); } -ufs_put_page (page); +ufs_put_page (page, kaddr); }return 0 ; }
@@ -485,10 +505,15 @@ ufs_readdir(struct file *file, struct dir_context *ctx)* previous entry. */ int ufs_delete_entry (struct inode *inode,struct ufs_dir_entry *dir, -struct page * page) +struct page *page) {struct super_block *sb = inode->i_sb; -char *kaddr =page_address (page); +/* +* The "dir" dentry points somewhere in the same page whose we need the +* address of; therefore, we can simply get the base address "kaddr" by +* masking the previous with PAGE_MASK. +*/ +char *kaddr = (char *)((unsigned long )dir &PAGE_MASK );unsigned int from =offset_in_page (dir) & ~(UFS_SB (sb)->s_uspi->s_dirblksize -1 );unsigned int to =offset_in_page (dir) +fs16_to_cpu (sb, dir->d_reclen);loff_t pos;
@@ -527,7 +552,7 @@ int ufs_delete_entry(struct inode *inode, struct ufs_dir_entry *dir, inode->i_ctime = inode->i_mtime =current_time (inode);mark_inode_dirty (inode); out: -ufs_put_page (page); +ufs_put_page (page, kaddr);UFSD ("EXIT\n" );return err; }
@@ -551,8 +576,7 @@ int ufs_make_empty(struct inode * inode, struct inode *dir) goto fail; } -kmap (page); - base = (char *)page_address (page); + base =kmap_local_page (page);memset (base,0 ,PAGE_SIZE ); de = (struct ufs_dir_entry *) base;
@@ -569,7 +593,7 @@ int ufs_make_empty(struct inode * inode, struct inode *dir) de->d_reclen =cpu_to_fs16 (sb, chunk_size -UFS_DIR_REC_LEN (1 ));ufs_set_de_namlen (sb, de,2 );strcpy (de->d_name,".." ); -kunmap (page); +kunmap_local (base); err =ufs_commit_chunk (page,0 , chunk_size); fail:
@@ -585,9 +609,9 @@ int ufs_empty_dir(struct inode * inode)struct super_block *sb = inode->i_sb;struct page *page =NULL ;unsigned long i, npages =dir_pages (inode); +char *kaddr;for (i =0 ; i < npages; i++) { -char *kaddr;struct ufs_dir_entry *de; kaddr =ufs_get_page (inode, i, &page);
@@ -620,12 +644,12 @@ int ufs_empty_dir(struct inode * inode) } de =ufs_next_entry (sb, de); } -ufs_put_page (page); +ufs_put_page (page, kaddr); }return 1 ; not_empty: -ufs_put_page (page); +ufs_put_page (page, kaddr);return 0 ; }
fs/ufs/namei.c +4 -4
@@ -250,7 +250,7 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,struct inode *old_inode =d_inode (old_dentry);struct inode *new_inode =d_inode (new_dentry);struct page *dir_page =NULL ; -struct ufs_dir_entry * dir_de =NULL ; +struct ufs_dir_entry *dir_de =NULL ;struct page *old_page;struct ufs_dir_entry *old_de;int err = -ENOENT ;
@@ -307,7 +307,7 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,if (old_dir != new_dir)ufs_set_link (old_inode, dir_de, dir_page, new_dir,0 );else { -ufs_put_page (dir_page); +ufs_put_page (dir_page, dir_de); }inode_dec_link_count (old_dir); }
@@ -316,10 +316,10 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir, out_dir:if (dir_de) { -ufs_put_page (dir_page); +ufs_put_page (dir_page, dir_de); } out_old: -ufs_put_page (old_page); +ufs_put_page (old_page, old_de); out:return err; }
fs/ufs/ufs.h +1 -1
@@ -98,7 +98,7 @@ extern struct ufs_cg_private_info * ufs_load_cylinder (struct super_block *, unsextern void ufs_put_cylinder (struct super_block *,unsigned );/* dir.c */ -extern void ufs_put_page (struct page *page); +extern void ufs_put_page (struct page *page,void *vaddr);extern const struct inode_operations ufs_dir_inode_operations;extern int ufs_add_link (struct dentry *,struct inode *);extern ino_t ufs_inode_by_name (struct inode *,const struct qstr *);
Signature · 1 line
2.39.0
Re: [PATCH v5 4/4] fs/ufs: Replace kmap() with kmap_local_page()
Quoted 19 lines · Fabio M. De Francesco
kmap()is being deprecated in favor ofkmap_local_page().There are two main problems with
kmap(): (1) It comes with an overhead as the mapping space is restricted and protected by a global lock for synchronization and (2) it also requires global TLB invalidation when the kmap’s pool wraps and it might block when the mapping space is fully utilized until a slot becomes available.With
kmap_local_page()the mappings are per thread, CPU local, can take page faults, and can be called from any context (including interrupts). It is faster thankmap()in kernels with HIGHMEM enabled. Furthermore, the tasks can be preempted and, when they are scheduled to run again, the kernel virtual addresses are restored and still valid.The use of
kmap_local_page()in fs/ufs is "safe" because (1) the kernel virtual addresses are exclusively re-used by the thread which established the mappings (i.e., thread locality is never violated) and (2) the nestings of mappings and un-mappings are always stack based (LIFO).
Quoted 6 lines
Therefore, replace
kmap()withkmap_local_page()in fs/ufs.kunmap_local()requires the mapping address, so return that address fromufs_get_page()and use it as parameter for the second argument ofufs_put_page().Suggested-by Al Viro <[email protected]>Suggested-by Ira Weiny <[email protected]>
Quoted 282 lines
Signed-off-by Fabio M. De Francesco <[email protected]>--- fs/ufs/dir.c | 72 +++++++++++++++++++++++++++++++++----------------- fs/ufs/namei.c | 8 +++--- fs/ufs/ufs.h | 2 +- 3 files changed, 53 insertions(+), 29 deletions(-)fs/ufs/dir.c +48
-24@@ -61,9 +61,9 @@ static int ufs_commit_chunk(struct page *page, loff_t pos, unsigned len)return err; } -inline void ufs_put_page (struct page *page) +inline void ufs_put_page (struct page *page,void *page_addr) { -kunmap (page); +kunmap_local (page_addr);put_page (page); }@@ -76,7 +76,7 @@ ino_t ufs_inode_by_name(struct inode *dir, const struct qstr *qstr) de =ufs_find_entry (dir, qstr, &page);if (de) { res =fs32_to_cpu (dir->i_sb, de->d_ino); -ufs_put_page (page); +ufs_put_page (page, de); }return res; }@@ -99,18 +99,17 @@ void ufs_set_link(struct inode *dir, struct ufs_dir_entry *de,ufs_set_de_type (dir->i_sb, de, inode->i_mode); err =ufs_commit_chunk (page, pos, len); -ufs_put_page (page); +ufs_put_page (page, de);if (update_times) dir->i_mtime = dir->i_ctime =current_time (dir);mark_inode_dirty (dir); } -static bool ufs_check_page (struct page *page) +static bool ufs_check_page (struct page *page,char *kaddr) {struct inode *dir = page->mapping->host;struct super_block *sb = dir->i_sb; -char *kaddr =page_address (page);unsigned offs, rec_len;unsigned limit =PAGE_SIZE ;const unsigned chunk_mask =UFS_SB (sb)->s_uspi->s_dirblksize -1 ;@@ -185,23 +184,32 @@ static bool ufs_check_page(struct page *page)return false; } +/* +* Calls to ufs_get_page()/ufs_put_page() must be nested according to the +* rules documented in kmap_local_page()/kunmap_local(). +* +* NOTE: ufs_find_entry() and ufs_dotdot() act as calls to ufs_get_page() +* and must be treated accordingly for nesting purposes. +*/ static void *ufs_get_page (struct inode *dir,unsigned long n,struct page **p) { +char *kaddr; +struct address_space *mapping = dir->i_mapping;struct page *page =read_mapping_page (mapping, n,NULL );if (!IS_ERR (page)) { -kmap (page); + kaddr =kmap_local_page (page);if (unlikely (!PageChecked (page))) { -if (!ufs_check_page (page)) +if (!ufs_check_page (page, kaddr)) goto fail; } *p = page; -return page_address (page); +return kaddr; }return ERR_CAST (page); fail: -ufs_put_page (page); +ufs_put_page (page, kaddr);return ERR_PTR (-EIO ); }@@ -227,6 +235,13 @@ ufs_next_entry(struct super_block *sb, struct ufs_dir_entry *p) fs16_to_cpu(sb, p->d_reclen)); } +/* +* Calls to ufs_get_page()/ufs_put_page() must be nested according to the +* rules documented in kmap_local_page()/kunmap_local(). +* +* ufs_dotdot() acts as a call to ufs_get_page() and must be treated +* accordingly for nesting purposes. +*/ struct ufs_dir_entry *ufs_dotdot (struct inode *dir,struct page **p) {struct ufs_dir_entry *de =ufs_get_page (dir,0 , p);@@ -244,6 +259,11 @@ struct ufs_dir_entry *ufs_dotdot(struct inode *dir, struct page **p)* returns the page in which the entry was found, and the entry itself * (as a parameter - res_dir). Page is returned mapped and unlocked. * Entry is guaranteed to be valid. +* +* On Success ufs_put_page() should be called on *res_page. +* +* ufs_find_entry() acts as a call to ufs_get_page() and must be treated +* accordingly for nesting purposes. */ struct ufs_dir_entry *ufs_find_entry (struct inode *dir,const struct qstr *qstr,struct page **res_page)@@ -282,7 +302,7 @@ struct ufs_dir_entry *ufs_find_entry(struct inode *dir, const struct qstr *qstr, goto found; de =ufs_next_entry (sb, de); } -ufs_put_page (page); +ufs_put_page (page, kaddr); }if (++n >= npages) n =0 ;@@ -360,7 +380,7 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode) de = (struct ufs_dir_entry *) ((char *) de + rec_len); }unlock_page (page); -ufs_put_page (page); +ufs_put_page (page, kaddr); }BUG ();return -EINVAL ;@@ -390,7 +410,7 @@ int ufs_add_link(struct dentry *dentry, struct inode *inode)mark_inode_dirty (dir);/* OFFSET_CACHE */ out_put: -ufs_put_page (page); +ufs_put_page (page, kaddr);return err; out_unlock:unlock_page (page);@@ -468,13 +488,13 @@ ufs_readdir(struct file *file, struct dir_context *ctx) ufs_get_de_namlen(sb, de), fs32_to_cpu(sb, de->d_ino), d_type)) { -ufs_put_page (page); +ufs_put_page (page, kaddr);return 0 ; } } ctx->pos +=fs16_to_cpu (sb, de->d_reclen); } -ufs_put_page (page); +ufs_put_page (page, kaddr); }return 0 ; }@@ -485,10 +505,15 @@ ufs_readdir(struct file *file, struct dir_context *ctx)* previous entry. */ int ufs_delete_entry (struct inode *inode,struct ufs_dir_entry *dir, -struct page * page) +struct page *page) {struct super_block *sb = inode->i_sb; -char *kaddr =page_address (page); +/* +* The "dir" dentry points somewhere in the same page whose we need the +* address of; therefore, we can simply get the base address "kaddr" by +* masking the previous with PAGE_MASK. +*/ +char *kaddr = (char *)((unsigned long )dir &PAGE_MASK );unsigned int from =offset_in_page (dir) & ~(UFS_SB (sb)->s_uspi->s_dirblksize -1 );unsigned int to =offset_in_page (dir) +fs16_to_cpu (sb, dir->d_reclen);loff_t pos;@@ -527,7 +552,7 @@ int ufs_delete_entry(struct inode *inode, struct ufs_dir_entry *dir, inode->i_ctime = inode->i_mtime =current_time (inode);mark_inode_dirty (inode); out: -ufs_put_page (page); +ufs_put_page (page, kaddr);UFSD ("EXIT\n" );return err; }@@ -551,8 +576,7 @@ int ufs_make_empty(struct inode * inode, struct inode *dir) goto fail; } -kmap (page); - base = (char *)page_address (page); + base =kmap_local_page (page);memset (base,0 ,PAGE_SIZE ); de = (struct ufs_dir_entry *) base;@@ -569,7 +593,7 @@ int ufs_make_empty(struct inode * inode, struct inode *dir) de->d_reclen =cpu_to_fs16 (sb, chunk_size -UFS_DIR_REC_LEN (1 ));ufs_set_de_namlen (sb, de,2 );strcpy (de->d_name,".." ); -kunmap (page); +kunmap_local (base); err =ufs_commit_chunk (page,0 , chunk_size); fail:@@ -585,9 +609,9 @@ int ufs_empty_dir(struct inode * inode)struct super_block *sb = inode->i_sb;struct page *page =NULL ;unsigned long i, npages =dir_pages (inode); +char *kaddr;for (i =0 ; i < npages; i++) { -char *kaddr;struct ufs_dir_entry *de; kaddr =ufs_get_page (inode, i, &page);@@ -620,12 +644,12 @@ int ufs_empty_dir(struct inode * inode) } de =ufs_next_entry (sb, de); } -ufs_put_page (page); +ufs_put_page (page, kaddr); }return 1 ; not_empty: -ufs_put_page (page); +ufs_put_page (page, kaddr);return 0 ; }fs/ufs/namei.c +4
-4@@ -250,7 +250,7 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,struct inode *old_inode =d_inode (old_dentry);struct inode *new_inode =d_inode (new_dentry);struct page *dir_page =NULL ; -struct ufs_dir_entry * dir_de =NULL ; +struct ufs_dir_entry *dir_de =NULL ;struct page *old_page;struct ufs_dir_entry *old_de;int err = -ENOENT ;@@ -307,7 +307,7 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,if (old_dir != new_dir)ufs_set_link (old_inode, dir_de, dir_page, new_dir,0 );else { -ufs_put_page (dir_page); +ufs_put_page (dir_page, dir_de); }inode_dec_link_count (old_dir); }@@ -316,10 +316,10 @@ static int ufs_rename(struct user_namespace *mnt_userns, struct inode *old_dir, out_dir:if (dir_de) { -ufs_put_page (dir_page); +ufs_put_page (dir_page, dir_de); } out_old: -ufs_put_page (old_page); +ufs_put_page (old_page, old_de); out:return err; }fs/ufs/ufs.h +1
-1@@ -98,7 +98,7 @@ extern struct ufs_cg_private_info * ufs_load_cylinder (struct super_block *, unsextern void ufs_put_cylinder (struct super_block *,unsigned );/* dir.c */ -extern void ufs_put_page (struct page *page); +extern void ufs_put_page (struct page *page,void *vaddr);extern const struct inode_operations ufs_dir_inode_operations;extern int ufs_add_link (struct dentry *,struct inode *);extern ino_t ufs_inode_by_name (struct inode *,const struct qstr *);Signature · 1 line
2.39.0
Quoted 14 lines · On giovedì 29 dicembre 2022 23:50:56 CEST Fabio M. De Francesco wrote:
kmap()is being deprecated in favor ofkmap_local_page().There are two main problems with
kmap(): (1) It comes with an overhead as the mapping space is restricted and protected by a global lock for synchronization and (2) it also requires global TLB invalidation when the kmap’s pool wraps and it might block when the mapping space is fully utilized until a slot becomes available.With
kmap_local_page()the mappings are per thread, CPU local, can take page faults, and can be called from any context (including interrupts). It is faster thankmap()in kernels with HIGHMEM enabled. Furthermore, the tasks can be preempted and, when they are scheduled to run again, the kernel virtual addresses are restored and still valid.
Quoted 5 lines
Since its use in fs/ufs is safe everywhere, it should be preferred.
Therefore, replace
kmap()withkmap_local_page()in fs/ufs.kunmap_local()requires the mapping address, so return that address fromufs_get_page()to be used inufs_put_page().
Hi Al,
I see that this series is here since Dec 29, 2022. Is there anything that prevents its merging? Can you please its four patches in your tree?
Thanks,
Fabio
Quoted 88 lines
This series could have not been ever made because nothing prevented the previous patch from working properly but Al Viro made a long series of very appreciated comments about how many unnecessary and redundant lines of code I could have removed. He could see things I was entirely unable to notice. Furthermore, he also provided solutions and details about how I could decompose a single patch into a small series of three independent units.[1][2][3]
I want to thank him so much for the patience, kindness and the time he decided to spend to provide those analysis and write three messages full of interesting insights.[1][2][3]
Cc Al Viro <[email protected]>Cc Ira Weiny <[email protected]>Signed-off-by Fabio M. De Francesco <[email protected]>Changes from v1:
1/3: No changes.
2/3: Restore the return of "err" that was mistakenly deleted
together with the removal of the "out" label inufs_add_link(). Thanks to Al Viro.[4]
Return the address of the kmap()'ed page instead of a pointer to a pointer to the mapped page; apage_address()had been overlooked inufs_get_page(). Thanks to Al Viro.[5]
3/3: Return the kernel virtual address got from the call to
kmap_local_page()after conversion fromkmap(). Again thanks to Al Viro.[6]Changes from v2:
1/3: No changes.
2/3: Reworkufs_get_page()because the previous version had two
errors: (1) It could return an invalid pages with the out argument "page" and (2) it could return "page_address(page)" also in cases whereread_mapping_page()returned an error and the page is never kmap()'ed. Thanks to Al Viro.[7]
3/3: Reworkufs_get_page()after conversion to
kmap_local_page(), in accordance to the last changes in 2/3.Changes from v3:
1/3: No changes. 2/3: No changes. 3/3: Replace kunmap() with kunmap_local().Changes from v4:
1/4: It was 1/3.
2/4: Move the declaration of a page into an inner loop. Add Ira
Weiny's "Reviewed-by" tag (thanks!).
3/4: Add this patch to useufs_put_page()to replace threekunmap()
andput_page()innamei.c. Thanks to Ira Weiny who noticed that I had overlooked their presence.
4/4: Remove an unnecessary masking that is already carried out by
kunmap_local()viakunmap_local_indexed(). Add a comment to clarify that aufs_dir_entrypassed toufs_delete_entry()points in the same page we need the address of. Suggested by Ira Weiny.[1] https://lore.kernel.org/lkml/Y4E++JERgUMoqfjG@ZenIV/#t
[2] https://lore.kernel.org/lkml/Y4FG0O7VWTTng5yh@ZenIV/#t
[3] https://lore.kernel.org/lkml/Y4ONIFJatIGsVNpf@ZenIV/#t
[4] https://lore.kernel.org/lkml/Y5Zc0qZ3+zsI74OZ@ZenIV/#t
[5] https://lore.kernel.org/lkml/Y5ZZy23FFAnQDR3C@ZenIV/#t
[6] https://lore.kernel.org/lkml/Y5ZcMPzPG9h6C9eh@ZenIV/#t
[7] https://lore.kernel.org/lkml/Y5glgpD7fFifC4Fi@ZenIV/#tThe cover letter of the v1 series is at
https://lore.kernel.org/lkml/[email protected]/
The cover letter of the v2 series is at
https://lore.kernel.org/lkml/[email protected]/
The cover letter of the v3 series is at
https://lore.kernel.org/lkml/[email protected]/
The cover letter of the v4 series is at
https://lore.kernel.org/lkml/[email protected]/Fabio M. De Francesco (4):
fs/ufs: Use theoffset_in_page()helper
fs/ufs: Change the signature ofufs_get_page()
fs/ufs: Useufs_put_page()inufs_rename()
fs/ufs: Replacekmap()withkmap_local_page()fs/ufs/dir.c | 131 +++++++++++++++++++++++++++---------------------- fs/ufs/namei.c | 11 ++--- fs/ufs/ufs.h | 1 + 3 files changed, 78 insertions(+), 65 deletions(-)--
2.39.0
Quoted 21 lines · Fabio M. De Francesco
Quoted 14 lines · On giovedì 29 dicembre 2022 23:50:56 CEST Fabio M. De Francesco wrote:
kmap()is being deprecated in favor ofkmap_local_page().There are two main problems with
kmap(): (1) It comes with an overhead as the mapping space is restricted and protected by a global lock for synchronization and (2) it also requires global TLB invalidation when the kmap’s pool wraps and it might block when the mapping space is fully utilized until a slot becomes available.With
kmap_local_page()the mappings are per thread, CPU local, can take page faults, and can be called from any context (including interrupts). It is faster thankmap()in kernels with HIGHMEM enabled. Furthermore, the tasks can be preempted and, when they are scheduled to run again, the kernel virtual addresses are restored and still valid.Quoted 5 lines
Since its use in fs/ufs is safe everywhere, it should be preferred.
Therefore, replace
kmap()withkmap_local_page()in fs/ufs.kunmap_local()requires the mapping address, so return that address fromufs_get_page()to be used inufs_put_page().
Quoted 5 lines
Hi Al,
I see that this series is here since Dec 29, 2022. Is there anything that prevents its merging? Can you please its four patches in your tree?
I'm pretty sure UFS directories should simply be allocated from lowmem. There's really no reason to put them in highmem these days.
Quoted 5 lines · On lunedì 27 marzo 2023 15:23:41 CEST Matthew Wilcox wrote:
Quoted 4 lines · Fabio M. De Francesco
Quoted 3 lines · On giovedì 29 dicembre 2022 23:50:56 CEST Fabio M. De Francesco wrote:
kmap()is being deprecated in favor ofkmap_local_page().There are two main problems with
kmap(): (1) It comes with an overhead
as
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the mapping space is restricted and protected by a global lock for synchronization and (2) it also requires global TLB invalidation when
the
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
kmap’s pool wraps and it might block when the mapping space is fully utilized until a slot becomes available.
With
kmap_local_page()the mappings are per thread, CPU local, can take page faults, and can be called from any context (including interrupts). It is faster thankmap()in kernels with HIGHMEM enabled. Furthermore, the tasks can be preempted and, when they are scheduled to run again,
the
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
kernel virtual addresses are restored and still valid.
Since its use in fs/ufs is safe everywhere, it should be preferred.
Therefore, replace
kmap()withkmap_local_page()in fs/ufs.
kunmap_local()
Quoted 11 lines
Quoted 8 lines
Quoted 2 lines
requires the mapping address, so return that address from
ufs_get_page()to be used inufs_put_page().Hi Al,
I see that this series is here since Dec 29, 2022. Is there anything that prevents its merging? Can you please its four patches in your tree?
I'm pretty sure UFS directories should simply be allocated from lowmem. There's really no reason to put them in highmem these days.
Matthew,
It's been a few days since you wrote but I still haven't done anything or replied. For now, I just want you to know that it's not my intention to ignore your comment.
The only reasons I hesitate to follow your directions are that I built this series based on specific suggestions from Al.
While I understand and, for sure, heed what you said, I also think I should wait for Al to chime in with what he thinks about all this before taking a different route for removing kmap() from fs/ufs.
Thanks for looking at this series.
Fabio