Skip to contentSkip to conversation

minor fd cleanup

PATCH In review Mateusz Guzik 3 participants 9 messages Oct 3, 2017 – Oct 4, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover minor fd cleanup

The first patch is a super minor nit and I'm fine with it being ignored.

Second patch is imho of actual value.

fd_install has an avoidable requirement to be called in a sleepable context. It matters e.g. in rhel7 to where the patch was recently backported and the new requirement could not be enforced.

I don't see any benefits from sleeping over locking and not sleeping makes the function slightly nicer.

Mateusz Guzik (2):
vfs: stop clearing close on exec when closing a fd
vfs: grab the lock instead of blocking in __fd_install during resizing

 Documentation/filesystems/porting |  4 ----
 fs/file.c                         | 12 +++++++-----
 2 files changed, 7 insertions(+), 9 deletions(-)
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/2 vfs: stop clearing close on exec when closing a fd

Codepaths allocating a fd always make sure the bit is set/unset depending on flags, thus clearing on close is redundant.

---
 fs/file.c | 1 -
 1 file changed, 1 deletion(-)
fs/file.c +0 -1
index 1fc7fbb..9d047bd 100644
@@ -631,7 +631,6 @@ int __close_fd(struct files_struct *files, unsigned fd)
 	if (!file)
 		goto out_unlock;
 	rcu_assign_pointer(fdt->fd[fd], NULL);
-	__clear_close_on_exec(fd, fdt);
 	__put_unused_fd(files, fd);
 	spin_unlock(&files->file_lock);
 	return filp_close(file, files);
Signature · 1 line
1.8.3.1
#3 lore raw

PATCH 2/2 vfs: grab the lock instead of blocking in __fd_install during resizing

In reply to Mateusz Guzik

Explicit locking in the fallback case provides a safe state of the table. Getting rid of blocking semantics makes __fd_install usable again in non-sleepable contexts, which easies backporting efforts.

There is a side effect of slightly nicer assembly for the common case as might_sleep can now be removed.

---
 Documentation/filesystems/porting |  4 ----
 fs/file.c                         | 11 +++++++----
 2 files changed, 7 insertions(+), 8 deletions(-)
Documentation/filesystems/porting +0 -4
index 93e0a24..17bb4dc 100644
@@ -502,10 +502,6 @@ in your dentry operations instead.
 	store it as cookie.
 --
 [mandatory]
-	__fd_install() & fd_install() can now sleep. Callers should not
-	hold a spinlock	or other resources that do not allow a schedule.
---
-[mandatory]
 	any symlink that might use page_follow_link_light/page_put_link() must
 	have inode_nohighmem(inode) called before anything might start playing with
 	its pagecache.  No highmem pages should end up in the pagecache of such
fs/file.c +7 -4
index 9d047bd..4115503 100644
@@ -592,13 +592,16 @@ void __fd_install(struct files_struct *files, unsigned int fd,
 {
 	struct fdtable *fdt;
 
-	might_sleep();
 	rcu_read_lock_sched();
 
-	while (unlikely(files->resize_in_progress)) {
+	if (unlikely(files->resize_in_progress)) {
 		rcu_read_unlock_sched();
-		wait_event(files->resize_wait, !files->resize_in_progress);
-		rcu_read_lock_sched();
+		spin_lock(&files->file_lock);
+		fdt = files_fdtable(files);
+		BUG_ON(fdt->fd[fd] != NULL);
+		rcu_assign_pointer(fdt->fd[fd], file);
+		spin_unlock(&files->file_lock);
+		return;
 	}
 	/* coupled with smp_wmb() in expand_fdtable() */
 	smp_rmb();
Signature · 1 line
1.8.3.1
Reviewed-by on v1 2/2
#4 lore raw

Re: [PATCH 2/2] vfs: grab the lock instead of blocking in __fd_install during resizing

Quoted 4 lines · Mateusz Guzik

Explicit locking in the fallback case provides a safe state of the table. Getting rid of blocking semantics makes __fd_install usable again in non-sleepable contexts, which easies backporting efforts.

Quoted 8 lines

There is a side effect of slightly nicer assembly for the common case as might_sleep can now be removed.

---
 Documentation/filesystems/porting |  4 ----
 fs/file.c                         | 11 +++++++----
 2 files changed, 7 insertions(+), 8 deletions(-)

Nice change !

#5 lore raw

Re: [PATCH 2/2] vfs: grab the lock instead of blocking in __fd_install during resizing

Quoted 14 lines · Eric Dumazet
Quoted 4 lines · Mateusz Guzik

Explicit locking in the fallback case provides a safe state of the table. Getting rid of blocking semantics makes __fd_install usable again in non-sleepable contexts, which easies backporting efforts.

Quoted 8 lines

There is a side effect of slightly nicer assembly for the common case as might_sleep can now be removed.

---
 Documentation/filesystems/porting |  4 ----
 fs/file.c                         | 11 +++++++----
 2 files changed, 7 insertions(+), 8 deletions(-)
Quoted 3 lines

Nice change !

Hey Eric,

Any chance you could review the patches from Sandhya that make this entire codepath obsolete?

https://lkml.org/lkml/2017/4/29/20

#6 lore raw

Re: [PATCH 2/2] vfs: grab the lock instead of blocking in __fd_install during resizing

Quoted 17 lines · Matthew Wilcox
Quoted 16 lines · Eric Dumazet
Quoted 4 lines · Mateusz Guzik

Explicit locking in the fallback case provides a safe state of the table. Getting rid of blocking semantics makes __fd_install usable again in non-sleepable contexts, which easies backporting efforts.

Quoted 8 lines

There is a side effect of slightly nicer assembly for the common case as might_sleep can now be removed.

---
 Documentation/filesystems/porting |  4 ----
 fs/file.c                         | 11 +++++++----
 2 files changed, 7 insertions(+), 8 deletions(-)

Nice change !

Quoted 9 lines
Quoted 1 line

Hey Eric,

Any chance you could review the patches from Sandhya that make this entire codepath obsolete?

https://lkml.org/lkml/2017/4/29/20

Hmm...

18 files changed, 578 insertions(+), 585 deletions(-)

Frankly I need to be convinced with solid performance numbers before I am taking a look at this series.

I do not believe an IDR will be faster than current implementation, so I am not quite convinced at this moment.

Thanks.

#7 lore raw

Re: [PATCH 2/2] vfs: grab the lock instead of blocking in __fd_install during resizing

Quoted 7 lines · Eric Dumazet
Quoted 5 lines · Matthew Wilcox

Any chance you could review the patches from Sandhya that make this entire codepath obsolete?

https://lkml.org/lkml/2017/4/29/20

Quoted 6 lines

Hmm...

18 files changed, 578 insertions(+), 585 deletions(-)

Frankly I need to be convinced with solid performance numbers before I am taking a look at this series.

I was hoping you'd help us get some solid performance numbers ... you seem to have workloads available to you that help find weaknesses in implementations.

The number of lines inserted is a bit of a red herring. Over 100 are in the test suite (you surely aren't going to review those) and another ~300 are adding enhancements to the IDR & radix tree that should be useful for other users (eg I think I have a way to speed up writing out dirty pages by using get_tag_batch()).

Quoted 2 lines

I do not believe an IDR will be faster than current implementation, so I am not quite convinced at this moment.

I don't think it should be significantly different in performance. Let's look at the layout of data for a typical bash shell (fds 0-2 and 255 open).

Current implementation:

files_struct -> fdt -> fd -> struct file

IDR:

files_struct -> radix node (shift 6) -> radix node (shift 0) -> struct file

In either case, it's the same number of dependent loads. It'll start to look worse for the radix tree above 4096 open fds in a given process.

#8 lore raw

Re: [PATCH 2/2] vfs: grab the lock instead of blocking in __fd_install during resizing

Quoted 33 lines · Matthew Wilcox
Quoted 7 lines · Eric Dumazet
Quoted 5 lines · Matthew Wilcox

Any chance you could review the patches from Sandhya that make this entire codepath obsolete?

https://lkml.org/lkml/2017/4/29/20

Quoted 6 lines

Hmm...

18 files changed, 578 insertions(+), 585 deletions(-)

Frankly I need to be convinced with solid performance numbers before I am taking a look at this series.

I was hoping you'd help us get some solid performance numbers ... you seem to have workloads available to you that help find weaknesses in implementations.

The number of lines inserted is a bit of a red herring. Over 100 are in the test suite (you surely aren't going to review those) and another ~300 are adding enhancements to the IDR & radix tree that should be useful for other users (eg I think I have a way to speed up writing out dirty pages by using get_tag_batch()).

Quoted 2 lines

I do not believe an IDR will be faster than current implementation, so I am not quite convinced at this moment.

I don't think it should be significantly different in performance. Let's look at the layout of data for a typical bash shell (fds 0-2 and 255 open).

Current implementation:

Quoted 8 lines

files_struct -> fdt -> fd -> struct file

IDR:

files_struct -> radix node (shift 6) -> radix node (shift 0) -> struct file

In either case, it's the same number of dependent loads. It'll start to look worse for the radix tree above 4096 open fds in a given process.

I am interested in performance for process with 10,000,000 fds, and ~100 threads constantly adding/deleting/using fds.

Thanks