Skip to contentSkip to conversation

glibc: header file and manual updates for file-private locks

RFC PATCH In review Jeff Layton 4 participants 9 messages Feb 11, 2014 – Feb 16, 2014 VFS

lore
Share & tools
lore
#0 lore raw

cover glibc: header file and manual updates for file-private locks

I've been working on a set of patches to add a new type file record lock to the Linux kernel that have less troublesome semantics on close and that are usable by threaded applications.

I have a blog post here with more details:

    http://jtlayton.wordpress.com/2014/01/07/file-private-posix-locks-aka-un-posix-locks/

...and the latest kernel patchset is here:

    https://lkml.org/lkml/2014/1/14/554

This patchset represents a set of corresponding updates for glibc. It just adds the new cmd values to fcntl.h, and a manual update that discusses how to use the new locks.

I'm currently shooting for a merge in v3.15 kernel so I'm posting this as an RFC for now.

Comments and suggestions appreciated...

Thanks,

Jeff Layton (2):
fcntl.h: add new definitions for file-private lock cmd values
manual: update manual to document file-private locks

 ChangeLog                                  |   5 +
 manual/llio.texi                           | 175 ++++++++++++++++++++++++++++-
 sysdeps/unix/sysv/linux/bits/fcntl-linux.h |  18 +++
 3 files changed, 196 insertions(+), 2 deletions(-)
Signature · 1 line
1.8.5.3
#1 lore raw

RFC PATCH 1/2 fcntl.h: add new definitions for file-private lock cmd values

---
 ChangeLog                                  |  4 ++++
 sysdeps/unix/sysv/linux/bits/fcntl-linux.h | 18 ++++++++++++++++++
 2 files changed, 22 insertions(+)
ChangeLog +4 -0
index 5e820f7e9900..db5b90385e2c 100644
@@ -1,3 +1,7 @@
+2014-02-11  Jeff Layton  <[email protected]>
+
+	* add new fcntl cmd macros for file-private locks
+
 2014-02-11  Stefan Liebler  <[email protected]>
 
 	[BZ #16447]
sysdeps/unix/sysv/linux/bits/fcntl-linux.h +18 -0
index cd4722b836ea..71fd80dba2d8 100644
@@ -117,6 +117,24 @@
 # define F_SETLKW64	14	/* Set record locking info (blocking).	*/
 #endif
 
+/*
+ * fd "private" POSIX locks.
+ *
+ * Usually POSIX locks held by a process are released on *any* close and are
+ * not inherited across a fork().
+ *
+ * These cmd values will set locks that conflict with normal POSIX locks, but
+ * are "owned" by the opened file, not the process. This means that they are
+ * inherited across fork() like BSD (flock) locks, and they are only released
+ * automatically when the last reference to the the open file against which
+ * they were acquired is put.
+ */
+#ifndef F_GETLKP
+# define F_GETLKP	36
+# define F_SETLKP	37
+# define F_SETLKPW	38
+#endif
+
 #ifdef __USE_LARGEFILE64
 # define O_LARGEFILE __O_LARGEFILE
 #endif
Signature · 1 line
1.8.5.3
#2 lore raw

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

ChangeLog line 3

Quoted 3 lines

+2014-02-11 Jeff Layton <[email protected]>

ChangeLog +2 -0
+
+	* add new fcntl cmd macros for file-private locks

Proper format is:

* sysdeps/unix/sysv/linux/bits/fcntl-linux.h (F_GETLKP, F_SETLKP, F_SETLKPW): New macros.

Quoted 5 lines

+/*
+ * fd "private" POSIX locks.
+ *
+ * Usually POSIX locks held by a process are released on *any* close and are
+ * not inherited across a fork().

Use GNU style for comments, as you see elsewhere in the file. Use two spaces between sentences. Don't use "function()" when referring to a function, just the name.

sysdeps/unix/sysv/linux/bits/fcntl-linux.h line 136

Quoted 5 lines
sysdeps/unix/sysv/linux/bits/fcntl-linux.h +5 -0
+#ifndef F_GETLKP
+# define F_GETLKP	36
+# define F_SETLKP	37
+# define F_SETLKPW	38
+#endif

These need to be protected by #ifdef __USE_GNU.

#3 lore raw

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

Quoted 19 lines · On Tue, 11 Feb 2014 13:49:15 -0800 (PST) Roland McGrath <[email protected]>…

ChangeLog line 3

Quoted 3 lines

+2014-02-11 Jeff Layton <[email protected]>

ChangeLog +2 -0
+
+	* add new fcntl cmd macros for file-private locks

Proper format is:

* sysdeps/unix/sysv/linux/bits/fcntl-linux.h (F_GETLKP, F_SETLKP, F_SETLKPW): New macros.

Quoted 5 lines

+/*
+ * fd "private" POSIX locks.
+ *
+ * Usually POSIX locks held by a process are released on *any* close and are
+ * not inherited across a fork().

Use GNU style for comments, as you see elsewhere in the file. Use two spaces between sentences. Don't use "function()" when referring to a function, just the name.

Quoted 7 lines

sysdeps/unix/sysv/linux/bits/fcntl-linux.h line 136

Quoted 5 lines
sysdeps/unix/sysv/linux/bits/fcntl-linux.h +5 -0
+#ifndef F_GETLKP
+# define F_GETLKP	36
+# define F_SETLKP	37
+# define F_SETLKPW	38
+#endif

These need to be protected by #ifdef __USE_GNU.

Thanks for the comments, Roland. I'll fix the patch accordingly.

My plan is to eventually submit this to become part of the POSIX standard though they won't take it until there's at least one shipping implementation. I assume that we just need to stick that inside of __USE_GNU for now, and if POSIX eventually picks it up then we'll just remove that protection?

Also, I'm a little curious in looking at this and have a (probably stupid) question:

Why does glibc have its own definitions for the fcntl F_* cmd values instead of using the ones in the uapi kernel headers? Would it not be simpler to just have the fcntl.h include those instead of duplicating them?

Thanks,

Signature · 1 line
Jeff Layton <[email protected]>
#4 lore raw

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

Quoted 4 lines · Jeff Layton

Why does glibc have its own definitions for the fcntl F_* cmd values instead of using the ones in the uapi kernel headers? Would it not be simpler to just have the fcntl.h include those instead of duplicating them?

The uapi headers are useless for user space. They refer to CONFIG_* macros, and don't have suitable namespace control.

Andreas.

Signature · 3 lines
Andreas Schwab, SUSE Labs, [email protected]
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
#5 lore raw

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

In reply to Jeff Layton
Quoted 5 lines

My plan is to eventually submit this to become part of the POSIX standard though they won't take it until there's at least one shipping implementation. I assume that we just need to stick that inside of __USE_GNU for now, and if POSIX eventually picks it up then we'll just remove that protection?

Change it as appropriate, yes.

Quoted 4 lines

Why does glibc have its own definitions for the fcntl F_* cmd values instead of using the ones in the uapi kernel headers? Would it not be simpler to just have the fcntl.h include those instead of duplicating them?

libc headers respect ISO C and POSIX name space constraints. Kernel headers (even uapi ones) fail to.

#6 lore raw

RFC PATCH 2/2 manual: update manual to document file-private locks

In reply to Jeff Layton
---
 ChangeLog        |   1 +
 manual/llio.texi | 175 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 174 insertions(+), 2 deletions(-)
ChangeLog +1 -0
index db5b90385e2c..d2f9a52dd283 100644
@@ -1,6 +1,7 @@
 2014-02-11  Jeff Layton  <[email protected]>
 
 	* add new fcntl cmd macros for file-private locks
+	* add manual section about file-private locks
 
 2014-02-11  Stefan Liebler  <[email protected]>
 
manual/llio.texi +173 -2

211 lines not on this page · raw message

Signature · 1 line
1.8.5.3
#7 lore raw
In reply to Jeff Layton
Quoted 12 lines · On Tuesday, February 11, 2014 14:03:54 Jeff Layton wrote:

I've been working on a set of patches to add a new type file record lock to the Linux kernel that have less troublesome semantics on close and that are usable by threaded applications.

I have a blog post here with more details:

http://jtlayton.wordpress.com/2014/01/07/file-private-posix-locks-aka-un-po
six-locks/

...and the latest kernel patchset is here:

Quoted 8 lines
    https://lkml.org/lkml/2014/1/14/554

This patchset represents a set of corresponding updates for glibc. It just adds the new cmd values to fcntl.h, and a manual update that discusses how to use the new locks.

I'm currently shooting for a merge in v3.15 kernel so I'm posting this as an RFC for now.

disclaimer: haven't read the kernel patches. just the patch summary and your blog post.

looks like the new API manipulates the existing POSIX lock only -- flock based locks are still parallel. too crazy to have the new API grab both since you're bringing proper semantics that cover both ?

a good summary of Linux locking from the perspective of userspace was posted here sometime ago:
http://0pointer.de/blog/projects/locking.html
random notes/limitations/bugs are also in the various man pages like fcntl(2)

would be nice if we could cover as many known limitations as possible with the new api so as to not need yet another in the future ;)
-mike

#8 lore raw
Quoted 22 lines · On Sun, 16 Feb 2014 03:27:05 -0500 Mike Frysinger <[email protected]> wrote:
Quoted 12 lines · On Tuesday, February 11, 2014 14:03:54 Jeff Layton wrote:

I've been working on a set of patches to add a new type file record lock to the Linux kernel that have less troublesome semantics on close and that are usable by threaded applications.

I have a blog post here with more details:

http://jtlayton.wordpress.com/2014/01/07/file-private-posix-locks-aka-un-po
six-locks/

...and the latest kernel patchset is here:

Quoted 8 lines
    https://lkml.org/lkml/2014/1/14/554

This patchset represents a set of corresponding updates for glibc. It just adds the new cmd values to fcntl.h, and a manual update that discusses how to use the new locks.

I'm currently shooting for a merge in v3.15 kernel so I'm posting this as an RFC for now.

Quoted 7 lines

disclaimer: haven't read the kernel patches. just the patch summary and your blog post.

looks like the new API manipulates the existing POSIX lock only -- flock based locks are still parallel. too crazy to have the new API grab both since you're bringing proper semantics that cover both ?

Correct, at least in the Linux implementation. The basic idea is that these *are* POSIX locks, they just have different behavior wrt inheritance and on close.

We should note that on BSD, flock() and fcntl() based locks do conflict. If they end up implementing file-private locks on BSD, then they may also conflict with flock() locks just like classic POSIX locks do there.

Quoted 7 lines

a good summary of Linux locking from the perspective of userspace was posted here sometime ago:
http://0pointer.de/blog/projects/locking.html
random notes/limitations/bugs are also in the various man pages like fcntl(2) would be nice if we could cover as many known limitations as possible with the new api so as to not need yet another in the future ;)
-mike

Yep, I've seen Lennart's summary. I think that the only thing this doesn't approach doesn't address is his comment about using locks on world-readable files. I don't see that there's much we can do about that.

Signature · 1 line
Jeff Layton <[email protected]>