Skip to contentSkip to conversation

FMODE_NONOTIFY, FMODE_RANDOM and FMODE_NEG_OFFSET bits

PATCH Superseded Wu Fengguang 3 participants 22 messages Jan 15, 2010 – Jan 19, 2010

Subsystems (10)
lore
Share & tools
lore
#0 lore raw

cover FMODE_NONOTIFY, FMODE_RANDOM and FMODE_NEG_OFFSET bits

Andrew,

This collects my recent patches related to FMODE_* and O_* discussed in LKML. They are mainly bug fixes.

O_* and FMODE_NONOTIFY collision fix/check
[PATCH 1/6] fanotify: fix FMODE_NONOTIFY bit number
[PATCH 2/6] bitops: compile time optimization for hweight_long(CONSTANT) [PATCH 3/6] vfs: O_* bit numbers uniqueness check

POSIX_FADV_RANDOM misbehavior fix
[PATCH 4/6] vfs: take f_lock on modifying f_mode after open time [PATCH 5/6] readahead: introduce FMODE_RANDOM for POSIX_FADV_RANDOM

allow negative f_pos for /dev/kmem
[PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Thanks,
Fengguang

#2 lore raw

PATCH 2/6 bitops: compile time optimization for hweight_long(CONSTANT)

This allows use of hweight_long() in BUILD_BUG_ON(). Suggested by Jamie.

CC Jamie Lokier <[email protected]>
CC Roland Dreier <[email protected]>
---
 include/linux/bitops.h |   12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
include/linux/bitops.h +8 -4
@@ -40,10 +40,14 @@ static __inline__ int get_count_order(un
 	return order;
 }
 
-static inline unsigned long hweight_long(unsigned long w)
-{
-	return sizeof(w) == 4 ? hweight32(w) : hweight64(w);
-}
+#define hweight_long(x)				\
+(						\
+	__builtin_constant_p(x) ?		\
+		__builtin_popcountl(x) :	\
+		(sizeof(x) <= 4 ?		\
+			 hweight32(x) :		\
+			 hweight64(x))		\
+)
 
 /**
  * rol32 - rotate a 32-bit value left
#5 lore raw

PATCH 5/6 readahead: introduce FMODE_RANDOM for POSIX_FADV_RANDOM

In reply to Wu Fengguang

This fixes inefficient page-by-page reads on POSIX_FADV_RANDOM.

POSIX_FADV_RANDOM used to set ra_pages=0, which leads to poor
performance: a 16K read will be carried out in 4 _sync_ 1-page reads.

In other places, ra_pages==0 means
- it's ramfs/tmpfs/hugetlbfs/sysfs/configfs
- some IO error happened
where multi-page read IO won't help or should be avoided.

POSIX_FADV_RANDOM actually want a different semantics: to disable the *heuristic* readahead algorithm, and to use a dumb one which faithfully submit read IO for whatever application requests.

So introduce a flag FMODE_RANDOM for POSIX_FADV_RANDOM.

Note that the random hint is not likely to help random reads performance noticeably. And it may be too permissive on huge request size (its IO size is not limited by read_ahead_kb).

In Quentin's report (http://lkml.org/lkml/2009/12/24/145), the overall (NFS read) performance of the application increased by 313%!

v6: use FMODE_RANDOM (proposed by Christoph Hellwig)
v5: use bit 0200000000; explicitly nuke the O_RANDOM bit in __dentry_open()
(Stephen Rothwell)
v4: resolve bit conflicts with sparc and parisc;
use bit 040000000(=FMODE_NONOTIFY), which will be masked out by __dentry_open(), so that open(O_RANDOM) is disabled (Stephen Rothwell and Christoph Hellwig)
v3: use O_RANDOM to indicate both read/write access pattern as in
posix_fadvise(), although it only takes effect for read() now (proposed by Quentin)
v2: use O_RANDOM_READ to avoid race conditions (pointed out by Andi)

CC Nick Piggin <[email protected]>
CC Andi Kleen <[email protected]>
CC Steven Whitehouse <[email protected]>
CC David Howells <[email protected]>
CC Al Viro <[email protected]>
CC Jonathan Corbet <[email protected]>
CC Christoph Hellwig <[email protected]>
---
 include/linux/fs.h |    3 +++
 mm/fadvise.c       |   10 +++++++++-
 mm/readahead.c     |    6 ++++++
 3 files changed, 18 insertions(+), 1 deletion(-)
mm/fadvise.c +9 -1
@@ -77,12 +77,20 @@ SYSCALL_DEFINE(fadvise64_64)(int fd, lof
 	switch (advice) {
 	case POSIX_FADV_NORMAL:
 		file->f_ra.ra_pages = bdi->ra_pages;
+		spin_lock(&file->f_lock);
+		file->f_flags &= ~FMODE_RANDOM;
+		spin_unlock(&file->f_lock);
 		break;
 	case POSIX_FADV_RANDOM:
-		file->f_ra.ra_pages = 0;
+		spin_lock(&file->f_lock);
+		file->f_flags |= FMODE_RANDOM;
+		spin_unlock(&file->f_lock);
 		break;
 	case POSIX_FADV_SEQUENTIAL:
 		file->f_ra.ra_pages = bdi->ra_pages * 2;
+		spin_lock(&file->f_lock);
+		file->f_flags &= ~FMODE_RANDOM;
+		spin_unlock(&file->f_lock);
 		break;
 	case POSIX_FADV_WILLNEED:
 		if (!mapping->a_ops->readpage) {
mm/readahead.c +6 -0
@@ -501,6 +501,12 @@ void page_cache_sync_readahead(struct ad
 	if (!ra->ra_pages)
 		return;
 
+	/* be dumb */
+	if (filp->f_mode & FMODE_RANDOM) {
+		force_page_cache_readahead(mapping, filp, offset, req_size);
+		return;
+	}
+
 	/* do read-ahead */
 	ondemand_readahead(mapping, ra, filp, false, offset, req_size);
 }
include/linux/fs.h +3 -0
@@ -90,6 +90,9 @@ struct inodes_stat_t {
 /* File was opened by fanotify and shouldn't generate fanotify events */
 #define FMODE_NONOTIFY		((__force fmode_t)0x1000000)
 
+/* Expect random access pattern */
+#define FMODE_RANDOM		((__force fmode_t)0x1000)
+
 /*
  * The below are the various read and write types that we support. Some of
  * them include behavioral modifiers that send information down to the
#6 lore raw

PATCH 6/6 vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

In reply to Wu Fengguang

From: KAMEZAWA Hiroyuki <[email protected]>

Now, rw_verify_area() checsk f_pos is negative or not. And if negative, returns -EINVAL.

But, some special files as /dev/(k)mem and /proc/<pid>/mem etc.. has negative offsets. And we can't do any access via read/write to the file(device).

So introduce FMODE_NEG_OFFSET to allow negative file offsets.

Changelog: v5->v6
- use FMODE_NEG_OFFSET (suggested by Al)
- rebased onto 2.6.33-rc1

Changelog: v4->v5
- clean up patches dor /dev/mem.
- rebased onto 2.6.32-rc1

Changelog: v3->v4
- make changes in mem.c aligned.
- change __negative_fpos_check() to return int.
- fixed bug in "pos" check.
- added comments.

Changelog: v2->v3
- fixed bug in rw_verify_area (it cannot be compiled)

---
 drivers/char/mem.c |    4 ++++
 fs/proc/base.c     |    2 ++
 fs/read_write.c    |   21 +++++++++++++++++++--
 include/linux/fs.h |    3 +++
 4 files changed, 28 insertions(+), 2 deletions(-)
fs/read_write.c +19 -2
@@ -205,6 +205,20 @@ bad:
 }
 #endif
 
+static int
+__negative_fpos_check(struct file *file, loff_t pos, size_t count)
+{
+	/*
+	 * pos or pos+count is negative here, check overflow.
+	 * too big "count" will be caught in rw_verify_area().
+	 */
+	if ((pos < 0) && (pos + count < pos))
+		return -EOVERFLOW;
+	if (file->f_mode & FMODE_NEG_OFFSET)
+		return 0;
+	return -EINVAL;
+}
+
 /*
  * rw_verify_area doesn't like huge counts. We limit
  * them to something that fits in "int" so that others
@@ -222,8 +236,11 @@ int rw_verify_area(int read_write, struc
 	if (unlikely((ssize_t) count < 0))
 		return retval;
 	pos = *ppos;
-	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0))
-		return retval;
+	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0)) {
+		retval = __negative_fpos_check(file, pos, count);
+		if (retval)
+			return retval;
+	}
 
 	if (unlikely(inode->i_flock && mandatory_lock(inode))) {
 		retval = locks_mandatory_area(
include/linux/fs.h +3 -0
@@ -93,6 +93,9 @@ struct inodes_stat_t {
 /* Expect random access pattern */
 #define FMODE_RANDOM		((__force fmode_t)0x1000)
 
+/* File is huge (eg. /dev/kmem): treat loff_t as unsigned */
+#define FMODE_NEG_OFFSET	((__force fmode_t)0x2000)
+
 /*
  * The below are the various read and write types that we support. Some of
  * them include behavioral modifiers that send information down to the
drivers/char/mem.c +4 -0
@@ -861,6 +861,10 @@ static int memory_open(struct inode *ino
 	if (dev->dev_info)
 		filp->f_mapping->backing_dev_info = dev->dev_info;
 
+	/* Is /dev/mem or /dev/kmem ? */
+	if (dev->dev_info == &directly_mappable_cdev_bdi)
+		filp->f_mode |= FMODE_NEG_OFFSET;
+
 	if (dev->fops->open)
 		return dev->fops->open(inode, filp);
 
fs/proc/base.c +2 -0
@@ -861,6 +861,8 @@ static const struct file_operations proc
 static int mem_open(struct inode* inode, struct file* file)
 {
 	file->private_data = (void*)((long)current->self_exec_id);
+	/* OK to pass negative loff_t, we can catch out-of-range */
+	file->f_mode |= FMODE_NEG_OFFSET;
 	return 0;
 }
 
#1 lore raw

PATCH 1/6 fanotify: fix FMODE_NONOTIFY bit number

In reply to Wu Fengguang

FMODE_NONOTIFY=0x800000 collides with __O_SYNC in sparc, so change it to 0x1000000.

---
 include/asm-generic/fcntl.h |    2 +-
 include/linux/fs.h          |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
include/asm-generic/fcntl.h +1 -1
@@ -5,7 +5,7 @@
 
 /*
  * FMODE_EXEC is 0x20
- * FMODE_NONOTIFY is 0x800000
+ * FMODE_NONOTIFY is 0x1000000
  * These cannot be used by userspace O_* until internal and external open
  * flags are split.
  * -Eric Paris
include/linux/fs.h +1 -1
@@ -88,7 +88,7 @@ struct inodes_stat_t {
 #define FMODE_NOCMTIME		((__force fmode_t)2048)
 
 /* File was opened by fanotify and shouldn't generate fanotify events */
-#define FMODE_NONOTIFY		((__force fmode_t)8388608)
+#define FMODE_NONOTIFY		((__force fmode_t)0x1000000)
 
 /*
  * The below are the various read and write types that we support. Some of
#4 lore raw

PATCH 4/6 vfs: take f_lock on modifying f_mode after open time

In reply to Wu Fengguang

We'll introduce FMODE_RANDOM which will be runtime modified. So protect all runtime modification to f_mode with f_lock to avoid races.

CC Al Viro <[email protected]>
CC Christoph Hellwig <[email protected]>
---
 fs/file_table.c     |    2 ++
 fs/nfsd/nfs4state.c |    2 ++
 2 files changed, 4 insertions(+)
fs/file_table.c +2 -0
@@ -392,7 +392,9 @@ retry:
 			continue;
 		if (!(f->f_mode & FMODE_WRITE))
 			continue;
+		spin_lock(&f->f_lock);
 		f->f_mode &= ~FMODE_WRITE;
+		spin_unlock(&f->f_lock);
 		if (file_check_writeable(f) != 0)
 			continue;
 		file_release_write(f);
fs/nfsd/nfs4state.c +2 -0
@@ -1998,7 +1998,9 @@ nfs4_file_downgrade(struct file *filp, u
 {
 	if (share_access & NFS4_SHARE_ACCESS_WRITE) {
 		drop_file_write_access(filp);
+		spin_lock(&filp->f_lock);
 		filp->f_mode = (filp->f_mode | FMODE_READ) & ~FMODE_WRITE;
+		spin_unlock(&filp->f_lock);
 	}
 }
 
#3 lore raw

PATCH 3/6 vfs: O_* bit numbers uniqueness check

In reply to Wu Fengguang

The O_* bit numbers are defined in 20+ arch/*, and can silently overlap. Add a compile time check to ensure the uniqueness as suggested by David Miller.

v4: use the nice hweight_long() (suggested by Jamie)
split O_SYNC to (__O_SYNC | O_DSYNC) (suggested by Andreas) take away the FMODE_* and O_RANDOM bits
v3: change to BUILD_BUG_ON() (suggested by Roland)

CC David Miller <[email protected]>
CC Stephen Rothwell <[email protected]>
CC Al Viro <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Eric Paris <[email protected]>
CC Roland Dreier <[email protected]>
CC Jamie Lokier <[email protected]>
CC Andreas Schwab <[email protected]>
---
 fs/fcntl.c                  |   14 ++++++++++++--
 include/asm-generic/fcntl.h |    2 ++
 2 files changed, 14 insertions(+), 2 deletions(-)
fs/fcntl.c +12 -2
@@ -709,11 +709,21 @@ void kill_fasync(struct fasync_struct **
 }
 EXPORT_SYMBOL(kill_fasync);
 
-static int __init fasync_init(void)
+static int __init fcntl_init(void)
 {
+	/* please add new bits here to ensure allocation uniqueness */
+	BUILD_BUG_ON(17 != hweight_long(
+		O_RDONLY	| O_WRONLY	| O_RDWR	|
+		O_CREAT		| O_EXCL	| O_NOCTTY	|
+		O_TRUNC		| O_APPEND	| O_NONBLOCK	|
+		__O_SYNC	| O_DSYNC	| FASYNC	|
+		O_DIRECT	| O_LARGEFILE	| O_DIRECTORY	|
+		O_NOFOLLOW	| O_NOATIME	| O_CLOEXEC
+		));
+
 	fasync_cache = kmem_cache_create("fasync_cache",
 		sizeof(struct fasync_struct), 0, SLAB_PANIC, NULL);
 	return 0;
 }
 
-module_init(fasync_init)
+module_init(fcntl_init)
include/asm-generic/fcntl.h +2 -0
@@ -4,6 +4,8 @@
 #include <linux/types.h>
 
 /*
+ * When introducing new O_* bits, please check its uniqueness in fcntl_init().
+ *
  * FMODE_EXEC is 0x20
  * FMODE_NONOTIFY is 0x1000000
  * These cannot be used by userspace O_* until internal and external open
#7 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

In reply to Wu Fengguang
Quoted 21 lines · Wu Fengguang
fs/read_write.c +14 -0
+static int
+__negative_fpos_check(struct file *file, loff_t pos, size_t count)
+{
+	/*
+	 * pos or pos+count is negative here, check overflow.
+	 * too big "count" will be caught in rw_verify_area().
+	 */
+	if ((pos < 0) && (pos + count < pos))
+		return -EOVERFLOW;
+	if (file->f_mode & FMODE_NEG_OFFSET)
+		return 0;
+	return -EINVAL;
+}
+
 /*
  * rw_verify_area doesn't like huge counts. We limit
  * them to something that fits in "int" so that others
@@ -222,8 +236,11 @@ int rw_verify_area(int read_write, struc
 	if (unlikely((ssize_t) count < 0))
 		return retval;
 	pos = *ppos;

fs/read_write.c int rw_verify_area(int read_write, struc · line 246

Quoted 10 lines
fs/read_write.c +5 -2
-	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0))
-		return retval;
+	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0)) {
+		retval = __negative_fpos_check(file, pos, count);
+		if (retval)
+			return retval;
+	}
 
 	if (unlikely(inode->i_flock && mandatory_lock(inode))) {
 		retval = locks_mandatory_area(

Um... How do lseek() work? It sounds like to violate error code range.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#8 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 31 lines · On Sat, 16 Jan 2010 21:54:39 +0900 OGAWA Hirofumi…
Quoted 29 lines · Wu Fengguang
fs/read_write.c +19 -2
+static int
+__negative_fpos_check(struct file *file, loff_t pos, size_t count)
+{
+	/*
+	 * pos or pos+count is negative here, check overflow.
+	 * too big "count" will be caught in rw_verify_area().
+	 */
+	if ((pos < 0) && (pos + count < pos))
+		return -EOVERFLOW;
+	if (file->f_mode & FMODE_NEG_OFFSET)
+		return 0;
+	return -EINVAL;
+}
+
 /*
  * rw_verify_area doesn't like huge counts. We limit
  * them to something that fits in "int" so that others
@@ -222,8 +236,11 @@ int rw_verify_area(int read_write, struc
 	if (unlikely((ssize_t) count < 0))
 		return retval;
 	pos = *ppos;
-	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0))
-		return retval;
+	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0)) {
+		retval = __negative_fpos_check(file, pos, count);
+		if (retval)
+			return retval;
+	}
 
Quoted 4 lines

fs/read_write.c int rw_verify_area(int read_write, struc · line 246

Quoted 2 lines
fs/read_write.c +0 -0
 	if (unlikely(inode->i_flock && mandatory_lock(inode))) {
 		retval = locks_mandatory_area(

Um... How do lseek() work? It sounds like to violate error code range.

This is for read-write. As far as I know,
- generic_file_llseek,
- default_llseek
- no_llseek

doesn't call this function.

Thanks,
-Kame

#9 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 34 lines · KAMEZAWA Hiroyuki
Quoted 29 lines
Quoted 29 lines
fs/read_write.c +19 -2
+static int
+__negative_fpos_check(struct file *file, loff_t pos, size_t count)
+{
+	/*
+	 * pos or pos+count is negative here, check overflow.
+	 * too big "count" will be caught in rw_verify_area().
+	 */
+	if ((pos < 0) && (pos + count < pos))
+		return -EOVERFLOW;
+	if (file->f_mode & FMODE_NEG_OFFSET)
+		return 0;
+	return -EINVAL;
+}
+
 /*
  * rw_verify_area doesn't like huge counts. We limit
  * them to something that fits in "int" so that others
@@ -222,8 +236,11 @@ int rw_verify_area(int read_write, struc
 	if (unlikely((ssize_t) count < 0))
 		return retval;
 	pos = *ppos;
-	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0))
-		return retval;
+	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0)) {
+		retval = __negative_fpos_check(file, pos, count);
+		if (retval)
+			return retval;
+	}
 
Quoted 4 lines

fs/read_write.c int rw_verify_area(int read_write, struc · line 246

Quoted 2 lines
fs/read_write.c +0 -0
 	if (unlikely(inode->i_flock && mandatory_lock(inode))) {
 		retval = locks_mandatory_area(

Um... How do lseek() work? It sounds like to violate error code range.

Quoted 6 lines

This is for read-write. As far as I know,
- generic_file_llseek,
- default_llseek
- no_llseek

doesn't call this function.

It seems to allow to set negative value to ->f_pos, right? So, lseek() returns (uses) it?

Thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#10 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 36 lines · On Mon, 18 Jan 2010 10:17:48 +0900 OGAWA Hirofumi…
Quoted 34 lines · KAMEZAWA Hiroyuki
Quoted 33 lines
Quoted 21 lines
fs/read_write.c +14 -0
+static int
+__negative_fpos_check(struct file *file, loff_t pos, size_t count)
+{
+	/*
+	 * pos or pos+count is negative here, check overflow.
+	 * too big "count" will be caught in rw_verify_area().
+	 */
+	if ((pos < 0) && (pos + count < pos))
+		return -EOVERFLOW;
+	if (file->f_mode & FMODE_NEG_OFFSET)
+		return 0;
+	return -EINVAL;
+}
+
 /*
  * rw_verify_area doesn't like huge counts. We limit
  * them to something that fits in "int" so that others
@@ -222,8 +236,11 @@ int rw_verify_area(int read_write, struc
 	if (unlikely((ssize_t) count < 0))
 		return retval;
 	pos = *ppos;

fs/read_write.c int rw_verify_area(int read_write, struc · line 246

Quoted 10 lines
fs/read_write.c +5 -2
-	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0))
-		return retval;
+	if (unlikely((pos < 0) || (loff_t) (pos + count) < 0)) {
+		retval = __negative_fpos_check(file, pos, count);
+		if (retval)
+			return retval;
+	}
 
 	if (unlikely(inode->i_flock && mandatory_lock(inode))) {
 		retval = locks_mandatory_area(

Um... How do lseek() work? It sounds like to violate error code range.

Quoted 8 lines
Quoted 6 lines

This is for read-write. As far as I know,
- generic_file_llseek,
- default_llseek
- no_llseek

doesn't call this function.

It seems to allow to set negative value to ->f_pos, right?

yes. Some file (/dev/kmem) requires that.

Quoted 1 line

So, lseek() returns (uses) it?

lseek can return negative value, as far as I know.

Thanks,
-Kame

#19 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

In reply to OGAWA Hirofumi
Quoted 11 lines · OGAWA Hirofumi
Quoted 8 lines
Quoted 1 line

Um... How do lseek() work? It sounds like to violate error code range.

This is for read-write. As far as I know,
- generic_file_llseek,
- default_llseek
- no_llseek

doesn't call this function.

It seems to allow to set negative value to ->f_pos, right? So, lseek() returns (uses) it?

BTW, another concern by negative "pos" value is, the following like code

pos >> shift_bits

it will break the above. So, I think it should be checked if not yet.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#11 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

In reply to KAMEZAWA Hiroyuki
Quoted 3 lines · KAMEZAWA Hiroyuki
Quoted 1 line

So, lseek() returns (uses) it?

lseek can return negative value, as far as I know.

Umm..., how do you know the difference of -EOVERFLOW and fpos == -75?

Thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#20 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

In reply to OGAWA Hirofumi
Quoted 11 lines · On Mon, 18 Jan 2010 10:32:49 +0900 OGAWA Hirofumi…
Quoted 9 lines · OGAWA Hirofumi
Quoted 8 lines
Quoted 1 line

Um... How do lseek() work? It sounds like to violate error code range.

This is for read-write. As far as I know,
- generic_file_llseek,
- default_llseek
- no_llseek

doesn't call this function.

Quoted 8 lines
Quoted 2 lines

It seems to allow to set negative value to ->f_pos, right? So, lseek() returns (uses) it?

BTW, another concern by negative "pos" value is, the following like code

pos >> shift_bits

it will break the above. So, I think it should be checked if not yet.

Where do we check ?

FMODE_NEG_OFFSET is just used by /dev/mem and /proc/<pid>/mem. And I don't think there are no additonal users. So, I myself don't have has such concerns...

Thanks,
-Kame

#21 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 10 lines · KAMEZAWA Hiroyuki
Quoted 5 lines

BTW, another concern by negative "pos" value is, the following like code

pos >> shift_bits

it will break the above. So, I think it should be checked if not yet.

Where do we check ?

FMODE_NEG_OFFSET is just used by /dev/mem and /proc/<pid>/mem. And I don't think there are no additonal users. So, I myself don't have has such concerns...

Sorry, it's just my concern. I'm not checking real path (e.g. vfs) of related to /dev/mem, if there is no user of such code, it's ok.

Thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#12 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

In reply to OGAWA Hirofumi
Quoted 8 lines · On Mon, 18 Jan 2010 10:38:27 +0900 OGAWA Hirofumi…
Quoted 3 lines · KAMEZAWA Hiroyuki
Quoted 1 line

So, lseek() returns (uses) it?

lseek can return negative value, as far as I know.

Umm..., how do you know the difference of -EOVERFLOW and fpos == -75?

Ah, sorry. I read wrong.

For /dev/mem, it uses its own lseek function which allows negative f_pos value. Other usual file system doesn't allow negative f_pos.

It's ok not to return -EOVEFLOW for /dev/mem because there is no file end.

Thanks,
-Kame

#13 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 12 lines · KAMEZAWA Hiroyuki
Quoted 8 lines · On Mon, 18 Jan 2010 10:38:27 +0900 OGAWA Hirofumi…
Quoted 3 lines · KAMEZAWA Hiroyuki
Quoted 1 line

So, lseek() returns (uses) it?

lseek can return negative value, as far as I know.

Umm..., how do you know the difference of -EOVERFLOW and fpos == -75?

Quoted 6 lines

Ah, sorry. I read wrong.

For /dev/mem, it uses its own lseek function which allows negative f_pos value. Other usual file system doesn't allow negative f_pos.

It's ok not to return -EOVEFLOW for /dev/mem because there is no file end.

No, no. I think it has the problem.

E.g. /dev/mem returns -75 as fpos, so, lseek(2) returns -75 to userland. Then the userland (e.g. glibc) convert it as error. I.e. finally, errno == -75, and lseek(3) returns -1, right?

Thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#14 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 19 lines · On Mon, 18 Jan 2010 11:13:04 +0900 OGAWA Hirofumi…
Quoted 17 lines · KAMEZAWA Hiroyuki
Quoted 8 lines · On Mon, 18 Jan 2010 10:38:27 +0900 OGAWA Hirofumi…
Quoted 3 lines · KAMEZAWA Hiroyuki
Quoted 1 line

So, lseek() returns (uses) it?

lseek can return negative value, as far as I know.

Umm..., how do you know the difference of -EOVERFLOW and fpos == -75?

Ah, sorry. I read wrong.

For /dev/mem, it uses its own lseek function which allows negative f_pos value. Other usual file system doesn't allow negative f_pos.

Quoted 8 lines
Quoted 1 line

It's ok not to return -EOVEFLOW for /dev/mem because there is no file end.

No, no. I think it has the problem.

E.g. /dev/mem returns -75 as fpos, so, lseek(2) returns -75 to userland. Then the userland (e.g. glibc) convert it as error. I.e. finally, errno == -75, and lseek(3) returns -1, right?

Maybe possible.

Hmm. Then, /dev/mem's llseek need some fix not to return pos < -PAGESIZE. Wu-san, could you add additional bug fix to lseek()'s f_pos handling in /dev/mem ?

Thanks,
-Kame

#15 lore raw

RE: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Hi,

[replying from webmail, sorry for top-posting]

memory_lseek() calls force_successful_syscall_return() to force success on negative vals. However that is a no-op for x86.

My experiment shows that lseek() does return negative pos. However, manual says that "a value of (off_t) -1 is returned" on error. So it's OK as long as your program is written as "err == -1" instead of "err < 0".

code:

        err = lseek64(fd, addr, SEEK_SET);
        if (err == -1)
                perror("seek " FILENAME);

output:

  # kmem-rw 0xffffffffa0094000
  addr=0xffffffffa0094000 val=0x441f0fe5894855

strace:
open("/dev/kmem", O_RDWR) = 3
lseek(3, 18446744072099545088, SEEK_SET) = 18446744072099545088
read(3, "UH\211\345\17\37D\0"..., 8) = 8

Thanks,
Fengguang
________________________________________
From: KAMEZAWA Hiroyuki [[email protected]]
Sent: Monday, January 18, 2010 10:30 AM
To: OGAWA Hirofumi
Cc: Wu, Fengguang; Andrew Morton; Al Viro; Heiko Carstens; Christoph Hellwig; LKML; Eric Paris; Nick Piggin; Andi Kleen; David Howells; Jonathan Corbet; [email protected]
Subject: Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 19 lines · On Mon, 18 Jan 2010 11:13:04 +0900 OGAWA Hirofumi…
Quoted 17 lines · KAMEZAWA Hiroyuki
Quoted 8 lines · On Mon, 18 Jan 2010 10:38:27 +0900 OGAWA Hirofumi…
Quoted 3 lines · KAMEZAWA Hiroyuki
Quoted 1 line

So, lseek() returns (uses) it?

lseek can return negative value, as far as I know.

Umm..., how do you know the difference of -EOVERFLOW and fpos == -75?

Ah, sorry. I read wrong.

For /dev/mem, it uses its own lseek function which allows negative f_pos value. Other usual file system doesn't allow negative f_pos.

Quoted 8 lines
Quoted 1 line

It's ok not to return -EOVEFLOW for /dev/mem because there is no file end.

No, no. I think it has the problem.

E.g. /dev/mem returns -75 as fpos, so, lseek(2) returns -75 to userland. Then the userland (e.g. glibc) convert it as error. I.e. finally, errno == -75, and lseek(3) returns -1, right?

Maybe possible.

Hmm. Then, /dev/mem's llseek need some fix not to return pos < -PAGESIZE. Wu-san, could you add additional bug fix to lseek()'s f_pos handling in /dev/mem ?

Thanks,
-Kame

#16 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 11 lines · Fengguang

Hi,

[replying from webmail, sorry for top-posting]

memory_lseek() calls force_successful_syscall_return() to force success on negative vals. However that is a no-op for x86.

My experiment shows that lseek() does return negative pos. However, manual says that "a value of (off_t) -1 is returned" on error. So it's OK as long as your program is written as "err == -1" instead of "err < 0".

On error, the kernel returns -EOVERFLOW (via %eax) and libc hides it by
errno = EOVERFLOW
ret = -1

The problem discussed here is the kernel's return value. So, the kernel's lseek should check that, I think.

Anyway, this lseek problem is not related to this patch itself and has existed for very long time. Fixing it later by another patch is not very bad, I think.
(I'm sorry I myself is not ready for writing a patch...)

Thaks,
-Kame

#17 lore raw

RE: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 5 lines

On error, the kernel returns -EOVERFLOW (via %eax) and libc hides it by
errno = EOVERFLOW
ret = -1

Ah got it. How about the attached patch?

Thanks,
Fengguang

Reviewed-by
#18 lore raw

Re: [PATCH 6/6] vfs: introduce FMODE_NEG_OFFSET for allowing negative f_pos

Quoted 8 lines · Fengguang
Quoted 5 lines

On error, the kernel returns -EOVERFLOW (via %eax) and libc hides it by
errno = EOVERFLOW
ret = -1

Ah got it. How about the attached patch?

Seems good to me. Thank you very much.