Skip to contentSkip to conversation

fanotify: fix support of large files

PATCH In review Justin Maggard 2 participants 5 messages Feb 14, 2013 – Jul 11, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fanotify: fix support of large files

Opening a file of >2GB in an area of the filesystem that has been marked for fanotify events currently results in an EOVERFLOW error. This is particularly problematic if you are using fanotify permissions checking, because it prevents large files from being opened at all. Fix this by setting the O_LARGEFILE flag on the new file handle.

---
 fs/notify/fanotify/fanotify_user.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
fs/notify/fanotify/fanotify_user.c +3 -3
index 9ff4a5e..ef02b3d 100644
@@ -64,6 +64,7 @@ static int create_fd(struct fsnotify_group *group,
 			struct file **file)
 {
 	int client_fd;
+	int f_flags;
 	struct file *new_file;
 
 	pr_debug("%s: group=%p event=%p\n", __func__, group, event);
@@ -82,12 +83,11 @@ static int create_fd(struct fsnotify_group *group,
 	 * we need a new file handle for the userspace program so it can read even if it was
 	 * originally opened O_WRONLY.
 	 */
+	f_flags = group->fanotify_data.f_flags | FMODE_NONOTIFY | O_LARGEFILE;
 	/* it's possible this event was an overflow event.  in that case dentry and mnt
 	 * are NULL;  That's fine, just don't call dentry open */
 	if (event->path.dentry && event->path.mnt)
-		new_file = dentry_open(&event->path,
-				       group->fanotify_data.f_flags | FMODE_NONOTIFY,
-				       current_cred());
+		new_file = dentry_open(&event->path, f_flags, current_cred());
 	else
 		new_file = ERR_PTR(-EOVERFLOW);
 	if (IS_ERR(new_file)) {
Signature · 1 line
1.7.9.5
#1 lore raw

Dear Justin,

looking at the example at
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
the large file support is enabled by passing O_LARGEFILE to fanotify_init:

if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
O_RDONLY | O_CLOEXEC |
O_LARGEFILE)) < 0)

Could you, please, check if this solves your issue.

(I am resending this message because HTML was rejected by [email protected]).

Best regards

Heinrich Schuchardt

Quoted 40 lines · Justin Maggard

Opening a file of>2GB in an area of the filesystem that has been marked for fanotify events currently results in an EOVERFLOW error. This is particularly problematic if you are using fanotify permissions checking, because it prevents large files from being opened at all. Fix this by setting the O_LARGEFILE flag on the new file handle.

---
fs/notify/fanotify/fanotify_user.c | 6 +++---

  1 file changed, 3 insertions(+), 3 deletions(-)
fs/notify/fanotify/fanotify_user.c +3 -3
index 9ff4a5e..ef02b3d 100644
@@ -64,6 +64,7 @@ static int create_fd(struct fsnotify_group *group,
  			struct file **file)
  {
  	int client_fd;
+	int f_flags;
  	struct file *new_file;
 
  	pr_debug("%s: group=%p event=%p\n", __func__, group, event);
@@ -82,12 +83,11 @@ static int create_fd(struct fsnotify_group *group,
  	 * we need a new file handle for the userspace program so it can read even if it was
  	 * originally opened O_WRONLY.
  	 */
+	f_flags = group->fanotify_data.f_flags | FMODE_NONOTIFY | O_LARGEFILE;
  	/* it's possible this event was an overflow event.  in that case dentry and mnt
  	 * are NULL;  That's fine, just don't call dentry open */
  	if (event->path.dentry&&  event->path.mnt)
-		new_file = dentry_open(&event->path,
-				       group->fanotify_data.f_flags | FMODE_NONOTIFY,
-				       current_cred());
+		new_file = dentry_open(&event->path, f_flags, current_cred());
  	else
  		new_file = ERR_PTR(-EOVERFLOW);
  	if (IS_ERR(new_file)) {
#2 lore raw
Quoted 10 lines · Heinrich Schuchardt

Dear Justin,

looking at the example at
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
the large file support is enabled by passing O_LARGEFILE to fanotify_init:

if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
O_RDONLY | O_CLOEXEC | O_LARGEFILE)) <
0)

Quoted 9 lines

Could you, please, check if this solves your issue.

(I am resending this message because HTML was rejected by [email protected]).

Best regards

Heinrich Schuchardt

No, unfortunately that doesn't help. I slightly modifed fanotify-example.c to call perror() when read() fails, and here's the output:

jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example . &
[1] 7248
jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
truncate -s 2047m 2047m
Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
FAN_OPEN
FAN_CLOSE_WRITE
jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
read: Value too large for defined data type

-Justin

#3 lore raw

Hello Justin,

I downloaded the example,
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
compiled it without modification

$ gcc fanotify-example.c -o fanotify-example
and started the executable. In a separate windows I executed
truncate -s 2048m 2048m

This is the output of the first window:

$ sudo ./fanotify-example /home/user/temp/
Started monitoring directory '/home/user/temp/'...
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
        FAN_OPEN
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
        FAN_CLOSE_WRITE
$ uname -a
Linux family2 3.8.0 #1 SMP Fri Feb 22 22:07:58 CET 2013 i686 GNU/Linux

To reproduce your problem, could you, please, provide the Linux configuration file (look in your /boot directory) and a link to the source of the Linux kernel version you use. Then I can use your configuration file to compile that kernel version.

Best regards

Heinrich Schuchardt

Quoted 37 lines · Justin Maggard
Quoted 10 lines · Heinrich Schuchardt

Dear Justin,

looking at the example at
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
the large file support is enabled by passing O_LARGEFILE to fanotify_init:

if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
O_RDONLY | O_CLOEXEC | O_LARGEFILE))<
0)

Quoted 9 lines

Could you, please, check if this solves your issue.

(I am resending this message because HTML was rejected by [email protected]).

Best regards

Heinrich Schuchardt

No, unfortunately that doesn't help. I slightly modifed fanotify-example.c to call perror() when read() fails, and here's the output:

jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example .&
[1] 7248
jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
truncate -s 2047m 2047m
Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
FAN_OPEN
FAN_CLOSE_WRITE
jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
read: Value too large for defined data type

-Justin

#4 lore raw
Quoted 18 lines · Heinrich Schuchardt

Hello Justin,

I downloaded the example,
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
compiled it without modification

$ gcc fanotify-example.c -o fanotify-example
and started the executable. In a separate windows I executed
truncate -s 2048m 2048m

This is the output of the first window:

$ sudo ./fanotify-example /home/user/temp/
Started monitoring directory '/home/user/temp/'...
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
        FAN_OPEN
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
        FAN_CLOSE_WRITE
Quoted 3 lines
$ uname -a
Linux family2 3.8.0 #1 SMP Fri Feb 22 22:07:58 CET 2013 i686 GNU/Linux

Ah yes, that's the reason. I'm running on x86_64. 32-bit platforms actually work if you set O_LARGEFILE. From /usr/incude/bits/fcntl.h:

#ifdef __USE_LARGEFILE64

# if __WORDSIZE == 64
#  define O_LARGEFILE   0
# else
#  define O_LARGEFILE   0100000
# endif
#endif

If I include <asm/fcntl.h> instead of <fcntl.h>, it gets set to 0x8000, and my large file fanotify test works. Or, of course, I can define that flag locally and use it. But it feels like I shouldn't have to do that.

-Justin

Quoted 49 lines

To reproduce your problem, could you, please, provide the Linux configuration file (look in your /boot directory) and a link to the source of the Linux kernel version you use. Then I can use your configuration file to compile that kernel version.

Best regards

Heinrich Schuchardt

Quoted 39 lines · Justin Maggard
Quoted 11 lines · Heinrich Schuchardt

Dear Justin,

looking at the example at
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
the large file support is enabled by passing O_LARGEFILE to fanotify_init:

if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
O_RDONLY | O_CLOEXEC | O_LARGEFILE))<
0)

Quoted 9 lines

Could you, please, check if this solves your issue.

(I am resending this message because HTML was rejected by [email protected]).

Best regards

Heinrich Schuchardt

No, unfortunately that doesn't help. I slightly modifed fanotify-example.c to call perror() when read() fails, and here's the output:

jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example .&
[1] 7248
jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
truncate -s 2047m 2047m
Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
FAN_OPEN
FAN_CLOSE_WRITE
jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
read: Value too large for defined data type

-Justin