Skip to contentSkip to conversation

Bug Fix for 2GB core limit in 2.4

DISCUSSION Javier Olivares 2 participants 2 messages May 30, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

We were having problems when running programs that used over 2GB of ram not being able to generate core files over 2GB, these are some very simple changes that fixed the problem.

linux-2.4.31/fs/binfmt_elf.c
1024c1024
< static int dump_seek(struct file *file, off_t off)
---

Quoted 1 line

static int dump_seek(struct file *file, loff_t off)

Changed the function parameter "off" from type "off_t" to "loff_t". The parameter was truncating the incoming long long type to a long, causing the seek to fail and kill the dump when off grew above 2GB.

/kernels/2.4.31/linux-2.4.31/fs/exec.c
1151c1151
< file = filp_open(corename, O_CREAT | 2 | O_NOFOLLOW, 0600);
---

Quoted 1 line
    file = filp_open(corename, O_CREAT | 2 | O_NOFOLLOW | O_LARGEFILE, 0600);

Included the O_LARGEFILE flag in order to create files over 2GB.

The changes have been running on many Debian systems for a couple of months. Valid core files just over 3GB have been created without any problem. I have never submitted anything like this before so please excuse any lack of proper protocol. Thank you.

-Javier Olivares

#1 lore raw

Hello,

Quoted 8 lines · Javier Olivares

We were having problems when running programs that used over 2GB of ram not being able to generate core files over 2GB, these are some very simple changes that fixed the problem.

linux-2.4.31/fs/binfmt_elf.c
1024c1024
< static int dump_seek(struct file *file, off_t off)
---

Quoted 5 lines
Quoted 1 line

static int dump_seek(struct file *file, loff_t off)

Changed the function parameter "off" from type "off_t" to "loff_t". The parameter was truncating the incoming long long type to a long, causing the seek to fail and kill the dump when off grew above 2GB.

You change looks right.

Quoted 5 lines

/kernels/2.4.31/linux-2.4.31/fs/exec.c
1151c1151
< file = filp_open(corename, O_CREAT | 2 | O_NOFOLLOW, 0600);
---

Quoted 1 line

file = filp_open(corename, O_CREAT | 2 | O_NOFOLLOW |

Quoted 8 lines

O_LARGEFILE, 0600);

Included the O_LARGEFILE flag in order to create files over 2GB.

The changes have been running on many Debian systems for a couple of months. Valid core files just over 3GB have been created without any problem. I have never submitted anything like this before so please excuse any lack of proper protocol.

You would be interested in reading this file in the kernel sources :

Documentation/SubmittingPatches

Most important to remember : use "diff -u" to make the patch, and don't forget to CC the maintainer, particularly on 2.4 patches, because very few patches are related to 2.4 right now on LKML and it's easy to miss them. Anyway, your changes are quite self-explanatory and are very easy to redo by hand. Here's the patch re-done as a diff -u (in fact, as exported by Git but the output from diff -u would be the same).

fs/binfmt_elf.c +1 -1
@@ -1026,7 +1026,7 @@ static int dump_write(struct file *file,
 	return file->f_op->write(file, addr, nr, &file->f_pos) == nr;
 }
 
-static int dump_seek(struct file *file, off_t off)
+static int dump_seek(struct file *file, loff_t off)
 {
 	if (file->f_op->llseek) {
 		if (file->f_op->llseek(file, off, 0) != off)
fs/exec.c +2 -1
@@ -1148,7 +1148,8 @@ int do_coredump(long signr, struct pt_re
 		goto fail;
 
  	format_corename(corename, core_pattern, signr);
-	file = filp_open(corename, O_CREAT | 2 | O_NOFOLLOW, 0600);
+	file = filp_open(corename,
+	                 O_CREAT | 2 | O_NOFOLLOW | O_LARGEFILE, 0600);
 	if (IS_ERR(file))
 		goto fail;
 	inode = file->f_dentry->d_inode;
Quoted 3 lines

Thank you.

-Javier Olivares

Thank you.

Marcelo, this looks valid to me, I've queued it.
Willy