Skip to contentSkip to conversation

2Gb file size limit on 2.4.24, LVM and ext3?

DISCUSSION Niccolo Rigacci 4 participants 6 messages May 6, 2004 – May 10, 2004

lore
Share & tools
lore
#0 lore raw

Hi to all!

I got a very strange problem: I can create files larger than 2 Gb (even 5 Gb), but I can't read them back.

- The simple "cat" command fails with:

  # ls -la pippo
  -rw-r--r--    1 root     root     2147483648 May  6 17:03 pippo
  # cat pippo
  cat: pippo: Operation not permitted

- A file just 2kb under 2Gb, reads fine.

- If I do an "strace cat pippo" it works fine! So how can I trace the problem further?

- The partition is an ext3 over LVM, kernel 2.4.24. Debian Woody (glibc-2.2.5-11.5). Pentium 4 2.80GHz.
I tried both a quoted filesystem and a non quoted with same results.

- On a very similar system I have no problem, the main difference is using LVM here.

Is there a known issue? Can someone tell me how can trace down the problem?

Many thanks!

Please CC to me if you can.

Some info on the machine:

# uname -a
Linux argo 2.4.24 #1 Wed Feb 11 19:03:31 CET 2004 i686 unknown
# cat /proc/lvm/global
LVM driver LVM version 1.0.7(28/03/2003)
Total:  1 VG  1 PV  2 LVs (2 LVs open 2 times)
Global: 41615 bytes malloced IOP version:10 57 days ... active
VG:  vg0  [1 PV, 2 LV/2 open]  PE Size: 32768 KB
  Usage [KB/PE]: 76709888/2341 total 76709888/2341 used 0/0 free
  PV:  [AA] md2  76709888 /2341   76709888 /2341     0 /0
    LVs: [AWDL  ] var             20971520 /640      1x open
         [AWDL  ] home            55738368 /1701     1x open
argo:~# mount
/dev/md0 on / type ext3 (rw,errors=remount-ro)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/vg0/var on /var type ext3 (rw)
/dev/vg0/home on /home type ext3 (rw,nodev,usrquota)
Signature · 4 lines
Niccolo Rigacci
Firenze - Italy

War against Iraq? Not in my name!
#1 lore raw
Quoted 21 lines · Niccolo Rigacci

Hi to all!

I got a very strange problem: I can create files larger than 2 Gb (even 5 Gb), but I can't read them back.

- The simple "cat" command fails with:

 # ls -la pippo
 -rw-r--r--    1 root     root     2147483648 May  6 17:03 pippo
 # cat pippo
 cat: pippo: Operation not permitted

- A file just 2kb under 2Gb, reads fine.

- If I do an "strace cat pippo" it works fine! So how can I trace the problem further?

- The partition is an ext3 over LVM, kernel 2.4.24. Debian Woody (glibc-2.2.5-11.5). Pentium 4 2.80GHz.
I tried both a quoted filesystem and a non quoted with same results.

Quoted 8 lines

- On a very similar system I have no problem, the main difference is using LVM here.

Is there a known issue? Can someone tell me how can trace down the problem?

What's your glibc version ?

Old glibces didn't support large files.

--
GJ

#2 lore raw
In reply to Niccolo Rigacci
Quoted 4 lines · Niccolo Rigacci

- The partition is an ext3 over LVM, kernel 2.4.24. Debian Woody (glibc-2.2.5-11.5). Pentium 4 2.80GHz.

Ooops, sorry ;) can see it now.
afair you need at least 2.3 for large files. But I am not 100% sure.

--
GJ

#3 lore raw
Quoted 7 lines · Grzegorz Piotr Jaskiewicz
Quoted 4 lines · Niccolo Rigacci

- The partition is an ext3 over LVM, kernel 2.4.24. Debian Woody (glibc-2.2.5-11.5). Pentium 4 2.80GHz.

Ooops, sorry ;) can see it now.
afair you need at least 2.3 for large files. But I am not 100% sure.

No, glibc-2.2.5-11.5 will also do large files. Just be sure that you open() the file with O_LARGEFILE. I usually have this piece of code in one of my header files to get it done:

#ifndef O_LARGEFILE
#define O_LARGEFILE 0100000
#endif

I always forget about the glibc #define-du-jour to get O_LARGEFILE defined, this always works.

Erik

Signature · 2 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
#4 lore raw
Quoted 2 lines · Erik Mouw

No, glibc-2.2.5-11.5 will also do large files. Just be sure that you open() the file with O_LARGEFILE.

I definitively think that cp uses O_LARGEFILE. Infact if I strace the command I can see:

open("argo_full.tar.gz", O_RDONLY|O_LARGEFILE) = 3

but the problem now is that with strace the command works OK! What the hell that stracing works and without strace it does not work?

Signature · 4 lines
Niccolo Rigacci
Firenze - Italy

War against Iraq? Not in my name!
#5 lore raw
In reply to Erik Mouw
Quoted 2 lines · Erik Mouw

I always forget about the glibc #define-du-jour to get O_LARGEFILE defined, this always works.

#define _GNU_SOURCE
#include <...>