Skip to contentSkip to conversation

Filesize limit on SMBFS

DISCUSSION Tyler BIRD 11 participants 14 messages Nov 22, 2001 – Nov 26, 2001

lore
Share & tools
lore

Tyler BIRD

author
#0 lore raw

Ext2 Filesystems I believe have the limit of 2 GB. Ext3 Extends that Limit to something?? Try making the ext3 filesystem partitions and sharing those.
I don't know limits on FAT32 or any other filesystem you can share

Tyler

Quoted 1 line
Quoted 1 line
Quoted 1 line

Petr Tite(ra <[email protected]> 11/22/01 02:10AM >>>

Hello,

is maximum file size on SMBFS really 2GB? I cannot create file
bigger than that.

Petr Titera
[email protected]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#1 lore raw

This limit is a kernel´s limit not a file system´s limit. Even vfat has a limitation of 2GB under linux. I thought with kernel 2.4.x this will be over.
----- Original Message -----
From: "Tyler BIRD" <[email protected]>
To: <[email protected]>; <[email protected]>
Sent: Thursday, November 22, 2001 4:53 PM
Subject: Re: Filesize limit on SMBFS

Quoted 1 line

Ext2 Filesystems I believe have the limit of 2 GB. Ext3 Extends that

Limit to something??

Quoted 27 lines

Try making the ext3 filesystem partitions and sharing those. I don't know limits on FAT32 or any other filesystem you can share

Tyler

Quoted 1 line
Quoted 1 line
Quoted 1 line

Petr Tite(ra <[email protected]> 11/22/01 02:10AM >>>

Hello,

is maximum file size on SMBFS really 2GB? I cannot create file
bigger than that.

Petr Titera
[email protected]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#3 lore raw
In reply to Tyler BIRD
Quoted 3 lines · Marcelo Borges Ribeiro

This limit is a kernel´s limit not a file system´s limit. Even vfat has a limitation of 2GB under linux. I thought with kernel 2.4.x this will be over.

Totally incorrect. 2.4 allows files larger than 2GB, and with a patch, you can do this on 2.2 as well. If you are having problems with a 2GB limit, then either your shell, libc, or tools is causing the problem.

VFAT does have a 2GB limit, AFAIK, but I could be wrong.

Quoted 4 lines

----- Original Message -----
From: "Tyler BIRD" <[email protected]>

Quoted 2 lines

Ext2 Filesystems I believe have the limit of 2 GB. Ext3 Extends that Limit to something??

No, the ext2 and ext3 limits are exactly the same, about 4TB right now, but they would be larger with a bit of bug fixing (up to 16TB). Note that the kernel has a limit of 2TB for a single device.

Quoted 2 lines
Quoted 2 lines

is maximum file size on SMBFS really 2GB? I cannot create file
bigger than that.

As for SMBFS, I don't know, but it can obviously not be larger than the limit on the server.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#4 lore raw
Quoted 1 line · Andreas Dilger

VFAT does have a 2GB limit, AFAIK, but I could be wrong.

Use "mkdosfs -F32" or use msdos fdisk,format to get >2GB.

I'm using 3GB for VFAT partition.

Jeff.

#5 lore raw

I have fat32 partition, but the problem isn´t the size of partition it is 8GB. The problem is that if you want to
create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2 to a reiserfs) to an available fat32 space, in my case the backup size is allways 2GB and when I tried to extract back I saw "unexpected end of file". So I thought it was that famous kernel limitation of 2GB under any kind of partition, but i was informed that fat has this limitation too. So the kernel may suport files bigger than 2GB (I really don´t know, I just know that in my case with fat32 it did not and I saw this too with some oracle databases that could not be used when they grow and reach 2GB, may be a library problem too).

That´s all.
----- Original Message -----
From: "Jeff Chua" <[email protected]>
To: "Andreas Dilger" <[email protected]>
Cc: "Marcelo Borges Ribeiro" <[email protected]>; "Tyler BIRD" <[email protected]>; <[email protected]>
Sent: Friday, November 23, 2001 12:35 AM
Subject: Re: Filesize limit on SMBFS

Quoted 17 lines · Andreas Dilger
Quoted 2 lines

VFAT does have a 2GB limit, AFAIK, but I could be wrong.

Use "mkdosfs -F32" or use msdos fdisk,format to get >2GB.

I'm using 3GB for VFAT partition.

Jeff.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#6 lore raw

You may want to try using glibc 2.2.x, but then you may fail to compile many other codes. I tried it once, and managed to get >2GB file, but had to revert back to 2.1.3 because 2.2.x broke compiling other codes. That was a few months back.

Thanks,
Jeff
[ [email protected] ]

Quoted 40 lines · Marcelo Borges Ribeiro

I have fat32 partition, but the problem isn�t the size of partition it is 8GB. The problem is that if you want to
create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2 to a reiserfs) to an available fat32 space, in my case the backup size is allways 2GB and when I tried to extract back I saw "unexpected end of file". So I thought it was that famous kernel limitation of 2GB under any kind of partition, but i was informed that fat has this limitation too. So the kernel may suport files bigger than 2GB (I really don�t know, I just know that in my case with fat32 it did not and I saw this too with some oracle databases that could not be used when they grow and reach 2GB, may be a library problem too).

That�s all.
----- Original Message -----
From: "Jeff Chua" <[email protected]>
To: "Andreas Dilger" <[email protected]>
Cc: "Marcelo Borges Ribeiro" <[email protected]>; "Tyler BIRD" <[email protected]>; <[email protected]>
Sent: Friday, November 23, 2001 12:35 AM
Subject: Re: Filesize limit on SMBFS

Quoted 17 lines · Andreas Dilger
Quoted 2 lines

VFAT does have a 2GB limit, AFAIK, but I could be wrong.

Use "mkdosfs -F32" or use msdos fdisk,format to get >2GB.

I'm using 3GB for VFAT partition.

Jeff.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#7 lore raw
In reply to Jeff Chua
Quoted 11 lines · Marcelo Borges Ribeiro

I have fat32 partition, but the problem isn´t the size of partition it is 8GB. The problem is that if you want to
create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2 to a reiserfs) to an available fat32 space, in my case the backup size is allways 2GB and when I tried to extract back I saw "unexpected end of file". So I thought it was that famous kernel limitation of 2GB under any kind of partition, but i was informed that fat has this limitation too. So the kernel may suport files bigger than 2GB (I really don´t know, I just know that in my case with fat32 it did not and I saw this too with some oracle databases that could not be used when they grow and reach 2GB, may be a library problem too).

ext2 has a 2GB filesize limitation.

-mike

--------------------------------------------------------------------------

   /~\  The ASCII                       all that is gold does not glitter
   \ /  Ribbon Campaign                 not all those who wander are lost
    X   Against HTML                          -- jrr tolkien
   / \  Email!

radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd
--------------------------------------------------------------------------

#8 lore raw
Quoted 1 line

ext2 has a 2GB filesize limitation.

No it doesn't.

#9 lore raw
In reply to Jeff Chua
Quoted 1 line · Mike Eldridge

ext2 has a 2GB filesize limitation.

Where do you get that idea from. I have created files up to 4TB (sparse ones, of course) without problems. After that you start hitting bugs in the VFS and ext2 _code_, but you should be able to have up to 16TB files on a 4kB block ext2 fs.

Please stop spreading misinformation. Maybe there is a 2GB limitation in libc, or your tools, or in 2.2 ext2 _implementation_ (which is fixed if you apply the LFS patch for ext2), but no such limit in the design of ext2 itself.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#10 lore raw
In reply to Jeff Chua

To put it simply, ext2 does not have a 2GB filesize limitation anymore, (in newer distributions).

Quoted 30 lines · Mike Eldridge
Quoted 11 lines · Marcelo Borges Ribeiro

I have fat32 partition, but the problem isn´t the size of partition it is 8GB. The problem is that if you want to
create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2 to a reiserfs) to an available fat32 space, in my case the backup size is allways 2GB and when I tried to extract back I saw "unexpected end of file". So I thought it was that famous kernel limitation of 2GB under any kind of partition, but i was informed that fat has this limitation too. So the kernel may suport files bigger than 2GB (I really don´t know, I just know that in my case with fat32 it did not and I saw this too with some oracle databases that could not be used when they grow and reach 2GB, may be a library problem too).

ext2 has a 2GB filesize limitation.

(quoted diff) +0 -4
-mike
 
--------------------------------------------------------------------------
   /~\  The ASCII                       all that is gold does not glitter
   \ /  Ribbon Campaign                 not all those who wander are lost
    X   Against HTML                          -- jrr tolkien
   / \  Email!
 
          radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd
--------------------------------------------------------------------------
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#11 lore raw
In reply to Jeff Chua
Quoted 8 lines · Andreas Dilger
Quoted 1 line · Mike Eldridge

ext2 has a 2GB filesize limitation.

Where do you get that idea from. I have created files up to 4TB (sparse ones, of course) without problems. After that you start hitting bugs in the VFS and ext2 _code_, but you should be able to have up to 16TB files on a 4kB block ext2 fs.

Quoted 4 lines

Please stop spreading misinformation. Maybe there is a 2GB limitation in libc, or your tools, or in 2.2 ext2 _implementation_ (which is fixed if you apply the LFS patch for ext2), but no such limit in the design of ext2 itself.

oooh. i apologize. i remember reading in several different places that little tidbit i rattled off.

i stand (majorly) corrected.

-mike

--------------------------------------------------------------------------

   /~\  The ASCII                       all that is gold does not glitter
   \ /  Ribbon Campaign                 not all those who wander are lost
    X   Against HTML                          -- jrr tolkien
   / \  Email!

radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd
--------------------------------------------------------------------------

#12 lore raw
In reply to Jeff Chua
Quoted 1 line · Mike Eldridge

ext2 has a 2GB filesize limitation.

Mike has already realized this, but still...
I have used ext2 at Linux 1.2 with file sizes exceeding 2GB. The requirement was 64 bit machine in those days, I had Alpha. (Yes, that was VERY LONG AGO!)

It is very late (2.3/2.4) development that 32-bit machines can do 2TB files. Sparse files can indeed be larger up to 4G times filesystem block size, but ext2 is limited on one physical partition, and those are still limited on 1 or 2 terabyte range. (2G * 512 or 4G * 512.) (long story why that limit is still in there, mainly because nobody has had need to rework it.)

Limitations on EXT2 (indeed of all "indirect block indexing" schemes of SysV FS style) are a bit complicated to calculate.
B = block size in bytes (512 to 4096 bytes) addressable_blocks = (B/4)**3 + (B/4)**2 + ... max_offset = B * addressable_blocks

So, say: max_offset = B**4 / 64 + epsilon

That gives you magnitude. Say with 4k blocks (long story why that is presently maximum block-size) you can have up to 4 TB file size. (plus a bit over, see the math.)

The 2.5 series may change the underlying block-device layer so that it can handle larger block devices than 2TB - the 64 bit machines can handle them, of course, but 32-bit i386 is a bit limited...

Quoted 2 lines

-mike
radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd

/Matti Aarnio

#13 lore raw
In reply to Jeff Chua
Quoted 3 lines · Matti Aarnio

The 2.5 series may change the underlying block-device layer so that it can handle larger block devices than 2TB - the 64 bit machines can handle them, of course, but 32-bit i386 is a bit limited...

Definitely. Right now the page cache on x86 has a limit (per file) of 2^32 * PAGE_SIZE == 2^32 * 2^12 = 2^44 = 16TB. And I doubt that will change. x86 will be going 64bit in the next 2-3 years, at which point I don't see it paying to push 32bit code into the larger data sizes. Especially when we our limit is still 2 orders of magnitude larger then the largest disk manufactured today.

Now the file size limit on x86 can be increased a little by increasing the internal PAGE_SIZE for the page cache but that will only give us a bit or two which really isn't significant.

Eric