Skip to contentSkip to conversation

LFS Support for Sendfile

DISCUSSION Doug O'Neil 5 participants 5 messages Feb 28, 2002 – Mar 3, 2002

lore
Share & tools
lore
#0 lore raw

Hello group.

First time poster. If this isn't the right place for this then please point me in the right direction and accept my apology.

I'm using Linux 2.4.12 on a PIII

I've been playing with sendfile and noticed that it doesn't like 64 bit filesystems ( _FILE_OFFSET_BITS=64). Is there a patch or kernel revision out there that supports LFS for sendfile? Or is there some method I can't see to work around this.

I'm opening a descriptor on a disk partition. I don't need to support individual block transfers larger than 2 Gig., but would like to serve up partitions that are larger than 2Gig.

Thanks in advance

Doug

#1 lore raw
Quoted 6 lines · Doug O'Neil

Hello group.

First time poster. If this isn't the right place for this then please point me in the right direction and accept my apology.

I'm using Linux 2.4.12 on a PIII

That's pretty darn old. Can you try it with a modern kernel (2.4.18)?

-Doug

Signature · 2 lines
Let us cross over the river, and rest under the shade of the trees.
   --T. J. Jackson, 1863
#2 lore raw
In reply to Doug O'Neil
Quoted 9 lines · Doug McNaught
Quoted 6 lines · Doug O'Neil

Hello group.

First time poster. If this isn't the right place for this then please point me in the right direction and accept my apology.

I'm using Linux 2.4.12 on a PIII

That's pretty darn old. Can you try it with a modern kernel (2.4.18)?

Won't help.

The API (kernel syscall) as defined does not support LFS.

mm/filemap.c:
asmlinkage ssize_t sys_sendfile(int out_fd, int in_fd, off_t *offset,
size_t count)

The problem there is that "off_t *" thing. A loff_t in that spot would solve the problem, but that needs rewriting the thing with a new syscall
number. Trivial in itself, though. (And changeing the count to 64-bit
value is equally trivial at the same time.)

There lurks another problem in the filesystems, though.

All SysV UFS tradition type filesystems with indirect indexing of disk blocks will need more and more of metadata traversal to reach actual data blocks on very large files. It slows things down.

The "extent based" filesystems offer flatter performance, and while I can't determine if ReiserFS is exactly of that type, it too offers fast and flat performance.

Quoted 1 line

-Doug

/Matti Aarnio

#3 lore raw
In reply to Doug O'Neil

On Thu, Feb 28, 2002 at 10:03:25AM +0200, Matti Aarnio wrote:

The API (kernel syscall) as defined does not support LFS.

I wonder does it really need to? I mean, a loop calling sendfile for 2GB (or whatever) at a time is almost as good, if not better in some ways.

The "extent based" filesystems offer flatter performance, and while I can't determine if ReiserFS is exactly of that type, it too offers fast and flat performance.

Reiserfs (v3) isn't extent based but does perform pretty well. When I was messing large numbers of with (what at the time were) large files of 50GB or so, XFS proved to be very effective.

--cw

#4 lore raw
In reply to Doug O'Neil
Quoted 17 lines · Chris Wedgwood

On Thu, Feb 28, 2002 at 10:03:25AM +0200, Matti Aarnio wrote:

The API (kernel syscall) as defined does not support LFS.

I wonder does it really need to? I mean, a loop calling sendfile for 2GB (or whatever) at a time is almost as good, if not better in some ways.

The "extent based" filesystems offer flatter performance, and while I can't determine if ReiserFS is exactly of that type, it too offers fast and flat performance.

Reiserfs (v3) isn't extent based but does perform pretty well. When I was messing large numbers of with (what at the time were) large files of 50GB or so, XFS proved to be very effective.

Quoted 8 lines

--cw
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

extents are in reiserfs v4.0 (September ship date), and should offer much improved performance for large files.

Hans