fs block size and PAGE_CACHE_SIZE
Hi all,
I've got a question regarding filesystem block size (the arbitary internal fs block size) between the system's PAGE_CACHE_SIZE . Since the most common block size of fs in Linux is 4k, it is because it provides easy implementation of fs which directly matches the PAGE_CACHE_SIZE . As you know, new CPUs such as IA-64's can have a page size of 16k and above, I am wondering is it possible to implement an fs that has 16k block size on a 4k system which this fs can be used both on IA-32 and IA-64 systems. In other words, is it possible (or someone already done) to implement a file system that has 16k block size on a 4k paging system. Firstly, I think of working on a readpage() aop on a non aligned page no (in the middle of a block), how could I efficient populate one 16k block to page cache(4 pages) at one readpage() op? Since the fs driver could not really create a mapped page or telling whether a page (page no) is in the page cache or not. From my knowledge, one way of doing this
is to use the read_cache_page() callback to tell whether the page is in page cache by knowing wether the callback is triggered. If I simply call read_cache_page() to aligned page in readpage() aop of unaligned pages, even the aligned page is up_to_date afterwards, how would it possible for the readpage() process to return an up_to_date page (guaranteed) for unaligned pages? I thought of locking the page, but the problem is that readpage() is called with the page locked, and call read_cache_page() on a cached aligned page will not trigger an fs read (page cache hit of aligned page doesn't mean an up_to_date for unaligned page). Since my fs implementation would be very inefficient if doing a reading in the middle of a block (compression file systems) or the overhead of reading will be 4 times higher on small page cache size systems. I am pleased to hear any suggestions on such implementations. Thanks.
regards,
David Chow
Quoted 5 lines · David Chow
size of 16k and above, I am wondering is it possible to implement an fs that has 16k block size on a 4k system which this fs can be used both on IA-32 and IA-64 systems. In other words, is it possible (or someone already done) to implement a file system that has 16k block size on a 4k paging system.
The new NTFS driver, in 2.5 or as a patch to 2.4.20, supports all power of 2 block sizes between 512 bytes and 64kB.
Szaka
I don't know why there would be any issue having page size != block size, as long as one is a multiple of the other. Maybe you have a particular issue in mind?
I know one area where having a block larger than a page is a pain: When you allocate a new block in order to write just one page, you have to separately initialize the rest of the block.
Quoted 1 line
how could I efficient populate one 16k block to page cache(4 pages) at one
readpage() op?
Why would you want to? If someone wants to access those other 3 pages, they'll have page faults of their own.
Quoted 8 lines · Bryan Henderson
Quoted 3 lines
how could I efficient populate one 16k block to page cache(4 pages) at one
readpage()op?Why would you want to? If someone wants to access those other 3 pages, they'll have page faults of their own.
In an uncompressed filesystem this is not a problem because the additional page faults (read_page), will read the pages out of the block cache. However, for a compressed filesystem, the 16K block would have to be (re-)decompressed evey time, because the block is compressed in the block cache. It is much more efficient to push the extra decompressed pages into the page cache.
Regards
Phillip Lougher
Quoted 12 lines · Bryan Henderson
I don't know why there would be any issue having page size != block size, as long as one is a multiple of the other. Maybe you have a particular issue in mind?
I know one area where having a block larger than a page is a pain: When you allocate a new block in order to write just one page, you have to separately initialize the rest of the block.
Quoted 9 lines
Quoted 3 lines
how could I efficient populate one 16k block to page cache(4 pages) at one
readpage()op?Why would you want to? If someone wants to access those other 3 pages, they'll have page faults of their own.
Yes, for uncompressed file systems it doesn't really matter. If I am writing a file system that supports 16k blocks compression, I have to spend 3 extra (12k extra) compression work for only one 4k data. Since there is no way for me to read in the middle of the block. If the extra 12k data read from the block didn't put into page cache and set up-to-date, it will be wasted. That's why.
regards,
David Chow
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
" " == David Chow <[email protected]> writes:
> In other words, is it possible (or someone already done) to > implement a file system that has 16k block size on a 4k paging > system.
The NFS client does that. In 2.4.x, I had to invent a custom system for coalescing the pages, but in 2.5.x, Andrew added the readpages() op, and the ability to control page readahead via the backing_dev_info struct.
Cheers,
Trond
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
" " == David Chow <[email protected]> writes:
> How can you make sure there are no race conditions in > populating multiple page caches and making them up-to-date > afterwards? Since readpage only make sures you lock one page,
Have a browse through mm/readahead.c (in particular read_pages() and read_cache_pages()).
The page is is anonymous when it gets locked, and is only added to the page cache) if there is no existing page. The lock is only released by the filler.
> If readpages() is added for 2.5, it doesn't mean read page > don't have to implement readpage(), right?
Right. You still have to implement readpage.
Cheers,
Trond
Quoted 14 lines · Trond Myklebust
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
" " == David Chow <[email protected]> writes:
How can you make sure there are no race conditions in populating multiple page caches and making them up-to-date afterwards? Since readpage only make sures you lock one page,
Have a browse through
mm/readahead.c(in particularread_pages()andread_cache_pages()).
The page is is anonymous when it gets locked, and is only added to the page cache) if there is no existing page. The lock is only released by the filler.
Quoted 5 lines
Quoted 2 lines
If
readpages()is added for 2.5, it doesn't mean read page don't have to implementreadpage(), right?Right. You still have to implement readpage.
So it is impossible to do under 2.4? Since all the design of VFS and page cache assumes the underlying fs uses a 4k page or something equivalent less to the size of a PAGE_CACHE_SIZE as the largest unit of a cache. If you read Phillip Lougher's mail which points to the squash fs (read-only), plus the problem of sleeping which really impossible to implement a read/write fs on smp systems (which I suppose to, and the current fs already supported). If it is impossible, I don't bother to think though.
regards,
David Chow
Quoted 2 lines
Quoted 16 lines · David Chow
Quoted 10 lines · Trond Myklebust
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
" " == David Chow <[email protected]> writes:
How can you make sure there are no race conditions in populating multiple page caches and making them up-to-date afterwards? Since readpage only make sures you lock one page,
Have a browse through
mm/readahead.c(in particularread_pages()andread_cache_pages()).
The page is is anonymous when it gets locked, and is only added to the page cache) if there is no existing page. The lock is only released by the filler.Quoted 5 lines
Quoted 2 lines
If
readpages()is added for 2.5, it doesn't mean read page don't have to implementreadpage(), right?Right. You still have to implement readpage.
Quoted 8 lines
So it is impossible to do under 2.4? Since all the design of VFS and page cache assumes the underlying fs uses a 4k page or something equivalent less to the size of a
PAGE_CACHE_SIZEas the largest unit of a cache. If you read Phillip Lougher's mail which points to the squash fs (read-only), plus the problem of sleeping which really impossible to implement a read/write fs on smp systems (which I suppose to, and the current fs already supported). If it is impossible, I don't bother to think though.
The new ntfs driver reads several (compressed data) pages transparently and at once using readpage and it works just fine both smp and preemption both on 2.4 and 2.5. Basically I see it as intelligent readahead. You can see the 2.5 code in the vanilla kernel and the 2.4 code by downloading the 2.4 kernel patch from:
http://linux-ntfs.sf.net/downloads.html
Or by pulling from our BK repository:
bk pull bk://linux-ntfs.bkbits.net/ntfs-2.4
You can also view the BK tree with your webbrowser:
http://linux-ntfs.bkbits.net:8080/ntfs-2.4/src?nav=index.html
You want to look at the files:
fs/ntfs/aops.c and fs/ntfs/compress.c.
Best regards,
Anton
Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
Quoted 16 lines · David Chow
Quoted 14 lines · Trond Myklebust
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
" " == David Chow <[email protected]> writes:
How can you make sure there are no race conditions in populating multiple page caches and making them up-to-date afterwards? Since readpage only make sures you lock one page,
Have a browse through
mm/readahead.c(in particularread_pages()andread_cache_pages()).
The page is is anonymous when it gets locked, and is only added to the page cache) if there is no existing page. The lock is only released by the filler.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
If
readpages()is added for 2.5, it doesn't mean read page don't have to implementreadpage(), right?Right. You still have to implement readpage.
So it is impossible to do under 2.4? Since all the design of VFS and page cache assumes the underlying fs uses a 4k page or something equivalent less to the size of a
PAGE_CACHE_SIZEas the largest unit of a cache. If you read Phillip Lougher's mail which points to the squash fs (read-only), plus the problem of sleeping which really impossible to implement a read/write fs on smp systems (which I suppose to, and the current fs already supported). If it is impossible, I don't bother to think though.
Hi,
I think you slightly mis-understood my post. There is no problem doing this with SMP or pre-emptive kernels.
What I did say was to point out it was better to use grab_page_nowait() rather than read_cache_page. The problem with read_cache_page is not to with smp or pre-emptive kernels, it is to do with the fact read_cache_page tries to grab the lock of the page it is called to fill. If the page is already locked by another process, it will sleep on the lock. Grab_page_nowait() doesn't wait if the page is locked.
Regards
Phillip Lougher
Quoted 12 lines
regards,
David ChowQuoted 2 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines
Quoted 3 lines
So it is impossible to do under 2.4? Since all the design of VFS and page cache assumes the underlying fs uses a 4k page or something equivalent less to the size of a
PAGE_CACHE_SIZEas the largest unit of a cache. If you read Phillip Lougher's mail which points to the squash fs (read-only), plus the problem of sleeping which really impossible to implement a read/write fs on smp systems (which I suppose to, and the current fs already supported). If it is impossible, I don't bother to think though.Hi,
Quoted 8 lines
I think you slightly mis-understood my post. There is no problem doing this with SMP or pre-emptive kernels.
What I did say was to point out it was better to use
grab_page_nowait()rather thanread_cache_page. The problem withread_cache_pageis not to with smp or pre-emptive kernels, it is to do with the factread_cache_pagetries to grab the lock of the page it is called to fill. If the page is already locked by another process, it will sleep on the lock.Grab_page_nowait()doesn't wait if the page is locked.Regards
Phillip Lougher
Thanks for your clarifications. But what to do if one page is locked out of 4 (a 16k block)? It is easy when readpage() so that I just fill the reset 12k with up-to-date data. But for write, it is likely I need something to compress, right? If I simply grab the page data without considering the lock, corruption will likely to occur. I should sleep and wait for the page lock since it might be never unlocked for some mmapped and locked pages in different vmarea. Have you consider this at all? This is where it comes different with rw fs.
regards,
David Chow
Quoted 3 lines · David Chow
Hi all,
I am wondering is it possible to implement an fs that has 16k block size on a 4k system which this fs can be used both on IA-32 and IA-64 systems. In other words, is it possible (or someone already done) to implement a file system that has 16k block size on a 4k paging system.
Both zisofs and squashfs (http://squashfs.sourceforge.net) use larger than 4K blocks. In the case of squashfs, it can uses blocks up to 32K.
Both of these are compressed read-only filesystems.
Quoted 1 line
Firstly, I think of working on a
readpage()aop on a non aligned page no (in the middle of a block), how could I efficient populate one 16k block to page cache(4 pages) at onereadpage()op? Since the fs driver could not really create a mapped page or telling whether a page (page no) is in the page cache or not. From my knowledge, one way of doing this is to use theread_cache_page()callback to tell whether the page is in page cache by knowing wether the callback is triggered. If I simply callread_cache_page()to aligned page inreadpage()aop of unaligned pages, even the aligned page isup_to_dateafterwards, how would it possible for thereadpage()process to return anup_to_datepage (guaranteed) for unaligned pages? I thought of locking the page, but the problem is thatreadpage()is called with the page locked, and callread_cache_page()on a cached aligned page will not trigger an fs read (page cache hit of aligned page doesn't mean anup_to_datefor unaligned page)
. Since my fs implementation would be very inefficient if doing a reading in the middle of a block (compression file systems) or the overhead of reading will be 4 times higher on small page cache size systems. I am pleased to hear any suggestions on such implementations. Thanks.
Quoted 1 line
Squashfs in the read_page routine pushes the extra pages (i.e. 3 extra 4K pages in the case of a 16K block) into the page cache using grab_page_nowait(), which returns a page with the appropriate mapping to be filled and marked uptodate. To avoid race conditions this routine will not sleep if the page is already locked.
Using read_cache_page() is dangerous, because it sleeps if the page you are trying to fill is already locked. When writing squashfs I hit the problem where read_page was being called almost simultaneously for different pages which were in the same (32K) block. The first invocation read the 32K block and locked it, then tried to push the pages into the page cache using read_cache_page(). However, read_cache_page slept when it tried to push the page that the second invocation had locked. The second invocation itself was sleeping holding that lock, whilst waiting for the overal 32K block lock .
Regards
Phillip Lougher
Quoted 11 lines
regards,
David Chow-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines · Phillip Lougher
Quoted 4 lines · David Chow
Hi all,
I am wondering is it possible to implement an fs that has 16k block
size on a 4k system which this fs can be used both on IA-32 and IA-64 systems. In other words, is it possible (or someone already done) to implement a file system that has 16k block size on a 4k paging system.
Quoted 7 lines
Both zisofs and squashfs (http://squashfs.sourceforge.net) use larger than 4K blocks. In the case of squashfs, it can uses blocks up to 32K. Both of these are compressed read-only filesystems.
Quoted 3 lines
Firstly, I think of working on a
readpage()aop on a non aligned page no (in the middle of a block), how could I efficient populate one 16k block to page cache(4 pages) at onereadpage()op? Since the fs driver could not really create a mapped page or telling whether a page (page no) is in the page cache or not. From my knowledge, one way of doing this is to use theread_cache_page()callback to tell whether the page is in page cache by knowing wether the callback is triggered. If I simply callread_cache_page()to aligned page inreadpage()aop of unaligned pages, even the aligned page isup_to_dateafterwards, how would it possible for thereadpage()process to return anup_to_datepage (guaranteed) for unaligned pages? I thought of locking the page, but the problem is thatreadpage()is called with the page locked, and callread_cache_page()on a cached aligned page will not trigger an fs read (page cache hit of aligned page doesn't mean anup_to_datefor unaligned page). Since my fs implementation would be very inefficient if doing a reading in the middle of a block (compression file systems) or the overhead of reading will be 4 times higher on small page cache size systems. I am pleased to hear any suggestions on such implementations. Thanks.Squashfs in the
read_pageroutine pushes the extra pages (i.e. 3 extra 4K pages in the case of a 16K block) into the page cache usinggrab_page_nowait(), which returns a page with the appropriate mapping to be filled and marked uptodate. To avoid race conditions this routine will not sleep if the page is already locked.
The case for sleep or no sleep only works on uni processor systems and non-preemptive kernel. Since the file system is suppose to work on smp systems which sleeping or not doesn't really helps. As you know, cases where simultaneous read to the same block really block my head.
Quoted 2 lines
Using
read_cache_page()is dangerous, because it sleeps if the page you are trying to fill is already locked. When writing squashfs I hit the problem whereread_pagewas being called almost simultaneously for different pages which were in the same (32K) block. The first invocation read the 32K block and locked it, then tried to push the pages into the page cache usingread_cache_page(). However,read_cache_pageslept when it tried to push the page that the second invocation had locked. The second invocation itself was sleeping holding that lock, whilst waiting for the overal 32K block lock .
Thanks for your pointers.
regards,
David Chow