btrfs: rework lzo_compress_pages() to make it subpage compatible
RFC PATCH 1/1 btrfs: rework lzo_compress_pages() to make it subpage compatible
There are several problems in lzo_compress_pages() preventing it from being subpage compatible:
- No page offset is calculated when reading from inode pages For subpage case, we could have @start which is not aligned to PAGE_SIZE.
Thus the destination where we read data from must take offset in page into consideration.
- The padding for segment header is bound to PAGE_SIZE
This means, for subpage case we can skip several corners where on x86 machines we need to add padding zeros.
The rework will:
- Update the comment to replace "page" with "sector"
- Introduce a new helper, copy_compressed_data_to_page(), to do the copy So that we don't need to bother page switches for both input and output.
Now in lzo_compress_pages() we only care about page switching for input, while in copy_compressed_data_to_page() we only care the page switching for output.
- Only one main cursor
For lzo_compress_pages() we use @cur_in as main curor. It will be the file offset we are currently at.
All other helper variables will be only declared inside the loop.
For copy_compressed_data_to_page() it's similar, we will have @cur_out at the main cursor, which records how many bytes are in the output.
- Get rid of kmap()/kunmap()
Instead of using __GFP_HIGHMEM and needs to do kmap()/kunmap(), just get rid of that GFP flag, so we can use page_address() and never bother the kmap()/kunmap() thing.
---
Reason for RFC:
The patch itself seems not only easier to read, but already passes fstests -g compress with "-o compress=lzo" tests.
Haven't yet run the full test suits with "-o compress=lzo", but so far so good.
So the main reason for RFC is, if the style is OK.
There are at least 4 functions need similar rework, and one of them is in high priority as it affects subpage read path for compressed data. (namely lzo_decompress_bio()).
While the lzo|zlib/zstd_compress_pages() functions are in low priority, as for subpage write support we will only support range which is PAGE_SIZE aligned, thus the existing functions can work without big problems.
Personally speaking I'm pretty happy with the refactor.
Even with my excessive comments style, we still reach net 0 for changed lines.
Thus any feedback on the style if appreciated.
--- fs/btrfs/lzo.c | 284 ++++++++++++++++++++++++------------------------- 1 file changed, 142 insertions(+), 142 deletions(-)
fs/btrfs/lzo.c +142 -142
@@ -14,6 +14,7 @@#include <linux/lzo.h> #include <linux/refcount.h> #include "compression.h" +#include "ctree.h" #define LZO_LEN 4
@@ -31,19 +32,19 @@* payload. * One regular LZO compressed extent can have one or more segments. * For inlined LZO compressed extent, only one segment is allowed. -* One segment represents at most one page of uncompressed data. +* One segment represents at most one sector of uncompressed data. * * 2.1 Segment header * Fixed size. LZO_LEN (4) bytes long, LE32. * Records the total size of the segment (not including the header). -* Segment header never crosses page boundary, thus it's possible to -* have at most 3 padding zeros at the end of the page. +* Segment header never crosses sector boundary, thus it's possible to +* have at most 3 padding zeros at the end of the sector. * * 2.2 Data Payload -* Variable size. Size up limit should be lzo1x_worst_compress(PAGE_SIZE) -* which is 4419 for a 4KiB page. +* Variable size. Size up limit should be lzo1x_worst_compress(sectorsize) +* which is 4419 for a 4KiB sectorsize. * -* Example: +* Example with 4K sectorsize: * Page 1: * 0 0x2 0x4 0x6 0x8 0xa 0xc 0xe 0x10 * 0x0000 | Header | SegHdr 01 | Data payload 01 ... |
@@ -111,170 +112,169 @@ static inline size_t read_compress_length(const char *buf)return le32_to_cpu (dlen); } +/* +* Will do: +* +* - Write a segment header into the destination +* - Copy the compressed buffer into the destination +* - Make sure we have enough space in the last sector to fit a segment header +* If not, we will pad at most (LZO_LEN (4)) - 1 bytes of zeros. +* +* Will allocate new pages when needed. +*/ +static int copy_compressed_data_to_page (char *compressed_data, +size_t compressed_size, +struct page **out_pages, +u32 *cur_out, +const u32 sectorsize) +{ +u32 sector_bytes_left; +u32 orig_out; +struct page *cur_page; + +/* +* We never allow a segment header crossing sector boundary, previous +* run should ensure we have enough space left inside the sector. +*/ +ASSERT ((*cur_out / sectorsize) == + (*cur_out +LZO_LEN -1 ) / sectorsize); + + cur_page = out_pages[*cur_out /PAGE_SIZE ]; +/* Allocate a new page */ +if (!cur_page) { + cur_page =alloc_page (GFP_NOFS ); +if (!cur_page) +return -ENOMEM ; + out_pages[*cur_out /PAGE_SIZE ] = cur_page; + } + +write_compress_length (page_address (cur_page) +offset_in_page (*cur_out), + compressed_size); + + *cur_out +=LZO_LEN ; + orig_out = *cur_out; +/* *cur_out is increased, let the main loop to grab a proper page */ + cur_page =NULL ; + +/* Copy compressed data */ +while (*cur_out - orig_out < compressed_size) { +u32 copy_len =min_t (u32, sectorsize - *cur_out % sectorsize, + orig_out + compressed_size - *cur_out); + +/* Grab a page or allocate a new one */ +if (!cur_page) { + cur_page = out_pages[*cur_out /PAGE_SIZE ]; +if (!cur_page) { + cur_page =alloc_page (GFP_NOFS ); +if (!cur_page) +return -ENOMEM ; + out_pages[*cur_out /PAGE_SIZE ] = cur_page; + } + } + +memcpy (page_address (cur_page) +offset_in_page (*cur_out), + compressed_data + *cur_out - orig_out, copy_len); + + *cur_out += copy_len; + +/* If we reached page boudnary, go to next page */ +if (IS_ALIGNED (*cur_out,PAGE_SIZE )) { +/* Let next iteration to grab a page */ + cur_page =NULL ; + } + } + +/* +* Check if we can fit the next segment header into the remaining space +* of the sector. +*/ + sector_bytes_left =round_up (*cur_out, sectorsize) - *cur_out; +if (sector_bytes_left >=LZO_LEN ) +return 0 ; + +/* The remaining size is not enough, pad it with zeros */ +memset (page_address (cur_page) +offset_in_page (*cur_out),0 , + sector_bytes_left); + *cur_out += sector_bytes_left; +return 0 ; +} +int lzo_compress_pages (struct list_head *ws,struct address_space *mapping,u64 start,struct page **pages,unsigned long *out_pages,unsigned long *total_in,unsigned long *total_out) { struct workspace *workspace = list_entry(ws, struct workspace, list); +const u32 sectorsize =btrfs_sb (mapping->host->i_sb)->sectorsize; +struct page *page_in =NULL ;int ret =0 ; -char *data_in; -char *cpage_out, *sizes_ptr; -int nr_pages =0 ; -struct page *in_page =NULL ; -struct page *out_page =NULL ; -unsigned long bytes_left; -unsigned long len = *total_out; -unsigned long nr_dest_pages = *out_pages; -const unsigned long max_out = nr_dest_pages *PAGE_SIZE ; -size_t in_len; -size_t out_len; -char *buf; -unsigned long tot_in =0 ; -unsigned long tot_out =0 ; -unsigned long pg_bytes_left; -unsigned long out_offset; -unsigned long bytes; +u64 cur_in = start;/* Points to the file offset of input data */ +u32 cur_out =0 ;/* Points to the current output byte */ +u32 len = *total_out; *out_pages =0 ; *total_out =0 ; *total_in =0 ; - in_page =find_get_page (mapping, start >>PAGE_SHIFT ); - data_in =kmap (in_page); -/* -* store the size of all chunks of compressed data in -* the first 4 bytes +* Skip the header for now, we will later come back and write the total +* compressed size */ - out_page =alloc_page (GFP_NOFS | __GFP_HIGHMEM); -if (out_page ==NULL ) { - ret = -ENOMEM ; - goto out; - } - cpage_out =kmap (out_page); - out_offset =LZO_LEN ; - tot_out =LZO_LEN ; - pages[0 ] = out_page; - nr_pages =1 ; - pg_bytes_left =PAGE_SIZE -LZO_LEN ; - -/* compress at most one page of data each time */ - in_len =min (len,PAGE_SIZE ); -while (tot_in < len) { - ret =lzo1x_1_compress (data_in, in_len, workspace->cbuf, - &out_len, workspace->mem); -if (ret !=LZO_E_OK ) { -pr_debug ("BTRFS: lzo in loop returned %d\n" , - ret); + cur_out +=LZO_LEN ; +while (cur_in < start + len) { +u32 sector_off = (cur_in - start) % sectorsize; +u32 in_len; +size_t out_len; + +/* Get the input page first */ +if (!page_in) { + page_in =find_get_page (mapping, cur_in >>PAGE_SHIFT ); +ASSERT (page_in); + } + +/* Compress at most one sector of data each time */ + in_len =min_t (u32, start + len - cur_in, + sectorsize - sector_off); +ASSERT (in_len); + ret =lzo1x_1_compress (page_address (page_in) + +offset_in_page (cur_in), in_len, + workspace->cbuf, &out_len, + workspace->mem); +if (ret <0 ) { +pr_debug ("BTRFS: lzo in loop returned %d\n" , ret); ret = -EIO ; goto out; } -/* store the size of this chunk of compressed data */ -write_compress_length (cpage_out + out_offset, out_len); - tot_out +=LZO_LEN ; - out_offset +=LZO_LEN ; - pg_bytes_left -=LZO_LEN ; - - tot_in += in_len; - tot_out += out_len; - -/* copy bytes from the working buffer into the pages */ - buf = workspace->cbuf; -while (out_len) { - bytes = min_t(unsigned long, pg_bytes_left, out_len); - -memcpy (cpage_out + out_offset, buf, bytes); - - out_len -= bytes; - pg_bytes_left -= bytes; - buf += bytes; - out_offset += bytes; - -/* -* we need another page for writing out. -* -* Note if there's less than 4 bytes left, we just -* skip to a new page. -*/ -if ((out_len ==0 && pg_bytes_left <LZO_LEN ) || - pg_bytes_left ==0 ) { -if (pg_bytes_left) { -memset (cpage_out + out_offset,0 , - pg_bytes_left); - tot_out += pg_bytes_left; - } - -/* we're done, don't allocate new page */ -if (out_len ==0 && tot_in >= len) -break ; - -kunmap (out_page); -if (nr_pages == nr_dest_pages) { - out_page =NULL ; - ret = -E2BIG ; - goto out; - } - - out_page =alloc_page (GFP_NOFS | __GFP_HIGHMEM); -if (out_page ==NULL ) { - ret = -ENOMEM ; - goto out; - } - cpage_out =kmap (out_page); - pages[nr_pages++] = out_page; + ret =copy_compressed_data_to_page (workspace->cbuf, out_len, + pages, &cur_out, sectorsize); +if (ret <0 ) + goto out; - pg_bytes_left =PAGE_SIZE ; - out_offset =0 ; - } - } + cur_in += in_len; -/* we're making it bigger, give up */ -if (tot_in >8192 && tot_in < tot_out) { +/* +* Check if we're making it bigger after two sectors. +* And if we're making it bigger, give up. +*/ +if (cur_in - start > sectorsize *2 && + cur_in - start < cur_out) { ret = -E2BIG ; goto out; } -/* we're all done */ -if (tot_in >= len) -break ; - -if (tot_out > max_out) -break ; - - bytes_left = len - tot_in; -kunmap (in_page); -put_page (in_page); - - start +=PAGE_SIZE ; - in_page =find_get_page (mapping, start >>PAGE_SHIFT ); - data_in =kmap (in_page); - in_len =min (bytes_left,PAGE_SIZE ); - } - -if (tot_out >= tot_in) { - ret = -E2BIG ; - goto out; +/* Check if we have reached page boundary */ +if (IS_ALIGNED (cur_in,PAGE_SIZE )) + page_in =NULL ; } -/* store the size of all chunks of compressed data */ - sizes_ptr =kmap_local_page (pages[0 ]); -write_compress_length (sizes_ptr, tot_out); -kunmap_local (sizes_ptr); +/* Store the size of all chunks of compressed data */ +write_compress_length (page_address (pages[0 ]), cur_out); ret =0 ; - *total_out = tot_out; - *total_in = tot_in; + *total_out = cur_out; + *total_in = cur_in - start; out: - *out_pages = nr_pages; -if (out_page) -kunmap (out_page); - -if (in_page) { -kunmap (in_page); -put_page (in_page); - } - + *out_pages =DIV_ROUND_UP (cur_out,PAGE_SIZE );return ret; }
Signature · 1 line
2.32.0
kernel test robot
build bot[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Qu,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on kdave/for-next]
[also build test ERROR on v5.13 next-20210629]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Qu-Wenruo/btrfs-rework-lzo_compress_pages-to-make-it-subpage-compatible/20210629-202452 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next config: i386-randconfig-r031-20210628 (attached as .config) compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/adeef0e74afe086c1c81296f5bd36a382f8af3ee
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Qu-Wenruo/btrfs-rework-lzo_compress_pages-to-make-it-subpage-compatible/20210629-202452 git checkout adeef0e74afe086c1c81296f5bd36a382f8af3ee
# save the attached .config to linux build tree
mkdir build_dir
make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
ld: fs/btrfs/lzo.o: in function `lzo_compress_pages':
>> lzo.c:(.text+0x1a3): undefined reference to `__umoddi3' >> ld: lzo.c:(.text+0x387): undefined reference to `__umoddi3'
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Quoted 170 lines · Qu Wenruo
There are several problems in
lzo_compress_pages()preventing it from being subpage compatible:- No page offset is calculated when reading from inode pages For subpage case, we could have @start which is not aligned to
PAGE_SIZE.Thus the destination where we read data from must take offset in page into consideration.
- The padding for segment header is bound to
PAGE_SIZE
This means, for subpage case we can skip several corners where on x86 machines we need to add padding zeros.The rework will:
- Update the comment to replace "page" with "sector"
- Introduce a new helper,
copy_compressed_data_to_page(), to do the copy So that we don't need to bother page switches for both input and output.Now in
lzo_compress_pages()we only care about page switching for input, while incopy_compressed_data_to_page()we only care the page switching for output.- Only one main cursor
Forlzo_compress_pages()we use @cur_in as main curor. It will be the file offset we are currently at.All other helper variables will be only declared inside the loop.
For
copy_compressed_data_to_page()it's similar, we will have @cur_out at the main cursor, which records how many bytes are in the output.- Get rid of kmap()/kunmap()
Instead of using__GFP_HIGHMEMand needs to do kmap()/kunmap(), just get rid of that GFP flag, so we can usepage_address()and never bother the kmap()/kunmap() thing.Signed-off-by Qu Wenruo <[email protected]>---
Reason for RFC:The patch itself seems not only easier to read, but already passes fstests -g compress with "-o compress=lzo" tests.
Haven't yet run the full test suits with "-o compress=lzo", but so far so good.
So the main reason for RFC is, if the style is OK.
There are at least 4 functions need similar rework, and one of them is in high priority as it affects subpage read path for compressed data. (namely
lzo_decompress_bio()).While the lzo|zlib/zstd_compress_pages() functions are in low priority, as for subpage write support we will only support range which is
PAGE_SIZEaligned, thus the existing functions can work without big problems.Personally speaking I'm pretty happy with the refactor.
Even with my excessive comments style, we still reach net 0 for changed lines.
Thus any feedback on the style if appreciated.
--- fs/btrfs/lzo.c | 284 ++++++++++++++++++++++++------------------------- 1 file changed, 142 insertions(+), 142 deletions(-)fs/btrfs/lzo.c +64
-6@@ -14,6 +14,7 @@#include <linux/lzo.h> #include <linux/refcount.h> #include "compression.h" +#include "ctree.h" #define LZO_LEN 4@@ -31,19 +32,19 @@* payload. * One regular LZO compressed extent can have one or more segments. * For inlined LZO compressed extent, only one segment is allowed. -* One segment represents at most one page of uncompressed data. +* One segment represents at most one sector of uncompressed data. * * 2.1 Segment header * Fixed size. LZO_LEN (4) bytes long, LE32. * Records the total size of the segment (not including the header). -* Segment header never crosses page boundary, thus it's possible to -* have at most 3 padding zeros at the end of the page. +* Segment header never crosses sector boundary, thus it's possible to +* have at most 3 padding zeros at the end of the sector. * * 2.2 Data Payload -* Variable size. Size up limit should be lzo1x_worst_compress(PAGE_SIZE) -* which is 4419 for a 4KiB page. +* Variable size. Size up limit should be lzo1x_worst_compress(sectorsize) +* which is 4419 for a 4KiB sectorsize. * -* Example: +* Example with 4K sectorsize: * Page 1: * 0 0x2 0x4 0x6 0x8 0xa 0xc 0xe 0x10 * 0x0000 | Header | SegHdr 01 | Data payload 01 ... | @@ -111,170 +112,169 @@ static inline size_t read_compress_length(const char *buf)return le32_to_cpu (dlen); } +/* +* Will do: +* +* - Write a segment header into the destination +* - Copy the compressed buffer into the destination +* - Make sure we have enough space in the last sector to fit a segment header +* If not, we will pad at most (LZO_LEN (4)) - 1 bytes of zeros. +* +* Will allocate new pages when needed. +*/ +static int copy_compressed_data_to_page (char *compressed_data, +size_t compressed_size, +struct page **out_pages, +u32 *cur_out, +const u32 sectorsize) +{ +u32 sector_bytes_left; +u32 orig_out; +struct page *cur_page; + +/* +* We never allow a segment header crossing sector boundary, previous +* run should ensure we have enough space left inside the sector. +*/ +ASSERT ((*cur_out / sectorsize) == + (*cur_out +LZO_LEN -1 ) / sectorsize); + + cur_page = out_pages[*cur_out /PAGE_SIZE ]; +/* Allocate a new page */ +if (!cur_page) { + cur_page =alloc_page (GFP_NOFS ); +if (!cur_page) +return -ENOMEM ; + out_pages[*cur_out /PAGE_SIZE ] = cur_page; + } + +write_compress_length (page_address (cur_page) +offset_in_page (*cur_out), + compressed_size); + + *cur_out +=LZO_LEN ; + orig_out = *cur_out; +/* *cur_out is increased, let the main loop to grab a proper page */ + cur_page =NULL ; + +/* Copy compressed data */ +while (*cur_out - orig_out < compressed_size) { +u32 copy_len =min_t (u32, sectorsize - *cur_out % sectorsize, + orig_out + compressed_size - *cur_out); + +/* Grab a page or allocate a new one */ +if (!cur_page) { + cur_page = out_pages[*cur_out /PAGE_SIZE ]; +if (!cur_page) { + cur_page =alloc_page (GFP_NOFS ); +if (!cur_page) +return -ENOMEM ; + out_pages[*cur_out /PAGE_SIZE ] = cur_page;
fs/btrfs/lzo.c static inline size_t read_compress_length(const char *buf) · line 181
Quoted 10 lines
fs/btrfs/lzo.c +10
-0+ } + } + +memcpy (page_address (cur_page) +offset_in_page (*cur_out), + compressed_data + *cur_out - orig_out, copy_len); + + *cur_out += copy_len; + +/* If we reached page boudnary, go to next page */ +if (IS_ALIGNED (*cur_out,PAGE_SIZE )) {
Here missing a put_page() and will cause memory leakage.
Already fixed in my github compression branch.
Thanks,
Qu
Quoted 236 lines
fs/btrfs/lzo.c +68
-136+/* Let next iteration to grab a page */ + cur_page =NULL ; + } + } + +/* +* Check if we can fit the next segment header into the remaining space +* of the sector. +*/ + sector_bytes_left =round_up (*cur_out, sectorsize) - *cur_out; +if (sector_bytes_left >=LZO_LEN ) +return 0 ; + +/* The remaining size is not enough, pad it with zeros */ +memset (page_address (cur_page) +offset_in_page (*cur_out),0 , + sector_bytes_left); + *cur_out += sector_bytes_left; +return 0 ; +} +int lzo_compress_pages (struct list_head *ws,struct address_space *mapping,u64 start,struct page **pages,unsigned long *out_pages,unsigned long *total_in,unsigned long *total_out) { struct workspace *workspace = list_entry(ws, struct workspace, list); +const u32 sectorsize =btrfs_sb (mapping->host->i_sb)->sectorsize; +struct page *page_in =NULL ;int ret =0 ; -char *data_in; -char *cpage_out, *sizes_ptr; -int nr_pages =0 ; -struct page *in_page =NULL ; -struct page *out_page =NULL ; -unsigned long bytes_left; -unsigned long len = *total_out; -unsigned long nr_dest_pages = *out_pages; -const unsigned long max_out = nr_dest_pages *PAGE_SIZE ; -size_t in_len; -size_t out_len; -char *buf; -unsigned long tot_in =0 ; -unsigned long tot_out =0 ; -unsigned long pg_bytes_left; -unsigned long out_offset; -unsigned long bytes; +u64 cur_in = start;/* Points to the file offset of input data */ +u32 cur_out =0 ;/* Points to the current output byte */ +u32 len = *total_out; *out_pages =0 ; *total_out =0 ; *total_in =0 ; - in_page =find_get_page (mapping, start >>PAGE_SHIFT ); - data_in =kmap (in_page); -/* -* store the size of all chunks of compressed data in -* the first 4 bytes +* Skip the header for now, we will later come back and write the total +* compressed size */ - out_page =alloc_page (GFP_NOFS | __GFP_HIGHMEM); -if (out_page ==NULL ) { - ret = -ENOMEM ; - goto out; - } - cpage_out =kmap (out_page); - out_offset =LZO_LEN ; - tot_out =LZO_LEN ; - pages[0 ] = out_page; - nr_pages =1 ; - pg_bytes_left =PAGE_SIZE -LZO_LEN ; - -/* compress at most one page of data each time */ - in_len =min (len,PAGE_SIZE ); -while (tot_in < len) { - ret =lzo1x_1_compress (data_in, in_len, workspace->cbuf, - &out_len, workspace->mem); -if (ret !=LZO_E_OK ) { -pr_debug ("BTRFS: lzo in loop returned %d\n" , - ret); + cur_out +=LZO_LEN ; +while (cur_in < start + len) { +u32 sector_off = (cur_in - start) % sectorsize; +u32 in_len; +size_t out_len; + +/* Get the input page first */ +if (!page_in) { + page_in =find_get_page (mapping, cur_in >>PAGE_SHIFT ); +ASSERT (page_in); + } + +/* Compress at most one sector of data each time */ + in_len =min_t (u32, start + len - cur_in, + sectorsize - sector_off); +ASSERT (in_len); + ret =lzo1x_1_compress (page_address (page_in) + +offset_in_page (cur_in), in_len, + workspace->cbuf, &out_len, + workspace->mem); +if (ret <0 ) { +pr_debug ("BTRFS: lzo in loop returned %d\n" , ret); ret = -EIO ; goto out; } -/* store the size of this chunk of compressed data */ -write_compress_length (cpage_out + out_offset, out_len); - tot_out +=LZO_LEN ; - out_offset +=LZO_LEN ; - pg_bytes_left -=LZO_LEN ; - - tot_in += in_len; - tot_out += out_len; - -/* copy bytes from the working buffer into the pages */ - buf = workspace->cbuf; -while (out_len) { - bytes = min_t(unsigned long, pg_bytes_left, out_len); - -memcpy (cpage_out + out_offset, buf, bytes); - - out_len -= bytes; - pg_bytes_left -= bytes; - buf += bytes; - out_offset += bytes; - -/* -* we need another page for writing out. -* -* Note if there's less than 4 bytes left, we just -* skip to a new page. -*/ -if ((out_len ==0 && pg_bytes_left <LZO_LEN ) || - pg_bytes_left ==0 ) { -if (pg_bytes_left) { -memset (cpage_out + out_offset,0 , - pg_bytes_left); - tot_out += pg_bytes_left; - } - -/* we're done, don't allocate new page */ -if (out_len ==0 && tot_in >= len) -break ; - -kunmap (out_page); -if (nr_pages == nr_dest_pages) { - out_page =NULL ; - ret = -E2BIG ; - goto out; - } - - out_page =alloc_page (GFP_NOFS | __GFP_HIGHMEM); -if (out_page ==NULL ) { - ret = -ENOMEM ; - goto out; - } - cpage_out =kmap (out_page); - pages[nr_pages++] = out_page; + ret =copy_compressed_data_to_page (workspace->cbuf, out_len, + pages, &cur_out, sectorsize); +if (ret <0 ) + goto out; - pg_bytes_left =PAGE_SIZE ; - out_offset =0 ; - } - } + cur_in += in_len; -/* we're making it bigger, give up */ -if (tot_in >8192 && tot_in < tot_out) { +/* +* Check if we're making it bigger after two sectors. +* And if we're making it bigger, give up. +*/ +if (cur_in - start > sectorsize *2 && + cur_in - start < cur_out) { ret = -E2BIG ; goto out; } -/* we're all done */ -if (tot_in >= len) -break ; - -if (tot_out > max_out) -break ; - - bytes_left = len - tot_in; -kunmap (in_page); -put_page (in_page); - - start +=PAGE_SIZE ; - in_page =find_get_page (mapping, start >>PAGE_SHIFT ); - data_in =kmap (in_page); - in_len =min (bytes_left,PAGE_SIZE ); - } - -if (tot_out >= tot_in) { - ret = -E2BIG ; - goto out; +/* Check if we have reached page boundary */ +if (IS_ALIGNED (cur_in,PAGE_SIZE )) + page_in =NULL ; } -/* store the size of all chunks of compressed data */ - sizes_ptr =kmap_local_page (pages[0 ]); -write_compress_length (sizes_ptr, tot_out); -kunmap_local (sizes_ptr); +/* Store the size of all chunks of compressed data */ +write_compress_length (page_address (pages[0 ]), cur_out); ret =0 ; - *total_out = tot_out; - *total_in = tot_in; + *total_out = cur_out; + *total_in = cur_in - start; out: - *out_pages = nr_pages; -if (out_page) -kunmap (out_page); - -if (in_page) { -kunmap (in_page); -put_page (in_page); - } - + *out_pages =DIV_ROUND_UP (cur_out,PAGE_SIZE );return ret; }