the max size of block device on 32bit os,when using do_generic_file_read() proceed.

DISCUSSION majianpeng 2 participants 4 messages May 24, 2012 – May 28, 2012

Subsystems (2)

Memory management VFS

lore

Share & tools

lore

majianpeng

author

2012-05-24 13:37 UTC #0 lore raw

Hi all:
I readed a raid5,which size 30T.OS is RHEL6 32bit.
I reaed the raid5(as a whole,not parted) and found read address which not i wanted.
So I tested the newest kernel code,the problem is still.
I review the code, in function do_generic_file_read()

index = *ppos >> PAGE_CACHE_SHIFT;
index is u32.and *ppos is long long.
So when *ppos is larger than 0xFFFF FFFF * PAGE_CACHE_SHIFT(16T Byte),then the index is error.

I wonder this .In 32bit os ,block devices size do not large then 16T,in other words, if block devices larger than 16T,must parted.

Thanks all.

--------------
majianpeng
2012-05-24

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Hugh Dickins

2012-05-26 21:23 UTC #1 lore raw

Quoted 11 lines · majianpeng

Hi all:
I readed a raid5,which size 30T.OS is RHEL6 32bit.
I reaed the raid5(as a whole,not parted) and found read address which not i wanted.
So I tested the newest kernel code,the problem is still.
I review the code, in function do_generic_file_read()
index = *ppos >> PAGE_CACHE_SHIFT;
index is u32.and *ppos is long long.
So when *ppos is larger than 0xFFFF FFFF * PAGE_CACHE_SHIFT(16T Byte),then the index is error.
I wonder this .In 32bit os ,block devices size do not large then 16T,in other words, if block devices larger than 16T,must parted.

I am not surprised that the page cache limitation prevents you from reading the whole device with a 32-bit kernel. See MAX_LFS_FILESIZE in include/linux/fs.h. Our answer to that is just to use a 64-bit kernel.

#if BITS_PER_LONG==32
#define MAX_LFS_FILESIZE (((u64)PAGE_CACHE_SIZE << (BITS_PER_LONG-1))-1) #elif BITS_PER_LONG==64
#define MAX_LFS_FILESIZE 0x7fffffffffffffffUL
#endif

But I am a little surprised that you get as far as 16TiB (with 4k page):
I would have expected you to be stopped just before 8TiB (although I suspect that the limitation to 8TiB rather than 16TiB is unnecessary).

And if I understand you correctly, read() or pread() gave you no error at those large offsets, but supplied data from the low offset instead?

That does surprise me - have we missed a check there?

Hugh

majianpeng

author

2012-05-28 06:25 UTC #2 lore raw

Re: Re: the max size of block device on 32bit os,when usingdo_generic_file_read() proceed.

In reply to majianpeng

U29ycnkgZm9yIGxhdGUgdG8gcmVwbHkuSSByZXZpZXdlZCB0aGUgY29kZSBhZ2FpbiBhbmQgZm91 bmQgc29tZSBwcm9ibGVhbS4NCkkgY3JlYXRlZCBhIHNvZnQtcmFpZCBhbmQgdGhlIHNpemUgd2Fz IGxhcmdlciB0aGFuIDE2VC4NClRoZSBvcyBpcyB1YnVudHUgMTIuMDQgMzJiaXQgeDg2Lg0KVGhl IHVkZXYgY3JlYXRlIHRoZSBibG9jayBub2RlIGlzIC9kZXYgZGlyKGFzIHRtcGZzKS4NCkFuZCBJ IHJlYWRlZCB0aGUgdG1wZnMgY29kZSA6DQppbiBtbS9zaG1lbS5jOnNobWVtX2ZpbGxfc3VwZXIo KQ0KPnNiLT5zX21heGJ5dGVzID0gTUFYX0xGU19GSUxFU0laRTsNCkluIG15IGNvbXB1dGVyLCBN QVhfTEZTX0ZJTEVTWkUgaXMgZXF1YWwgOFQgLTEuDQpCdXQgdGhlIHJlYWQgY29kZToNCmdlbmVy aWNfZmlsZV9haW9fcmVhZC0tPmRvX2dlbmVyaWNfZmlsZV9yZWFkW25vdCB1c2UgZGlyZWN0IGZs YWcNCkluIGZ1bmN0aW9uOmRvX2dlbmVyaWNfZmlsZV9yZWFkKCk6DQo+aW5kZXggPSAqcHBvcyA+ PiBQQUdFX0NBQ0hFX1NISUZUOw0KaW5kZXggaXMgdGhlIHR5cGUgb2YgcGdvZmZfdC4NClNvIGlm ICAqcHBvcyBpcyBsYXJnZXIgdGhhbiAxNlQsIHRoZSBpbmRleCBpcyBvdmVyZmxvdy5BcyB5b3Ug c2FpZCwgaXQgd2lsbCByZWFkIGxvdyBwb3NpdGlvbiBkYXRhLg0KDQpCdXQgSSB0ZXN0ZWQgdGhl IHdyaXRlIG9wZXJhdGlvbjoNCmJsa2Rldl9haW9fd3JpdGUtPl9fZ2VuZXJpY19maWxlX2Fpb193 cml0ZS4NCkluIGZ1bmN0aW9uOl9fZ2VuZXJpY19maWxlX2Fpb193cml0ZSgpDQpJdCB3aWxsIGNo ZWNrIGJ5IGZ1bmN0aW9uOmdlbmVyaWNfd3JpdGVfY2hlY2tzKCkNCkJ1dCBJbiBmdW5jdGlvbg0K PmlmIChsaWtlbHkoIWlzYmxrKSkgew0KPgkJaWYgKHVubGlrZWx5KCpwb3MgPj0gaW5vZGUtPmlf c2ItPnNfbWF4Ynl0ZXMpKSB7DQo+CQkJaWYgKCpjb3VudCB8fCAqcG9zID4gaW5vZGUtPmlfc2It PnNfbWF4Ynl0ZXMpIHsNCj4JCQkJcmV0dXJuIC1FRkJJRzsNCj4JCQl9DQo+CQkJLyogemVyby1s ZW5ndGggd3JpdGVzIGF0IC0+c19tYXhieXRlcyBhcmUgT0sgKi8NCj4JCX0NCg0KPgkJaWYgKHVu bGlrZWx5KCpwb3MgKyAqY291bnQgPiBpbm9kZS0+aV9zYi0+c19tYXhieXRlcykpDQo+CQkJKmNv dW50ID0gaW5vZGUtPmlfc2ItPnNfbWF4Ynl0ZXMgLSAqcG9zOw0KPgl9IGVsc2Ugew0KPiNpZmRl ZiBDT05GSUdfQkxPQ0sNCj4JCWxvZmZfdCBpc2l6ZTsNCj4JCWlmIChiZGV2X3JlYWRfb25seShJ X0JERVYoaW5vZGUpKSkNCj4JCQlyZXR1cm4gLUVQRVJNOw0KPgkJaXNpemUgPSBpX3NpemVfcmVh ZChpbm9kZSk7DQo+CQlpZiAoKnBvcyA+PSBpc2l6ZSkgew0KPgkJCWlmICgqY291bnQgfHwgKnBv cyA+IGlzaXplKQ0KPgkJCQlyZXR1cm4gLUVOT1NQQzsNCj4JCX0NCg0KPgkJaWYgKCpwb3MgKyAq Y291bnQgPiBpc2l6ZSkNCj4JCQkqY291bnQgPSBpc2l6ZSAtICpwb3M7DQo+I2Vsc2UNCj4JCXJl dHVybiAtRVBFUk07DQo+I2VuZGlmDQpBbHRob3VnaCBpdCBjaGVjayAoc19tYXhieXRlcylNQVhf TEZTX0ZJTEVTSVpFLkJ1dCBpcyBmaWxlIGlzIGJsb2NrIGRldmljZSxpdCBkaWQgbm90IGNoZWNr LGl0IG9ubHkgY2hlY2sgdGhlIHJlYWwgc2l6ZS4NCkJ1dCB0aGVyZSBpcyBhbHNvIGEgYnVnLkJl Y2F1c2UgaWYgYmxvY2sgc2l6ZSA+IDE2VCx0aGVyZSB3YXMgbm90IGVycm9yIGFuZCBleGVjZWQg Y29udGludWUuDQpXaGVuIGV4ZWMgZ2VuZXJpY19maWxlX2J1ZmZlcmVkX3dyaXRlKClbbm8gb2Ry aWVjdCBhY3Rpb25dIC0tLT5nZW5lcmljX3BlcmZvcm1fd3JpdGUtLT53cml0ZV9iZWdpbltibGtk ZXZfd3JpdGVfYmVnaW5dDQotLS0+YmxvY2tfd3JpdGVfYmVnaW4NCkluIGZ1bmN0aW9uOmJsb2Nr X3dyaXRlX2JlZ2luKCkNCj5wZ29mZl90IGluZGV4ID0gcG9zID4+IFBBR0VfQ0FDSEVfU0hJRlQ7 DQppbmRleCB3aWxsIG92ZXJmbG93Lg0KDQpJIG9uY2UgdGhvdWdodCB0byBwYXRjaCB0aG9zZSBi dWcoSSBtYXkgYmUgd2VsbC1rbm93biAsaGFoYSkuQnV0IEkgY2FuJ3QsYXMgaXMgZ2VuZXJpY193 cml0ZV9jaGVja3MoKToNCj4vKg0KPgkgKiBBcmUgd2UgYWJvdXQgdG8gZXhjZWVkIHRoZSBmcyBi bG9jayBsaW1pdCA/DQo+CSAqDQo+CSAqIElmIHdlIGhhdmUgd3JpdHRlbiBkYXRhIGl0IGJlY29t ZXMgYSBzaG9ydCB3cml0ZS4gIElmIHdlIGhhdmUNCj4JICogZXhjZWVkZWQgd2l0aG91dCB3cml0 aW5nIGRhdGEgd2Ugc2VuZCBhIHNpZ25hbCBhbmQgcmV0dXJuIEVGQklHLg0KPgkgKiBMaW51cyBm cmVzdHJpY3QgaWRlYSB3aWxsIGNsZWFuIHRoZXNlIHVwIG5pY2VseS4uDQo+CSAqLw0KPglpZiAo bGlrZWx5KCFpc2JsaykpIHsNCmhvdyB0byBkZWFsIHdpdGggYmxvY2s/IEFzIGEgcmVndWxhciBm aWxlIG9yIG5vdD8NCgkJCQkJCQ0KDQoNCg0KLS0tLS0tLS0tLS0tLS0tLS0tCQkJCSANCm1hamlh bnBlbmcNCjIwMTItMDUtMjgNCg0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0Kt6K8/sjLo7pIdWdoIERpY2tpbnMNCreiy83I1cba o7oyMDEyLTA1LTI3IDA1OjI0OjEzDQrK1bz+yMujum1hamlhbnBlbmcNCrOty82jukFsIFZpcm87 IEFuZHJldyBNb3J0b247IGxpbnV4LW1tOyBsaW51eC1mc2RldmVsDQrW98zio7pSZTogdGhlIG1h eCBzaXplIG9mIGJsb2NrIGRldmljZSBvbiAzMmJpdCBvcyx3aGVuIHVzaW5nZG9fZ2VuZXJpY19m aWxlX3JlYWQoKSBwcm9jZWVkLg0KDQpPbiBUaHUsIDI0IE1heSAyMDEyLCBtYWppYW5wZW5nIHdy b3RlOg0KPiAgIEhpIGFsbDoNCj4gCQlJIHJlYWRlZCBhIHJhaWQ1LHdoaWNoIHNpemUgMzBULk9T IGlzIFJIRUw2IDMyYml0Lg0KPiAJICAgIEkgcmVhZWQgdGhlIHJhaWQ1KGFzIGEgd2hvbGUsbm90 IHBhcnRlZCkgYW5kIGZvdW5kIHJlYWQgYWRkcmVzcyB3aGljaCBub3QgaSB3YW50ZWQuDQo+IAkJ U28gSSB0ZXN0ZWQgdGhlIG5ld2VzdCBrZXJuZWwgY29kZSx0aGUgcHJvYmxlbSBpcyBzdGlsbC4N Cj4gCQlJIHJldmlldyB0aGUgY29kZSwgaW4gZnVuY3Rpb24gZG9fZ2VuZXJpY19maWxlX3JlYWQo KQ0KPiANCj4gCQlpbmRleCA9ICpwcG9zID4+IFBBR0VfQ0FDSEVfU0hJRlQ7DQo+IAkJaW5kZXgg aXMgdTMyLmFuZCAqcHBvcyBpcyBsb25nIGxvbmcuDQo+IAkJU28gd2hlbiAqcHBvcyBpcyBsYXJn ZXIgdGhhbiAweEZGRkYgRkZGRiAqICBQQUdFX0NBQ0hFX1NISUZUKDE2VCBCeXRlKSx0aGVuIHRo ZSBpbmRleCBpcyBlcnJvci4NCj4gDQo+IAkJSSB3b25kZXIgdGhpcyAuSW4gMzJiaXQgb3MgLGJs b2NrIGRldmljZXMgc2l6ZSBkbyBub3QgbGFyZ2UgdGhlbiAxNlQsaW4gb3RoZXIgd29yZHMsIGlm IGJsb2NrIGRldmljZXMgbGFyZ2VyIHRoYW4gMTZULG11c3QgcGFydGVkLg0KDQpJIGFtIG5vdCBz dXJwcmlzZWQgdGhhdCB0aGUgcGFnZSBjYWNoZSBsaW1pdGF0aW9uIHByZXZlbnRzIHlvdSBmcm9t DQpyZWFkaW5nIHRoZSB3aG9sZSBkZXZpY2Ugd2l0aCBhIDMyLWJpdCBrZXJuZWwuICBTZWUgTUFY X0xGU19GSUxFU0laRSBpbg0KaW5jbHVkZS9saW51eC9mcy5oLiAgT3VyIGFuc3dlciB0byB0aGF0 IGlzIGp1c3QgdG8gdXNlIGEgNjQtYml0IGtlcm5lbC4NCg0KI2lmIEJJVFNfUEVSX0xPTkc9PTMy DQojZGVmaW5lIE1BWF9MRlNfRklMRVNJWkUgKCgodTY0KVBBR0VfQ0FDSEVfU0laRSA8PCAoQklU U19QRVJfTE9ORy0xKSktMSkgDQojZWxpZiBCSVRTX1BFUl9MT05HPT02NA0KI2RlZmluZSBNQVhf TEZTX0ZJTEVTSVpFIDB4N2ZmZmZmZmZmZmZmZmZmZlVMDQojZW5kaWYNCg0KQnV0IEkgYW0gYSBs aXR0bGUgc3VycHJpc2VkIHRoYXQgeW91IGdldCBhcyBmYXIgYXMgMTZUaUIgKHdpdGggNGsgcGFn ZSk6DQpJIHdvdWxkIGhhdmUgZXhwZWN0ZWQgeW91IHRvIGJlIHN0b3BwZWQganVzdCBiZWZvcmUg OFRpQiAoYWx0aG91Z2ggSQ0Kc3VzcGVjdCB0aGF0IHRoZSBsaW1pdGF0aW9uIHRvIDhUaUIgcmF0 aGVyIHRoYW4gMTZUaUIgaXMgdW5uZWNlc3NhcnkpLg0KDQpBbmQgaWYgSSB1bmRlcnN0YW5kIHlv dSBjb3JyZWN0bHksIHJlYWQoKSBvciBwcmVhZCgpIGdhdmUgeW91IG5vIGVycm9yDQphdCB0aG9z ZSBsYXJnZSBvZmZzZXRzLCBidXQgc3VwcGxpZWQgZGF0YSBmcm9tIHRoZSBsb3cgb2Zmc2V0IGlu c3RlYWQ/DQoNClRoYXQgZG9lcyBzdXJwcmlzZSBtZSAtIGhhdmUgd2UgbWlzc2VkIGEgY2hlY2sg dGhlcmU/DQoNCkh1Z2gNCg==

majianpeng

author

2012-05-28 06:25 UTC #3 lore raw

Re: Re: the max size of block device on 32bit os,when usingdo_generic_file_read() proceed.

In reply to majianpeng

Sorry for late to reply.I reviewed the code again and found some probleam. I created a soft-raid and the size was larger than 16T.
The os is ubuntu 12.04 32bit x86.
The udev create the block node is /dev dir(as tmpfs).
And I readed the tmpfs code :
in mm/shmem.c:shmem_fill_super()

Quoted 1 line

sb->s_maxbytes = MAX_LFS_FILESIZE;

In my computer, MAX_LFS_FILESZE is equal 8T -1.
But the read code:
generic_file_aio_read-->do_generic_file_read[not use direct flag In function:do_generic_file_read():

Quoted 1 line

index = *ppos >> PAGE_CACHE_SHIFT;

index is the type of pgoff_t.
So if *ppos is larger than 16T, the index is overflow.As you said, it will read low position data.

But I tested the write operation:
blkdev_aio_write->__generic_file_aio_write.
In function:__generic_file_aio_write()
It will check by function:generic_write_checks()
But In function

Quoted 25 lines

if (likely(!isblk)) {
		if (unlikely(*pos >= inode->i_sb->s_maxbytes)) {
			if (*count || *pos > inode->i_sb->s_maxbytes) {
				return -EFBIG;
			}
			/* zero-length writes at ->s_maxbytes are OK */
		}

		if (unlikely(*pos + *count > inode->i_sb->s_maxbytes))
			*count = inode->i_sb->s_maxbytes - *pos;
	} else {
#ifdef CONFIG_BLOCK
		loff_t isize;
		if (bdev_read_only(I_BDEV(inode)))
			return -EPERM;
		isize = i_size_read(inode);
		if (*pos >= isize) {
			if (*count || *pos > isize)
				return -ENOSPC;
		}

		if (*pos + *count > isize)
			*count = isize - *pos;
#else
		return -EPERM;

Quoted 1 line

#endif

Although it check (s_maxbytes)MAX_LFS_FILESIZE.But is file is block device,it did not check,it only check the real size. But there is also a bug.Because if block size > 16T,there was not error and execed continue.
When exec generic_file_buffered_write()[no odriect action] --->generic_perform_write-->write_begin[blkdev_write_begin] --->block_write_begin
In function:block_write_begin()

Quoted 1 line

pgoff_t index = pos >> PAGE_CACHE_SHIFT;

index will overflow.

I once thought to patch those bug(I may be well-known ,haha).But I can't,as is generic_write_checks():

Quoted 8 lines

/*
* Are we about to exceed the fs block limit ?
*
* If we have written data it becomes a short write. If we have
* exceeded without writing data we send a signal and return EFBIG.
* Linus frestrict idea will clean these up nicely..
*/
if (likely(!isblk)) {

how to deal with block? As a regular file or not?

------------------
majianpeng
2012-05-28

-------------------------------------------------------------
发件人：Hugh Dickins
发送日期：2012-05-27 05:24:13
收件人：majianpeng
抄送：Al Viro; Andrew Morton; linux-mm; linux-fsdevel
主题：Re: the max size of block device on 32bit os,when usingdo_generic_file_read() proceed.

Quoted 11 lines · majianpeng

Hi all:
I readed a raid5,which size 30T.OS is RHEL6 32bit.
I reaed the raid5(as a whole,not parted) and found read address which not i wanted.
So I tested the newest kernel code,the problem is still.
I review the code, in function do_generic_file_read()
index = *ppos >> PAGE_CACHE_SHIFT;
index is u32.and *ppos is long long.
So when *ppos is larger than 0xFFFF FFFF * PAGE_CACHE_SHIFT(16T Byte),then the index is error.
I wonder this .In 32bit os ,block devices size do not large then 16T,in other words, if block devices larger than 16T,must parted.

#if BITS_PER_LONG==32
#define MAX_LFS_FILESIZE (((u64)PAGE_CACHE_SIZE << (BITS_PER_LONG-1))-1) #elif BITS_PER_LONG==64
#define MAX_LFS_FILESIZE 0x7fffffffffffffffUL
#endif

And if I understand you correctly, read() or pread() gave you no error at those large offsets, but supplied data from the low offset instead?

That does surprise me - have we missed a check there?

Hugh