Skip to contentSkip to conversation

kernel 2.6.22 pdflush consumes ~90% cpu

DISCUSSION Muralitharan Perumal 3 participants 4 messages Jan 13, 2014

lore
Share & tools
lore
#0 lore raw

Hi,

I am currently working on an embedded platform involving XFS as the preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit is working fine with ext3 but after replacing with XFS, I am seeing a few issues:

a) Occassionally I am getting the following error when we write files to the fs:

attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096
attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096

b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?

Any pointers would be helpful.

Thanks,
Murali

IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw

Hi Dave,

Is this related to log buffer size when formatting and mounting as I am just using the default 32KB buffer size?

Any pointers will be helpful.

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Muralitharan Perumal
Sent: 13 January 2014 14:59
To: [email protected]
Subject: kernel 2.6.22 pdflush consumes ~90% cpu

Hi,

I am currently working on an embedded platform involving XFS as the preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit is working fine with ext3 but after replacing with XFS, I am seeing a few issues:

a) Occassionally I am getting the following error when we write files to the fs:

attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096
attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096

b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?

Any pointers would be helpful.

Thanks,
Murali

IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender. IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
In reply to Muralitharan Perumal
Quoted 6 lines · Muralitharan Perumal

Hi,

I am currently working on an embedded platform involving XFS as the preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit is working fine with ext3 but after replacing with XFS, I am seeing a few issues:

a 7 year old kernel on an embedded platform? Nobody here will be able to spend much time on this I'm afraid.

Quoted 9 lines

a) Occassionally I am getting the following error when we write files to the fs:

attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096
attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096

looks like on-disk corruption. xfs_repair is probably warranted.

Quoted 1 line

b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

do a sysrq-w to get backtraces of all blocked tasks, and that might yield a clue for you.

-Eric

Quoted 8 lines

Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?

Any pointers would be helpful.

Thanks,
Murali

Quoted 9 lines

IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw
In reply to Muralitharan Perumal
Quoted 10 lines · Muralitharan Perumal

Hi,

I am currently working on an embedded platform involving XFS as the preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit is working fine with ext3 but after replacing with XFS, I am seeing a few issues:

a) Occassionally I am getting the following error when we write files to the fs:

Quoted 6 lines

attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096
attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8 ("xfs_trans_read_buf") error 5 buf count 4096

It's trying to write way beyond the end of the device, hence the error. I'd suggest updating to xfsprogs 3.11 and running xfs_repair -n /dev/sda1 on the device and seeing if it finds a corrupt block map on an inode somewhere.

Quoted 1 line

b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

It can't be stuck in an uninterruptible sleep (D state) and consuming CPU at the same time....

Quoted 1 line

Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?

That's a 6 year old kernel. I'd suggest that an upgrade to something more recent might be in order...

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs