Skip to contentSkip to conversation

ionice and FUSE-based filesystems?

DISCUSSION Chris Friesen 3 participants 6 messages Sep 2, 2010 – Sep 8, 2010

lore
Share & tools
lore
#0 lore raw

I'm curious about the limits of using ionice with multiple layers of filesystems and devices.

In particular, we have a scenario with a FUSE-based filesystem running on top of xfs on top of LVM, on top of software RAID, on top of spinning disks. (Something like that, anyways.) The IO scheduler is CFQ.

In the above scenario would you expect the IO nice value of the writes done by a task to be propagated all the way down to the disk writes? Or would they get stripped off at some point?

Thanks,
Chris

Signature · 5 lines
Chris Friesen
Software Developer
GENBAND
[email protected]
www.genband.com
#1 lore raw

I also wanna know this!!

Manuel Amador (Rudd-O)
Cloud.com, Inc. -- http://www.cloud.com

Quoted 35 lines · Chris Friesen

I'm curious about the limits of using ionice with multiple layers of filesystems and devices.

In particular, we have a scenario with a FUSE-based filesystem running on top of xfs on top of LVM, on top of software RAID, on top of spinning disks. (Something like that, anyways.) The IO scheduler is CFQ.

In the above scenario would you expect the IO nice value of the writes done by a task to be propagated all the way down to the disk writes? Or would they get stripped off at some point?

Thanks,
Chris

Signature · 19 lines
Chris Friesen
Software Developer
GENBAND
[email protected]
www.genband.com

--- 
--- 
--- 
---------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
fuse-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
#2 lore raw
In reply to Chris Friesen
Quoted 11 lines · Chris Friesen

I'm curious about the limits of using ionice with multiple layers of filesystems and devices.

In particular, we have a scenario with a FUSE-based filesystem running on top of xfs on top of LVM, on top of software RAID, on top of spinning disks. (Something like that, anyways.) The IO scheduler is CFQ.

In the above scenario would you expect the IO nice value of the writes done by a task to be propagated all the way down to the disk writes? Or would they get stripped off at some point?

Miklos should be able to expand on what fuse does, but at least on the write side priorities will only be carried through for non-buffered writes with the current design (since actual write out happens out of context of the submitting application).

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Chris Friesen

I'm curious about the limits of using ionice with multiple layers of filesystems and devices.

In particular, we have a scenario with a FUSE-based filesystem running on top of xfs on top of LVM, on top of software RAID, on top of spinning disks. (Something like that, anyways.) The IO scheduler is CFQ.

Quoted 8 lines
Quoted 3 lines

In the above scenario would you expect the IO nice value of the writes done by a task to be propagated all the way down to the disk writes? Or would they get stripped off at some point?

Miklos should be able to expand on what fuse does, but at least on the write side priorities will only be carried through for non-buffered writes with the current design (since actual write out happens out of context of the submitting application).

So we're talking either O_SYNC or O_DIRECT only? That seems an unfortunate limitation given that it then forces the app to block. Has any thought been given to somehow associating the priority with the actual operation so that it would affect buffered writes as well?

As for fuse...I was concerned that the addition of userspace tasks to handle the filesystem operations would result in the I/O operations taking on the priority of the fuse tasks rather than the originating task. Or does fuse adjust its I/O nice level according to that of the incoming requests?

Chris

Signature · 5 lines
Chris Friesen
Software Developer
GENBAND
[email protected]
www.genband.com
#4 lore raw
In reply to Jens Axboe
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Chris Friesen

I'm curious about the limits of using ionice with multiple layers of filesystems and devices.

In particular, we have a scenario with a FUSE-based filesystem running on top of xfs on top of LVM, on top of software RAID, on top of spinning disks. (Something like that, anyways.) The IO scheduler is CFQ.

Quoted 8 lines
Quoted 3 lines

In the above scenario would you expect the IO nice value of the writes done by a task to be propagated all the way down to the disk writes? Or would they get stripped off at some point?

Miklos should be able to expand on what fuse does, but at least on the write side priorities will only be carried through for non-buffered writes with the current design (since actual write out happens out of context of the submitting application).

So we're talking either O_SYNC or O_DIRECT only? That seems an unfortunate limitation given that it then forces the app to block. Has any thought been given to somehow associating the priority with the actual operation so that it would affect buffered writes as well?

As for fuse...I was concerned that the addition of userspace tasks to handle the filesystem operations would result in the I/O operations taking on the priority of the fuse tasks rather than the originating task. Or does fuse adjust its I/O nice level according to that of the incoming requests?

Chris

Signature · 5 lines
Chris Friesen
Software Developer
GENBAND
[email protected]
www.genband.com
#5 lore raw
Quoted 19 lines · Chris Friesen
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Chris Friesen

I'm curious about the limits of using ionice with multiple layers of filesystems and devices.

In particular, we have a scenario with a FUSE-based filesystem running on top of xfs on top of LVM, on top of software RAID, on top of spinning disks. (Something like that, anyways.) The IO scheduler is CFQ.

Quoted 8 lines
Quoted 3 lines

In the above scenario would you expect the IO nice value of the writes done by a task to be propagated all the way down to the disk writes? Or would they get stripped off at some point?

Miklos should be able to expand on what fuse does, but at least on the write side priorities will only be carried through for non-buffered writes with the current design (since actual write out happens out of context of the submitting application).

Quoted 4 lines

So we're talking either O_SYNC or O_DIRECT only? That seems an unfortunate limitation given that it then forces the app to block. Has any thought been given to somehow associating the priority with the actual operation so that it would affect buffered writes as well?

Yes. Work is progressing to allow to track dirty pages which would then allow you to use ionice for buffered writes as well.

Signature · 1 line
Jens Axboe