Priorities in Anticipatory I/O scheduler

Modifications to the Anticipatory I/O scheduler to add multiple priority levels. It makes use of anticipation and batching in current anticipatory scheduler to implement priorities.

- Minimizes the latency of highest priority level.
- Low priority requests wait for high priority requests.
- Higher priority request break any anticipating low priority request.
- If single priority level is used the scheduler behaves as an anticipatory scheduler. So no change for existing users.

With this change, it is possible for a latency sensitive job to coexist with background job.

Other possible use of this patch is in context of I/O subsystem controller. It can add another dimension to the parameters controlling a particular cgroup. While we can easily divide b/w among existing croups, setting a bound on latency is not a feasible solution. Hence in context of storage devices bandwidth and priority can be two parameters controlling I/O. Though it can be a standalone patch to separate latency sensitive jobs and need not be tied to I/O controller.

In this patch I have added a new class IOPRIO_CLASS_LATENCY to differentiate notion of absolute priority over existing uses of various time-slice based priority classes in cfq. Though internally within anticipatory scheduler all of them map to best-effort levels. Hence, one can also use various best-effort priority levels.

Quoted 22 lines

Modifications to the Anticipatory I/O scheduler to add multiple priority levels. It makes use of anticipation and batching in current anticipatory scheduler to implement priorities.

- Minimizes the latency of highest priority level.
- Low priority requests wait for high priority requests.
- Higher priority request break any anticipating low priority request.
- If single priority level is used the scheduler behaves as an anticipatory scheduler. So no change for existing users.

With this change, it is possible for a latency sensitive job to coexist with background job.

Other possible use of this patch is in context of I/O subsystem controller. It can add another dimension to the parameters controlling a particular cgroup. While we can easily divide b/w among existing croups, setting a bound on latency is not a feasible solution. Hence in context of storage devices bandwidth and priority can be two parameters controlling I/O. Though it can be a standalone patch to separate latency sensitive jobs and need not be tied to I/O controller.

Quoted 5 lines

In this patch I have added a new class IOPRIO_CLASS_LATENCY to differentiate notion of absolute priority over existing uses of various time-slice based priority classes in cfq. Though internally within anticipatory scheduler all of them map to best-effort levels. Hence, one can also use various best-effort priority levels.

Please don't introduce yet another incompatible behaviour between I/O schedulers. It's bad enough from an optimisation point of view that BIO_RW_SYNC and BIO_RW_META mean different things to different schedulers, let alone that only CFQ currently understands priorities. If you are going to introduce priorities into AS, then please, please, please make it use the same interface as CFQ.

Why? Both the extN and XFS devs have been considering bumping the priority of journal writes using the existing CFQ-based I/O priority mechanism - the last thing I want to see is a different scheduler requiring a different priority configuration to acheive the same optimisation. There is no way we can support this sort of optimisation in the filesystem code if the interface changes when the I/O scheduler changes. So please use the existing IOPRIO classes to map the priorities for the AS scheduler.

Cheers,

Dave.
--
Dave Chinner
[email protected]

2008/10/27 Dave Chinner <[email protected]>:

Quoted 5 lines · On Mon, Oct 27, 2008 at 12:01:32PM -0700, [email protected] wrote:

Quoted 4 lines

Modifications to the Anticipatory I/O scheduler to add multiple priority levels. It makes use of anticipation and batching in current anticipatory scheduler to implement priorities.

Quoted 13 lines

Quoted 5 lines

In this patch I have added a new class IOPRIO_CLASS_LATENCY to differentiate notion of absolute priority over existing uses of various time-slice based priority classes in cfq. Though internally within anticipatory scheduler all of them map to best-effort levels. Hence, one can also use various best-effort priority levels.

Please don't introduce yet another incompatible behaviour between I/O schedulers. It's bad enough from an optimisation point of view that BIO_RW_SYNC and BIO_RW_META mean different things to different schedulers, let alone that only CFQ currently understands priorities. If you are going to introduce priorities into AS, then please, please, please make it use the same interface as CFQ.

Quoted 9 lines

Why? Both the extN and XFS devs have been considering bumping the priority of journal writes using the existing CFQ-based I/O priority mechanism - the last thing I want to see is a different scheduler requiring a different priority configuration to acheive the same optimisation. There is no way we can support this sort of optimisation in the filesystem code if the interface changes when the I/O scheduler changes. So please use the existing IOPRIO classes to map the priorities for the AS scheduler.

The anticipatory scheduler chooses it's next i/o to be of highest available priority level.

So, in some sense it kind of implements
absolute priority and is best used for jobs which are latency sensitive. Since the priorities can be and are mapped internally in anticipatory scheduler, BEST_EFFORT class is mapped one-one with the LATENCY class.

A filesystem can use best-effort class using similar interface as for cfq.

Quoted 6 lines

2008/10/27 Dave Chinner <[email protected]>:

Quoted 5 lines · On Mon, Oct 27, 2008 at 12:01:32PM -0700, [email protected] wrote:

Quoted 4 lines

Modifications to the Anticipatory I/O scheduler to add multiple priority levels. It makes use of anticipation and batching in current anticipatory scheduler to implement priorities.

.....

Quoted 23 lines

Quoted 22 lines

Quoted 5 lines

In this patch I have added a new class IOPRIO_CLASS_LATENCY to differentiate notion of absolute priority over existing uses of various time-slice based priority classes in cfq. Though internally within anticipatory scheduler all of them map to best-effort levels. Hence, one can also use various best-effort priority levels.

Please don't introduce yet another incompatible behaviour between I/O schedulers. It's bad enough from an optimisation point of view that BIO_RW_SYNC and BIO_RW_META mean different things to different schedulers, let alone that only CFQ currently understands priorities. If you are going to introduce priorities into AS, then please, please, please make it use the same interface as CFQ.

Why? Both the extN and XFS devs have been considering bumping the priority of journal writes using the existing CFQ-based I/O priority mechanism - the last thing I want to see is a different scheduler requiring a different priority configuration to acheive the same optimisation. There is no way we can support this sort of optimisation in the filesystem code if the interface changes when the I/O scheduler changes. So please use the existing IOPRIO classes to map the priorities for the AS scheduler.

Quoted 2 lines

The anticipatory scheduler chooses it's next i/o to be of highest available priority level.

That sounds exactly like what the current RT class is supposed to be used for - defining the absolute priority of dispatch. How is this latency class different to the current RT class semantics that are defined for CFQ?

Quoted 5 lines

So, in some sense it kind of implements
absolute priority and is best used for jobs which are latency sensitive. Since the priorities can be and are mapped internally in anticipatory scheduler, BEST_EFFORT class is mapped one-one with the LATENCY class.

So you map the BE class to something with the same semantics as the RT class? What mapping do you do when an application uses the RT class?

Quoted 2 lines

A filesystem can use best-effort class using similar interface as for cfq.

The folk using the RT priority classes greatly objected to using the RT class for journal I/O precisely because it would then preempt their application's RT I/O and introduce unpredictable latencies.

Journal I/O will typically use the highest priority BE class so that it is promoted above BE I/O but does not preempt RT I/O. With your mapping of BE classes to this new "absolute priority latency" class, this configuration will give journal I/O the highest priority in the scheduler. This will cause preemption of your latency sensitive I/O and so those latencies you are trying to avoid won't go away....

Cheers,

Dave.
--
Dave Chinner
[email protected]

2008/10/28 Dave Chinner <[email protected]>:

Quoted 31 lines · Naveen Gupta

Quoted 6 lines

2008/10/27 Dave Chinner <[email protected]>:

Quoted 5 lines · On Mon, Oct 27, 2008 at 12:01:32PM -0700, [email protected] wrote:

Quoted 4 lines

Modifications to the Anticipatory I/O scheduler to add multiple priority levels. It makes use of anticipation and batching in current anticipatory scheduler to implement priorities.

.....

Quoted 23 lines

Quoted 22 lines

Quoted 5 lines

In this patch I have added a new class IOPRIO_CLASS_LATENCY to differentiate notion of absolute priority over existing uses of various time-slice based priority classes in cfq. Though internally within anticipatory scheduler all of them map to best-effort levels. Hence, one can also use various best-effort priority levels.

Please don't introduce yet another incompatible behaviour between I/O schedulers. It's bad enough from an optimisation point of view that BIO_RW_SYNC and BIO_RW_META mean different things to different schedulers, let alone that only CFQ currently understands priorities. If you are going to introduce priorities into AS, then please, please, please make it use the same interface as CFQ.

Why? Both the extN and XFS devs have been considering bumping the priority of journal writes using the existing CFQ-based I/O priority mechanism - the last thing I want to see is a different scheduler requiring a different priority configuration to acheive the same optimisation. There is no way we can support this sort of optimisation in the filesystem code if the interface changes when the I/O scheduler changes. So please use the existing IOPRIO classes to map the priorities for the AS scheduler.

Quoted 8 lines

Quoted 2 lines

The anticipatory scheduler chooses it's next i/o to be of highest available priority level.

That sounds exactly like what the current RT class is supposed to be used for - defining the absolute priority of dispatch. How is this latency class different to the current RT class semantics that are defined for CFQ?

I/O from RT class in CFQ can still see a bubble with this new latency class. An easy way to check this would be to submit ios at multiple levels both in CFQ and AS and check max latency of the highest levels. I will let Jens or Satoshi comment on exact algorithm for RT class.

Quoted 10 lines

Quoted 5 lines

So, in some sense it kind of implements absolute priority and is best used for jobs which are latency sensitive. Since the priorities can be and are mapped internally in anticipatory scheduler, BEST_EFFORT class is mapped one-one with the LATENCY class.

So you map the BE class to something with the same semantics as the RT class? What mapping do you do when an application uses the RT class?

Yes I could have used RT class but it was used in CFQ to implement it's time-sliced based highest priority class. If an application uses RT class, AS maps all levels of RT class to BE class level 0 (i.e. to the highest priority available)

Quoted 8 lines

Quoted 2 lines

A filesystem can use best-effort class using similar interface as for cfq.

The folk using the RT priority classes greatly objected to using the RT class for journal I/O precisely because it would then preempt their application's RT I/O and introduce unpredictable latencies.

Quoted 8 lines

Journal I/O will typically use the highest priority BE class so that it is promoted above BE I/O but does not preempt RT I/O. With your mapping of BE classes to this new "absolute priority latency" class, this configuration will give journal I/O the highest priority in the scheduler. This will cause preemption of your latency sensitive I/O and so those latencies you are trying to avoid won't go away....

I see your problem, we could make the LATENCY class different from and above BE class (instead of one-one mapping).

Quoted 41 lines

2008/10/28 Dave Chinner <[email protected]>:

Quoted 31 lines · Naveen Gupta

Quoted 6 lines

2008/10/27 Dave Chinner <[email protected]>:

Quoted 5 lines · On Mon, Oct 27, 2008 at 12:01:32PM -0700, [email protected] wrote:

Quoted 4 lines

Modifications to the Anticipatory I/O scheduler to add multiple priority levels. It makes use of anticipation and batching in current anticipatory scheduler to implement priorities.

.....

Quoted 23 lines

Quoted 22 lines

Quoted 5 lines

In this patch I have added a new class IOPRIO_CLASS_LATENCY to differentiate notion of absolute priority over existing uses of various time-slice based priority classes in cfq. Though internally within anticipatory scheduler all of them map to best-effort levels. Hence, one can also use various best-effort priority levels.

Please don't introduce yet another incompatible behaviour between I/O schedulers. It's bad enough from an optimisation point of view that BIO_RW_SYNC and BIO_RW_META mean different things to different schedulers, let alone that only CFQ currently understands priorities. If you are going to introduce priorities into AS, then please, please, please make it use the same interface as CFQ.

Why? Both the extN and XFS devs have been considering bumping the priority of journal writes using the existing CFQ-based I/O priority mechanism - the last thing I want to see is a different scheduler requiring a different priority configuration to acheive the same optimisation. There is no way we can support this sort of optimisation in the filesystem code if the interface changes when the I/O scheduler changes. So please use the existing IOPRIO classes to map the priorities for the AS scheduler.

Quoted 8 lines

Quoted 2 lines

The anticipatory scheduler chooses it's next i/o to be of highest available priority level.

That sounds exactly like what the current RT class is supposed to be used for - defining the absolute priority of dispatch. How is this latency class different to the current RT class semantics that are defined for CFQ?

Quoted 4 lines

I/O from RT class in CFQ can still see a bubble with this new latency class. An easy way to check this would be to submit ios at multiple levels both in CFQ and AS and check max latency of the highest levels. I will let Jens or Satoshi comment on exact algorithm for RT class.

You're missing my point entirely.

You're defining a new class that has the exact same meaning as the current RT class definition, then mapping the BE class over the top of that, hence changing what that means for everyone.

The fact that the *implementation* of AS and CFQ is different is irrelevant; if you use the RT class then on CFQ you get the current RT behaviour, if you use the RT class on AS you should get your new priority dispatch mechanism. We don't need a new API just because the implementations are different.

Quoted 11 lines

Quoted 10 lines

Quoted 5 lines

So, in some sense it kind of implements absolute priority and is best used for jobs which are latency sensitive. Since the priorities can be and are mapped internally in anticipatory scheduler, BEST_EFFORT class is mapped one-one with the LATENCY class.

So you map the BE class to something with the same semantics as the RT class? What mapping do you do when an application uses the RT class?

Quoted 4 lines

Yes I could have used RT class but it was used in CFQ to implement it's time-sliced based highest priority class. If an application uses RT class, AS maps all levels of RT class to BE class level 0 (i.e. to the highest priority available)

Which means you are throwing away all the RT priority levels and so an application using the RT class would be subtly broken on AS....

Quoted 17 lines

Quoted 16 lines

Quoted 2 lines

A filesystem can use best-effort class using similar interface as for cfq.

The folk using the RT priority classes greatly objected to using the RT class for journal I/O precisely because it would then preempt their application's RT I/O and introduce unpredictable latencies.

Journal I/O will typically use the highest priority BE class so that it is promoted above BE I/O but does not preempt RT I/O. With your mapping of BE classes to this new "absolute priority latency" class, this configuration will give journal I/O the highest priority in the scheduler. This will cause preemption of your latency sensitive I/O and so those latencies you are trying to avoid won't go away....

Quoted 2 lines

I see your problem, we could make the LATENCY class different from and above BE class (instead of one-one mapping).

Like the RT class is currently defined to be? ;)

Cheers,

Dave.
--
Dave Chinner
[email protected]

2008/10/28 Dave Chinner <[email protected]>:

Quoted 11 lines · Naveen Gupta

Quoted 4 lines

I/O from RT class in CFQ can still see a bubble with this new latency class. An easy way to check this would be to submit ios at multiple levels both in CFQ and AS and check max latency of the highest levels. I will let Jens or Satoshi comment on exact algorithm for RT class.

You're missing my point entirely.

You're defining a new class that has the exact same meaning as the current RT class definition, then mapping the BE class over the top of that, hence changing what that means for everyone.

Quoted 6 lines

The fact that the *implementation* of AS and CFQ is different is irrelevant; if you use the RT class then on CFQ you get the current RT behaviour, if you use the RT class on AS you should get your new priority dispatch mechanism. We don't need a new API just because the implementations are different.

There is nothing "real-time" about the current RT class anyways. It is

basically these small *implementation* differences that defines these classes in current scheme of things, precise definitions of which would be very hard to find if one started looking around.

The current implementation of AS is basically a flat structure with multiple priority levels. Initially I planned them to be different levels of best-effort class, which is exactly what we are doing "best-effort" from the scheduler/software point of view. So, the question is what you do with other classes for which you don't have a significantly different behavior: to keep things simple you map them to existing flat structure. And, I mapped RT (all levels to BE 0), idle (all levels to BE 7).

This leaves these RT and IDLE classes open for future implementations, where one could use hardware priorities (may be in NCQ) to implement RT class or other improvisations in software other than schedulers to map to RT class.

Now the initial feedback was since this *implementation* is different from anything we have in CFQ which is our current *standard* way of thinking and comparing (that is the only thing that exists) why not make them into a new class :). And somehow map others so that they make some sense till we get something for those classes as well.

Quoted 13 lines

Quoted 13 lines

Quoted 9 lines

Quoted 5 lines

So, in some sense it kind of implements absolute priority and is best used for jobs which are latency sensitive. Since the priorities can be and are mapped internally in anticipatory scheduler, BEST_EFFORT class is mapped one-one with the LATENCY class.

So you map the BE class to something with the same semantics as the RT class? What mapping do you do when an application uses the RT class?

Yes I could have used RT class but it was used in CFQ to implement it's time-sliced based highest priority class. If an application uses RT class, AS maps all levels of RT class to BE class level 0 (i.e. to the highest priority available)

Quoted 3 lines

Which means you are throwing away all the RT priority levels and so an application using the RT class would be subtly broken on AS....

As I said earlier the organization of the AS levels is flat, so we could use any class (RT, BE, LATENCY) and fold the remaining ones. The other way which you would probably like is to increase number of levels and map different classes so that they are not folded.

Quoted 15 lines

Quoted 15 lines

Quoted 15 lines

Quoted 2 lines

A filesystem can use best-effort class using similar interface as for cfq.

The folk using the RT priority classes greatly objected to using the RT class for journal I/O precisely because it would then preempt their application's RT I/O and introduce unpredictable latencies.

Journal I/O will typically use the highest priority BE class so that it is promoted above BE I/O but does not preempt RT I/O. With your mapping of BE classes to this new "absolute priority latency" class, this configuration will give journal I/O the highest priority in the scheduler. This will cause preemption of your latency sensitive I/O and so those latencies you are trying to avoid won't go away....

Quoted 4 lines

Quoted 2 lines

I see your problem, we could make the LATENCY class different from and above BE class (instead of one-one mapping).

Like the RT class is currently defined to be? ;)

I agree with you and we could use RT (though you and I know that basically it is best effort). LATENCY was invented due to a previous suggestion.

Quoted 12 lines · Naveen Gupta

2008/10/28 Dave Chinner <[email protected]>:

Quoted 11 lines · Naveen Gupta

Quoted 4 lines

I/O from RT class in CFQ can still see a bubble with this new latency class. An easy way to check this would be to submit ios at multiple levels both in CFQ and AS and check max latency of the highest levels. I will let Jens or Satoshi comment on exact algorithm for RT class.

You're missing my point entirely.

You're defining a new class that has the exact same meaning as the current RT class definition, then mapping the BE class over the top of that, hence changing what that means for everyone.

Quoted 8 lines

Quoted 6 lines

The fact that the *implementation* of AS and CFQ is different is irrelevant; if you use the RT class then on CFQ you get the current RT behaviour, if you use the RT class on AS you should get your new priority dispatch mechanism. We don't need a new API just because the implementations are different.

There is nothing "real-time" about the current RT class anyways. It is

Yes, this is stupid. IMO the real time class should be strict priorities within the class, and within the same priority level, round robin. As it stands, RT seems to be just like a second BE class.

Quoted 4 lines

basically these small *implementation* differences that defines these classes in current scheme of things, precise definitions of which would be very hard to find if one started looking around.

Quoted 8 lines

The current implementation of AS is basically a flat structure with multiple priority levels. Initially I planned them to be different levels of best-effort class, which is exactly what we are doing "best-effort" from the scheduler/software point of view. So, the question is what you do with other classes for which you don't have a significantly different behavior: to keep things simple you map them to existing flat structure. And, I mapped RT (all levels to BE 0), idle (all levels to BE 7).

Even compared to CFQs broken RT handling, this is wrong, because now any old BE0 process is equal in priority to any RT process.

Quoted 24 lines

This leaves these RT and IDLE classes open for future implementations, where one could use hardware priorities (may be in NCQ) to implement RT class or other improvisations in software other than schedulers to map to RT class.

Now the initial feedback was since this *implementation* is different from anything we have in CFQ which is our current *standard* way of thinking and comparing (that is the only thing that exists) why not make them into a new class :). And somehow map others so that they make some sense till we get something for those classes as well.

Quoted 13 lines

Quoted 13 lines

Quoted 9 lines

Quoted 5 lines

So, in some sense it kind of implements absolute priority and is best used for jobs which are latency sensitive. Since the priorities can be and are mapped internally in anticipatory scheduler, BEST_EFFORT class is mapped one-one with the LATENCY class.

So you map the BE class to something with the same semantics as the RT class? What mapping do you do when an application uses the RT class?

Yes I could have used RT class but it was used in CFQ to implement it's time-sliced based highest priority class. If an application uses RT class, AS maps all levels of RT class to BE class level 0 (i.e. to the highest priority available)

Quoted 8 lines

Quoted 3 lines

Which means you are throwing away all the RT priority levels and so an application using the RT class would be subtly broken on AS....

As I said earlier the organization of the AS levels is flat, so we could use any class (RT, BE, LATENCY) and fold the remaining ones. The other way which you would probably like is to increase number of levels and map different classes so that they are not folded.

As I said in my reply to the initial posting of this, I think there are only two sensible ways of handling this:

1) Maintain the full number of I/O priorities (1 IDLE, 8 BE, 8 RT);

2) Collapse the levels and only deal with the classes;

Any other mapping seems arbitrary and likely to confuse.

Quoted 15 lines

Quoted 15 lines

Quoted 15 lines

Quoted 15 lines

Quoted 2 lines

A filesystem can use best-effort class using similar interface as for cfq.

The folk using the RT priority classes greatly objected to using the RT class for journal I/O precisely because it would then preempt their application's RT I/O and introduce unpredictable latencies.

Journal I/O will typically use the highest priority BE class so that it is promoted above BE I/O but does not preempt RT I/O. With your mapping of BE classes to this new "absolute priority latency" class, this configuration will give journal I/O the highest priority in the scheduler. This will cause preemption of your latency sensitive I/O and so those latencies you are trying to avoid won't go away....

Quoted 8 lines

Quoted 4 lines

Quoted 2 lines

I see your problem, we could make the LATENCY class different from and above BE class (instead of one-one mapping).

Like the RT class is currently defined to be? ;)

I agree with you and we could use RT (though you and I know that basically it is best effort). LATENCY was invented due to a previous suggestion.

Maybe what you want to do is make RT really real-time, and then use this latency class to differentiate latency-sensitive BE traffic from regular BE traffic. Not necessarily ``higher'' priority, just a different kind of best-effort. One way of implementing this in CFQ might be to have smaller but more frequent dispatches.

Also from the original posting, I think the weights are still broken (especially in the context of RT) but I won't repeat that here.

-- Aaron

2008/10/28 Aaron Carroll <[email protected]>:

Quoted 5 lines

Quoted 4 lines · Naveen Gupta

As I said earlier the organization of the AS levels is flat, so we could use any class (RT, BE, LATENCY) and fold the remaining ones. The other way which you would probably like is to increase number of levels and map different classes so that they are not folded.

Quoted 4 lines

As I said in my reply to the initial posting of this, I think there are only two sensible ways of handling this:

1) Maintain the full number of I/O priorities (1 IDLE, 8 BE, 8 RT);

But then we are assuming that we are providing different quality of service according to classes.

Quoted 1 line

2) Collapse the levels and only deal with the classes;

I am not sure if this is meaningful. When all we have is different levels of BE, it wouldn't make sense to call them different classes.

Quoted 6 lines · Naveen Gupta

2008/10/28 Aaron Carroll <[email protected]>:

Quoted 5 lines

Quoted 4 lines · Naveen Gupta

As I said earlier the organization of the AS levels is flat, so we could use any class (RT, BE, LATENCY) and fold the remaining ones. The other way which you would probably like is to increase number of levels and map different classes so that they are not folded.

Quoted 7 lines

Quoted 4 lines

As I said in my reply to the initial posting of this, I think there are only two sensible ways of handling this:

1) Maintain the full number of I/O priorities (1 IDLE, 8 BE, 8 RT);

But then we are assuming that we are providing different quality of service according to classes.

Right. The ideal solution is a scheduler-independent definition of RT (Jens?) which you can apply here. However, it seems to me that you want to basically ignore RT and IDLE. If you're going to do that, at least implement sane alternate behaviour.

This solution applies the the principle of least surprise; RT requests always have higher priority than BE requests, and within the class, higher level means higher priority. In your implementation, BE 0 == RT x and IDLE == BE 7. This is surprising behaviour.

Quoted 5 lines

Quoted 2 lines

2) Collapse the levels and only deal with the classes;

I am not sure if this is meaningful. When all we have is different levels of BE, it wouldn't make sense to call them different classes.

It's not meaningful as it stands. This difference here is that you at least maintain the ordering of the classes with respect to priority.

-- Aaron

2008/10/28 Dave Chinner <[email protected]>:

Quoted 23 lines · Naveen Gupta

Quoted 11 lines

2008/10/28 Dave Chinner <[email protected]>:

Quoted 9 lines · Naveen Gupta

Quoted 2 lines

The anticipatory scheduler chooses it's next i/o to be of highest available priority level.

That sounds exactly like what the current RT class is supposed to be used for - defining the absolute priority of dispatch. How is this latency class different to the current RT class semantics that are defined for CFQ?

Quoted 4 lines

I/O from RT class in CFQ can still see a bubble with this new latency class. An easy way to check this would be to submit ios at multiple levels both in CFQ and AS and check max latency of the highest levels. I will let Jens or Satoshi comment on exact algorithm for RT class.

You're missing my point entirely.

You're defining a new class that has the exact same meaning as the current RT class definition, then mapping the BE class over the top of that, hence changing what that means for everyone.

Quoted 6 lines

The fact that the *implementation* of AS and CFQ is different is irrelevant; if you use the RT class then on CFQ you get the current RT behaviour, if you use the RT class on AS you should get your new priority dispatch mechanism. We don't need a new API just because the implementations are different.

There is nothing "real-time" about the current RT class anyways.

It is
basically these small *implementation* differences that defines these classes in current scheme of things, precise definitions of which would be very hard to find if one started looking around.

Now the initial feedback was since this *implementation* is different from anything we have in CFQ which is our current *standard* way of thinking and comparing (that is the only thing that exists) why not make them into a new class :).

Quoted 4 lines

Quoted 2 lines

I see your problem, we could make the LATENCY class different from and above BE class (instead of one-one mapping).

Like the RT class is currently defined to be? ;)

I agree with you and we could use RT (though you and I know that basically it is best effort). LATENCY was invented due to a previous suggestion.

Quoted 24 lines

2008/10/28 Dave Chinner <[email protected]>:

Quoted 23 lines · Naveen Gupta

Quoted 11 lines

2008/10/28 Dave Chinner <[email protected]>:

Quoted 9 lines · Naveen Gupta

Quoted 2 lines

The anticipatory scheduler chooses it's next i/o to be of highest available priority level.

That sounds exactly like what the current RT class is supposed to be used for - defining the absolute priority of dispatch. How is this latency class different to the current RT class semantics that are defined for CFQ?

Quoted 4 lines

I/O from RT class in CFQ can still see a bubble with this new latency class. An easy way to check this would be to submit ios at multiple levels both in CFQ and AS and check max latency of the highest levels. I will let Jens or Satoshi comment on exact algorithm for RT class.

You're missing my point entirely.

You're defining a new class that has the exact same meaning as the current RT class definition, then mapping the BE class over the top of that, hence changing what that means for everyone.

Quoted 8 lines

Quoted 6 lines

The fact that the *implementation* of AS and CFQ is different is irrelevant; if you use the RT class then on CFQ you get the current RT behaviour, if you use the RT class on AS you should get your new priority dispatch mechanism. We don't need a new API just because the implementations are different.

There is nothing "real-time" about the current RT class anyways.

That's an implementation problem, not an API definition problem.

Quoted 4 lines

It is
basically these small *implementation* differences that defines these classes in current scheme of things, precise definitions of which would be very hard to find if one started looking around.

Please, disconnect what you think about implementation and ask yourself what makes sense from an API if you were trying to use this stuff.

I want to be able to use this stuff to optimise filesystem I/O, but if the priority class I need to use is dependent on the elevator the *user selects* and can change dynamically, then I simply cannot make that optimisation.

Quoted 4 lines

Now the initial feedback was since this *implementation* is different from anything we have in CFQ which is our current *standard* way of thinking and comparing (that is the only thing that exists) why not make them into a new class :).

Because it make it impossible to optimise application code as the class that needs to be used is entirely dependent on the configuration of the machine that it is running on. Application writers are not going to probe the I/O scheduler the block device is using to determine if they should be using RT or LATENCY class prioritisation. From a user POV they do *exactly the same thing*, so they should use the same behavioural classes defined by the API.

Quoted 9 lines

Quoted 5 lines

Quoted 3 lines

I see your problem, we could make the LATENCY class different from and above BE class (instead of one-one mapping).

Like the RT class is currently defined to be? ;)

I agree with you and we could use RT (though you and I know that basically it is best effort). LATENCY was invented due to a previous suggestion.

As someone who is actually trying to use this stuff, I'm saying that the LATENCY suggestion was a *bad idea* because of the complexity it introduces when trying to optimise performance by applying I/O priorities to different I/O types. I want *one* API that is implemented by all schedulers, not an API per scheduler.....

Cheers,

Dave.
--
Dave Chinner
[email protected]

2008/10/28 Dave Chinner <[email protected]>:

Quoted 5 lines

Quoted 4 lines

Now the initial feedback was since this *implementation* is different from anything we have in CFQ which is our current *standard* way of thinking and comparing (that is the only thing that exists) why not make them into a new class :).

Quoted 7 lines

Because it make it impossible to optimise application code as the class that needs to be used is entirely dependent on the configuration of the machine that it is running on. Application writers are not going to probe the I/O scheduler the block device is using to determine if they should be using RT or LATENCY class prioritisation. From a user POV they do *exactly the same thing*, so they should use the same behavioural classes defined by the API.

I agree with you that we need an API which is valid across schedulers. But one has to agree that this sort of thing has it's own limitations. We are assuming that every scheduler which implements any kind of priority has a valid implementation of RT, BE, Idle class, which in this we we don't have. What happens tomorrow once we have a scheduler which decides that it needs to divide b/w. Which class would one map it to?

As I understand what you are asking for is: filesystem i/o can use BE 0 across all schedulers for journal updates. And you still have RT levels to take care of any higher priority i/o which need not wait for journal updates.

Here is what we can do:
1. Add 17 levels. top 8 RT, next 8 BE and last 1 idle. Though we know they all are similar in implementation. It's just that RT > BE > idle in importance.

And if the LATENCY camp is still active, add another
class LATENCY which in context of AS is same as RT. So you get to keep RT > BE and they get Latency.

2. Add 10 levels instead of current 8. top 1 level maps all 8 RT levels. next 8 are BE and last 1 maps to idle. This also gives you access to BE 0, while all RT levels are higher priority than BE. It discourages people from using different RT levels unless we find a new meaning for it in context of AS.