request: teach the device more intelligent

RFC PATCH In review Kyungmin Park 5 participants 17 messages Aug 9, 2011 – Aug 12, 2011

Subsystems (2)

Block layer MMC, SD and SDIO

lore

Share & tools

lore

Kyungmin Park

author

2011-08-09 03:47 UTC #0 lore raw

RFC PATCH 1/1 request: teach the device more intelligent

Hi Jens

Now eMMC device requires the upper layer information to improve the data performance and reliability.

. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead

. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.

With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.

Can you give your opinions and does it proper fields at requests?

Thank you,
Kyungmin Park
---

block/cfq-iosched.c +1 -0

index 1f96ad6..d73bfa7 100644
@@ -3790,6 +3790,7 @@ new_queue:
 	rq->elevator_private[0] = cic;
 	rq->elevator_private[1] = cfqq;
 	rq->elevator_private[2] = cfq_ref_get_cfqg(cfqq->cfqg);
+	rq->hint.context = (int) cfqq->pid;
 	spin_unlock_irqrestore(q->queue_lock, flags);
 	return 0;

include/linux/blkdev.h +9 -0

index 0e67c45..0145c10 100644
@@ -69,6 +69,14 @@ enum rq_cmd_type_bits {
 	REQ_TYPE_ATA_PC,
 };
 
+/*
+ * request hint
+ */
+struct request_hint {
+	int context;			/* Context ID */
+	int hot:1;			/* Hot/cold data */
+};
+
 #define BLK_MAX_CDB	16
 
 /*

@@ -139,6 +147,7 @@ struct request {
 
 	int ref_count;
 
+	struct request_hint hint;	/* request hint information */
 	void *special;		/* opaque pointer available for LLD use */
 	char *buffer;		/* kaddr of the current segment if available */

Kyungmin Park

author

2011-08-09 14:04 UTC #1 lore raw

Cc the correct email address of jens

Quoted 66 lines · Kyungmin Park

Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
Thank you,
Kyungmin Park
---
block/cfq-iosched.c +1 -0
index 1f96ad6..d73bfa7 100644
@@ -3790,6 +3790,7 @@ new_queue:
        rq->elevator_private[0] = cic;
        rq->elevator_private[1] = cfqq;
        rq->elevator_private[2] = cfq_ref_get_cfqg(cfqq->cfqg);
+       rq->hint.context = (int) cfqq->pid;
        spin_unlock_irqrestore(q->queue_lock, flags);
        return 0;
 
include/linux/blkdev.h +9 -0
index 0e67c45..0145c10 100644
@@ -69,6 +69,14 @@ enum rq_cmd_type_bits {
        REQ_TYPE_ATA_PC,
  };
 
+/*
+ * request hint
+ */
+struct request_hint {
+       int context;                    /* Context ID */
+       int hot:1;                      /* Hot/cold data */
+};
+
  #define BLK_MAX_CDB    16
 
  /*
@@ -139,6 +147,7 @@ struct request {
 
        int ref_count;
 
+       struct request_hint hint;       /* request hint information */
void *special; /* opaque pointer available for LLD use */ char *buffer; /* kaddr of the current segment if available */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Namhyung Kim

2011-08-10 02:16 UTC #2 lore raw

Quoted 44 lines · Kyungmin Park

Cc the correct email address of jens
Quoted 41 lines · Kyungmin Park
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
Thank you,
Kyungmin Park
---
block/cfq-iosched.c +1 -0
index 1f96ad6..d73bfa7 100644
@@ -3790,6 +3790,7 @@ new_queue:
        rq->elevator_private[0] = cic;
        rq->elevator_private[1] = cfqq;
        rq->elevator_private[2] = cfq_ref_get_cfqg(cfqq->cfqg);
+       rq->hint.context = (int) cfqq->pid;
        spin_unlock_irqrestore(q->queue_lock, flags);
        return 0;
 
include/linux/blkdev.h +0 -0
index 0e67c45..0145c10 100644
@@ -69,6 +69,14 @@ enum rq_cmd_type_bits {
        REQ_TYPE_ATA_PC,
  };
 

Quoted 6 lines

include/linux/blkdev.h enum rq_cmd_type_bits { · line 77
Quoted 6 lines
include/linux/blkdev.h +6 -0
+/*
+ * request hint
+ */
+struct request_hint {
+       int context;                    /* Context ID */
+       int hot:1;                      /* Hot/cold data */

This 1-bit member should be a type of unsigned?

Thanks.

Jens Axboe

maintainer

2011-08-09 19:47 UTC #3 lore raw

In reply to Kyungmin Park

Quoted 10 lines · Kyungmin Park

Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead

Quoted 7 lines

. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?

You need this to work on all IO schedulers, not just cfq. And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:

static int cfq_foo(struct request *rq)

{
        struct cfq_queue *cfqq = rq->elevator_private[1];

        if (cfqq)
                return cfqq->pid;

        return -1;
}

As to the hot part, I think that would be better as just a request flag like eg the meta flag.

Signature · 1 line

Jens Axboe

Kyungmin Park

author

2011-08-09 23:43 UTC #4 lore raw

Quoted 15 lines · Jens Axboe

Quoted 14 lines · Kyungmin Park
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.

Quoted 5 lines

Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.

Of course if the concept is acceptable, I'll add the other IO schedulers also.

Quoted 7 lines

And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];

Quoted 5 lines

if (cfqq)
return cfqq->pid;
return -1;
}

The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.

Quoted 3 lines

As to the hot part, I think that would be better as just a request flag like eg the meta flag.

Yes it can use the JBD_flags at cfq. but same reason it's not proper reference at device drivers. that's reason to make a filed at request.

Device driver should or must see the request data structure and don't refer the upper layer data structures.

Thank you,
Kyungmin Park

Jens Axboe

maintainer

2011-08-10 08:08 UTC #5 lore raw

Quoted 33 lines · Kyungmin Park

Quoted 15 lines · Jens Axboe
Quoted 14 lines · Kyungmin Park
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
       struct cfq_queue *cfqq = rq->elevator_private[1];

       if (cfqq)
               return cfqq->pid;

Quoted 7 lines

Quoted 2 lines
       return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.

No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.

It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.

Quoted 7 lines

Quoted 2 lines
As to the hot part, I think that would be better as just a request flag like eg the meta flag.
Yes it can use the JBD_flags at cfq. but same reason it's not proper reference at device drivers. that's reason to make a filed at request.
Device driver should or must see the request data structure and don't refer the upper layer data structures.

The device driver sees the struct request, which is where this flag ends up.

Signature · 1 line

Jens Axboe

Kyungmin Park

author

2011-08-10 08:42 UTC #6 lore raw

Quoted 48 lines · Jens Axboe

Quoted 33 lines · Kyungmin Park
Quoted 15 lines · Jens Axboe
Quoted 14 lines · Kyungmin Park
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];
if (cfqq)
return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.

Quoted 4 lines

It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.

Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?

The quick-n-dirty call is like this at "drivers/mmc/card/block.c"

                struct elevator_queue *e = md->queue.queue->elevator;
                int context = -1;

                if (e->ops->elevator_get_req_hint_fn && req) {
                        context = e->ops->elevator_get_req_hint_fn(req);

Quoted 11 lines

Quoted 8 lines
Quoted 3 lines
As to the hot part, I think that would be better as just a request flag like eg the meta flag.
Yes it can use the JBD_flags at cfq. but same reason it's not proper reference at device drivers. that's reason to make a filed at request.
Device driver should or must see the request data structure and don't refer the upper layer data structures.
The device driver sees the struct request, which is where this flag ends up.

It's ext{3,4} series specific detection method. if user use the another filesystem, it's difficult to detect hot/cold. I'll more think about it.

Thank you for suggestion.
Kyungmin Park

Quoted 5 lines

--
Jens Axboe

Shaohua Li

2011-08-10 08:47 UTC #7 lore raw

2011/8/10 Kyungmin Park <[email protected]>:

Quoted 56 lines · Jens Axboe

Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];
if (cfqq)
return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?

Quoted 7 lines

The quick-n-dirty call is like this at "drivers/mmc/card/block.c"
struct elevator_queue *e = md->queue.queue->elevator; int context = -1;
if (e->ops->elevator_get_req_hint_fn && req) { context = e->ops->elevator_get_req_hint_fn(req);

I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.

Thanks,
Shaohua

Kyungmin Park

author

2011-08-10 08:55 UTC #8 lore raw

Quoted 59 lines · Shaohua Li

2011/8/10 Kyungmin Park <[email protected]>:
Quoted 58 lines · Jens Axboe
Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];
if (cfqq)
return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?
The quick-n-dirty call is like this at "drivers/mmc/card/block.c"

Quoted 7 lines

Quoted 5 lines
struct elevator_queue *e = md->queue.queue->elevator; int context = -1;
if (e->ops->elevator_get_req_hint_fn && req) { context = e->ops->elevator_get_req_hint_fn(req);
I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.

In case of MMC. it doesn't need to persistent pid. it's enough to use the temporal context ID.
Please see the test output.

PID TTY STAT TIME COMMAND
...

  346 ?        S      0:01 [kworker/0:1]
  348 ?        S      0:01 [mmcqd/0]
  349 ?        S      0:00 [mmcqd/0boot0]
  350 ?        S      0:00 [mmcqd/0boot1]
  363 ?        S      0:00 [kworker/1:1]
  365 ?        S      0:00 [mmcqd/1]
  369 ?        S      0:00 [jbd2/mmcblk0p3-]
  370 ?        S<     0:00 [ext4-dio-unwrit]
  386 ?        S      0:00 [jbd2/mmcblk0p6-]
  387 ?        S<     0:00 [ext4-dio-unwrit]
  448 ttySAC2  Ss     0:00 -sh
  461 ?        Ss     0:00 /usr/sbin/inetd
  464 ttySAC2  R+     0:00 ps ax

[ 1021.153100] mmc_blk_issue_rq[1203] req df94e7e0, context 448 <- sh
[ 1021.236799] mmc_blk_issue_rq[1203] req df82c8c0, context 448

[ 1023.712727] mmc_blk_issue_rq[1203] req df82c9a0, context 464
[ 1023.722319] mmc_blk_issue_rq[1203] req df82c9a0, context 464
[ 1023.737660] mmc_blk_issue_rq[1203] req df94e700, context 464
[ 1023.744930] mmc_blk_issue_rq[1203] req df94e620, context 464

[ 1025.189639] mmc_blk_issue_rq[1203] req df82c7e0, context 369 <- jbd2
[ 1025.194334] mmc_blk_issue_rq[1203] req df82c9a0, context 369
[ 1025.200157] mmc_blk_issue_rq[1203] req df82cee0, context 369
[ 1025.213215] mmc_blk_issue_rq[1203] req df9a5b08, context 0 <- flush
[ 1025.217627] mmc_blk_issue_rq[1203] req df82cee0, context 0 <- flush

Quoted 8 lines

Thanks,
Shaohua
--
To unsubscribe from this list: send the line "unsubscribe linux-mmc" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jens Axboe

maintainer

2011-08-10 09:06 UTC #9 lore raw

In reply to Shaohua Li

Quoted 59 lines · Shaohua Li

2011/8/10 Kyungmin Park <[email protected]>:
Quoted 58 lines · Jens Axboe
Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
       struct cfq_queue *cfqq = rq->elevator_private[1];

       if (cfqq)
               return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
       return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?
The quick-n-dirty call is like this at "drivers/mmc/card/block.c"

Quoted 7 lines

Quoted 5 lines
               struct elevator_queue *e = md->queue.queue->elevator;
               int context = -1;

               if (e->ops->elevator_get_req_hint_fn && req) {
                       context = e->ops->elevator_get_req_hint_fn(req);
I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.

Any request with sched private data is drained prior to switching over. This problem isn't unique to this context id, we have other per-request IO scheduler data structures associated with the request, too.

Signature · 1 line

Jens Axboe

Shaohua Li

2011-08-11 00:29 UTC #10 lore raw

2011/8/10 Jens Axboe <[email protected]>:

Quoted 63 lines · Shaohua Li

Quoted 62 lines
2011/8/10 Kyungmin Park <[email protected]>:
Quoted 61 lines · Jens Axboe
Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];
if (cfqq)
return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?
The quick-n-dirty call is like this at "drivers/mmc/card/block.c"
struct elevator_queue *e = md->queue.queue->elevator; int context = -1;

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
if (e->ops->elevator_get_req_hint_fn && req) { context = e->ops->elevator_get_req_hint_fn(req);
I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.
Any request with sched private data is drained prior to switching over. This problem isn't unique to this context id, we have other per-request IO scheduler data structures associated with the request, too.

what I'm afraid is the context id isn't consistent. Say in cfq, context id for app1 is 1, app2 2. Then switching to deadline, context id for app1 is 2, app2 1. Will the driver be confused about this?

Thanks,
Shaohua

Kyungmin Park

author

2011-08-11 00:36 UTC #11 lore raw

Quoted 69 lines · Shaohua Li

2011/8/10 Jens Axboe <[email protected]>:
Quoted 68 lines · Shaohua Li
Quoted 66 lines
2011/8/10 Kyungmin Park <[email protected]>:
Quoted 56 lines · Jens Axboe
Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];
if (cfqq)
return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?
Quoted 7 lines
The quick-n-dirty call is like this at "drivers/mmc/card/block.c"
struct elevator_queue *e = md->queue.queue->elevator; int context = -1;
if (e->ops->elevator_get_req_hint_fn && req) { context = e->ops->elevator_get_req_hint_fn(req);
I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.

Quoted 6 lines

Quoted 3 lines
Any request with sched private data is drained prior to switching over. This problem isn't unique to this context id, we have other per-request IO scheduler data structures associated with the request, too.
what I'm afraid is the context id isn't consistent. Say in cfq, context id for app1 is 1, app2 2. Then switching to deadline, context id for app1 is 2, app2 1. Will the driver be confused about this?

No, no need to consistent. the context id id only valid when several requests are request the I/O simultaneously
e.g.,
App1 requests A, B, C, D, ...
App2 requests a, b, c, d, ...
App2 requests 1, 2, 3, 4, 5, ...
with following order, A, B, a, 1, C, b, 2, ...

The current eMMC can't handle these operation.

Instead using context, it can teach the these operation comes from using context ID. and finally can place the request in-order at card internally.

Open Context ID operation, perform I/O with context Id, ...., and Close Context ID operation until queue is empty.

Thank you,
Kyungmin Park

Quoted 4 lines

Thanks,
Shaohua

Kyungmin Park

author

2011-08-11 00:41 UTC #12 lore raw

Quoted 85 lines · Kyungmin Park

Quoted 69 lines · Shaohua Li
2011/8/10 Jens Axboe <[email protected]>:
Quoted 68 lines · Shaohua Li
Quoted 66 lines
2011/8/10 Kyungmin Park <[email protected]>:
Quoted 56 lines · Jens Axboe
Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];
if (cfqq)
return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?
Quoted 7 lines
The quick-n-dirty call is like this at "drivers/mmc/card/block.c"
struct elevator_queue *e = md->queue.queue->elevator; int context = -1;
if (e->ops->elevator_get_req_hint_fn && req) { context = e->ops->elevator_get_req_hint_fn(req);
I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.
Quoted 6 lines
Quoted 3 lines
Any request with sched private data is drained prior to switching over. This problem isn't unique to this context id, we have other per-request IO scheduler data structures associated with the request, too.
what I'm afraid is the context id isn't consistent. Say in cfq, context id for app1 is 1, app2 2. Then switching to deadline, context id for app1 is 2, app2 1. Will the driver be confused about this?
No, no need to consistent. the context id id only valid when several requests are request the I/O simultaneously
e.g.,
App1 requests A, B, C, D, ...
App2 requests a, b, c, d, ...
App2 requests 1, 2, 3, 4, 5, ...
with following order, A, B, a, 1, C, b, 2, ...

Quoted 1 line

The current eMMC can't handle these operation.

I mean it can't handle it efficiently.

Quoted 17 lines

Instead using context, it can teach the these operation comes from using context ID. and finally can place the request in-order at card internally.
Open Context ID operation, perform I/O with context Id, ...., and Close Context ID operation until queue is empty.
Thank you,
Kyungmin Park
Quoted 4 lines
Thanks,
Shaohua

Shaohua Li

2011-08-11 01:01 UTC #13 lore raw

In reply to Kyungmin Park

2011/8/11 Kyungmin Park <[email protected]>:

Quoted 85 lines · Shaohua Li

Quoted 69 lines
2011/8/10 Jens Axboe <[email protected]>:
Quoted 68 lines · Shaohua Li
Quoted 66 lines
2011/8/10 Kyungmin Park <[email protected]>:
Quoted 56 lines · Jens Axboe
Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context".
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
struct cfq_queue *cfqq = rq->elevator_private[1];
if (cfqq)
return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?
Quoted 7 lines
The quick-n-dirty call is like this at "drivers/mmc/card/block.c"
struct elevator_queue *e = md->queue.queue->elevator; int context = -1;
if (e->ops->elevator_get_req_hint_fn && req) { context = e->ops->elevator_get_req_hint_fn(req);
I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.
Quoted 6 lines
Quoted 3 lines
Any request with sched private data is drained prior to switching over. This problem isn't unique to this context id, we have other per-request IO scheduler data structures associated with the request, too.
what I'm afraid is the context id isn't consistent. Say in cfq, context id for app1 is 1, app2 2. Then switching to deadline, context id for app1 is 2, app2 1. Will the driver be confused about this?
No, no need to consistent. the context id id only valid when several requests are request the I/O simultaneously
e.g.,
App1 requests A, B, C, D, ...
App2 requests a, b, c, d, ...
App2 requests 1, 2, 3, 4, 5, ...
with following order, A, B, a, 1, C, b, 2, ...

Quoted 8 lines

The current eMMC can't handle these operation.
Instead using context, it can teach the these operation comes from using context ID. and finally can place the request in-order at card internally.
Open Context ID operation, perform I/O with context Id, ...., and Close Context ID operation until queue is empty.

so just sort the request according to context id in a small interval. Makes sense. Thanks for the explanation.

Thanks,
Shaohua

Vivek Goyal

2011-08-11 14:20 UTC #14 lore raw

In reply to Kyungmin Park

On Thu, Aug 11, 2011 at 09:36:04AM +0900, Kyungmin Park wrote:

[...]

Quoted 8 lines

No, no need to consistent. the context id id only valid when several requests are request the I/O simultaneously
e.g.,
App1 requests A, B, C, D, ...
App2 requests a, b, c, d, ...
App2 requests 1, 2, 3, 4, 5, ...
with following order, A, B, a, 1, C, b, 2, ...

Quoted 8 lines

The current eMMC can't handle these operation.
Instead using context, it can teach the these operation comes from using context ID. and finally can place the request in-order at card internally.
Open Context ID operation, perform I/O with context Id, ...., and Close Context ID operation until queue is empty.

Hi Kyungmin,

So once we have context information, we want to process all the requests from the same context together? IOW, we expect that driver will have multiple requests in flight from various context and it will batch the requests from same context together?

How does that help? Is it something related to seeking? If yes, then just setting the rotational=1 should help as CFQ will sequence all the sequential IO where it can. For random IO, anyway optimization will not help. May be it is about something else which I am completely missing. Some details here will help.

Thanks
Vivek

Kyungmin Park

author

2011-08-12 02:20 UTC #15 lore raw

Quoted 20 lines · Vivek Goyal

On Thu, Aug 11, 2011 at 09:36:04AM +0900, Kyungmin Park wrote:
[...]
Quoted 8 lines
No, no need to consistent. the context id id only valid when several requests are request the I/O simultaneously
e.g.,
App1 requests A, B, C, D, ...
App2 requests a, b, c, d, ...
App2 requests 1, 2, 3, 4, 5, ...
with following order, A, B, a, 1, C, b, 2, ...
Quoted 8 lines
The current eMMC can't handle these operation.
Instead using context, it can teach the these operation comes from using context ID. and finally can place the request in-order at card internally.
Open Context ID operation, perform I/O with context Id, ...., and Close Context ID operation until queue is empty.

Quoted 6 lines

Hi Kyungmin,
So once we have context information, we want to process all the requests from the same context together? IOW, we expect that driver will have multiple requests in flight from various context and it will batch the requests from same context together?

It's current implementation. Actually there's no context until eMMC v4.41. After eMMC v4.5. introduce the context and handle it efficiently if context is provided from host

Quoted 6 lines

How does that help? Is it something related to seeking? If yes, then just setting the rotational=1 should help as CFQ will sequence all the sequential IO where it can. For random IO, anyway optimization will not help. May be it is about something else which I am completely missing. Some details here will help.

No, it's dependent on firmware implementation at each vendor. but some mixed workload, several programs requests the sequential write, and one or two random write, it can assign the blocks one for sequential, another for ramdon. don't mix it if context is provided.

Thank you,
Kyungmin Park

Quoted 4 lines

Thanks
Vivek

Jens Axboe

maintainer

2011-08-11 07:56 UTC #16 lore raw

In reply to Shaohua Li

Quoted 69 lines · Shaohua Li

2011/8/10 Jens Axboe <[email protected]>:
Quoted 68 lines · Shaohua Li
Quoted 66 lines
2011/8/10 Kyungmin Park <[email protected]>:
Quoted 56 lines · Jens Axboe
Quoted 48 lines · Kyungmin Park
Quoted 33 lines · Jens Axboe
Quoted 15 lines · Kyungmin Park
Quoted 14 lines
Hi Jens
Now eMMC device requires the upper layer information to improve the data performance and reliability.
. Context ID
Using the context information, it can sort out the data internally and improve the performance. The main problem is that it's needed to define "What's the context"..
Actually I expect cfq queue has own unique ID but it doesn't so decide to use the pid instead
. Data Tag
Using the Data Tag (1-bit information), It writes the data at SLC area when it's hot data. So it can make the chip more reliable. First I expect the REQ_META but current ext4 doesn't pass the WRITE_META. only use the READ_META. so it needs to investigate it.
Quoted 5 lines
Quoted 3 lines
With these characteristics, it's helpful to teach the device. After some consideration. it's needed to pass out these information at request data structure.
Can you give your opinions and does it proper fields at requests?
You need this to work on all IO schedulers, not just cfq.
Of course if the concept is acceptable, I'll add the other IO schedulers also.
Quoted 10 lines
And since that's the case, there's no need to add this field since you can just retrieve it if the driver asks for it. For CFQ, it could look like this:
static int cfq_foo(struct request *rq)
{
       struct cfq_queue *cfqq = rq->elevator_private[1];

       if (cfqq)
               return cfqq->pid;
Quoted 7 lines
Quoted 2 lines
       return -1;
}
The actual user of these information is device driver. e.g., drivers/mmc/card/block.c
So it's not good to use cfq data structure at D/D. some time later these are also used at scsi device drivers.
No, what I'm suggesting above is the CFQ implementation. You would need to wire up an elv_ops->get_foo() and have the IO schedulers fill it in.. If you notice, the above function does not take or output anything related to CFQ in particular, it'll just return you the unique key you need.
Quoted 4 lines
It's either the above, or a field in the request that the schedulers fill out. However, it'd be somewhat annoying to grow struct request for something that has a narrow scope of use. Hence the suggestion to add a strategy helper for this.
Okay, I'll add new elevator function one for getting context or more hints. BTW, does it okay to call elevator function call at D/D?
Quoted 7 lines
The quick-n-dirty call is like this at "drivers/mmc/card/block.c"
               struct elevator_queue *e = md->queue.queue->elevator;
               int context = -1;

               if (e->ops->elevator_get_req_hint_fn && req) {
                       context = e->ops->elevator_get_req_hint_fn(req);
I'm wondering how the driver deal with elevator switch. A context id from one elevator might just be garbage for another elevator.

Quoted 6 lines

Quoted 3 lines
Any request with sched private data is drained prior to switching over. This problem isn't unique to this context id, we have other per-request IO scheduler data structures associated with the request, too.
what I'm afraid is the context id isn't consistent. Say in cfq, context id for app1 is 1, app2 2. Then switching to deadline, context id for app1 is 2, app2 1. Will the driver be confused about this?

It's a hint, so should not be a worry at all. Things should function perfectly fine with just returning 1 all the time, the idea is to allow some more efficiency in scheduling on the hw side if we can. Realistically, the device isn't going to be tracking a ton of pids anyway. In the rare event of an IO scheduler switch, things will settle down very quickly.

Signature · 1 line

Jens Axboe