Skip to contentSkip to conversation

pktcdvd stack usage regression

DISCUSSION Adrian Bunk 2 participants 4 messages Feb 9, 2006 – Feb 10, 2006 Block layer

lore
Share & tools
lore
#0 lore raw

Hi Phillip,

your recent patch "pktcdvd: Allow larger packets" changed PACKET_MAX_SIZE in the pktcdvd driver from 32 to 128.

Unfortunately, drivers/block/pktcdvd.c contains the following:

<-- snip -->

...
static void pkt_start_write(struct pktcdvd_device *pd, struct packet_data *pkt)

{
        struct bio *bio;
        struct page *pages[PACKET_MAX_SIZE];
        int offsets[PACKET_MAX_SIZE];

...

<-- snip -->

With PACKET_MAX_SIZE=128, this allocates more than 1 kB on the stack which is not acceptable considering that we might have only 4 kB stack altogether.

Please either fix this before 2.6.16 or ask Linus to revert commit 5c55ac9bbca22ee134408f83de5f2bda3b1b2a53.

TIA
Adrian

Signature · 4 lines
       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed
#1 lore raw
Quoted 16 lines · Adrian Bunk

Hi Phillip,

your recent patch "pktcdvd: Allow larger packets" changed PACKET_MAX_SIZE in the pktcdvd driver from 32 to 128.

Unfortunately, drivers/block/pktcdvd.c contains the following:

<-- snip -->

...
static void pkt_start_write(struct pktcdvd_device *pd, struct packet_data *pkt)

{
        struct bio *bio;
        struct page *pages[PACKET_MAX_SIZE];
        int offsets[PACKET_MAX_SIZE];
Quoted 5 lines

...

<-- snip -->

With PACKET_MAX_SIZE=128, this allocates more than 1 kB on the stack

Yes, I know.

Quoted 2 lines

which is not acceptable considering that we might have only 4 kB stack altogether.

Why is it not acceptable? The pkt_start_write() function is only called from the kcdrwd() kernel thread and the pkt_start_write() function doesn't call anything else in the kernel that could require lots of stack space.

The actual I/O is started from pkt_iosched_process_queue(), which calls generic_make_request(). However pkt_iosched_process_queue() is on a different call chain than pkt_start_write(), so I don't see how this could be a problem.

Signature · 2 lines
Peter Osterlund - [email protected]
http://web.telia.com/~u89404340
#2 lore raw
Quoted 34 lines · Peter Osterlund
Quoted 16 lines · Adrian Bunk

Hi Phillip,

your recent patch "pktcdvd: Allow larger packets" changed PACKET_MAX_SIZE in the pktcdvd driver from 32 to 128.

Unfortunately, drivers/block/pktcdvd.c contains the following:

<-- snip -->

...
static void pkt_start_write(struct pktcdvd_device *pd, struct packet_data *pkt)

{
        struct bio *bio;
        struct page *pages[PACKET_MAX_SIZE];
        int offsets[PACKET_MAX_SIZE];
Quoted 5 lines

...

<-- snip -->

With PACKET_MAX_SIZE=128, this allocates more than 1 kB on the stack

Yes, I know.

Quoted 2 lines

which is not acceptable considering that we might have only 4 kB stack altogether.

Why is it not acceptable? The pkt_start_write() function is only called from the kcdrwd() kernel thread and the pkt_start_write() function doesn't call anything else in the kernel that could require lots of stack space.

Quoted 4 lines

The actual I/O is started from pkt_iosched_process_queue(), which calls generic_make_request(). However pkt_iosched_process_queue() is on a different call chain than pkt_start_write(), so I don't see how this could be a problem.

You might be able to verify this is true today, but it is a bit fragile and other changes might always add even more stack usage in some places. Therefore, functions using 1 kB stack aren't a good idea.

As an exercise, pkt_open() currently uses more than 0,5 kB stack (kernel 2.6.16-rc2-mm1, i386, gcc 4.0). Try to understand where this stack usage comes from (hint: add up the stack usages of all only once used static functions gcc automatically inlines).

Quoted 1 line

Peter Osterlund

cu
Adrian

Signature · 4 lines
       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed
#3 lore raw
Quoted 15 lines · Adrian Bunk
Quoted 14 lines · Peter Osterlund
Quoted 1 line · Adrian Bunk

With PACKET_MAX_SIZE=128, this allocates more than 1 kB on the stack

Yes, I know.

Quoted 2 lines

which is not acceptable considering that we might have only 4 kB stack altogether.

Why is it not acceptable? The pkt_start_write() function is only called from the kcdrwd() kernel thread and the pkt_start_write() function doesn't call anything else in the kernel that could require lots of stack space.

Quoted 7 lines
Quoted 4 lines

The actual I/O is started from pkt_iosched_process_queue(), which calls generic_make_request(). However pkt_iosched_process_queue() is on a different call chain than pkt_start_write(), so I don't see how this could be a problem.

You might be able to verify this is true today, but it is a bit fragile and other changes might always add even more stack usage in some places.

I don't think that would happen in this driver, but it doesn't really matter, because I realized that those vectors aren't actually needed at all. This patch removes them:

pktcdvd: Reduce stack usage.

Reduce stack usage in the pkt_start_write() function.

---

 drivers/block/pktcdvd.c |   43 +++++++++++++++++--------------------------
 1 files changed, 17 insertions(+), 26 deletions(-)
drivers/block/pktcdvd.c +17 -26
index d794f2b..f783af7 100644
@@ -646,7 +646,7 @@ static void pkt_copy_bio_data(struct bio
  * b) The data can be used as cache to avoid read requests if we receive a
  *    new write request for the same zone.
  */
-static void pkt_make_local_copy(struct packet_data *pkt, struct page **pages, int *offsets)
+static void pkt_make_local_copy(struct packet_data *pkt, struct bio_vec *bvec)
 {
 	int f, p, offs;
 
@@ -654,15 +654,15 @@ static void pkt_make_local_copy(struct p
 	p = 0;
 	offs = 0;
 	for (f = 0; f < pkt->frames; f++) {
-		if (pages[f] != pkt->pages[p]) {
-			void *vfrom = kmap_atomic(pages[f], KM_USER0) + offsets[f];
+		if (bvec[f].bv_page != pkt->pages[p]) {
+			void *vfrom = kmap_atomic(bvec[f].bv_page, KM_USER0) + bvec[f].bv_offset;
 			void *vto = page_address(pkt->pages[p]) + offs;
 			memcpy(vto, vfrom, CD_FRAMESIZE);
 			kunmap_atomic(vfrom, KM_USER0);
-			pages[f] = pkt->pages[p];
-			offsets[f] = offs;
+			bvec[f].bv_page = pkt->pages[p];
+			bvec[f].bv_offset = offs;
 		} else {
-			BUG_ON(offsets[f] != offs);
+			BUG_ON(bvec[f].bv_offset != offs);
 		}
 		offs += CD_FRAMESIZE;
 		if (offs >= PAGE_SIZE) {
@@ -992,18 +992,17 @@ try_next_bio:
 static void pkt_start_write(struct pktcdvd_device *pd, struct packet_data *pkt)
 {
 	struct bio *bio;
-	struct page *pages[PACKET_MAX_SIZE];
-	int offsets[PACKET_MAX_SIZE];
 	int f;
 	int frames_write;
+	struct bio_vec *bvec = pkt->w_bio->bi_io_vec;
 
 	for (f = 0; f < pkt->frames; f++) {
-		pages[f] = pkt->pages[(f * CD_FRAMESIZE) / PAGE_SIZE];
-		offsets[f] = (f * CD_FRAMESIZE) % PAGE_SIZE;
+		bvec[f].bv_page = pkt->pages[(f * CD_FRAMESIZE) / PAGE_SIZE];
+		bvec[f].bv_offset = (f * CD_FRAMESIZE) % PAGE_SIZE;
 	}
 
 	/*
-	 * Fill-in pages[] and offsets[] with data from orig_bios.
+	 * Fill-in bvec with data from orig_bios.
 	 */
 	frames_write = 0;
 	spin_lock(&pkt->lock);
@@ -1025,11 +1024,11 @@ static void pkt_start_write(struct pktcd
 			}
 
 			if (src_bvl->bv_len - src_offs >= CD_FRAMESIZE) {
-				pages[f] = src_bvl->bv_page;
-				offsets[f] = src_bvl->bv_offset + src_offs;
+				bvec[f].bv_page = src_bvl->bv_page;
+				bvec[f].bv_offset = src_bvl->bv_offset + src_offs;
 			} else {
 				pkt_copy_bio_data(bio, segment, src_offs,
-						  pages[f], offsets[f]);
+						  bvec[f].bv_page, bvec[f].bv_offset);
 			}
 			src_offs += CD_FRAMESIZE;
 			frames_write++;
@@ -1043,7 +1042,7 @@ static void pkt_start_write(struct pktcd
 	BUG_ON(frames_write != pkt->write_size);
 
 	if (test_bit(PACKET_MERGE_SEGS, &pd->flags) || (pkt->write_size < pkt->frames)) {
-		pkt_make_local_copy(pkt, pages, offsets);
+		pkt_make_local_copy(pkt, bvec);
 		pkt->cache_valid = 1;
 	} else {
 		pkt->cache_valid = 0;
@@ -1056,17 +1055,9 @@ static void pkt_start_write(struct pktcd
 	pkt->w_bio->bi_bdev = pd->bdev;
 	pkt->w_bio->bi_end_io = pkt_end_io_packet_write;
 	pkt->w_bio->bi_private = pkt;
-	for (f = 0; f < pkt->frames; f++) {
-		if ((f + 1 < pkt->frames) && (pages[f + 1] == pages[f]) &&
-		    (offsets[f + 1] = offsets[f] + CD_FRAMESIZE)) {
-			if (!bio_add_page(pkt->w_bio, pages[f], CD_FRAMESIZE * 2, offsets[f]))
-				BUG();
-			f++;
-		} else {
-			if (!bio_add_page(pkt->w_bio, pages[f], CD_FRAMESIZE, offsets[f]))
-				BUG();
-		}
-	}
+	for (f = 0; f < pkt->frames; f++)
+		if (!bio_add_page(pkt->w_bio, bvec[f].bv_page, CD_FRAMESIZE, bvec[f].bv_offset))
+			BUG();
 	VPRINTK("pktcdvd: vcnt=%d\n", pkt->w_bio->bi_vcnt);
 
 	atomic_set(&pkt->io_wait, 1);
Signature · 2 lines
Peter Osterlund - [email protected]
http://web.telia.com/~u89404340