Skip to contentSkip to conversation

System unresponsitive when copying HD/HD

DISCUSSION Delta 7 participants 9 messages Feb 2, 2001 – Feb 5, 2001

Subsystems (8)
lore
Share & tools
lore

Delta

author
#0 lore raw

Hi,

I backup my linux partition once a month from my second IDE drive to an empty partition
on the first IDE disk. I have about 2.8 Gig to copy, and I use <cp -ax / /mnt/hd> to do the copy.

While cp is copying from the second hard disk to the first hard disk, I find my system performance
drop VERY sharply. X is sloppy, even bash takes many seconds to respond. I using two
recent IDE disk (Fudjisu 13 gig, Maxtor 20 Gig), so I'm wondering why the system is so slow? My mobo is a FIC SD11 and I have an athlon 550 Mhz.

I tried renicing the process priority to 20, but I don't see any improvement on system usability.
Hard disk activity is still frenzy, even if there are other task runnable (X, g++ jobs).
Note that I'm running as root when I'm doing the copy.

So I'd like to know why the linux kernel can't schedule the task less often?
I guess that copying file doesn't eat too many CPU cycles, but is running almost all the time
in kernel mode doing I/O... Is there a way to prevent a process from "hogging" the hard disk like that?
It's pretty annoying when the system is sluggish like that.

Thanks a lot,
Laurent Birtz <[email protected]>

If you wish to reply, I'm not on the list so reply directly to me.. ý:.žË›±Êâmçë¢kaŠÉb²ßìzwm…éb�ïîžË›±Êâmébžìÿ‘êçz_âžØ^n‡r¡ö¦zË�ëh™¨è­Ú&£ûàz¿äz¹Þ—ú+€ù^jÇ«y§m…á@A«a¶Úÿÿü0ÃûnÇú+ƒùd

#1 lore raw
Quoted 7 lines · Delta

While cp is copying from the second hard disk to the first hard disk, I find my system performance
drop VERY sharply. X is sloppy, even bash takes many seconds to respond. I using two
recent IDE disk (Fudjisu 13 gig, Maxtor 20 Gig), so I'm wondering why the system is so slow? My mobo is a FIC SD11 and I have an athlon 550 Mhz.

You don't say what kernel you're running. Some versions (2.2.16ish) have very bad interactive response under I/O load. 2.2.18 is supposed to be better, or try 2.2.19pre.

Also 'hdparm -u' may help, and turning on DMA if you're not using it.

-Doug
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
In reply to Delta
Quoted 7 lines · Delta

Hi,

I backup my linux partition once a month from my second IDE drive to an empty partition
on the first IDE disk. I have about 2.8 Gig to copy, and I use <cp -ax / /mnt/hd> to do the copy.

# cd / ; nice -n 20 tar -clf - . | (cd /mnt/hd; tar -xpf -)

Will probably do what you want.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#3 lore raw
In reply to Delta

I've noticed less responsive disk response on 2.4.0 vs. 2.2.17. For example -- I run vmware and suspend it frequently when I'm not using it. One of them requires a 158Mb save file. Before, I could suspend that one, then start another which reads in a smaller 50M save file. The smaller one would come up while the other was still saving. As of 2.4, the smaller one doesn't come up -- I can't even do an 'ls' until the big save finishes.

Now big image program has actually exited and I can close the window -- the disk writes are going on from the disk cache with 'kupdate' taking some minor fraction (<1%) of the CPU and the rest of the system being mostly idle.

If I have vmstat running, I notice blocks trickling out to the disk, 5sec averages 495,142,151,155,136,257,15,0. Note that the maximum read rate (hdparm -t) of this disk is in the 12-14M/s range. I'm getting about 1-5% of that on output with the system's disk subsystem being apparently unable to do anything else.

This is with IDE hard disk with DMA enabled.

a) is this expected performance on a large linear write?
b) should I expect other disk operations to be denied service as long as the write is 'flushing'?

-l
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#4 lore raw
Quoted 8 lines

If I have vmstat running, I notice blocks trickling out to the disk, 5sec averages 495,142,151,155,136,257,15,0. Note that the maximum read rate (hdparm -t) of this disk is in the 12-14M/s range. I'm getting about 1-5% of that on output with the system's disk subsystem being apparently unable to do anything else.

This is with IDE hard disk with DMA enabled.

a) is this expected performance on a large linear write?

No

Quoted 2 lines

b) should I expect other disk operations to be denied service as long as the write is 'flushing'?

No

But try 2.4.1 before worrying too much. That fixed a lot of the block performance problems I was seeing (2.4.1 ruins the VM performance under paging loads but the I/O speed is fixed ;))

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#5 lore raw
Quoted 3 lines · Alan Cox

But try 2.4.1 before worrying too much. That fixed a lot of the block performance problems I was seeing (2.4.1 ruins the VM performance under paging loads but the I/O speed is fixed ;))

---
Seems to have gotten a bit worse. Vmstat output after 'vmware' had completed
write -- but system unresponsive and writing out a 155M file...

 1  0  0      0 113960  47528 277152   0   0     0     0  397   861   1  24  75
 1  0  0      0 114060  47560 277152   0   0     4   350  432  1435   4  17  79
 0  0  1      0 127380  47560 266196   0   0     0   516  216   435   7   3  90
 1  0  1      0 127380  47560 266196   0   0     0   240  203   173   0   1  99
 0  0  1      0 127380  47560 266196   0   0     0   434  275   180   0   2  98
 1  0  1      0 127376  47560 266196   0   0     0   218  204   173   0   2  98
 0  0  1      0 127376  47560 266196   0   0     0   288  203   174   0   0 100
 0  0  1      0 127376  47560 266196   0   0     0   337  230   176   0   1  99
 0  0  1      0 127376  47560 266196   0   0     0   267  241   177   0   1  99
 0  0  1      0 127376  47560 266196   0   0     0   210  204   173   0   1  99
 0  0  1      0 127376  47560 266196   0   0     0   204  203   173   0   1  99
 0  0  1      0 127376  47560 266196   0   0     0   216  212   250   0   1  99
 0  0  1      0 127376  47560 266196   0   0     0   208  205   172   0   2  98
 0  0  1      0 127372  47560 266196   0   0     0   225  203   160   0   2  98
 0  0  1      0 127372  47560 266196   0   0     0   316  214   212   0   1  99
 1  0  1      0 127144  47560 266196   0   0     0   281  218   304   1   2  96
 0  0  0      0 127144  47560 266196   0   0     0     1  161   240   1   0  99
 0  0  0      0 127144  47560 266196   0   0     0     0  101   232   0   1  99 

---
What is the meaning of having a process in the 'w' column? On other
systems, I was used to that meaning an executable had been *swapped* out completely (as opposed to no pages mapped in) and that it meant your system vm was 'thrashing'. But that obviously isn't the case here.

Those columns are output from a 'vmstat 5'. Meaning it took about 70 seconds
to write out 158M. Or about 2.2M/s. That's probably not bad. It still locks up the system for over a minute though -- which is really undesirable performance for interactive use. I'm guessing the vmstat output numbers are showing 4K? 8K? blocks? 8K would about make sense for the 2.2M average.

Signature · 6 lines
Linda A Walsh                    | Trust Technology, Core Linux, SGI
[email protected]                      | Voice: (650) 933-5338
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
#6 lore raw
In reply to Alan Cox
Quoted 2 lines · LA Walsh

Seems to have gotten a bit worse. Vmstat output after 'vmware' had completed
write -- but system unresponsive and writing out a 155M file...

Your numbers seem way too much off to have much to do with i/o scheduling fairness, but there is a slight bug due to the merge and insertion scan being done at once now. So you could try with attached patch and see if it makes any difference whatsoever.

Quoted 5 lines

Those columns are output from a 'vmstat 5'. Meaning it took about 70 seconds
to write out 158M. Or about 2.2M/s. That's probably not bad. It still locks up the system for over a minute though -- which is really undesirable performance for interactive use. I'm guessing the vmstat output numbers are showing 4K? 8K? blocks? 8K would about make sense for the 2.2M average.

Most likely 4kB blocks. I'd say the numbers are pretty bad...

Signature · 1 line
Jens Axboe
block/elevator.c renamed from opt/kernel/linux-2.4.2-pre1/drivers/block/elevator.c +5 -2
@@ -35,6 +35,7 @@
 	struct list_head *entry = &q->queue_head;
 	unsigned int count = bh->b_size >> 9, ret = ELEVATOR_NO_MERGE;
 
+	*req = NULL;
 	while ((entry = entry->prev) != head) {
 		struct request *__rq = blkdev_entry_to_request(entry);
 
@@ -46,10 +47,13 @@
 			break;
 		}
 
+
 		if (__rq->sem)
 			continue;
 		if (__rq->cmd != rw)
 			continue;
+		if (!*req && BHRQ_IN_ORDER(bh, __rq))
+			*req = __rq;
 		if (__rq->rq_dev != bh->b_rdev)
 			continue;
 		if (__rq->nr_sectors + count > max_sectors)
@@ -65,8 +69,7 @@
 			__rq->elevator_sequence -= count;
 			*req = __rq;
 			break;
-		} else if (!*req && BHRQ_IN_ORDER(bh, __rq))
-			*req = __rq;
+		}
 	}
 
 	return ret;
block/ll_rw_blk.c renamed from opt/kernel/linux-2.4.2-pre1/drivers/block/ll_rw_blk.c +15 -3
@@ -628,11 +628,19 @@
 		    && atomic_read(&queued_sectors) < low_queued_sectors)
 			wake_up(&blk_buffers_wait);
 
+		if (!list_empty(&q->request_freelist[rw])) {
+			blk_refill_freelist(q, rw);
+			list_add(&req->table, &q->request_freelist[rw]);
+			if (waitqueue_active(&q->wait_for_request))
+				wake_up_nr(&q->wait_for_request, 2);
+			return;
+		}
+
 		/*
-		 * Add to pending free list and batch wakeups
+		 * free list is empty, add to pending free list and
+		 * batch wakeups
 		 */
 		list_add(&req->table, &q->pending_freelist[rw]);
-
 		if (++q->pending_free[rw] >= batch_requests) {
 			int wake_up = q->pending_free[rw];
 			blk_refill_freelist(q, rw);
@@ -705,7 +713,7 @@
 {
 	unsigned int sector, count;
 	int max_segments = MAX_SEGMENTS;
-	struct request * req = NULL, *freereq = NULL;
+	struct request *req, *freereq = NULL;
 	int rw_ahead, max_sectors, el_ret;
 	struct list_head *head, *insert_here;
 	int latency;
@@ -767,6 +775,10 @@
 
 	el_ret = elevator->elevator_merge_fn(q, &req, head, bh, rw,
 					     max_sectors, max_segments);
+	if (el_ret && q->head_active && !q->plugged
+	    && req->queue.prev == &q->queue_head)
+		BUG();
+
 	switch (el_ret) {
 
 		case ELEVATOR_BACK_MERGE:
#7 lore raw
In reply to Alan Cox
Quoted 3 lines · Alan Cox

But try 2.4.1 before worrying too much. That fixed a lot of the block performance problems I was seeing (2.4.1 ruins the VM performance under paging loads but the I/O speed is fixed ;))

The patch below should fix the paging performance.

I haven't had a whole lot of time to test this (and I won't have much time due to yet another LVM bug making my workstation unusable under 2.4 ;(), but it seems to curb worst-case behaviour during heavy swapping and IO load very fine.

On my home machine (64MB K6-2 500) I didn't manage to make my mp3 skip while under both heavy paging load and heavy IO load (using a few of the "standard" benchmark programs).

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...

		http://www.surriel.com/

http://www.conectiva.com/ http://distro.conectiva.com/

fs/buffer.c renamed from fs/buffer.c.orig +0 -10
@@ -1052,16 +1052,6 @@
 		return 0;
 	}
 
-	/*
-	 * If we are about to get low on free pages and
-	 * cleaning the inactive_dirty pages would help
-	 * fix this, wake up bdflush.
-	 */
-	shortage = free_shortage();
-	if (shortage && nr_inactive_dirty_pages > shortage &&
-			nr_inactive_dirty_pages > freepages.high)
-		return 0;
-
 	return -1;
 }
 
mm/filemap.c renamed from mm/filemap.c.orig +36 -9

70 lines not on this page · raw message

mm/page_alloc.c renamed from mm/page_alloc.c.orig +0 -15
@@ -299,21 +299,6 @@
 			!(current->flags & PF_MEMALLOC))
 		direct_reclaim = 1;
 
-	/*
-	 * If we are about to get low on free pages and we also have
-	 * an inactive page shortage, wake up kswapd.
-	 */
-	if (inactive_shortage() > inactive_target / 2 && free_shortage())
-		wakeup_kswapd();
-	/*
-	 * If we are about to get low on free pages and cleaning
-	 * the inactive_dirty pages would fix the situation,
-	 * wake up bdflush.
-	 */
-	else if (free_shortage() && nr_inactive_dirty_pages > free_shortage()
-			&& nr_inactive_dirty_pages >= freepages.high)
-		wakeup_bdflush(0);
-
 try_again:
 	/*
 	 * First, see if we have any zones with lots of free memory.
mm/page_io.c renamed from mm/page_io.c.orig +0 -5
@@ -42,11 +42,6 @@
 	int block_size;
 	struct inode *swapf = 0;
 
-	/* Don't allow too many pending pages in flight.. */
-	if ((rw == WRITE) && atomic_read(&nr_async_pages) >
-			pager_daemon.swap_cluster * (1 << page_cluster))
-		wait = 1;
-
 	if (rw == READ) {
 		ClearPageUptodate(page);
 		kstat.pswpin++;
mm/vmscan.c renamed from mm/vmscan.c.orig +119 -57

332 lines not on this page · raw message

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#8 lore raw
In reply to LA Walsh
Quoted 7 lines · LA Walsh

I've noticed less responsive disk response on 2.4.0 vs. 2.2.17. For example -- I run vmware and suspend it frequently when I'm not using it. One of them requires a 158Mb save file. Before, I could suspend that one, then start another which reads in a smaller 50M save file. The smaller one would come up while the other was still saving. As of 2.4, the smaller one doesn't come up -- I can't even do an 'ls' until the big save finishes.

[snip]

This could be related with (from the linux-mm bugzilla):

http://distro.conectiva.com/bugzilla/show_bug.cgi?id=1178

I have a patch against 2.4.1 and 2.4.1-ac which {c,sh}ould fix this bug, but I don't know if it will fix your situation. If your problem still happens with this patch, let me know and I'll see how I can fix it:

http://www.surriel.com/patches/2.4/2.4.1-vmpatch

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...

		http://www.surriel.com/

http://www.conectiva.com/ http://distro.conectiva.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/