scsi_debug issues · kernelthread

Nishanth Aravamudan

author

2004-10-15 19:01 UTC #0 lore raw

Hi,

At the recommendation of Pat Mansfield, I'm posting this issue to linux-scsi. I have run into a big problem with the scsi_debug driver which is causing my machine to hang.

I was finally able to get a large number (10000) of scsi_debug disks via modprobe (no partitioning, no mounting, no fs), so I decided to go ahead and try to continue the experiment. I found that I was not able to ls in a mounted directory from one of the scsi_debug disks. In the chance that it was somehow due to the high number of disks present, I tried just

modprobe scsi_debug

so that I would only get one disk. I then ran these commands [0]. I did not set the SCSI logging on until the last moment, after I synced the existent SCSI disks in the machine (there are two actual SCSI disks) [1]. I found that the ls command never fails to hang (although sometimes it will not hang immediately, I have to

cd lost+found

first. Interestingly, if the machine does not have highmem (i.e. less than 896 MB of RAM) or the kernel is booted with mem=800 (or some other number less than 896), the problem dissappears. I had to put the check in my patch in for the NULL dereference, as it would cause a segmentation fault otherwise. I'm guessing that is where the problem may be, as the NULL-check was not there before (an assumption that it (scatg2virt) would always work?).

I'm more than happy to run further tests, post further output, apply patches, etc. or anything else to resolve this bug. Please CC me on reply, as I'm not subscribed to linux-scsi, thanks.

-Nish

[0] commands run to produce hang:

/etc/init.d/klogd stop
/etc/init.d/sysklogd stop
modprobe scsi_debug
fdisk /dev/sdc (primary partition, default values selected) mkfs /dev/sdc1
mount /dev/sdc1 /home/test1
cd /home/test1
sync
echo 0xfffffff > /proc/sys/dev/scsi/logging_level ls

[1] hw/sw description

2-way 700MHz with 2GB RAM
2.6.9-rc3 kernel with patch[3] applied

[2] output of serial console from modprobe onwards:

scsi2 : scsi_debug, version 1.73 [20040518], dev_size_mb=8, opts=0x0
Vendor: Linux Model: scsi_debug Rev: 0004
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:scatg2virt(&sgpnt[0] [==c2268b60]) returned f7531000 unknown partition table
Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0 Attached scsi generic sg3 at scsi2, channel 0, id 0, lun 0, type 0 scatg2virt(&sgpnt[0] [==c23d6b80]) returned f7532000
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:scatg2virt(&sgpnt[0] [==c2268960]) returned f767c000 sdc1
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:scatg2virt(&sgpnt[0] [==f76efd60]) returned f7515000 sdc1
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7667000 scatg2virt(&sgpnt[0] [==c23d6b80]) returned f767d000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f76c6000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f74bb000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f74b7000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7ec9000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f75eb000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7452000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f74cd000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7ff8000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7e72000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f766d000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f74e4000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7689000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f76ab000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7586000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7632000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7554000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f745f000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7408000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f74ba000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7589000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7400000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f76b4000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f76b6000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f765c000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f765e000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f765f000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7674000 scatg2virt(&sgpnt[0] [==f76efe60]) returned f7675000 scatg2virt(&sgpnt[0] [==c23d6a80]) returned f7516000 scatg2virt(&sgpnt[0] [==c2268960]) returned f751b400 scatg2virt(&sgpnt[0] [==c2268960]) returned f751b800 scatg2virt(&sgpnt[0] [==c2268960]) returned f7661400
sd_init_command: disk=sdc, block=554, count=2
sdc : block=554
sdc : reading 2/2 512 byte blocks.
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 SUCCESS 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 70000
sd_rw_intr: sdc: res=0x70000
sd_rw_intr: sb[0,2,asc,ascq]=0,0,0,0
2 sectors total, 0 bytes done.
use_sg is 1
SCSI error : <2 0 0 0> return code = 0x70000
end_request: I/O error, dev sdc, sector 554
sd_init_command: disk=sdc, block=554, count=2 sdc : block=554
sdc : reading 2/2 512 byte blocks.
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030 scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 SUCCESS 70000 scsi2 : destination target 0, lun 0

        command = 0x28 00 00 00 02 2a 00 00 02 00

scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 70000
sd_rw_intr: sdc: res=0x70000
sd_rw_intr: sb[0,2,asc,ascq]=0,0,0,0
2 sectors total, 0 bytes done.
use_sg is 1
SCSI error : <2 0 0 0> return code = 0x70000
end_request: I/O error, dev sdc, sector 554
sd_init_command: disk=sdc, block=34, count=2 sdc : block=34
sdc : writing 2/2 512 byte blocks.
scsi_add_timer: scmd: f76e2ca0, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2ca0 scsi2 : destination target 0, lun 0

        command = 0x2a 00 00 00 00 22 00 00 02 00

buffer = 0xf76ef060, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
sd_init_command: disk=sda, block=7575, count=80
sda : block=7575
sda : writing 80/80 512 byte blocks.
scsi_add_timer: scmd: f76e2880, time: 30000, (c023dc50)
scsi <0:0:0:0> send 0xf76e2880 scsi0 : destination target 0, lun 0

        command = 0x2a 00 00 00 1d 97 00 00 50 00

buffer = 0xc23d6180, bufflen = 40960, done = 0xc0264770, queuecommand 0xc025b350 leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2880, rtn: 1
scsi <0:0:0:0> done 0xf76e2880 SUCCESS 0 scsi0 : destination target 0, lun 0

        command = 0x2a 00 00 00 1d 97 00 00 50 00

scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 0
sd_rw_intr: sda: res=0x0
80 sectors total, 40960 bytes done.
use_sg is 9
sd_init_command: disk=sda, block=7655, count=8 sda : block=7655
sda : writing 8/8 512 byte blocks.
scsi_add_timer: scmd: f76e2880, time: 30000, (c023dc50)
scsi <0:0:0:0> send 0xf76e2880 scsi0 : destination target 0, lun 0

        command = 0x2a 00 00 00 1d e7 00 00 08 00

buffer = 0xc2268ce0, bufflen = 4096, done = 0xc0264770, queuecommand 0xc025b350 leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2880, rtn: 1
scsi <0:0:0:0> done 0xf76e2880 SUCCESS 0 scsi0 : destination target 0, lun 0

        command = 0x2a 00 00 00 1d e7 00 00 08 00

scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 0
sd_rw_intr: sda: res=0x0
8 sectors total, 4096 bytes done.
use_sg is 1

[3] patch applied:

drivers/scsi/scsi_debug.c +14 -0

@@ -423,6 +423,10 @@ int scsi_debug_queuecommand(struct scsi_
 			num = cmd[4];
 		}
 		errsts = resp_read(SCpnt, upper_blk, block, num, devip);
+		if (errsts == (DID_ERROR << 16)) {
+			printk ("Invalid memory access in resp_read...\n");
+			break;
+		}
 		if (inj_recovered && (0 == errsts)) {
 			mk_sense_buffer(devip, RECOVERED_ERROR,
 					THRESHHOLD_EXCEEDED, 0, 18);

@@ -801,6 +805,12 @@ static int resp_read(struct scsi_cmnd * 
 		sgcount = 0;
 		sgpnt = (struct scatterlist *) buff;
 		buff = scatg2virt(&sgpnt[sgcount]);
+		printk("scatg2virt(&sgpnt[0] [==%p]) returned ", &sgpnt[sgcount]);
+		if (buff == NULL) {
+			printk("NULL\n");
+			return (DID_ERROR << 16);
+		}
+		printk("%p\n", buff);
 		bufflen = sgpnt[sgcount].length;
 	}
 	do {

@@ -1243,6 +1253,10 @@ static int schedule_resp(struct scsi_cmn
 			printk(KERN_INFO "scsi_debug: ... <%u %u %u %u> "
 			       "non-zero result=0x%x\n", sdp->host->host_no,
 			       sdp->channel, sdp->id, sdp->lun, scsi_result);
+			if (scsi_result == (DID_ERROR << 16)) {
+				printk("scsi_debug: WARNING: DID_ERROR "
+						"returned!\n");
+			}
 		}
 	}
 	if (cmnd && devip) {

Douglas Gilbert

2004-10-16 06:52 UTC #1 lore raw

Quoted 22 lines · Nishanth Aravamudan

Hi,
At the recommendation of Pat Mansfield, I'm posting this issue to linux-scsi. I have run into a big problem with the scsi_debug driver which is causing my machine to hang.
I was finally able to get a large number (10000) of scsi_debug disks via modprobe (no partitioning, no mounting, no fs), so I decided to go ahead and try to continue the experiment. I found that I was not able to ls in a mounted directory from one of the scsi_debug disks. In the chance that it was somehow due to the high number of disks present, I tried just
modprobe scsi_debug
so that I would only get one disk. I then ran these commands [0]. I did not set the SCSI logging on until the last moment, after I synced the existent SCSI disks in the machine (there are two actual SCSI disks) [1]. I found that the ls command never fails to hang (although sometimes it will not hang immediately, I have to
cd lost+found

Quoted 7 lines

first. Interestingly, if the machine does not have highmem (i.e. less than 896 MB of RAM) or the kernel is booted with mem=800 (or some other number less than 896), the problem dissappears. I had to put the check in my patch in for the NULL dereference, as it would cause a segmentation fault otherwise. I'm guessing that is where the problem may be, as the NULL-check was not there before (an assumption that it (scatg2virt) would always work?).

Nishanth,
So this problem seems related to highmem.

The attachment is against the current scsi_debug driver (at least in lk 2.6.9-rc4). It uses dma_map_sg() and friends together with phys_to_virt() which comes highly recommended:
"in almost all conceivable cases a device driver should not be using this function" :-) Could you try the patch.

Perhaps others my be able to answer this: is setting scsi_host_template::dma_boundary (to some figure other than 0xffffffff) going to help in this case?

Doug Gilbert

drivers/scsi/scsi_debug.c2681dma1 renamed from drivers/scsi/scsi_debug.c +34 ~~-25~~

@@ -262,16 +262,6 @@
 static struct device pseudo_primary;
 static struct bus_type pseudo_lld_bus;
 
-static unsigned char * scatg2virt(const struct scatterlist * sclp)
-{
-	if (NULL == sclp)
-		return NULL;
-	else if (sclp->page)
-		return (unsigned char *)page_address(sclp->page) +
-		       sclp->offset;
-	else
-		return NULL;
-}
 
 static
 int scsi_debug_queuecommand(struct scsi_cmnd * SCpnt, done_funct_t done)

@@ -286,6 +276,7 @@
 	struct sdebug_dev_info * devip = NULL;
 	unsigned char * sbuff;
 	int inj_recovered = 0;
+        int num_elems = 0;
 
 	if (done == NULL)
 		return 0;	/* assume mid level reprocessing command */

@@ -294,8 +285,12 @@
 		struct scatterlist *sgpnt = (struct scatterlist *)
 						SCpnt->request_buffer;
 
-		buff = scatg2virt(&sgpnt[0]);
-		bufflen = sgpnt[0].length;
+		num_elems = dma_map_sg(NULL, sgpnt, SCpnt->use_sg,
+				       SCpnt->sc_data_direction);
+		buff = phys_to_virt(sg_dma_address(&sgpnt[0]));
+		bufflen = sg_dma_len(&sgpnt[0]);
+		dma_unmap_sg(NULL, sgpnt, SCpnt->use_sg,
+			     SCpnt->sc_data_direction);
 		/* READ and WRITE process scatterlist themselves */
 	}
 	else

@@ -778,6 +773,7 @@
         struct scatterlist *sgpnt = NULL;
         int bufflen = SCpnt->request_bufflen;
 	unsigned long iflags;
+        int num_elems = 0;
 
 	if (upper_blk || (block + num > sdebug_capacity)) {
 		mk_sense_buffer(devip, ILLEGAL_REQUEST, ADDR_OUT_OF_RANGE,

@@ -799,9 +795,11 @@
 	       block, bufflen); */
 	if (SCpnt->use_sg) {
 		sgcount = 0;
-		sgpnt = (struct scatterlist *) buff;
-		buff = scatg2virt(&sgpnt[sgcount]);
-		bufflen = sgpnt[sgcount].length;
+		sgpnt = (struct scatterlist *)buff;
+		num_elems = dma_map_sg(NULL, sgpnt, SCpnt->use_sg,
+				       SCpnt->sc_data_direction);
+		buff = phys_to_virt(sg_dma_address(&sgpnt[0]));
+		bufflen = sg_dma_len(&sgpnt[0]);
 	}
 	do {
 		memcpy(buff, fake_storep + (block * SECT_SIZE), bufflen);

@@ -809,14 +807,18 @@
 		if (SCpnt->use_sg) {
 			block += bufflen >> POW2_SECT_SIZE;
 			sgcount++;
-			if (nbytes) {
-				buff = scatg2virt(&sgpnt[sgcount]);
-				bufflen = sgpnt[sgcount].length;
+			if (nbytes > 0) {
+				buff = phys_to_virt(
+					sg_dma_address(&sgpnt[sgcount]));
+				bufflen = sg_dma_len(&sgpnt[sgcount]);
 			}
 		} else if (nbytes > 0)
 			printk(KERN_WARNING "scsi_debug:resp_read: unexpected "
 			       "nbytes=%d\n", nbytes);
-	} while (nbytes);
+	} while (nbytes > 0);
+	if (SCpnt->use_sg)
+		dma_unmap_sg(NULL, sgpnt, SCpnt->use_sg,
+			     SCpnt->sc_data_direction);
 	read_unlock_irqrestore(&atomic_rw, iflags);
 	return 0;
 }

@@ -829,6 +831,7 @@
         struct scatterlist *sgpnt = NULL;
         int bufflen = SCpnt->request_bufflen;
 	unsigned long iflags;
+        int num_elems = 0;
 
 	if (upper_blk || (block + num > sdebug_capacity)) {
 		mk_sense_buffer(devip, ILLEGAL_REQUEST, ADDR_OUT_OF_RANGE,

@@ -842,8 +845,10 @@
 	if (SCpnt->use_sg) {
 		sgcount = 0;
 		sgpnt = (struct scatterlist *) buff;
-		buff = scatg2virt(&sgpnt[sgcount]);
-		bufflen = sgpnt[sgcount].length;
+		num_elems = dma_map_sg(NULL, sgpnt, SCpnt->use_sg,
+				       SCpnt->sc_data_direction);
+		buff = phys_to_virt(sg_dma_address(&sgpnt[0]));
+		bufflen = sg_dma_len(&sgpnt[0]);
 	}
 	do {
 		memcpy(fake_storep + (block * SECT_SIZE), buff, bufflen);

@@ -852,14 +857,18 @@
 		if (SCpnt->use_sg) {
 			block += bufflen >> POW2_SECT_SIZE;
 			sgcount++;
-			if (nbytes) {
-				buff = scatg2virt(&sgpnt[sgcount]);
-				bufflen = sgpnt[sgcount].length;
+			if (nbytes > 0) {
+				buff = phys_to_virt(
+					sg_dma_address(&sgpnt[sgcount]));
+				bufflen = sg_dma_len(&sgpnt[sgcount]);
 			}
 		} else if (nbytes > 0)
 			printk(KERN_WARNING "scsi_debug:resp_write: "
 			       "unexpected nbytes=%d\n", nbytes);
-	} while (nbytes);
+	} while (nbytes > 0);
+	if (SCpnt->use_sg)
+		dma_unmap_sg(NULL, sgpnt, SCpnt->use_sg,
+			     SCpnt->sc_data_direction);
 	write_unlock_irqrestore(&atomic_rw, iflags);
 	return 0;
 }

Christoph Hellwig

2004-10-16 10:50 UTC #2 lore raw

A dma address is not a physical address. You need to kmap the virtual addresses in the sg lists to get a kernel virtual address.

James Bottomley

2004-10-16 13:13 UTC #3 lore raw

In reply to Douglas Gilbert

Quoted 11 lines · Douglas Gilbert

So this problem seems related to highmem.
The attachment is against the current scsi_debug driver (at least in lk 2.6.9-rc4). It uses dma_map_sg() and friends together with phys_to_virt() which comes highly recommended:
"in almost all conceivable cases a device driver should not be using this function" :-) Could you try the patch.
Perhaps others my be able to answer this: is setting scsi_host_template::dma_boundary (to some figure other than 0xffffffff) going to help in this case?

No, the problem is with highmem as you correctly point out, but the issue is that your driver cannot see beyond it. On x86, highmem begins at about 900Mb. The kernel has no page tables for any memory beyond this. In order to see the memory you need to kmap it (i.e. ask the kernel to create a temporary page table for it); so the virtual address sg uses should be got by kmapping the pages in the sg list:

kaddr = kmap(sg[i].page) + sg[i].offset

If you make sure clustering is disabled, you should never get multiple pages in the initial sg list.

When you've finished you need to release the mapping with kunmap().

There are also atomic versions of these depending on where you are.

James

Douglas Gilbert

2004-10-18 13:47 UTC #4 lore raw

PATCH 1/1 scsi_debug [was: scsi_debug issues]

Quoted 22 lines · James Bottomley

Quoted 11 lines · Douglas Gilbert
So this problem seems related to highmem.
The attachment is against the current scsi_debug driver (at least in lk 2.6.9-rc4). It uses dma_map_sg() and friends together with phys_to_virt() which comes highly recommended:
"in almost all conceivable cases a device driver should not be using this function" :-) Could you try the patch.
Perhaps others my be able to answer this: is setting scsi_host_template::dma_boundary (to some figure other than 0xffffffff) going to help in this case?
No, the problem is with highmem as you correctly point out, but the issue is that your driver cannot see beyond it. On x86, highmem begins at about 900Mb. The kernel has no page tables for any memory beyond this. In order to see the memory you need to kmap it (i.e. ask the kernel to create a temporary page table for it); so the virtual address sg uses should be got by kmapping the pages in the sg list:

Quoted 4 lines

kaddr = kmap(sg[i].page) + sg[i].offset
If you make sure clustering is disabled, you should never get multiple pages in the initial sg list.

done

Quoted 1 line

When you've finished you need to release the mapping with kunmap().

ouch, that requires a re-org

Quoted 1 line

There are also atomic versions of these depending on where you are.

hopefully not required

The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .

Nishanth, could you test this with highmem?

Changelog:
- use kmap/kunmap to handle normal and highmem cases
- disable clustering
- previously could only handle 'use_sg > 1' for read and write commands
- VERIFY (SBC) and REWIND (SSC) command support (dummies)
- 'dsense' parameter (sysfs + load time) for descriptor sense format
- allow negative 'every_nth' to fail continually after |every_nth| commands (or until sysfs intervention)
- clean up debug messages sent to the log (when opts=1)
- correct ordering of log messages
- set scsi_cmnd::resid when underflow on DMA_FROM_DEVICE operations
- clean up std inquiry response; add version descriptors
- reject mode sense subpage commands since no subpages currently supported (i.e. better command filtering)

Doug Gilbert

Nishanth Aravamudan

author

2004-10-18 18:37 UTC #5 lore raw

Re: [PATCH] scsi_debug [was: scsi_debug issues]

Quoted 4 lines · Douglas Gilbert

Quoted 3 lines · James Bottomley
Quoted 1 line · Douglas Gilbert
So this problem seems related to highmem.

<snip>

Quoted 7 lines

The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .
Nishanth, could you test this with highmem?

Doug,

I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s occur (complaining about sleeping in an invalid context) and a final panic during mkfs :) I think something is still wrong... ;) Output is below and attached.

-Nish

scsi2 : scsi_debug, version 1.74 [20041018], dev_size_mb=8, opts=0x0
Vendor: Linux Model: scsi_debug Rev: 0004
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a10d>] register_disk+0xdd/0x190
 [<c02255d0>] blk_register_region+0x40/0x50
 [<c022568c>] add_disk+0x4c/0x60
 [<c0225610>] exact_match+0x0/0x10
 [<c0225620>] exact_lock+0x0/0x20
 [<c0265e0f>] sd_probe+0x24f/0x3a0
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<c0243e52>] scsi_sysfs_add_sdev+0x52/0x200
 [<f88c5880>] scsi_debug_slave_configure+0x0/0xa0 [scsi_debug]
 [<c0242674>] scsi_add_lun+0x2f4/0x3b0
 [<c0242809>] scsi_probe_and_add_lun+0xd9/0x240
 [<c02430b8>] scsi_scan_target+0xa8/0x130
 [<c02431c9>] scsi_scan_channel+0x89/0xa0
 [<c02432e1>] scsi_scan_host_selected+0x101/0x140
 [<c0243350>] scsi_scan_host+0x30/0x40
 [<f88c6d8d>] sdebug_driver_probe+0x9d/0xd0 [scsi_debug]
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<f88c6bbf>] sdebug_add_adapter+0x12f/0x1f0 [scsi_debug]
 [<f88cf36c>] scsi_debug_init+0x16c/0x1d2 [scsi_debug]
 [<c0132f59>] sys_init_module+0x199/0x230
 [<c01060cf>] syscall_call+0x7/0xb
 unknown partition table

Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg3 at scsi2, channel 0, id 0, lun 0, type 0
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5 in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c48ab>] fetch_to_dev_buffer+0x9b/0x140 [scsi_debug]
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221da0>] blk_unplug_work+0x10/0x20
 [<c012ab79>] worker_thread+0x1b9/0x260
 [<c0221d90>] blk_unplug_work+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c012a9c0>] worker_thread+0x0/0x260
 [<c012edba>] kthread+0xba/0xc0
 [<c012ed00>] kthread+0x0/0xc0
 [<c0104145>] kernel_thread_helper+0x5/0x10

SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1

SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c015527f>] sync_buffer+0x3f/0x50
 [<c0155408>] __wait_on_buffer+0xa8/0xb0
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0157706>] __block_prepare_write+0x156/0x440
 [<c01ef2cf>] radix_tree_node_alloc+0x1f/0x70
 [<c0158224>] block_prepare_write+0x34/0x50
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c013675f>] generic_file_buffered_write+0x1cf/0x600
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c016f540>] inode_update_time+0xd0/0xe0
 [<c0136e1a>] generic_file_aio_write_nolock+0x28a/0x4c0
 [<c01370f3>] generic_file_write_nolock+0xa3/0xc0
 [<c011328c>] do_page_fault+0x1cc/0x5f1
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<c015d038>] blkdev_file_write+0x38/0x40
 [<c01540d8>] vfs_write+0xb8/0x130
 [<c0154221>] sys_write+0x51/0x80
 [<c01060cf>] syscall_call+0x7/0xb
------------[ cut here ]------------
kernel BUG at arch/i386/mm/highmem.c:14!
invalid operand: 0000 [#1]
SMP 
Modules linked in: scsi_debug
CPU:    1
EIP:    0060:[<c0114784>]    Not tainted VLI
EFLAGS: 00010006   (2.6.9-rc4)

EIP is at kunmap+0x14/0x30
eax: f7f82000 ebx: 00001000 ecx: 00000000 edx: c16eeb00
esi: f7759000 edi: f93ea000 ebp: f75e1c00 esp: f7f83bf4
ds: 007b es: 007b ss: 0068
Process mkfs.ext2 (pid: 1573, threadinfo=f7f82000 task=f7694150)

Stack: f88c4906 c16eeb00 f7f83bfc f7f83bfc 00000000 00000000 00000000 00000006 
       00000000 f93e9000 0001c000 f88c563d f7e015c0 f93e9000 0001c000 00004000 
       00000000 000000e0 f7e01614 00000000 f88c44a0 f7e015c0 00000000 00003f20 
Call Trace:
 [<f88c4906>] fetch_to_dev_buffer+0xf6/0x140 [scsi_debug]
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c023b2b3>] scsi_put_command+0x73/0xa0
 [<c0221e7d>] blk_run_queue+0x2d/0x50
 [<c02405fd>] scsi_end_request+0xdd/0xf0
 [<c024092c>] scsi_io_completion+0x15c/0x4d0
 [<c013d970>] cache_alloc_refill+0x150/0x230
 [<c0264ab8>] sd_rw_intr+0x88/0x310
 [<c023deba>] scsi_delete_timer+0x1a/0x70
 [<c023bd11>] scsi_finish_command+0x81/0xe0
 [<c023bc0e>] scsi_softirq+0xbe/0xf0
 [<c011f6ba>] __do_softirq+0xba/0xd0
 [<c011f6fd>] do_softirq+0x2d/0x30
 [<c01106fd>] smp_apic_timer_interrupt+0x8d/0x100
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c0106abe>] apic_timer_interrupt+0x1a/0x20
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c023b877>] scsi_dispatch_cmd+0x187/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c0134875>] wait_on_page_bit+0xe5/0xf0
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c013437c>] wait_on_page_writeback_range+0x10c/0x150
 [<c0134541>] filemap_fdatawait+0x71/0x80
 [<c0155599>] sync_blockdev+0x39/0x50
 [<c0155aa7>] sys_fsync+0xb7/0xf0
 [<c01060cf>] syscall_call+0x7/0xb
Code: d8 8b 5c 24 08 83 c4 0c e9 6a d5 02 00 8d 76 00 8d bc 27 00 00 00 00 b8 00 e0 ff ff 8b 54 24 04 21 e0 f7 40 14 00 ff ff 00 74 08 <0f> 0b 0e 00 82 d4 30 c0 3b 15 34 cb 44 c0 73 01 c3 89 d0 e9 e4 
 <0>Kernel panic - not syncing: Fatal exception in interrupt

scsi2 : scsi_debug, version 1.74 [20041018], dev_size_mb=8, opts=0x0
Vendor: Linux Model: scsi_debug Rev: 0004
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a10d>] register_disk+0xdd/0x190
 [<c02255d0>] blk_register_region+0x40/0x50
 [<c022568c>] add_disk+0x4c/0x60
 [<c0225610>] exact_match+0x0/0x10
 [<c0225620>] exact_lock+0x0/0x20
 [<c0265e0f>] sd_probe+0x24f/0x3a0
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<c0243e52>] scsi_sysfs_add_sdev+0x52/0x200
 [<f88c5880>] scsi_debug_slave_configure+0x0/0xa0 [scsi_debug]
 [<c0242674>] scsi_add_lun+0x2f4/0x3b0
 [<c0242809>] scsi_probe_and_add_lun+0xd9/0x240
 [<c02430b8>] scsi_scan_target+0xa8/0x130
 [<c02431c9>] scsi_scan_channel+0x89/0xa0
 [<c02432e1>] scsi_scan_host_selected+0x101/0x140
 [<c0243350>] scsi_scan_host+0x30/0x40
 [<f88c6d8d>] sdebug_driver_probe+0x9d/0xd0 [scsi_debug]
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<f88c6bbf>] sdebug_add_adapter+0x12f/0x1f0 [scsi_debug]
 [<f88cf36c>] scsi_debug_init+0x16c/0x1d2 [scsi_debug]
 [<c0132f59>] sys_init_module+0x199/0x230
 [<c01060cf>] syscall_call+0x7/0xb
 unknown partition table

Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg3 at scsi2, channel 0, id 0, lun 0, type 0
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5 in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c48ab>] fetch_to_dev_buffer+0x9b/0x140 [scsi_debug]
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221da0>] blk_unplug_work+0x10/0x20
 [<c012ab79>] worker_thread+0x1b9/0x260
 [<c0221d90>] blk_unplug_work+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c012a9c0>] worker_thread+0x0/0x260
 [<c012edba>] kthread+0xba/0xc0
 [<c012ed00>] kthread+0x0/0xc0
 [<c0104145>] kernel_thread_helper+0x5/0x10

SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1

SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1

 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c015527f>] sync_buffer+0x3f/0x50
 [<c0155408>] __wait_on_buffer+0xa8/0xb0
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0157706>] __block_prepare_write+0x156/0x440
 [<c01ef2cf>] radix_tree_node_alloc+0x1f/0x70
 [<c0158224>] block_prepare_write+0x34/0x50
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c013675f>] generic_file_buffered_write+0x1cf/0x600
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c016f540>] inode_update_time+0xd0/0xe0
 [<c0136e1a>] generic_file_aio_write_nolock+0x28a/0x4c0
 [<c01370f3>] generic_file_write_nolock+0xa3/0xc0
 [<c011328c>] do_page_fault+0x1cc/0x5f1
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<c015d038>] blkdev_file_write+0x38/0x40
 [<c01540d8>] vfs_write+0xb8/0x130
 [<c0154221>] sys_write+0x51/0x80
 [<c01060cf>] syscall_call+0x7/0xb
------------[ cut here ]------------
kernel BUG at arch/i386/mm/highmem.c:14!
invalid operand: 0000 [#1]
SMP 
Modules linked in: scsi_debug
CPU:    1
EIP:    0060:[<c0114784>]    Not tainted VLI
EFLAGS: 00010006   (2.6.9-rc4)

EIP is at kunmap+0x14/0x30
eax: f7f82000 ebx: 00001000 ecx: 00000000 edx: c16eeb00
esi: f7759000 edi: f93ea000 ebp: f75e1c00 esp: f7f83bf4
ds: 007b es: 007b ss: 0068
Process mkfs.ext2 (pid: 1573, threadinfo=f7f82000 task=f7694150)

Stack: f88c4906 c16eeb00 f7f83bfc f7f83bfc 00000000 00000000 00000000 00000006 
       00000000 f93e9000 0001c000 f88c563d f7e015c0 f93e9000 0001c000 00004000 
       00000000 000000e0 f7e01614 00000000 f88c44a0 f7e015c0 00000000 00003f20 
Call Trace:
 [<f88c4906>] fetch_to_dev_buffer+0xf6/0x140 [scsi_debug]
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c023b2b3>] scsi_put_command+0x73/0xa0
 [<c0221e7d>] blk_run_queue+0x2d/0x50
 [<c02405fd>] scsi_end_request+0xdd/0xf0
 [<c024092c>] scsi_io_completion+0x15c/0x4d0
 [<c013d970>] cache_alloc_refill+0x150/0x230
 [<c0264ab8>] sd_rw_intr+0x88/0x310
 [<c023deba>] scsi_delete_timer+0x1a/0x70
 [<c023bd11>] scsi_finish_command+0x81/0xe0
 [<c023bc0e>] scsi_softirq+0xbe/0xf0
 [<c011f6ba>] __do_softirq+0xba/0xd0
 [<c011f6fd>] do_softirq+0x2d/0x30
 [<c01106fd>] smp_apic_timer_interrupt+0x8d/0x100
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c0106abe>] apic_timer_interrupt+0x1a/0x20
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c023b877>] scsi_dispatch_cmd+0x187/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c0134875>] wait_on_page_bit+0xe5/0xf0
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c013437c>] wait_on_page_writeback_range+0x10c/0x150
 [<c0134541>] filemap_fdatawait+0x71/0x80
 [<c0155599>] sync_blockdev+0x39/0x50
 [<c0155aa7>] sys_fsync+0xb7/0xf0
 [<c01060cf>] syscall_call+0x7/0xb
Code: d8 8b 5c 24 08 83 c4 0c e9 6a d5 02 00 8d 76 00 8d bc 27 00 00 00 00 b8 00 e0 ff ff 8b 54 24 04 21 e0 f7 40 14 00 ff ff 00 74 08 <0f> 0b 0e 00 82 d4 30 c0 3b 15 34 cb 44 c0 73 01 c3 89 d0 e9 e4 
 <0>Kernel panic - not syncing: Fatal exception in interrupt

Douglas Gilbert

2004-10-18 22:07 UTC #6 lore raw

Re: [PATCH] scsi_debug [was: scsi_debug issues]

Quoted 21 lines · Nishanth Aravamudan

Quoted 6 lines · Douglas Gilbert
Quoted 4 lines · James Bottomley
Quoted 1 line · Douglas Gilbert
So this problem seems related to highmem.
<snip>
Quoted 7 lines
The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .
Nishanth, could you test this with highmem?

Quoted 6 lines

Doug,
I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s occur (complaining about sleeping in an invalid context) and a final panic during mkfs :) I think something is still wrong... ;) Output is below and attached.

Ok, it looks like kmap_atomic() is needed.
Could you try this additional patch.

Doug Gilbert

drivers/scsi/scsi_debug.c174b renamed from drivers/scsi/scsi_debug.c +6 -5

@@ -56,7 +56,7 @@
 #include "scsi_debug.h"
 
 #define SCSI_DEBUG_VERSION "1.74"
-static const char * scsi_debug_version_date = "20041018";
+static const char * scsi_debug_version_date = "20041019";
 
 /* Additional Sense Code (ASC) used */
 #define NO_ADDED_SENSE 0x0

@@ -505,7 +505,8 @@
 	active = 1;
 	for (k = 0, req_len = 0, act_len = 0; k < scp->use_sg; ++k, ++sgpnt) {
 		if (active) {
-			kaddr = (unsigned char *)kmap(sgpnt->page);
+			kaddr = (unsigned char *)
+				kmap_atomic(sgpnt->page, KM_USER0);
 			if (NULL == kaddr)
 				return (DID_ERROR << 16);
 			kaddr = (unsigned char *)kaddr + sgpnt->offset;

@@ -515,7 +516,7 @@
 				len = arr_len - req_len;
 			}
 			memcpy(kaddr, arr + req_len, len);
-			kunmap(sgpnt->page);
+			kunmap_atomic(sgpnt->page, KM_USER0);
 			act_len += len;
 		}
 		req_len += sgpnt->length;

@@ -547,7 +548,7 @@
 	}
 	sgpnt = (struct scatterlist *)scp->request_buffer;
 	for (k = 0, req_len = 0, fin = 0; k < scp->use_sg; ++k, ++sgpnt) {
-		kaddr = (unsigned char *)kmap(sgpnt->page);
+		kaddr = (unsigned char *)kmap_atomic(sgpnt->page, KM_USER0);
 		if (NULL == kaddr)
 			return -1;
 		kaddr = (unsigned char *)kaddr + sgpnt->offset;

@@ -557,7 +558,7 @@
 			fin = 1;
 		}
 		memcpy(arr + req_len, kaddr, len);
-		kunmap(sgpnt->page);
+		kunmap_atomic(sgpnt->page, KM_USER0);
 		if (fin)
 			return req_len + len;
 		req_len += sgpnt->length;

Nishanth Aravamudan

author

2004-10-18 23:24 UTC #7 lore raw

Re: [PATCH] scsi_debug [was: scsi_debug issues]

Quoted 24 lines · Douglas Gilbert

Quoted 23 lines · Nishanth Aravamudan
Quoted 6 lines · Douglas Gilbert
Quoted 4 lines · James Bottomley
Quoted 1 line · Douglas Gilbert
So this problem seems related to highmem.
<snip>
Quoted 7 lines
The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .
Nishanth, could you test this with highmem?
Doug,

Quoted 7 lines

Quoted 4 lines
I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s occur (complaining about sleeping in an invalid context) and a final panic during mkfs :) I think something is still wrong... ;) Output is below and attached.
Ok, it looks like kmap_atomic() is needed.
Could you try this additional patch.

Great! That patch seems to have fixed it. I am able to vi files, sync the scsi_debug 'disks' and not have any hangs. It is noticeably slower, though, it seems. Might that just be a side effect of the atomic calls?

Thanks again!

-Nish

Douglas Gilbert

2004-10-19 06:58 UTC #8 lore raw

Re: [PATCH] scsi_debug [was: scsi_debug issues]

Quoted 35 lines · Nishanth Aravamudan

Quoted 33 lines · Douglas Gilbert
Quoted 30 lines · Nishanth Aravamudan
Quoted 8 lines · Douglas Gilbert
Quoted 5 lines · James Bottomley
Quoted 1 line · Douglas Gilbert
So this problem seems related to highmem.
<snip>
Quoted 7 lines
The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .
Nishanth, could you test this with highmem?
Doug,
I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s occur (complaining about sleeping in an invalid context) and a final panic during mkfs :) I think something is still wrong... ;) Output is below and attached.

Quoted 7 lines

Quoted 2 lines
Ok, it looks like kmap_atomic() is needed.
Could you try this additional patch.
Great! That patch seems to have fixed it. I am able to vi files, sync the scsi_debug 'disks' and not have any hangs. It is noticeably slower, though, it seems. Might that just be a side effect of the atomic calls?

One would expect it to be pretty fast since only a ram copy is involved.

Perhaps James can comment on this strategy:

  if (( kaddr = page_address(scatp->page)) {
        kaddr += scatp->offset;
	memcpy(....);
  } else {
	kaddr = kmap_atomic(scatp->page) + scatp-offset;
	memcpy(....);
	kunmap_atomic(scatp->page);

}

Doug Gilbert

Nishanth Aravamudan

author

2004-10-21 21:36 UTC #9 lore raw

Re: [PATCH] scsi_debug [was: scsi_debug issues]

Quoted 40 lines · Douglas Gilbert

Quoted 39 lines · Nishanth Aravamudan
Quoted 35 lines · Douglas Gilbert
Quoted 24 lines · Nishanth Aravamudan
Quoted 8 lines · Douglas Gilbert
Quoted 5 lines · James Bottomley
Quoted 1 line · Douglas Gilbert
So this problem seems related to highmem.
<snip>
Quoted 7 lines
The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .
Nishanth, could you test this with highmem?
Quoted 6 lines
Doug,
I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s occur (complaining about sleeping in an invalid context) and a final panic during mkfs :) I think something is still wrong... ;) Output is below and attached.
Ok, it looks like kmap_atomic() is needed.
Could you try this additional patch.

Quoted 6 lines

Quoted 3 lines
Great! That patch seems to have fixed it. I am able to vi files, sync the scsi_debug 'disks' and not have any hangs. It is noticeably slower, though, it seems. Might that just be a side effect of the atomic calls?
One would expect it to be pretty fast since only a ram copy is involved.

I agree, and in fact I'm not seeing any perf. related problems anymore. Now it's on to debug udev's relation to thousands of disks :)

Thanks for all your help!

-Nish

Jens Axboe

2004-10-22 10:04 UTC #10 lore raw

Re: [PATCH] scsi_debug [was: scsi_debug issues]

In reply to Douglas Gilbert

Quoted 49 lines · Douglas Gilbert

Quoted 35 lines · Nishanth Aravamudan
Quoted 33 lines · Douglas Gilbert
Quoted 30 lines · Nishanth Aravamudan
Quoted 8 lines · Douglas Gilbert
Quoted 5 lines · James Bottomley
Quoted 1 line · Douglas Gilbert
So this problem seems related to highmem.
<snip>
Quoted 7 lines
The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .
Nishanth, could you test this with highmem?
Doug,
I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s occur (complaining about sleeping in an invalid context) and a final panic during mkfs :) I think something is still wrong... ;) Output is below and attached.
Quoted 7 lines
Quoted 2 lines
Ok, it looks like kmap_atomic() is needed.
Could you try this additional patch.
Great! That patch seems to have fixed it. I am able to vi files, sync the scsi_debug 'disks' and not have any hangs. It is noticeably slower, though, it seems. Might that just be a side effect of the atomic calls?
One would expect it to be pretty fast since only a ram copy is involved.
Perhaps James can comment on this strategy:

Quoted 8 lines

  if (( kaddr = page_address(scatp->page)) {
        kaddr += scatp->offset;
	memcpy(....);
  } else {
	kaddr = kmap_atomic(scatp->page) + scatp-offset;
	memcpy(....);
	kunmap_atomic(scatp->page);

}

Don't do that, always kmap_atomic(). It already does various checks to avoid unecessary work.

Signature · 1 line

Jens Axboe

Jens Axboe

2004-10-22 10:02 UTC #11 lore raw

Re: [PATCH] scsi_debug [was: scsi_debug issues]

In reply to Douglas Gilbert

Quoted 56 lines · Douglas Gilbert

Quoted 21 lines · Nishanth Aravamudan
Quoted 6 lines · Douglas Gilbert
Quoted 4 lines · James Bottomley
Quoted 1 line · Douglas Gilbert
So this problem seems related to highmem.
<snip>
Quoted 7 lines
The above required a fair few changes to scsi_debug. Attached is a patch that rolls "kmap" changes with patches I have sent recently for scsi_debug. Attachment is gzipped (due to size) and applies against lk 2.6.8.1 -> lk 2.6.9-rc4 .
Nishanth, could you test this with highmem?
Quoted 6 lines
Doug,
I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s occur (complaining about sleeping in an invalid context) and a final panic during mkfs :) I think something is still wrong... ;) Output is below and attached.
Ok, it looks like kmap_atomic() is needed.
Could you try this additional patch.
Doug Gilbert
drivers/scsi/scsi_debug.c174b renamed from drivers/scsi/scsi_debug.c +3 -2
@@ -56,7 +56,7 @@
 #include "scsi_debug.h"
 
 #define SCSI_DEBUG_VERSION "1.74"
-static const char * scsi_debug_version_date = "20041018";
+static const char * scsi_debug_version_date = "20041019";
 
 /* Additional Sense Code (ASC) used */
 #define NO_ADDED_SENSE 0x0
@@ -505,7 +505,8 @@
 	active = 1;
 	for (k = 0, req_len = 0, act_len = 0; k < scp->use_sg; ++k, ++sgpnt) {
 		if (active) {
-			kaddr = (unsigned char *)kmap(sgpnt->page);
+			kaddr = (unsigned char *)
+				kmap_atomic(sgpnt->page, KM_USER0);
 			if (NULL == kaddr)
 				return (DID_ERROR << 16);
 			kaddr = (unsigned char *)kaddr + sgpnt->offset;

drivers/scsi/scsi_debug.c174b line 519

Quoted 5 lines

drivers/scsi/scsi_debug.c174b renamed from drivers/scsi/scsi_debug.c +1 -1

@@ -515,7 +516,7 @@
 				len = arr_len - req_len;
 			}
 			memcpy(kaddr, arr + req_len, len);
-			kunmap(sgpnt->page);
+			kunmap_atomic(sgpnt->page, KM_USER0);

BUG, kunmap_atomic(kaddr) (without the offset).

Also do make sure that you stay on the same cpu from calling kmap_atomic to kunmap_atomic(). If you are under the queue_lock you are fine, I didn't check if this is the case for all your calls.

Signature · 1 line

Jens Axboe