s390/dasd: data corruption fixes for thin provisioning
Subsystems (3)
Stefan Haberland
maintainercover s390/dasd: data corruption fixes for thin provisioning
Hi Jens,
please apply the following patches. There are 4 patches to fix potential data corruption on thin provisioned DASD devices and one cosmetic patch.
Haowen Bai (1):
s390/dasd: Use kzalloc instead of kmalloc/memset
Jan Höppner (2):
s390/dasd: Fix read for ESE with blksize < 4k s390/dasd: Fix read inconsistency for ESE DASD devices
Stefan Haberland (2):
s390/dasd: fix data corruption for ESE devices s390/dasd: prevent double format of tracks for ESE devices
drivers/s390/block/dasd.c | 18 +++++++++++++++--- drivers/s390/block/dasd_eckd.c | 33 ++++++++++++++++++++++----------- drivers/s390/block/dasd_int.h | 14 ++++++++++++++ 3 files changed, 51 insertions(+), 14 deletions(-)
Signature · 1 line
2.32.0
Stefan Haberland
maintainerPATCH 1/5 s390/dasd: fix data corruption for ESE devices
For ESE devices we get an error when accessing an unformatted track. The handling of this error will return zero data for read requests and format the track on demand before writing to it. To do this the code needs to distinguish between read and write requests. This is done with data from the blocklayer request. A pointer to the blocklayer request is stored in the CQR.
If there is an error on the device an ERP request is built to do error recovery. While the ERP request is mostly a copy of the original CQR the pointer to the blocklayer request is not copied to not accidentally pass it back to the blocklayer without cleanup.
This leads to the error that during ESE handling after an ERP request was built it is not possible to determine the IO direction. This leads to the formatting of a track for read requests which might in turn lead to data corruption.
--- drivers/s390/block/dasd.c | 8 +++++++- drivers/s390/block/dasd_eckd.c | 2 +- drivers/s390/block/dasd_int.h | 12 ++++++++++++ 3 files changed, 20 insertions(+), 2 deletions(-)
drivers/s390/block/dasd.c +7 -1
@@ -1639,6 +1639,7 @@ void dasd_int_handler(struct ccw_device *cdev, unsigned long intparm,unsigned long now;int nrf_suppressed =0 ;int fp_suppressed =0 ; +struct request *req;u8 *sense =NULL ;int expires;
@@ -1739,7 +1740,12 @@ void dasd_int_handler(struct ccw_device *cdev, unsigned long intparm, }if (dasd_ese_needs_format (cqr->block, irb)) { -if (rq_data_dir ((struct request *)cqr->callback_data) ==READ ) { + req =dasd_get_callback_data (cqr); +if (!req) { + cqr->status =DASD_CQR_ERROR ; +return ; + } +if (rq_data_dir (req) ==READ ) { device->discipline->ese_read (cqr, irb); cqr->status =DASD_CQR_SUCCESS ; cqr->stopclk = now;
drivers/s390/block/dasd_eckd.c +1 -1
@@ -3145,7 +3145,7 @@ dasd_eckd_ese_format(struct dasd_device *startdev, struct dasd_ccw_req *cqr,sector_t curr_trk;int rc; - req = cqr->callback_data; + req =dasd_get_callback_data (cqr); block = cqr->block; base = block->base; private = base->private;
drivers/s390/block/dasd_int.h +12 -0
@@ -756,6 +756,18 @@ dasd_check_blocksize(int bsize)return 0 ; } +/* +* return the callback data of the original request in case there are +* ERP requests build on top of it +*/ +static inline void *dasd_get_callback_data (struct dasd_ccw_req *cqr) +{ +while (cqr->refers) + cqr = cqr->refers; + +return cqr->callback_data; +} +/* externals in dasd.c */ #define DASD_PROFILE_OFF 0#define DASD_PROFILE_ON 1
Signature · 1 line
2.32.0
Stefan Haberland
maintainerPATCH 2/5 s390/dasd: prevent double format of tracks for ESE devices
In reply to Stefan HaberlandFor ESE devices we get an error for write operations on an unformatted track. Afterwards the track will be formatted and the IO operation restarted.
When using alias devices a track might be accessed by multiple requests simultaneously and there is a race window that a track gets formatted twice resulting in data loss.
Prevent this by remembering the amount of formatted tracks when starting a request and comparing this number before actually formatting a track on the fly. If the number has changed there is a chance that the current track was finally formatted in between. As a result do not format the track and restart the current IO to check.
The number of formatted tracks does not match the overall number of formatted tracks on the device and it might wrap around but this is no problem. It is only needed to recognize that a track has been formatted at all in between.
--- drivers/s390/block/dasd.c | 7 +++++++ drivers/s390/block/dasd_eckd.c | 19 +++++++++++++++++-- drivers/s390/block/dasd_int.h | 2 ++ 3 files changed, 26 insertions(+), 2 deletions(-)
drivers/s390/block/dasd.c +7 -0
@@ -1422,6 +1422,13 @@ int dasd_start_IO(struct dasd_ccw_req *cqr)if (!cqr->lpm) cqr->lpm =dasd_path_get_opm (device); } +/* +* remember the amount of formatted tracks to prevent double format on +* ESE devices +*/ +if (cqr->block) + cqr->trkcount =atomic_read (&cqr->block->trkcount); +if (cqr->cpmode ==1 ) { rc =ccw_device_tm_start (device->cdev, cqr->cpaddr, (long ) cqr, cqr->lpm);
drivers/s390/block/dasd_eckd.c +17 -2
@@ -3083,13 +3083,24 @@ static int dasd_eckd_format_device(struct dasd_device *base, }static bool test_and_set_format_track (struct dasd_format_entry *to_format, -struct dasd_block *block) +struct dasd_ccw_req *cqr) { +struct dasd_block *block = cqr->block;struct dasd_format_entry *format;unsigned long flags;bool rc = false;spin_lock_irqsave (&block->format_lock, flags); +if (cqr->trkcount !=atomic_read (&block->trkcount)) { +/* +* The number of formatted tracks has changed after request +* start and we can not tell if the current track was involved. +* To avoid data corruption treat it as if the current track is +* involved +*/ + rc = true; + goto out; + }list_for_each_entry (format, &block->format_list, list) {if (format->track == to_format->track) { rc = true;
@@ -3109,6 +3120,7 @@ static void clear_format_track(struct dasd_format_entry *format,unsigned long flags;spin_lock_irqsave (&block->format_lock, flags); +atomic_inc (&block->trkcount);list_del_init (&format->list);spin_unlock_irqrestore (&block->format_lock, flags); }
@@ -3170,8 +3182,11 @@ dasd_eckd_ese_format(struct dasd_device *startdev, struct dasd_ccw_req *cqr, } format->track = curr_trk;/* test if track is already in formatting by another thread */ -if (test_and_set_format_track (format, block)) +if (test_and_set_format_track (format, cqr)) { +/* this is no real error so do not count down retries */ + cqr->retries++;return ERR_PTR (-EEXIST ); + } fdata.start_unit = curr_trk; fdata.stop_unit = curr_trk;
drivers/s390/block/dasd_int.h +2 -0
@@ -187,6 +187,7 @@ struct dasd_ccw_req {void (*callback)(struct dasd_ccw_req *,void *data);void *callback_data;unsigned int proc_bytes;/* bytes for partial completion */ +unsigned int trkcount;/* count formatted tracks */ };/*
@@ -610,6 +611,7 @@ struct dasd_block {struct list_head format_list;spinlock_t format_lock; +atomic_t trkcount; };struct dasd_attention_data {
Signature · 1 line
2.32.0
Stefan Haberland
maintainerPATCH 3/5 s390/dasd: Fix read for ESE with blksize < 4k
In reply to Stefan HaberlandFrom: Jan Höppner <[email protected]>
When reading unformatted tracks on ESE devices, the corresponding memory areas are simply set to zero for each segment. This is done incorrectly for blocksizes < 4096.
There are two problems. First, the increment of dst is done using the counter of the loop (off), which is increased by blksize every iteration. This leads to a much bigger increment for dst as actually intended. Second, the increment of dst is done before the memory area is set to 0, skipping a significant amount of bytes of memory.
This leads to illegal overwriting of memory and ultimately to a kernel panic.
This is not a problem with 4k blocksize because blk_queue_max_segment_size is set to PAGE_SIZE, always resulting in a single iteration for the inner segment loop (bv.bv_len == blksize). The incorrectly used 'off' value to increment dst is 0 and the correct memory area is used.
In order to fix this for blksize < 4k, increment dst correctly using the blksize and only do it at the end of the loop.
--- drivers/s390/block/dasd_eckd.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/s390/block/dasd_eckd.c +3 -4
@@ -3285,12 +3285,11 @@ static int dasd_eckd_ese_read(struct dasd_ccw_req *cqr, struct irb *irb) cqr->proc_bytes = blk_count * blksize;return 0 ; } -if (dst && !skip_block) { - dst += off; +if (dst && !skip_block)memset (dst,0 , blksize); - }else { +else skip_block--; - } + dst += blksize; blk_count++; } }
Signature · 1 line
2.32.0
Stefan Haberland
maintainerPATCH 4/5 s390/dasd: Fix read inconsistency for ESE DASD devices
In reply to Stefan HaberlandFrom: Jan Höppner <[email protected]>
Read requests that return with NRF error are partially completed in dasd_eckd_ese_read(). The function keeps track of the amount of processed bytes and the driver will eventually return this information back to the block layer for further processing via __dasd_cleanup_cqr() when the request is in the final stage of processing (from the driver's perspective).
For this, blk_update_request() is used which requires the number of bytes to complete the request. As per documentation the nr_bytes parameter is described as follows:
"number of bytes to complete for @req".
This was mistakenly interpreted as "number of bytes _left_ for @req" leading to new requests with incorrect data length. The consequence are inconsistent and completely wrong read requests as data from random memory areas are read back.
Fix this by correctly specifying the amount of bytes that should be used to complete the request.
--- drivers/s390/block/dasd.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/s390/block/dasd.c +1 -2
@@ -2778,8 +2778,7 @@ static void __dasd_cleanup_cqr(struct dasd_ccw_req *cqr)* complete a request partially. */ if (proc_bytes) { -blk_update_request (req,BLK_STS_OK , -blk_rq_bytes (req) - proc_bytes); +blk_update_request (req,BLK_STS_OK , proc_bytes);blk_mq_requeue_request (req, true); }else if (likely (!blk_should_fake_timeout (req->q))) {blk_mq_complete_request (req);
Signature · 1 line
2.32.0
Stefan Haberland
maintainerPATCH 5/5 s390/dasd: Use kzalloc instead of kmalloc/memset
In reply to Stefan HaberlandFrom: Haowen Bai <[email protected]>
Use kzalloc rather than duplicating its implementation, which makes code simple and easy to understand.
--- drivers/s390/block/dasd_eckd.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
drivers/s390/block/dasd_eckd.c +1 -4
@@ -1480,7 +1480,7 @@ static int dasd_eckd_pe_handler(struct dasd_device *device, {struct pe_handler_work_data *data; - data =kmalloc (sizeof (*data),GFP_ATOMIC |GFP_DMA ); + data =kzalloc (sizeof (*data),GFP_ATOMIC |GFP_DMA );if (!data) {if (mutex_trylock (&dasd_pe_handler_mutex)) { data = pe_handler_worker;
@@ -1488,9 +1488,6 @@ static int dasd_eckd_pe_handler(struct dasd_device *device, }else {return -ENOMEM ; } - }else { -memset (data,0 ,sizeof (*data)); - data->isglobal =0 ; }INIT_WORK (&data->worker, do_pe_handler_work);dasd_get_device (device);
Signature · 1 line
2.32.0
Quoted 11 lines · Stefan Haberland
please apply the following patches. There are 4 patches to fix potential data corruption on thin provisioned DASD devices and one cosmetic patch.
Haowen Bai (1):
s390/dasd: Use kzalloc instead of kmalloc/memsetJan Höppner (2):
s390/dasd: Fix read for ESE with blksize < 4k s390/dasd: Fix read inconsistency for ESE DASD devices[...]
Applied, thanks!
[1/5] s390/dasd: fix data corruption for ESE devices
commit: 5b53a405e4658580e1faf7c217db3f55a21ba849
[2/5] s390/dasd: prevent double format of tracks for ESE devices
commit: 71f3871657370dbbaf942a1c758f64e49a36c70f
[3/5] s390/dasd: Fix read for ESE with blksize < 4k
commit: cd68c48ea15c85f1577a442dc4c285e112ff1b37
[4/5] s390/dasd: Fix read inconsistency for ESE DASD devices
commit: b9c10f68e23c13f56685559a0d6fdaca9f838324
[5/5] s390/dasd: Use kzalloc instead of kmalloc/memset
commit: f1c8781ac9d87650ccf45a354c0bbfa3f9230371Best regards,
Signature · 1 line
Jens Axboe