Skip to content

Use block pr_ops in LIO

Mike Christie dm-devel linux-block linux-mm linux-nvme linux-scsi target-devel 5 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest18 patches 31 repliesIn review
  2. v518 patches 8 repliesSuperseded
  3. v418 patches 29 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v319 patches34 repliesSuperseded
v220 patches26 repliesSuperseded

Changes in v6 from the cover letter

- Drop dm use comment.
- Move scsi pr type conversion code to scsi_common.c/.h.
- Fix le NVME_EXTENDED_DATA_STRUCT use.
- Fix sparse warnings like sense_reason_t use.

Patches in v6 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/18block: Add PR callouts for read keys and reservationinclude/linux/pr.h+25 -01 reply
2/18block: Rename BLK_STS_NEXUS to BLK_STS_RESV_CONFLICTblock/blk-core.c drivers/nvme/host/core.c drivers/s390/block/dasd.c +2 more+11 -63 replies
3/18scsi: Rename sd_pr_commanddrivers/scsi/sd.c+6 -61 reply
4/18scsi: Move sd_pr_type to scsi_commondrivers/scsi/scsi_common.c drivers/scsi/sd.c include/scsi/scsi_common.h+42 -253 replies
5/18scsi: Add support for block PR read keys/reservationdrivers/scsi/scsi_common.c drivers/scsi/sd.c include/scsi/scsi_common.h +1 more+118 -01 reply
6/18dm: Add support for block PR read keys/reservationdrivers/md/dm.c+69 -02 replies
7/18nvme: Fix reservation status related structsinclude/linux/nvme.h+30 -82 replies
8/18nvme: Don't hardcode the data len for pr commandsdrivers/nvme/host/core.c+7 -61 reply
9/18nvme: Move pr code to it's own filedrivers/nvme/host/Makefile drivers/nvme/host/core.c drivers/nvme/host/nvme.h +1 more+161 -1492 replies
10/18nvme: Add helper to send pr commanddrivers/nvme/host/pr.c+14 -92 replies
11/18nvme: Add pr_ops read_keys supportdrivers/nvme/host/pr.c include/linux/nvme.h+73 -02 replies
12/18nvme: Add a nvme_pr_type enumdrivers/nvme/host/pr.c include/linux/nvme.h+21 -121 reply
13/18nvme: Add pr_ops read_reservation supportdrivers/nvme/host/pr.c+83 -02 replies
14/18scsi: target: Rename sbc_ops to exec_cmd_opsdrivers/target/target_core_file.c drivers/target/target_core_iblock.c drivers/target/target_core_rd.c +3 more+17 -161 reply
15/18scsi: target: Allow backends to hook into PR handlingdrivers/target/target_core_pr.c include/target/target_core_backend.h+65 -11 reply
16/18scsi: target: Pass struct target_opcode_descriptor to enableddrivers/target/target_core_spc.c include/target/target_core_base.h+27 -162 replies
17/18scsi: target: Report and detect unsupported PR commandsdrivers/target/target_core_pr.c drivers/target/target_core_spc.c+72 -202 replies
18/18scsi: target: Add block PR support to iblockdrivers/target/target_core_iblock.c+266 -52 replies

Reviews trailers per patch

All 18 patches have a Reviewed-by or Acked-by.

PatchStefan HaberlandBart Van AsscheChaitanya KulkarniChristoph HellwigKeith BuschHannes ReineckeMike Snitzer
1/18R-b R-b R-b R-b
2/18A-b R-b R-b R-b R-b
3/18R-b R-b R-b R-b
4/18R-b R-b R-b R-b R-b
5/18R-b R-b R-b
6/18R-b A-b
7/18R-b R-b
8/18R-b R-b R-b
9/18R-b R-b R-b
10/18R-b R-b R-b
11/18R-b R-b
12/18R-b R-b R-b
13/18R-b R-b
14/18R-b R-b
15/18R-b
16/18R-b R-b
17/18R-b R-b
18/18R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 7 reviewers.

Read the v6 thread

Cover letter [PATCH v6 00/18] Use block pr_ops in LIO

The patches in this thread allow us to use the block pr_ops with LIO's target_core_iblock module to support cluster applications in VMs. They were built over Linus's tree. They also apply over linux-next and Martin's tree and Jens's trees.

Currently, to use windows clustering or linux clustering (pacemaker + cluster labs scsi fence agents) in VMs with LIO and vhost-scsi, you have to use tcmu or pscsi or use a cluster aware FS/framework for the LIO pr file. Setting up a cluster FS/framework is pain and waste when your real backend device is already a distributed device, and pscsi and tcmu are nice for specific use cases, but iblock gives you the best performance and allows you to use stacked devices like dm-multipath. So these patches allow iblock to work like pscsi/tcmu where they can pass a PR command to the backend module. And then iblock will use the pr_ops to pass the PR command to the real devices similar to what we do for unmap today.

The patches are separated in the following groups:
Patch 1 - 2:
- Add block layer callouts for reading reservations and rename reservation error code.
Patch 3 - 5:
- SCSI support for new callouts.
Patch 6:
- DM support for new callouts.
Patch 7 - 13:
- NVMe support for new callouts.
Patch 14 - 18:
- LIO support for new callouts.

This patchset has been tested with the libiscsi PGR ops and with window's failover cluster verification test. Note that for scsi backend devices we need this patchset:

https://lore.kernel.org/linux-scsi/[email protected]/T/#m4834a643ffb5bac2529d65d40906d3cfbdd9b1b7

to handle UAs. To reduce the size of this patchset that's being done separately to make reviewing easier. And to make merging easier this patchset and the one above do not have any conflicts so can be merged in different trees.

v6:
- Drop dm use comment.
- Move scsi pr type conversion code to scsi_common.c/.h.
- Fix le NVME_EXTENDED_DATA_STRUCT use.
- Fix sparse warnings like sense_reason_t use.

v5:
- Use []/struct_size with nvme reservation structs
- Add Keith's copywrite to pr.c
- Drop else in nvme_send_pr_command
- Fix PR_EXCLUSIVE_ACCESS_ALL_REGS use in block_pr_type_from_nvme

v4:
- Pass read_keys number of keys instead of array len
- Keep the switch use when converting between block and scsi/nvme PR types. Drop default case so compiler spits out warning if in the future a new value is added.
- Add helper for handling nvme_send_ns_head_pr_command/nvme_send_ns_pr_command
- Use void * instead of u8* for passing data buffer.
- Rename status variable to rs.
- Have caller init buffer/structs instead of nvme/scsi callouts.
- Drop blk_status to err code.

v3:
- Fix patch subject formatting.
- Fix coding style.
- Rearrange patches so helpers are added with users to avoid compilation errors.
- Move pr type conversion to array and add nvme_pr_type.
- Add Extended Data Structure control flag enum and use in code for checks.
- Move nvme pr code to new file.
- Add more info to patch subjects about why we need to add blk_status to pr_ops.
- Use generic SCSI passthrough error handling interface.
- Fix checkpatch --strict errors. Note that I kept the existing coding style that it complained about because it looked like it was the preferred style for the code and I didn't want a mix and match.

v2:
- Drop BLK_STS_NEXUS rename changes. Will do separately.
- Add NVMe support.
- Fixed bug in target_core_iblock where a variable was not initialized mentioned by Christoph.
- Fixed sd pr_ops UA handling issue found when running libiscsi PGR tests.
- Added patches to allow pr_ops to pass up a BLK_STS so we could return a RESERVATION_CONFLICT status when a pr_ops callout fails.