xfs: CIL and log optimisations
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v5 Latest39 patches 10 repliesIn review
- v439 patches 44 repliesSuperseded
- v18 patches 38 repliesSuperseded
Patches in v5 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/39 | xfs: log stripe roundoff is a property of the logfs/ fs/ fs/ | +27 | 0 replies | |
| 2/39 | xfs: separate CIL commit record IOfs/ fs/ fs/ | +16 | 0 replies | |
| 3/39 | xfs: remove xfs_blkdev_issue_flushfs/ fs/ fs/ +2 more | +5 | 0 replies | |
| 4/39 | xfs: async blkdev cache flushfs/ fs/ | +37 | 0 replies | |
| 5/39 | xfs: CIL checkpoint flushes caches unconditionallyfs/ | +21 | 0 replies | |
| 6/39 | xfs: remove need_start_rec parameter from xlog_write()fs/ fs/ fs/ | +25 | 1 reply | |
| 7/39 | xfs: journal IO cache flush reductionsfs/ fs/ fs/ +1 more | +43 | 1 reply | |
| 8/39 | xfs: Fix CIL throttle hang when CIL space used going backwardsfs/ fs/ fs/ | +49 | 1 reply | |
| 9/39 | xfs: xfs_log_force_lsn isn't passed a LSNfs/ fs/ fs/ +10 more | +56 | 1 reply | |
| 10/39 | xfs: AIL needs asynchronous CIL forcingfs/ fs/ fs/ +5 more | +91 | 1 reply | |
| 11/39 | xfs: CIL work is serialised, not pipelinedfs/ fs/ fs/ | +48 | 0 replies | |
| 12/39 | xfs: factor out the CIL transaction header buildingfs/ | +47 | 0 replies | |
| 13/39 | xfs: only CIL pushes require a start recordfs/ fs/ | +81 | 0 replies | |
| 14/39 | xfs: embed the xlog_op_header in the unmount recordfs/ | +28 | 0 replies | |
| 15/39 | xfs: embed the xlog_op_header in the commit recordfs/ | +15 | 0 replies | |
| 16/39 | xfs: log tickets don't need log client idfs/ fs/ fs/ +3 more | +19 | 0 replies | |
| 17/39 | xfs: move log iovec alignment to preparation functionfs/ | +14 | 0 replies | |
| 18/39 | xfs: reserve space and initialise xlog_op_header in item formattingfs/ fs/ fs/ | +99 | 0 replies | |
| 19/39 | xfs: log ticket region debug is largely uselessfs/ fs/ | +6 | 0 replies | |
| 20/39 | xfs: pass lv chain length into xlog_write()fs/ fs/ fs/ | +18 | 0 replies | |
| 21/39 | xfs: introduce xlog_write_single()fs/ | +56 | 0 replies | |
| 22/39 | xfs:_introduce xlog_write_partial()fs/ | +240 | 0 replies | |
| 23/39 | xfs: xlog_write() no longer needs contwr statefs/ | +10 | 0 replies | |
| 24/39 | xfs: xlog_write() doesn't need optype anymorefs/ fs/ fs/ | +6 | 0 replies | |
| 25/39 | xfs: CIL context doesn't need to count iovecsfs/ fs/ | +6 | 0 replies | |
| 26/39 | xfs: use the CIL space used counter for emptiness checksfs/ fs/ | +29 | 0 replies | |
| 27/39 | xfs: lift init CIL reservation out of xc_cil_lockfs/ | +12 | 0 replies | |
| 28/39 | xfs: rework per-iclog header CIL reservationfs/ fs/ fs/ | +59 | 0 replies | |
| 29/39 | xfs: introduce per-cpu CIL tracking structurefs/ fs/ include/ | +121 | 1 reply | |
| 30/39 | xfs: implement percpu cil space used calculationfs/ fs/ | +62 | 1 reply | |
| 31/39 | xfs: track CIL ticket reservation in percpu structurefs/ fs/ | +17 | 1 reply | |
| 32/39 | xfs: convert CIL busy extents to per-cpufs/ | +19 | 0 replies | |
| 33/39 | xfs: Add order IDs to log items in CILfs/ fs/ fs/ | +32 | 0 replies | |
| 34/39 | xfs: convert CIL to unordered per cpu listsfs/ fs/ | +31 | 0 replies | |
| 35/39 | xfs: convert log vector chain to use list headsfs/ fs/ fs/ +3 more | +57 | 0 replies | |
| 36/39 | xfs: move CIL ordering to the logvec chainfs/ fs/ | +15 | 0 replies | |
| 37/39 | xfs: avoid cil push lock if possiblefs/ | +11 | 0 replies | |
| 38/39 | xfs: xlog_sync() manually adjusts grant head spacefs/ fs/ fs/ | +41 | 0 replies | |
| 39/39 | xfs: expanding delayed logging design with background materialDocumentation/ | +322 | 0 replies |
Reviews trailers per patch
All 39 patches have a Reviewed-by or Acked-by.
| Patch | Christoph Hellwig | Darrick J. Wong | Chandan Babu R | Brian Foster | Allison Henderson | Donald Buczek |
|---|---|---|---|---|---|---|
| 1/39 | R-b | R-b | R-b | R-b | ||
| 2/39 | R-b | R-b | R-b | R-b | ||
| 3/39 | R-b | R-b | R-b | R-b | R-b | |
| 4/39 | R-b | R-b | R-b | |||
| 5/39 | R-b | R-b | R-b | R-b | ||
| 6/39 | R-b | R-b | R-b | |||
| 7/39 | R-b | R-b | R-b | |||
| 8/39 | R-b | R-b | R-b | R-b | Reported-and-tested-by | |
| 9/39 | R-b | R-b | R-b | |||
| 10/39 | R-b | R-b | ||||
| 11/39 | R-b | |||||
| 12/39 | R-b | R-b | R-b | |||
| 13/39 | R-b | |||||
| 14/39 | R-b | R-b | ||||
| 15/39 | R-b | |||||
| 16/39 | R-b | R-b | R-b | |||
| 17/39 | R-b | R-b | R-b | |||
| 18/39 | R-b | |||||
| 19/39 | R-b | R-b | ||||
| 20/39 | R-b | |||||
| 21/39 | R-b | |||||
| 22/39 | R-b | |||||
| 23/39 | R-b | |||||
| 24/39 | R-b | |||||
| 25/39 | R-b | |||||
| 26/39 | R-b | |||||
| 27/39 | R-b | |||||
| 28/39 | R-b | |||||
| 29/39 | R-b | |||||
| 30/39 | R-b | |||||
| 31/39 | R-b | |||||
| 32/39 | R-b | |||||
| 33/39 | R-b | |||||
| 34/39 | R-b | |||||
| 35/39 | R-b | |||||
| 36/39 | R-b | |||||
| 37/39 | R-b | |||||
| 38/39 | R-b | |||||
| 39/39 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.
Cover letter [PATCH 00/39 v5] xfs: CIL and log optimisations
Hi folks,
This is an update of the consolidated log scalability patchset I've been working on. Version 4 was posted here:
https://lore.kernel.org/linux-xfs/[email protected]/
This version contains the changes Darrick requested during review. The only patch remaining without at least one RVB tag is patch 30.
Performance improvements are largely documented in the change logs of the individual patches. Headline numbers are an increase in transaction rate from 700k commits/s to 1.7M commits/s, and a reduction in fua/flush operations by 2-3 orders of magnitude on metadata heavy workloads that don't use fsync.
Summary of series:
Patches Modifications
------- -------------
1-7: log write FUA/FLUSH optimisations
8: bug fix
9-11: Async CIL pushes
12-25: xlog_write() rework
26-39: CIL commit scalability
Change log is below.
Cheers,
Dave.
Version 5:
- fix typo in comment (patch 4)
- fix typo in commit msg (patch 7)
- removed unnecessary update of num_iovecs (patch 20)
- fixed whitespace (patch 29)
- removed unused space_used/curr_res from CIL pcp structure (patch 29)
- added space_used to CIL pcp structure (patch 30)
- add comment to xlog_cil_pcp_aggregate() (patch 30)
- added space_reserved to CIL pcp structure (patch 31)
- removed unnecessary commentary about CIL ordering issues from commit msg in patch 33 as patch 35 addresses these issues..
- fix whitespace in xlog_cil_free_logvec() (patch 35)
- fix weird sentence in docco (patch 39)
- propagate changes through patches to fix conflicts due to dependent changes. (various patches)
Version 4:
- rebase on 5.13-rc2+
- fixed completion logic for async cache flush
- trimmed superflous comments about not requiring REQ_PREFLUSH for iclog IO anymore.
- ensure that setting/clearing XLOG_ICL_NEED_FLUSH is atomic (i.e. only modified while holding the icloglock)
- ensure callers only add iclog flush/fua flags appropriately before releasing the iclog so that multiple independent writes to an iclog doesn't clear flags other writes into the iclog depend on.
- buffer log item dirty tracking patches merged so removed from series
- replaced XFS_LSN_CMP() checks with direct lsn1 == lsn2 comparisons to simplify the code
- changed "push_async" to "push_commit_stable" to indicate that the push caller wants the entire CIL checkpoint and commit record to be on stable storage when it completes.
- updated comment to indicate that iclog sync state is set according to the caller's desire for a stable checkpoint to be performed.
- Added comment explaining why the CIL workqueue is limited to 4 concurrent works per filesystem.
- debug overhead reduction patches merged so removed from series
- cleaned up a couple of typedef uses.
- updated pahole output for checkpoint header in commit message
- Added BUILD_BUG_ON() to check the size of unmount records.
- got rid of XFS_VOLUME define.
- got rid of XLOG_TIC_LEN_MAX define.
- cleaned up extra blank lines.
- fixed double initialisation of lv in xlog_write_single().
- removed the unnecessary change for reserved iclog space in xlog_state_get_iclog_space().
- no need to check for XLOG_CONTINUE_TRANS in xlog_write_partial() as it will always be set.
- added a patch for removing the optype parameter from xlog_write()
- removed unused nvecs from struct xfs_cil_ctx
- fixed whitespace damage in xlog_cil_pcp_dead()
- added missing cpu dead accounting transfer in xlog_cil_pcp_dead().
- factored out CIL push percpu structure aggregation into xlog_cil_pcp_aggregate()
- added the ctx->ticket->t_unit_res update back into the code even though it is largely unnecessary.
- cleaned up the pcp, cilpcp, pcptr mess in xlog_cil_pcp_alloc() and elsewhere to use variable names consistently.
- simplified the CIL sort comparison functions to a single comparison operation
- fixed percpu CIL item list sort order where items in the same transaction (order id) were reversed, leading to intents being replayed in the wrong order.
- split out log vector chain conversion to list_head into separate patch
- Updated documentation with all the fixes and suggestions made.
Version 3:
- rebase onto 5.12-rc1+
- aggregate many small dependent patchsets in one large one.
- simplify xlog_wait_on_iclog_lsn() back to just a call to xlog_wait_on_iclog()
- remove xfs_blkdev_issue_flush() instead of moving and renaming it.
- pass bio to xfs_flush_bdev_async() so it doesn't need allocation.
- skip cache flush in xfs_flush_bdev_async() if the underlying queue does not require it.
- fixed whitespace in xfs_flush_bdev_async()
- remove the implicit external log's data device cache flush code and replace it with an explicit flush in the unmount record write so that it works the same as the new CIL checkpoint cache pre-flush mechanism. This mechanism now guarantees metadata vs journal ordering for both internal and external logs.
- updated various commit messages
- fixed incorrect/unintended changes to xfs_log_force() behaviour
- typedef uint64_t xfs_csn_t; and conversion.
- removed stray trace_printk()s that were used for debugging.
- fixed minor formatting details.
- uninlined xlog_prepare_iovec()
- fixed up "lv chain vector and size calculation" commit message to reflect we are only calculating and passin gin the vector byte count.
- reworked the loop in xlog_write_single() based on Christoph's suggestion. Much cleaner!
- added patch to pass log ticket down to xlog_sync() so that it accounts the roundoff to the log ticket rather than directly modifying grant heads. Grant heads are hot, so every little bit helps.
- added patch to update delayed logging design doc with background material on how transactions and log space accounting works in XFS.
Version 2:
- fix ticket reservation roundoff to include 2 roundoffs
- removed stale copied comment from roundoff initialisation.
- clarified "separation" to mean "separation for ordering purposes" in commit message.
- added comment that newly activated, clean, empty iclogs have a LSN of 0 so are captured by the "iclog lsn < start_lsn" case that avoids needing to wait before releasing the commit iclog to be written.
- added async cache flush infrastructure
- convert CIL checkpoint push work it issue an unconditional metadata device cache flush rather than asking the first iclog write to issue it via REQ_PREFLUSH.
- cleaned up xlog_write() to remove a redundant parameter and prepare the logic for setting flags on the iclog based on the type of operational data is being written to the log.
- added XLOG_ICL_NEED_FUA flag to complement the NEED_FLUSH flag, allowing callers to issue explicit flushes and clear the NEED_FLUSH flag before the iclog is written without dropping the REQ_FUA requirement in /dev/null...
- added CIL commit-in-start-iclog optimisation that clears the NEED_FLUSH flag to avoid an unnecessary cache flush when issuing the iclog.
- fixed typo in CIL throttle bugfix comment.
- fixed trailing whitespace in commit message.