Skip to content

gfs2-utils: Introduce extent allocation and speed up journal creation

Steven Whitehouse cluster-devel 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest2 of 19 patches 0 repliesNew

2 of 19 patches are in this archive. The cover letter announces 19.

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
6/19libgfs2: Add functions for finding free extents+0 -01 reply
6/19libgfs2: Add functions for finding free extents+0 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [Cluster-devel] [PATCH 00/19] gfs2-utils: Introduce extent allocation and speed up journal creation

Hi,

Aside from the question on patch 6, the other patches all look good,

Steve.

Quoted 79 lines · Andrew Price

Hi,

This patch set introduces extent allocation to libgfs2 and adds functions which decouple file creation, allocation and writing so that mkfs.gfs2 can be re-worked to write journals and resource groups sequentially.

With these patches, mkfs.gfs2 typically takes around 20% of the time that it did before in my tests. The main speed-up has been from the journal data allocation functions not having to re-read and write a resource group for each block allocated as it did before (this was a performance regression introduced by previous memory footprint improvement patches, hence the significant perf improvement). Journals now each occupy an extent spanning an entire resource group specifically sized for the journal, and the resource group headers are written only once, after the journal blocks have been allocated in the in-memory bitmaps. Resource groups are still only kept in memory for as long as they are needed so peak memory usage should be largely unchanged.

One thing to note is that, with these patches, the root and master inodes are no longer the first objects in the first resource group. The master inode is written in the first free block after the journals and then the other metafs structures are placed. The root directory inode is then finally created. This is not a format change but it may cause some confusion after years of expecting the root and master inodes to be at certain addresses so I thought it worth mentioning.

Coverity and valgrind are happy about these patches and I've encountered no problems after various tests which mount the fs.

Cheers,
Andy

Andrew Price (19):
libgfs2: Keep a pointer to the sbd in lgfs2_rgrps_t
libgfs2: Move bitmap buffers inside struct gfs2_bitmap
libgfs2: Fix an impossible loop condition in gfs2_rgrp_read
libgfs2: Introduce struct lgfs2_rbm
libgfs2: Move struct _lgfs2_rgrps into rgrp.h
libgfs2: Add functions for finding free extents
tests: Add unit tests for the new extent search functions
libgfs2: Ignore an empty rgrp plan if a length is specified
libgfs2: Add back-pointer to rgrps in lgfs2_rgrp_t
libgfs2: Const-ify the parameters of print functions
libgfs2: Allow init_dinode to accept a preallocated bh
libgfs2: Add extent allocation functions
libgfs2: Add support for allocating entire rgrp headers
libgfs2: Write file metadata sequentially
libgfs2: Fix alignment in lgfs2_rgsize_for_data
libgfs2: Handle non-zero bitmaps in lgfs2_rgrp_write
libgfs2: Add a speedier journal data block writing function
libgfs2: Create jindex directory separately from journals mkfs.gfs2: Improve journal creation performance

  .gitignore                  |   3 +-
  gfs2/convert/gfs2_convert.c |  49 +++--
  gfs2/edit/journal.c         |   6 +-
  gfs2/fsck/fs_recovery.c     |   2 +-
  gfs2/fsck/initialize.c      |  27 +--
  gfs2/fsck/metawalk.c        |  10 +-
  gfs2/fsck/pass5.c           |   9 +-
  gfs2/fsck/rgrepair.c        |  14 +-
  gfs2/fsck/util.c            |   2 +-
  gfs2/libgfs2/Makefile.am    |   2 +-
  gfs2/libgfs2/fs_bits.c      |  10 +-
  gfs2/libgfs2/fs_geometry.c  |   6 +-
  gfs2/libgfs2/fs_ops.c       | 184 ++++++++++++++---
  gfs2/libgfs2/libgfs2.h      |  50 +++--
  gfs2/libgfs2/ondisk.c       |  26 +--
  gfs2/libgfs2/rgrp.c         | 491 ++++++++++++++++++++++++++++++++++++--------
  gfs2/libgfs2/rgrp.h         |  50 +++++
  gfs2/libgfs2/structures.c   | 103 +++++++++-
  gfs2/mkfs/main_grow.c       |   4 +-
  gfs2/mkfs/main_mkfs.c       | 155 ++++++++++----
  tests/Makefile.am           |  33 ++-
  tests/check_rgrp.c          | 143 +++++++++++++
  tests/libgfs2.at            |   8 +-
  23 files changed, 1113 insertions(+), 274 deletions(-)

create mode 100644 gfs2/libgfs2/rgrp.h
create mode 100644 tests/check_rgrp.c