erofs: promote erofs from staging
Subsystems (4)
v6 cover erofs: promote erofs from staging
Hi folks,
We'd like to hear if there are potential suggestions or maybe objections of EROFS upstreaming stuffs. If any, we still have time and we can improve it in this round.
As related materials mentioned before, the goal of EROFS filesystem is to save extra storage space with guaranteed end-to-end performance for read-only files, which has better performance over exist Linux compression filesystems based on fixed-sized output compression and inplace decompression. It even has better performance in a large compression ratio range compared with generic uncompressed filesystems with proper CPU-storage combinations. And we think this direction is right and we have a dedicated kernel team working on it, enough testers and beta / end users using it.
EROFS has been applied to 10+ million HUAWEI mobile phones (Yes, the number is still increasing by time) and it seems like a success. It can be used in more wider scenarios. We think it's useful for Linux / Android OS community and it's the time moving out of staging.
EROFS is a self-contained filesystem driver. Although there are still some TODOs, we will keep on developping / tuning EROFS with the evolution of Linux kernel as the other in-kernel filesystems.
Kindly share your comments about EROFS. Thank you in advance!
Thanks,
Gao Xiang
Changelog from v5:
o keep up with "[PATCH v2] staging: erofs: updates according to erofs-outofstaging v4"
https://lore.kernel.org/lkml/20190731155752.210602-1-gaoxiang25 at huawei.com/
which mainly addresses review comments from Chao:
- keep the marco EROFS_IO_MAX_RETRIES_NOFAIL in internal.h;
- kill a redundant NULL check in "__stagingpage_alloc";
- add some descriptions in document about "use_vmap";
- rearrange erofs_vmap of "staging: erofs: kill CONFIG_EROFS_FS_USE_VM_MAP_RAM";
o all changes have been merged into staging tree, which are under staging-testing:
https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/log/?h=staging-testing
It can also be found in git at tag "erofs_2019-08-02" at:
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/linux.git/
and the latest fs code is available at:
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/linux.git/tree/fs/erofs?h=erofs-outofstaging
Changelog from v4:
o rebase on Linus 5.3-rc1;
o keep up with "staging: erofs: updates according to erofs-outofstaging v4" in order to get main code bit-for-bit identical with staging tree:
https://lore.kernel.org/lkml/20190729065159.62378-1-gaoxiang25 at huawei.com/
Changelog from v3:
o use GPL-2.0-only for SPDX-License-Identifier suggested by Stephen;
o kill all kconfig cache strategies and turn them into mount options "cache_strategy={disable|readahead|readaround}" suggested by Ted. As the first step, cached pages can still be usable after cache is disabled by remounting, and these pages will be fallen out over time, which can be refined in the later version if some requirement is needed. Update related document as well;
o turn on CONFIG_EROFS_FS_SECURITY by default suggested by David;
o kill CONFIG_EROFS_FS_IO_MAX_RETRIES and fold it into code; turn EROFS_FS_USE_VM_MAP_RAM into a module parameter ("use_vmap") suggested by David.
Changelog from v2:
o kill sbi->dev_name and clean up all failure handling in fill_super() suggested by Al.
Note that the initialzation of managed_cache is now moved after s_root is assigned since it's more preferred to iput() in .put_super() and all inodes should be evicted before the end of generic_shutdown_super(sb);
o fold in the following staging patches (and thanks):
staging: erofs:converting all 'unsigned' to 'unsigned int'
staging: erofs: Remove function erofs_kill_sb()
- However it was revoked due to erofs_kill_sb reused...
staging: erofs: avoid opened loop codes
staging: erofs: support bmap
o move EROFS_SUPER_MAGIC_V1 from linux/fs/erofs/erofs_fs.h to include/uapi/linux/magic.h for userspace utilities.
Changelog from v1:
o resend the whole filesystem into a patchset suggested by Greg;
o code is more cleaner, especially for decompression frontend.
--8<----(original cover)----
Hi,
EROFS file system has been in Linux-staging for about a year. It has been proved to be stable enough to move out of staging by 10+ millions of HUAWEI Android mobile phones on the market from EMUI 9.0.1, and it was promoted as one of the key features of EMUI 9.1 [1], including P30(pro).
EROFS is a read-only file system designed to save extra storage space with guaranteed end-to-end performance by applying fixed-size output compression, inplace I/O and decompression inplace technologies [2] to Linux filesystem.
In our observation, EROFS is one of the fastest Linux compression filesystem using buffered I/O in the world. It will support direct I/O in the future if needed. EROFS even has better read performance in a large CR range compared with generic uncompressed file systems with proper CPU-storage combination, which is a reason why EROFS can be landed to speed up mobile phone performance, and which can be probably used for other use cases such as LiveCD and Docker image as well.
Currently EROFS supports 4k LZ4 fixed-size output compression since LZ4 is the fastest widely-used decompression solution in the world and 4k leads to unnoticable read amplification for the worst case. More compression algorithms and cluster sizes could be added later, which depends on the real requirement.
More information about EROFS itself are available at:
Documentation/filesystems/erofs.txt
https://kccncosschn19eng.sched.com/event/Nru2/erofs-an-introduction-and-our-smartphone-practice-xiang-gao-huawei
erofs-utils (mainly mkfs.erofs now) is available atgit://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git
Preliminary iomap support has been pending in EROFS mailing list by Chao Yu. The key issue is that current iomap doesn't support tail-end packing inline data yet, it should be resolved later.
Thanks to many contributors in the last year, the code is more clean and improved. We hope EROFS can be used in wider use cases so let's promote erofs out of staging and enhance it more actively.
Kindly share comments about EROFS! We think EROFS is useful to community as a part of Linux upstream.
Thank you very much,
Gao Xiang
[1] http://web.archive.org/web/20190627021241/https://consumer.huawei.com/en/emui/
[2] https://lore.kernel.org/lkml/20190624072258.28362-1-hsiangkao at aol.com/
Gao Xiang (24):
erofs: add on-disk layout
erofs: add erofs in-memory stuffs
erofs: add super block operations
erofs: add raw address_space operations
erofs: add inode operations
erofs: support special inode
erofs: add directory operations
erofs: add namei functions
erofs: support tracepoint
erofs: update Kconfig and Makefile
erofs: introduce xattr & posixacl support
erofs: introduce tagged pointer
erofs: add compression indexes support
erofs: introduce superblock registration
erofs: introduce erofs shrinker
erofs: introduce workstation for decompression
erofs: introduce per-CPU buffers implementation
erofs: introduce pagevec for decompression subsystem
erofs: add erofs_allocpage()
erofs: introduce generic decompression backend
erofs: introduce LZ4 decompression inplace
erofs: introduce the decompression frontend
erofs: introduce cached decompression
erofs: add document
Documentation/filesystems/erofs.txt | 225 +++++ fs/Kconfig | 1 + fs/Makefile | 1 + fs/erofs/Kconfig | 98 ++ fs/erofs/Makefile | 11 + fs/erofs/compress.h | 62 ++ fs/erofs/data.c | 429 ++++++++ fs/erofs/decompressor.c | 360 +++++++ fs/erofs/dir.c | 147 +++ fs/erofs/erofs_fs.h | 316 ++++++ fs/erofs/inode.c | 326 +++++++ fs/erofs/internal.h | 554 +++++++++++ fs/erofs/namei.c | 251 +++++ fs/erofs/super.c | 666 +++++++++++++ fs/erofs/tagptr.h | 110 +++ fs/erofs/utils.c | 335 +++++++ fs/erofs/xattr.c | 700 +++++++++++++ fs/erofs/xattr.h | 94 ++ fs/erofs/zdata.c | 1405 +++++++++++++++++++++++++++ fs/erofs/zdata.h | 195 ++++ fs/erofs/zmap.c | 462 +++++++++ fs/erofs/zpvec.h | 159 +++ include/trace/events/erofs.h | 256 +++++ include/uapi/linux/magic.h | 1 + 24 files changed, 7164 insertions(+) create mode 100644 Documentation/filesystems/erofs.txt create mode 100644 fs/erofs/Kconfig create mode 100644 fs/erofs/Makefile create mode 100644 fs/erofs/compress.h create mode 100644 fs/erofs/data.c create mode 100644 fs/erofs/decompressor.c create mode 100644 fs/erofs/dir.c create mode 100644 fs/erofs/erofs_fs.h create mode 100644 fs/erofs/inode.c create mode 100644 fs/erofs/internal.h create mode 100644 fs/erofs/namei.c create mode 100644 fs/erofs/super.c create mode 100644 fs/erofs/tagptr.h create mode 100644 fs/erofs/utils.c create mode 100644 fs/erofs/xattr.c create mode 100644 fs/erofs/xattr.h create mode 100644 fs/erofs/zdata.c create mode 100644 fs/erofs/zdata.h create mode 100644 fs/erofs/zmap.c create mode 100644 fs/erofs/zpvec.h create mode 100644 include/trace/events/erofs.h
Signature · 1 line
2.17.1
PATCH v6 1/24 erofs: add on-disk layout
This commit adds the on-disk layout header file of erofs. On-disk format is compatible with erofs-staging added in 4.19.
In addition, add EROFS_SUPER_MAGIC_V1 to magic.h.
--- fs/erofs/erofs_fs.h | 316 +++++++++++++++++++++++++++++++++++++ include/uapi/linux/magic.h | 1 + 2 files changed, 317 insertions(+) create mode 100644 fs/erofs/erofs_fs.h
fs/erofs/erofs_fs.h new file +316 -0
@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#ifndef __EROFS_FS_H +#define __EROFS_FS_H + +/* Enhanced(Extended) ROM File System */ +#define EROFS_SUPER_OFFSET 1024 + +/* +* Any bits that aren't in EROFS_ALL_REQUIREMENTS should be +* incompatible with this kernel version. +*/ +#define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001 +#define EROFS_ALL_REQUIREMENTS 0 + +struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ +/* 13 */ __u8 reserved; + +/* 14 */ __le16 root_nid; +/* 16 */ __le64 inos;/* total valid ino # (== f_files - f_favail) */ + +/* 24 */ __le64 build_time;/* inode v1 time derivation */ +/* 32 */ __le32 build_time_nsec; +/* 36 */ __le32 blocks;/* used for statfs */ +/* 40 */ __le32 meta_blkaddr; +/* 44 */ __le32 xattr_blkaddr; +/* 48 */ __u8 uuid[16 ];/* 128-bit uuid for volume */ +/* 64 */ __u8 volume_name[16 ];/* volume name */ +/* 80 */ __le32 requirements;/* (aka. feature_incompat) */ + +/* 84 */ __u8 reserved2[44 ]; +} __packed;/* 128 bytes */ + +/* +* erofs inode data mapping: +* 0 - inode plain without inline data A: +* inode, [xattrs], ... | ... | no-holed data +* 1 - inode VLE compression B (legacy): +* inode, [xattrs], extents ... | ... +* 2 - inode plain with inline data C: +* inode, [xattrs], last_inline_data, ... | ... | no-holed data +* 3 - inode compression D: +* inode, [xattrs], map_header, extents ... | ... +* 4~7 - reserved +*/ +enum { +EROFS_INODE_FLAT_PLAIN , +EROFS_INODE_FLAT_COMPRESSION_LEGACY , +EROFS_INODE_FLAT_INLINE , +EROFS_INODE_FLAT_COMPRESSION , +EROFS_INODE_LAYOUT_MAX +}; + +static bool erofs_inode_is_data_compressed (unsigned int datamode) +{ +if (datamode ==EROFS_INODE_FLAT_COMPRESSION ) +return true; +return datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; +} + +/* bit definitions of inode i_advise */ +#define EROFS_I_VERSION_BITS 1 +#define EROFS_I_DATA_MAPPING_BITS 3 + +#define EROFS_I_VERSION_BIT 0 +#define EROFS_I_DATA_MAPPING_BIT 1 + +struct erofs_inode_v1 { +/* 0 */ __le16 i_advise; + +/* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ +/* 2 */ __le16 i_xattr_icount; +/* 4 */ __le16 i_mode; +/* 6 */ __le16 i_nlink; +/* 8 */ __le32 i_size; +/* 12 */ __le32 i_reserved; +/* 16 */ union { +/* file total compressed blocks for data mapping 1 */ +__le32 compressed_blocks; +__le32 raw_blkaddr; + +/* for device files, used to indicate old/new device # */ +__le32 rdev; + } i_u __packed; +/* 20 */ __le32 i_ino;/* only used for 32-bit stat compatibility */ +/* 24 */ __le16 i_uid; +/* 26 */ __le16 i_gid; +/* 28 */ __le32 i_reserved2; +} __packed; + +/* 32 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V1 0 +/* 64 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V2 1 + +struct erofs_inode_v2 { +/* 0 */ __le16 i_advise; + +/* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ +/* 2 */ __le16 i_xattr_icount; +/* 4 */ __le16 i_mode; +/* 6 */ __le16 i_reserved; +/* 8 */ __le64 i_size; +/* 16 */ union { +/* file total compressed blocks for data mapping 1 */ +__le32 compressed_blocks; +__le32 raw_blkaddr; + +/* for device files, used to indicate old/new device # */ +__le32 rdev; + } i_u __packed; + +/* only used for 32-bit stat compatibility */ +/* 20 */ __le32 i_ino; + +/* 24 */ __le32 i_uid; +/* 28 */ __le32 i_gid; +/* 32 */ __le64 i_ctime; +/* 40 */ __le32 i_ctime_nsec; +/* 44 */ __le32 i_nlink; +/* 48 */ __u8 i_reserved2[16 ]; +} __packed;/* 64 bytes */ + +#define EROFS_MAX_SHARED_XATTRS (128) +/* h_shared_count between 129 ... 255 are special # */ +#define EROFS_SHARED_XATTR_EXTENT (255) + +/* +* inline xattrs (n == i_xattr_icount): +* erofs_xattr_ibody_header(1) + (n - 1) * 4 bytes +* 12 bytes / \ +* / \ +* /-----------------------\ +* | erofs_xattr_entries+ | +* +-----------------------+ +* inline xattrs must starts in erofs_xattr_ibody_header, +* for read-only fs, no need to introduce h_refcount +*/ +struct erofs_xattr_ibody_header { +__le32 h_reserved; +__u8 h_shared_count; +__u8 h_reserved2[7 ]; +__le32 h_shared_xattrs[0 ];/* shared xattr id array */ +} __packed; + +/* Name indexes */ +#define EROFS_XATTR_INDEX_USER 1 +#define EROFS_XATTR_INDEX_POSIX_ACL_ACCESS 2 +#define EROFS_XATTR_INDEX_POSIX_ACL_DEFAULT 3 +#define EROFS_XATTR_INDEX_TRUSTED 4 +#define EROFS_XATTR_INDEX_LUSTRE 5 +#define EROFS_XATTR_INDEX_SECURITY 6 + +/* xattr entry (for both inline & shared xattrs) */ +struct erofs_xattr_entry { +__u8 e_name_len;/* length of name */ +__u8 e_name_index;/* attribute name index */ +__le16 e_value_size;/* size of attribute value */ +/* followed by e_name and e_value */ +char e_name[0 ];/* attribute name */ +} __packed; + +#define ondisk_xattr_ibody_size (count) ({\ + u32 __count = le16_to_cpu(count); \ + ((__count) == 0) ? 0 : \ + sizeof(struct erofs_xattr_ibody_header) + \ + sizeof(__u32) * ((__count) - 1); }) + +#define EROFS_XATTR_ALIGN (size) round_up(size, sizeof(struct erofs_xattr_entry)) +#define EROFS_XATTR_ENTRY_SIZE (entry) EROFS_XATTR_ALIGN( \ + sizeof(struct erofs_xattr_entry) + \ + (entry)->e_name_len + le16_to_cpu((entry)->e_value_size)) + +/* available compression algorithm types */ +enum { +Z_EROFS_COMPRESSION_LZ4 , +Z_EROFS_COMPRESSION_MAX +}; + +/* +* bit 0 : COMPACTED_2B indexes (0 - off; 1 - on) +* e.g. for 4k logical cluster size, 4B if compacted 2B is off; +* (4B) + 2B + (4B) if compacted 2B is on. +*/ +#define Z_EROFS_ADVISE_COMPACTED_2B_BIT 0 + +#define Z_EROFS_ADVISE_COMPACTED_2B (1 << Z_EROFS_ADVISE_COMPACTED_2B_BIT) + +struct z_erofs_map_header { +__le32 h_reserved1; +__le16 h_advise; +/* +* bit 0-3 : algorithm type of head 1 (logical cluster type 01); +* bit 4-7 : algorithm type of head 2 (logical cluster type 11). +*/ +__u8 h_algorithmtype; +/* +* bit 0-2 : logical cluster bits - 12, e.g. 0 for 4096; +* bit 3-4 : (physical - logical) cluster bits of head 1: +* For example, if logical clustersize = 4096, 1 for 8192. +* bit 5-7 : (physical - logical) cluster bits of head 2. +*/ +__u8 h_clusterbits; +}; + +#define Z_EROFS_VLE_LEGACY_HEADER_PADDING 8 + +/* +* Z_EROFS Variable-sized Logical Extent cluster type: +* 0 - literal (uncompressed) cluster +* 1 - compressed cluster (for the head logical cluster) +* 2 - compressed cluster (for the other logical clusters) +* +* In detail, +* 0 - literal (uncompressed) cluster, +* di_advise = 0 +* di_clusterofs = the literal data offset of the cluster +* di_blkaddr = the blkaddr of the literal cluster +* +* 1 - compressed cluster (for the head logical cluster) +* di_advise = 1 +* di_clusterofs = the decompressed data offset of the cluster +* di_blkaddr = the blkaddr of the compressed cluster +* +* 2 - compressed cluster (for the other logical clusters) +* di_advise = 2 +* di_clusterofs = +* the decompressed data offset in its own head cluster +* di_u.delta[0] = distance to its corresponding head cluster +* di_u.delta[1] = distance to its corresponding tail cluster +* (di_advise could be 0, 1 or 2) +*/ +enum { +Z_EROFS_VLE_CLUSTER_TYPE_PLAIN , +Z_EROFS_VLE_CLUSTER_TYPE_HEAD , +Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD , +Z_EROFS_VLE_CLUSTER_TYPE_RESERVED , +Z_EROFS_VLE_CLUSTER_TYPE_MAX +}; + +#define Z_EROFS_VLE_DI_CLUSTER_TYPE_BITS 2 +#define Z_EROFS_VLE_DI_CLUSTER_TYPE_BIT 0 + +struct z_erofs_vle_decompressed_index { +__le16 di_advise; +/* where to decompress in the head cluster */ +__le16 di_clusterofs; + +union { +/* for the head cluster */ +__le32 blkaddr; +/* +* for the rest clusters +* eg. for 4k page-sized cluster, maximum 4K*64k = 256M) +* [0] - pointing to the head cluster +* [1] - pointing to the tail cluster +*/ +__le16 delta[2 ]; + } di_u __packed; /* 8 bytes */ +} __packed; + +#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN (size) \ + (round_up(size, sizeof(struct z_erofs_vle_decompressed_index)) + \ + sizeof(struct z_erofs_map_header) + Z_EROFS_VLE_LEGACY_HEADER_PADDING) + +/* dirent sorts in alphabet order, thus we can do binary search */ +struct erofs_dirent { +__le64 nid;/* 0, node number */ +__le16 nameoff;/* 8, start offset of file name */ +__u8 file_type;/* 10, file type */ +__u8 reserved;/* 11, reserved */ +} __packed; + +/* file types used in inode_info->flags */ +enum { +EROFS_FT_UNKNOWN , +EROFS_FT_REG_FILE , +EROFS_FT_DIR , +EROFS_FT_CHRDEV , +EROFS_FT_BLKDEV , +EROFS_FT_FIFO , +EROFS_FT_SOCK , +EROFS_FT_SYMLINK , +EROFS_FT_MAX +}; + +#define EROFS_NAME_LEN 255 + +/* check the EROFS on-disk layout strictly at compile time */ +static inline void erofs_check_ondisk_layout_definitions (void ) +{ +BUILD_BUG_ON (sizeof (struct erofs_super_block ) !=128 ); +BUILD_BUG_ON (sizeof (struct erofs_inode_v1 ) !=32 ); +BUILD_BUG_ON (sizeof (struct erofs_inode_v2 ) !=64 ); +BUILD_BUG_ON (sizeof (struct erofs_xattr_ibody_header ) !=12 ); +BUILD_BUG_ON (sizeof (struct erofs_xattr_entry ) !=4 ); +BUILD_BUG_ON (sizeof (struct z_erofs_map_header ) !=8 ); +BUILD_BUG_ON (sizeof (struct z_erofs_vle_decompressed_index ) !=8 ); +BUILD_BUG_ON (sizeof (struct erofs_dirent ) !=12 ); + +BUILD_BUG_ON (BIT (Z_EROFS_VLE_DI_CLUSTER_TYPE_BITS ) < +Z_EROFS_VLE_CLUSTER_TYPE_MAX -1 ); +} + +#endif +
include/uapi/linux/magic.h +1 -0
@@ -19,6 +19,7 @@#define SQUASHFS_MAGIC 0x73717368#define ECRYPTFS_SUPER_MAGIC 0xf15f#define EFS_SUPER_MAGIC 0x414A53 +#define EROFS_SUPER_MAGIC_V1 0xE0F5E1E2#define EXT2_SUPER_MAGIC 0xEF53#define EXT3_SUPER_MAGIC 0xEF53#define XENFS_SUPER_MAGIC 0xabba1974
Signature · 1 line
2.17.1
Re: [PATCH v6 01/24] erofs: add on-disk layout
fs/erofs/erofs_fs.h line 3
Quoted 6 lines
fs/erofs/erofs_fs.h new file +3
-0@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h
Please remove the pointless file names in the comment headers.
fs/erofs/erofs_fs.h line 26
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */
Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
fs/erofs/erofs_fs.h line 42
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+/* 64 */ __u8 volume_name[16 ];/* volume name */ +/* 80 */ __le32 requirements;/* (aka. feature_incompat) */ + +/* 84 */ __u8 reserved2[44 ]; +} __packed;/* 128 bytes */
Please don't add __packed. In this case I think you don't need it (but double check with pahole), but even if you would need it using proper padding fields and making sure all fields are naturally aligned will give you much better code generation on architectures that don't support native unaligned access.
Quoted 4 lines
fs/erofs/erofs_fs.h new file +4
-0+/* +* erofs inode data mapping: +* 0 - inode plain without inline data A: +* inode, [xattrs], ... | ... | no-holed data
fs/erofs/erofs_fs.h line 57
Quoted 10 lines
fs/erofs/erofs_fs.h new file +10
-0+* 1 - inode VLE compression B (legacy): +* inode, [xattrs], extents ... | ... +* 2 - inode plain with inline data C: +* inode, [xattrs], last_inline_data, ... | ... | no-holed data +* 3 - inode compression D: +* inode, [xattrs], map_header, extents ... | ... +* 4~7 - reserved +*/ +enum { +EROFS_INODE_FLAT_PLAIN ,
This one doesn't actually seem to be used.
fs/erofs/erofs_fs.h line 58
Quoted 1 line
fs/erofs/erofs_fs.h new file +1
-0+ EROFS_INODE_FLAT_COMPRESSION_LEGACY,
why are we adding a legacy field to a brand new file system?
fs/erofs/erofs_fs.h line 61
Quoted 3 lines
fs/erofs/erofs_fs.h new file +3
-0+EROFS_INODE_FLAT_INLINE , +EROFS_INODE_FLAT_COMPRESSION , +EROFS_INODE_LAYOUT_MAX
It seems like these come from the on-disk format, in which case they should have explicit values assigned to them.
Btw, I think it generally helps file system implementation quality if you use a separate header for the on-disk structures vs in-memory structures, as that keeps it clear in everyones mind what needs to stay persistent and what can be chenged easily.
fs/erofs/erofs_fs.h line 69
Quoted 6 lines
fs/erofs/erofs_fs.h new file +6
-0+static bool erofs_inode_is_data_compressed (unsigned int datamode) +{ +if (datamode ==EROFS_INODE_FLAT_COMPRESSION ) +return true; +return datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; +}
This looks like a really obsfucated way to write:
return datamode == EROFS_INODE_FLAT_COMPRESSION || datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY;
fs/erofs/erofs_fs.h line 39
Quoted 2 lines
fs/erofs/erofs_fs.h new file +2
-0+/* 28 */ __le32 i_reserved2; +} __packed;
Sane comment as above.
fs/erofs/erofs_fs.h line 107
Quoted 8 lines
fs/erofs/erofs_fs.h new file +8
-0+ +/* 32 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V1 0 +/* 64 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V2 1 + +struct erofs_inode_v2 { +/* 0 */ __le16 i_advise;
Why do we have two inode version in a newly added file system?
fs/erofs/erofs_fs.h line 177
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+#define ondisk_xattr_ibody_size (count) ({\ + u32 __count = le16_to_cpu(count); \ + ((__count) == 0) ? 0 : \ + sizeof(struct erofs_xattr_ibody_header) + \ + sizeof(__u32) * ((__count) - 1); })
This would be much more readable as a function.
fs/erofs/erofs_fs.h line 182
Quoted 3 lines
fs/erofs/erofs_fs.h new file +3
-0+#define EROFS_XATTR_ENTRY_SIZE (entry) EROFS_XATTR_ALIGN( \ + sizeof(struct erofs_xattr_entry) + \ + (entry)->e_name_len + le16_to_cpu((entry)->e_value_size))
Same here.
fs/erofs/erofs_fs.h line 188
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+/* available compression algorithm types */ +enum { +Z_EROFS_COMPRESSION_LZ4 , +Z_EROFS_COMPRESSION_MAX +};
Seems like an on-disk value again that should use explicitly assigned numbers.
Re: [PATCH v6 01/24] erofs: add on-disk layout
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
fs/erofs/erofs_fs.h line 3
Quoted 6 lines
fs/erofs/erofs_fs.h new file +3
-0@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h Please remove the pointless file names in the comment headers.
Already removed in the latest version.
Quoted 9 lines
fs/erofs/erofs_fs.h line 26
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
I have no idea the actual issue here.
It will help all developpers better add fields or calculate these offsets in their mind, and with care.
Rather than they didn't run "gdb" or "pahole" and change it by mistake.
Quoted 12 lines
fs/erofs/erofs_fs.h line 42
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+/* 64 */ __u8 volume_name[16 ];/* volume name */ +/* 80 */ __le32 requirements;/* (aka. feature_incompat) */ + +/* 84 */ __u8 reserved2[44 ]; +} __packed;/* 128 bytes */ Please don't add
__packed. In this case I think you don't need it (but double check with pahole), but even if you would need it using proper padding fields and making sure all fields are naturally aligned will give you much better code generation on architectures that don't support native unaligned access.
If you can see more, all on-disk fields in EROFS are naturally aligned, I can remove all of these as you like, but I think that is not very urgent.
Quoted 17 lines
Quoted 5 lines
fs/erofs/erofs_fs.h new file +4
-0+/* +* erofs inode data mapping: +* 0 - inode plain without inline data A: +* inode, [xattrs], ... | ... | no-holed data fs/erofs/erofs_fs.h line 57
Quoted 10 lines
fs/erofs/erofs_fs.h new file +10
-0+* 1 - inode VLE compression B (legacy): +* inode, [xattrs], extents ... | ... +* 2 - inode plain with inline data C: +* inode, [xattrs], last_inline_data, ... | ... | no-holed data +* 3 - inode compression D: +* inode, [xattrs], map_header, extents ... | ... +* 4~7 - reserved +*/ +enum { +EROFS_INODE_FLAT_PLAIN ,This one doesn't actually seem to be used.
It could be better has a name though, because erofs.mkfs uses it, and we keep this on-disk file up with erofs-utils.
Quoted 4 lines
fs/erofs/erofs_fs.h line 58
Quoted 2 lines
fs/erofs/erofs_fs.h new file +1
-0+ EROFS_INODE_FLAT_COMPRESSION_LEGACY,why are we adding a legacy field to a brand new file system?
the difference is just EROFS_INODE_FLAT_COMPRESSION_LEGACY doesn't have z_erofs_map_header, nothing special at all.
Quoted 8 lines
fs/erofs/erofs_fs.h line 61
Quoted 4 lines
fs/erofs/erofs_fs.h new file +3
-0+EROFS_INODE_FLAT_INLINE , +EROFS_INODE_FLAT_COMPRESSION , +EROFS_INODE_LAYOUT_MAX It seems like these come from the on-disk format, in which case they should have explicit values assigned to them.
Quoted 4 lines
Btw, I think it generally helps file system implementation quality if you use a separate header for the on-disk structures vs in-memory structures, as that keeps it clear in everyones mind what needs to stay persistent and what can be chenged easily.
All fields in this file are on-disk representation.
Quoted 10 lines
fs/erofs/erofs_fs.h line 69
Quoted 7 lines
fs/erofs/erofs_fs.h new file +6
-0+static bool erofs_inode_is_data_compressed (unsigned int datamode) +{ +if (datamode ==EROFS_INODE_FLAT_COMPRESSION ) +return true; +return datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; +}This looks like a really obsfucated way to write:
Quoted 2 lines
return datamode == EROFS_INODE_FLAT_COMPRESSION || datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY;
It depends on the personal choise, if you like, I will change into your form.
Quoted 6 lines
fs/erofs/erofs_fs.h line 39
Quoted 3 lines
fs/erofs/erofs_fs.h new file +2
-0+/* 28 */ __le32 i_reserved2; +} __packed;Sane comment as above.
Quoted 10 lines
fs/erofs/erofs_fs.h line 107
Quoted 8 lines
fs/erofs/erofs_fs.h new file +8
-0+ +/* 32 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V1 0 +/* 64 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V2 1 + +struct erofs_inode_v2 { +/* 0 */ __le16 i_advise;Why do we have two inode version in a newly added file system?
v2 is an exhanced on-disk inode form, it has 64 bytes, v1 is more compacted one, which is already suitable for Android use case of course.
There is no new and old, both are used for the current EROFS.
Quoted 9 lines
fs/erofs/erofs_fs.h line 177
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+#define ondisk_xattr_ibody_size (count) ({\ + u32 __count = le16_to_cpu(count); \ + ((__count) == 0) ? 0 : \ + sizeof(struct erofs_xattr_ibody_header) + \ + sizeof(__u32) * ((__count) - 1); })This would be much more readable as a function.
Quoted 5 lines
fs/erofs/erofs_fs.h line 182
Quoted 3 lines
fs/erofs/erofs_fs.h new file +3
-0+#define EROFS_XATTR_ENTRY_SIZE (entry) EROFS_XATTR_ALIGN( \ + sizeof(struct erofs_xattr_entry) + \ + (entry)->e_name_len + le16_to_cpu((entry)->e_value_size))Same here.
Personal tendency, because we are working in a dedicated team rather than an individual person.
But I can fix as you like.
Quoted 9 lines
fs/erofs/erofs_fs.h line 188
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+/* available compression algorithm types */ +enum { +Z_EROFS_COMPRESSION_LZ4 , +Z_EROFS_COMPRESSION_MAX +};Seems like an on-disk value again that should use explicitly assigned numbers.
I can fix it up as you like but I still cannot get what is critical issues here.
Thanks,
Gao Xiang
Re: [PATCH v6 01/24] erofs: add on-disk layout
Quoted 2 lines · Gao Xiang
I can fix it up as you like but I still cannot get what is critical issues here.
The problem is that the whole codebase is way substandard quality, looking a lot like Linux code from 20 years ago. Yes, we already have plenty of code of that standard in the tree, but we should not add more.
Re: [PATCH v6 01/24] erofs: add on-disk layout
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Gao Xiang
I can fix it up as you like but I still cannot get what is critical issues here.
The problem is that the whole codebase is way substandard quality, looking a lot like Linux code from 20 years ago. Yes, we already have plenty of code of that standard in the tree, but we should not add more.
I still cannot get your point what does your substandard quality mean, please refer to some thing critical in EROFS (and I noticed that your new code still has bug) rather than naming.
Thanks,
Gao Xiang
Re: [PATCH v6 01/24] erofs: add on-disk layout
In reply to Gao XiangQuoted 23 lines · Gao Xiang
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
fs/erofs/erofs_fs.h line 3
Quoted 6 lines
fs/erofs/erofs_fs.h new file +3
-0@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h Please remove the pointless file names in the comment headers.
Already removed in the latest version.
Quoted 8 lines
fs/erofs/erofs_fs.h line 26
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
Quoted 5 lines
I have no idea the actual issue here.
It will help all developpers better add fields or calculate these offsets in their mind, and with care.Rather than they didn't run "gdb" or "pahole" and change it by mistake.
I think Christoph is not right here.
Using external tools for validation is extra work when necessary for understanding the code.
The expected offset is somewhat valuable, but perhaps the form is a bit off given the visual run-in to the field types.
The extra work with this form is manipulating all the offsets whenever a structure change occurs.
The comments might be better with a form more like:
struct erofs_super_block { /* offset description */
__le32 magic; /* 0 */ __le32 checksum; /* 4 crc32c(super_block) */ __le32 features; /* 8 (aka. feature_compat) */ __u8 blkszbits; /* 12 support block_size == PAGE_SIZE only */
Re: [PATCH v6 01/24] erofs: add on-disk layout
Hi Joe,
Quoted 35 lines · Joe Perches
Quoted 23 lines · Gao Xiang
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
fs/erofs/erofs_fs.h line 3
Quoted 6 lines
fs/erofs/erofs_fs.h new file +3
-0@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h Please remove the pointless file names in the comment headers.
Already removed in the latest version.
Quoted 8 lines
fs/erofs/erofs_fs.h line 26
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
Quoted 5 lines
I have no idea the actual issue here.
It will help all developpers better add fields or calculate these offsets in their mind, and with care.Rather than they didn't run "gdb" or "pahole" and change it by mistake.
I think Christoph is not right here.
Using external tools for validation is extra work when necessary for understanding the code.
Quoted 8 lines
The expected offset is somewhat valuable, but perhaps the form is a bit off given the visual run-in to the field types.
The extra work with this form is manipulating all the offsets whenever a structure change occurs.
The comments might be better with a form more like:
Thanks for your comment.
I will change those places as you suggested, that is fine.
Thanks,
Gao Xiang
Quoted 8 lines
struct
erofs_super_block{ /* offset description */__le32 magic; /* 0 */ __le32 checksum; /* 4 crc32c(super_block) */ __le32 features; /* 8 (aka. feature_compat) */ __u8 blkszbits; /* 12 support block_size == PAGE_SIZE only */
Re: [PATCH v6 01/24] erofs: add on-disk layout
In reply to Joe PerchesQuoted 28 lines · Joe Perches
Quoted 27 lines · Gao Xiang
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
fs/erofs/erofs_fs.h line 3
Quoted 6 lines
fs/erofs/erofs_fs.h new file +3
-0@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h Please remove the pointless file names in the comment headers.
Already removed in the latest version.
Quoted 8 lines
fs/erofs/erofs_fs.h line 26
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
I have no idea the actual issue here.
It will help all developpers better add fields or calculate these offsets in their mind, and with care.
Quoted 6 lines
Quoted 1 line
Rather than they didn't run "gdb" or "pahole" and change it by mistake.
I think Christoph is not right here.
Using external tools for validation is extra work when necessary for understanding the code.
The advantage of using the external tools that the information about offsets is provably correct ...
Quoted 6 lines
The expected offset is somewhat valuable, but perhaps the form is a bit off given the visual run-in to the field types.
The extra work with this form is manipulating all the offsets whenever a structure change occurs.
... while this is error prone.
Quoted 7 lines
The comments might be better with a form more like:
struct
erofs_super_block{ /* offset description */__le32 magic; /* 0 */ __le32 checksum; /* 4 crc32c(super_block) */ __le32 features; /* 8 (aka. feature_compat) */ __u8 blkszbits; /* 12 support block_size == PAGE_SIZE only */
Re: [PATCH v6 01/24] erofs: add on-disk layout
Hi David,
Quoted 39 lines · David Sterba
Quoted 28 lines · Joe Perches
Quoted 27 lines · Gao Xiang
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
fs/erofs/erofs_fs.h line 3
Quoted 6 lines
fs/erofs/erofs_fs.h new file +3
-0@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h Please remove the pointless file names in the comment headers.
Already removed in the latest version.
Quoted 8 lines
fs/erofs/erofs_fs.h line 26
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
I have no idea the actual issue here.
It will help all developpers better add fields or calculate these offsets in their mind, and with care.Quoted 6 lines
Quoted 1 line
Rather than they didn't run "gdb" or "pahole" and change it by mistake.
I think Christoph is not right here.
Using external tools for validation is extra work when necessary for understanding the code.
The advantage of using the external tools that the information about offsets is provably correct ...
Quoted 8 lines
Quoted 6 lines
The expected offset is somewhat valuable, but perhaps the form is a bit off given the visual run-in to the field types.
The extra work with this form is manipulating all the offsets whenever a structure change occurs.
... while this is error prone.
I will redo a full patchset and comments addressing what Christoph all said yesterday.
Either form is fine with me for this case, let's remove them instead.
Thanks,
Gao Xiang
Quoted 8 lines
Quoted 8 lines
The comments might be better with a form more like:
struct
erofs_super_block{ /* offset description */__le32 magic; /* 0 */ __le32 checksum; /* 4 crc32c(super_block) */ __le32 features; /* 8 (aka. feature_compat) */ __u8 blkszbits; /* 12 support block_size == PAGE_SIZE only */
Re: [PATCH v6 01/24] erofs: add on-disk layout
In reply to David SterbaHi!
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
Rather than they didn't run "gdb" or "pahole" and change it by mistake.
I think Christoph is not right here.
Using external tools for validation is extra work when necessary for understanding the code.
The advantage of using the external tools that the information about offsets is provably correct ...
No. gdb tells you what the actual offsets _are_.
Quoted 8 lines
Quoted 6 lines
The expected offset is somewhat valuable, but perhaps the form is a bit off given the visual run-in to the field types.
The extra work with this form is manipulating all the offsets whenever a structure change occurs.
... while this is error prone.
While the comment tells you what they _should be_.
Pavel
Signature · 2 lines
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Re: [PATCH v6 01/24] erofs: add on-disk layout
Quoted 11 lines · Pavel Machek
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
Rather than they didn't run "gdb" or "pahole" and change it by mistake.
I think Christoph is not right here.
Using external tools for validation is extra work when necessary for understanding the code.
The advantage of using the external tools that the information about offsets is provably correct ...
No. gdb tells you what the actual offsets _are_.
Ok, reading your reply twice, I think we have different perspectives. I don't trust the comments.
The tool I had in mind is pahole that parses dwarf information about the structures, the same as gdb does. The actual value of the struct members is the thing that needs to be investigated in memory dumps or disk image dumps.
Quoted 10 lines
Quoted 8 lines
Quoted 6 lines
The expected offset is somewhat valuable, but perhaps the form is a bit off given the visual run-in to the field types.
The extra work with this form is manipulating all the offsets whenever a structure change occurs.
... while this is error prone.
While the comment tells you what they
_shouldbe_.
That's exactly the source of confusion and bugs. For me an acceptable way of asserting that a value has certain offset is a build check, eg. like
BUILD_BUG_ON(strct my_superblock, magic, 16);
Re: [PATCH v6 01/24] erofs: add on-disk layout
Hi!
Quoted 19 lines
Quoted 1 line
No. gdb tells you what the actual offsets _are_.
Ok, reading your reply twice, I think we have different perspectives. I don't trust the comments.
The tool I had in mind is pahole that parses dwarf information about the structures, the same as gdb does. The actual value of the struct members is the thing that needs to be investigated in memory dumps or disk image dumps.
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
The expected offset is somewhat valuable, but perhaps the form is a bit off given the visual run-in to the field types.
The extra work with this form is manipulating all the offsets whenever a structure change occurs.
... while this is error prone.
Quoted 7 lines
Quoted 1 line
While the comment tells you what they
_shouldbe_.That's exactly the source of confusion and bugs. For me an acceptable way of asserting that a value has certain offset is a build check, eg. like
BUILD_BUG_ON(strct my_superblock, magic, 16);
Yes, that would work, too. As would documentation file with the disk structures.
Best regards,
Pavel
Signature · 2 lines
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Re: [PATCH v6 01/24] erofs: add on-disk layout
In reply to Christoph HellwigHi Christoph,
On Thu, Aug 29, 2019 at 02:59:54AM -0700, Christoph Hellwig wrote:
[]
Quoted 10 lines
fs/erofs/erofs_fs.h line 69
Quoted 7 lines
fs/erofs/erofs_fs.h new file +6
-0+static bool erofs_inode_is_data_compressed (unsigned int datamode) +{ +if (datamode ==EROFS_INODE_FLAT_COMPRESSION ) +return true; +return datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; +}This looks like a really obsfucated way to write:
Quoted 2 lines
return datamode == EROFS_INODE_FLAT_COMPRESSION || datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY;
Add a word about this, the above approach is not horrible if more datamode add here and comments, e.g
static bool erofs_inode_is_data_compressed(unsigned int datamode)
{
/* has z_erofs_map_header */
if (datamode == EROFS_INODE_FLAT_COMPRESSION)
return true;
/* some blablabla */
if (datamode == (1) )
return true;
/* some blablablabla */
if (datamode == (2) )
return true;
/* no z_erofs_map_header */
return datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY;
}vs.
static bool erofs_inode_is_data_compressed(unsigned int datamode)
{
/* has z_erofs_map_header */
return datamode == EROFS_INODE_FLAT_COMPRESSION ||
/* some blablabla */
datamode == (1) ||
/* some blablablabla */
datamode == (2) ||
/* no z_erofs_map_header */
datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY;
}I have no idea which one is better.
Anyway, if you still like the form, I will change it.
Thanks,
Gao Xiang
Re: [PATCH v6 01/24] erofs: add on-disk layout
In reply to Christoph HellwigHi Christoph,
Sorry about my first response, sincerely... Here is my redo-ed comments to all your suggestions...
Quoted 8 lines · Christoph Hellwig
fs/erofs/erofs_fs.h line 3
Quoted 6 lines
fs/erofs/erofs_fs.h new file +3
-0@@ -0,0 +1,316 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR Apache-2.0 */ +/* +* linux/fs/erofs/erofs_fs.h Please remove the pointless file names in the comment headers.
Has already fixed in the latest version, and I will resend the whole v9 addressing all suggestions from you these days...
However it's somewhat hard to spilt the whole code prefectly since erofs is ~7KLOC code and linux-fsdevel mailing list have some limitation, I have spilted it in the form of features...
Quoted 9 lines
fs/erofs/erofs_fs.h line 26
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 12 lines
fs/erofs/erofs_fs.h line 42
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+/* 64 */ __u8 volume_name[16 ];/* volume name */ +/* 80 */ __le32 requirements;/* (aka. feature_incompat) */ + +/* 84 */ __u8 reserved2[44 ]; +} __packed;/* 128 bytes */ Please don't add
__packed. In this case I think you don't need it (but double check with pahole), but even if you would need it using proper padding fields and making sure all fields are naturally aligned will give you much better code generation on architectures that don't support native unaligned access.
fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 17 lines
Quoted 5 lines
fs/erofs/erofs_fs.h new file +4
-0+/* +* erofs inode data mapping: +* 0 - inode plain without inline data A: +* inode, [xattrs], ... | ... | no-holed data fs/erofs/erofs_fs.h line 57
Quoted 10 lines
fs/erofs/erofs_fs.h new file +10
-0+* 1 - inode VLE compression B (legacy): +* inode, [xattrs], extents ... | ... +* 2 - inode plain with inline data C: +* inode, [xattrs], last_inline_data, ... | ... | no-holed data +* 3 - inode compression D: +* inode, [xattrs], map_header, extents ... | ... +* 4~7 - reserved +*/ +enum { +EROFS_INODE_FLAT_PLAIN ,This one doesn't actually seem to be used.
It could be better has a name though, because 1) erofs.mkfs uses this definition explicitly, and we keep this on-disk definition erofs_fs.h file up with erofs-utils.
2) For kernel use, first we have,
datamode < EROFS_INODE_LAYOUT_MAX; and !erofs_inode_is_data_compressed, so there are only two mode here,
1) EROFS_INODE_FLAT_INLINE,
2) EROFS_INODE_FLAT_PLAIN
if its datamode isn't EROFS_INODE_FLAT_INLINE (tail-end block packing), it should be EROFS_INODE_FLAT_PLAIN.
The detailed logic in erofs_read_inode and
erofs_map_blocks_flatmode....
Quoted 4 lines
fs/erofs/erofs_fs.h line 58
Quoted 2 lines
fs/erofs/erofs_fs.h new file +1
-0+ EROFS_INODE_FLAT_COMPRESSION_LEGACY,why are we adding a legacy field to a brand new file system?
The difference is just EROFS_INODE_FLAT_COMPRESSION_LEGACY doesn't have z_erofs_map_header, so it only supports default (4k clustersize) fixed-sized output compression rather than per-file setting, nothing special at all...
Quoted 7 lines
fs/erofs/erofs_fs.h line 61
Quoted 4 lines
fs/erofs/erofs_fs.h new file +3
-0+EROFS_INODE_FLAT_INLINE , +EROFS_INODE_FLAT_COMPRESSION , +EROFS_INODE_LAYOUT_MAX It seems like these come from the on-disk format, in which case they should have explicit values assigned to them.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 5 lines
Btw, I think it generally helps file system implementation quality if you use a separate header for the on-disk structures vs in-memory structures, as that keeps it clear in everyones mind what needs to stay persistent and what can be chenged easily.
All fields in this file are on-disk representation by design (no logic for in-memory presentation).
Quoted 10 lines
fs/erofs/erofs_fs.h line 69
Quoted 7 lines
fs/erofs/erofs_fs.h new file +6
-0+static bool erofs_inode_is_data_compressed (unsigned int datamode) +{ +if (datamode ==EROFS_INODE_FLAT_COMPRESSION ) +return true; +return datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; +}This looks like a really obsfucated way to write:
Quoted 2 lines
return datamode == EROFS_INODE_FLAT_COMPRESSION || datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY;
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 5 lines
fs/erofs/erofs_fs.h line 39
Quoted 3 lines
fs/erofs/erofs_fs.h new file +2
-0+/* 28 */ __le32 i_reserved2; +} __packed;Sane comment as above.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 11 lines
fs/erofs/erofs_fs.h line 107
Quoted 9 lines
fs/erofs/erofs_fs.h new file +8
-0+ +/* 32 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V1 0 +/* 64 bytes on-disk inode */ +#define EROFS_INODE_LAYOUT_V2 1 + +struct erofs_inode_v2 { +/* 0 */ __le16 i_advise;Why do we have two inode version in a newly added file system?
There is no new or old, both can be used for the current EROFS in one image.
v2 is an exhanced on-disk inode form, it has 64 bytes,
v1 is more reduced one, which is already suitable for Android use case.
Quoted 8 lines
fs/erofs/erofs_fs.h line 177
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+#define ondisk_xattr_ibody_size (count) ({\ + u32 __count = le16_to_cpu(count); \ + ((__count) == 0) ? 0 : \ + sizeof(struct erofs_xattr_ibody_header) + \ + sizeof(__u32) * ((__count) - 1); })This would be much more readable as a function.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 6 lines
fs/erofs/erofs_fs.h line 182
Quoted 4 lines
fs/erofs/erofs_fs.h new file +3
-0+#define EROFS_XATTR_ENTRY_SIZE (entry) EROFS_XATTR_ALIGN( \ + sizeof(struct erofs_xattr_entry) + \ + (entry)->e_name_len + le16_to_cpu((entry)->e_value_size))Same here.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 9 lines
fs/erofs/erofs_fs.h line 188
Quoted 6 lines
fs/erofs/erofs_fs.h new file +5
-0+/* available compression algorithm types */ +enum { +Z_EROFS_COMPRESSION_LZ4 , +Z_EROFS_COMPRESSION_MAX +};Seems like an on-disk value again that should use explicitly assigned numbers.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Thanks,
Gao Xiang
Re: [PATCH v6 01/24] erofs: add on-disk layout
Quoted 12 lines · Gao Xiang
It could be better has a name though, because 1) erofs.mkfs uses this definition explicitly, and we keep this on-disk definition
erofs_fs.hfile up with erofs-utils.2) For kernel use, first we have,
datamode <EROFS_INODE_LAYOUT_MAX; and !erofs_inode_is_data_compressed, so there are only two mode here,
1)EROFS_INODE_FLAT_INLINE,
2)EROFS_INODE_FLAT_PLAIN
if its datamode isn'tEROFS_INODE_FLAT_INLINE(tail-end block packing), it should beEROFS_INODE_FLAT_PLAIN.
Quoted 2 lines
The detailed logic in
erofs_read_inodeand
erofs_map_blocks_flatmode....
Ok. At least the explicit numbering makes this a little more obvious now. What seems fairly odd is that there are only various places that check for some inode layouts/formats but nothing that does a switch over all of them.
Quoted 6 lines
Quoted 1 line
why are we adding a legacy field to a brand new file system?
The difference is just
EROFS_INODE_FLAT_COMPRESSION_LEGACYdoesn't havez_erofs_map_header, so it only supports default (4k clustersize) fixed-sized output compression rather than per-file setting, nothing special at all...
It still seems odd to add a legacy field to a brand new file system.
Quoted 5 lines
Quoted 2 lines
structures, as that keeps it clear in everyones mind what needs to stay persistent and what can be chenged easily.
All fields in this file are on-disk representation by design (no logic for in-memory presentation).
Ok, make sense. Maybe add a note to the top of the file comment
that this is the on-disk format.
One little oddity is that erofs_inode_is_data_compressed is here, while is_inode_flat_inline is in internal.h. There are arguments for either place, but I'd suggest to keep the related macros together.
Re: [PATCH v6 01/24] erofs: add on-disk layout
Hi Christoph,
Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · Gao Xiang
It could be better has a name though, because 1) erofs.mkfs uses this definition explicitly, and we keep this on-disk definition
erofs_fs.hfile up with erofs-utils.2) For kernel use, first we have,
datamode <EROFS_INODE_LAYOUT_MAX; and !erofs_inode_is_data_compressed, so there are only two mode here,
1)EROFS_INODE_FLAT_INLINE,
2)EROFS_INODE_FLAT_PLAIN
if its datamode isn'tEROFS_INODE_FLAT_INLINE(tail-end block packing), it should beEROFS_INODE_FLAT_PLAIN.
Quoted 7 lines
Quoted 2 lines
The detailed logic in
erofs_read_inodeand
erofs_map_blocks_flatmode....Ok. At least the explicit numbering makes this a little more obvious now. What seems fairly odd is that there are only various places that check for some inode layouts/formats but nothing that does a switch over all of them.
(Maybe not explicitly for this part....)
erofs_map_blocks_flatmode()
...
97 nblocks = DIV_ROUND_UP(inode->i_size, PAGE_SIZE);
98 lastblk = nblocks - is_inode_flat_inline(inode);
^ here
...
Believe me EROFS_INODE_FLAT_PLAIN is used widely for EROFS images.... (if EROFS_INODE_FLAT_INLINE tail-end packing is not suitable and
no compression....)
Quoted 9 lines
Quoted 7 lines
Quoted 2 lines
why are we adding a legacy field to a brand new file system?
The difference is just
EROFS_INODE_FLAT_COMPRESSION_LEGACYdoesn't havez_erofs_map_header, so it only supports default (4k clustersize) fixed-sized output compression rather than per-file setting, nothing special at all...It still seems odd to add a legacy field to a brand new file system.
Since 4.19 EROFS only supports EROFS_INODE_FLAT_COMPRESSION_LEGACY (per-filesystem setting), we'd like to introduce per-file setting and more configration for future requirements....
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
structures, as that keeps it clear in everyones mind what needs to stay persistent and what can be chenged easily.
All fields in this file are on-disk representation by design (no logic for in-memory presentation).
Quoted 6 lines
Ok, make sense. Maybe add a note to the top of the file comment
that this is the on-disk format.One little oddity is that
erofs_inode_is_data_compressedis here, whileis_inode_flat_inlineis ininternal.h. There are arguments for either place, but I'd suggest to keep the related macros together.
(Just my personal thought... erofs_inode_is_data_compressed operates ondisk field like datamode (because we have 2 datamode for compression, need to wrap them to judge if the file is compressed...)
so it stays at erofs_fs.h... is_inode_flat_inline operates in-memory struct inode so it in internal.h....)
Thanks,
Gao Xiang
Re: [PATCH v6 01/24] erofs: add on-disk layout
In reply to Christoph HellwigHi!
Quoted 8 lines
fs/erofs/erofs_fs.h line 26
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+struct erofs_super_block { +/* 0 */ __le32 magic;/* in the little endian */ +/* 4 */ __le32 checksum;/* crc32c(super_block) */ +/* 8 */ __le32 features;/* (aka. feature_compat) */ +/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ Please remove all the byte offset comments. That is something that can easily be checked with gdb or pahole.
I don't think I agree. gdb will tell you byte offsets _on one architecture_. But filesystem is supposed to be portable between them.
Quoted 11 lines
fs/erofs/erofs_fs.h line 42
Quoted 5 lines
fs/erofs/erofs_fs.h new file +5
-0+/* 64 */ __u8 volume_name[16 ];/* volume name */ +/* 80 */ __le32 requirements;/* (aka. feature_incompat) */ + +/* 84 */ __u8 reserved2[44 ]; +} __packed;/* 128 bytes */ Please don't add
__packed. In this case I think you don't need it (but double check with pahole), but even if you would need it using proper padding fields and making sure all fields are naturally aligned will give you much better code generation on architectures that don't support native unaligned access.
This is on-disk structure, right?
drivers/staging/erofs/super.c: struct erofs_super_block *layout;drivers/staging/erofs/super.c: layout = (struct erofs_super_block
*)((u8 *)bh->b_data
So __packed is right thing to do. If architecture accesses that slowly, that's ungood, but different structures between architectures would be really bad.
Best regards,
Pavel
Signature · 2 lines
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Re: [PATCH v6 01/24] erofs: add on-disk layout
Hi Pavel,
(Thanks...)
Quoted 4 lines · Pavel Machek
So
__packedis right thing to do. If architecture accesses that slowly, that's ungood, but different structures between architectures would be really bad.
(...a little word, it seems that Christoph was trying to say that
it's unnecessary to __packed for this case since we designed most erofs on-disk format in natural alignment... Anyway, I updated, that seems okay...)
Thanks,
Gao Xiang
Quoted 6 lines
Best regards,
PavelSignature · 2 lines
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
PATCH v6 2/24 erofs: add erofs in-memory stuffs
In reply to Gao Xiang - erofs_sb_info:
contains erofs-specific in-memory information.
- erofs_vnode:
contains vfs_inode and other fs-specific information. same as super block, the only one in-memory definition exists.
- erofs_map_blocks
Logical to physical block mapping, used by erofs_map_blocks().
--- fs/erofs/internal.h | 356 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 356 insertions(+) create mode 100644 fs/erofs/internal.h
fs/erofs/internal.h new file +356 -0
@@ -0,0 +1,356 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* linux/fs/erofs/internal.h +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#ifndef __EROFS_INTERNAL_H +#define __EROFS_INTERNAL_H + +#include <linux/fs.h> +#include <linux/dcache.h> +#include <linux/mm.h> +#include <linux/pagemap.h> +#include <linux/bio.h> +#include <linux/buffer_head.h> +#include <linux/cleancache.h> +#include <linux/magic.h> +#include <linux/slab.h> +#include <linux/vmalloc.h> +#include "erofs_fs.h" + +/* redefine pr_fmt "erofs: " */ +#undef pr_fmt +#define pr_fmt (fmt) "erofs: " fmt + +#define errln (x, ...) pr_err(x "\n", ##__VA_ARGS__) +#define infoln (x, ...) pr_info(x "\n", ##__VA_ARGS__) +#ifdef CONFIG_EROFS_FS_DEBUG +#define debugln (x, ...) pr_debug(x "\n", ##__VA_ARGS__) +#define DBG_BUGON BUG_ON +#else +#define debugln (x, ...) ((void)0) +#define DBG_BUGON (x) ((void)(x)) +#endif /* !CONFIG_EROFS_FS_DEBUG */ + +enum { +FAULT_KMALLOC , +FAULT_READ_IO , +FAULT_MAX , +}; + +#ifdef CONFIG_EROFS_FAULT_INJECTION +extern const char *erofs_fault_name[FAULT_MAX ]; +#define IS_FAULT_SET (fi, type) ((fi)->inject_type & (1 << (type))) + +struct erofs_fault_info { +atomic_t inject_ops; +unsigned int inject_rate; +unsigned int inject_type; +}; +#endif /* CONFIG_EROFS_FAULT_INJECTION */ + +/* EROFS_SUPER_MAGIC_V1 to represent the whole file system */ +#define EROFS_SUPER_MAGIC EROFS_SUPER_MAGIC_V1 + +typedef u64 erofs_nid_t ; +typedef u64 erofs_off_t ; +/* data type for filesystem-wide blocks number */ +typedef u32 erofs_blk_t ; + +struct erofs_sb_info { +u32 blocks; +u32 meta_blkaddr; + +/* inode slot unit size in bit shift */ +unsigned char islotbits; + +u32 build_time_nsec; +u64 build_time; + +/* what we really care is nid, rather than ino.. */ +erofs_nid_t root_nid; +/* used for statfs, f_files - f_favail */ +u64 inos; + +u8 uuid[16 ];/* 128-bit uuid for volume */ +u8 volume_name[16 ];/* volume name */ +u32 requirements; + +unsigned int mount_opt; + +#ifdef CONFIG_EROFS_FAULT_INJECTION +struct erofs_fault_info fault_info;/* For fault injection */ +#endif +}; + +#ifdef CONFIG_EROFS_FAULT_INJECTION +#define erofs_show_injection_info (type) \ + infoln("inject %s in %s of %pS", erofs_fault_name[type], \ + __func__, __builtin_return_address(0)) + +static inline bool time_to_inject (struct erofs_sb_info *sbi,int type) +{ +struct erofs_fault_info *ffi = &sbi->fault_info; + +if (!ffi->inject_rate) +return false; + +if (!IS_FAULT_SET (ffi, type)) +return false; + +atomic_inc (&ffi->inject_ops); +if (atomic_read (&ffi->inject_ops) >= ffi->inject_rate) { +atomic_set (&ffi->inject_ops,0 ); +return true; + } +return false; +} +#else +static inline bool time_to_inject (struct erofs_sb_info *sbi,int type) +{ +return false; +} + +static inline void erofs_show_injection_info (int type) +{ +} +#endif /* !CONFIG_EROFS_FAULT_INJECTION */ + +static inline void *erofs_kmalloc (struct erofs_sb_info *sbi, +size_t size,gfp_t flags) +{ +if (time_to_inject (sbi,FAULT_KMALLOC )) { +erofs_show_injection_info (FAULT_KMALLOC ); +return NULL ; + } +return kmalloc (size, flags); +} + +#define EROFS_SB (sb) ((struct erofs_sb_info *)(sb)->s_fs_info) +#define EROFS_I_SB (inode) ((struct erofs_sb_info *)(inode)->i_sb->s_fs_info) + +/* Mount flags set via mount options or defaults */ +#define EROFS_MOUNT_FAULT_INJECTION 0x00000040 + +#define clear_opt (sbi, option) ((sbi)->mount_opt &= ~EROFS_MOUNT_##option) +#define set_opt (sbi, option) ((sbi)->mount_opt |= EROFS_MOUNT_##option) +#define test_opt (sbi, option) ((sbi)->mount_opt & EROFS_MOUNT_##option) + +/* we strictly follow PAGE_SIZE and no buffer head yet */ +#define LOG_BLOCK_SIZE PAGE_SHIFT + +#undef LOG_SECTORS_PER_BLOCK +#define LOG_SECTORS_PER_BLOCK (PAGE_SHIFT - 9) + +#undef SECTORS_PER_BLOCK +#define SECTORS_PER_BLOCK (1 << SECTORS_PER_BLOCK) + +#define EROFS_BLKSIZ (1 << LOG_BLOCK_SIZE) + +#if (EROFS_BLKSIZ %4096 || !EROFS_BLKSIZ ) +#error erofs cannot be used in this platform +#endif + +#define EROFS_IO_MAX_RETRIES_NOFAIL 5 + +#define ROOT_NID (sb) ((sb)->root_nid) + +#define erofs_blknr (addr) ((addr) / EROFS_BLKSIZ) +#define erofs_blkoff (addr) ((addr) % EROFS_BLKSIZ) +#define blknr_to_addr (nr) ((erofs_off_t)(nr) * EROFS_BLKSIZ) + +static inline erofs_off_t iloc (struct erofs_sb_info *sbi,erofs_nid_t nid) +{ +return blknr_to_addr (sbi->meta_blkaddr) + (nid << sbi->islotbits); +} + +struct erofs_vnode { +erofs_nid_t nid; + +unsigned char datamode; +unsigned char inode_isize; +unsigned short xattr_isize; + +erofs_blk_t raw_blkaddr; +/* the corresponding vfs inode */ +struct inode vfs_inode; +}; + +#define EROFS_V (ptr) \ + container_of(ptr, struct erofs_vnode, vfs_inode) + +#define __inode_advise (x, bit, bits) \ + (((x) >> (bit)) & ((1 << (bits)) - 1)) + +#define __inode_version (advise) \ + __inode_advise(advise, EROFS_I_VERSION_BIT, \ + EROFS_I_VERSION_BITS) + +#define __inode_data_mapping (advise) \ + __inode_advise(advise, EROFS_I_DATA_MAPPING_BIT,\ + EROFS_I_DATA_MAPPING_BITS) + +static inline unsigned long inode_datablocks (struct inode *inode) +{ +/* since i_size cannot be changed */ +return DIV_ROUND_UP (inode->i_size,EROFS_BLKSIZ ); +} + +static inline bool is_inode_layout_compression (struct inode *inode) +{ +return erofs_inode_is_data_compressed (EROFS_V (inode)->datamode); +} + +static inline bool is_inode_flat_inline (struct inode *inode) +{ +return EROFS_V (inode)->datamode ==EROFS_INODE_FLAT_INLINE ; +} + +extern const struct super_operations erofs_sops; + +extern const struct address_space_operations erofs_raw_access_aops; + +/* +* Logical to physical block mapping, used by erofs_map_blocks() +* +* Different with other file systems, it is used for 2 access modes: +* +* 1) RAW access mode: +* +* Users pass a valid (m_lblk, m_lofs -- usually 0) pair, +* and get the valid m_pblk, m_pofs and the longest m_len(in bytes). +* +* Note that m_lblk in the RAW access mode refers to the number of +* the compressed ondisk block rather than the uncompressed +* in-memory block for the compressed file. +* +* m_pofs equals to m_lofs except for the inline data page. +* +* 2) Normal access mode: +* +* If the inode is not compressed, it has no difference with +* the RAW access mode. However, if the inode is compressed, +* users should pass a valid (m_lblk, m_lofs) pair, and get +* the needed m_pblk, m_pofs, m_len to get the compressed data +* and the updated m_lblk, m_lofs which indicates the start +* of the corresponding uncompressed data in the file. +*/ +enum { + BH_Zipped = BH_PrivateStart, + BH_FullMapped, +}; + +/* Has a disk mapping */ +#define EROFS_MAP_MAPPED (1 << BH_Mapped) +/* Located in metadata (could be copied from bd_inode) */ +#define EROFS_MAP_META (1 << BH_Meta) + +struct erofs_map_blocks { +erofs_off_t m_pa, m_la; +u64 m_plen, m_llen; + +unsigned int m_flags; + +struct page *mpage; +}; + +/* Flags used by erofs_map_blocks() */ +#define EROFS_GET_BLOCKS_RAW 0x0001 + +/* data.c */ +static inline struct bio *erofs_grab_bio (struct super_block *sb, +erofs_blk_t blkaddr, +unsigned int nr_pages, +void *bi_private,bio_end_io_t endio, +bool nofail) +{ +const gfp_t gfp =GFP_NOIO ; +struct bio *bio; + +do { +if (nr_pages ==1 ) { + bio =bio_alloc (gfp | (nofail ? __GFP_NOFAIL :0 ),1 ); +if (unlikely (!bio)) { +DBG_BUGON (nofail); +return ERR_PTR (-ENOMEM ); + } +break ; + } + bio =bio_alloc (gfp, nr_pages); + nr_pages /=2 ; + }while (unlikely (!bio)); + + bio->bi_end_io = endio; +bio_set_dev (bio, sb->s_bdev); + bio->bi_iter.bi_sector = (sector_t )blkaddr <<LOG_SECTORS_PER_BLOCK ; + bio->bi_private = bi_private; +return bio; +} + +static inline void __submit_bio (struct bio *bio,unsigned int op, +unsigned int op_flags) +{ +bio_set_op_attrs (bio, op, op_flags); +submit_bio (bio); +} + +struct page *__erofs_get_meta_page (struct super_block *sb,erofs_blk_t blkaddr, +bool prio,bool nofail); + +static inline struct page *erofs_get_meta_page (struct super_block *sb, +erofs_blk_t blkaddr,bool prio) +{ +return __erofs_get_meta_page (sb, blkaddr, prio, false); +} + +int erofs_map_blocks (struct inode *,struct erofs_map_blocks *,int ); + +static inline struct page *erofs_get_inline_page (struct inode *inode, +erofs_blk_t blkaddr) +{ +return erofs_get_meta_page (inode->i_sb, blkaddr, +S_ISDIR (inode->i_mode)); +} + +/* inode.c */ +static inline unsigned long erofs_inode_hash (erofs_nid_t nid) +{ +#if BITS_PER_LONG ==32 +return (nid >>32 ) ^ (nid &0xffffffff ); +#else +return nid; +#endif +} + +extern const struct inode_operations erofs_generic_iops; +extern const struct inode_operations erofs_symlink_iops; +extern const struct inode_operations erofs_fast_symlink_iops; + +static inline void set_inode_fast_symlink (struct inode *inode) +{ + inode->i_op = &erofs_fast_symlink_iops; +} + +static inline bool is_inode_fast_symlink (struct inode *inode) +{ +return inode->i_op == &erofs_fast_symlink_iops; +} + +struct inode *erofs_iget (struct super_block *sb,erofs_nid_t nid,bool dir); +int erofs_getattr (const struct path *path,struct kstat *stat, +u32 request_mask,unsigned int query_flags); + +/* namei.c */ +extern const struct inode_operations erofs_dir_iops; + +int erofs_namei (struct inode *dir,struct qstr *name, +erofs_nid_t *nid,unsigned int *d_type); + +/* dir.c */ +extern const struct file_operations erofs_dir_fops; + +#endif /* __EROFS_INTERNAL_H */ +
Signature · 1 line
2.17.1
PATCH v6 3/24 erofs: add super block operations
In reply to Gao XiangThis commit adds erofs super block operations, including (u)mount, remount_fs, show_options, statfs, in addition to some private icache management functions.
--- fs/erofs/super.c | 437 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 437 insertions(+) create mode 100644 fs/erofs/super.c
fs/erofs/super.c new file +437 -0
@@ -0,0 +1,437 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/super.c +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#include <linux/module.h> +#include <linux/buffer_head.h> +#include <linux/statfs.h> +#include <linux/parser.h> +#include <linux/seq_file.h> +#include "internal.h" + +#define CREATE_TRACE_POINTS +#include <trace/events/erofs.h> + +static struct kmem_cache *erofs_inode_cachep__read_mostly ; + +static void init_once (void *ptr) +{ +struct erofs_vnode *vi = ptr; + +inode_init_once (&vi->vfs_inode); +} + +static int __init erofs_init_inode_cache (void ) +{ + erofs_inode_cachep =kmem_cache_create ("erofs_inode" , +sizeof (struct erofs_vnode ),0 , +SLAB_RECLAIM_ACCOUNT , + init_once); + +return erofs_inode_cachep ?0 : -ENOMEM ; +} + +static void erofs_exit_inode_cache (void ) +{ +kmem_cache_destroy (erofs_inode_cachep); +} + +static struct inode *alloc_inode (struct super_block *sb) +{ +struct erofs_vnode *vi = +kmem_cache_alloc (erofs_inode_cachep,GFP_KERNEL ); + +if (!vi) +return NULL ; + +/* zero out everything except vfs_inode */ +memset (vi,0 , offsetof(struct erofs_vnode , vfs_inode)); +return &vi->vfs_inode; +} + +static void free_inode (struct inode *inode) +{ +struct erofs_vnode *vi =EROFS_V (inode); + +/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ +if (is_inode_fast_symlink (inode)) +kfree (inode->i_link); + +kmem_cache_free (erofs_inode_cachep, vi); +} + +static bool check_layout_compatibility (struct super_block *sb, +struct erofs_super_block *layout) +{ +const unsigned int requirements =le32_to_cpu (layout->requirements); + +EROFS_SB (sb)->requirements = requirements; + +/* check if current kernel meets all mandatory requirements */ +if (requirements & (~EROFS_ALL_REQUIREMENTS )) { +errln ("unidentified requirements %x, please upgrade kernel version" , + requirements & ~EROFS_ALL_REQUIREMENTS ); +return false; + } +return true; +} + +static int superblock_read (struct super_block *sb) +{ +struct erofs_sb_info *sbi; +struct buffer_head *bh; +struct erofs_super_block *layout; +unsigned int blkszbits; +int ret; + + bh =sb_bread (sb,0 ); + +if (!bh) { +errln ("cannot read erofs superblock" ); +return -EIO ; + } + + sbi =EROFS_SB (sb); + layout = (struct erofs_super_block *)((u8 *)bh->b_data + +EROFS_SUPER_OFFSET ); + + ret = -EINVAL ; +if (le32_to_cpu (layout->magic) !=EROFS_SUPER_MAGIC_V1 ) { +errln ("cannot find valid erofs superblock" ); + goto out; + } + + blkszbits = layout->blkszbits; +/* 9(512 bytes) + LOG_SECTORS_PER_BLOCK == LOG_BLOCK_SIZE */ +if (unlikely (blkszbits !=LOG_BLOCK_SIZE )) { +errln ("blksize %u isn't supported on this platform" , +1 << blkszbits); + goto out; + } + +if (!check_layout_compatibility (sb, layout)) + goto out; + + sbi->blocks =le32_to_cpu (layout->blocks); + sbi->meta_blkaddr =le32_to_cpu (layout->meta_blkaddr); + sbi->islotbits =ffs (sizeof (struct erofs_inode_v1 )) -1 ; + sbi->root_nid =le16_to_cpu (layout->root_nid); + sbi->inos =le64_to_cpu (layout->inos); + + sbi->build_time =le64_to_cpu (layout->build_time); + sbi->build_time_nsec =le32_to_cpu (layout->build_time_nsec); + +memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name)); + + ret =0 ; +out: +brelse (bh); +return ret; +} + +#ifdef CONFIG_EROFS_FAULT_INJECTION +const char *erofs_fault_name[FAULT_MAX ] = { + [FAULT_KMALLOC ] ="kmalloc" , + [FAULT_READ_IO ] ="read IO error" , +}; + +static void __erofs_build_fault_attr (struct erofs_sb_info *sbi, +unsigned int rate) +{ +struct erofs_fault_info *ffi = &sbi->fault_info; + +if (rate) { +atomic_set (&ffi->inject_ops,0 ); + ffi->inject_rate = rate; + ffi->inject_type = (1 <<FAULT_MAX ) -1 ; + }else { +memset (ffi,0 ,sizeof (struct erofs_fault_info )); + } + +set_opt (sbi,FAULT_INJECTION ); +} + +static int erofs_build_fault_attr (struct erofs_sb_info *sbi, +substring_t *args) +{ +int rate =0 ; + +if (args->from &&match_int (args, &rate)) +return -EINVAL ; + +__erofs_build_fault_attr (sbi, rate); +return 0 ; +} + +static unsigned int erofs_get_fault_rate (struct erofs_sb_info *sbi) +{ +return sbi->fault_info.inject_rate; +} +#else +static void __erofs_build_fault_attr (struct erofs_sb_info *sbi, +unsigned int rate) +{ +} + +static int erofs_build_fault_attr (struct erofs_sb_info *sbi, +substring_t *args) +{ +infoln ("fault_injection options not supported" ); +return 0 ; +} + +static unsigned int erofs_get_fault_rate (struct erofs_sb_info *sbi) +{ +return 0 ; +} +#endif + +/* set up default EROFS parameters */ +static void default_options (struct erofs_sb_info *sbi) +{ +} + +enum { + Opt_fault_injection, + Opt_err +}; + +static match_table_t erofs_tokens = { + {Opt_fault_injection,"fault_injection=%u" }, + {Opt_err,NULL } +}; + +static int parse_options (struct super_block *sb,char *options) +{ +substring_t args[MAX_OPT_ARGS ]; +char *p; +int err; + +if (!options) +return 0 ; + +while ((p =strsep (&options,"," ))) { +int token; + +if (!*p) +continue ; + + args[0 ].to = args[0 ].from =NULL ; + token =match_token (p, erofs_tokens, args); + +switch (token) { +case Opt_fault_injection: + err =erofs_build_fault_attr (EROFS_SB (sb), args); +if (err) +return err; +break ; +default : +errln ("Unrecognized mount option \"%s\" or missing value" , p); +return -EINVAL ; + } + } +return 0 ; +} + +static int erofs_fill_super (struct super_block *sb,void *data,int silent) +{ +struct inode *inode; +struct erofs_sb_info *sbi; +int err; + +infoln ("fill_super, device -> %s" , sb->s_id); +infoln ("options -> %s" , (char *)data); + + sb->s_magic =EROFS_SUPER_MAGIC ; + +if (unlikely (!sb_set_blocksize (sb,EROFS_BLKSIZ ))) { +errln ("failed to set erofs blksize" ); +return -EINVAL ; + } + + sbi =kzalloc (sizeof (*sbi),GFP_KERNEL ); +if (unlikely (!sbi)) +return -ENOMEM ; + + sb->s_fs_info = sbi; + err =superblock_read (sb); +if (err) +return err; + + sb->s_flags |=SB_RDONLY |SB_NOATIME ; + sb->s_maxbytes =MAX_LFS_FILESIZE ; + sb->s_time_gran =1 ; + + sb->s_op = &erofs_sops; + +/* set erofs default mount options */ +default_options (sbi); + + err =parse_options (sb, data); +if (unlikely (err)) +return err; + +if (!silent) +infoln ("root inode @ nid %llu" ,ROOT_NID (sbi)); + +/* get the root inode */ + inode =erofs_iget (sb,ROOT_NID (sbi), true); +if (IS_ERR (inode)) +return PTR_ERR (inode); + +if (unlikely (!S_ISDIR (inode->i_mode))) { +errln ("rootino(nid %llu) is not a directory(i_mode %o)" , +ROOT_NID (sbi), inode->i_mode); +iput (inode); +return -EINVAL ; + } + + sb->s_root =d_make_root (inode); +if (unlikely (!sb->s_root)) +return -ENOMEM ; + +if (!silent) +infoln ("mounted on %s with opts: %s." , sb->s_id, (char *)data); +return 0 ; +} + +static struct dentry *erofs_mount (struct file_system_type *fs_type,int flags, +const char *dev_name,void *data) +{ +return mount_bdev (fs_type, flags, dev_name, data, erofs_fill_super); +} + +/* +* could be triggered after deactivate_locked_super() +* is called, thus including umount and failed to initialize. +*/ +static void erofs_kill_sb (struct super_block *sb) +{ +struct erofs_sb_info *sbi; + +WARN_ON (sb->s_magic !=EROFS_SUPER_MAGIC ); +infoln ("unmounting for %s" , sb->s_id); + +kill_block_super (sb); + + sbi =EROFS_SB (sb); +if (!sbi) +return ; +kfree (sbi); + sb->s_fs_info =NULL ; +} + +static struct file_system_type erofs_fs_type = { + .owner =THIS_MODULE , + .name ="erofs" , + .mount = erofs_mount, + .kill_sb = erofs_kill_sb, + .fs_flags =FS_REQUIRES_DEV , +}; +MODULE_ALIAS_FS ("erofs" ); + +static int __init erofs_module_init (void ) +{ +int err; + +erofs_check_ondisk_layout_definitions (); +infoln ("initializing erofs " EROFS_VERSION ); + + err =erofs_init_inode_cache (); +if (err) + goto icache_err; + + err =register_filesystem (&erofs_fs_type); +if (err) + goto fs_err; + +infoln ("successfully to initialize erofs" ); +return 0 ; + +fs_err: +erofs_exit_inode_cache (); +icache_err: +return err; +} + +static void __exit erofs_module_exit(void) +{ +unregister_filesystem (&erofs_fs_type); +erofs_exit_inode_cache (); +infoln ("successfully finalize erofs" ); +} + +/* get filesystem statistics */ +static int erofs_statfs (struct dentry *dentry,struct kstatfs *buf) +{ +struct super_block *sb = dentry->d_sb; +struct erofs_sb_info *sbi =EROFS_SB (sb); +u64 id =huge_encode_dev (sb->s_bdev->bd_dev); + + buf->f_type = sb->s_magic; + buf->f_bsize =EROFS_BLKSIZ ; + buf->f_blocks = sbi->blocks; + buf->f_bfree = buf->f_bavail =0 ; + + buf->f_files =ULLONG_MAX ; + buf->f_ffree =ULLONG_MAX - sbi->inos; + + buf->f_namelen =EROFS_NAME_LEN ; + + buf->f_fsid.val[0 ] = (u32 )id; + buf->f_fsid.val[1 ] = (u32)(id >>32 ); +return 0 ; +} + +static int erofs_show_options (struct seq_file *seq,struct dentry *root) +{ +struct erofs_sb_info *sbi__maybe_unused =EROFS_SB (root->d_sb); + +if (test_opt (sbi,FAULT_INJECTION )) +seq_printf (seq,",fault_injection=%u" , +erofs_get_fault_rate (sbi)); +return 0 ; +} + +static int erofs_remount (struct super_block *sb,int *flags,char *data) +{ +struct erofs_sb_info *sbi =EROFS_SB (sb); +unsigned int org_mnt_opt = sbi->mount_opt; +unsigned int org_inject_rate =erofs_get_fault_rate (sbi); +int err; + +DBG_BUGON (!sb_rdonly (sb)); + err =parse_options (sb, data); +if (err) + goto out; + + *flags |=SB_RDONLY ; +return 0 ; +out: +__erofs_build_fault_attr (sbi, org_inject_rate); + sbi->mount_opt = org_mnt_opt; + +return err; +} + +const struct super_operations erofs_sops = { + .alloc_inode = alloc_inode, + .free_inode = free_inode, + .statfs = erofs_statfs, + .show_options = erofs_show_options, + .remount_fs = erofs_remount, +}; + +module_init (erofs_module_init); +module_exit (erofs_module_exit); + +MODULE_DESCRIPTION ("Enhanced ROM File System" ); +MODULE_AUTHOR ("Gao Xiang, Chao Yu, Miao Xie, CONSUMER BG, HUAWEI Inc." ); +MODULE_LICENSE ("GPL" ); +
Signature · 1 line
2.17.1
Re: [PATCH v6 03/24] erofs: add super block operations
fs/erofs/super.c line 35
Quoted 8 lines · Gao Xiang
fs/erofs/super.c new file +8
-0+static int __init erofs_init_inode_cache (void ) +{ + erofs_inode_cachep =kmem_cache_create ("erofs_inode" , +sizeof (struct erofs_vnode ),0 , +SLAB_RECLAIM_ACCOUNT , + init_once); + +return erofs_inode_cachep ?0 : -ENOMEM ;
Please just use normal if/else. Also having this function seems entirely pointless.
fs/erofs/super.c line 41
Quoted 4 lines
fs/erofs/super.c new file +4
-0+static void erofs_exit_inode_cache (void ) +{ +kmem_cache_destroy (erofs_inode_cachep); +}
Same for this one.
Quoted 1 line
+static void
free_inode(struct inode *inode)
Please use an erofs_ prefix for all your functions.
Review
Quoted 2 lines
(quoted diff) +2
-0+{ + struct erofs_vnode *vi = EROFS_V(inode);
Why is this called vnode instead of inode? That seems like a rather odd naming for a Linux file system.
fs/erofs/super.c line 62
Quoted 4 lines
fs/erofs/super.c new file +4
-0+ +/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ +if (is_inode_fast_symlink (inode)) +kfree (inode->i_link);
is_inode_fast_symlink only shows up in a later patch. And really obsfucates the check here in the only caller as you can just do an unconditional kfree here - i_link will be NULL except for the case where you explicitly set it.
Also this code is nothing like ext4, so the code seems a little confusing.
fs/erofs/super.c line 70
Quoted 4 lines
fs/erofs/super.c new file +4
-0+static bool check_layout_compatibility (struct super_block *sb, +struct erofs_super_block *layout) +{ +const unsigned int requirements =le32_to_cpu (layout->requirements);
Why is the variable name for the on-disk subperblock layout? We usually still calls this something with sb in the name, e.g. dsb. for disk super block.
fs/erofs/super.c line 80
Quoted 9 lines
fs/erofs/super.c new file +9
-0+EROFS_SB (sb)->requirements = requirements; + +/* check if current kernel meets all mandatory requirements */ +if (requirements & (~EROFS_ALL_REQUIREMENTS )) { +errln ("unidentified requirements %x, please upgrade kernel version" , + requirements & ~EROFS_ALL_REQUIREMENTS ); +return false; + } +return true;
Note that normally we call this features, but that doesn't really matter too much.
fs/erofs/super.c line 91
Quoted 9 lines
fs/erofs/super.c new file +9
-0+static int superblock_read (struct super_block *sb) +{ +struct erofs_sb_info *sbi; +struct buffer_head *bh; +struct erofs_super_block *layout; +unsigned int blkszbits; +int ret; + + bh =sb_bread (sb,0 );
Is there any good reasons to use buffer heads like this in new code vs directly using bios?
fs/erofs/super.c line 130
Quoted 13 lines
fs/erofs/super.c new file +13
-0+ + sbi->blocks =le32_to_cpu (layout->blocks); + sbi->meta_blkaddr =le32_to_cpu (layout->meta_blkaddr); + sbi->islotbits =ffs (sizeof (struct erofs_inode_v1 )) -1 ; + sbi->root_nid =le16_to_cpu (layout->root_nid); + sbi->inos =le64_to_cpu (layout->inos); + + sbi->build_time =le64_to_cpu (layout->build_time); + sbi->build_time_nsec =le32_to_cpu (layout->build_time_nsec); + +memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name));
s_uuid should preferably be a uuid_t (assuming it is a real BE uuid, if it is le it should be a guid_t).
fs/erofs/super.c line 198
Quoted 4 lines
fs/erofs/super.c new file +4
-0+/* set up default EROFS parameters */ +static void default_options (struct erofs_sb_info *sbi) +{ +}
No need to add an empty function.
fs/erofs/super.c line 249
Quoted 8 lines
fs/erofs/super.c new file +8
-0+static int erofs_fill_super (struct super_block *sb,void *data,int silent) +{ +struct inode *inode; +struct erofs_sb_info *sbi; +int err; + +infoln ("fill_super, device -> %s" , sb->s_id); +infoln ("options -> %s" , (char *)data);
That is some very verbose debug info. We usually don't add that and let people trace the function instead. Also you should probably implement the new mount API.
new mount API.
Quoted 5 lines
fs/erofs/super.c new file +5
-0+static void erofs_kill_sb (struct super_block *sb) +{ +struct erofs_sb_info *sbi; + +WARN_ON (sb->s_magic !=EROFS_SUPER_MAGIC );
fs/erofs/super.c line 328
Quoted 10 lines
fs/erofs/super.c new file +10
-0+infoln ("unmounting for %s" , sb->s_id); + +kill_block_super (sb); + + sbi =EROFS_SB (sb); +if (!sbi) +return ; +kfree (sbi); + sb->s_fs_info =NULL ; +}
Why is this needed? You can just free your sb privatte information in ->put_super and wire up kill_block_super as the ->kill_sb method directly.
Re: [PATCH v6 03/24] erofs: add super block operations
Hi Christoph,
Quoted 20 lines · Christoph Hellwig
fs/erofs/super.c line 35
Quoted 8 lines · Gao Xiang
fs/erofs/super.c new file +8
-0+static int __init erofs_init_inode_cache (void ) +{ + erofs_inode_cachep =kmem_cache_create ("erofs_inode" , +sizeof (struct erofs_vnode ),0 , +SLAB_RECLAIM_ACCOUNT , + init_once); + +return erofs_inode_cachep ?0 : -ENOMEM ;Please just use normal if/else. Also having this function seems entirely pointless.
fs/erofs/super.c line 41
Quoted 4 lines
fs/erofs/super.c new file +4
-0+static void erofs_exit_inode_cache (void ) +{ +kmem_cache_destroy (erofs_inode_cachep); +}Same for this one.
Quoted 3 lines
Quoted 1 line
+static void
free_inode(struct inode *inode)Please use an
erofs_prefix for all your functions.
It is already a static function, I have no idea what is wrong here.
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+{ + struct erofs_vnode *vi = EROFS_V(inode);Why is this called vnode instead of inode? That seems like a rather odd naming for a Linux file system.
I don't know anything difference of that, it is just a naming.
Quoted 10 lines
fs/erofs/super.c line 62
Quoted 5 lines
fs/erofs/super.c new file +4
-0+ +/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ +if (is_inode_fast_symlink (inode)) +kfree (inode->i_link);
is_inode_fast_symlinkonly shows up in a later patch. And really obsfucates the check here in the only caller as you can just do an unconditional kfree here -i_linkwill be NULL except for the case where you explicitly set it.
I cannot fully understand your point (sorry about my English), I will reply you about this later.
Quoted 11 lines
Also this code is nothing like ext4, so the code seems a little confusing.
fs/erofs/super.c line 70
Quoted 4 lines
fs/erofs/super.c new file +4
-0+static bool check_layout_compatibility (struct super_block *sb, +struct erofs_super_block *layout) +{ +const unsigned int requirements =le32_to_cpu (layout->requirements);Why is the variable name for the on-disk subperblock layout? We usually still calls this something with sb in the name, e.g. dsb. for disk super block.
I can change it later, sbi and dsb (It has not good meaning in Chinese, although).
Quoted 14 lines
fs/erofs/super.c line 80
Quoted 10 lines
fs/erofs/super.c new file +9
-0+EROFS_SB (sb)->requirements = requirements; + +/* check if current kernel meets all mandatory requirements */ +if (requirements & (~EROFS_ALL_REQUIREMENTS )) { +errln ("unidentified requirements %x, please upgrade kernel version" , + requirements & ~EROFS_ALL_REQUIREMENTS ); +return false; + } +return true;Note that normally we call this features, but that doesn't really matter too much.
Quoted 12 lines
fs/erofs/super.c line 91
Quoted 9 lines
fs/erofs/super.c new file +9
-0+static int superblock_read (struct super_block *sb) +{ +struct erofs_sb_info *sbi; +struct buffer_head *bh; +struct erofs_super_block *layout; +unsigned int blkszbits; +int ret; + + bh =sb_bread (sb,0 );Is there any good reasons to use buffer heads like this in new code vs directly using bios?
This page can save in bdev page cache, it contains not only the erofs superblock so it can be fetched in page cache later.
Quoted 17 lines
Quoted 4 lines
fs/erofs/super.c new file +3
-0+ + sbi->blocks =le32_to_cpu (layout->blocks); + sbi->meta_blkaddr =le32_to_cpu (layout->meta_blkaddr);fs/erofs/super.c line 130
Quoted 10 lines
fs/erofs/super.c new file +10
-0+ sbi->islotbits =ffs (sizeof (struct erofs_inode_v1 )) -1 ; + sbi->root_nid =le16_to_cpu (layout->root_nid); + sbi->inos =le64_to_cpu (layout->inos); + + sbi->build_time =le64_to_cpu (layout->build_time); + sbi->build_time_nsec =le32_to_cpu (layout->build_time_nsec); + +memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name));
s_uuidshould preferably be auuid_t(assuming it is a real BE uuid, if it is le it should be aguid_t).
I just copied it from f2fs, I have no idea which one is best and which fs I could refer to.
Quoted 7 lines
fs/erofs/super.c line 198
Quoted 5 lines
fs/erofs/super.c new file +4
-0+/* set up default EROFS parameters */ +static void default_options (struct erofs_sb_info *sbi) +{ +}No need to add an empty function.
Later patch will fill this function.
Quoted 14 lines
fs/erofs/super.c line 249
Quoted 9 lines
fs/erofs/super.c new file +8
-0+static int erofs_fill_super (struct super_block *sb,void *data,int silent) +{ +struct inode *inode; +struct erofs_sb_info *sbi; +int err; + +infoln ("fill_super, device -> %s" , sb->s_id); +infoln ("options -> %s" , (char *)data);That is some very verbose debug info. We usually don't add that and let people trace the function instead. Also you should probably implement the new mount API.
new mount API.
Al think it is not urgent as well,
https://lore.kernel.org/driverdev-devel/[email protected]/
Al said,
Quoted 19 lines
Quoted 4 lines
I agree with you, it seems better to just use
s_idin community and deleteerofs_mount_privatestuffs...
Yet I don't look into how to use newfs_context, could I keep using legacy mount interface and fix them all?Sure.
Quoted 11 lines
fs/erofs/super.c new file +11
-0+static void erofs_kill_sb (struct super_block *sb) +{ +struct erofs_sb_info *sbi; + +WARN_ON (sb->s_magic !=EROFS_SUPER_MAGIC ); +infoln ("unmounting for %s" , sb->s_id); + +kill_block_super (sb); + + sbi =EROFS_SB (sb); +if (!sbi)
Quoted 8 lines
fs/erofs/super.c line 328
Quoted 4 lines
fs/erofs/super.c new file +4
-0+return ; +kfree (sbi); + sb->s_fs_info =NULL ; +}Why is this needed? You can just free your sb privatte information in ->put_super and wire up
kill_block_superas the ->kill_sb method directly.
See Al's comments,
https://lore.kernel.org/r/[email protected]/
Thanks,
Gao Xiang
Re: [PATCH v6 03/24] erofs: add super block operations
Quoted 3 lines · Gao Xiang
Quoted 1 line
Please use an
erofs_prefix for all your functions.It is already a static function, I have no idea what is wrong here.
Which part of all wasn't clear? Have you looked at the prefixes for most functions in the various other big filesystems?
Quoted 11 lines
Quoted 8 lines
fs/erofs/super.c line 62
Quoted 3 lines
fs/erofs/super.c new file +3
-0+/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ +if (is_inode_fast_symlink (inode)) +kfree (inode->i_link);
is_inode_fast_symlinkonly shows up in a later patch. And really obsfucates the check here in the only caller as you can just do an unconditional kfree here -i_linkwill be NULL except for the case where you explicitly set it.I cannot fully understand your point (sorry about my English), I will reply you about this later.
With that I mean that you should:
1) remove is_inode_fast_symlink and just opencode it in the few places using it
2) remove the check in this place entirely as it is not needed
3) remove the comment quoted above as it is more confusing than not having the comment
Quoted 5 lines
Quoted 2 lines
Is there any good reasons to use buffer heads like this in new code vs directly using bios?
This page can save in bdev page cache, it contains not only the erofs superblock so it can be fetched in page cache later.
If you want it in the page cache why not use read_mapping_page or similar?
Quoted 8 lines
Quoted 6 lines
fs/erofs/super.c line 198
Quoted 4 lines
fs/erofs/super.c new file +4
-0+/* set up default EROFS parameters */ +static void default_options (struct erofs_sb_info *sbi) +{ +}No need to add an empty function.
Later patch will fill this function.
Please only add the function in the patch actually adding the functionality.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
+}
Why is this needed? You can just free your sb privatte information in ->put_super and wire up
kill_block_superas the ->kill_sb method directly.See Al's comments,
https://lore.kernel.org/r/[email protected]/
With that code it makes sense. In this paticular patch it does not. So please add it only when actually needed.
Re: [PATCH v6 03/24] erofs: add super block operations
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Gao Xiang
Quoted 1 line
Please use an
erofs_prefix for all your functions.It is already a static function, I have no idea what is wrong here.
Which part of all wasn't clear? Have you looked at the prefixes for most functions in the various other big filesystems?
I will add erofs prefix to free_inode as you said.
At least, all non-prefix functions in erofs are all static functions, it won't pollute namespace... I will add "erofs_" to other meaningful callbacks...And as you can see...
cifs/cifsfs.c
1303:cifs_init_inodecache(void)
1509: rc = cifs_init_inodecache();
hpfs/super.c
254:static int init_inodecache(void)
771: int err = init_inodecache();
minix/inode.c
84:static int __init init_inodecache(void)
665: int err = init_inodecache();
isofs/inode.c
88:static int __init init_inodecache(void)
1580: int err = init_inodecache();
bfs/inode.c
261:static int __init init_inodecache(void)
468: int err = init_inodecache();
ext4/super.c
1144:static int __init init_inodecache(void)
6115: err = init_inodecache();
reiserfs/super.c
666:static int __init init_inodecache(void)
2606: ret = init_inodecache();
squashfs/super.c
406:static int __init init_inodecache(void)
430: int err = init_inodecache();
udf/super.c
177:static int __init init_inodecache(void)
232: err = init_inodecache();
qnx4/inode.c
358:static int init_inodecache(void)
399: err = init_inodecache();
ufs/super.c
1463:static int __init init_inodecache(void)
1517: int err = init_inodecache();
qnx6/inode.c
618:static int init_inodecache(void)
659: err = init_inodecache();
f2fs/super.c
3540:static int __init init_inodecache(void)
3572: err = init_inodecache();
Quoted 13 lines
Quoted 10 lines
Quoted 9 lines
fs/erofs/super.c line 62
Quoted 4 lines
fs/erofs/super.c new file +3
-0+/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ +if (is_inode_fast_symlink (inode)) +kfree (inode->i_link);
is_inode_fast_symlinkonly shows up in a later patch. And really obsfucates the check here in the only caller as you can just do an unconditional kfree here -i_linkwill be NULL except for the case where you explicitly set it.Quoted 2 lines
I cannot fully understand your point (sorry about my English), I will reply you about this later.
Quoted 7 lines
With that I mean that you should:
1) remove
is_inode_fast_symlinkand just opencode it in the few places using it
2) remove the check in this place entirely as it is not needed
3) remove the comment quoted above as it is more confusing than not having the comment
Got it, thanks!
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Is there any good reasons to use buffer heads like this in new code vs directly using bios?
This page can save in bdev page cache, it contains not only the erofs superblock so it can be fetched in page cache later.
If you want it in the page cache why not use
read_mapping_pageor similar?
It's reasonable, I will change as you suggested.
(The difference is whether it has some buffer_head to the sb page or not...)
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
fs/erofs/super.c line 198
Quoted 5 lines
fs/erofs/super.c new file +4
-0+/* set up default EROFS parameters */ +static void default_options (struct erofs_sb_info *sbi) +{ +}No need to add an empty function.
Quoted 4 lines
Quoted 1 line
Later patch will fill this function.
Please only add the function in the patch actually adding the functionality.
That was my fault when spilting patches...considering it's an >7KLOC filesystem (maybe spilting the whole xfs or ext4 properly is more harder)... Anyway, that is my fault.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
+}
Why is this needed? You can just free your sb privatte information in ->put_super and wire up
kill_block_superas the ->kill_sb method directly.
Quoted 5 lines
Quoted 2 lines
See Al's comments,
https://lore.kernel.org/r/[email protected]/With that code it makes sense. In this paticular patch it does not. So please add it only when actually needed.
Same as above...
Thanks,
Gao Xiang
Re: [PATCH v6 03/24] erofs: add super block operations
Hi Christoph,
On Sat, Aug 31, 2019 at 01:15:10AM +0800, Gao Xiang wrote:
[]
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
fs/erofs/super.c line 62
Quoted 4 lines
fs/erofs/super.c new file +3
-0+/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ +if (is_inode_fast_symlink (inode)) +kfree (inode->i_link);
is_inode_fast_symlinkonly shows up in a later patch. And really obsfucates the check here in the only caller as you can just do an unconditional kfree here -i_linkwill be NULL except for the case where you explicitly set it.
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
I cannot fully understand your point (sorry about my English), I will reply you about this later.
With that I mean that you should:
1) remove
is_inode_fast_symlinkand just opencode it in the few places using it
2) remove the check in this place entirely as it is not needed
Add some words about this suggestion since I'm addressing this place, it seems it could not (or I am not sure at least) be freed unconditionally
union {
struct pipe_inode_info *i_pipe;
struct block_device *i_bdev;
struct cdev *i_cdev;
char *i_link;
unsigned i_dir_seq;
};while I saw what shmem did, it seems that they handle as follows:
3636 static void shmem_free_in_core_inode(struct inode *inode) 3637 {
3638 if (S_ISLNK(inode->i_mode))
3639 kfree(inode->i_link);
3640 kmem_cache_free(shmem_inode_cachep, SHMEM_I(inode));
3641 }
I think that would be some check on it to get it is a symlink (for i_dir_seq it seems unsafe).... I think the original check is ok but I will opencode it instead.
Thanks,
Gao Xiang
Re: [PATCH v6 03/24] erofs: add super block operations
In reply to Gao XiangQuoted 22 lines · Gao Xiang
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Gao Xiang
Quoted 1 line
Please use an
erofs_prefix for all your functions.It is already a static function, I have no idea what is wrong here.
Which part of all wasn't clear? Have you looked at the prefixes for most functions in the various other big filesystems?
I will add erofs prefix to
free_inodeas you said.At least, all non-prefix functions in erofs are all static functions, it won't pollute namespace... I will add "
erofs_" to other meaningful callbacks...And as you can see...
cifs/cifsfs.c
1303:cifs_init_inodecache(void)
1509: rc =cifs_init_inodecache();
Quoted 8 lines
hpfs/super.c
254:static intinit_inodecache(void)
771: int err =init_inodecache();
minix/inode.c
84:static int__initinit_inodecache(void)
665: int err =init_inodecache();
Hi Gao,
"They did it first" is never a good reply for code review comments. Nobody cares if you copy&paste code with init_inodecache(). I understand why you thought static function names do not pollute the (linker) namespace, but they do pollute the global namespace.
free_inode() as a local function name is one of the worst examples for VFS namespace pollution.
VFS code uses function names like those a lot in the global namespace, e.g.:
clear_inode(),new_inode().
For example from recent history of namespace collision caused by your line of thinking, see:e6fd2093a85d md: namespace private helper names
Besides, you really have nothing to loose from prefixing everything with erofs_, do you? It's better for review, for debugging...
Thanks,
Amir.
Re: [PATCH v6 03/24] erofs: add super block operations
Quoted 45 lines · Amir Goldstein
Quoted 22 lines · Gao Xiang
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Gao Xiang
Quoted 1 line
Please use an
erofs_prefix for all your functions.It is already a static function, I have no idea what is wrong here.
Which part of all wasn't clear? Have you looked at the prefixes for most functions in the various other big filesystems?
I will add erofs prefix to
free_inodeas you said.At least, all non-prefix functions in erofs are all static functions, it won't pollute namespace... I will add "
erofs_" to other meaningful callbacks...And as you can see...
cifs/cifsfs.c
1303:cifs_init_inodecache(void)
1509: rc =cifs_init_inodecache();Quoted 8 lines
hpfs/super.c
254:static intinit_inodecache(void)
771: int err =init_inodecache();
minix/inode.c
84:static int__initinit_inodecache(void)
665: int err =init_inodecache();Hi Gao,
"They did it first" is never a good reply for code review comments. Nobody cares if you copy&paste code with
init_inodecache(). I understand why you thought static function names do not pollute the (linker) namespace, but they do pollute the global namespace.
free_inode()as a local function name is one of the worst examples for VFS namespace pollution.VFS code uses function names like those a lot in the global namespace, e.g.:
clear_inode(),new_inode().
Quoted 6 lines
For example from recent history of namespace collision caused by your line of thinking, see:
e6fd2093a85dmd: namespace private helper namesBesides, you really have nothing to loose from prefixing everything with
erofs_, do you? It's better for review, for debugging...
Hi Amir,
Thanks for you kind reply...
Yes, I understand that some generic header files could have the same function names and cause bad behaviors...
I will fix them, my only one question is "if all function/variable names are prefixed with "erofs_" (including all inline helpers in header files), it seems somewhat strange... (too many statements start "erofs_" in the source code...)"
I will fix common and short names at once...
Thanks,
Gao Xiang
Quoted 3 lines
Thanks,
Amir.
Re: [PATCH v6 03/24] erofs: add super block operations
In reply to Christoph HellwigHi Christoph,
Here is also my redo-ed comments...
Quoted 12 lines · Christoph Hellwig
fs/erofs/super.c line 35
Quoted 8 lines · Gao Xiang
fs/erofs/super.c new file +8
-0+static int __init erofs_init_inode_cache (void ) +{ + erofs_inode_cachep =kmem_cache_create ("erofs_inode" , +sizeof (struct erofs_vnode ),0 , +SLAB_RECLAIM_ACCOUNT , + init_once); + +return erofs_inode_cachep ?0 : -ENOMEM ;Please just use normal if/else. Also having this function seems entirely pointless.
Fixed in
https://lore.kernel.org/r/[email protected]/
Quoted 7 lines
fs/erofs/super.c line 41
Quoted 5 lines
fs/erofs/super.c new file +4
-0+static void erofs_exit_inode_cache (void ) +{ +kmem_cache_destroy (erofs_inode_cachep); +}Same for this one.
Fixed in
https://lore.kernel.org/r/[email protected]/
Quoted 4 lines
Quoted 2 lines
+static void
free_inode(struct inode *inode)Please use an
erofs_prefix for all your functions.
free_inode and most short, common static functions are fixed in
https://lore.kernel.org/r/[email protected]/
For all non-static functions, all are prefixed with "erofs_"
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+{ + struct erofs_vnode *vi = EROFS_V(inode);Why is this called vnode instead of inode? That seems like a rather odd naming for a Linux file system.
Fixed in
https://lore.kernel.org/r/[email protected]/
Quoted 10 lines
fs/erofs/super.c line 62
Quoted 5 lines
fs/erofs/super.c new file +4
-0+ +/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ +if (is_inode_fast_symlink (inode)) +kfree (inode->i_link);
is_inode_fast_symlinkonly shows up in a later patch. And really obsfucates the check here in the only caller as you can just do an unconditional kfree here -i_linkwill be NULL except for the case where you explicitly set it.
Fixed in
https://lore.kernel.org/r/[email protected]/
and with my following comments....
https://lore.kernel.org/r/20190831005446.GA233871@architecture4/
Quoted 11 lines
Also this code is nothing like ext4, so the code seems a little confusing.
fs/erofs/super.c line 70
Quoted 4 lines
fs/erofs/super.c new file +4
-0+static bool check_layout_compatibility (struct super_block *sb, +struct erofs_super_block *layout) +{ +const unsigned int requirements =le32_to_cpu (layout->requirements);Why is the variable name for the on-disk subperblock layout? We usually still calls this something with sb in the name, e.g. dsb. for disk super block.
Fixed in
https://lore.kernel.org/r/[email protected]/
Quoted 13 lines
fs/erofs/super.c line 80
Quoted 10 lines
fs/erofs/super.c new file +9
-0+EROFS_SB (sb)->requirements = requirements; + +/* check if current kernel meets all mandatory requirements */ +if (requirements & (~EROFS_ALL_REQUIREMENTS )) { +errln ("unidentified requirements %x, please upgrade kernel version" , + requirements & ~EROFS_ALL_REQUIREMENTS ); +return false; + } +return true;Note that normally we call this features, but that doesn't really matter too much.
No modification at this... (some comments already right here...)
20 /* 128-byte erofs on-disk super block */
21 struct erofs_super_block {
...
24 __le32 features; /* (aka. feature_compat) */
...
38 __le32 requirements; /* (aka. feature_incompat) */
...
41 };
Quoted 13 lines
fs/erofs/super.c line 91
Quoted 10 lines
fs/erofs/super.c new file +9
-0+static int superblock_read (struct super_block *sb) +{ +struct erofs_sb_info *sbi; +struct buffer_head *bh; +struct erofs_super_block *layout; +unsigned int blkszbits; +int ret; + + bh =sb_bread (sb,0 );Is there any good reasons to use buffer heads like this in new code vs directly using bios?
As you said, I want it in the page cache.
The reason "why not use read_mapping_page or similar?" is simply read_mapping_page -> .readpage -> (for bdev inode) block_read_full_page
-> create_page_buffers anyway...
sb_bread haven't obsoleted... It has similar function though...
Quoted 17 lines
Quoted 4 lines
fs/erofs/super.c new file +3
-0+ + sbi->blocks =le32_to_cpu (layout->blocks); + sbi->meta_blkaddr =le32_to_cpu (layout->meta_blkaddr);fs/erofs/super.c line 130
Quoted 10 lines
fs/erofs/super.c new file +10
-0+ sbi->islotbits =ffs (sizeof (struct erofs_inode_v1 )) -1 ; + sbi->root_nid =le16_to_cpu (layout->root_nid); + sbi->inos =le64_to_cpu (layout->inos); + + sbi->build_time =le64_to_cpu (layout->build_time); + sbi->build_time_nsec =le32_to_cpu (layout->build_time_nsec); + +memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name));
s_uuidshould preferably be auuid_t(assuming it is a real BE uuid, if it is le it should be aguid_t).
For this case, I have no idea how to deal with...
I have little knowledge about this uuid stuff, so I just copied from f2fs... (Could be no urgent of this field...)
Quoted 7 lines
fs/erofs/super.c line 198
Quoted 5 lines
fs/erofs/super.c new file +4
-0+/* set up default EROFS parameters */ +static void default_options (struct erofs_sb_info *sbi) +{ +}No need to add an empty function.
My fault of spilting patches...
Quoted 14 lines
fs/erofs/super.c line 249
Quoted 9 lines
fs/erofs/super.c new file +8
-0+static int erofs_fill_super (struct super_block *sb,void *data,int silent) +{ +struct inode *inode; +struct erofs_sb_info *sbi; +int err; + +infoln ("fill_super, device -> %s" , sb->s_id); +infoln ("options -> %s" , (char *)data);That is some very verbose debug info. We usually don't add that and let people trace the function instead. Also you should probably implement the new mount API.
new mount API.
Fixed in
https://lore.kernel.org/r/[email protected]/
(For new mount API,
https://lore.kernel.org/r/[email protected]/
, I will a look later)
Quoted 12 lines
Quoted 12 lines
fs/erofs/super.c new file +11
-0+static void erofs_kill_sb (struct super_block *sb) +{ +struct erofs_sb_info *sbi; + +WARN_ON (sb->s_magic !=EROFS_SUPER_MAGIC ); +infoln ("unmounting for %s" , sb->s_id); + +kill_block_super (sb); + + sbi =EROFS_SB (sb); +if (!sbi)
Quoted 8 lines
fs/erofs/super.c line 328
Quoted 4 lines
fs/erofs/super.c new file +4
-0+return ; +kfree (sbi); + sb->s_fs_info =NULL ; +}Why is this needed? You can just free your sb privatte information in ->put_super and wire up
kill_block_superas the ->kill_sb method directly.
The background is Al's comments in erofs v2....
(which simplify erofs_fill_super logic)
https://lore.kernel.org/r/[email protected]/
with a specific notation...
https://lore.kernel.org/r/[email protected]/
"
Quoted 2 lines
OTOH, for the case of NULL ->s_root ->put_super() won't be called at all, so in that case you need it directly in ->kill_sb().
"
Thanks,
Gao Xiang
Re: [PATCH v6 03/24] erofs: add super block operations
Quoted 10 lines · Gao Xiang
No modification at this... (some comments already right here...)
20 /* 128-byte erofs on-disk super block */
21 structerofs_super_block{
...
24__le32features; /* (aka.feature_compat) */
...
38__le32requirements; /* (aka.feature_incompat) */
...
41 };
This is only cosmetic, why not stick to feature_compat and feature_incompat?
Quoted 5 lines
Quoted 4 lines
fs/erofs/super.c line 91
Quoted 1 line
fs/erofs/super.c new file +1
-0+ bh =sb_bread (sb,0 );Is there any good reasons to use buffer heads like this in new code vs directly using bios?
Quoted 7 lines
As you said, I want it in the page cache.
The reason "why not use
read_mapping_pageor similar?" is simplyread_mapping_page-> .readpage -> (for bdev inode)block_read_full_page
->create_page_buffersanyway...
sb_breadhaven't obsoleted... It has similar function though...
With the different that it keeps you isolated from the buffer_head internals. This seems to be your only direct use of buffer heads, which while not deprecated are a bit of an ugly step child. So if you can easily avoid creating a buffer_head dependency in a new filesystem I think you should avoid it.
Quoted 10 lines
Quoted 9 lines
fs/erofs/super.c line 130
Quoted 6 lines
fs/erofs/super.c new file +6
-0+ sbi->build_time =le64_to_cpu (layout->build_time); + sbi->build_time_nsec =le32_to_cpu (layout->build_time_nsec); + +memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name));
s_uuidshould preferably be auuid_t(assuming it is a real BE uuid, if it is le it should be aguid_t).
Quoted 3 lines
For this case, I have no idea how to deal with...
I have little knowledge about this uuid stuff, so I just copied from f2fs... (Could be no urgent of this field...)
Who fills out this field in the on-disk format and how?
Quoted 11 lines
The background is Al's comments in erofs v2....
(which simplifyerofs_fill_superlogic)
https://lore.kernel.org/r/[email protected]/with a specific notation...
https://lore.kernel.org/r/[email protected]/"
Quoted 2 lines
OTOH, for the case of NULL ->s_root ->put_super() won't be called at all, so in that case you need it directly in ->kill_sb().
"
Yes. Although none of that is relevant for this initial version, just after more features are added.
Re: [PATCH v6 03/24] erofs: add super block operations
Hi Christoph,
Quoted 12 lines · Christoph Hellwig
Quoted 10 lines · Gao Xiang
No modification at this... (some comments already right here...)
20 /* 128-byte erofs on-disk super block */
21 structerofs_super_block{
...
24__le32features; /* (aka.feature_compat) */
...
38__le32requirements; /* (aka.feature_incompat) */
...
41 };
Quoted 2 lines
This is only cosmetic, why not stick to
feature_compatandfeature_incompat?
Okay, will fix. (however, in my mind, I'm some confused why "features" could be incompatible...)
Quoted 12 lines
Quoted 12 lines
Quoted 5 lines
fs/erofs/super.c line 91
Quoted 2 lines
fs/erofs/super.c new file +1
-0+ bh =sb_bread (sb,0 );Is there any good reasons to use buffer heads like this in new code vs directly using bios?
As you said, I want it in the page cache.
The reason "why not use
read_mapping_pageor similar?" is simplyread_mapping_page-> .readpage -> (for bdev inode)block_read_full_page
->create_page_buffersanyway...
Quoted 7 lines
Quoted 1 line
sb_breadhaven't obsoleted... It has similar function though...With the different that it keeps you isolated from the
buffer_headinternals. This seems to be your only direct use of buffer heads, which while not deprecated are a bit of an ugly step child. So if you can easily avoid creating abuffer_headdependency in a new filesystem I think you should avoid it.
OK, let's use read_mapping_page instead.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
fs/erofs/super.c line 130
Quoted 7 lines
fs/erofs/super.c new file +6
-0+ sbi->build_time =le64_to_cpu (layout->build_time); + sbi->build_time_nsec =le32_to_cpu (layout->build_time_nsec); + +memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name));
s_uuidshould preferably be auuid_t(assuming it is a real BE uuid, if it is le it should be aguid_t).
Quoted 5 lines
Quoted 3 lines
For this case, I have no idea how to deal with...
I have little knowledge about this uuid stuff, so I just copied from f2fs... (Could be no urgent of this field...)Who fills out this field in the on-disk format and how?
mkfs.erofs, but this field leaves 0 for now. Is that reasonable? (using libuuid can generate it easily...)
Quoted 8 lines
Quoted 8 lines
The background is Al's comments in erofs v2....
(which simplifyerofs_fill_superlogic)
https://lore.kernel.org/r/[email protected]/with a specific notation...
https://lore.kernel.org/r/[email protected]/
Quoted 7 lines
Quoted 4 lines
"
Quoted 2 lines
OTOH, for the case of NULL ->s_root ->put_super() won't be called at all, so in that case you need it directly in ->kill_sb().
"
Yes. Although none of that is relevant for this initial version, just after more features are added.
This patch uses it actually... since no failure path in erofs_fill_super() and s_root will be filled nearly at the end of the function...
Thanks,
Gao Xiang
Re: [PATCH v6 03/24] erofs: add super block operations
Quoted 8 lines · Gao Xiang
Hi Christoph,
Quoted 7 lines
Quoted 6 lines
...
24__le32features; /* (aka.feature_compat) */
...
38__le32requirements; /* (aka.feature_incompat) */
...
41 };
Quoted 5 lines
Quoted 2 lines
This is only cosmetic, why not stick to
feature_compatandfeature_incompat?Okay, will fix. (however, in my mind, I'm some confused why "features" could be incompatible...)
The feature is incompatible if it requires changes to the driver. An easy to understand historic example is that ext3 originally did not have the file types in the directory entry. Adding them means old file system drivers can not read a file system with this new feature, so an incompat flag has to be added.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
fs/erofs/super.c line 130
Quoted 3 lines
fs/erofs/super.c new file +3
-0+memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name));
s_uuidshould preferably be auuid_t(assuming it is a real BE uuid, if it is le it should be aguid_t).
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
For this case, I have no idea how to deal with...
I have little knowledge about this uuid stuff, so I just copied from f2fs... (Could be no urgent of this field...)Who fills out this field in the on-disk format and how?
mkfs.erofs, but this field leaves 0 for now. Is that reasonable? (using libuuid can generate it easily...)
If the filed is always zero for now please don't fill it out. If you decide it is worth adding the uuid eventually please add a compat feature flag that you have a valid uuid and only fill out the field if the file system actualy has a valid uuid.
Re: [PATCH v6 03/24] erofs: add super block operations
Hi Christoph,
Quoted 12 lines · Christoph Hellwig
Quoted 11 lines · Gao Xiang
Hi Christoph,
Quoted 9 lines
Quoted 6 lines
...
24__le32features; /* (aka.feature_compat) */
...
38__le32requirements; /* (aka.feature_incompat) */
...
41 };This is only cosmetic, why not stick to
feature_compatandfeature_incompat?
Quoted 8 lines
Quoted 2 lines
Okay, will fix. (however, in my mind, I'm some confused why "features" could be incompatible...)
The feature is incompatible if it requires changes to the driver. An easy to understand historic example is that ext3 originally did not have the file types in the directory entry. Adding them means old file system drivers can not read a file system with this new feature, so an incompat flag has to be added.
Got it.
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 11 lines
Quoted 7 lines
fs/erofs/super.c line 130
Quoted 4 lines
fs/erofs/super.c new file +3
-0+memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (sbi->volume_name, layout->volume_name, +sizeof (layout->volume_name));
s_uuidshould preferably be auuid_t(assuming it is a real BE uuid, if it is le it should be aguid_t).For this case, I have no idea how to deal with...
I have little knowledge about this uuid stuff, so I just copied from f2fs... (Could be no urgent of this field...)Who fills out this field in the on-disk format and how?
Quoted 7 lines
Quoted 2 lines
mkfs.erofs, but this field leaves 0 for now. Is that reasonable? (using libuuid can generate it easily...)
If the filed is always zero for now please don't fill it out. If you decide it is worth adding the uuid eventually please add a compat feature flag that you have a valid uuid and only fill out the field if the file system actualy has a valid uuid.
Okay. Will do that then (as a note here).
Thanks,
Gao Xiang
PATCH v6 4/24 erofs: add raw address_space operations
In reply to Gao XiangThis commit adds functions for meta and raw data, and also provides address_space_operations for raw data access.
--- fs/erofs/data.c | 423 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 423 insertions(+) create mode 100644 fs/erofs/data.c
fs/erofs/data.c new file +423 -0
@@ -0,0 +1,423 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/data.c +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#include "internal.h" +#include <linux/prefetch.h> + +#include <trace/events/erofs.h> + +static inline void read_endio (struct bio *bio) +{ +struct super_block *const sb = bio->bi_private; +struct bio_vec *bvec; +blk_status_t err = bio->bi_status; +struct bvec_iter_all iter_all; + +if (time_to_inject (EROFS_SB (sb),FAULT_READ_IO )) { +erofs_show_injection_info (FAULT_READ_IO ); + err =BLK_STS_IOERR ; + } + +bio_for_each_segment_all (bvec, bio, iter_all) { +struct page *page = bvec->bv_page; + +/* page is already locked */ +DBG_BUGON (PageUptodate (page)); + +if (unlikely (err)) +SetPageError (page); +else +SetPageUptodate (page); + +unlock_page (page); +/* page could be reclaimed now */ + } +bio_put (bio); +} + +/* prio -- true is used for dir */ +struct page *__erofs_get_meta_page (struct super_block *sb, +erofs_blk_t blkaddr,bool prio,bool nofail) +{ +struct inode *const bd_inode = sb->s_bdev->bd_inode; +struct address_space *const mapping = bd_inode->i_mapping; +/* prefer retrying in the allocator to blindly looping below */ +const gfp_t gfp =mapping_gfp_constraint (mapping, ~__GFP_FS) | + (nofail ? __GFP_NOFAIL :0 ); +unsigned int io_retries = nofail ?EROFS_IO_MAX_RETRIES_NOFAIL :0 ; +struct page *page; +int err; + +repeat: + page =find_or_create_page (mapping, blkaddr, gfp); +if (unlikely (!page)) { +DBG_BUGON (nofail); +return ERR_PTR (-ENOMEM ); + } +DBG_BUGON (!PageLocked (page)); + +if (!PageUptodate (page)) { +struct bio *bio; + + bio =erofs_grab_bio (sb, blkaddr,1 , sb, read_endio, nofail); +if (IS_ERR (bio)) { +DBG_BUGON (nofail); + err =PTR_ERR (bio); + goto err_out; + } + + err =bio_add_page (bio, page,PAGE_SIZE ,0 ); +if (unlikely (err !=PAGE_SIZE )) { + err = -EFAULT ; + goto err_out; + } + +__submit_bio (bio,REQ_OP_READ , +REQ_META | (prio ?REQ_PRIO :0 )); + +lock_page (page); + +/* this page has been truncated by others */ +if (unlikely (page->mapping != mapping)) { +unlock_repeat: +unlock_page (page); +put_page (page); + goto repeat; + } + +/* more likely a read error */ +if (unlikely (!PageUptodate (page))) { +if (io_retries) { + --io_retries; + goto unlock_repeat; + } + err = -EIO ; + goto err_out; + } + } +return page; + +err_out: +unlock_page (page); +put_page (page); +return ERR_PTR (err); +} + +static int erofs_map_blocks_flatmode (struct inode *inode, +struct erofs_map_blocks *map, +int flags) +{ +int err =0 ; +erofs_blk_t nblocks, lastblk; +u64 offset = map->m_la; +struct erofs_vnode *vi =EROFS_V (inode); + +trace_erofs_map_blocks_flatmode_enter (inode, map, flags); + + nblocks =DIV_ROUND_UP (inode->i_size,PAGE_SIZE ); + lastblk = nblocks -is_inode_flat_inline (inode); + +if (unlikely (offset >= inode->i_size)) { +/* leave out-of-bound access unmapped */ + map->m_flags =0 ; + map->m_plen =0 ; + goto out; + } + +/* there is no hole in flatmode */ + map->m_flags =EROFS_MAP_MAPPED ; + +if (offset <blknr_to_addr (lastblk)) { + map->m_pa =blknr_to_addr (vi->raw_blkaddr) + map->m_la; + map->m_plen =blknr_to_addr (lastblk) - offset; + }else if (is_inode_flat_inline (inode)) { +/* 2 - inode inline B: inode, [xattrs], inline last blk... */ +struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); + + map->m_pa =iloc (sbi, vi->nid) + vi->inode_isize + + vi->xattr_isize +erofs_blkoff (map->m_la); + map->m_plen = inode->i_size - offset; + +/* inline data should locate in one meta block */ +if (erofs_blkoff (map->m_pa) + map->m_plen >PAGE_SIZE ) { +DBG_BUGON (1 ); + err = -EIO ; + goto err_out; + } + + map->m_flags |=EROFS_MAP_META ; + }else { +errln ("internal error @ nid: %llu (size %llu), m_la 0x%llx" , + vi->nid, inode->i_size, map->m_la); +DBG_BUGON (1 ); + err = -EIO ; + goto err_out; + } + +out: + map->m_llen = map->m_plen; + +err_out: +trace_erofs_map_blocks_flatmode_exit (inode, map, flags,0 ); +return err; +} + +int erofs_map_blocks (struct inode *inode, +struct erofs_map_blocks *map,int flags) +{ +if (is_inode_layout_compression (inode)) +return -ENOTSUPP ; + +return erofs_map_blocks_flatmode (inode, map, flags); +} + +static inline struct bio *erofs_read_raw_page (struct bio *bio, +struct address_space *mapping, +struct page *page, +erofs_off_t *last_block, +unsigned int nblocks, +bool ra) +{ +struct inode *const inode = mapping->host; +struct super_block *const sb = inode->i_sb; +erofs_off_t current_block = (erofs_off_t )page->index; +int err; + +DBG_BUGON (!nblocks); + +if (PageUptodate (page)) { + err =0 ; + goto has_updated; + } + +if (cleancache_get_page (page) ==0 ) { + err =0 ; +SetPageUptodate (page); + goto has_updated; + } + +/* note that for readpage case, bio also equals to NULL */ +if (bio && +/* not continuous */ + *last_block +1 != current_block) { +submit_bio_retry: +__submit_bio (bio,REQ_OP_READ ,0 ); + bio =NULL ; + } + +if (!bio) { +struct erofs_map_blocks map = { + .m_la =blknr_to_addr (current_block), + }; +erofs_blk_t blknr; +unsigned int blkoff; + + err =erofs_map_blocks (inode, &map,EROFS_GET_BLOCKS_RAW ); +if (unlikely (err)) + goto err_out; + +/* zero out the holed page */ +if (unlikely (!(map.m_flags &EROFS_MAP_MAPPED ))) { +zero_user_segment (page,0 ,PAGE_SIZE ); +SetPageUptodate (page); + +/* imply err = 0, see erofs_map_blocks */ + goto has_updated; + } + +/* for RAW access mode, m_plen must be equal to m_llen */ +DBG_BUGON (map.m_plen != map.m_llen); + + blknr =erofs_blknr (map.m_pa); + blkoff =erofs_blkoff (map.m_pa); + +/* deal with inline page */ +if (map.m_flags &EROFS_MAP_META ) { +void *vsrc, *vto; +struct page *ipage; + +DBG_BUGON (map.m_plen >PAGE_SIZE ); + + ipage =erofs_get_meta_page (inode->i_sb, blknr,0 ); + +if (IS_ERR (ipage)) { + err =PTR_ERR (ipage); + goto err_out; + } + + vsrc =kmap_atomic (ipage); + vto =kmap_atomic (page); +memcpy (vto, vsrc + blkoff, map.m_plen); +memset (vto + map.m_plen,0 ,PAGE_SIZE - map.m_plen); +kunmap_atomic (vto); +kunmap_atomic (vsrc); +flush_dcache_page (page); + +SetPageUptodate (page); +/* TODO: could we unlock the page earlier? */ +unlock_page (ipage); +put_page (ipage); + +/* imply err = 0, see erofs_map_blocks */ + goto has_updated; + } + +/* pa must be block-aligned for raw reading */ +DBG_BUGON (erofs_blkoff (map.m_pa)); + +/* max # of continuous pages */ +if (nblocks >DIV_ROUND_UP (map.m_plen,PAGE_SIZE )) + nblocks =DIV_ROUND_UP (map.m_plen,PAGE_SIZE ); +if (nblocks >BIO_MAX_PAGES ) + nblocks =BIO_MAX_PAGES ; + + bio =erofs_grab_bio (sb, blknr, nblocks, sb, + read_endio, false); +if (IS_ERR (bio)) { + err =PTR_ERR (bio); + bio =NULL ; + goto err_out; + } + } + + err =bio_add_page (bio, page,PAGE_SIZE ,0 ); +/* out of the extent or bio is full */ +if (err <PAGE_SIZE ) + goto submit_bio_retry; + + *last_block = current_block; + +/* shift in advance in case of it followed by too many gaps */ +if (bio->bi_iter.bi_size >= bio->bi_max_vecs *PAGE_SIZE ) { +/* err should reassign to 0 after submitting */ + err =0 ; + goto submit_bio_out; + } + +return bio; + +err_out: +/* for sync reading, set page error immediately */ +if (!ra) { +SetPageError (page); +ClearPageUptodate (page); + } +has_updated: +unlock_page (page); + +/* if updated manually, continuous pages has a gap */ +if (bio) +submit_bio_out: +__submit_bio (bio,REQ_OP_READ ,0 ); + +return unlikely (err) ?ERR_PTR (err) :NULL ; +} + +/* +* since we dont have write or truncate flows, so no inode +* locking needs to be held at the moment. +*/ +static int erofs_raw_access_readpage (struct file *file,struct page *page) +{ +erofs_off_t last_block; +struct bio *bio; + +trace_erofs_readpage (page, true); + + bio =erofs_read_raw_page (NULL , page->mapping, + page, &last_block,1 , false); + +if (IS_ERR (bio)) +return PTR_ERR (bio); + +DBG_BUGON (bio);/* since we have only one bio -- must be NULL */ +return 0 ; +} + +static int erofs_raw_access_readpages (struct file *filp, +struct address_space *mapping, +struct list_head *pages, +unsigned int nr_pages) +{ +erofs_off_t last_block; +struct bio *bio =NULL ; +gfp_t gfp =readahead_gfp_mask (mapping); + struct page *page = list_last_entry(pages, struct page, lru); + +trace_erofs_readpages (mapping->host, page, nr_pages, true); + +for (; nr_pages; --nr_pages) { + page = list_entry(pages->prev, struct page, lru); + +prefetchw (&page->flags); +list_del (&page->lru); + +if (!add_to_page_cache_lru (page, mapping, page->index, gfp)) { + bio =erofs_read_raw_page (bio, mapping, page, + &last_block, nr_pages, true); + +/* all the page errors are ignored when readahead */ +if (IS_ERR (bio)) { +pr_err ("%s, readahead error at page %lu of nid %llu\n" , + __func__, page->index, +EROFS_V (mapping->host)->nid); + + bio =NULL ; + } + } + +/* pages could still be locked */ +put_page (page); + } +DBG_BUGON (!list_empty (pages)); + +/* the rare case (end in gaps) */ +if (unlikely (bio)) +__submit_bio (bio,REQ_OP_READ ,0 ); +return 0 ; +} + +static int erofs_get_block (struct inode *inode,sector_t iblock, +struct buffer_head *bh,int create) +{ +struct erofs_map_blocks map = { + .m_la = iblock <<9 , + }; +int err; + + err =erofs_map_blocks (inode, &map,EROFS_GET_BLOCKS_RAW ); +if (err) +return err; + +if (map.m_flags &EROFS_MAP_MAPPED ) + bh->b_blocknr =erofs_blknr (map.m_pa); + +return err; +} + +static sector_t erofs_bmap (struct address_space *mapping,sector_t block) +{ +struct inode *inode = mapping->host; + +if (is_inode_flat_inline (inode)) { +erofs_blk_t blks =i_size_read (inode) >>LOG_BLOCK_SIZE ; + +if (block >>LOG_SECTORS_PER_BLOCK >= blks) +return 0 ; + } + +return generic_block_bmap (mapping, block, erofs_get_block); +} + +/* for uncompressed (aligned) files and raw access for other files */ +const struct address_space_operations erofs_raw_access_aops = { + .readpage = erofs_raw_access_readpage, + .readpages = erofs_raw_access_readpages, + .bmap = erofs_bmap, +}; +
Signature · 1 line
2.17.1
Re: [PATCH v6 04/24] erofs: add raw address_space operations
The actual address_space operations seem to largely duplicate the iomap versions. Please use those instead. Also I don't think any new file system should write up ->bmap these days.
Re: [PATCH v6 04/24] erofs: add raw address_space operations
Hi Christoph,
Quoted 3 lines · Christoph Hellwig
The actual
address_spaceoperations seem to largely duplicate the iomap versions. Please use those instead. Also I don't think any new file system should write up ->bmap these days.
iomap doesn't support tail-end packing inline data till now, I think Chao and I told you and Andreas before [1].
Since EROFS keeps a self-contained driver for now, we will use iomap if it supports tail-end packing inline data later.
[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
Thanks,
Gao Xiang
Re: [PATCH v6 04/24] erofs: add raw address_space operations
Quoted 7 lines · Gao Xiang
Hi Christoph,
Quoted 3 lines · Christoph Hellwig
The actual
address_spaceoperations seem to largely duplicate the iomap versions. Please use those instead. Also I don't think any new file system should write up ->bmap these days.
Quoted 5 lines
iomap doesn't support tail-end packing inline data till now, I think Chao and I told you and Andreas before [1].
Since EROFS keeps a self-contained driver for now, we will use iomap if it supports tail-end packing inline data later.
Well, so work with the maintainers to enhance the core kernel. That is how Linux development works. We've added various iomap enhancements for gfs in the last merge windows, and we've added more for the brand new zonefs file system we plan to merge for 5.4.
Re: [PATCH v6 04/24] erofs: add raw address_space operations
Hi Christoph,
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Gao Xiang
Hi Christoph,
Quoted 3 lines · Christoph Hellwig
The actual
address_spaceoperations seem to largely duplicate the iomap versions. Please use those instead. Also I don't think any new file system should write up ->bmap these days.iomap doesn't support tail-end packing inline data till now, I think Chao and I told you and Andreas before [1].
Quoted 7 lines
Quoted 2 lines
Since EROFS keeps a self-contained driver for now, we will use iomap if it supports tail-end packing inline data later.
Well, so work with the maintainers to enhance the core kernel. That is how Linux development works. We've added various iomap enhancements for gfs in the last merge windows, and we've added more for the brand new zonefs file system we plan to merge for 5.4.
That is a good idea, I think Chao will continue working on this (adding tail-end packing inline approach into iomap, thus we can
have few code in data.c.)
Thanks,
Gao Xiang
PATCH v6 5/24 erofs: add inode operations
In reply to Gao XiangThis adds core functions to get, read an inode.
It adds statx support as well.
--- fs/erofs/inode.c | 291 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 291 insertions(+) create mode 100644 fs/erofs/inode.c
fs/erofs/inode.c new file +291 -0
@@ -0,0 +1,291 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/inode.c +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#include "internal.h" + +#include <trace/events/erofs.h> + +/* no locking */ +static int read_inode (struct inode *inode,void *data) +{ +struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode_v1 *v1 = data; +const unsigned int advise =le16_to_cpu (v1->i_advise); +erofs_blk_t nblks =0 ; + + vi->datamode =__inode_data_mapping (advise); + +if (unlikely (vi->datamode >=EROFS_INODE_LAYOUT_MAX )) { +errln ("unsupported data mapping %u of nid %llu" , + vi->datamode, vi->nid); +DBG_BUGON (1 ); +return -EIO ; + } + +if (__inode_version (advise) ==EROFS_INODE_LAYOUT_V2 ) { +struct erofs_inode_v2 *v2 = data; + + vi->inode_isize =sizeof (struct erofs_inode_v2 ); + vi->xattr_isize =ondisk_xattr_ibody_size (v2->i_xattr_icount); + + inode->i_mode =le16_to_cpu (v2->i_mode); + vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr); + +i_uid_write (inode,le32_to_cpu (v2->i_uid)); +i_gid_write (inode,le32_to_cpu (v2->i_gid)); +set_nlink (inode,le32_to_cpu (v2->i_nlink)); + +/* ns timestamp */ + inode->i_mtime.tv_sec = inode->i_ctime.tv_sec = +le64_to_cpu (v2->i_ctime); + inode->i_mtime.tv_nsec = inode->i_ctime.tv_nsec = +le32_to_cpu (v2->i_ctime_nsec); + + inode->i_size =le64_to_cpu (v2->i_size); + +/* total blocks for compressed files */ +if (is_inode_layout_compression (inode)) + nblks =le32_to_cpu (v2->i_u.compressed_blocks); + }else if (__inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) { +struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); + + vi->inode_isize =sizeof (struct erofs_inode_v1 ); + vi->xattr_isize =ondisk_xattr_ibody_size (v1->i_xattr_icount); + + inode->i_mode =le16_to_cpu (v1->i_mode); + vi->raw_blkaddr =le32_to_cpu (v1->i_u.raw_blkaddr); + +i_uid_write (inode,le16_to_cpu (v1->i_uid)); +i_gid_write (inode,le16_to_cpu (v1->i_gid)); +set_nlink (inode,le16_to_cpu (v1->i_nlink)); + +/* use build time to derive all file time */ + inode->i_mtime.tv_sec = inode->i_ctime.tv_sec = + sbi->build_time; + inode->i_mtime.tv_nsec = inode->i_ctime.tv_nsec = + sbi->build_time_nsec; + + inode->i_size =le32_to_cpu (v1->i_size); +if (is_inode_layout_compression (inode)) + nblks =le32_to_cpu (v1->i_u.compressed_blocks); + }else { +errln ("unsupported on-disk inode version %u of nid %llu" , +__inode_version (advise), vi->nid); +DBG_BUGON (1 ); +return -EIO ; + } + +if (!nblks) +/* measure inode.i_blocks as generic filesystems */ + inode->i_blocks =roundup (inode->i_size,EROFS_BLKSIZ ) >>9 ; +else + inode->i_blocks = nblks <<LOG_SECTORS_PER_BLOCK ; +return 0 ; +} + +/* +* try_lock can be required since locking order is: +* file data(fs_inode) +* meta(bd_inode) +* but the majority of the callers is "iget", +* in that case we are pretty sure no deadlock since +* no data operations exist. However I tend to +* try_lock since it takes no much overhead and +* will success immediately. +*/ +static int fill_inline_data (struct inode *inode,void *data, +unsigned int m_pofs) +{ +struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_sb_info *sbi =EROFS_I_SB (inode); + +/* should be inode inline C */ +if (!is_inode_flat_inline (inode)) +return 0 ; + +/* fast symlink (following ext4) */ +if (S_ISLNK (inode->i_mode) && inode->i_size <PAGE_SIZE ) { +char *lnk =erofs_kmalloc (sbi, inode->i_size +1 ,GFP_KERNEL ); + +if (unlikely (!lnk)) +return -ENOMEM ; + + m_pofs += vi->inode_isize + vi->xattr_isize; + +/* inline symlink data shouldn't across page boundary as well */ +if (unlikely (m_pofs + inode->i_size >PAGE_SIZE )) { +DBG_BUGON (1 ); +kfree (lnk); +return -EIO ; + } + +/* get in-page inline data */ +memcpy (lnk, data + m_pofs, inode->i_size); + lnk[inode->i_size] ='\0' ; + + inode->i_link = lnk; +set_inode_fast_symlink (inode); + } +return 0 ; +} + +static int fill_inode (struct inode *inode,int isdir) +{ +struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); +struct erofs_vnode *vi =EROFS_V (inode); +struct page *page; +void *data; +int err; +erofs_blk_t blkaddr; +unsigned int ofs; + +trace_erofs_fill_inode (inode, isdir); + + blkaddr =erofs_blknr (iloc (sbi, vi->nid)); + ofs =erofs_blkoff (iloc (sbi, vi->nid)); + +debugln ("%s, reading inode nid %llu at %u of blkaddr %u" , + __func__, vi->nid, ofs, blkaddr); + + page =erofs_get_meta_page (inode->i_sb, blkaddr, isdir); + +if (IS_ERR (page)) { +errln ("failed to get inode (nid: %llu) page, err %ld" , + vi->nid,PTR_ERR (page)); +return PTR_ERR (page); + } + +DBG_BUGON (!PageUptodate (page)); + data =page_address (page); + + err =read_inode (inode, data + ofs); +if (!err) { +/* setup the new inode */ +if (S_ISREG (inode->i_mode)) { + inode->i_op = &erofs_generic_iops; + inode->i_fop = &generic_ro_fops; + }else if (S_ISDIR (inode->i_mode)) { + inode->i_op = &erofs_dir_iops; + inode->i_fop = &erofs_dir_fops; + }else if (S_ISLNK (inode->i_mode)) { +/* by default, page_get_link is used for symlink */ + inode->i_op = &erofs_symlink_iops; +inode_nohighmem (inode); + }else { + err = -EIO ; + goto out_unlock; + } + +if (is_inode_layout_compression (inode)) { + err = -ENOTSUPP ; + goto out_unlock; + } + + inode->i_mapping->a_ops = &erofs_raw_access_aops; + +/* fill last page if inline data is available */ + err =fill_inline_data (inode, data, ofs); + } + +out_unlock: +unlock_page (page); +put_page (page); +return err; +} + +/* +* erofs nid is 64bits, but i_ino is 'unsigned long', therefore +* we should do more for 32-bit platform to find the right inode. +*/ +#if BITS_PER_LONG ==32 +static int erofs_ilookup_test_actor (struct inode *inode,void *opaque) +{ +const erofs_nid_t nid = *(erofs_nid_t *)opaque; + +return EROFS_V (inode)->nid == nid; +} + +static int erofs_iget_set_actor (struct inode *inode,void *opaque) +{ +const erofs_nid_t nid = *(erofs_nid_t *)opaque; + + inode->i_ino =erofs_inode_hash (nid); +return 0 ; +} +#endif + +static inline struct inode *erofs_iget_locked (struct super_block *sb, +erofs_nid_t nid) +{ +const unsigned long hashval =erofs_inode_hash (nid); + +#if BITS_PER_LONG >=64 +/* it is safe to use iget_locked for >= 64-bit platform */ +return iget_locked (sb, hashval); +#else +return iget5_locked (sb, hashval, erofs_ilookup_test_actor, + erofs_iget_set_actor, &nid); +#endif +} + +struct inode *erofs_iget (struct super_block *sb, +erofs_nid_t nid, +bool isdir) +{ +struct inode *inode =erofs_iget_locked (sb, nid); + +if (unlikely (!inode)) +return ERR_PTR (-ENOMEM ); + +if (inode->i_state &I_NEW ) { +int err; +struct erofs_vnode *vi =EROFS_V (inode); + + vi->nid = nid; + + err =fill_inode (inode, isdir); +if (likely (!err)) +unlock_new_inode (inode); +else { +iget_failed (inode); + inode =ERR_PTR (err); + } + } +return inode; +} + +int erofs_getattr (const struct path *path,struct kstat *stat, +u32 request_mask,unsigned int query_flags) +{ +struct inode *const inode =d_inode (path->dentry); + +if (is_inode_layout_compression (inode)) + stat->attributes |=STATX_ATTR_COMPRESSED ; + + stat->attributes |=STATX_ATTR_IMMUTABLE ; + stat->attributes_mask |= (STATX_ATTR_COMPRESSED | +STATX_ATTR_IMMUTABLE ); + +generic_fillattr (inode, stat); +return 0 ; +} + +const struct inode_operations erofs_generic_iops = { + .getattr = erofs_getattr, +}; + +const struct inode_operations erofs_symlink_iops = { + .get_link = page_get_link, + .getattr = erofs_getattr, +}; + +const struct inode_operations erofs_fast_symlink_iops = { + .get_link = simple_get_link, + .getattr = erofs_getattr, +}; +
Signature · 1 line
2.17.1
Re: [PATCH v6 05/24] erofs: add inode operations
Quoted 26 lines · Gao Xiang
This adds core functions to get, read an inode.
It adds statx support as well.Signed-off-by Gao Xiang <[email protected]>--- fs/erofs/inode.c | 291 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 291 insertions(+) create mode 100644 fs/erofs/inode.cfs/erofs/inode.c new file +11
-0@@ -0,0 +1,291 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/inode.c +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <[email protected]> +*/ +#include "internal.h" + +#include <trace/events/erofs.h>
fs/erofs/inode.c line 21
Quoted 10 lines
fs/erofs/inode.c new file +10
-0+ +/* no locking */ +static int read_inode (struct inode *inode,void *data) +{ +struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode_v1 *v1 = data; +const unsigned int advise =le16_to_cpu (v1->i_advise); +erofs_blk_t nblks =0 ; + + vi->datamode =__inode_data_mapping (advise);
What is the deal with these magic underscores here and various other similar helpers?
fs/erofs/inode.c line 111
Quoted 1 line
fs/erofs/inode.c new file +1
-0+/* fast symlink (following ext4) */
This actually originates in FFS. But it is so common that the comment seems a little pointless.
fs/erofs/inode.c line 113
Quoted 2 lines
fs/erofs/inode.c new file +2
-0+if (S_ISLNK (inode->i_mode) && inode->i_size <PAGE_SIZE ) { +char *lnk =erofs_kmalloc (sbi, inode->i_size +1 ,GFP_KERNEL );
Please just use plain kmalloc everywhere and let the normal kernel error injection code take care of injeting any errors.
fs/erofs/inode.c line 120
Quoted 1 line
fs/erofs/inode.c new file +1
-0+/* inline symlink data shouldn't across page boundary as well */
... should not cross ..
fs/erofs/inode.c line 127
Quoted 7 lines
fs/erofs/inode.c new file +7
-0+if (unlikely (m_pofs + inode->i_size >PAGE_SIZE )) { +DBG_BUGON (1 ); +kfree (lnk); +return -EIO ; + } + +/* get in-page inline data */
s/get/copy/, but the comment seems rather pointless.
fs/erofs/inode.c line 132
Quoted 5 lines
fs/erofs/inode.c new file +5
-0+memcpy (lnk, data + m_pofs, inode->i_size); + lnk[inode->i_size] ='\0' ; + + inode->i_link = lnk; +set_inode_fast_symlink (inode);
Please just set the ops directly instead of obsfucating that in a single caller, single line inline function. And please set it instead of the normal symlink iops in the same place where you also set those.:w
fs/erofs/inode.c line 167
Quoted 2 lines
fs/erofs/inode.c new file +2
-0+ err =read_inode (inode, data + ofs); +if (!err) {
if (err) goto out_unlock;
.. and save one level of indentation.
fs/erofs/inode.c line 184
Quoted 1 line
fs/erofs/inode.c new file +1
-0+if (is_inode_layout_compression (inode)) {
The name of this helper is a little odd. But I think just opencoding it seems generally cleaner anyway.
fs/erofs/inode.c line 192
Quoted 8 lines
fs/erofs/inode.c new file +8
-0+ err = -ENOTSUPP ; + goto out_unlock; + } + + inode->i_mapping->a_ops = &erofs_raw_access_aops; + +/* fill last page if inline data is available */ + err =fill_inline_data (inode, data, ofs);
Well, I think you should move the is_inode_flat_inline and (S_ISLNK(inode->i_mode) && inode->i_size < PAGE_SIZE) checks from that helper here, as otherwise you make everyone wonder why you'd always fill out the inline data.
fs/erofs/inode.c line 233
Quoted 12 lines
fs/erofs/inode.c new file +12
-0+static inline struct inode *erofs_iget_locked (struct super_block *sb, +erofs_nid_t nid) +{ +const unsigned long hashval =erofs_inode_hash (nid); + +#if BITS_PER_LONG >=64 +/* it is safe to use iget_locked for >= 64-bit platform */ +return iget_locked (sb, hashval); +#else +return iget5_locked (sb, hashval, erofs_ilookup_test_actor, + erofs_iget_set_actor, &nid); +#endif
Just use the slightly more complicated 32-bit version everywhere so that you have a single actually tested code path. And then remove this helper.
Re: [PATCH v6 05/24] erofs: add inode operations
On Thu, Aug 29, 2019 at 03:24:26AM -0700, Christoph Hellwig wrote:
[]
Quoted 9 lines
fs/erofs/inode.c line 192
Quoted 4 lines
fs/erofs/inode.c new file +3
-0+ +/* fill last page if inline data is available */ + err =fill_inline_data (inode, data, ofs);Well, I think you should move the
is_inode_flat_inlineand (S_ISLNK(inode->i_mode)&&inode->i_size<PAGE_SIZE) checks from that helper here, as otherwise you make everyone wonder why you'd always fill out the inline data.
Currently, fill_inline_data() only fills for fast symlink, later we can fill any tail-end block (such as dir block) for our requirements.
And I think that is minor.
Quoted 17 lines
fs/erofs/inode.c line 233
Quoted 13 lines
fs/erofs/inode.c new file +12
-0+static inline struct inode *erofs_iget_locked (struct super_block *sb, +erofs_nid_t nid) +{ +const unsigned long hashval =erofs_inode_hash (nid); + +#if BITS_PER_LONG >=64 +/* it is safe to use iget_locked for >= 64-bit platform */ +return iget_locked (sb, hashval); +#else +return iget5_locked (sb, hashval, erofs_ilookup_test_actor, + erofs_iget_set_actor, &nid); +#endif Just use the slightly more complicated 32-bit version everywhere so that you have a single actually tested code path. And then remove this helper.
The consideration is simply because iget_locked performs better than iget5_locked.
Thanks,
Gao Xiang
Re: [PATCH v6 05/24] erofs: add inode operations
Quoted 14 lines · Gao Xiang
On Thu, Aug 29, 2019 at 03:24:26AM -0700, Christoph Hellwig wrote:
[]
Quoted 9 lines
fs/erofs/inode.c line 192
Quoted 4 lines
fs/erofs/inode.c new file +3
-0+ +/* fill last page if inline data is available */ + err =fill_inline_data (inode, data, ofs);Well, I think you should move the
is_inode_flat_inlineand (S_ISLNK(inode->i_mode)&&inode->i_size<PAGE_SIZE) checks from that helper here, as otherwise you make everyone wonder why you'd always fill out the inline data.
Quoted 3 lines
Currently,
fill_inline_data()only fills for fast symlink, later we can fill any tail-end block (such as dir block) for our requirements.
So change it when that later changes actually come in. And even then having the checks outside the function is a lot more obvious.
Quoted 1 line
And I think that is minor.
The problem is that each of these issues might appear minor on their own. But combined a lot of the coding style choices lead to code that is more suitable an obsfucated code contest than the Linux kernel as trying to understand even just a few places requires jumping through tons of helpers with misleading names and spread over various files.
Quoted 2 lines
The consideration is simply because
iget_lockedperforms better thaniget5_locked.
In what benchmark do the differences show up?
Re: [PATCH v6 05/24] erofs: add inode operations
Hi Christoph,
Quoted 15 lines · Christoph Hellwig
Quoted 14 lines · Gao Xiang
On Thu, Aug 29, 2019 at 03:24:26AM -0700, Christoph Hellwig wrote:
[]
Quoted 9 lines
fs/erofs/inode.c line 192
Quoted 4 lines
fs/erofs/inode.c new file +3
-0+ +/* fill last page if inline data is available */ + err =fill_inline_data (inode, data, ofs);Well, I think you should move the
is_inode_flat_inlineand (S_ISLNK(inode->i_mode)&&inode->i_size<PAGE_SIZE) checks from that helper here, as otherwise you make everyone wonder why you'd always fill out the inline data.
Quoted 6 lines
Quoted 3 lines
Currently,
fill_inline_data()only fills for fast symlink, later we can fill any tail-end block (such as dir block) for our requirements.So change it when that later changes actually come in. And even then having the checks outside the function is a lot more obvious.
Okay.
Quoted 9 lines
Quoted 2 lines
And I think that is minor.
The problem is that each of these issues might appear minor on their own. But combined a lot of the coding style choices lead to code that is more suitable an obsfucated code contest than the Linux kernel as trying to understand even just a few places requires jumping through tons of helpers with misleading names and spread over various files.
Quoted 4 lines
Quoted 2 lines
The consideration is simply because
iget_lockedperforms better thaniget5_locked.In what benchmark do the differences show up?
In a word, no benchmark here, just because "unsigned long on 32-bit platforms is 4 bytes." but erofs nid is a 64-bit number.
iget_locked will do find_inode_fast (no callback at all) rather than iget5_locked --> find_inode (test callback) ->
inode_insert5(set callback) for each new inode.
For most 64-bit platforms, iget_locked is enough, 32-bit platforms become rare...
Thanks,
Gao Xiang
Re: [PATCH v6 05/24] erofs: add inode operations
In reply to Christoph HellwigHi Christoph,
Here are redo-ed comments of your suggestions...
Quoted 32 lines · Christoph Hellwig
Quoted 31 lines · Gao Xiang
This adds core functions to get, read an inode.
It adds statx support as well.Signed-off-by Gao Xiang <[email protected]>--- fs/erofs/inode.c | 291 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 291 insertions(+) create mode 100644 fs/erofs/inode.cfs/erofs/inode.c new file +16
-0@@ -0,0 +1,291 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/inode.c +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <[email protected]> +*/ +#include "internal.h" + +#include <trace/events/erofs.h> + +/* no locking */ +static int read_inode (struct inode *inode,void *data) +{ +struct erofs_vnode *vi =EROFS_V (inode);
Quoted 8 lines
fs/erofs/inode.c line 21
Quoted 5 lines
fs/erofs/inode.c new file +5
-0+struct erofs_inode_v1 *v1 = data; +const unsigned int advise =le16_to_cpu (v1->i_advise); +erofs_blk_t nblks =0 ; + + vi->datamode =__inode_data_mapping (advise);What is the deal with these magic underscores here and various other similar helpers?
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
underscores means 'internal' in my thought, it seems somewhat some common practice of Linux kernel, or some recent discussions about it?... I didn't notice these discussions...
Quoted 5 lines
fs/erofs/inode.c line 111
Quoted 2 lines
fs/erofs/inode.c new file +1
-0+/* fast symlink (following ext4) */ This actually originates in FFS. But it is so common that the comment seems a little pointless.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 6 lines
fs/erofs/inode.c line 113
Quoted 3 lines
fs/erofs/inode.c new file +2
-0+if (S_ISLNK (inode->i_mode) && inode->i_size <PAGE_SIZE ) { +char *lnk =erofs_kmalloc (sbi, inode->i_size +1 ,GFP_KERNEL );Please just use plain kmalloc everywhere and let the normal kernel error injection code take care of injeting any errors.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 4 lines
fs/erofs/inode.c line 120
Quoted 2 lines
fs/erofs/inode.c new file +1
-0+/* inline symlink data shouldn't across page boundary as well */ ... should not cross ..
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 10 lines
fs/erofs/inode.c line 127
Quoted 8 lines
fs/erofs/inode.c new file +7
-0+if (unlikely (m_pofs + inode->i_size >PAGE_SIZE )) { +DBG_BUGON (1 ); +kfree (lnk); +return -EIO ; + } + +/* get in-page inline data */ s/get/copy/, but the comment seems rather pointless.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 10 lines
fs/erofs/inode.c line 132
Quoted 6 lines
fs/erofs/inode.c new file +5
-0+memcpy (lnk, data + m_pofs, inode->i_size); + lnk[inode->i_size] ='\0' ; + + inode->i_link = lnk; +set_inode_fast_symlink (inode);Please just set the ops directly instead of obsfucating that in a single caller, single line inline function. And please set it instead of the normal symlink iops in the same place where you also set those.:w
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 8 lines
fs/erofs/inode.c line 167
Quoted 3 lines
fs/erofs/inode.c new file +2
-0+ err =read_inode (inode, data + ofs); +if (!err) {if (err) goto out_unlock;.. and save one level of indentation.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 5 lines
fs/erofs/inode.c line 184
Quoted 2 lines
fs/erofs/inode.c new file +1
-0+if (is_inode_layout_compression (inode)) {The name of this helper is a little odd. But I think just opencoding it seems generally cleaner anyway.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Quoted 15 lines
fs/erofs/inode.c line 192
Quoted 10 lines
fs/erofs/inode.c new file +8
-0+ err = -ENOTSUPP ; + goto out_unlock; + } + + inode->i_mapping->a_ops = &erofs_raw_access_aops; + +/* fill last page if inline data is available */ + err =fill_inline_data (inode, data, ofs);Well, I think you should move the
is_inode_flat_inlineand (S_ISLNK(inode->i_mode)&&inode->i_size<PAGE_SIZE) checks from that helper here, as otherwise you make everyone wonder why you'd always fill out the inline data.
fill_inline_data is killed, and the similar function turns into erofs_fill_symlink which is called at erofs_fill_inode():
case S_IFLNK:
- /* by default, page_get_link is used for symlink */
- inode->i_op = &erofs_symlink_iops;
+ err = erofs_fill_symlink(inode, data, ofs);
+ if (err)
+ goto out_unlock;
inode_nohighmem(inode);
break;
Quoted 17 lines
fs/erofs/inode.c line 233
Quoted 13 lines
fs/erofs/inode.c new file +12
-0+static inline struct inode *erofs_iget_locked (struct super_block *sb, +erofs_nid_t nid) +{ +const unsigned long hashval =erofs_inode_hash (nid); + +#if BITS_PER_LONG >=64 +/* it is safe to use iget_locked for >= 64-bit platform */ +return iget_locked (sb, hashval); +#else +return iget5_locked (sb, hashval, erofs_ilookup_test_actor, + erofs_iget_set_actor, &nid); +#endif Just use the slightly more complicated 32-bit version everywhere so that you have a single actually tested code path. And then remove this helper.
As I said before, 64-bit platforms is common currently, I think iget_locked is enough.
https://lore.kernel.org/r/20190830184606.GA175612@architecture4/
Thanks,
Gao Xiang
Re: [PATCH v6 05/24] erofs: add inode operations
Quoted 11 lines · Gao Xiang
Quoted 7 lines
fs/erofs/inode.c line 233
Quoted 3 lines
fs/erofs/inode.c new file +3
-0+return iget5_locked (sb, hashval, erofs_ilookup_test_actor, + erofs_iget_set_actor, &nid); +#endif Just use the slightly more complicated 32-bit version everywhere so that you have a single actually tested code path. And then remove this helper.
As I said before, 64-bit platforms is common currently, I think
iget_lockedis enough.
https://lore.kernel.org/r/20190830184606.GA175612@architecture4/
The problem with that is that you now have two entirely different code paths. And the 32-bit one will probably get very little testing and eventually bitrot. We defintively had problems of that sort in XFS in the past, so my suggestion is to not go down the root of separate code for 32-bit vs 64-bit unless it makes a real difference for a real-life workload.
Re: [PATCH v6 05/24] erofs: add inode operations
In reply to Gao XiangQuoted 12 lines · Gao Xiang
Quoted 11 lines
fs/erofs/inode.c line 21
Quoted 8 lines
fs/erofs/inode.c new file +8
-0+static int read_inode (struct inode *inode,void *data) +{ +struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode_v1 *v1 = data; +const unsigned int advise =le16_to_cpu (v1->i_advise); +erofs_blk_t nblks =0 ; + + vi->datamode =__inode_data_mapping (advise);What is the deal with these magic underscores here and various other similar helpers?
Quoted 6 lines
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/underscores means 'internal' in my thought, it seems somewhat some common practice of Linux kernel, or some recent discussions about it?... I didn't notice these discussions...
I know about a few valid uses of the underscores:
* pattern where the __underscored version does not do locking, while the other does
* similarly for atomic and non-atomic version
* macro that needs to manipulate the argument name (like glue some prefix, so the macro does not have underscores and is supposed to be used instead of the function with underscores that needs the full name of a variable/constant/..
* underscore function takes a few more parameters to further tune the behaviour, but most users are fine with the defaults and that is provided as a function without underscores
* in case you have just one function of the kind, don't use the underscores
I can lookup examples if you're interested or if the brief description is not sufficient. The list covers what I've seen and used, but the list may be incomplete.
Re: [PATCH v6 05/24] erofs: add inode operations
Hi David,
Quoted 34 lines · David Sterba
Quoted 12 lines · Gao Xiang
Quoted 11 lines
fs/erofs/inode.c line 21
Quoted 8 lines
fs/erofs/inode.c new file +8
-0+static int read_inode (struct inode *inode,void *data) +{ +struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode_v1 *v1 = data; +const unsigned int advise =le16_to_cpu (v1->i_advise); +erofs_blk_t nblks =0 ; + + vi->datamode =__inode_data_mapping (advise);What is the deal with these magic underscores here and various other similar helpers?
Quoted 6 lines
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/underscores means 'internal' in my thought, it seems somewhat some common practice of Linux kernel, or some recent discussions about it?... I didn't notice these discussions...
I know about a few valid uses of the underscores:
* pattern where the
__underscoredversion does not do locking, while the other does
* similarly for atomic and non-atomic version
* macro that needs to manipulate the argument name (like glue some prefix, so the macro does not have underscores and is supposed to be used instead of the function with underscores that needs the full name of a variable/constant/..
* underscore function takes a few more parameters to further tune the behaviour, but most users are fine with the defaults and that is provided as a function without underscores
* in case you have just one function of the kind, don't use the underscores
Quoted 3 lines
I can lookup examples if you're interested or if the brief description is not sufficient. The list covers what I've seen and used, but the list may be incomplete.
Thanks, I learn a lot from the above. [thumb]
Thanks,
Gao Xiang
PATCH v6 6/24 erofs: support special inode
In reply to Gao XiangThis patch adds to support special inode, such as
block dev, char, socket, pipe inode.
--- fs/erofs/inode.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)
fs/erofs/inode.c +25 -2
@@ -34,7 +34,16 @@ static int read_inode(struct inode *inode, void *data) vi->xattr_isize =ondisk_xattr_ibody_size (v2->i_xattr_icount); inode->i_mode =le16_to_cpu (v2->i_mode); - vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr); +if (S_ISREG (inode->i_mode) ||S_ISDIR (inode->i_mode) || +S_ISLNK (inode->i_mode)) + vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr); +else if (S_ISCHR (inode->i_mode) ||S_ISBLK (inode->i_mode)) + inode->i_rdev = +new_decode_dev (le32_to_cpu (v2->i_u.rdev)); +else if (S_ISFIFO (inode->i_mode) ||S_ISSOCK (inode->i_mode)) + inode->i_rdev =0 ; +else +return -EIO ;i_uid_write (inode,le32_to_cpu (v2->i_uid));i_gid_write (inode,le32_to_cpu (v2->i_gid));
@@ -58,7 +67,16 @@ static int read_inode(struct inode *inode, void *data) vi->xattr_isize =ondisk_xattr_ibody_size (v1->i_xattr_icount); inode->i_mode =le16_to_cpu (v1->i_mode); - vi->raw_blkaddr =le32_to_cpu (v1->i_u.raw_blkaddr); +if (S_ISREG (inode->i_mode) ||S_ISDIR (inode->i_mode) || +S_ISLNK (inode->i_mode)) + vi->raw_blkaddr =le32_to_cpu (v1->i_u.raw_blkaddr); +else if (S_ISCHR (inode->i_mode) ||S_ISBLK (inode->i_mode)) + inode->i_rdev = +new_decode_dev (le32_to_cpu (v1->i_u.rdev)); +else if (S_ISFIFO (inode->i_mode) ||S_ISSOCK (inode->i_mode)) + inode->i_rdev =0 ; +else +return -EIO ;i_uid_write (inode,le16_to_cpu (v1->i_uid));i_gid_write (inode,le16_to_cpu (v1->i_gid));
@@ -176,6 +194,11 @@ static int fill_inode(struct inode *inode, int isdir)/* by default, page_get_link is used for symlink */ inode->i_op = &erofs_symlink_iops;inode_nohighmem (inode); + }else if (S_ISCHR (inode->i_mode) ||S_ISBLK (inode->i_mode) || +S_ISFIFO (inode->i_mode) ||S_ISSOCK (inode->i_mode)) { + inode->i_op = &erofs_generic_iops; +init_special_inode (inode, inode->i_mode, inode->i_rdev); + goto out_unlock; }else { err = -EIO ; goto out_unlock;
Signature · 1 line
2.17.1
Re: [PATCH v6 06/24] erofs: support special inode
Quoted 17 lines · Gao Xiang
This patch adds to support special inode, such as
block dev, char, socket, pipe inode.Signed-off-by Gao Xiang <[email protected]>--- fs/erofs/inode.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)fs/erofs/inode.c +0
-1@@ -34,7 +34,16 @@ static int read_inode(struct inode *inode, void *data) vi->xattr_isize =ondisk_xattr_ibody_size (v2->i_xattr_icount); inode->i_mode =le16_to_cpu (v2->i_mode); - vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr);
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 46
Quoted 10 lines
fs/erofs/inode.c +10
-0+if (S_ISREG (inode->i_mode) ||S_ISDIR (inode->i_mode) || +S_ISLNK (inode->i_mode)) + vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr); +else if (S_ISCHR (inode->i_mode) ||S_ISBLK (inode->i_mode)) + inode->i_rdev = +new_decode_dev (le32_to_cpu (v2->i_u.rdev)); +else if (S_ISFIFO (inode->i_mode) ||S_ISSOCK (inode->i_mode)) + inode->i_rdev =0 ; +else +return -EIO ;
Please use a switch statement when dealing with the file modes to make everything easier to read.
Re: [PATCH v6 06/24] erofs: support special inode
Hi Christoph,
Quoted 16 lines · Christoph Hellwig
Quoted 15 lines · Gao Xiang
This patch adds to support special inode, such as
block dev, char, socket, pipe inode.Signed-off-by Gao Xiang <[email protected]>--- fs/erofs/inode.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)fs/erofs/inode.c +0
-0@@ -34,7 +34,16 @@ static int read_inode(struct inode *inode, void *data) vi->xattr_isize =ondisk_xattr_ibody_size (v2->i_xattr_icount);
Quoted 15 lines
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 46
Quoted 12 lines
fs/erofs/inode.c +10
-1inode->i_mode =le16_to_cpu (v2->i_mode); - vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr); +if (S_ISREG (inode->i_mode) ||S_ISDIR (inode->i_mode) || +S_ISLNK (inode->i_mode)) + vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr); +else if (S_ISCHR (inode->i_mode) ||S_ISBLK (inode->i_mode)) + inode->i_rdev = +new_decode_dev (le32_to_cpu (v2->i_u.rdev)); +else if (S_ISFIFO (inode->i_mode) ||S_ISSOCK (inode->i_mode)) + inode->i_rdev =0 ; +else +return -EIO ;Please use a switch statement when dealing with the file modes to make everything easier to read.
Fixed in
https://lore.kernel.org/linux-fsdevel/[email protected]/
Thanks,
Gao Xiang
PATCH v6 7/24 erofs: add directory operations
In reply to Gao XiangThis adds functions for directory, mainly readdir.
--- fs/erofs/dir.c | 147 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 fs/erofs/dir.c
fs/erofs/dir.c new file +147 -0
@@ -0,0 +1,147 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/dir.c +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#include "internal.h" + +static const unsigned char erofs_filetype_table[EROFS_FT_MAX ] = { + [EROFS_FT_UNKNOWN ] =DT_UNKNOWN , + [EROFS_FT_REG_FILE ] =DT_REG , + [EROFS_FT_DIR ] =DT_DIR , + [EROFS_FT_CHRDEV ] =DT_CHR , + [EROFS_FT_BLKDEV ] =DT_BLK , + [EROFS_FT_FIFO ] =DT_FIFO , + [EROFS_FT_SOCK ] =DT_SOCK , + [EROFS_FT_SYMLINK ] =DT_LNK , +}; + +static void debug_one_dentry (unsigned char d_type,const char *de_name, +unsigned int de_namelen) +{ +#ifdef CONFIG_EROFS_FS_DEBUG +/* since the on-disk name could not have the trailing '\0' */ +unsigned char dbg_namebuf[EROFS_NAME_LEN +1 ]; + +memcpy (dbg_namebuf, de_name, de_namelen); + dbg_namebuf[de_namelen] ='\0' ; + +debugln ("found dirent %s de_len %u d_type %d" , dbg_namebuf, + de_namelen, d_type); +#endif +} + +static int erofs_fill_dentries (struct dir_context *ctx, +void *dentry_blk,unsigned int *ofs, +unsigned int nameoff,unsigned int maxsize) +{ +struct erofs_dirent *de = dentry_blk + *ofs; +const struct erofs_dirent *end = dentry_blk + nameoff; + +while (de < end) { +const char *de_name; +unsigned int de_namelen; +unsigned char d_type; + +if (de->file_type <EROFS_FT_MAX ) + d_type = erofs_filetype_table[de->file_type]; +else + d_type =DT_UNKNOWN ; + + nameoff =le16_to_cpu (de->nameoff); + de_name = (char *)dentry_blk + nameoff; + +/* the last dirent in the block? */ +if (de +1 >= end) + de_namelen =strnlen (de_name, maxsize - nameoff); +else + de_namelen =le16_to_cpu (de[1 ].nameoff) - nameoff; + +/* a corrupted entry is found */ +if (unlikely (nameoff + de_namelen > maxsize || + de_namelen >EROFS_NAME_LEN )) { +DBG_BUGON (1 ); +return -EIO ; + } + +debug_one_dentry (d_type, de_name, de_namelen); +if (!dir_emit (ctx, de_name, de_namelen, +le64_to_cpu (de->nid), d_type)) +/* stopped by some reason */ +return 1 ; + ++de; + *ofs +=sizeof (struct erofs_dirent ); + } + *ofs = maxsize; +return 0 ; +} + +static int erofs_readdir (struct file *f,struct dir_context *ctx) +{ +struct inode *dir =file_inode (f); +struct address_space *mapping = dir->i_mapping; +const size_t dirsize =i_size_read (dir); +unsigned int i = ctx->pos /EROFS_BLKSIZ ; +unsigned int ofs = ctx->pos %EROFS_BLKSIZ ; +int err =0 ; +bool initial = true; + +while (ctx->pos < dirsize) { +struct page *dentry_page; +struct erofs_dirent *de; +unsigned int nameoff, maxsize; + + dentry_page =read_mapping_page (mapping, i,NULL ); +if (IS_ERR (dentry_page)) +continue ; + + de = (struct erofs_dirent *)kmap (dentry_page); + + nameoff =le16_to_cpu (de->nameoff); + +if (unlikely (nameoff <sizeof (struct erofs_dirent ) || + nameoff >=PAGE_SIZE )) { +errln ("%s, invalid de[0].nameoff %u" , + __func__, nameoff); + + err = -EIO ; + goto skip_this; + } + + maxsize = min_t(unsigned int, + dirsize - ctx->pos + ofs,PAGE_SIZE ); + +/* search dirents at the arbitrary position */ +if (unlikely (initial)) { + initial = false; + + ofs =roundup (ofs,sizeof (struct erofs_dirent )); +if (unlikely (ofs >= nameoff)) + goto skip_this; + } + + err =erofs_fill_dentries (ctx, de, &ofs, nameoff, maxsize); +skip_this: +kunmap (dentry_page); + +put_page (dentry_page); + + ctx->pos =blknr_to_addr (i) + ofs; + +if (unlikely (err)) +break ; + ++i; + ofs =0 ; + } +return err <0 ? err :0 ; +} + +const struct file_operations erofs_dir_fops = { + .llseek = generic_file_llseek, + .read = generic_read_dir, + .iterate_shared = erofs_readdir, +}; +
Signature · 1 line
2.17.1
PATCH v6 8/24 erofs: add namei functions
In reply to Gao XiangThis commit adds functions that transfer names to inodes.
--- fs/erofs/namei.c | 247 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 247 insertions(+) create mode 100644 fs/erofs/namei.c
fs/erofs/namei.c new file +247 -0
@@ -0,0 +1,247 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/namei.c +* +* Copyright (C) 2017-2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#include "internal.h" + +#include <trace/events/erofs.h> + +struct erofs_qstr { +const unsigned char *name; +const unsigned char *end; +}; + +/* based on the end of qn is accurate and it must have the trailing '\0' */ +static inline int dirnamecmp (const struct erofs_qstr *qn, +const struct erofs_qstr *qd, +unsigned int *matched) +{ +unsigned int i = *matched; + +/* +* on-disk error, let's only BUG_ON in the debugging mode. +* otherwise, it will return 1 to just skip the invalid name +* and go on (in consideration of the lookup performance). +*/ +DBG_BUGON (qd->name > qd->end); + +/* qd could not have trailing '\0' */ +/* However it is absolutely safe if < qd->end */ +while (qd->name + i < qd->end && qd->name[i] !='\0' ) { +if (qn->name[i] != qd->name[i]) { + *matched = i; +return qn->name[i] > qd->name[i] ?1 :-1 ; + } + ++i; + } + *matched = i; +/* See comments in __d_alloc on the terminating NUL character */ +return qn->name[i] =='\0' ?0 :1 ; +} + +#define nameoff_from_disk (off, sz) (le16_to_cpu(off) & ((sz) - 1)) + +static struct erofs_dirent *find_target_dirent (struct erofs_qstr *name, +u8 *data, +unsigned int dirblksize, +const int ndirents) +{ +int head, back; +unsigned int startprfx, endprfx; +struct erofs_dirent *const de = (struct erofs_dirent *)data; + +/* since the 1st dirent has been evaluated previously */ + head =1 ; + back = ndirents -1 ; + startprfx = endprfx =0 ; + +while (head <= back) { +const int mid = head + (back - head) /2 ; +const int nameoff =nameoff_from_disk (de[mid].nameoff, + dirblksize); +unsigned int matched =min (startprfx, endprfx); +struct erofs_qstr dname = { + .name = data + nameoff, + .end =unlikely (mid >= ndirents -1 ) ? + data + dirblksize : + data +nameoff_from_disk (de[mid +1 ].nameoff, + dirblksize) + }; + +/* string comparison without already matched prefix */ +int ret =dirnamecmp (name, &dname, &matched); + +if (unlikely (!ret)) { +return de + mid; + }else if (ret >0 ) { + head = mid +1 ; + startprfx = matched; + }else { + back = mid -1 ; + endprfx = matched; + } + } + +return ERR_PTR (-ENOENT ); +} + +static struct page *find_target_block_classic (struct inode *dir, +struct erofs_qstr *name, +int *_ndirents) +{ +unsigned int startprfx, endprfx; +int head, back; +struct address_space *const mapping = dir->i_mapping; +struct page *candidate =ERR_PTR (-ENOENT ); + + startprfx = endprfx =0 ; + head =0 ; + back =inode_datablocks (dir) -1 ; + +while (head <= back) { +const int mid = head + (back - head) /2 ; +struct page *page =read_mapping_page (mapping, mid,NULL ); + +if (!IS_ERR (page)) { +struct erofs_dirent *de =kmap_atomic (page); +const int nameoff =nameoff_from_disk (de->nameoff, +EROFS_BLKSIZ ); +const int ndirents = nameoff /sizeof (*de); +int diff; +unsigned int matched; +struct erofs_qstr dname; + +if (unlikely (!ndirents)) { +DBG_BUGON (1 ); +kunmap_atomic (de); +put_page (page); + page =ERR_PTR (-EIO ); + goto out; + } + + matched =min (startprfx, endprfx); + + dname.name = (u8 *)de + nameoff; +if (ndirents ==1 ) + dname.end = (u8 *)de +EROFS_BLKSIZ ; +else + dname.end = (u8 *)de + +nameoff_from_disk (de[1 ].nameoff, +EROFS_BLKSIZ ); + +/* string comparison without already matched prefix */ + diff =dirnamecmp (name, &dname, &matched); +kunmap_atomic (de); + +if (unlikely (!diff)) { + *_ndirents =0 ; + goto out; + }else if (diff >0 ) { + head = mid +1 ; + startprfx = matched; + +if (!IS_ERR (candidate)) +put_page (candidate); + candidate = page; + *_ndirents = ndirents; + }else { +put_page (page); + + back = mid -1 ; + endprfx = matched; + } +continue ; + } +out:/* free if the candidate is valid */ +if (!IS_ERR (candidate)) +put_page (candidate); +return page; + } +return candidate; +} + +int erofs_namei (struct inode *dir, +struct qstr *name, +erofs_nid_t *nid,unsigned int *d_type) +{ +int ndirents; +struct page *page; +void *data; +struct erofs_dirent *de; +struct erofs_qstr qn; + +if (unlikely (!dir->i_size)) +return -ENOENT ; + + qn.name = name->name; + qn.end = name->name + name->len; + + ndirents =0 ; + page =find_target_block_classic (dir, &qn, &ndirents); + +if (IS_ERR (page)) +return PTR_ERR (page); + + data =kmap_atomic (page); +/* the target page has been mapped */ +if (ndirents) + de =find_target_dirent (&qn, data,EROFS_BLKSIZ , ndirents); +else + de = (struct erofs_dirent *)data; + +if (!IS_ERR (de)) { + *nid =le64_to_cpu (de->nid); + *d_type = de->file_type; + } + +kunmap_atomic (data); +put_page (page); + +return PTR_ERR_OR_ZERO (de); +} + +/* NOTE: i_mutex is already held by vfs */ +static struct dentry *erofs_lookup (struct inode *dir, +struct dentry *dentry, +unsigned int flags) +{ +int err; +erofs_nid_t nid; +unsigned int d_type; +struct inode *inode; + +DBG_BUGON (!d_really_is_negative (dentry)); +/* dentry must be unhashed in lookup, no need to worry about */ +DBG_BUGON (!d_unhashed (dentry)); + +trace_erofs_lookup (dir, dentry, flags); + +/* file name exceeds fs limit */ +if (unlikely (dentry->d_name.len >EROFS_NAME_LEN )) +return ERR_PTR (-ENAMETOOLONG ); + +/* false uninitialized warnings on gcc 4.8.x */ + err =erofs_namei (dir, &dentry->d_name, &nid, &d_type); + +if (err == -ENOENT ) { +/* negative dentry */ + inode =NULL ; + }else if (unlikely (err)) { + inode =ERR_PTR (err); + }else { +debugln ("%s, %s (nid %llu) found, d_type %u" , __func__, + dentry->d_name.name, nid, d_type); + inode =erofs_iget (dir->i_sb, nid, d_type ==EROFS_FT_DIR ); + } +return d_splice_alias (inode, dentry); +} + +const struct inode_operations erofs_dir_iops = { + .lookup = erofs_lookup, + .getattr = erofs_getattr, +}; +
Signature · 1 line
2.17.1
Re: [PATCH v6 08/24] erofs: add namei functions
fs/erofs/namei.c line 16
Quoted 4 lines · Gao Xiang
fs/erofs/namei.c new file +4
-0+struct erofs_qstr { +const unsigned char *name; +const unsigned char *end; +};
Maybe erofs_name? The q in qstr stands for quick, because of the existing hash and len, which this doesn't really provide.
Also I don't really see why you don't just pass the actual qstr and just document that dirnamecmp does not look at the hash and thus doesn't require it to be filled out.
Re: [PATCH v6 08/24] erofs: add namei functions
Quoted 9 lines · Christoph Hellwig
fs/erofs/namei.c line 16
Quoted 4 lines · Gao Xiang
fs/erofs/namei.c new file +4
-0+struct erofs_qstr { +const unsigned char *name; +const unsigned char *end; +};Maybe
erofs_name? The q in qstr stands for quick, because of the existing hash and len, which this doesn't really provide.
Quoted 3 lines
Also I don't really see why you don't just pass the actual qstr and just document that dirnamecmp does not look at the hash and thus doesn't require it to be filled out.
q in erofs_qstr also means quick substring.
If you have some time to look into it more, it uses a prefixed binary search algorithm (rather than linear traversal), which provides similar proformance with hashed approach but no need to save such hash field and it's natively sorted in alphabet order.
Thanks,
Gao Xiang
Quoted 1 line
PATCH v6 9/24 erofs: support tracepoint
In reply to Gao XiangAdd basic tracepoints for ->readpage{,s}, ->lookup, ->destroy_inode, fill_inode and map_blocks.
--- include/trace/events/erofs.h | 241 +++++++++++++++++++++++++++++++++++ 1 file changed, 241 insertions(+) create mode 100644 include/trace/events/erofs.h
include/trace/events/erofs.h new file +241 -0
@@ -0,0 +1,241 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM erofs + +#if !defined(_TRACE_EROFS_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _TRACE_EROFS_H + +#include <linux/tracepoint.h> + +#define show_dev (dev) MAJOR(dev), MINOR(dev) +#define show_dev_nid (entry) show_dev(entry->dev), entry->nid + +#define show_file_type (type) \ + __print_symbolic(type, \ + { 0, "FILE" }, \ + { 1, "DIR" }) + +#define show_map_flags (flags) __print_flags(flags, "|", \ + { EROFS_GET_BLOCKS_RAW, "RAW" }) + +#define show_mflags (flags) __print_flags(flags, "", \ + { EROFS_MAP_MAPPED, "M" }, \ + { EROFS_MAP_META, "I" }) + +TRACE_EVENT (erofs_lookup, + + TP_PROTO(struct inode *dir, struct dentry *dentry, unsigned int flags), + +TP_ARGS (dir, dentry, flags), + +TP_STRUCT__entry ( + __field(dev_t, dev ) + __field(erofs_nid_t, nid ) + __field(const char *, name ) + __field(unsigned int, flags ) + ), + +TP_fast_assign ( + __entry->dev = dir->i_sb->s_dev; + __entry->nid = EROFS_V(dir)->nid; + __entry->name = dentry->d_name.name; + __entry->flags = flags; + ), + +TP_printk ("dev = (%d,%d), pnid = %llu, name:%s, flags:%x" , +show_dev_nid (__entry), + __entry->name, + __entry->flags) +); + +TRACE_EVENT (erofs_fill_inode, + TP_PROTO(struct inode *inode, int isdir), +TP_ARGS (inode, isdir), + +TP_STRUCT__entry ( + __field(dev_t, dev ) + __field(erofs_nid_t, nid ) + __field(erofs_blk_t, blkaddr ) + __field(unsigned int, ofs ) +__field (int, isdir ) + ), + +TP_fast_assign ( + __entry->dev = inode->i_sb->s_dev; + __entry->nid = EROFS_V(inode)->nid; + __entry->blkaddr = erofs_blknr(iloc(EROFS_I_SB(inode), __entry->nid)); + __entry->ofs = erofs_blkoff(iloc(EROFS_I_SB(inode), __entry->nid)); + __entry->isdir = isdir; + ), + +TP_printk ("dev = (%d,%d), nid = %llu, blkaddr %u ofs %u, isdir %d" , +show_dev_nid (__entry), + __entry->blkaddr, __entry->ofs, + __entry->isdir) +); + +TRACE_EVENT (erofs_readpage, + + TP_PROTO(struct page *page, bool raw), + +TP_ARGS (page, raw), + +TP_STRUCT__entry ( + __field(dev_t, dev ) + __field(erofs_nid_t, nid ) + __field(int, dir ) + __field(pgoff_t, index ) + __field(int, uptodate) +__field (bool, raw ) + ), + +TP_fast_assign ( + __entry->dev = page->mapping->host->i_sb->s_dev; + __entry->nid = EROFS_V(page->mapping->host)->nid; + __entry->dir = S_ISDIR(page->mapping->host->i_mode); + __entry->index = page->index; + __entry->uptodate = PageUptodate(page); + __entry->raw = raw; + ), + +TP_printk ("dev = (%d,%d), nid = %llu, %s, index = %lu, uptodate = %d " +"raw = %d" , +show_dev_nid (__entry), +show_file_type (__entry->dir), + (unsigned long )__entry->index, + __entry->uptodate, + __entry->raw) +); + +TRACE_EVENT (erofs_readpages, + + TP_PROTO(struct inode *inode, struct page *page, unsigned int nrpage, + bool raw), + +TP_ARGS (inode, page, nrpage, raw), + +TP_STRUCT__entry ( + __field(dev_t, dev ) + __field(erofs_nid_t, nid ) + __field(pgoff_t, start ) + __field(unsigned int, nrpage ) +__field (bool, raw ) + ), + +TP_fast_assign ( + __entry->dev = inode->i_sb->s_dev; + __entry->nid = EROFS_V(inode)->nid; + __entry->start = page->index; + __entry->nrpage = nrpage; + __entry->raw = raw; + ), + +TP_printk ("dev = (%d,%d), nid = %llu, start = %lu nrpage = %u raw = %d" , +show_dev_nid (__entry), + (unsigned long )__entry->start, + __entry->nrpage, + __entry->raw) +); + +DECLARE_EVENT_CLASS (erofs__map_blocks_enter, + TP_PROTO(struct inode *inode, struct erofs_map_blocks *map, + unsigned int flags), + +TP_ARGS (inode, map, flags), + +TP_STRUCT__entry ( + __field( dev_t, dev ) + __field( erofs_nid_t, nid ) + __field( erofs_off_t, la ) + __field( u64, llen ) + __field( unsigned int, flags ) + ), + +TP_fast_assign ( + __entry->dev = inode->i_sb->s_dev; + __entry->nid = EROFS_V(inode)->nid; + __entry->la = map->m_la; + __entry->llen = map->m_llen; + __entry->flags = flags; + ), + +TP_printk ("dev = (%d,%d), nid = %llu, la %llu llen %llu flags %s" , +show_dev_nid (__entry), + __entry->la, __entry->llen, + __entry->flags ?show_map_flags (__entry->flags) :"NULL" ) +); + +DEFINE_EVENT (erofs__map_blocks_enter, erofs_map_blocks_flatmode_enter, + TP_PROTO(struct inode *inode, struct erofs_map_blocks *map, + unsigned flags), + +TP_ARGS (inode, map, flags) +); + +DECLARE_EVENT_CLASS (erofs__map_blocks_exit, + TP_PROTO(struct inode *inode, struct erofs_map_blocks *map, + unsigned int flags, int ret), + +TP_ARGS (inode, map, flags, ret), + +TP_STRUCT__entry ( + __field( dev_t, dev ) + __field( erofs_nid_t, nid ) + __field( unsigned int, flags ) + __field( erofs_off_t, la ) + __field( erofs_off_t, pa ) + __field( u64, llen ) + __field( u64, plen ) + __field( unsigned int, mflags ) +__field ( int, ret ) + ), + +TP_fast_assign ( + __entry->dev = inode->i_sb->s_dev; + __entry->nid = EROFS_V(inode)->nid; + __entry->flags = flags; + __entry->la = map->m_la; + __entry->pa = map->m_pa; + __entry->llen = map->m_llen; + __entry->plen = map->m_plen; + __entry->mflags = map->m_flags; + __entry->ret = ret; + ), + +TP_printk ("dev = (%d,%d), nid = %llu, flags %s " +"la %llu pa %llu llen %llu plen %llu mflags %s ret %d" , +show_dev_nid (__entry), + __entry->flags ?show_map_flags (__entry->flags) :"NULL" , + __entry->la, __entry->pa, __entry->llen, __entry->plen, +show_mflags (__entry->mflags), __entry->ret) +); + +DEFINE_EVENT (erofs__map_blocks_exit, erofs_map_blocks_flatmode_exit, + TP_PROTO(struct inode *inode, struct erofs_map_blocks *map, + unsigned flags, int ret), + +TP_ARGS (inode, map, flags, ret) +); + +TRACE_EVENT (erofs_destroy_inode, + TP_PROTO(struct inode *inode), + +TP_ARGS (inode), + +TP_STRUCT__entry ( + __field( dev_t, dev ) +__field ( erofs_nid_t, nid ) + ), + +TP_fast_assign ( + __entry->dev = inode->i_sb->s_dev; + __entry->nid = EROFS_V(inode)->nid; + ), + +TP_printk ("dev = (%d,%d), nid = %llu" ,show_dev_nid (__entry)) +); + +#endif /* _TRACE_EROFS_H */ + +/* This part must be outside protection */ +#include <trace/define_trace.h>
Signature · 1 line
2.17.1
PATCH v6 10/24 erofs: update Kconfig and Makefile
In reply to Gao XiangThis commit adds Makefile and Kconfig for erofs, and updates Makefile and Kconfig files in the fs directory.
--- fs/Kconfig | 1 + fs/Makefile | 1 + fs/erofs/Kconfig | 36 ++++++++++++++++++++++++++++++++++++ fs/erofs/Makefile | 9 +++++++++ 4 files changed, 47 insertions(+) create mode 100644 fs/erofs/Kconfig create mode 100644 fs/erofs/Makefile
fs/Kconfig +1 -0
@@ -261,6 +261,7 @@ source "fs/romfs/Kconfig" source "fs/pstore/Kconfig" source "fs/sysv/Kconfig" source "fs/ufs/Kconfig" +source "fs/erofs/Kconfig" endif # MISC_FILESYSTEMS
fs/Makefile +1 -0
@@ -130,3 +130,4 @@ obj-$(CONFIG_F2FS_FS) += f2fs/ obj-$(CONFIG_CEPH_FS) += ceph/ obj-$(CONFIG_PSTORE) += pstore/ obj-$(CONFIG_EFIVAR_FS) += efivarfs/ +obj-$(CONFIG_EROFS_FS) += erofs/
fs/erofs/Kconfig new file +36 -0
@@ -0,0 +1,36 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config EROFS_FS + tristate "EROFS filesystem support" + depends on BLOCK + help + EROFS (Enhanced Read-Only File System) is a lightweight + read-only file system with modern designs (eg. page-sized + blocks, inline xattrs/data, etc.) for scenarios which need + high-performance read-only requirements, e.g. Android OS + for mobile phones and LIVECDs. + + It also provides fixed-sized output compression support, + which improves storage density, keeps relatively higher + compression ratios, which is more useful to achieve high + performance for embedded devices with limited memory. + + If unsure, say N. + +config EROFS_FS_DEBUG + bool "EROFS debugging feature" + depends on EROFS_FS + help + Print debugging messages and enable more BUG_ONs which check + filesystem consistency and find potential issues aggressively, + which can be used for Android eng build, for example. + + For daily use, say N. + +config EROFS_FAULT_INJECTION + bool "EROFS fault injection facility" + depends on EROFS_FS + help + Test EROFS to inject faults such as ENOMEM, EIO, and so on. + If unsure, say N. +
fs/erofs/Makefile new file +9 -0
@@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0-only + +EROFS_VERSION = "1.0" + +ccflags-y += -DEROFS_VERSION=\"$(EROFS_VERSION)\" + +obj-$(CONFIG_EROFS_FS) += erofs.o +erofs-objs := super.o inode.o data.o namei.o dir.o +
Signature · 1 line
2.17.1
PATCH v6 11/24 erofs: introduce xattr & posixacl support
In reply to Gao XiangThis implements xattr and posixacl functionalities.
1) Inline and shared xattrs are introduced for flexibility.
Specifically, if the same xattr is used for a large number of inodes or the size of a xattr is so large that unsuitable to inline, a shared xattr will be used instead in xattr meta.
2) Add .get_acl() to read the file's acl from its xattrs to make POSIX ACL usable.
Here is the on-disk detail,
fullname: system.posix_acl_access
struct erofs_xattr_entry:
.e_name_len = 0
.e_name_index = EROFS_XATTR_INDEX_POSIX_ACL_ACCESS (2)
fullname: system.posix_acl_default
struct erofs_xattr_entry:
.e_name_len = 0
.e_name_index = EROFS_XATTR_INDEX_POSIX_ACL_DEFAULT (3)
--- fs/erofs/Kconfig | 38 +++ fs/erofs/Makefile | 1 + fs/erofs/inode.c | 14 +- fs/erofs/internal.h | 16 + fs/erofs/namei.c | 6 +- fs/erofs/super.c | 77 ++++- fs/erofs/xattr.c | 700 ++++++++++++++++++++++++++++++++++++++++++++ fs/erofs/xattr.h | 94 ++++++ 8 files changed, 943 insertions(+), 3 deletions(-) create mode 100644 fs/erofs/xattr.c create mode 100644 fs/erofs/xattr.h
fs/erofs/Kconfig +38 -0
@@ -34,3 +34,41 @@ config EROFS_FAULT_INJECTION Test EROFS to inject faults such as ENOMEM, EIO, and so on. If unsure, say N. +config EROFS_FS_XATTR + bool "EROFS extended attributes" + depends on EROFS_FS + default y + help + Extended attributes are name:value pairs associated with inodes by + the kernel or by users (see the attr(5) manual page, or visit + <http://acl.bestbits.at/> for details). + + If unsure, say N. + +config EROFS_FS_POSIX_ACL + bool "EROFS Access Control Lists" + depends on EROFS_FS_XATTR + select FS_POSIX_ACL + default y + help + Posix Access Control Lists (ACLs) support permissions for users and + groups beyond the owner/group/world scheme. + + To learn more about Access Control Lists, visit the POSIX ACLs for + Linux website <http://acl.bestbits.at/>. + + If you don't know what Access Control Lists are, say N. + +config EROFS_FS_SECURITY + bool "EROFS Security Labels" + depends on EROFS_FS_XATTR + default y + help + Security labels provide an access control facility to support Linux + Security Models (LSMs) accepted by AppArmor, SELinux, Smack and TOMOYO + Linux. This option enables an extended attribute handler for file + security labels in the erofs filesystem, so that it requires enabling + the extended attribute support in advance. + + If you are not using a security module, say N. +
fs/erofs/Makefile +1 -0
@@ -6,4 +6,5 @@ ccflags-y += -DEROFS_VERSION=\"$(EROFS_VERSION)\" obj-$(CONFIG_EROFS_FS) += erofs.o erofs-objs := super.o inode.o data.o namei.o dir.o +erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o
fs/erofs/inode.c +13 -1
@@ -6,7 +6,7 @@* http://www.huawei.com/ * Created by Gao Xiang <gaoxiang25 at huawei.com> */ -#include "internal.h" +#include "xattr.h" #include <trace/events/erofs.h>
@@ -300,15 +300,27 @@ int erofs_getattr(const struct path *path, struct kstat *stat,const struct inode_operations erofs_generic_iops = { .getattr = erofs_getattr, +#ifdef CONFIG_EROFS_FS_XATTR + .listxattr = erofs_listxattr, +#endif + .get_acl = erofs_get_acl, };const struct inode_operations erofs_symlink_iops = { .get_link = page_get_link, .getattr = erofs_getattr, +#ifdef CONFIG_EROFS_FS_XATTR + .listxattr = erofs_listxattr, +#endif + .get_acl = erofs_get_acl, };const struct inode_operations erofs_fast_symlink_iops = { .get_link = simple_get_link, .getattr = erofs_getattr, +#ifdef CONFIG_EROFS_FS_XATTR + .listxattr = erofs_listxattr, +#endif + .get_acl = erofs_get_acl, };
fs/erofs/internal.h +16 -0
@@ -63,6 +63,9 @@ typedef u32 erofs_blk_t;struct erofs_sb_info {u32 blocks;u32 meta_blkaddr; +#ifdef CONFIG_EROFS_FS_XATTR +u32 xattr_blkaddr; +#endif /* inode slot unit size in bit shift */ unsigned char islotbits;
@@ -133,6 +136,8 @@ static inline void *erofs_kmalloc(struct erofs_sb_info *sbi,#define EROFS_I_SB (inode) ((struct erofs_sb_info *)(inode)->i_sb->s_fs_info)/* Mount flags set via mount options or defaults */ +#define EROFS_MOUNT_XATTR_USER 0x00000010 +#define EROFS_MOUNT_POSIX_ACL 0x00000020#define EROFS_MOUNT_FAULT_INJECTION 0x00000040#define clear_opt (sbi, option) ((sbi)->mount_opt &= ~EROFS_MOUNT_##option)
@@ -167,13 +172,24 @@ static inline erofs_off_t iloc(struct erofs_sb_info *sbi, erofs_nid_t nid)return blknr_to_addr (sbi->meta_blkaddr) + (nid << sbi->islotbits); } +/* atomic flag definitions */ +#define EROFS_V_EA_INITED_BIT 0 + +/* bitlock definitions (arranged in reverse order) */ +#define EROFS_V_BL_XATTR_BIT (BITS_PER_LONG - 1)struct erofs_vnode {erofs_nid_t nid; +/* atomic flags (including bitlocks) */ +unsigned long flags; +unsigned char datamode;unsigned char inode_isize;unsigned short xattr_isize; +unsigned int xattr_shared_count; +unsigned int *xattr_shared_xattrs; +erofs_blk_t raw_blkaddr;/* the corresponding vfs inode */ struct inode vfs_inode;
fs/erofs/namei.c +5 -1
@@ -6,7 +6,7 @@* http://www.huawei.com/ * Created by Gao Xiang <gaoxiang25 at huawei.com> */ -#include "internal.h" +#include "xattr.h" #include <trace/events/erofs.h>
@@ -243,5 +243,9 @@ static struct dentry *erofs_lookup(struct inode *dir,const struct inode_operations erofs_dir_iops = { .lookup = erofs_lookup, .getattr = erofs_getattr, +#ifdef CONFIG_EROFS_FS_XATTR + .listxattr = erofs_listxattr, +#endif + .get_acl = erofs_get_acl, };
fs/erofs/super.c +76 -1
139 lines not on this page · raw message
fs/erofs/xattr.c new file +700 -0
700 lines not on this page · raw message
fs/erofs/xattr.h new file +94 -0
94 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v6 12/24 erofs: introduce tagged pointer
In reply to Gao XiangCurrently kernel has scattered tagged pointer usages hacked by hand in plain code, without a unique and portable functionset to highlight the tagged pointer itself and wrap these hacked code in order to clean up all over meaningless magic masks.
This patch introduces simple generic methods to fold tags into a pointer integer. Currently it supports the last n bits of the pointer for tags, which can be selected by users.
In addition, it will also be used for the upcoming EROFS filesystem, which heavily uses tagged pointer pproach
to reduce extra memory allocation.
--- fs/erofs/tagptr.h | 110 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 fs/erofs/tagptr.h
fs/erofs/tagptr.h new file +110 -0
@@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* A tagged pointer implementation +* +* Copyright (C) 2018 Gao Xiang <gaoxiang25 at huawei.com> +*/ +#ifndef __EROFS_FS_TAGPTR_H +#define __EROFS_FS_TAGPTR_H + +#include <linux/types.h> +#include <linux/build_bug.h> + +/* +* the name of tagged pointer types are tagptr{1, 2, 3...}_t +* avoid directly using the internal structs __tagptr{1, 2, 3...} +*/ +#define __MAKE_TAGPTR (n) \ +typedef struct __tagptr##n { \ + uintptr_t v; \ +} tagptr##n##_t; + +__MAKE_TAGPTR(1) +__MAKE_TAGPTR(2) +__MAKE_TAGPTR(3) +__MAKE_TAGPTR(4) + +#undef __MAKE_TAGPTR + +extern void __compiletime_error("bad tagptr tags") +__bad_tagptr_tags (void); + +extern void __compiletime_error("bad tagptr type") +__bad_tagptr_type (void); + +/* fix the broken usage of "#define tagptr2_t tagptr3_t" by users */ +#define __tagptr_mask_1 (ptr, n) \ + __builtin_types_compatible_p(typeof(ptr), struct __tagptr##n) ? \ + (1UL << (n)) - 1 : + +#define __tagptr_mask (ptr) (\ + __tagptr_mask_1(ptr, 1) ( \ + __tagptr_mask_1(ptr, 2) ( \ + __tagptr_mask_1(ptr, 3) ( \ + __tagptr_mask_1(ptr, 4) ( \ + __bad_tagptr_type(), 0))))) + +/* generate a tagged pointer from a raw value */ +#define tagptr_init (type, val) \ + ((typeof(type)){ .v = (uintptr_t)(val) }) + +/* +* directly cast a tagged pointer to the native pointer type, which +* could be used for backward compatibility of existing code. +*/ +#define tagptr_cast_ptr (tptr) ((void *)(tptr).v) + +/* encode tagged pointers */ +#define tagptr_fold (type, ptr, _tags) ({ \ + const typeof(_tags) tags = (_tags); \ + if (__builtin_constant_p(tags) && (tags & ~__tagptr_mask(type))) \ + __bad_tagptr_tags(); \ +tagptr_init(type, (uintptr_t)(ptr) | tags); }) + +/* decode tagged pointers */ +#define tagptr_unfold_ptr (tptr) \ + ((void *)((tptr).v & ~__tagptr_mask(tptr))) + +#define tagptr_unfold_tags (tptr) \ + ((tptr).v & __tagptr_mask(tptr)) + +/* operations for the tagger pointer */ +#define tagptr_eq (_tptr1, _tptr2) ({ \ + typeof(_tptr1) tptr1 = (_tptr1); \ + typeof(_tptr2) tptr2 = (_tptr2); \ + (void)(&tptr1 == &tptr2); \ +(tptr1).v == (tptr2).v; }) + +/* lock-free CAS operation */ +#define tagptr_cmpxchg (_ptptr, _o, _n) ({ \ + typeof(_ptptr) ptptr = (_ptptr); \ + typeof(_o) o = (_o); \ + typeof(_n) n = (_n); \ + (void)(&o == &n); \ + (void)(&o == ptptr); \ +tagptr_init(o, cmpxchg(&ptptr->v, o.v, n.v)); }) + +/* wrap WRITE_ONCE if atomic update is needed */ +#define tagptr_replace_tags (_ptptr, tags) ({ \ + typeof(_ptptr) ptptr = (_ptptr); \ + *ptptr = tagptr_fold(*ptptr, tagptr_unfold_ptr(*ptptr), tags); \ +*ptptr; }) + +#define tagptr_set_tags (_ptptr, _tags) ({ \ + typeof(_ptptr) ptptr = (_ptptr); \ + const typeof(_tags) tags = (_tags); \ + if (__builtin_constant_p(tags) && (tags & ~__tagptr_mask(*ptptr))) \ + __bad_tagptr_tags(); \ + ptptr->v |= tags; \ +*ptptr; }) + +#define tagptr_clear_tags (_ptptr, _tags) ({ \ + typeof(_ptptr) ptptr = (_ptptr); \ + const typeof(_tags) tags = (_tags); \ + if (__builtin_constant_p(tags) && (tags & ~__tagptr_mask(*ptptr))) \ + __bad_tagptr_tags(); \ + ptptr->v &= ~tags; \ +*ptptr; }) + +#endif /* __EROFS_FS_TAGPTR_H */ +
Signature · 1 line
2.17.1
PATCH v6 13/24 erofs: add compression indexes support
In reply to Gao XiangThis patch adds EROFS compression indexes support, including legacy and compacted 2/4B indexes.
In addition, it introduces an iterable L2P mapping operation called 'z_erofs_map_blocks_iter'.
Compared with 'erofs_map_blocks', it avoids a number of redundant 'release and regrab' processes if they request the same meta page.
--- fs/erofs/Kconfig | 9 + fs/erofs/Makefile | 1 + fs/erofs/data.c | 10 +- fs/erofs/inode.c | 2 +- fs/erofs/internal.h | 35 ++- fs/erofs/zmap.c | 460 +++++++++++++++++++++++++++++++++++ include/trace/events/erofs.h | 17 +- 7 files changed, 529 insertions(+), 5 deletions(-) create mode 100644 fs/erofs/zmap.c
fs/erofs/Kconfig +9 -0
@@ -72,3 +72,12 @@ config EROFS_FS_SECURITY If you are not using a security module, say N. +config EROFS_FS_ZIP + bool "EROFS Data Compression Support" + depends on EROFS_FS + select LZ4_DECOMPRESS + help + Enable fixed-sized output compression for EROFS. + + If you don't want to enable compression feature, say N. +
fs/erofs/Makefile +1 -0
@@ -7,4 +7,5 @@ ccflags-y += -DEROFS_VERSION=\"$(EROFS_VERSION)\" obj-$(CONFIG_EROFS_FS) += erofs.o erofs-objs := super.o inode.o data.o namei.o dir.o erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o +erofs-$(CONFIG_EROFS_FS_ZIP) += zmap.o
fs/erofs/data.c +8 -2
@@ -170,9 +170,15 @@ static int erofs_map_blocks_flatmode(struct inode *inode,int erofs_map_blocks (struct inode *inode,struct erofs_map_blocks *map,int flags) { -if (is_inode_layout_compression (inode)) -return -ENOTSUPP ; +if (unlikely (is_inode_layout_compression (inode))) { +int err =z_erofs_map_blocks_iter (inode, map, flags); +if (map->mpage) { +put_page (map->mpage); + map->mpage =NULL ; + } +return err; + }return erofs_map_blocks_flatmode (inode, map, flags); }
fs/erofs/inode.c +1 -1
@@ -205,7 +205,7 @@ static int fill_inode(struct inode *inode, int isdir) }if (is_inode_layout_compression (inode)) { - err = -ENOTSUPP ; + err =z_erofs_fill_inode (inode); goto out_unlock; }
fs/erofs/internal.h +34 -1
@@ -174,9 +174,12 @@ static inline erofs_off_t iloc(struct erofs_sb_info *sbi, erofs_nid_t nid)/* atomic flag definitions */ #define EROFS_V_EA_INITED_BIT 0 +#define EROFS_V_Z_INITED_BIT 1/* bitlock definitions (arranged in reverse order) */ #define EROFS_V_BL_XATTR_BIT (BITS_PER_LONG - 1) +#define EROFS_V_BL_Z_BIT (BITS_PER_LONG - 2) +struct erofs_vnode {erofs_nid_t nid;
@@ -190,7 +193,17 @@ struct erofs_vnode {unsigned int xattr_shared_count;unsigned int *xattr_shared_xattrs; -erofs_blk_t raw_blkaddr; +union { +erofs_blk_t raw_blkaddr; +#ifdef CONFIG_EROFS_FS_ZIP +struct { +unsigned short z_advise; +unsigned char z_algorithmtype[2 ]; +unsigned char z_logical_clusterbits; +unsigned char z_physical_clusterbits[2 ]; + }; +#endif /* CONFIG_EROFS_FS_ZIP */ + };/* the corresponding vfs inode */ struct inode vfs_inode; };
@@ -263,6 +276,10 @@ enum {#define EROFS_MAP_MAPPED (1 << BH_Mapped)/* Located in metadata (could be copied from bd_inode) */ #define EROFS_MAP_META (1 << BH_Meta) +/* The extent has been compressed */ +#define EROFS_MAP_ZIPPED (1 << BH_Zipped) +/* The length of extent is full */ +#define EROFS_MAP_FULL_MAPPED (1 << BH_FullMapped)struct erofs_map_blocks {erofs_off_t m_pa, m_la;
@@ -276,6 +293,22 @@ struct erofs_map_blocks {/* Flags used by erofs_map_blocks() */ #define EROFS_GET_BLOCKS_RAW 0x0001 +/* zmap.c */ +#ifdef CONFIG_EROFS_FS_ZIP +int z_erofs_fill_inode (struct inode *inode); +int z_erofs_map_blocks_iter (struct inode *inode, +struct erofs_map_blocks *map, +int flags); +#else +static inline int z_erofs_fill_inode (struct inode *inode) {return -ENOTSUPP ; } +static inline int z_erofs_map_blocks_iter (struct inode *inode, +struct erofs_map_blocks *map, +int flags) +{ +return -ENOTSUPP ; +} +#endif /* !CONFIG_EROFS_FS_ZIP */ +/* data.c */ static inline struct bio *erofs_grab_bio(struct super_block *sb, erofs_blk_t blkaddr,
fs/erofs/zmap.c new file +460 -0
460 lines not on this page · raw message
include/trace/events/erofs.h +16 -1
@@ -20,7 +20,8 @@#define show_mflags (flags) __print_flags(flags, "", \ { EROFS_MAP_MAPPED, "M" }, \ - { EROFS_MAP_META, "I" }) + { EROFS_MAP_META, "I" }, \ + { EROFS_MAP_ZIPPED, "Z" }) TRACE_EVENT(erofs_lookup,
@@ -172,6 +173,13 @@ DEFINE_EVENT(erofs__map_blocks_enter, erofs_map_blocks_flatmode_enter,TP_ARGS (inode, map, flags) ); +DEFINE_EVENT (erofs__map_blocks_enter, z_erofs_map_blocks_iter_enter, + TP_PROTO(struct inode *inode, struct erofs_map_blocks *map, + unsigned int flags), + +TP_ARGS (inode, map, flags) +); + DECLARE_EVENT_CLASS(erofs__map_blocks_exit, TP_PROTO(struct inode *inode, struct erofs_map_blocks *map, unsigned int flags, int ret),
@@ -217,6 +225,13 @@ DEFINE_EVENT(erofs__map_blocks_exit, erofs_map_blocks_flatmode_exit,TP_ARGS (inode, map, flags, ret) ); +DEFINE_EVENT (erofs__map_blocks_exit, z_erofs_map_blocks_iter_exit, + TP_PROTO(struct inode *inode, struct erofs_map_blocks *map, + unsigned int flags, int ret), + +TP_ARGS (inode, map, flags, ret) +); + TRACE_EVENT(erofs_destroy_inode, TP_PROTO(struct inode *inode),
Signature · 1 line
2.17.1
PATCH v6 14/24 erofs: introduce superblock registration
In reply to Gao XiangIn order to introducing shrinker solution for erofs, let's manage all mounted erofs instances at first.
--- fs/erofs/Makefile | 2 +- fs/erofs/internal.h | 13 +++++++++++++ fs/erofs/super.c | 9 +++++++++ fs/erofs/utils.c | 32 ++++++++++++++++++++++++++++++++ 4 files changed, 55 insertions(+), 1 deletion(-) create mode 100644 fs/erofs/utils.c
fs/erofs/Makefile +1 -1
@@ -5,7 +5,7 @@ EROFS_VERSION = "1.0" ccflags-y += -DEROFS_VERSION=\"$(EROFS_VERSION)\" obj-$(CONFIG_EROFS_FS) += erofs.o -erofs-objs := super.o inode.o data.o namei.o dir.o +erofs-objs := super.o inode.o data.o namei.o dir.o utils.o erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o erofs-$(CONFIG_EROFS_FS_ZIP) += zmap.o
fs/erofs/internal.h +13 -0
@@ -61,6 +61,10 @@ typedef u64 erofs_off_t;typedef u32 erofs_blk_t ;struct erofs_sb_info { +#ifdef CONFIG_EROFS_FS_ZIP +/* list for all registered superblocks, mainly for shrinker */ +struct list_head list; +#endif /* CONFIG_EROFS_FS_ZIP */ u32 blocks;u32 meta_blkaddr;#ifdef CONFIG_EROFS_FS_XATTR
@@ -401,5 +405,14 @@ int erofs_namei(struct inode *dir, struct qstr *name,/* dir.c */ extern const struct file_operations erofs_dir_fops; +/* utils.c */ +#ifdef CONFIG_EROFS_FS_ZIP +void erofs_shrinker_register (struct super_block *sb); +void erofs_shrinker_unregister (struct super_block *sb); +#else +static inline void erofs_shrinker_register (struct super_block *sb) {} +static inline void erofs_shrinker_unregister (struct super_block *sb) {} +#endif /* !CONFIG_EROFS_FS_ZIP */ +#endif /* __EROFS_INTERNAL_H */
fs/erofs/super.c +9 -0
@@ -354,6 +354,8 @@ static int erofs_fill_super(struct super_block *sb, void *data, int silent)if (unlikely (!sb->s_root))return -ENOMEM ; +erofs_shrinker_register (sb); +if (!silent)infoln ("mounted on %s with opts: %s." , sb->s_id, (char *)data);return 0 ;
@@ -385,6 +387,12 @@ static void erofs_kill_sb(struct super_block *sb) sb->s_fs_info =NULL ; } +/* called when ->s_root is non-NULL */ +static void erofs_put_super (struct super_block *sb) +{ +erofs_shrinker_unregister (sb); +} +static struct file_system_type erofs_fs_type = { .owner =THIS_MODULE , .name ="erofs" ,
@@ -496,6 +504,7 @@ static int erofs_remount(struct super_block *sb, int *flags, char *data) }const struct super_operations erofs_sops = { + .put_super = erofs_put_super, .alloc_inode = alloc_inode, .free_inode = free_inode, .statfs = erofs_statfs,
fs/erofs/utils.c new file +32 -0
@@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/fs/erofs/utils.c +* +* Copyright (C) 2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#include "internal.h" + +#ifdef CONFIG_EROFS_FS_ZIP +/* protects the mounted 'erofs_sb_list' */ +static DEFINE_SPINLOCK (erofs_sb_list_lock); +static LIST_HEAD (erofs_sb_list); + +void erofs_shrinker_register (struct super_block *sb) +{ +struct erofs_sb_info *sbi =EROFS_SB (sb); + +spin_lock (&erofs_sb_list_lock); +list_add (&sbi->list, &erofs_sb_list); +spin_unlock (&erofs_sb_list_lock); +} + +void erofs_shrinker_unregister (struct super_block *sb) +{ +spin_lock (&erofs_sb_list_lock); +list_del (&EROFS_SB (sb)->list); +spin_unlock (&erofs_sb_list_lock); +} +#endif /* !CONFIG_EROFS_FS_ZIP */ +
Signature · 1 line
2.17.1
PATCH v6 15/24 erofs: introduce erofs shrinker
In reply to Gao XiangThis patch adds a dedicated shrinker targeting to free unneeded memory consumed by a number of erofs in-memory data structures.
Like F2FS and UBIFS, it also adds:
- sbi->umount_mutex to avoid races on shrinker and put_super;
- sbi->shrinker_run_no to not revisit recently scanned objects.
--- fs/erofs/internal.h | 7 ++++ fs/erofs/super.c | 6 +++ fs/erofs/utils.c | 93 ++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 105 insertions(+), 1 deletion(-)
fs/erofs/internal.h +7 -0
@@ -64,6 +64,9 @@ struct erofs_sb_info {#ifdef CONFIG_EROFS_FS_ZIP /* list for all registered superblocks, mainly for shrinker */ struct list_head list; +struct mutex umount_mutex; + +unsigned int shrinker_run_no;#endif /* CONFIG_EROFS_FS_ZIP */ u32 blocks;u32 meta_blkaddr;
@@ -409,9 +412,13 @@ extern const struct file_operations erofs_dir_fops;#ifdef CONFIG_EROFS_FS_ZIP void erofs_shrinker_register (struct super_block *sb);void erofs_shrinker_unregister (struct super_block *sb); +int __init erofs_init_shrinker (void ); +void erofs_exit_shrinker (void );#else static inline void erofs_shrinker_register (struct super_block *sb) {}static inline void erofs_shrinker_unregister (struct super_block *sb) {} +static inline int erofs_init_shrinker (void ) {return 0 ; } +static inline void erofs_exit_shrinker (void ) {}#endif /* !CONFIG_EROFS_FS_ZIP */ #endif /* __EROFS_INTERNAL_H */
fs/erofs/super.c +6 -0
@@ -413,6 +413,9 @@ static int __init erofs_module_init(void)if (err) goto icache_err; + err =erofs_init_shrinker (); +if (err) + goto shrinker_err; err =register_filesystem (&erofs_fs_type);if (err) goto fs_err;
@@ -421,6 +424,8 @@ static int __init erofs_module_init(void)return 0 ; fs_err: +erofs_exit_shrinker (); +shrinker_err:erofs_exit_inode_cache (); icache_err:return err;
@@ -429,6 +434,7 @@ static int __init erofs_module_init(void) static void __exit erofs_module_exit(void) {unregister_filesystem (&erofs_fs_type); +erofs_exit_shrinker ();erofs_exit_inode_cache ();infoln ("successfully finalize erofs" ); }
fs/erofs/utils.c +92 -1
113 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v6 16/24 erofs: introduce workstation for decompression
In reply to Gao XiangThis patch introduces another concept used by decompress subsystem called 'workstation'. It can be seen as a sparse array that stores pointers pointed to data structures related to the corresponding physical clusters.
All lookups are protected by RCU read lock. Besides, reference count and spin_lock are also introduced to manage its lifetime and serialize all update operations.
`workstation' is currently implemented on the in-kernel radix tree approach for backward compatibility. With the evolution of linux kernel, it will be migrated into new XArray implementation in the future.
--- fs/erofs/internal.h | 80 +++++++++++++++++++++ fs/erofs/super.c | 4 ++ fs/erofs/utils.c | 166 +++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 248 insertions(+), 2 deletions(-)
fs/erofs/internal.h +80 -0
@@ -66,6 +66,9 @@ struct erofs_sb_info {struct list_head list;struct mutex umount_mutex; +/* the dedicated workstation for compression */ +struct radix_tree_root workstn_tree; +unsigned int shrinker_run_no;#endif /* CONFIG_EROFS_FS_ZIP */ u32 blocks;
@@ -151,6 +154,77 @@ static inline void *erofs_kmalloc(struct erofs_sb_info *sbi,#define set_opt (sbi, option) ((sbi)->mount_opt |= EROFS_MOUNT_##option)#define test_opt (sbi, option) ((sbi)->mount_opt & EROFS_MOUNT_##option) +#ifdef CONFIG_EROFS_FS_ZIP +#define EROFS_LOCKED_MAGIC (INT_MIN | 0xE0F510CCL) + +/* basic unit of the workstation of a super_block */ +struct erofs_workgroup { +/* the workgroup index in the workstation */ +pgoff_t index; + +/* overall workgroup reference count */ +atomic_t refcount; +}; + +#if defined(CONFIG_SMP ) +static inline bool erofs_workgroup_try_to_freeze (struct erofs_workgroup *grp, +int val) +{ +preempt_disable (); +if (val !=atomic_cmpxchg (&grp->refcount, val,EROFS_LOCKED_MAGIC )) { +preempt_enable (); +return false; + } +return true; +} + +static inline void erofs_workgroup_unfreeze (struct erofs_workgroup *grp, +int orig_val) +{ +/* +* other observers should notice all modifications +* in the freezing period. +*/ +smp_mb (); +atomic_set (&grp->refcount, orig_val); +preempt_enable (); +} + +static inline int erofs_wait_on_workgroup_freezed (struct erofs_workgroup *grp) +{ +return atomic_cond_read_relaxed (&grp->refcount, +VAL !=EROFS_LOCKED_MAGIC ); +} +#else +static inline bool erofs_workgroup_try_to_freeze (struct erofs_workgroup *grp, +int val) +{ +preempt_disable (); +/* no need to spin on UP platforms, let's just disable preemption. */ +if (val !=atomic_read (&grp->refcount)) { +preempt_enable (); +return false; + } +return true; +} + +static inline void erofs_workgroup_unfreeze (struct erofs_workgroup *grp, +int orig_val) +{ +preempt_enable (); +} + +static inline int erofs_wait_on_workgroup_freezed (struct erofs_workgroup *grp) +{ +int v =atomic_read (&grp->refcount); + +/* workgroup is never freezed on uniprocessor systems */ +DBG_BUGON (v ==EROFS_LOCKED_MAGIC ); +return v; +} +#endif /* !CONFIG_SMP */ +#endif /* !CONFIG_EROFS_FS_ZIP */ +/* we strictly follow PAGE_SIZE and no buffer head yet */ #define LOG_BLOCK_SIZE PAGE_SHIFT
@@ -410,6 +484,12 @@ extern const struct file_operations erofs_dir_fops;/* utils.c */ #ifdef CONFIG_EROFS_FS_ZIP +int erofs_workgroup_put (struct erofs_workgroup *grp); +struct erofs_workgroup *erofs_find_workgroup (struct super_block *sb, +pgoff_t index,bool *tag); +int erofs_register_workgroup (struct super_block *sb, +struct erofs_workgroup *grp,bool tag); +static inline void erofs_workgroup_free_rcu (struct erofs_workgroup *grp) {}void erofs_shrinker_register (struct super_block *sb);void erofs_shrinker_unregister (struct super_block *sb);int __init erofs_init_shrinker (void );
fs/erofs/super.c +4 -0
@@ -338,6 +338,10 @@ static int erofs_fill_super(struct super_block *sb, void *data, int silent)else sb->s_flags &= ~SB_POSIXACL ; +#ifdef CONFIG_EROFS_FS_ZIP +INIT_RADIX_TREE (&sbi->workstn_tree,GFP_ATOMIC ); +#endif +/* get the root inode */ inode =erofs_iget (sb,ROOT_NID (sbi), true);if (IS_ERR (inode))
fs/erofs/utils.c +164 -2
189 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v6 17/24 erofs: introduce per-CPU buffers implementation
In reply to Gao XiangThis patch introduces per-CPU buffers in order for the upcoming generic decompression framework to use.
Note that I tried to use in-kernel per-CPU buffer or per-CPU page approaches to clean up further, however noticeable performanace regression (about 2% for sequential read) was observed.
Let's leave it as-is for now.
--- fs/erofs/Kconfig | 14 ++++++++++++++ fs/erofs/internal.h | 21 +++++++++++++++++++++ fs/erofs/utils.c | 12 ++++++++++++ 3 files changed, 47 insertions(+)
fs/erofs/Kconfig +14 -0
@@ -81,3 +81,17 @@ config EROFS_FS_ZIP If you don't want to enable compression feature, say N. +config EROFS_FS_CLUSTER_PAGE_LIMIT + int "EROFS Cluster Pages Hard Limit" + depends on EROFS_FS_ZIP + range 1 256 + default "1" + help + Indicates maximum # of pages of a compressed + physical cluster. + + For example, if files in a image were compressed + into 8k-unit, hard limit should not be configured + less than 2. Otherwise, the image will be refused + to mount on this kernel. +
fs/erofs/internal.h +21 -0
@@ -223,6 +223,12 @@ static inline int erofs_wait_on_workgroup_freezed(struct erofs_workgroup *grp)return v; }#endif /* !CONFIG_SMP */ + +/* hard limit of pages per compressed cluster */ +#define Z_EROFS_CLUSTER_MAX_PAGES (CONFIG_EROFS_FS_CLUSTER_PAGE_LIMIT) +#define EROFS_PCPUBUF_NR_PAGES Z_EROFS_CLUSTER_MAX_PAGES +#else +#define EROFS_PCPUBUF_NR_PAGES 0#endif /* !CONFIG_EROFS_FS_ZIP */ /* we strictly follow PAGE_SIZE and no buffer head yet */
@@ -483,6 +489,21 @@ int erofs_namei(struct inode *dir, struct qstr *name,extern const struct file_operations erofs_dir_fops;/* utils.c */ +#if (EROFS_PCPUBUF_NR_PAGES >0 ) +void *erofs_get_pcpubuf (unsigned int pagenr); +#define erofs_put_pcpubuf (buf) do { \ + (void)&(buf); \ + preempt_enable(); \ +} while (0) +#else +static inline void *erofs_get_pcpubuf (unsigned int pagenr) +{ +return ERR_PTR (-ENOTSUPP ); +} + +#define erofs_put_pcpubuf (buf) do {} while (0) +#endif +#ifdef CONFIG_EROFS_FS_ZIP int erofs_workgroup_put (struct erofs_workgroup *grp); struct erofs_workgroup *erofs_find_workgroup(struct super_block *sb,
fs/erofs/utils.c +12 -0
@@ -9,6 +9,18 @@#include "internal.h" #include <linux/pagevec.h> +#if (EROFS_PCPUBUF_NR_PAGES >0 ) +static struct { +u8 data[PAGE_SIZE *EROFS_PCPUBUF_NR_PAGES ]; +}____cacheline_aligned_in_smp erofs_pcpubuf[NR_CPUS ]; + +void *erofs_get_pcpubuf (unsigned int pagenr) +{ +preempt_disable (); +return &erofs_pcpubuf[smp_processor_id ()].data[pagenr *PAGE_SIZE ]; +} +#endif +#ifdef CONFIG_EROFS_FS_ZIP /* global shrink count (for all mounted EROFS instances) */ static atomic_long_t erofs_global_shrink_cnt;
Signature · 1 line
2.17.1
PATCH v6 18/24 erofs: introduce pagevec for decompression subsystem
In reply to Gao XiangFor each physical cluster, there is a straight-forward way of allocating a fixed or variable-sized array to record the corresponding file pages for its decompression if we decide to decompress these pages asynchronously (eg. read-ahead case), however it will take variable-sized on-heap memory compared with traditional uncompressed filesystems.
This patch introduces a pagevec solution to reuse some allocated file page in the time-sharing approach to store parts of the array itself in order to minimize the extra memory overhead, thus only a small-sized constant array used for booting the whole array itself up will be needed.
--- fs/erofs/zpvec.h | 159 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 159 insertions(+) create mode 100644 fs/erofs/zpvec.h
fs/erofs/zpvec.h new file +159 -0
@@ -0,0 +1,159 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* linux/fs/erofs/zpvec.h +* +* Copyright (C) 2018 HUAWEI, Inc. +* http://www.huawei.com/ +* Created by Gao Xiang <gaoxiang25 at huawei.com> +*/ +#ifndef __EROFS_FS_ZPVEC_H +#define __EROFS_FS_ZPVEC_H + +#include "tagptr.h" + +/* page type in pagevec for decompress subsystem */ +enum z_erofs_page_type { +/* including Z_EROFS_VLE_PAGE_TAIL_EXCLUSIVE */ +Z_EROFS_PAGE_TYPE_EXCLUSIVE , + +Z_EROFS_VLE_PAGE_TYPE_TAIL_SHARED , + +Z_EROFS_VLE_PAGE_TYPE_HEAD , +Z_EROFS_VLE_PAGE_TYPE_MAX +}; + +extern void __compiletime_error("Z_EROFS_PAGE_TYPE_EXCLUSIVE != 0") +__bad_page_type_exclusive (void); + +/* pagevec tagged pointer */ +typedef tagptr2_t erofs_vtptr_t ; + +/* pagevec collector */ +struct z_erofs_pagevec_ctor { +struct page *curr, *next; +erofs_vtptr_t *pages; + +unsigned int nr, index; +}; + +static inline void z_erofs_pagevec_ctor_exit (struct z_erofs_pagevec_ctor *ctor, +bool atomic) +{ +if (!ctor->curr) +return ; + +if (atomic) +kunmap_atomic (ctor->pages); +else +kunmap (ctor->curr); +} + +static inline struct page * +z_erofs_pagevec_ctor_next_page (struct z_erofs_pagevec_ctor *ctor, +unsigned int nr) +{ +unsigned int index; + +/* keep away from occupied pages */ +if (ctor->next) +return ctor->next; + +for (index =0 ; index < nr; ++index) { +const erofs_vtptr_t t = ctor->pages[index]; +const unsigned int tags =tagptr_unfold_tags (t); + +if (tags ==Z_EROFS_PAGE_TYPE_EXCLUSIVE ) +return tagptr_unfold_ptr (t); + } +DBG_BUGON (nr >= ctor->nr); +return NULL ; +} + +static inline void +z_erofs_pagevec_ctor_pagedown (struct z_erofs_pagevec_ctor *ctor, +bool atomic) +{ +struct page *next =z_erofs_pagevec_ctor_next_page (ctor, ctor->nr); + +z_erofs_pagevec_ctor_exit (ctor, atomic); + + ctor->curr = next; + ctor->next =NULL ; + ctor->pages = atomic ? +kmap_atomic (ctor->curr) :kmap (ctor->curr); + + ctor->nr =PAGE_SIZE /sizeof (struct page *); + ctor->index =0 ; +} + +static inline void z_erofs_pagevec_ctor_init (struct z_erofs_pagevec_ctor *ctor, +unsigned int nr, +erofs_vtptr_t *pages, +unsigned int i) +{ + ctor->nr = nr; + ctor->curr = ctor->next =NULL ; + ctor->pages = pages; + +if (i >= nr) { + i -= nr; +z_erofs_pagevec_ctor_pagedown (ctor, false); +while (i > ctor->nr) { + i -= ctor->nr; +z_erofs_pagevec_ctor_pagedown (ctor, false); + } + } + ctor->next =z_erofs_pagevec_ctor_next_page (ctor, i); + ctor->index = i; +} + +static inline bool z_erofs_pagevec_enqueue (struct z_erofs_pagevec_ctor *ctor, +struct page *page, +enum z_erofs_page_type type, +bool *occupied) +{ + *occupied = false; +if (unlikely (!ctor->next && type)) +if (ctor->index +1 == ctor->nr) +return false; + +if (unlikely (ctor->index >= ctor->nr)) +z_erofs_pagevec_ctor_pagedown (ctor, false); + +/* exclusive page type must be 0 */ +if (Z_EROFS_PAGE_TYPE_EXCLUSIVE != (uintptr_t )NULL ) +__bad_page_type_exclusive (); + +/* should remind that collector->next never equal to 1, 2 */ +if (type == (uintptr_t )ctor->next) { + ctor->next = page; + *occupied = true; + } + ctor->pages[ctor->index++] =tagptr_fold (erofs_vtptr_t, page, type); +return true; +} + +static inline struct page * +z_erofs_pagevec_dequeue (struct z_erofs_pagevec_ctor *ctor, +enum z_erofs_page_type *type) +{ +erofs_vtptr_t t; + +if (unlikely (ctor->index >= ctor->nr)) { +DBG_BUGON (!ctor->next); +z_erofs_pagevec_ctor_pagedown (ctor, true); + } + + t = ctor->pages[ctor->index]; + + *type =tagptr_unfold_tags (t); + +/* should remind that collector->next never equal to 1, 2 */ +if (*type == (uintptr_t )ctor->next) + ctor->next =tagptr_unfold_ptr (t); + + ctor->pages[ctor->index++] =tagptr_fold (erofs_vtptr_t,NULL ,0 ); +return tagptr_unfold_ptr (t); +} +#endif +
Signature · 1 line
2.17.1
PATCH v6 19/24 erofs: add erofs_allocpage()
In reply to Gao XiangThis patch introduces an temporary _on-stack_ page pool to reuse the freed page directly as much as it can for better performance and release all pages at a time, it also slightly reduces the possibility of the potential memory allocation failure.
--- fs/erofs/internal.h | 2 ++ fs/erofs/utils.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+)
fs/erofs/internal.h +2 -0
@@ -489,6 +489,8 @@ int erofs_namei(struct inode *dir, struct qstr *name,extern const struct file_operations erofs_dir_fops;/* utils.c */ +struct page *erofs_allocpage (struct list_head *pool,gfp_t gfp,bool nofail); +#if (EROFS_PCPUBUF_NR_PAGES >0 )void *erofs_get_pcpubuf (unsigned int pagenr);#define erofs_put_pcpubuf (buf) do { \
fs/erofs/utils.c +14 -0
@@ -9,6 +9,20 @@#include "internal.h" #include <linux/pagevec.h> +struct page *erofs_allocpage (struct list_head *pool,gfp_t gfp,bool nofail) +{ +struct page *page; + +if (!list_empty (pool)) { + page =lru_to_page (pool); +DBG_BUGON (page_ref_count (page) !=1 ); +list_del (&page->lru); + }else { + page =alloc_pages (gfp | (nofail ? __GFP_NOFAIL :0 ),0 ); + } +return page; +} + #if (EROFS_PCPUBUF_NR_PAGES > 0) static struct { u8 data[PAGE_SIZE * EROFS_PCPUBUF_NR_PAGES];
Signature · 1 line
2.17.1
PATCH v6 20/24 erofs: introduce generic decompression backend
In reply to Gao XiangThis patch adds decompression backend to EROFS, which supports uncompressed and LZ4 compressed data.
For compressed data, it uses the following strategy:
1) If outputsize is very small (totally less than a threshold), decompress to the per-CPU buffer and do memcpy directly in order to avoid vmap() overhead;
2) Otherwise it will fill bounced pages if needed and vmap all output pages into a continuous virtual memory area, memcpy compressed data to the per-CPU buffer for inplace I/O [1] and decompress.
Since LZ4 is a lz77-based algorithm which has a dynamically populated ("sliding window") dictionary and the maximum lookback distance is 65535. Therefore the number of bounced pages could be limited by erofs based on this property.
[1] `LZ4 decompression inplace' will eliminate the extra memcpy if iend - oend margin is safe enough, see the following patch.
--- fs/erofs/Makefile | 2 +- fs/erofs/compress.h | 62 ++++++++ fs/erofs/decompressor.c | 332 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 395 insertions(+), 1 deletion(-) create mode 100644 fs/erofs/compress.h create mode 100644 fs/erofs/decompressor.c
fs/erofs/Makefile +1 -1
@@ -7,5 +7,5 @@ ccflags-y += -DEROFS_VERSION=\"$(EROFS_VERSION)\" obj-$(CONFIG_EROFS_FS) += erofs.o erofs-objs := super.o inode.o data.o namei.o dir.o utils.o erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o -erofs-$(CONFIG_EROFS_FS_ZIP) += zmap.o +erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o
fs/erofs/compress.h new file +62 -0
62 lines not on this page · raw message
fs/erofs/decompressor.c new file +332 -0
332 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v6 21/24 erofs: introduce LZ4 decompression inplace
In reply to Gao Xiangcompressed data will be usually loaded into last pages of the extent (the last page for 4k) for in-place decompression (more specifically, in-place IO), as ilustration below,
start of compressed logical extent
| end of this logical extent
| |
______v___________________________v________
... | page 6 | page 7 | page 8 | page 9 | ...
|__________|__________|__________|__________|
. ^ . ^
. |compressed|
. | data |
. . .
|< dstsize >|<margin>|
oend iendop ip
Therefore, it's possible to do decompression inplace (thus no memcpy at all) if the margin is sufficient and safe enough [1], and it can be implemented only for fixed-size output compression compared with fixed-size input compression.
No memcpy for most of in-place IO (about 99% of enwik9) after decompression inplace is implemented and sequential read will be improved of course (see the following patches for test results).
[1] https://github.com/lz4/lz4/commit/b17f578a919b7e6b078cede2d52be29dd48c8e8c
https://github.com/lz4/lz4/commit/5997e139f53169fa3a1c1b4418d2452a90b01602
--- fs/erofs/decompressor.c | 36 ++++++++++++++++++++++++++++++++---- fs/erofs/erofs_fs.h | 2 +- 2 files changed, 33 insertions(+), 5 deletions(-)
fs/erofs/decompressor.c +32 -4
@@ -15,6 +15,9 @@#endif #define LZ4_MAX_DISTANCE_PAGES (DIV_ROUND_UP(LZ4_DISTANCE_MAX, PAGE_SIZE) + 1) +#ifndef LZ4_DECOMPRESS_INPLACE_MARGIN +#define LZ4_DECOMPRESS_INPLACE_MARGIN (srcsize) (((srcsize) >> 8) + 32) +#endif struct z_erofs_decompressor {/*
@@ -117,7 +120,7 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) {unsigned int inputmargin, inlen;u8 *src; -bool copied; +bool copied, support_0padding;int ret;if (rq->inputsize >PAGE_SIZE )
@@ -125,13 +128,38 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) src =kmap_atomic (*rq->in); inputmargin =0 ; + support_0padding = false; + +/* decompression inplace is only safe when 0padding is enabled */ +if (EROFS_SB (rq->sb)->requirements &EROFS_REQUIREMENT_LZ4_0PADDING ) { + support_0padding = true; + +while (!src[inputmargin & ~PAGE_MASK ]) +if (!(++inputmargin & ~PAGE_MASK )) +break ; + +if (inputmargin >= rq->inputsize) { +kunmap_atomic (src); +return -EIO ; + } + } copied = false; inlen = rq->inputsize - inputmargin;if (rq->inplace_io) { - src =generic_copy_inplace_data (rq, src, inputmargin); - inputmargin =0 ; - copied = true; +const uint oend = (rq->pageofs_out + + rq->outputsize) & ~PAGE_MASK ; +const uint nr =PAGE_ALIGN (rq->pageofs_out + + rq->outputsize) >>PAGE_SHIFT ; + +if (rq->partial_decoding || !support_0padding || + rq->out[nr -1 ] != rq->in[0 ] || + rq->inputsize - oend < +LZ4_DECOMPRESS_INPLACE_MARGIN (inlen)) { + src =generic_copy_inplace_data (rq, src, inputmargin); + inputmargin =0 ; + copied = true; + } } ret = LZ4_decompress_safe_partial(src + inputmargin, out,
fs/erofs/erofs_fs.h +1 -1
@@ -17,7 +17,7 @@* incompatible with this kernel version. */ #define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001 -#define EROFS_ALL_REQUIREMENTS 0 +#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDINGstruct erofs_super_block {/* 0 */ __le32 magic;/* in the little endian */
Signature · 1 line
2.17.1
PATCH v6 22/24 erofs: introduce the decompression frontend
In reply to Gao XiangThis patch introduces the basic inplace fixed-sized output decompression implementation for erofs filesystem.
In constant to fixed-sized input compression, it has fixed-sized capacity for each compressed cluster to contain compressed data with the following advantages:
1) improved storage density;
2) decompression inplace support;
3) all data in a compressed physical cluster can be decompressed and utilized.
The key point of inplace refers to one of all erofs decompression strategies: Instead of allocating extra compressed pages and data management structures, it reuses the allocated file cache pages as much as possible to store its compressed data (called inplace I/O) and the corresponding pagevec in a time-sharing approach, which is particularly useful for low memory scenario.
In addition, decompression inplace technology is based on inplace I/O, which eliminates page allocation and all extra compressed data memcpy.
--- fs/erofs/Kconfig | 1 + fs/erofs/Makefile | 2 +- fs/erofs/internal.h | 14 +- fs/erofs/super.c | 11 + fs/erofs/zdata.c | 1254 +++++++++++++++++++++++++++++++++++++++++++ fs/erofs/zdata.h | 192 +++++++ fs/erofs/zmap.c | 4 +- 7 files changed, 1474 insertions(+), 4 deletions(-) create mode 100644 fs/erofs/zdata.c create mode 100644 fs/erofs/zdata.h
fs/erofs/Kconfig +1 -0
@@ -76,6 +76,7 @@ config EROFS_FS_ZIP bool "EROFS Data Compression Support" depends on EROFS_FS select LZ4_DECOMPRESS + default y help Enable fixed-sized output compression for EROFS.
fs/erofs/Makefile +1 -1
@@ -7,5 +7,5 @@ ccflags-y += -DEROFS_VERSION=\"$(EROFS_VERSION)\" obj-$(CONFIG_EROFS_FS) += erofs.o erofs-objs := super.o inode.o data.o namei.o dir.o utils.o erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o -erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o +erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o zdata.o
fs/erofs/internal.h +12 -2
@@ -69,6 +69,9 @@ struct erofs_sb_info {/* the dedicated workstation for compression */ struct radix_tree_root workstn_tree; +/* threshold for decompression synchronously */ +unsigned int max_sync_decompress_pages; +unsigned int shrinker_run_no;#endif /* CONFIG_EROFS_FS_ZIP */ u32 blocks;
@@ -328,6 +331,9 @@ static inline bool is_inode_flat_inline(struct inode *inode)extern const struct super_operations erofs_sops;extern const struct address_space_operations erofs_raw_access_aops; +#ifdef CONFIG_EROFS_FS_ZIP +extern const struct address_space_operations z_erofs_vle_normalaccess_aops; +#endif /* * Logical to physical block mapping, used by erofs_map_blocks()
@@ -488,7 +494,7 @@ int erofs_namei(struct inode *dir, struct qstr *name,/* dir.c */ extern const struct file_operations erofs_dir_fops; -/* utils.c */ +/* utils.c / zdata.c */ struct page *erofs_allocpage (struct list_head *pool,gfp_t gfp,bool nofail);#if (EROFS_PCPUBUF_NR_PAGES >0 )
@@ -512,16 +518,20 @@ struct erofs_workgroup *erofs_find_workgroup(struct super_block *sb, pgoff_t index, bool *tag);int erofs_register_workgroup (struct super_block *sb,struct erofs_workgroup *grp,bool tag); -static inline void erofs_workgroup_free_rcu (struct erofs_workgroup *grp) {} +void erofs_workgroup_free_rcu (struct erofs_workgroup *grp);void erofs_shrinker_register (struct super_block *sb);void erofs_shrinker_unregister (struct super_block *sb);int __init erofs_init_shrinker (void );void erofs_exit_shrinker (void ); +int __init z_erofs_init_zip_subsystem (void ); +void z_erofs_exit_zip_subsystem (void );#else static inline void erofs_shrinker_register (struct super_block *sb) {}static inline void erofs_shrinker_unregister (struct super_block *sb) {}static inline int erofs_init_shrinker (void ) {return 0 ; }static inline void erofs_exit_shrinker (void ) {} +static inline int z_erofs_init_zip_subsystem (void ) {return 0 ; } +static inline void z_erofs_exit_zip_subsystem (void ) {}#endif /* !CONFIG_EROFS_FS_ZIP */ #endif /* __EROFS_INTERNAL_H */
fs/erofs/super.c +11 -0
@@ -200,6 +200,9 @@ static unsigned int erofs_get_fault_rate(struct erofs_sb_info *sbi)/* set up default EROFS parameters */ static void default_options (struct erofs_sb_info *sbi) { +#ifdef CONFIG_EROFS_FS_ZIP + sbi->max_sync_decompress_pages =3 ; +#endif #ifdef CONFIG_EROFS_FS_XATTR set_opt (sbi,XATTR_USER );#endif
@@ -420,6 +423,11 @@ static int __init erofs_module_init(void) err =erofs_init_shrinker ();if (err) goto shrinker_err; + + err =z_erofs_init_zip_subsystem (); +if (err) + goto zip_err; + err =register_filesystem (&erofs_fs_type);if (err) goto fs_err;
@@ -428,6 +436,8 @@ static int __init erofs_module_init(void)return 0 ; fs_err: +z_erofs_exit_zip_subsystem (); +zip_err:erofs_exit_shrinker (); shrinker_err:erofs_exit_inode_cache ();
@@ -438,6 +448,7 @@ static int __init erofs_module_init(void) static void __exit erofs_module_exit(void) {unregister_filesystem (&erofs_fs_type); +z_erofs_exit_zip_subsystem ();erofs_exit_shrinker ();erofs_exit_inode_cache ();infoln ("successfully finalize erofs" );
fs/erofs/zdata.c new file +1254 -0
1,254 lines not on this page · raw message
fs/erofs/zdata.h new file +192 -0
192 lines not on this page · raw message
fs/erofs/zmap.c +3 -1
@@ -23,7 +23,9 @@ int z_erofs_fill_inode(struct inode *inode) vi->z_physical_clusterbits[1 ] = vi->z_logical_clusterbits;set_bit (EROFS_V_Z_INITED_BIT , &vi->flags); } -return -ENOTSUPP ; + + inode->i_mapping->a_ops = &z_erofs_vle_normalaccess_aops; +return 0 ; }static int fill_inode_lazy (struct inode *inode)
Signature · 1 line
2.17.1
PATCH v6 23/24 erofs: introduce cached decompression
In reply to Gao XiangThis patch adds strategies which can be selected by users in order to cache both incomplete ends of compressed physical clusters as a complement of in-place I/O in order to boost random read, but it costs more memory than the in-place I/O only.
--- fs/erofs/internal.h | 16 +++++ fs/erofs/super.c | 126 ++++++++++++++++++++++++++++++++- fs/erofs/utils.c | 40 ++++++++--- fs/erofs/zdata.c | 165 ++++++++++++++++++++++++++++++++++++++++++-- fs/erofs/zdata.h | 7 +- 5 files changed, 336 insertions(+), 18 deletions(-)
fs/erofs/internal.h +16 -0
@@ -73,6 +73,12 @@ struct erofs_sb_info {unsigned int max_sync_decompress_pages;unsigned int shrinker_run_no; + +/* current strategy of how to use managed cache */ +unsigned char cache_strategy; + +/* pseudo inode to manage cached pages */ +struct inode *managed_cache;#endif /* CONFIG_EROFS_FS_ZIP */ u32 blocks;u32 meta_blkaddr;
@@ -158,6 +164,12 @@ static inline void *erofs_kmalloc(struct erofs_sb_info *sbi,#define test_opt (sbi, option) ((sbi)->mount_opt & EROFS_MOUNT_##option)#ifdef CONFIG_EROFS_FS_ZIP +enum { +EROFS_ZIP_CACHE_DISABLED , +EROFS_ZIP_CACHE_READAHEAD , +EROFS_ZIP_CACHE_READAROUND +}; +#define EROFS_LOCKED_MAGIC (INT_MIN | 0xE0F510CCL)/* basic unit of the workstation of a super_block */
@@ -525,6 +537,10 @@ int __init erofs_init_shrinker(void);void erofs_exit_shrinker (void );int __init z_erofs_init_zip_subsystem (void );void z_erofs_exit_zip_subsystem (void ); +int erofs_try_to_free_all_cached_pages (struct erofs_sb_info *sbi, +struct erofs_workgroup *egrp); +int erofs_try_to_free_cached_page (struct address_space *mapping, +struct page *page);#else static inline void erofs_shrinker_register (struct super_block *sb) {}static inline void erofs_shrinker_unregister (struct super_block *sb) {}
fs/erofs/super.c +125 -1
185 lines not on this page · raw message
fs/erofs/utils.c +32 -8
@@ -144,24 +144,48 @@ int erofs_workgroup_put(struct erofs_workgroup *grp)return count; } -/* for nocache case, no customized reclaim path at all */ +static void erofs_workgroup_unfreeze_final (struct erofs_workgroup *grp) +{ +erofs_workgroup_unfreeze (grp,0 ); +__erofs_workgroup_free (grp); +} +static bool erofs_try_to_release_workgroup (struct erofs_sb_info *sbi,struct erofs_workgroup *grp,bool cleanup) { -int cnt =atomic_read (&grp->refcount); - -DBG_BUGON (cnt <=0 ); -DBG_BUGON (cleanup && cnt !=1 ); +/* +* If managed cache is on, refcount of workgroups +* themselves could be < 0 (freezed). In other words, +* there is no guarantee that all refcounts > 0. +*/ +if (!erofs_workgroup_try_to_freeze (grp,1 )) +return false; -if (cnt >1 ) +/* +* Note that all cached pages should be unattached +* before deleted from the radix tree. Otherwise some +* cached pages could be still attached to the orphan +* old workgroup when the new one is available in the tree. +*/ +if (erofs_try_to_free_all_cached_pages (sbi, grp)) { +erofs_workgroup_unfreeze (grp,1 );return false; + } +/* +* It's impossible to fail after the workgroup is freezed, +* however in order to avoid some race conditions, add a +* DBG_BUGON to observe this in advance. +*/ DBG_BUGON (xa_untag_pointer (radix_tree_delete (&sbi->workstn_tree, grp->index)) != grp); -/* (rarely) could be grabbed again when freeing */ -erofs_workgroup_put (grp); +/* +* If managed cache is on, last refcount should indicate +* the related workstation. +*/ +erofs_workgroup_unfreeze_final (grp);return true; }
fs/erofs/zdata.c +158 -7
213 lines not on this page · raw message
fs/erofs/zdata.h +5 -2
@@ -101,9 +101,12 @@ struct z_erofs_unzip_io_sb {struct super_block *sb; }; -#define MNGD_MAPPING (sbi) (NULL) +#define MNGD_MAPPING (sbi) ((sbi)->managed_cache->i_mapping)static inline bool erofs_page_is_managed (const struct erofs_sb_info *sbi, -struct page *page) {return false; } +struct page *page) +{ +return page->mapping ==MNGD_MAPPING (sbi); +}#define Z_EROFS_ONLINEPAGE_COUNT_BITS 2#define Z_EROFS_ONLINEPAGE_COUNT_MASK ((1 << Z_EROFS_ONLINEPAGE_COUNT_BITS) - 1)
Signature · 1 line
2.17.1
PATCH v6 24/24 erofs: add document
In reply to Gao XiangThis documents key features, usage, and
on-disk design of erofs.
--- Documentation/filesystems/erofs.txt | 225 ++++++++++++++++++++++++++++ 1 file changed, 225 insertions(+) create mode 100644 Documentation/filesystems/erofs.txt
Documentation/filesystems/erofs.txt new file +225 -0
@@ -0,0 +1,225 @@ +Overview +======== + +EROFS file-system stands for Enhanced Read-Only File System. Different +from other read-only file systems, it aims to be designed for flexibility, +scalability, but be kept simple and high performance. + +It is designed as a better filesystem solution for the following scenarios: + - read-only storage media or + + - part of a fully trusted read-only solution, which means it needs to be + immutable and bit-for-bit identical to the official golden image for + their releases due to security and other considerations and + + - hope to save some extra storage space with guaranteed end-to-end performance + by using reduced metadata and transparent file compression, especially + for those embedded devices with limited memory (ex, smartphone); + +Here is the main features of EROFS: + - Little endian on-disk design; + + - Currently 4KB block size (nobh) and therefore maximum 16TB address space; + + - Metadata & data could be mixed by design; + + - 2 inode versions for different requirements: + v1 v2 + Inode metadata size: 32 bytes 64 bytes + Max file size: 4 GB 16 EB (also limited by max. vol size) + Max uids/gids: 65536 4294967296 + File creation time: no yes (64 + 32-bit timestamp) + Max hardlinks: 65536 4294967296 + Metadata reserved: 4 bytes 14 bytes + + - Support extended attributes (xattrs) as an option; + + - Support xattr inline and tail-end data inline for all files; + + - Support POSIX.1e ACLs by using xattrs; + + - Support statx(); + + - Support transparent file compression as an option: + LZ4 algorithm with 4 KB fixed-output compression for high performance; + +The following git tree provides the file system user-space tools under +development (ex, formatting tool mkfs.erofs): +>> git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git + +Bugs and patches are welcome, please kindly help us and send to the following +linux-erofs mailing list: +>> linux-erofs mailing list <linux-erofs at lists.ozlabs.org> + +Note that EROFS is still working in progress as a Linux staging driver, +Cc the staging mailing list as well is highly recommended: +>> Linux Driver Project Developer List <devel at driverdev.osuosl.org> + +Mount options +============= + +fault_injection=%d Enable fault injection in all supported types with + specified injection rate. Supported injection type: + Type_Name Type_Value + FAULT_KMALLOC 0x000000001 + FAULT_READ_IO 0x000000002 +(no)user_xattr Setup Extended User Attributes. Note: xattr is enabled + by default if CONFIG_EROFS_FS_XATTR is selected. +(no)acl Setup POSIX Access Control List. Note: acl is enabled + by default if CONFIG_EROFS_FS_POSIX_ACL is selected. +cache_strategy=%s Select a strategy for cached decompression from now on: + disabled: In-place I/O decompression only; + readahead: Cache the last incomplete compressed physical + cluster for further reading. It still does + in-place I/O decompression for the rest + compressed physical clusters; + readaround: Cache the both ends of incomplete compressed + physical clusters for further reading. + It still does in-place I/O decompression + for the rest compressed physical clusters. + +Module parameters +================= +use_vmap=[0|1] Use vmap() instead of vm_map_ram() (default 0). + +On-disk details +=============== + +Summary +------- +Different from other read-only file systems, an EROFS volume is designed +to be as simple as possible: + + |-> aligned with the block size + ____________________________________________________________ + | |SB| | ... | Metadata | ... | Data | Metadata | ... | Data | + |_|__|_|_____|__________|_____|______|__________|_____|______| + 0 +1K + +All data areas should be aligned with the block size, but metadata areas +may not. All metadatas can be now observed in two different spaces (views): + 1. Inode metadata space + Each valid inode should be aligned with an inode slot, which is a fixed + value (32 bytes) and designed to be kept in line with v1 inode size. + + Each inode can be directly found with the following formula: + inode offset = meta_blkaddr * block_size + 32 * nid + + |-> aligned with 8B + |-> followed closely + + meta_blkaddr blocks |-> another slot + _____________________________________________________________________ + | ... | inode | xattrs | extents | data inline | ... | inode ... + |________|_______|(optional)|(optional)|__(optional)_|_____|__________ + |-> aligned with the inode slot size + . . + . . + . . + . . + . . + . . + .____________________________________________________|-> aligned with 4B + | xattr_ibody_header | shared xattrs | inline xattrs | + |____________________|_______________|_______________| + |-> 12 bytes <-|->x * 4 bytes<-| . + . . . + . . . + . . . + ._______________________________.______________________. + | id | id | id | id | ... | id | ent | ... | ent| ... | + |____|____|____|____|______|____|_____|_____|____|_____| + |-> aligned with 4B + |-> aligned with 4B + + Inode could be 32 or 64 bytes, which can be distinguished from a common + field which all inode versions have -- i_advise: + + __________________ __________________ + | i_advise | | i_advise | + |__________________| |__________________| + | ... | | ... | + | | | | + |__________________| 32 bytes | | + | | + |__________________| 64 bytes + + Xattrs, extents, data inline are followed by the corresponding inode with + proper alignes, and they could be optional for different data mappings, + _currently_ there are totally 3 valid data mappings supported: + + 1) flat file data without data inline (no extent); + 2) fixed-output size data compression (must have extents); + 3) flat file data with tail-end data inline (no extent); + + The size of the optional xattrs is indicated by i_xattr_count in inode + header. Large xattrs or xattrs shared by many different files can be + stored in shared xattrs metadata rather than inlined right after inode. + + 2. Shared xattrs metadata space + Shared xattrs space is similar to the above inode space, started with + a specific block indicated by xattr_blkaddr, organized one by one with + proper align. + + Each share xattr can also be directly found by the following formula: + xattr offset = xattr_blkaddr * block_size + 4 * xattr_id + + |-> aligned by 4 bytes + + xattr_blkaddr blocks |-> aligned with 4 bytes + _________________________________________________________________________ + | ... | xattr_entry | xattr data | ... | xattr_entry | xattr data ... + |________|_____________|_____________|_____|______________|_______________ + +Directories +----------- +All directories are now organized in a compact on-disk format. Note that +each directory block is divided into index and name areas in order to support +random file lookup, and all directory entries are _strictly_ recorded in +alphabetical order in order to support improved prefix binary search +algorithm (could refer to the related source code). + + ___________________________ + / | + / ______________|________________ + / / | nameoff1 | nameoffN-1 + ____________.______________._______________v________________v__________ +| dirent | dirent | ... | dirent | filename | filename | ... | filename | +|___.0___|____1___|_____|___N-1__|____0_____|____1_____|_____|___N-1____| + \ ^ + \ | * could have + \ | trailing '\0' + \________________________| nameoff0 + + Directory block + +Note that apart from the offset of the first filename, nameoff0 also indicates +the total number of directory entries in this block since it is no need to +introduce another on-disk field at all. + +Compression +----------- +Currently, EROFS supports 4KB fixed-output clustersize transparent file +compression, as illustrated below: + + |---- Variant-Length Extent ----|-------- VLE --------|----- VLE ----- + clusterofs clusterofs clusterofs + | | | logical data +_________v_______________________________v_____________________v_______________ +... | . | | . | | . | ... +____|____.________|_____________|________.____|_____________|__.__________|____ + |-> cluster <-|-> cluster <-|-> cluster <-|-> cluster <-|-> cluster <-| + size size size size size + . . . . + . . . . + . . . . + _______._____________._____________._____________._____________________ + ... | | | | ... physical data + _______|_____________|_____________|_____________|_____________________ + |-> cluster <-|-> cluster <-|-> cluster <-| + size size size + +Currently each on-disk physical cluster can contain 4KB (un)compressed data +at most. For each logical cluster, there is a corresponding on-disk index to +describe its cluster type, physical cluster address, etc. + +See "struct z_erofs_vle_decompressed_index" in erofs_fs.h for more details. +
Signature · 1 line
2.17.1
cover erofs: patchset addressing Christoph's comments
In reply to Gao XiangHi,
This patchset is based on the following patch by Pratik Shinde,
https://lore.kernel.org/linux-erofs/[email protected]/
All patches addressing Christoph's comments on v6, which are trivial, most deleted code are from erofs specific fault injection, which was followed f2fs and previously discussed in earlier topic [1], but let's follow what Christoph's said now.
Comments and suggestions are welcome...
[1] https://lore.kernel.org/r/[email protected]/
Thanks,
Gao Xiang
Gao Xiang (21):
erofs: remove all the byte offset comments
erofs: on-disk format should have explicitly assigned numbers
erofs: some macros are much more readable as a function
erofs: kill __packed for on-disk structures
erofs: update erofs_inode_is_data_compressed helper
erofs: kill erofs_{init,exit}_inode_cache
erofs: use erofs_inode naming
erofs: update comments in inode.c
erofs: update erofs symlink stuffs
erofs: kill is_inode_layout_compression()
erofs: use dsb instead of layout for ondisk super_block
erofs: kill verbose debug info in erofs_fill_super
erofs: simplify erofs_grab_bio() since bio_alloc() never fail
erofs: kill prio and nofail of erofs_get_meta_page()
erofs: kill __submit_bio()
erofs: kill magic underscores
erofs: use a switch statement when dealing with the file modes
erofs: add "erofs_" prefix for common and short functions
erofs: kill all erofs specific fault injection
erofs: kill use_vmap module parameter
erofs: save one level of indentation
Documentation/filesystems/erofs.txt | 9 -- fs/erofs/Kconfig | 7 -- fs/erofs/data.c | 62 +++------- fs/erofs/decompressor.c | 34 ++--- fs/erofs/dir.c | 6 +- fs/erofs/erofs_fs.h | 162 ++++++++++++------------ fs/erofs/inode.c | 176 +++++++++++++------------- fs/erofs/internal.h | 156 +++-------------------- fs/erofs/namei.c | 12 +- fs/erofs/super.c | 185 ++++++++-------------------- fs/erofs/xattr.c | 33 +++-- fs/erofs/xattr.h | 4 +- fs/erofs/zdata.c | 44 +++---- fs/erofs/zmap.c | 32 ++--- include/trace/events/erofs.h | 14 +-- 15 files changed, 338 insertions(+), 598 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/21 erofs: remove all the byte offset comments
From: Gao Xiang <[email protected]>
As Christoph suggested [1], "Please remove all the byte offset comments. that is something that can easily be checked with gdb or pahole."
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 105 +++++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 51 deletions(-)
fs/erofs/erofs_fs.h +54 -51
@@ -17,27 +17,28 @@#define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING +/* 128-byte erofs on-disk super block */ struct erofs_super_block { -/* 0 */ __le32 magic;/* in the little endian */ -/* 4 */ __le32 checksum;/* crc32c(super_block) */ -/* 8 */ __le32 features;/* (aka. feature_compat) */ -/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ -/* 13 */ __u8 reserved; - -/* 14 */ __le16 root_nid; -/* 16 */ __le64 inos;/* total valid ino # (== f_files - f_favail) */ - -/* 24 */ __le64 build_time;/* inode v1 time derivation */ -/* 32 */ __le32 build_time_nsec; -/* 36 */ __le32 blocks;/* used for statfs */ -/* 40 */ __le32 meta_blkaddr; -/* 44 */ __le32 xattr_blkaddr; -/* 48 */ __u8 uuid[16 ];/* 128-bit uuid for volume */ -/* 64 */ __u8 volume_name[16 ];/* volume name */ -/* 80 */ __le32 requirements;/* (aka. feature_incompat) */ - -/* 84 */ __u8 reserved2[44 ]; -} __packed;/* 128 bytes */ +__le32 magic;/* file system magic number */ +__le32 checksum;/* crc32c(super_block) */ +__le32 features;/* (aka. feature_compat) */ +__u8 blkszbits;/* support block_size == PAGE_SIZE only */ +__u8 reserved; + +__le16 root_nid;/* nid of root directory */ +__le64 inos;/* total valid ino # (== f_files - f_favail) */ + +__le64 build_time;/* inode v1 time derivation */ +__le32 build_time_nsec;/* inode v1 time derivation in nano scale */ +__le32 blocks;/* used for statfs */ +__le32 meta_blkaddr;/* start block address of metadata area */ +__le32 xattr_blkaddr;/* start block address of shared xattr area */ +__u8 uuid[16 ];/* 128-bit uuid for volume */ +__u8 volume_name[16 ];/* volume name */ +__le32 requirements;/* (aka. feature_incompat) */ + +__u8 reserved2[44 ]; +} __packed;/* * erofs inode data mapping:
@@ -73,16 +74,17 @@ static inline bool erofs_inode_is_data_compressed(unsigned int datamode)#define EROFS_I_VERSION_BIT 0#define EROFS_I_DATA_MAPPING_BIT 1 +/* 32-byte reduced form of an ondisk inode */ struct erofs_inode_v1 { -/* 0 */ __le16 i_advise; +__le16 i_advise;/* inode hints */ /* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ -/* 2 */ __le16 i_xattr_icount; -/* 4 */ __le16 i_mode; -/* 6 */ __le16 i_nlink; -/* 8 */ __le32 i_size; -/* 12 */ __le32 i_reserved; -/* 16 */ union { +__le16 i_xattr_icount; +__le16 i_mode; +__le16 i_nlink; +__le32 i_size; +__le32 i_reserved; +union {/* file total compressed blocks for data mapping 1 */ __le32 compressed_blocks;__le32 raw_blkaddr;
@@ -90,10 +92,10 @@ struct erofs_inode_v1 {/* for device files, used to indicate old/new device # */ __le32 rdev; }i_u __packed; -/* 20 */ __le32 i_ino;/* only used for 32-bit stat compatibility */ -/* 24 */ __le16 i_uid; -/* 26 */ __le16 i_gid; -/* 28 */ __le32 i_reserved2; +__le32 i_ino;/* only used for 32-bit stat compatibility */ +__le16 i_uid; +__le16 i_gid; +__le32 i_reserved2; } __packed;/* 32 bytes on-disk inode */
@@ -101,15 +103,16 @@ struct erofs_inode_v1 {/* 64 bytes on-disk inode */ #define EROFS_INODE_LAYOUT_V2 1 +/* 64-byte complete form of an ondisk inode */ struct erofs_inode_v2 { -/* 0 */ __le16 i_advise; +__le16 i_advise;/* inode hints */ /* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ -/* 2 */ __le16 i_xattr_icount; -/* 4 */ __le16 i_mode; -/* 6 */ __le16 i_reserved; -/* 8 */ __le64 i_size; -/* 16 */ union { +__le16 i_xattr_icount; +__le16 i_mode; +__le16 i_reserved; +__le64 i_size; +union {/* file total compressed blocks for data mapping 1 */ __le32 compressed_blocks;__le32 raw_blkaddr;
@@ -119,15 +122,15 @@ struct erofs_inode_v2 { }i_u __packed;/* only used for 32-bit stat compatibility */ -/* 20 */ __le32 i_ino; - -/* 24 */ __le32 i_uid; -/* 28 */ __le32 i_gid; -/* 32 */ __le64 i_ctime; -/* 40 */ __le32 i_ctime_nsec; -/* 44 */ __le32 i_nlink; -/* 48 */ __u8 i_reserved2[16 ]; -} __packed;/* 64 bytes */ +__le32 i_ino; + +__le32 i_uid; +__le32 i_gid; +__le64 i_ctime; +__le32 i_ctime_nsec; +__le32 i_nlink; +__u8 i_reserved2[16 ]; +} __packed;#define EROFS_MAX_SHARED_XATTRS (128)/* h_shared_count between 129 ... 255 are special # */
@@ -264,7 +267,7 @@ struct z_erofs_vle_decompressed_index {* [1] - pointing to the tail cluster */ __le16 delta[2 ]; - }di_u __packed;/* 8 bytes */ + }di_u __packed; } __packed;#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN (size) \
@@ -273,10 +276,10 @@ struct z_erofs_vle_decompressed_index {/* dirent sorts in alphabet order, thus we can do binary search */ struct erofs_dirent { -__le64 nid;/* 0, node number */ -__le16 nameoff;/* 8, start offset of file name */ -__u8 file_type;/* 10, file type */ -__u8 reserved;/* 11, reserved */ +__le64 nid;/* node number */ +__le16 nameoff;/* start offset of file name */ +__u8 file_type;/* file type */ +__u8 reserved;/* reserved */ } __packed;/*
Signature · 1 line
2.17.1
Re: [PATCH 01/21] erofs: remove all the byte offset comments
Quoted 4 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph suggested [1], "Please remove all the byte offset comments. that is something that can easily be checked with gdb or pahole."
Looks good. If you want to keep them after the field names as someone pointed out feel free to - I don't think it actually is very useful but we've also heard other opinions.
PATCH 2/21 erofs: on-disk format should have explicitly assigned numbers
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph claimed [1], on-disk format should have explicitly assigned numbers. I have to change it.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
fs/erofs/erofs_fs.h +9 -9
@@ -53,10 +53,10 @@ struct erofs_super_block {* 4~7 - reserved */ enum { -EROFS_INODE_FLAT_PLAIN , -EROFS_INODE_FLAT_COMPRESSION_LEGACY , -EROFS_INODE_FLAT_INLINE , -EROFS_INODE_FLAT_COMPRESSION , +EROFS_INODE_FLAT_PLAIN =0 , +EROFS_INODE_FLAT_COMPRESSION_LEGACY =1 , +EROFS_INODE_FLAT_INLINE =2 , +EROFS_INODE_FLAT_COMPRESSION =3 ,EROFS_INODE_LAYOUT_MAX };
@@ -184,7 +184,7 @@ struct erofs_xattr_entry {/* available compression algorithm types */ enum { -Z_EROFS_COMPRESSION_LZ4 , +Z_EROFS_COMPRESSION_LZ4 =0 ,Z_EROFS_COMPRESSION_MAX };
@@ -242,10 +242,10 @@ struct z_erofs_map_header {* (di_advise could be 0, 1 or 2) */ enum { -Z_EROFS_VLE_CLUSTER_TYPE_PLAIN , -Z_EROFS_VLE_CLUSTER_TYPE_HEAD , -Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD , -Z_EROFS_VLE_CLUSTER_TYPE_RESERVED , +Z_EROFS_VLE_CLUSTER_TYPE_PLAIN =0 , +Z_EROFS_VLE_CLUSTER_TYPE_HEAD =1 , +Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD =2 , +Z_EROFS_VLE_CLUSTER_TYPE_RESERVED =3 ,Z_EROFS_VLE_CLUSTER_TYPE_MAX };
Signature · 1 line
2.17.1
Re: [PATCH 02/21] erofs: on-disk format should have explicitly assigned numbers
Quoted 13 lines · Gao Xiang
enum {
fs/erofs/erofs_fs.h +4
-4-EROFS_INODE_FLAT_PLAIN , -EROFS_INODE_FLAT_COMPRESSION_LEGACY , -EROFS_INODE_FLAT_INLINE , -EROFS_INODE_FLAT_COMPRESSION , +EROFS_INODE_FLAT_PLAIN =0 , +EROFS_INODE_FLAT_COMPRESSION_LEGACY =1 , +EROFS_INODE_FLAT_INLINE =2 , +EROFS_INODE_FLAT_COMPRESSION =3 , EROFS_INODE_LAYOUT_MAX };
fs/erofs/erofs_fs.h struct erofs_xattr_entry { · line 189
Quoted 7 lines
fs/erofs/erofs_fs.h +1
-1@@ -184,7 +184,7 @@ struct erofs_xattr_entry {/* available compression algorithm types */ enum { -Z_EROFS_COMPRESSION_LZ4 , +Z_EROFS_COMPRESSION_LZ4 =0 ,Z_EROFS_COMPRESSION_MAX };
This all looks ok - it somtimes also helps to have a comment near the numbers to indicate where they are stored, must that isn't a must.
PATCH 3/21 erofs: some macros are much more readable as a function
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph suggested [1], these macros are much
more readable as a function.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 22 ++++++++++++++-------- fs/erofs/inode.c | 4 ++-- fs/erofs/xattr.c | 2 +- 3 files changed, 17 insertions(+), 11 deletions(-)
fs/erofs/erofs_fs.h +14 -8
@@ -171,16 +171,22 @@ struct erofs_xattr_entry {char e_name[0 ];/* attribute name */ } __packed; -#define ondisk_xattr_ibody_size (count) ({\ - u32 __count = le16_to_cpu(count); \ - ((__count) == 0) ? 0 : \ - sizeof(struct erofs_xattr_ibody_header) + \ - sizeof(__u32) * ((__count) - 1); }) +static inline unsigned int erofs_xattr_ibody_size (__le16 i_xattr_icount) +{ +if (!i_xattr_icount) +return 0 ; + +return sizeof (struct erofs_xattr_ibody_header ) + +sizeof (__u32) * (le16_to_cpu (i_xattr_icount) -1 ); +}#define EROFS_XATTR_ALIGN (size) round_up(size, sizeof(struct erofs_xattr_entry)) -#define EROFS_XATTR_ENTRY_SIZE (entry) EROFS_XATTR_ALIGN( \ - sizeof(struct erofs_xattr_entry) + \ - (entry)->e_name_len + le16_to_cpu((entry)->e_value_size)) + +static inline unsigned int erofs_xattr_entry_size (struct erofs_xattr_entry *e) +{ +return EROFS_XATTR_ALIGN (sizeof (struct erofs_xattr_entry ) + + e->e_name_len +le16_to_cpu (e->e_value_size)); +}/* available compression algorithm types */ enum {
fs/erofs/inode.c +2 -2
@@ -29,7 +29,7 @@ static int read_inode(struct inode *inode, void *data)struct erofs_inode_v2 *v2 = data; vi->inode_isize =sizeof (struct erofs_inode_v2 ); - vi->xattr_isize =ondisk_xattr_ibody_size (v2->i_xattr_icount); + vi->xattr_isize =erofs_xattr_ibody_size (v2->i_xattr_icount); inode->i_mode =le16_to_cpu (v2->i_mode); if (S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
@@ -62,7 +62,7 @@ static int read_inode(struct inode *inode, void *data)struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); vi->inode_isize =sizeof (struct erofs_inode_v1 ); - vi->xattr_isize =ondisk_xattr_ibody_size (v1->i_xattr_icount); + vi->xattr_isize =erofs_xattr_ibody_size (v1->i_xattr_icount); inode->i_mode =le16_to_cpu (v1->i_mode); if (S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
fs/erofs/xattr.c +1 -1
@@ -231,7 +231,7 @@ static int xattr_foreach(struct xattr_iter *it,*/ entry = *(struct erofs_xattr_entry *)(it->kaddr + it->ofs);if (tlimit) { -unsigned int entry_sz =EROFS_XATTR_ENTRY_SIZE (&entry); +unsigned int entry_sz =erofs_xattr_entry_size (&entry);/* xattr on-disk corruption: xattr entry beyond xattr_isize */ if (*tlimit < entry_sz) {
Signature · 1 line
2.17.1
Re: [PATCH 03/21] erofs: some macros are much more readable as a function
This looks much better now.
PATCH 4/21 erofs: kill __packed for on-disk structures
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph suggested "Please don't add __packed" [1], remove all __packed except struct erofs_dirent here.
Note that all on-disk fields except struct erofs_dirent (12 bytes with a 8-byte nid) in EROFS are naturally aligned.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
fs/erofs/erofs_fs.h +9 -9
@@ -38,7 +38,7 @@ struct erofs_super_block {__le32 requirements;/* (aka. feature_incompat) */ __u8 reserved2[44 ]; -} __packed; +};/* * erofs inode data mapping:
@@ -91,12 +91,12 @@ struct erofs_inode_v1 {/* for device files, used to indicate old/new device # */ __le32 rdev; - }i_u __packed; + } i_u;__le32 i_ino;/* only used for 32-bit stat compatibility */ __le16 i_uid;__le16 i_gid;__le32 i_reserved2; -} __packed; +};/* 32 bytes on-disk inode */ #define EROFS_INODE_LAYOUT_V1 0
@@ -119,7 +119,7 @@ struct erofs_inode_v2 {/* for device files, used to indicate old/new device # */ __le32 rdev; - }i_u __packed; + } i_u;/* only used for 32-bit stat compatibility */ __le32 i_ino;
@@ -130,7 +130,7 @@ struct erofs_inode_v2 {__le32 i_ctime_nsec;__le32 i_nlink;__u8 i_reserved2[16 ]; -} __packed; +};#define EROFS_MAX_SHARED_XATTRS (128)/* h_shared_count between 129 ... 255 are special # */
@@ -152,7 +152,7 @@ struct erofs_xattr_ibody_header {__u8 h_shared_count;__u8 h_reserved2[7 ];__le32 h_shared_xattrs[0 ];/* shared xattr id array */ -} __packed; +};/* Name indexes */ #define EROFS_XATTR_INDEX_USER 1
@@ -169,7 +169,7 @@ struct erofs_xattr_entry {__le16 e_value_size;/* size of attribute value */ /* followed by e_name and e_value */ char e_name[0 ];/* attribute name */ -} __packed; +};static inline unsigned int erofs_xattr_ibody_size (__le16 i_xattr_icount) {
@@ -273,8 +273,8 @@ struct z_erofs_vle_decompressed_index {* [1] - pointing to the tail cluster */ __le16 delta[2 ]; - }di_u __packed; -} __packed; + } di_u; +};#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN (size) \ (round_up(size, sizeof(struct z_erofs_vle_decompressed_index)) + \
Signature · 1 line
2.17.1
Re: [PATCH 04/21] erofs: kill __packed for on-disk structures
Quoted 7 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph suggested "Please don't add
__packed" [1], remove all__packedexcept structerofs_direnthere.Note that all on-disk fields except struct
erofs_dirent(12 bytes with a 8-byte nid) in EROFS are naturally aligned.
Thanks. The users of various architectures where this generates a lot better code will thank you.
PATCH 5/21 erofs: update erofs_inode_is_data_compressed helper
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph said, "This looks like a really obsfucated way to write:
return datamode == EROFS_INODE_FLAT_COMPRESSION || datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY; "
Although I had my own consideration, it's the right way for now.
[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
--- fs/erofs/erofs_fs.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
fs/erofs/erofs_fs.h +2 -3
@@ -62,9 +62,8 @@ enum {static inline bool erofs_inode_is_data_compressed (unsigned int datamode) { -if (datamode ==EROFS_INODE_FLAT_COMPRESSION ) -return true; -return datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; +return datamode ==EROFS_INODE_FLAT_COMPRESSION || + datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; }/* bit definitions of inode i_advise */
Signature · 1 line
2.17.1
Re: [PATCH 05/21] erofs: update erofs_inode_is_data_compressed helper
Quoted 8 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph said, "This looks like a really obsfucated way to write:
return datamode == EROFS_INODE_FLAT_COMPRESSION || datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY; "Although I had my own consideration, it's the right way for now.
Well, if you do check one field for two values it really helps to do the same style of check for both. All your choice how you do the check, but don't mix multiple styles..
So this looks good.
PATCH 6/21 erofs: kill erofs_{init,exit}_inode_cache
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph said [1] "having this function seems
entirely pointless", let's kill those.
filesystem function name
ext2,f2fs,ext4,isofs,squashfs,cifs,... init_inodecache
In addition, add a necessary "rcu_barrier()" on exit_fs();
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/super.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-)
fs/erofs/super.c +12 -19
@@ -23,21 +23,6 @@ static void init_once(void *ptr)inode_init_once (&vi->vfs_inode); } -static int __init erofs_init_inode_cache (void ) -{ - erofs_inode_cachep =kmem_cache_create ("erofs_inode" , -sizeof (struct erofs_vnode ),0 , -SLAB_RECLAIM_ACCOUNT , - init_once); - -return erofs_inode_cachep ?0 : -ENOMEM ; -} - -static void erofs_exit_inode_cache (void ) -{ -kmem_cache_destroy (erofs_inode_cachep); -} -static struct inode *alloc_inode (struct super_block *sb) { struct erofs_vnode *vi =
@@ -531,9 +516,14 @@ static int __init erofs_module_init(void)erofs_check_ondisk_layout_definitions ();infoln ("initializing erofs " EROFS_VERSION ); - err =erofs_init_inode_cache (); -if (err) + erofs_inode_cachep =kmem_cache_create ("erofs_inode" , +sizeof (struct erofs_vnode ),0 , +SLAB_RECLAIM_ACCOUNT , + init_once); +if (!erofs_inode_cachep) { + err = -ENOMEM ; goto icache_err; + } err =erofs_init_shrinker ();if (err)
@@ -555,7 +545,7 @@ static int __init erofs_module_init(void) zip_err:erofs_exit_shrinker (); shrinker_err: -erofs_exit_inode_cache (); +kmem_cache_destroy (erofs_inode_cachep); icache_err:return err; }
@@ -565,7 +555,10 @@ static void __exit erofs_module_exit(void)unregister_filesystem (&erofs_fs_type);z_erofs_exit_zip_subsystem ();erofs_exit_shrinker (); -erofs_exit_inode_cache (); + +/* Ensure all RCU free inodes are safe before cache is destroyed. */ +rcu_barrier (); +kmem_cache_destroy (erofs_inode_cachep);infoln ("successfully finalize erofs" ); }
Signature · 1 line
2.17.1
Re: [PATCH 06/21] erofs: kill erofs_{init,exit}_inode_cache
Quoted 4 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph said [1] "having this function seems
entirely pointless", let's kill those.
Looks much better.
PATCH 7/21 erofs: use erofs_inode naming
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph suggested [1], "Why is this called vnode instead of inode? That seems like a rather odd naming for a Linux file system."
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/data.c | 4 ++-- fs/erofs/dir.c | 6 +++--- fs/erofs/inode.c | 10 +++++----- fs/erofs/internal.h | 18 +++++++++--------- fs/erofs/namei.c | 2 +- fs/erofs/super.c | 12 +++++------- fs/erofs/xattr.c | 18 +++++++++--------- fs/erofs/xattr.h | 4 ++-- fs/erofs/zdata.c | 9 +++------ fs/erofs/zmap.c | 28 ++++++++++++++-------------- include/trace/events/erofs.h | 14 +++++++------- 11 files changed, 60 insertions(+), 65 deletions(-)
fs/erofs/data.c +2 -2
@@ -112,7 +112,7 @@ static int erofs_map_blocks_flatmode(struct inode *inode,int err =0 ;erofs_blk_t nblocks, lastblk;u64 offset = map->m_la; -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);trace_erofs_map_blocks_flatmode_enter (inode, map, flags);
@@ -364,7 +364,7 @@ static int erofs_raw_access_readpages(struct file *filp,if (IS_ERR (bio)) {pr_err ("%s, readahead error at page %lu of nid %llu\n" , __func__, page->index, -EROFS_V (mapping->host)->nid); +EROFS_I (mapping->host)->nid); bio =NULL ; }
fs/erofs/dir.c +3 -3
@@ -47,7 +47,7 @@ static int erofs_fill_dentries(struct inode *dir, struct dir_context *ctx,/* a corrupted entry is found */ if (nameoff + de_namelen > maxsize || de_namelen >EROFS_NAME_LEN ) { -errln ("bogus dirent @ nid %llu" ,EROFS_V (dir)->nid); +errln ("bogus dirent @ nid %llu" ,EROFS_I (dir)->nid);DBG_BUGON (1 );return -EFSCORRUPTED ; }
@@ -85,7 +85,7 @@ static int erofs_readdir(struct file *f, struct dir_context *ctx)break ; }else if (IS_ERR (dentry_page)) {errln ("fail to readdir of logical block %u of nid %llu" , - i,EROFS_V (dir)->nid); + i,EROFS_I (dir)->nid); err = -EFSCORRUPTED ;break ; }
@@ -97,7 +97,7 @@ static int erofs_readdir(struct file *f, struct dir_context *ctx)if (nameoff <sizeof (struct erofs_dirent ) || nameoff >=PAGE_SIZE ) {errln ("%s, invalid de[0].nameoff %u @ nid %llu" , - __func__, nameoff,EROFS_V (dir)->nid); + __func__, nameoff,EROFS_I (dir)->nid); err = -EFSCORRUPTED ; goto skip_this; }
fs/erofs/inode.c +5 -5
@@ -11,7 +11,7 @@/* no locking */ static int read_inode (struct inode *inode,void *data) { -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);struct erofs_inode_v1 *v1 = data;const unsigned int advise =le16_to_cpu (v1->i_advise);erofs_blk_t nblks =0 ;
@@ -122,7 +122,7 @@ static int read_inode(struct inode *inode, void *data)static int fill_inline_data (struct inode *inode,void *data,unsigned int m_pofs) { -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);struct erofs_sb_info *sbi =EROFS_I_SB (inode);/* should be inode inline C */
@@ -160,7 +160,7 @@ static int fill_inline_data(struct inode *inode, void *data,static int fill_inode (struct inode *inode,int isdir) {struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);struct page *page;void *data;int err;
@@ -242,7 +242,7 @@ static int erofs_ilookup_test_actor(struct inode *inode, void *opaque) {const erofs_nid_t nid = *(erofs_nid_t *)opaque; -return EROFS_V (inode)->nid == nid; +return EROFS_I (inode)->nid == nid; }static int erofs_iget_set_actor (struct inode *inode,void *opaque)
@@ -279,7 +279,7 @@ struct inode *erofs_iget(struct super_block *sb,if (inode->i_state &I_NEW ) {int err; -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode); vi->nid = nid;
fs/erofs/internal.h +9 -9
@@ -272,14 +272,14 @@ static inline erofs_off_t iloc(struct erofs_sb_info *sbi, erofs_nid_t nid) }/* atomic flag definitions */ -#define EROFS_V_EA_INITED_BIT 0 -#define EROFS_V_Z_INITED_BIT 1 +#define EROFS_I_EA_INITED_BIT 0 +#define EROFS_I_Z_INITED_BIT 1/* bitlock definitions (arranged in reverse order) */ -#define EROFS_V_BL_XATTR_BIT (BITS_PER_LONG - 1) -#define EROFS_V_BL_Z_BIT (BITS_PER_LONG - 2) +#define EROFS_I_BL_XATTR_BIT (BITS_PER_LONG - 1) +#define EROFS_I_BL_Z_BIT (BITS_PER_LONG - 2) -struct erofs_vnode { +struct erofs_inode {erofs_nid_t nid;/* atomic flags (including bitlocks) */
@@ -307,8 +307,8 @@ struct erofs_vnode {struct inode vfs_inode; }; -#define EROFS_V (ptr) \ - container_of(ptr, struct erofs_vnode, vfs_inode) +#define EROFS_I (ptr) \ + container_of(ptr, struct erofs_inode, vfs_inode)#define __inode_advise (x, bit, bits) \ (((x) >> (bit)) & ((1 << (bits)) - 1))
@@ -329,12 +329,12 @@ static inline unsigned long inode_datablocks(struct inode *inode)static inline bool is_inode_layout_compression (struct inode *inode) { -return erofs_inode_is_data_compressed (EROFS_V (inode)->datamode); +return erofs_inode_is_data_compressed (EROFS_I (inode)->datamode); }static inline bool is_inode_flat_inline (struct inode *inode) { -return EROFS_V (inode)->datamode ==EROFS_INODE_FLAT_INLINE ; +return EROFS_I (inode)->datamode ==EROFS_INODE_FLAT_INLINE ; }extern const struct super_operations erofs_sops;
fs/erofs/namei.c +1 -1
@@ -117,7 +117,7 @@ static struct page *find_target_block_classic(struct inode *dir,kunmap_atomic (de);put_page (page);errln ("corrupted dir block %d @ nid %llu" , - mid,EROFS_V (dir)->nid); + mid,EROFS_I (dir)->nid);DBG_BUGON (1 ); page =ERR_PTR (-EFSCORRUPTED ); goto out;
fs/erofs/super.c +5 -7
@@ -18,27 +18,25 @@ static struct kmem_cache *erofs_inode_cachep __read_mostly;static void init_once (void *ptr) { -struct erofs_vnode *vi = ptr; - -inode_init_once (&vi->vfs_inode); +inode_init_once (&((struct erofs_inode *)ptr)->vfs_inode); }static struct inode *alloc_inode (struct super_block *sb) { -struct erofs_vnode *vi = +struct erofs_inode *vi =kmem_cache_alloc (erofs_inode_cachep,GFP_KERNEL );if (!vi)return NULL ;/* zero out everything except vfs_inode */ -memset (vi,0 , offsetof(struct erofs_vnode , vfs_inode)); +memset (vi,0 , offsetof(struct erofs_inode , vfs_inode));return &vi->vfs_inode; }static void free_inode (struct inode *inode) { -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ if (is_inode_fast_symlink (inode))
@@ -517,7 +515,7 @@ static int __init erofs_module_init(void)infoln ("initializing erofs " EROFS_VERSION ); erofs_inode_cachep =kmem_cache_create ("erofs_inode" , -sizeof (struct erofs_vnode ),0 , +sizeof (struct erofs_inode ),0 ,SLAB_RECLAIM_ACCOUNT , init_once);if (!erofs_inode_cachep) {
fs/erofs/xattr.c +9 -9
@@ -38,7 +38,7 @@ static inline void xattr_iter_end_final(struct xattr_iter *it)static int init_inode_xattrs (struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct xattr_iter it;unsigned int i;struct erofs_xattr_ibody_header *ih;
@@ -48,14 +48,14 @@ static int init_inode_xattrs(struct inode *inode)int ret =0 ;/* the most case is that xattrs of this inode are initialized. */ -if (test_bit (EROFS_V_EA_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_EA_INITED_BIT , &vi->flags))return 0 ; -if (wait_on_bit_lock (&vi->flags,EROFS_V_BL_XATTR_BIT ,TASK_KILLABLE )) +if (wait_on_bit_lock (&vi->flags,EROFS_I_BL_XATTR_BIT ,TASK_KILLABLE ))return -ERESTARTSYS ;/* someone has initialized xattrs for us? */ -if (test_bit (EROFS_V_EA_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_EA_INITED_BIT , &vi->flags)) goto out_unlock;/*
@@ -136,10 +136,10 @@ static int init_inode_xattrs(struct inode *inode) }xattr_iter_end (&it, atomic_map); -set_bit (EROFS_V_EA_INITED_BIT , &vi->flags); +set_bit (EROFS_I_EA_INITED_BIT , &vi->flags); out_unlock: -clear_and_wake_up_bit (EROFS_V_BL_XATTR_BIT , &vi->flags); +clear_and_wake_up_bit (EROFS_I_BL_XATTR_BIT , &vi->flags);return ret; }
@@ -184,7 +184,7 @@ static inline int xattr_iter_fixup(struct xattr_iter *it)static int inline_xattr_iter_begin (struct xattr_iter *it,struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct erofs_sb_info *const sbi =EROFS_SB (inode->i_sb);unsigned int xattr_header_sz, inline_xattr_ofs;
@@ -385,7 +385,7 @@ static int inline_getxattr(struct inode *inode, struct getxattr_iter *it)static int shared_getxattr (struct inode *inode,struct getxattr_iter *it) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct super_block *const sb = inode->i_sb;struct erofs_sb_info *const sbi =EROFS_SB (sb);unsigned int i;
@@ -608,7 +608,7 @@ static int inline_listxattr(struct listxattr_iter *it)static int shared_listxattr (struct listxattr_iter *it) {struct inode *const inode =d_inode (it->dentry); -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct super_block *const sb = inode->i_sb;struct erofs_sb_info *const sbi =EROFS_SB (sb);unsigned int i;
fs/erofs/xattr.h +2 -2
@@ -16,8 +16,8 @@static inline unsigned int inlinexattr_header_size (struct inode *inode) { -return sizeof (struct erofs_xattr_ibody_header ) - +sizeof (u32) *EROFS_V (inode)->xattr_shared_count; +return sizeof (struct erofs_xattr_ibody_header ) + +sizeof (u32) *EROFS_I (inode)->xattr_shared_count; } static inline erofs_blk_t xattrblock_addr(struct erofs_sb_info *sbi,
fs/erofs/zdata.c +3 -6
@@ -421,7 +421,7 @@ static struct z_erofs_collection *clregister(struct z_erofs_collector *clt,else pcl->algorithmformat =Z_EROFS_COMPRESSION_SHIFTED ; - pcl->clusterbits =EROFS_V (inode)->z_physical_clusterbits[0 ]; + pcl->clusterbits =EROFS_I (inode)->z_physical_clusterbits[0 ]; pcl->clusterbits -=PAGE_SHIFT ;/* new pclusters should be claimed as type 1, primary and followed */
@@ -1404,12 +1404,9 @@ static int z_erofs_vle_normalaccess_readpages(struct file *filp, head = (void *)page_private (page); err =z_erofs_do_read_page (&f, page, &pagepool); -if (err) { -struct erofs_vnode *vi =EROFS_V (inode); - +if (err)errln ("%s, readahead error at page %lu of nid %llu" , - __func__, page->index, vi->nid); - } + __func__, page->index,EROFS_I (inode)->nid);put_page (page); }
fs/erofs/zmap.c +14 -14
@@ -10,7 +10,7 @@int z_erofs_fill_inode (struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);if (vi->datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ) { vi->z_advise =0 ;
@@ -19,7 +19,7 @@ int z_erofs_fill_inode(struct inode *inode) vi->z_logical_clusterbits =LOG_BLOCK_SIZE ; vi->z_physical_clusterbits[0 ] = vi->z_logical_clusterbits; vi->z_physical_clusterbits[1 ] = vi->z_logical_clusterbits; -set_bit (EROFS_V_Z_INITED_BIT , &vi->flags); +set_bit (EROFS_I_Z_INITED_BIT , &vi->flags); } inode->i_mapping->a_ops = &z_erofs_vle_normalaccess_aops;
@@ -28,7 +28,7 @@ int z_erofs_fill_inode(struct inode *inode)static int fill_inode_lazy (struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct super_block *const sb = inode->i_sb;int err;erofs_off_t pos;
@@ -36,14 +36,14 @@ static int fill_inode_lazy(struct inode *inode)void *kaddr;struct z_erofs_map_header *h; -if (test_bit (EROFS_V_Z_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_Z_INITED_BIT , &vi->flags))return 0 ; -if (wait_on_bit_lock (&vi->flags,EROFS_V_BL_Z_BIT ,TASK_KILLABLE )) +if (wait_on_bit_lock (&vi->flags,EROFS_I_BL_Z_BIT ,TASK_KILLABLE ))return -ERESTARTSYS ; err =0 ; -if (test_bit (EROFS_V_Z_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_Z_INITED_BIT , &vi->flags)) goto out_unlock;DBG_BUGON (vi->datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY );
@@ -83,13 +83,13 @@ static int fill_inode_lazy(struct inode *inode) vi->z_physical_clusterbits[1 ] = vi->z_logical_clusterbits + ((h->h_clusterbits >>5 ) &7 ); -set_bit (EROFS_V_Z_INITED_BIT , &vi->flags); +set_bit (EROFS_I_Z_INITED_BIT , &vi->flags); unmap_done:kunmap_atomic (kaddr);unlock_page (page);put_page (page); out_unlock: -clear_and_wake_up_bit (EROFS_V_BL_Z_BIT , &vi->flags); +clear_and_wake_up_bit (EROFS_I_BL_Z_BIT , &vi->flags);return err; }
@@ -142,7 +142,7 @@ static int vle_legacy_load_cluster_from_disk(struct z_erofs_maprecorder *m, unsigned long lcn) {struct inode *const inode = m->inode; -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);const erofs_off_t ibase =iloc (EROFS_I_SB (inode), vi->nid);const erofs_off_t pos = Z_EROFS_VLE_LEGACY_INDEX_ALIGN(ibase + vi->inode_isize +
@@ -196,7 +196,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m, unsigned int amortizedshift, unsigned int eofs) { -struct erofs_vnode *const vi =EROFS_V (m->inode); +struct erofs_inode *const vi =EROFS_I (m->inode);const unsigned int lclusterbits = vi->z_logical_clusterbits;const unsigned int lomask = (1 << lclusterbits) -1 ;unsigned int vcnt, base, lo, encodebits, nblk;
@@ -260,7 +260,7 @@ static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m, unsigned long lcn) {struct inode *const inode = m->inode; -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);const unsigned int lclusterbits = vi->z_logical_clusterbits;const erofs_off_t ebase =ALIGN (iloc (EROFS_I_SB (inode), vi->nid) + vi->inode_isize + vi->xattr_isize, 8) +
@@ -314,7 +314,7 @@ static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,static int vle_load_cluster_from_disk (struct z_erofs_maprecorder *m,unsigned int lcn) { -const unsigned int datamode =EROFS_V (m->inode)->datamode; +const unsigned int datamode =EROFS_I (m->inode)->datamode;if (datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY )return vle_legacy_load_cluster_from_disk (m, lcn);
@@ -328,7 +328,7 @@ static int vle_load_cluster_from_disk(struct z_erofs_maprecorder *m,static int vle_extent_lookback (struct z_erofs_maprecorder *m,unsigned int lookback_distance) { -struct erofs_vnode *const vi =EROFS_V (m->inode); +struct erofs_inode *const vi =EROFS_I (m->inode);struct erofs_map_blocks *const map = m->map;const unsigned int lclusterbits = vi->z_logical_clusterbits;unsigned long lcn = m->lcn;
@@ -374,7 +374,7 @@ int z_erofs_map_blocks_iter(struct inode *inode, struct erofs_map_blocks *map, int flags) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct z_erofs_maprecorder m = { .inode = inode, .map = map,
include/trace/events/erofs.h +7 -7
@@ -41,7 +41,7 @@ TRACE_EVENT(erofs_lookup,TP_fast_assign ( __entry->dev = dir->i_sb->s_dev; - __entry->nid =EROFS_V (dir)->nid; + __entry->nid =EROFS_I (dir)->nid; __entry->name = dentry->d_name.name; __entry->flags = flags; ),
@@ -66,7 +66,7 @@ TRACE_EVENT(erofs_fill_inode,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->blkaddr =erofs_blknr (iloc (EROFS_I_SB (inode), __entry->nid)); __entry->ofs =erofs_blkoff (iloc (EROFS_I_SB (inode), __entry->nid)); __entry->isdir = isdir;
@@ -95,7 +95,7 @@ TRACE_EVENT(erofs_readpage,TP_fast_assign ( __entry->dev = page->mapping->host->i_sb->s_dev; - __entry->nid =EROFS_V (page->mapping->host)->nid; + __entry->nid =EROFS_I (page->mapping->host)->nid; __entry->dir =S_ISDIR (page->mapping->host->i_mode); __entry->index = page->index; __entry->uptodate =PageUptodate (page);
@@ -128,7 +128,7 @@ TRACE_EVENT(erofs_readpages,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->start = page->index; __entry->nrpage = nrpage; __entry->raw = raw;
@@ -157,7 +157,7 @@ DECLARE_EVENT_CLASS(erofs__map_blocks_enter,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->la = map->m_la; __entry->llen = map->m_llen; __entry->flags = flags;
@@ -203,7 +203,7 @@ DECLARE_EVENT_CLASS(erofs__map_blocks_exit,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->flags = flags; __entry->la = map->m_la; __entry->pa = map->m_pa;
@@ -247,7 +247,7 @@ TRACE_EVENT(erofs_destroy_inode,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; ),TP_printk ("dev = (%d,%d), nid = %llu" ,show_dev_nid (__entry))
Signature · 1 line
2.17.1
Re: [PATCH 07/21] erofs: use erofs_inode naming
fs/erofs/super.c static struct kmem_cache *erofs_inode_cachep __read_mostly; · line 21
Quoted 5 lines
{
fs/erofs/super.c +1
-3-struct erofs_vnode *vi = ptr; - -inode_init_once (&vi->vfs_inode); +inode_init_once (&((struct erofs_inode *)ptr)->vfs_inode);
Why doesn't this use EROFS_I? This looks a little odd.
Re: [PATCH 07/21] erofs: use erofs_inode naming
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
fs/erofs/super.c static struct kmem_cache *erofs_inode_cachep __read_mostly; · line 21
Quoted 5 lines
{
fs/erofs/super.c +1
-3-struct erofs_vnode *vi = ptr; - -inode_init_once (&vi->vfs_inode); +inode_init_once (&((struct erofs_inode *)ptr)->vfs_inode);Why doesn't this use
EROFS_I? This looks a little odd.
Thanks for your reply and suggestion...EROFS_I seems the revert direction ---> inode to erofs_inode here we need "erofs_inode" to inode...
Am I missing something?.... Hope not....
Thanks,
Gao Xiang
Re: [PATCH 07/21] erofs: use erofs_inode naming
Quoted 11 lines · Gao Xiang
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
fs/erofs/super.c static struct kmem_cache *erofs_inode_cachep __read_mostly; · line 21
Quoted 5 lines
{
fs/erofs/super.c +1
-3-struct erofs_vnode *vi = ptr; - -inode_init_once (&vi->vfs_inode); +inode_init_once (&((struct erofs_inode *)ptr)->vfs_inode);Why doesn't this use
EROFS_I? This looks a little odd.
Quoted 5 lines
Thanks for your reply and suggestion...
EROFS_Iseems the revert direction ---> inode toerofs_inodehere we need "erofs_inode" to inode...Am I missing something?.... Hope not....
No, you are not. But the cast still looks odd. Why not:
struct erofs_inode *ei = ptr; inode_init_once(&ei->vfs_inode);
Re: [PATCH 07/21] erofs: use erofs_inode naming
Hi Christoph,
Quoted 16 lines · Christoph Hellwig
Quoted 15 lines · Gao Xiang
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
fs/erofs/super.c static struct kmem_cache *erofs_inode_cachep __read_mostly; · line 21
Quoted 5 lines
{
fs/erofs/super.c +1
-3-struct erofs_vnode *vi = ptr; - -inode_init_once (&vi->vfs_inode); +inode_init_once (&((struct erofs_inode *)ptr)->vfs_inode);Why doesn't this use
EROFS_I? This looks a little odd.Thanks for your reply and suggestion...
EROFS_Iseems the revert direction ---> inode toerofs_inodehere we need "erofs_inode" to inode...
Quoted 7 lines
Quoted 1 line
Am I missing something?.... Hope not....
No, you are not. But the cast still looks odd. Why not:
struct erofs_inode *ei = ptr; inode_init_once(&ei->vfs_inode);
That is the old way, I thought you don't like the extra variable...
https://lore.kernel.org/linux-erofs/[email protected]/
I am ok with either form, anyway, let me use the old way....
Thanks,
Gao Xiang
PATCH 8/21 erofs: update comments in inode.c
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph suggested [1], update them all.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/inode.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
fs/erofs/inode.c +2 -3
@@ -129,7 +129,7 @@ static int fill_inline_data(struct inode *inode, void *data,if (!is_inode_flat_inline (inode))return 0 ; -/* fast symlink (following ext4) */ +/* fast symlink */ if (S_ISLNK (inode->i_mode) && inode->i_size <PAGE_SIZE ) {char *lnk =erofs_kmalloc (sbi, inode->i_size +1 ,GFP_KERNEL );
@@ -138,7 +138,7 @@ static int fill_inline_data(struct inode *inode, void *data, m_pofs += vi->inode_isize + vi->xattr_isize; -/* inline symlink data shouldn't across page boundary as well */ +/* inline symlink data shouldn't cross page boundary as well */ if (m_pofs + inode->i_size >PAGE_SIZE ) {kfree (lnk); errln("inline data cross block boundary @ nid %llu",
@@ -147,7 +147,6 @@ static int fill_inline_data(struct inode *inode, void *data,return -EFSCORRUPTED ; } -/* get in-page inline data */ memcpy (lnk, data + m_pofs, inode->i_size); lnk[inode->i_size] ='\0' ;
Signature · 1 line
2.17.1
PATCH 9/21 erofs: update erofs symlink stuffs
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
Fix as Christoph suggested [1] [2], "remove is_inode_fast_symlink and just opencode it in the few places using it"
and
"Please just set the ops directly instead of obsfucating that in a single caller, single line inline function. And please set it instead of the normal symlink iops in the same place where you also set those."
[1] https://lore.kernel.org/r/[email protected]/
[2] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/inode.c | 35 ++++++++++------------------------- fs/erofs/internal.h | 10 ---------- fs/erofs/super.c | 5 ++--- 3 files changed, 12 insertions(+), 38 deletions(-)
fs/erofs/inode.c +10 -25
@@ -109,28 +109,14 @@ static int read_inode(struct inode *inode, void *data)return -EFSCORRUPTED ; } -/* -* try_lock can be required since locking order is: -* file data(fs_inode) -* meta(bd_inode) -* but the majority of the callers is "iget", -* in that case we are pretty sure no deadlock since -* no data operations exist. However I tend to -* try_lock since it takes no much overhead and -* will success immediately. -*/ -static int fill_inline_data (struct inode *inode,void *data, -unsigned int m_pofs) +static int erofs_fill_symlink (struct inode *inode,void *data, +unsigned int m_pofs) {struct erofs_inode *vi =EROFS_I (inode);struct erofs_sb_info *sbi =EROFS_I_SB (inode); -/* should be inode inline C */ -if (!is_inode_flat_inline (inode)) -return 0 ; - -/* fast symlink */ -if (S_ISLNK (inode->i_mode) && inode->i_size <PAGE_SIZE ) { +/* if it can be handled with fast symlink scheme */ +if (is_inode_flat_inline (inode) && inode->i_size <PAGE_SIZE ) {char *lnk =erofs_kmalloc (sbi, inode->i_size +1 ,GFP_KERNEL );if (!lnk)
@@ -151,7 +137,9 @@ static int fill_inline_data(struct inode *inode, void *data, lnk[inode->i_size] ='\0' ; inode->i_link = lnk; -set_inode_fast_symlink (inode); + inode->i_op = &erofs_fast_symlink_iops; + }else { + inode->i_op = &erofs_symlink_iops; }return 0 ; }
@@ -199,8 +187,9 @@ static int fill_inode(struct inode *inode, int isdir) inode->i_fop = &erofs_dir_fops;break ;case S_IFLNK : -/* by default, page_get_link is used for symlink */ - inode->i_op = &erofs_symlink_iops; + err =erofs_fill_symlink (inode, data, ofs); +if (err) + goto out_unlock;inode_nohighmem (inode);break ;case S_IFCHR :
@@ -219,11 +208,7 @@ static int fill_inode(struct inode *inode, int isdir) err =z_erofs_fill_inode (inode); goto out_unlock; } - inode->i_mapping->a_ops = &erofs_raw_access_aops; - -/* fill last page if inline data is available */ - err =fill_inline_data (inode, data, ofs); } out_unlock:
fs/erofs/internal.h +0 -10
@@ -480,16 +480,6 @@ extern const struct inode_operations erofs_generic_iops;extern const struct inode_operations erofs_symlink_iops;extern const struct inode_operations erofs_fast_symlink_iops; -static inline void set_inode_fast_symlink (struct inode *inode) -{ - inode->i_op = &erofs_fast_symlink_iops; -} - -static inline bool is_inode_fast_symlink (struct inode *inode) -{ -return inode->i_op == &erofs_fast_symlink_iops; -} -struct inode *erofs_iget (struct super_block *sb,erofs_nid_t nid,bool dir);int erofs_getattr (const struct path *path,struct kstat *stat,u32 request_mask,unsigned int query_flags);
fs/erofs/super.c +2 -3
@@ -38,10 +38,9 @@ static void free_inode(struct inode *inode) {struct erofs_inode *vi =EROFS_I (inode); -/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ -if (is_inode_fast_symlink (inode)) +/* be careful of RCU symlink path */ +if (inode->i_op == &erofs_fast_symlink_iops)kfree (inode->i_link); -kfree (vi->xattr_shared_xattrs);kmem_cache_free (erofs_inode_cachep, vi);
Signature · 1 line
2.17.1
Re: [PATCH 09/21] erofs: update erofs symlink stuffs
Thanks, this looks much better.
Quoted 4 lines
fs/erofs/inode.c | 35 ++++++++++------------------------- fs/erofs/internal.h | 10 ---------- fs/erofs/super.c | 5 ++--- 3 files changed, 12 insertions(+), 38 deletions(-)
And that diffstat ain't bad either.
PATCH 10/21 erofs: kill is_inode_layout_compression()
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph suggested [1], "The name of this helper is a little odd. But I think just opencoding it seems generally cleaner anyway. "
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/data.c | 2 +- fs/erofs/inode.c | 8 ++++---- fs/erofs/internal.h | 5 ----- 3 files changed, 5 insertions(+), 10 deletions(-)
fs/erofs/data.c +1 -1
@@ -169,7 +169,7 @@ static int erofs_map_blocks_flatmode(struct inode *inode,int erofs_map_blocks (struct inode *inode,struct erofs_map_blocks *map,int flags) { -if (is_inode_layout_compression (inode)) { +if (erofs_inode_is_data_compressed (EROFS_I (inode)->datamode)) {int err =z_erofs_map_blocks_iter (inode, map, flags);if (map->mpage) {
fs/erofs/inode.c +4 -4
@@ -56,7 +56,7 @@ static int read_inode(struct inode *inode, void *data) inode->i_size =le64_to_cpu (v2->i_size);/* total blocks for compressed files */ -if (is_inode_layout_compression (inode)) +if (erofs_inode_is_data_compressed (vi->datamode)) nblks =le32_to_cpu (v2->i_u.compressed_blocks); }else if (__inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) {struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb);
@@ -87,7 +87,7 @@ static int read_inode(struct inode *inode, void *data) sbi->build_time_nsec; inode->i_size =le32_to_cpu (v1->i_size); -if (is_inode_layout_compression (inode)) +if (erofs_inode_is_data_compressed (vi->datamode)) nblks =le32_to_cpu (v1->i_u.compressed_blocks); }else { errln("unsupported on-disk inode version %u of nid %llu",
@@ -204,7 +204,7 @@ static int fill_inode(struct inode *inode, int isdir) goto out_unlock; } -if (is_inode_layout_compression (inode)) { +if (erofs_inode_is_data_compressed (vi->datamode)) { err =z_erofs_fill_inode (inode); goto out_unlock; }
@@ -283,7 +283,7 @@ int erofs_getattr(const struct path *path, struct kstat *stat, {struct inode *const inode =d_inode (path->dentry); -if (is_inode_layout_compression (inode)) +if (erofs_inode_is_data_compressed (EROFS_I (inode)->datamode)) stat->attributes |=STATX_ATTR_COMPRESSED ; stat->attributes |=STATX_ATTR_IMMUTABLE ;
fs/erofs/internal.h +0 -5
@@ -327,11 +327,6 @@ static inline unsigned long inode_datablocks(struct inode *inode)return DIV_ROUND_UP (inode->i_size,EROFS_BLKSIZ ); } -static inline bool is_inode_layout_compression (struct inode *inode) -{ -return erofs_inode_is_data_compressed (EROFS_I (inode)->datamode); -} -static inline bool is_inode_flat_inline (struct inode *inode) {return EROFS_I (inode)->datamode ==EROFS_INODE_FLAT_INLINE ;
Signature · 1 line
2.17.1
Re: [PATCH 10/21] erofs: kill is_inode_layout_compression()
Thanks,
this looks much better.
PATCH 11/21 erofs: use dsb instead of layout for ondisk super_block
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph pointed out [1], "Why is the variable name for the on-disk subperblock layout? We usually still calls this something with sb in the name, e.g. dsb. for disksuper block. " Let's fix it.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/super.c | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-)
fs/erofs/super.c +18 -18
@@ -47,9 +47,9 @@ static void free_inode(struct inode *inode) }static bool check_layout_compatibility (struct super_block *sb, -struct erofs_super_block *layout) +struct erofs_super_block *dsb) { -const unsigned int requirements =le32_to_cpu (layout->requirements); +const unsigned int requirements =le32_to_cpu (dsb->requirements);EROFS_SB (sb)->requirements = requirements;
@@ -66,7 +66,7 @@ static int superblock_read(struct super_block *sb) {struct erofs_sb_info *sbi;struct buffer_head *bh; -struct erofs_super_block *layout; +struct erofs_super_block *dsb;unsigned int blkszbits;int ret;
@@ -78,16 +78,16 @@ static int superblock_read(struct super_block *sb) } sbi =EROFS_SB (sb); - layout = (struct erofs_super_block *)((u8 *)bh->b_data - +EROFS_SUPER_OFFSET ); + dsb = (struct erofs_super_block *)((u8 *)bh->b_data + +EROFS_SUPER_OFFSET ); ret = -EINVAL ; -if (le32_to_cpu (layout->magic) !=EROFS_SUPER_MAGIC_V1 ) { +if (le32_to_cpu (dsb->magic) !=EROFS_SUPER_MAGIC_V1 ) {errln ("cannot find valid erofs superblock" ); goto out; } - blkszbits = layout->blkszbits; + blkszbits = dsb->blkszbits;/* 9(512 bytes) + LOG_SECTORS_PER_BLOCK == LOG_BLOCK_SIZE */ if (blkszbits !=LOG_BLOCK_SIZE ) { errln("blksize %u isn't supported on this platform",
@@ -95,25 +95,25 @@ static int superblock_read(struct super_block *sb) goto out; } -if (!check_layout_compatibility (sb, layout)) +if (!check_layout_compatibility (sb, dsb)) goto out; - sbi->blocks =le32_to_cpu (layout->blocks); - sbi->meta_blkaddr =le32_to_cpu (layout->meta_blkaddr); + sbi->blocks =le32_to_cpu (dsb->blocks); + sbi->meta_blkaddr =le32_to_cpu (dsb->meta_blkaddr);#ifdef CONFIG_EROFS_FS_XATTR - sbi->xattr_blkaddr =le32_to_cpu (layout->xattr_blkaddr); + sbi->xattr_blkaddr =le32_to_cpu (dsb->xattr_blkaddr);#endif sbi->islotbits =ffs (sizeof (struct erofs_inode_v1 )) -1 ; - sbi->root_nid =le16_to_cpu (layout->root_nid); - sbi->inos =le64_to_cpu (layout->inos); + sbi->root_nid =le16_to_cpu (dsb->root_nid); + sbi->inos =le64_to_cpu (dsb->inos); - sbi->build_time =le64_to_cpu (layout->build_time); - sbi->build_time_nsec =le32_to_cpu (layout->build_time_nsec); + sbi->build_time =le64_to_cpu (dsb->build_time); + sbi->build_time_nsec =le32_to_cpu (dsb->build_time_nsec); -memcpy (&sb->s_uuid, layout->uuid,sizeof (layout->uuid)); +memcpy (&sb->s_uuid, dsb->uuid,sizeof (dsb->uuid)); - ret =strscpy (sbi->volume_name, layout->volume_name, -sizeof (layout->volume_name)); + ret =strscpy (sbi->volume_name, dsb->volume_name, +sizeof (dsb->volume_name));if (ret <0 ) {/* -E2BIG */ errln ("bad volume name without NIL terminator" ); ret = -EFSCORRUPTED ;
Signature · 1 line
2.17.1
Re: [PATCH 11/21] erofs: use dsb instead of layout for ondisk super_block
fs/erofs/super.c static int superblock_read(struct super_block *sb) · line 82
Quoted 2 lines
fs/erofs/super.c +2
-0+ dsb = (struct erofs_super_block *)((u8 *)bh->b_data + +EROFS_SUPER_OFFSET );
Not new in this patch, but that u8 cast shouldn't be needed.
Re: [PATCH 11/21] erofs: use dsb instead of layout for ondisk super_block
Hi Christoph,
Quoted 4 lines · Christoph Hellwig
fs/erofs/super.c static int superblock_read(struct super_block *sb) · line 82
Quoted 2 lines
fs/erofs/super.c +2
-0+ dsb = (struct erofs_super_block *)((u8 *)bh->b_data + +EROFS_SUPER_OFFSET );Not new in this patch, but that u8 cast shouldn't be needed.
Yes, thanks and will fix it...
Thanks,
Gao Xiang
PATCH 12/21 erofs: kill verbose debug info in erofs_fill_super
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph said [1], "That is some very verbose debug info. We usually don't add that and let people trace the function instead. "
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/super.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-)
fs/erofs/super.c +2 -7
@@ -383,9 +383,6 @@ static int erofs_fill_super(struct super_block *sb, void *data, int silent)struct erofs_sb_info *sbi;int err; -infoln ("fill_super, device -> %s" , sb->s_id); -infoln ("options -> %s" , (char *)data); - sb->s_magic =EROFS_SUPER_MAGIC ;if (!sb_set_blocksize (sb,EROFS_BLKSIZ )) {
@@ -418,9 +415,6 @@ static int erofs_fill_super(struct super_block *sb, void *data, int silent)if (err)return err; -if (!silent) -infoln ("root inode @ nid %llu" ,ROOT_NID (sbi)); -if (test_opt (sbi,POSIX_ACL )) sb->s_flags |=SB_POSIXACL ;else
@@ -453,7 +447,8 @@ static int erofs_fill_super(struct super_block *sb, void *data, int silent)return err;if (!silent) -infoln ("mounted on %s with opts: %s." , sb->s_id, (char *)data); +infoln ("mounted on %s with opts: %s, root inode @ nid %llu." , + sb->s_id, (char *)data,ROOT_NID (sbi));return 0 ; }
Signature · 1 line
2.17.1
Re: [PATCH 12/21] erofs: kill verbose debug info in erofs_fill_super
Quoted 5 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph said [1], "That is some very verbose
debug info. We usually don't add that and let
people trace the function instead. "
Note that this applies to most of the infoln users as far as I can tell. And if you want to keep some of those I think you should converted them to use pr_info directly, and also print sb->s_id as a prefix before the actual message so that the user knows which file system is affected.
Re: [PATCH 12/21] erofs: kill verbose debug info in erofs_fill_super
Hi Christoph,
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph said [1], "That is some very verbose
debug info. We usually don't add that and let
people trace the function instead. "
Quoted 5 lines
Note that this applies to most of the infoln users as far as I can tell. And if you want to keep some of those I think you should converted them to use
pr_infodirectly, and also printsb->s_idas a prefix before the actual message so that the user knows which file system is affected.
Thanks for your suggestion...
I think I will turn them into erofs_errln and etc... and print sb->s_id as a prefix...
Thanks,
Gao Xiang
PATCH 13/21 erofs: simplify erofs_grab_bio() since bio_alloc() never fail
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph pointed out [1], "erofs_grab_bio tries to handle a bio_alloc failure, except that the function will not actually fail due the mempool backing it."
Sorry about useless code, fix it now.
[1] https://lore.kernel.org/lkml/[email protected]/
--- fs/erofs/data.c | 15 ++------------- fs/erofs/internal.h | 19 ++----------------- fs/erofs/zdata.c | 2 +- 3 files changed, 5 insertions(+), 31 deletions(-)
fs/erofs/data.c +2 -13
@@ -62,12 +62,7 @@ struct page *__erofs_get_meta_page(struct super_block *sb,if (!PageUptodate (page)) {struct bio *bio; - bio =erofs_grab_bio (sb, blkaddr,1 , sb, read_endio, nofail); -if (IS_ERR (bio)) { -DBG_BUGON (nofail); - err =PTR_ERR (bio); - goto err_out; - } + bio =erofs_grab_bio (sb, blkaddr,1 , sb, read_endio);if (bio_add_page (bio, page,PAGE_SIZE ,0 ) !=PAGE_SIZE ) { err = -EFAULT ;
@@ -275,13 +270,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio,if (nblocks >BIO_MAX_PAGES ) nblocks =BIO_MAX_PAGES ; - bio =erofs_grab_bio (sb, blknr, nblocks, sb, - read_endio, false); -if (IS_ERR (bio)) { - err =PTR_ERR (bio); - bio =NULL ; - goto err_out; - } + bio =erofs_grab_bio (sb, blknr, nblocks, sb, read_endio); } err =bio_add_page (bio, page,PAGE_SIZE ,0 );
fs/erofs/internal.h +2 -17
@@ -410,24 +410,9 @@ static inline int z_erofs_map_blocks_iter(struct inode *inode,static inline struct bio *erofs_grab_bio (struct super_block *sb,erofs_blk_t blkaddr,unsigned int nr_pages, -void *bi_private,bio_end_io_t endio, -bool nofail) +void *bi_private,bio_end_io_t endio) { -const gfp_t gfp =GFP_NOIO ; -struct bio *bio; - -do { -if (nr_pages ==1 ) { - bio =bio_alloc (gfp | (nofail ? __GFP_NOFAIL :0 ),1 ); -if (!bio) { -DBG_BUGON (nofail); -return ERR_PTR (-ENOMEM ); - } -break ; - } - bio =bio_alloc (gfp, nr_pages); - nr_pages /=2 ; - }while (!bio); +struct bio *bio =bio_alloc (GFP_NOIO , nr_pages); bio->bi_end_io = endio;bio_set_dev (bio, sb->s_bdev);
fs/erofs/zdata.c +1 -1
@@ -1265,7 +1265,7 @@ static bool z_erofs_vle_submit_all(struct super_block *sb,if (!bio) { bio =erofs_grab_bio (sb, first_index + i,BIO_MAX_PAGES , bi_private, - z_erofs_vle_read_endio, true); + z_erofs_vle_read_endio); ++nr_bios; }
Signature · 1 line
2.17.1
Re: [PATCH 13/21] erofs: simplify erofs_grab_bio() since bio_alloc() never fail
Quoted 7 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph pointed out [1], "
erofs_grab_biotries to handle abio_allocfailure, except that the function will not actually fail due the mempool backing it."Sorry about useless code, fix it now.
A lot of file systems used to have this, it is a classic that gets copy and pasted everywhere. If you feel like doing a little project it might make sense to check for other places that do the same.
Quoted 6 lines
fs/erofs/data.c +1
-0+ bio =erofs_grab_bio (sb, blkaddr,1 , sb, read_endio);if (bio_add_page (bio, page,PAGE_SIZE ,0 ) !=PAGE_SIZE ) { err = -EFAULT ;@@ -275,13 +270,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio,if (nblocks >BIO_MAX_PAGES )
fs/erofs/data.c static inline struct bio *erofs_read_raw_page(struct bio *bio, · line 273
Quoted 10 lines
fs/erofs/data.c +1
-7nblocks =BIO_MAX_PAGES ; - bio =erofs_grab_bio (sb, blknr, nblocks, sb, - read_endio, false); -if (IS_ERR (bio)) { - err =PTR_ERR (bio); - bio =NULL ; - goto err_out; - } + bio =erofs_grab_bio (sb, blknr, nblocks, sb, read_endio);
Btw, I think you should remove erofs_grab_bio in its current form. The full code in data.c is indeed used in two places, so a local erofs_raw_page_alloc_bio (or whatever name you prefer) helper here that hardcodes the sb amd read_endio argument to simplify it a bit makes sense.
fs/erofs/zdata.c static bool z_erofs_vle_submit_all(struct super_block *sb, · line 1268
Quoted 10 lines
fs/erofs/zdata.c +1
-1@@ -1265,7 +1265,7 @@ static bool z_erofs_vle_submit_all(struct super_block *sb,if (!bio) { bio =erofs_grab_bio (sb, first_index + i,BIO_MAX_PAGES , bi_private, - z_erofs_vle_read_endio, true); + z_erofs_vle_read_endio);
But I think here you might as well open code it entirely, or at least us a separate (and local to zdata.c) helper.
PATCH 14/21 erofs: kill prio and nofail of erofs_get_meta_page()
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph pointed out [1],
"Why is there __erofs_get_meta_page with the two weird booleans instead of a single erofs_get_meta_page that gets and gfp_t for additional flags and an unsigned int for additional bio op flags."
And since all callers can handle errors, let's kill prio and nofail and erofs_get_inline_page() now.
[1] https://lore.kernel.org/lkml/[email protected]/
--- fs/erofs/data.c | 24 ++++++------------------ fs/erofs/inode.c | 2 +- fs/erofs/internal.h | 18 +----------------- fs/erofs/xattr.c | 13 ++++++------- fs/erofs/zmap.c | 4 ++-- 5 files changed, 16 insertions(+), 45 deletions(-)
fs/erofs/data.c +6 -18
@@ -38,25 +38,20 @@ static inline void read_endio(struct bio *bio)bio_put (bio); } -/* prio -- true is used for dir */ -struct page *__erofs_get_meta_page (struct super_block *sb, -erofs_blk_t blkaddr,bool prio,bool nofail) +struct page *erofs_get_meta_page (struct super_block *sb,erofs_blk_t blkaddr) {struct inode *const bd_inode = sb->s_bdev->bd_inode;struct address_space *const mapping = bd_inode->i_mapping;/* prefer retrying in the allocator to blindly looping below */ -const gfp_t gfp =mapping_gfp_constraint (mapping, ~__GFP_FS) | - (nofail ? __GFP_NOFAIL :0 ); -unsigned int io_retries = nofail ?EROFS_IO_MAX_RETRIES_NOFAIL :0 ; +const gfp_t gfp =mapping_gfp_constraint (mapping, ~__GFP_FS);struct page *page;int err; repeat: page =find_or_create_page (mapping, blkaddr, gfp); -if (!page) { -DBG_BUGON (nofail); +if (!page)return ERR_PTR (-ENOMEM ); - } +DBG_BUGON (!PageLocked (page));if (!PageUptodate (page)) {
@@ -69,14 +64,11 @@ struct page *__erofs_get_meta_page(struct super_block *sb, goto err_out; } -__submit_bio (bio,REQ_OP_READ , -REQ_META | (prio ?REQ_PRIO :0 )); - +__submit_bio (bio,REQ_OP_READ ,REQ_META );lock_page (page);/* this page has been truncated by others */ if (page->mapping != mapping) { -unlock_repeat:unlock_page (page);put_page (page); goto repeat;
@@ -84,10 +76,6 @@ struct page *__erofs_get_meta_page(struct super_block *sb,/* more likely a read error */ if (!PageUptodate (page)) { -if (io_retries) { - --io_retries; - goto unlock_repeat; - } err = -EIO ; goto err_out; }
@@ -237,7 +225,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio,DBG_BUGON (map.m_plen >PAGE_SIZE ); - ipage =erofs_get_meta_page (inode->i_sb, blknr,0 ); + ipage =erofs_get_meta_page (inode->i_sb, blknr);if (IS_ERR (ipage)) { err =PTR_ERR (ipage);
fs/erofs/inode.c +1 -1
@@ -163,7 +163,7 @@ static int fill_inode(struct inode *inode, int isdir)debugln ("%s, reading inode nid %llu at %u of blkaddr %u" , __func__, vi->nid, ofs, blkaddr); - page =erofs_get_meta_page (inode->i_sb, blkaddr, isdir); + page =erofs_get_meta_page (inode->i_sb, blkaddr);if (IS_ERR (page)) { errln("failed to get inode (nid: %llu) page, err %ld",
fs/erofs/internal.h +1 -17
@@ -258,8 +258,6 @@ static inline int erofs_wait_on_workgroup_freezed(struct erofs_workgroup *grp)#error erofs cannot be used in this platform#endif -#define EROFS_IO_MAX_RETRIES_NOFAIL 5 -#define ROOT_NID (sb) ((sb)->root_nid)#define erofs_blknr (addr) ((addr) / EROFS_BLKSIZ)
@@ -428,24 +426,10 @@ static inline void __submit_bio(struct bio *bio, unsigned int op,submit_bio (bio); } -struct page *__erofs_get_meta_page (struct super_block *sb,erofs_blk_t blkaddr, -bool prio,bool nofail); - -static inline struct page *erofs_get_meta_page (struct super_block *sb, -erofs_blk_t blkaddr,bool prio) -{ -return __erofs_get_meta_page (sb, blkaddr, prio, false); -} +struct page *erofs_get_meta_page (struct super_block *sb,erofs_blk_t blkaddr);int erofs_map_blocks (struct inode *,struct erofs_map_blocks *,int ); -static inline struct page *erofs_get_inline_page (struct inode *inode, -erofs_blk_t blkaddr) -{ -return erofs_get_meta_page (inode->i_sb, blkaddr, -S_ISDIR (inode->i_mode)); -} -/* inode.c */ static inline unsigned long erofs_inode_hash (erofs_nid_t nid) {
fs/erofs/xattr.c +6 -7
@@ -87,7 +87,7 @@ static int init_inode_xattrs(struct inode *inode) it.blkaddr =erofs_blknr (iloc (sbi, vi->nid) + vi->inode_isize); it.ofs =erofs_blkoff (iloc (sbi, vi->nid) + vi->inode_isize); - it.page =erofs_get_inline_page (inode, it.blkaddr); + it.page =erofs_get_meta_page (sb, it.blkaddr);if (IS_ERR (it.page)) { ret =PTR_ERR (it.page); goto out_unlock;
@@ -117,8 +117,7 @@ static int init_inode_xattrs(struct inode *inode)DBG_BUGON (it.ofs !=EROFS_BLKSIZ );xattr_iter_end (&it, atomic_map); - it.page =erofs_get_meta_page (sb, ++it.blkaddr, -S_ISDIR (inode->i_mode)); + it.page =erofs_get_meta_page (sb, ++it.blkaddr);if (IS_ERR (it.page)) {kfree (vi->xattr_shared_xattrs); vi->xattr_shared_xattrs =NULL ;
@@ -168,7 +167,7 @@ static inline int xattr_iter_fixup(struct xattr_iter *it) it->blkaddr +=erofs_blknr (it->ofs); - it->page =erofs_get_meta_page (it->sb, it->blkaddr, false); + it->page =erofs_get_meta_page (it->sb, it->blkaddr);if (IS_ERR (it->page)) {int err =PTR_ERR (it->page);
@@ -199,7 +198,7 @@ static int inline_xattr_iter_begin(struct xattr_iter *it, it->blkaddr =erofs_blknr (iloc (sbi, vi->nid) + inline_xattr_ofs); it->ofs =erofs_blkoff (iloc (sbi, vi->nid) + inline_xattr_ofs); - it->page =erofs_get_inline_page (inode, it->blkaddr); + it->page =erofs_get_meta_page (inode->i_sb, it->blkaddr);if (IS_ERR (it->page))return PTR_ERR (it->page);
@@ -401,7 +400,7 @@ static int shared_getxattr(struct inode *inode, struct getxattr_iter *it)if (i)xattr_iter_end (&it->it, true); - it->it.page =erofs_get_meta_page (sb, blkaddr, false); + it->it.page =erofs_get_meta_page (sb, blkaddr);if (IS_ERR (it->it.page))return PTR_ERR (it->it.page);
@@ -623,7 +622,7 @@ static int shared_listxattr(struct listxattr_iter *it)if (i)xattr_iter_end (&it->it, true); - it->it.page =erofs_get_meta_page (sb, blkaddr, false); + it->it.page =erofs_get_meta_page (sb, blkaddr);if (IS_ERR (it->it.page))return PTR_ERR (it->it.page);
fs/erofs/zmap.c +2 -2
@@ -50,7 +50,7 @@ static int fill_inode_lazy(struct inode *inode) pos =ALIGN (iloc (EROFS_SB (sb), vi->nid) + vi->inode_isize + vi->xattr_isize,8 ); - page =erofs_get_meta_page (sb,erofs_blknr (pos), false); + page =erofs_get_meta_page (sb,erofs_blknr (pos));if (IS_ERR (page)) { err =PTR_ERR (page); goto out_unlock;
@@ -127,7 +127,7 @@ static int z_erofs_reload_indexes(struct z_erofs_maprecorder *m,put_page (mpage); } - mpage =erofs_get_meta_page (sb, eblk, false); + mpage =erofs_get_meta_page (sb, eblk);if (IS_ERR (mpage)) { map->mpage =NULL ;return PTR_ERR (mpage);
Signature · 1 line
2.17.1
Re: [PATCH 14/21] erofs: kill prio and nofail of erofs_get_meta_page()
Thanks,
much better. I'm still hoping REQ_PRIO in this current form will go entirely away soon as well.
PATCH 15/21 erofs: kill __submit_bio()
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph pointed out [1], "
Why is there __submit_bio which really just obsfucates what is going on? Also why is __submit_bio using bio_set_op_attrs instead of opencode it as the comment right next to it asks you to? "
Let's use submit_bio directly instead.
[1] https://lore.kernel.org/lkml/[email protected]/
--- fs/erofs/data.c | 15 ++++++++------- fs/erofs/internal.h | 9 ++------- fs/erofs/zdata.c | 6 +++--- 3 files changed, 13 insertions(+), 17 deletions(-)
fs/erofs/data.c +8 -7
@@ -57,14 +57,15 @@ struct page *erofs_get_meta_page(struct super_block *sb, erofs_blk_t blkaddr)if (!PageUptodate (page)) {struct bio *bio; - bio =erofs_grab_bio (sb, blkaddr,1 , sb, read_endio); + bio =erofs_grab_bio (sb,REQ_OP_READ |REQ_META , + blkaddr,1 , sb, read_endio);if (bio_add_page (bio, page,PAGE_SIZE ,0 ) !=PAGE_SIZE ) { err = -EFAULT ; goto err_out; } -__submit_bio (bio,REQ_OP_READ ,REQ_META ); +submit_bio (bio);lock_page (page);/* this page has been truncated by others */
@@ -188,7 +189,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio,/* not continuous */ *last_block + 1 != current_block) { submit_bio_retry: -__submit_bio (bio,REQ_OP_READ ,0 ); +submit_bio (bio); bio =NULL ; }
@@ -258,7 +259,8 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio,if (nblocks >BIO_MAX_PAGES ) nblocks =BIO_MAX_PAGES ; - bio =erofs_grab_bio (sb, blknr, nblocks, sb, read_endio); + bio =erofs_grab_bio (sb,REQ_OP_READ , blknr, nblocks, + sb, read_endio); } err =bio_add_page (bio, page,PAGE_SIZE ,0 );
@@ -289,8 +291,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio,/* if updated manually, continuous pages has a gap */ if (bio) submit_bio_out: -__submit_bio (bio,REQ_OP_READ ,0 ); - +submit_bio (bio);return err ?ERR_PTR (err) :NULL ; }
@@ -354,7 +355,7 @@ static int erofs_raw_access_readpages(struct file *filp,/* the rare case (end in gaps) */ if (bio) -__submit_bio (bio,REQ_OP_READ ,0 ); +submit_bio (bio);return 0 ; }
fs/erofs/internal.h +2 -7
@@ -406,6 +406,7 @@ static inline int z_erofs_map_blocks_iter(struct inode *inode,/* data.c */ static inline struct bio *erofs_grab_bio (struct super_block *sb, +unsigned int bi_opf,erofs_blk_t blkaddr,unsigned int nr_pages,void *bi_private,bio_end_io_t endio)
@@ -416,16 +417,10 @@ static inline struct bio *erofs_grab_bio(struct super_block *sb,bio_set_dev (bio, sb->s_bdev); bio->bi_iter.bi_sector = (sector_t )blkaddr <<LOG_SECTORS_PER_BLOCK ; bio->bi_private = bi_private; + bio->bi_opf = bi_opf;return bio; } -static inline void __submit_bio (struct bio *bio,unsigned int op, -unsigned int op_flags) -{ -bio_set_op_attrs (bio, op, op_flags); -submit_bio (bio); -} -struct page *erofs_get_meta_page (struct super_block *sb,erofs_blk_t blkaddr);int erofs_map_blocks (struct inode *,struct erofs_map_blocks *,int );
fs/erofs/zdata.c +3 -3
@@ -1258,12 +1258,12 @@ static bool z_erofs_vle_submit_all(struct super_block *sb,if (bio && force_submit) { submit_bio_retry: -__submit_bio (bio,REQ_OP_READ ,0 ); +submit_bio (bio); bio =NULL ; }if (!bio) { - bio =erofs_grab_bio (sb, first_index + i, + bio =erofs_grab_bio (sb,REQ_OP_READ , first_index + i,BIO_MAX_PAGES , bi_private, z_erofs_vle_read_endio); ++nr_bios;
@@ -1286,7 +1286,7 @@ static bool z_erofs_vle_submit_all(struct super_block *sb, }while (owned_head !=Z_EROFS_PCLUSTER_TAIL );if (bio) -__submit_bio (bio,REQ_OP_READ ,0 ); +submit_bio (bio);if (postsubmit_is_all_bypassed (q, nr_bios, force_fg))return true;
Signature · 1 line
2.17.1
PATCH 16/21 erofs: kill magic underscores
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph said [1], "
Review
Quoted 1 line
(quoted diff) +1
-0+ vi->datamode = __inode_data_mapping(advise);
What is the deal with these magic underscores here and various other similar helpers? "
Let avoid magic underscores now...
[1] https://lore.kernel.org/lkml/[email protected]/
--- fs/erofs/inode.c | 8 ++++---- fs/erofs/internal.h | 14 ++++++-------- 2 files changed, 10 insertions(+), 12 deletions(-)
fs/erofs/inode.c +4 -4
@@ -16,7 +16,7 @@ static int read_inode(struct inode *inode, void *data)const unsigned int advise =le16_to_cpu (v1->i_advise);erofs_blk_t nblks =0 ; - vi->datamode =__inode_data_mapping (advise); + vi->datamode =erofs_inode_data_mapping (advise);if (vi->datamode >=EROFS_INODE_LAYOUT_MAX ) { errln("unsupported data mapping %u of nid %llu",
@@ -25,7 +25,7 @@ static int read_inode(struct inode *inode, void *data)return -EOPNOTSUPP ; } -if (__inode_version (advise) ==EROFS_INODE_LAYOUT_V2 ) { +if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V2 ) {struct erofs_inode_v2 *v2 = data; vi->inode_isize =sizeof (struct erofs_inode_v2 );
@@ -58,7 +58,7 @@ static int read_inode(struct inode *inode, void *data)/* total blocks for compressed files */ if (erofs_inode_is_data_compressed (vi->datamode)) nblks =le32_to_cpu (v2->i_u.compressed_blocks); - }else if (__inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) { + }else if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) {struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); vi->inode_isize =sizeof (struct erofs_inode_v1 );
@@ -91,7 +91,7 @@ static int read_inode(struct inode *inode, void *data) nblks =le32_to_cpu (v1->i_u.compressed_blocks); }else {errln ("unsupported on-disk inode version %u of nid %llu" , -__inode_version (advise), vi->nid); +erofs_inode_version (advise), vi->nid);DBG_BUGON (1 );return -EOPNOTSUPP ; }
fs/erofs/internal.h +6 -8
@@ -308,16 +308,14 @@ struct erofs_inode {#define EROFS_I (ptr) \ container_of(ptr, struct erofs_inode, vfs_inode) -#define __inode_advise (x, bit, bits) \ - (((x) >> (bit)) & ((1 << (bits)) - 1)) +#define erofs_bitrange (x, bit, bits) (((x) >> (bit)) & ((1 << (bits)) - 1)) -#define __inode_version (advise) \ - __inode_advise(advise, EROFS_I_VERSION_BIT, \ - EROFS_I_VERSION_BITS) +#define erofs_inode_version (advise) \ + erofs_bitrange(advise, EROFS_I_VERSION_BIT, EROFS_I_VERSION_BITS) -#define __inode_data_mapping (advise) \ - __inode_advise(advise, EROFS_I_DATA_MAPPING_BIT,\ - EROFS_I_DATA_MAPPING_BITS) +#define erofs_inode_data_mapping (advise) \ + erofs_bitrange(advise, EROFS_I_DATA_MAPPING_BIT, \ + EROFS_I_DATA_MAPPING_BITS)static inline unsigned long inode_datablocks (struct inode *inode) {
Signature · 1 line
2.17.1
Re: [PATCH 16/21] erofs: kill magic underscores
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 21
Quoted 5 lines
fs/erofs/inode.c +1
-1- vi->datamode =__inode_data_mapping (advise); + vi->datamode =erofs_inode_data_mapping (advise);if (vi->datamode >=EROFS_INODE_LAYOUT_MAX ) {
While you are at it can we aim for some naming consistency here? The inode member is called is called datamode, the helper is called inode_data_mapping, and the enum uses layout? To me data_layout seems most descriptive, datamode is probably ok, but mapping seems very misleading given that we've already overloaded that terms for multiple other uses.
And while we are at naming choices - maybe i_format might be a better name for the i_advise field in the on-disk inode?
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 28
Quoted 1 line
fs/erofs/inode.c +1
-0+if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V2 ) {
I still need to wade through the old thread - but didn't you say this wasn't really a new vs old version but a compat vs full inode? Maybe the names aren't that suitable either?
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 61
Quoted 9 lines
fs/erofs/inode.c +1
-1struct erofs_inode_v2 *v2 = data; vi->inode_isize =sizeof (struct erofs_inode_v2 );@@ -58,7 +58,7 @@ static int read_inode(struct inode *inode, void *data)/* total blocks for compressed files */ if (erofs_inode_is_data_compressed (vi->datamode)) nblks =le32_to_cpu (v2->i_u.compressed_blocks); - }else if (__inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) { + }else if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) {
Also a lot of code would use a switch statements to switch for different matches on the same value instead of chained if/else if/else if statements.
fs/erofs/internal.h struct erofs_inode { · line 318
Quoted 8 lines
fs/erofs/internal.h +6
-0+#define erofs_bitrange (x, bit, bits) (((x) >> (bit)) & ((1 << (bits)) - 1)) +#define erofs_inode_version (advise) \ + erofs_bitrange(advise, EROFS_I_VERSION_BIT, EROFS_I_VERSION_BITS) +#define erofs_inode_data_mapping (advise) \ + erofs_bitrange(advise, EROFS_I_DATA_MAPPING_BIT, \ + EROFS_I_DATA_MAPPING_BITS)
All these should probably be inline functions.
Re: [PATCH 16/21] erofs: kill magic underscores
Hi Christoph,
Quoted 4 lines · Christoph Hellwig
Quoted 4 lines
fs/erofs/inode.c +1
-1- vi->datamode =__inode_data_mapping (advise); + vi->datamode =erofs_inode_data_mapping (advise);
Quoted 8 lines
Quoted 1 line
if (
vi->datamode>=EROFS_INODE_LAYOUT_MAX) {While you are at it can we aim for some naming consistency here? The inode member is called is called datamode, the helper is called
inode_data_mapping, and the enum uses layout? To medata_layoutseems most descriptive, datamode is probably ok, but mapping seems very misleading given that we've already overloaded that terms for multiple other uses.
the naming changed for many times...
Initially, it was called "data_mapping_mode", and I think it was too long (and as you said confusing, since confusing "mapping" meaning, sorry about my awkward English) so I simplified some into datamode....
In a word, I will change all data_mapping_mode into datalayout....
Quoted 3 lines
And while we are at naming choices - maybe
i_formatmight be a better name for thei_advisefield in the on-disk inode?
That is a good idea, I will resend v2 to address it...
Quoted 6 lines
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 28
Quoted 2 lines
fs/erofs/inode.c +1
-0+if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V2 ) {I still need to wade through the old thread - but didn't you say this wasn't really a new vs old version but a compat vs full inode? Maybe the names aren't that suitable either?
Could you kindly give some suggestions for better naming about it? there are all supported by EROFS... (Actually we only mainly use v1...)
Quoted 14 lines
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 61
Quoted 10 lines
fs/erofs/inode.c +1
-1struct erofs_inode_v2 *v2 = data; vi->inode_isize =sizeof (struct erofs_inode_v2 );@@ -58,7 +58,7 @@ static int read_inode(struct inode *inode, void *data)/* total blocks for compressed files */ if (erofs_inode_is_data_compressed (vi->datamode)) nblks =le32_to_cpu (v2->i_u.compressed_blocks); - }else if (__inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) { + }else if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V1 ) {Also a lot of code would use a switch statements to switch for different matches on the same value instead of chained if/else if/else if statements.
I will change it as well.
Quoted 11 lines
fs/erofs/internal.h struct erofs_inode { · line 318
Quoted 9 lines
fs/erofs/internal.h +6
-0+#define erofs_bitrange (x, bit, bits) (((x) >> (bit)) & ((1 << (bits)) - 1)) +#define erofs_inode_version (advise) \ + erofs_bitrange(advise, EROFS_I_VERSION_BIT, EROFS_I_VERSION_BITS) +#define erofs_inode_data_mapping (advise) \ + erofs_bitrange(advise, EROFS_I_DATA_MAPPING_BIT, \ + EROFS_I_DATA_MAPPING_BITS)All these should probably be inline functions.
Will fix...
Thanks,
Gao Xiang
Re: [PATCH 16/21] erofs: kill magic underscores
Quoted 9 lines · Gao Xiang
Quoted 6 lines
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 28
Quoted 2 lines
fs/erofs/inode.c +1
-0+if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V2 ) {I still need to wade through the old thread - but didn't you say this wasn't really a new vs old version but a compat vs full inode? Maybe the names aren't that suitable either?
Could you kindly give some suggestions for better naming about it? there are all supported by EROFS... (Actually we only mainly use v1...)
Maybe EROFS_INODE_LAYOUT_COMPACT and EROFS_INODE_LAYOUT_EXTENDED?
Re: [PATCH 16/21] erofs: kill magic underscores
Hi Christoph,
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Gao Xiang
Quoted 6 lines
fs/erofs/inode.c static int read_inode(struct inode *inode, void *data) · line 28
Quoted 2 lines
fs/erofs/inode.c +1
-0+if (erofs_inode_version (advise) ==EROFS_INODE_LAYOUT_V2 ) {I still need to wade through the old thread - but didn't you say this wasn't really a new vs old version but a compat vs full inode? Maybe the names aren't that suitable either?
Quoted 4 lines
Quoted 2 lines
Could you kindly give some suggestions for better naming about it? there are all supported by EROFS... (Actually we only mainly use v1...)
Maybe
EROFS_INODE_LAYOUT_COMPACTandEROFS_INODE_LAYOUT_EXTENDED?
Okay, that is fine.
Thanks,
Gao Xiang
PATCH 17/21 erofs: use a switch statement when dealing with the file modes
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph suggested [1], "
Please use a switch statement when dealing with the file modes to make everything easier to read."
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/inode.c | 38 ++++++++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 12 deletions(-)
fs/erofs/inode.c +26 -12
@@ -32,17 +32,24 @@ static int read_inode(struct inode *inode, void *data) vi->xattr_isize =erofs_xattr_ibody_size (v2->i_xattr_icount); inode->i_mode =le16_to_cpu (v2->i_mode); -if (S_ISREG (inode->i_mode) ||S_ISDIR (inode->i_mode) || -S_ISLNK (inode->i_mode)) +switch (inode->i_mode &S_IFMT ) { +case S_IFREG : +case S_IFDIR : +case S_IFLNK : vi->raw_blkaddr =le32_to_cpu (v2->i_u.raw_blkaddr); -else if (S_ISCHR (inode->i_mode) ||S_ISBLK (inode->i_mode)) +break ; +case S_IFCHR : +case S_IFBLK : inode->i_rdev =new_decode_dev (le32_to_cpu (v2->i_u.rdev)); -else if (S_ISFIFO (inode->i_mode) ||S_ISSOCK (inode->i_mode)) +break ; +case S_IFIFO : +case S_IFSOCK : inode->i_rdev =0 ; -else +break ; +default : goto bogusimode; - + }i_uid_write (inode,le32_to_cpu (v2->i_uid));i_gid_write (inode,le32_to_cpu (v2->i_gid));set_nlink (inode,le32_to_cpu (v2->i_nlink));
@@ -65,17 +72,24 @@ static int read_inode(struct inode *inode, void *data) vi->xattr_isize =erofs_xattr_ibody_size (v1->i_xattr_icount); inode->i_mode =le16_to_cpu (v1->i_mode); -if (S_ISREG (inode->i_mode) ||S_ISDIR (inode->i_mode) || -S_ISLNK (inode->i_mode)) +switch (inode->i_mode &S_IFMT ) { +case S_IFREG : +case S_IFDIR : +case S_IFLNK : vi->raw_blkaddr =le32_to_cpu (v1->i_u.raw_blkaddr); -else if (S_ISCHR (inode->i_mode) ||S_ISBLK (inode->i_mode)) +break ; +case S_IFCHR : +case S_IFBLK : inode->i_rdev =new_decode_dev (le32_to_cpu (v1->i_u.rdev)); -else if (S_ISFIFO (inode->i_mode) ||S_ISSOCK (inode->i_mode)) +break ; +case S_IFIFO : +case S_IFSOCK : inode->i_rdev =0 ; -else +break ; +default : goto bogusimode; - + }i_uid_write (inode,le16_to_cpu (v1->i_uid));i_gid_write (inode,le16_to_cpu (v1->i_gid));set_nlink (inode,le16_to_cpu (v1->i_nlink));
Signature · 1 line
2.17.1
Re: [PATCH 17/21] erofs: use a switch statement when dealing with the file modes
Thanks,
this looks much nicer.
PATCH 18/21 erofs: add "erofs_" prefix for common and short functions
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
Add erofs_ prefix to free_inode, alloc_inode, ...
--- fs/erofs/data.c | 6 +++--- fs/erofs/decompressor.c | 22 +++++++++++----------- fs/erofs/inode.c | 8 ++++---- fs/erofs/namei.c | 10 +++++----- fs/erofs/super.c | 26 +++++++++++++------------- fs/erofs/zdata.c | 19 ++++++++++--------- 6 files changed, 46 insertions(+), 45 deletions(-)
fs/erofs/data.c +3 -3
@@ -9,7 +9,7 @@#include <trace/events/erofs.h> -static inline void read_endio (struct bio *bio) +static void erofs_readendio (struct bio *bio) {struct super_block *const sb = bio->bi_private;struct bio_vec *bvec;
@@ -58,7 +58,7 @@ struct page *erofs_get_meta_page(struct super_block *sb, erofs_blk_t blkaddr)struct bio *bio; bio =erofs_grab_bio (sb,REQ_OP_READ |REQ_META , - blkaddr,1 , sb, read_endio); + blkaddr,1 , sb, erofs_readendio);if (bio_add_page (bio, page,PAGE_SIZE ,0 ) !=PAGE_SIZE ) { err = -EFAULT ;
@@ -260,7 +260,7 @@ static inline struct bio *erofs_read_raw_page(struct bio *bio, nblocks =BIO_MAX_PAGES ; bio =erofs_grab_bio (sb,REQ_OP_READ , blknr, nblocks, - sb, read_endio); + sb, erofs_readendio); } err =bio_add_page (bio, page,PAGE_SIZE ,0 );
fs/erofs/decompressor.c +11 -11
@@ -32,8 +32,8 @@ static bool use_vmap;module_param (use_vmap, bool,0444 );MODULE_PARM_DESC (use_vmap,"Use vmap() instead of vm_map_ram() (default 0)" ); -static int lz4_prepare_destpages (struct z_erofs_decompress_req *rq, -struct list_head *pagepool) +static int z_erofs_lz4_prepare_destpages (struct z_erofs_decompress_req *rq, +struct list_head *pagepool) {const unsigned int nr =PAGE_ALIGN (rq->pageofs_out + rq->outputsize) >>PAGE_SHIFT ;
@@ -114,7 +114,7 @@ static void *generic_copy_inplace_data(struct z_erofs_decompress_req *rq,return tmp; } -static int lz4_decompress (struct z_erofs_decompress_req *rq,u8 *out) +static int z_erofs_lz4_decompress (struct z_erofs_decompress_req *rq,u8 *out) {unsigned int inputmargin, inlen;u8 *src;
@@ -187,8 +187,8 @@ static struct z_erofs_decompressor decompressors[] = { .name = "shifted" }, [Z_EROFS_COMPRESSION_LZ4] = { - .prepare_destpages = lz4_prepare_destpages, - .decompress = lz4_decompress, + .prepare_destpages = z_erofs_lz4_prepare_destpages, + .decompress = z_erofs_lz4_decompress, .name ="lz4" }, };
@@ -246,8 +246,8 @@ static void erofs_vunmap(const void *mem, unsigned int count)vunmap (mem); } -static int decompress_generic (struct z_erofs_decompress_req *rq, -struct list_head *pagepool) +static int z_erofs_decompress_generic (struct z_erofs_decompress_req *rq, +struct list_head *pagepool) {const unsigned int nrpages_out =PAGE_ALIGN (rq->pageofs_out + rq->outputsize) >>PAGE_SHIFT ;
@@ -307,8 +307,8 @@ static int decompress_generic(struct z_erofs_decompress_req *rq,return ret; } -static int shifted_decompress (const struct z_erofs_decompress_req *rq, -struct list_head *pagepool) +static int z_erofs_shifted_transform (const struct z_erofs_decompress_req *rq, +struct list_head *pagepool) {const unsigned int nrpages_out =PAGE_ALIGN (rq->pageofs_out + rq->outputsize) >>PAGE_SHIFT ;
@@ -352,7 +352,7 @@ int z_erofs_decompress(struct z_erofs_decompress_req *rq, struct list_head *pagepool) {if (rq->alg ==Z_EROFS_COMPRESSION_SHIFTED ) -return shifted_decompress (rq, pagepool); -return decompress_generic (rq, pagepool); +return z_erofs_shifted_transform (rq, pagepool); +return z_erofs_decompress_generic (rq, pagepool); }
fs/erofs/inode.c +4 -4
@@ -9,7 +9,7 @@#include <trace/events/erofs.h> /* no locking */ -static int read_inode (struct inode *inode,void *data) +static int erofs_read_inode (struct inode *inode,void *data) {struct erofs_inode *vi =EROFS_I (inode);struct erofs_inode_v1 *v1 = data;
@@ -158,7 +158,7 @@ static int erofs_fill_symlink(struct inode *inode, void *data,return 0 ; } -static int fill_inode (struct inode *inode,int isdir) +static int erofs_fill_inode (struct inode *inode,int isdir) {struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb);struct erofs_inode *vi =EROFS_I (inode);
@@ -188,7 +188,7 @@ static int fill_inode(struct inode *inode, int isdir)DBG_BUGON (!PageUptodate (page)); data =page_address (page); - err =read_inode (inode, data + ofs); + err =erofs_read_inode (inode, data + ofs);if (!err) {/* setup the new inode */ switch (inode->i_mode &S_IFMT ) {
@@ -281,7 +281,7 @@ struct inode *erofs_iget(struct super_block *sb, vi->nid = nid; - err =fill_inode (inode, isdir); + err =erofs_fill_inode (inode, isdir);if (!err)unlock_new_inode (inode);else {
fs/erofs/namei.c +5 -5
@@ -14,9 +14,9 @@ struct erofs_qstr { };/* based on the end of qn is accurate and it must have the trailing '\0' */ -static inline int dirnamecmp (const struct erofs_qstr *qn, -const struct erofs_qstr *qd, -unsigned int *matched) +static inline int erofs_dirnamecmp (const struct erofs_qstr *qn, +const struct erofs_qstr *qd, +unsigned int *matched) {unsigned int i = *matched;
@@ -71,7 +71,7 @@ static struct erofs_dirent *find_target_dirent(struct erofs_qstr *name, };/* string comparison without already matched prefix */ -int ret =dirnamecmp (name, &dname, &matched); +int ret =erofs_dirnamecmp (name, &dname, &matched);if (!ret) {return de + mid;
@@ -134,7 +134,7 @@ static struct page *find_target_block_classic(struct inode *dir, EROFS_BLKSIZ);/* string comparison without already matched prefix */ - diff =dirnamecmp (name, &dname, &matched); + diff =erofs_dirnamecmp (name, &dname, &matched);kunmap_atomic (de);if (!diff) {
fs/erofs/super.c +13 -13
@@ -16,12 +16,12 @@static struct kmem_cache *erofs_inode_cachep__read_mostly ; -static void init_once (void *ptr) +static void erofs_inode_init_once (void *ptr) {inode_init_once (&((struct erofs_inode *)ptr)->vfs_inode); } -static struct inode *alloc_inode (struct super_block *sb) +static struct inode *erofs_alloc_inode (struct super_block *sb) {struct erofs_inode *vi =kmem_cache_alloc (erofs_inode_cachep,GFP_KERNEL );
@@ -34,7 +34,7 @@ static struct inode *alloc_inode(struct super_block *sb)return &vi->vfs_inode; } -static void free_inode (struct inode *inode) +static void erofs_free_inode (struct inode *inode) {struct erofs_inode *vi =EROFS_I (inode);
@@ -62,7 +62,7 @@ static bool check_layout_compatibility(struct super_block *sb,return true; } -static int superblock_read (struct super_block *sb) +static int erofs_read_superblock (struct super_block *sb) {struct erofs_sb_info *sbi;struct buffer_head *bh;
@@ -217,7 +217,7 @@ static int erofs_build_cache_strategy(struct erofs_sb_info *sbi,#endif /* set up default EROFS parameters */ -static void default_options (struct erofs_sb_info *sbi) +static void erofs_default_options (struct erofs_sb_info *sbi) {#ifdef CONFIG_EROFS_FS_ZIP sbi->cache_strategy =EROFS_ZIP_CACHE_READAROUND ;
@@ -251,7 +251,7 @@ static match_table_t erofs_tokens = { {Opt_err, NULL} }; -static int parse_options (struct super_block *sb,char *options) +static int erofs_parse_options (struct super_block *sb,char *options) {substring_t args[MAX_OPT_ARGS ];char *p;
@@ -395,7 +395,7 @@ static int erofs_fill_super(struct super_block *sb, void *data, int silent)return -ENOMEM ; sb->s_fs_info = sbi; - err =superblock_read (sb); + err =erofs_read_superblock (sb);if (err)return err;
@@ -409,9 +409,9 @@ static int erofs_fill_super(struct super_block *sb, void *data, int silent) sb->s_xattr = erofs_xattr_handlers;#endif /* set erofs default mount options */ -default_options (sbi); +erofs_default_options (sbi); - err =parse_options (sb, data); + err =erofs_parse_options (sb, data);if (err)return err;
@@ -511,7 +511,7 @@ static int __init erofs_module_init(void) erofs_inode_cachep =kmem_cache_create ("erofs_inode" ,sizeof (struct erofs_inode ),0 ,SLAB_RECLAIM_ACCOUNT , - init_once); + erofs_inode_init_once);if (!erofs_inode_cachep) { err = -ENOMEM ; goto icache_err;
@@ -618,7 +618,7 @@ static int erofs_remount(struct super_block *sb, int *flags, char *data)int err;DBG_BUGON (!sb_rdonly (sb)); - err =parse_options (sb, data); + err =erofs_parse_options (sb, data);if (err) goto out;
@@ -638,8 +638,8 @@ static int erofs_remount(struct super_block *sb, int *flags, char *data)const struct super_operations erofs_sops = { .put_super = erofs_put_super, - .alloc_inode = alloc_inode, - .free_inode = free_inode, + .alloc_inode = erofs_alloc_inode, + .free_inode = erofs_free_inode, .statfs = erofs_statfs, .show_options = erofs_show_options, .remount_fs = erofs_remount,
fs/erofs/zdata.c +10 -9
@@ -40,7 +40,7 @@ void z_erofs_exit_zip_subsystem(void)kmem_cache_destroy (pcluster_cachep); } -static inline int init_unzip_workqueue (void ) +static inline int z_erofs_init_workqueue (void ) {const unsigned int onlinecpus =num_possible_cpus ();const unsigned int flags =WQ_UNBOUND |WQ_HIGHPRI |WQ_CPU_INTENSIVE ;
@@ -54,7 +54,7 @@ static inline int init_unzip_workqueue(void)return z_erofs_workqueue ?0 : -ENOMEM ; } -static void init_once (void *ptr) +static void z_erofs_pcluster_init_once (void *ptr) {struct z_erofs_pcluster *pcl = ptr;struct z_erofs_collection *cl =z_erofs_primarycollection (pcl);
@@ -67,7 +67,7 @@ static void init_once(void *ptr) pcl->compressed_pages[i] =NULL ; } -static void init_always (struct z_erofs_pcluster *pcl) +static void z_erofs_pcluster_init_always (struct z_erofs_pcluster *pcl) {struct z_erofs_collection *cl =z_erofs_primarycollection (pcl);
@@ -81,9 +81,10 @@ int __init z_erofs_init_zip_subsystem(void) { pcluster_cachep =kmem_cache_create ("erofs_compress" ,Z_EROFS_WORKGROUP_SIZE ,0 , -SLAB_RECLAIM_ACCOUNT , init_once); +SLAB_RECLAIM_ACCOUNT , + z_erofs_pcluster_init_once);if (pcluster_cachep) { -if (!init_unzip_workqueue ()) +if (!z_erofs_init_workqueue ())return 0 ;kmem_cache_destroy (pcluster_cachep);
@@ -272,8 +273,8 @@ int erofs_try_to_free_cached_page(struct address_space *mapping, }/* page_type must be Z_EROFS_PAGE_TYPE_EXCLUSIVE */ -static inline bool try_inplace_io (struct z_erofs_collector *clt, -struct page *page) +static inline bool z_erofs_try_inplace_io (struct z_erofs_collector *clt, +struct page *page) {struct z_erofs_pcluster *const pcl = clt->pcl;const unsigned int clusterpages =BIT (pcl->clusterbits);
@@ -296,7 +297,7 @@ static int z_erofs_attach_page(struct z_erofs_collector *clt,/* give priority for inplaceio */ if (clt->mode >=COLLECT_PRIMARY && type ==Z_EROFS_PAGE_TYPE_EXCLUSIVE && -try_inplace_io (clt, page)) +z_erofs_try_inplace_io (clt, page))return 0 ; ret = z_erofs_pagevec_enqueue(&clt->vector,
@@ -409,7 +410,7 @@ static struct z_erofs_collection *clregister(struct z_erofs_collector *clt,if (!pcl)return ERR_PTR (-ENOMEM ); -init_always (pcl); +z_erofs_pcluster_init_always (pcl); pcl->obj.index = map->m_pa >>PAGE_SHIFT ; pcl->length = (map->m_llen << Z_EROFS_PCLUSTER_LENGTH_BIT) |
Signature · 1 line
2.17.1
Re: [PATCH 18/21] erofs: add "erofs_" prefix for common and short functions
Thanks. I don't have a tree with all these applies, but please make sure this covers at least all inlines in headers and all methods wired up to operation structures.
PATCH 19/21 erofs: kill all erofs specific fault injection
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph suggested [1], "Please just use plain kmalloc everywhere and let the normal kernel error injection code take care of injeting any errors."
[1] https://lore.kernel.org/r/[email protected]/
--- Documentation/filesystems/erofs.txt | 5 --- fs/erofs/Kconfig | 7 --- fs/erofs/data.c | 6 --- fs/erofs/inode.c | 3 +- fs/erofs/internal.h | 65 --------------------------- fs/erofs/super.c | 70 ----------------------------- fs/erofs/zdata.c | 8 +--- 7 files changed, 2 insertions(+), 162 deletions(-)
Documentation/filesystems/erofs.txt +0 -5
@@ -52,11 +52,6 @@ linux-erofs mailing list: Mount options ============= -fault_injection=%d Enable fault injection in all supported types with - specified injection rate. Supported injection type: - Type_Name Type_Value - FAULT_KMALLOC 0x000000001 - FAULT_READ_IO 0x000000002 (no)user_xattr Setup Extended User Attributes. Note: xattr is enabled by default if CONFIG_EROFS_FS_XATTR is selected. (no)acl Setup POSIX Access Control List. Note: acl is enabled
fs/erofs/Kconfig +0 -7
@@ -27,13 +27,6 @@ config EROFS_FS_DEBUG For daily use, say N. -config EROFS_FAULT_INJECTION - bool "EROFS fault injection facility" - depends on EROFS_FS - help - Test EROFS to inject faults such as ENOMEM, EIO, and so on. - If unsure, say N. - config EROFS_FS_XATTR bool "EROFS extended attributes" depends on EROFS_FS
fs/erofs/data.c +0 -6
@@ -11,16 +11,10 @@static void erofs_readendio (struct bio *bio) { -struct super_block *const sb = bio->bi_private;struct bio_vec *bvec;blk_status_t err = bio->bi_status;struct bvec_iter_all iter_all; -if (time_to_inject (EROFS_SB (sb),FAULT_READ_IO )) { -erofs_show_injection_info (FAULT_READ_IO ); - err =BLK_STS_IOERR ; - } -bio_for_each_segment_all (bvec, bio, iter_all) {struct page *page = bvec->bv_page;
fs/erofs/inode.c +1 -2
@@ -127,11 +127,10 @@ static int erofs_fill_symlink(struct inode *inode, void *data, unsigned int m_pofs) {struct erofs_inode *vi =EROFS_I (inode); -struct erofs_sb_info *sbi =EROFS_I_SB (inode);/* if it can be handled with fast symlink scheme */ if (is_inode_flat_inline (inode) && inode->i_size <PAGE_SIZE ) { -char *lnk =erofs_kmalloc (sbi, inode->i_size +1 ,GFP_KERNEL ); +char *lnk =kmalloc (inode->i_size +1 ,GFP_KERNEL );if (!lnk)return -ENOMEM ;
fs/erofs/internal.h +0 -65
85 lines not on this page · raw message
fs/erofs/super.c +0 -70
113 lines not on this page · raw message
fs/erofs/zdata.c +1 -7
@@ -724,15 +724,9 @@ static inline void z_erofs_vle_read_endio(struct bio *bio)DBG_BUGON (PageUptodate (page));DBG_BUGON (!page->mapping); -if (!sbi && !z_erofs_page_is_staging (page)) { +if (!sbi && !z_erofs_page_is_staging (page)) sbi =EROFS_SB (page->mapping->host->i_sb); -if (time_to_inject (sbi,FAULT_READ_IO )) { -erofs_show_injection_info (FAULT_READ_IO ); - err =BLK_STS_IOERR ; - } - } -/* sbi should already be gotten if the page is managed */ if (sbi) cachemngd =erofs_page_is_managed (sbi, page);
Signature · 1 line
2.17.1
Re: [PATCH 19/21] erofs: kill all erofs specific fault injection
Quoted 5 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph suggested [1], "Please just use plain kmalloc everywhere and let the normal kernel error injection code take care of injeting any errors."
Thanks.
PATCH 20/21 erofs: kill use_vmap module parameter
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph said [1],
"vm_map_ram is supposed to generally behave better. So if it doesn't please report that that to the arch maintainer and linux-mm so that they can look into the issue. Having user make choices of deep down kernel internals is just a horrible interface.
Please talk to maintainers of other bits of the kernel if you see issues and / or need enhancements. "
Let's redo the previous conclusion and kill the vmap approach.
[1] https://lore.kernel.org/r/[email protected]/
--- Documentation/filesystems/erofs.txt | 4 ---- fs/erofs/decompressor.c | 12 +----------- 2 files changed, 1 insertion(+), 15 deletions(-)
Documentation/filesystems/erofs.txt +0 -4
@@ -67,10 +67,6 @@ cache_strategy=%s Select a strategy for cached decompression from now on: It still does in-place I/O decompression for the rest compressed physical clusters. -Module parameters -================= -use_vmap=[0|1] Use vmap() instead of vm_map_ram() (default 0). - On-disk details ===============
fs/erofs/decompressor.c +1 -11
@@ -28,10 +28,6 @@ struct z_erofs_decompressor {char *name; }; -static bool use_vmap; -module_param (use_vmap, bool,0444 ); -MODULE_PARM_DESC (use_vmap,"Use vmap() instead of vm_map_ram() (default 0)" ); -static int z_erofs_lz4_prepare_destpages (struct z_erofs_decompress_req *rq,struct list_head *pagepool) {
@@ -224,9 +220,6 @@ static void *erofs_vmap(struct page **pages, unsigned int count) {int i =0 ; -if (use_vmap) -return vmap (pages, count,VM_MAP ,PAGE_KERNEL ); -while (1 ) {void *addr =vm_map_ram (pages, count,-1 ,PAGE_KERNEL );
@@ -240,10 +233,7 @@ static void *erofs_vmap(struct page **pages, unsigned int count)static void erofs_vunmap (const void *mem,unsigned int count) { -if (!use_vmap) -vm_unmap_ram (mem, count); -else -vunmap (mem); +vm_unmap_ram (mem, count); } static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq,
Signature · 1 line
2.17.1
Re: [PATCH 20/21] erofs: kill use_vmap module parameter
fs/erofs/decompressor.c static void *erofs_vmap(struct page **pages, unsigned int count) · line 224
Quoted 9 lines
fs/erofs/decompressor.c +0
-3@@ -224,9 +220,6 @@ static void *erofs_vmap(struct page **pages, unsigned int count) {int i =0 ; -if (use_vmap) -return vmap (pages, count,VM_MAP ,PAGE_KERNEL ); -while (1 ) {void *addr =vm_map_ram (pages, count,-1 ,PAGE_KERNEL );
I think you can just open code this in the caller.
fs/erofs/decompressor.c static void *erofs_vmap(struct page **pages, unsigned int count) · line 237
Quoted 8 lines
fs/erofs/decompressor.c +0
-0static void erofs_vunmap (const void *mem,unsigned int count){
fs/erofs/decompressor.c +1
-4-if (!use_vmap) -vm_unmap_ram (mem, count); -else -vunmap (mem); +vm_unmap_ram (mem, count); }
And this wrapper can go away entirely.
And don't forget to report your performance observations to the arm64 maintainers!
Re: [PATCH 20/21] erofs: kill use_vmap module parameter
Hi Christoph,
Quoted 11 lines · Christoph Hellwig
fs/erofs/decompressor.c static void *erofs_vmap(struct page **pages, unsigned int count) · line 224
Quoted 9 lines
fs/erofs/decompressor.c +0
-3@@ -224,9 +220,6 @@ static void *erofs_vmap(struct page **pages, unsigned int count) {int i =0 ; -if (use_vmap) -return vmap (pages, count,VM_MAP ,PAGE_KERNEL ); -while (1 ) {void *addr =vm_map_ram (pages, count,-1 ,PAGE_KERNEL );I think you can just open code this in the caller.
Yes, the only one user... will fix...
Quoted 11 lines
fs/erofs/decompressor.c static void *erofs_vmap(struct page **pages, unsigned int count) · line 237
Quoted 9 lines
fs/erofs/decompressor.c +0
-0static void erofs_vunmap (const void *mem,unsigned int count){
fs/erofs/decompressor.c +1
-4-if (!use_vmap) -vm_unmap_ram (mem, count); -else -vunmap (mem); +vm_unmap_ram (mem, count); }And this wrapper can go away entirely.
Got it. will fix.
Quoted 3 lines
And don't forget to report your performance observations to the arm64 maintainers!
In my observation, vm_map_ram always performs better... If there are something strange later, I will report to them immediately... :)
Thanks,
Gao Xiang
PATCH 21/21 erofs: save one level of indentation
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
As Christoph said [1], ".. and save one
level of indentation."
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/inode.c | 65 ++++++++++++++++++++++++------------------------ 1 file changed, 33 insertions(+), 32 deletions(-)
fs/erofs/inode.c +33 -32
@@ -188,41 +188,42 @@ static int erofs_fill_inode(struct inode *inode, int isdir) data =page_address (page); err =erofs_read_inode (inode, data + ofs); -if (!err) { -/* setup the new inode */ -switch (inode->i_mode &S_IFMT ) { -case S_IFREG : - inode->i_op = &erofs_generic_iops; - inode->i_fop = &generic_ro_fops; -break ; -case S_IFDIR : - inode->i_op = &erofs_dir_iops; - inode->i_fop = &erofs_dir_fops; -break ; -case S_IFLNK : - err =erofs_fill_symlink (inode, data, ofs); -if (err) - goto out_unlock; -inode_nohighmem (inode); -break ; -case S_IFCHR : -case S_IFBLK : -case S_IFIFO : -case S_IFSOCK : - inode->i_op = &erofs_generic_iops; -init_special_inode (inode, inode->i_mode, inode->i_rdev); - goto out_unlock; -default : - err = -EFSCORRUPTED ; +if (err) + goto out_unlock; + +/* setup the new inode */ +switch (inode->i_mode &S_IFMT ) { +case S_IFREG : + inode->i_op = &erofs_generic_iops; + inode->i_fop = &generic_ro_fops; +break ; +case S_IFDIR : + inode->i_op = &erofs_dir_iops; + inode->i_fop = &erofs_dir_fops; +break ; +case S_IFLNK : + err =erofs_fill_symlink (inode, data, ofs); +if (err) goto out_unlock; - } +inode_nohighmem (inode); +break ; +case S_IFCHR : +case S_IFBLK : +case S_IFIFO : +case S_IFSOCK : + inode->i_op = &erofs_generic_iops; +init_special_inode (inode, inode->i_mode, inode->i_rdev); + goto out_unlock; +default : + err = -EFSCORRUPTED ; + goto out_unlock; + } -if (erofs_inode_is_data_compressed (vi->datamode)) { - err =z_erofs_fill_inode (inode); - goto out_unlock; - } - inode->i_mapping->a_ops = &erofs_raw_access_aops; +if (erofs_inode_is_data_compressed (vi->datamode)) { + err =z_erofs_fill_inode (inode); + goto out_unlock; } + inode->i_mapping->a_ops = &erofs_raw_access_aops; out_unlock:unlock_page (page);
Signature · 1 line
2.17.1
Re: [PATCH 21/21] erofs: save one level of indentation
Quoted 4 lines · Gao Xiang
From: Gao Xiang <[email protected]>
As Christoph said [1], ".. and save one
level of indentation."
Thanks. Just a little cleanup, but cumulated things like this really help readability.
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
In reply to Gao XiangQuoted 9 lines · Gao Xiang
Hi,
This patchset is based on the following patch by Pratik Shinde,
https://lore.kernel.org/linux-erofs/[email protected]/All patches addressing Christoph's comments on v6, which are trivial, most deleted code are from erofs specific fault injection, which was followed f2fs and previously discussed in earlier topic [1], but let's follow what Christoph's said now.
I like where the cleanups are going. But this is really just basic code quality stuff. We're not addressing the issues with large amounts of functionality duplicating VFS helpers.
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
Hi Christoph,
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Gao Xiang
Hi,
This patchset is based on the following patch by Pratik Shinde,
https://lore.kernel.org/linux-erofs/[email protected]/
Quoted 6 lines
Quoted 4 lines
All patches addressing Christoph's comments on v6, which are trivial, most deleted code are from erofs specific fault injection, which was followed f2fs and previously discussed in earlier topic [1], but let's follow what Christoph's said now.
I like where the cleanups are going. But this is really just basic
For now, I think it will go to Greg's tree. Before that, I will do patchset v2 to address all remaining comments...
Quoted 2 lines
code quality stuff. We're not addressing the issues with large amounts of functionality duplicating VFS helpers.
You means killing erofs_get_meta_page or avoid erofs_read_raw_page?
- For killing erofs_get_meta_page, here is the current erofs_get_meta_page:
35 struct page *erofs_get_meta_page(struct super_block *sb, erofs_blk_t blkaddr) 36 {
37 struct inode *const bd_inode = sb->s_bdev->bd_inode;
38 struct address_space *const mapping = bd_inode->i_mapping;
39 /* prefer retrying in the allocator to blindly looping below */
40 const gfp_t gfp = mapping_gfp_constraint(mapping, ~__GFP_FS);
41 struct page *page;
42 int err;
43
44 repeat:
45 page = find_or_create_page(mapping, blkaddr, gfp);
46 if (!page)
47 return ERR_PTR(-ENOMEM);
48
49 DBG_BUGON(!PageLocked(page));
50
51 if (!PageUptodate(page)) {
52 struct bio *bio;
53
54 bio = erofs_grab_bio(sb, REQ_OP_READ | REQ_META,
55 blkaddr, 1, sb, erofs_readendio);
56
57 if (bio_add_page(bio, page, PAGE_SIZE, 0) != PAGE_SIZE) {
58 err = -EFAULT;
59 goto err_out;
60 }
61
62 submit_bio(bio);
63 lock_page(page);
64
65 /* this page has been truncated by others */
66 if (page->mapping != mapping) {
67 unlock_page(page);
68 put_page(page);
69 goto repeat;
70 }
71
72 /* more likely a read error */
73 if (!PageUptodate(page)) {
74 err = -EIO;
75 goto err_out;
76 }
77 }
78 return page;
79
80 err_out:
81 unlock_page(page);
82 put_page(page);
83 return ERR_PTR(err);84 }
I think it is simple enough. read_cache_page need write a similar filler, or read_cache_page_gfp will call .readpage, and then introduce buffer_heads, that is what I'd like to avoid now (no need these bd_inode buffer_heads in memory...)
- For erofs_read_raw_page, it can be avoided after iomap tail-end packing feature is done... If we remove it now, it will make EROFS broken. It is no urgent and Chao will focus on iomap tail-end packing feature.
Thanks,
Gao Xiang
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
Quoted 11 lines · Gao Xiang
Quoted 2 lines
code quality stuff. We're not addressing the issues with large amounts of functionality duplicating VFS helpers.
You means killing
erofs_get_meta_pageor avoiderofs_read_raw_page?- For killing
erofs_get_meta_page, here is the currenterofs_get_meta_page:I think it is simple enough.
read_cache_pageneed write a similar filler, orread_cache_page_gfpwill call .readpage, and then introducebuffer_heads, that is what I'd like to avoid now (no need thesebd_inodebuffer_headsin memory...)
If using read_cache_page_gfp and ->readpage works, please do. The fact that the block device inode uses buffer heads is an implementation detail that might not last very long and should be invisible to you. It also means you can get rid of a lot of code that you don't have to maintain and others don't have to update for global API changes.
Quoted 3 lines
- For
erofs_read_raw_page, it can be avoided after iomap tail-end packing feature is done... If we remove it now, it will make EROFS broken. It is no urgent and Chao will focus on iomap tail-end packing feature.
Ok. I wish we would have just sorted this out beforehand, which we could have trivially done without all that staging mess.
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
Hi Christoph,
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Gao Xiang
Quoted 2 lines
code quality stuff. We're not addressing the issues with large amounts of functionality duplicating VFS helpers.
You means killing
erofs_get_meta_pageor avoiderofs_read_raw_page?- For killing
erofs_get_meta_page, here is the currenterofs_get_meta_page:I think it is simple enough.
read_cache_pageneed write a similar filler, orread_cache_page_gfpwill call .readpage, and then introducebuffer_heads, that is what I'd like to avoid now (no need thesebd_inodebuffer_headsin memory...)
Quoted 5 lines
If using
read_cache_page_gfpand ->readpage works, please do. The fact that the block device inode uses buffer heads is an implementation detail that might not last very long and should be invisible to you. It also means you can get rid of a lot of code that you don't have to maintain and others don't have to update for global API changes.
I care about those useless buffer_heads in memory for our products...
Since we are nobh filesystem (a little request, could I use it after buffer_heads are fully avoided, I have no idea why I need those buffer_heads in memory.... But I think bd_inode is good for caching metadata...)
Quoted 7 lines
Quoted 4 lines
- For
erofs_read_raw_page, it can be avoided after iomap tail-end packing feature is done... If we remove it now, it will make EROFS broken. It is no urgent and Chao will focus on iomap tail-end packing feature.Ok. I wish we would have just sorted this out beforehand, which we could have trivially done without all that staging mess.
Firstly, I'd like to introduce EROFS as a self-contained filesystem to introduce new fixed-sized output compression to upstream and promote it...
And then we can do many improvements for EROFS in parallel... (if we introduce EROFS and touch many core modules like iomap, mm readahead code or modify LZ4 code at once... It could be more careful... Let's improve it step-by-step... We are a dedicated team if the Linux community needs us, we will still here... It will be actively maintained.)
Thanks,
Gao Xiang
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
Quoted 15 lines · Gao Xiang
Quoted 9 lines
Quoted 8 lines
You means killing
erofs_get_meta_pageor avoiderofs_read_raw_page?- For killing
erofs_get_meta_page, here is the currenterofs_get_meta_page:I think it is simple enough.
read_cache_pageneed write a similar filler, orread_cache_page_gfpwill call .readpage, and then introducebuffer_heads, that is what I'd like to avoid now (no need thesebd_inodebuffer_headsin memory...)Quoted 5 lines
If using
read_cache_page_gfpand ->readpage works, please do. The fact that the block device inode uses buffer heads is an implementation detail that might not last very long and should be invisible to you. It also means you can get rid of a lot of code that you don't have to maintain and others don't have to update for global API changes.
Quoted 6 lines
I care about those useless
buffer_headsin memory for our products...Since we are nobh filesystem (a little request, could I use it after
buffer_headsare fully avoided, I have no idea why I need thosebuffer_headsin memory.... But I thinkbd_inodeis good for caching metadata...)
Then please use read_cache_page with iomap_readpage(s), and write comment explaining why your are not using read_cache_page_gfp.
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
Hi Christoph,
Quoted 18 lines · Christoph Hellwig
Quoted 17 lines · Gao Xiang
Quoted 9 lines
Quoted 8 lines
You means killing
erofs_get_meta_pageor avoiderofs_read_raw_page?- For killing
erofs_get_meta_page, here is the currenterofs_get_meta_page:I think it is simple enough.
read_cache_pageneed write a similar filler, orread_cache_page_gfpwill call .readpage, and then introducebuffer_heads, that is what I'd like to avoid now (no need thesebd_inodebuffer_headsin memory...)Quoted 5 lines
If using
read_cache_page_gfpand ->readpage works, please do. The fact that the block device inode uses buffer heads is an implementation detail that might not last very long and should be invisible to you. It also means you can get rid of a lot of code that you don't have to maintain and others don't have to update for global API changes.I care about those useless
buffer_headsin memory for our products...
Quoted 7 lines
Quoted 4 lines
Since we are nobh filesystem (a little request, could I use it after
buffer_headsare fully avoided, I have no idea why I need thosebuffer_headsin memory.... But I thinkbd_inodeis good for caching metadata...)Then please use
read_cache_pagewithiomap_readpage(s), and write comment explaining why your are not usingread_cache_page_gfp.
I implement a prelimitary version, but I have no idea it is a really cleanup for now.
From 001e3e64c81e4ced0d22b147e6abf90060e704b5 Mon Sep 17 00:00:00 2001
From: Gao Xiang <[email protected]>
Date: Tue, 3 Sep 2019 16:13:00 +0800
Subject: [PATCH] erofs: use iomap_readpage for erofs_get_meta_page
--- fs/erofs/Kconfig | 1 + fs/erofs/data.c | 91 ++++++++++++++++++++++++++---------------------- 2 files changed, 51 insertions(+), 41 deletions(-)
fs/erofs/Kconfig +1 -0
@@ -3,6 +3,7 @@ config EROFS_FS tristate "EROFS filesystem support" depends on BLOCK + select FS_IOMAP help EROFS (Enhanced Read-Only File System) is a lightweight read-only file system with modern designs (eg. page-sized
fs/erofs/data.c +50 -41
110 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
Quoted 2 lines · Gao Xiang
I implement a prelimitary version, but I have no idea it is a really cleanup for now.
The fact that this has to guess the block device address_space implementation is indeed pretty ugly. I'd much prefer to just use read_cache_page_gfp, and live with the fact that this allocates bufferheads behind you for now. I'll try to speed up my attempts to get rid of the buffer heads on the block device mapping instead.
Re: [PATCH 00/21] erofs: patchset addressing Christoph's comments
Hi Christoph,
Quoted 9 lines · Christoph Hellwig
Quoted 2 lines · Gao Xiang
I implement a prelimitary version, but I have no idea it is a really cleanup for now.
The fact that this has to guess the block device
address_spaceimplementation is indeed pretty ugly. I'd much prefer to just useread_cache_page_gfp, and live with the fact that this allocates bufferheads behind you for now. I'll try to speed up my attempts to get rid of the buffer heads on the block device mapping instead.
Fully agree with your point. Let me use read_cache_page_gfp instead for now, hoping block device quickly avoiding from buffer_heads...
Thanks,
Gao Xiang
v2 cover erofs: patchset addressing Christoph's comments
In reply to Gao XiangHi,
This patchset is based on the following patch by Pratik Shinde,
https://lore.kernel.org/linux-erofs/[email protected]/
All patches addressing Christoph's comments on v6, which are trivial, most deleted code are from erofs specific fault injection, which was followed f2fs and previously discussed in earlier topic [1], but let's follow what Christoph's said now.
Comments and suggestions are welcome...
[1] https://lore.kernel.org/r/[email protected]/
changes since v1:
- leave some comments near the numbers to indicate where they are stored;
- avoid a u8 cast;
- use erofs_{err,info,dbg} and print sb->s_id as a prefix before the actual message;
- add a on-disk title in erofs_fs.h
- use feature_{compat,incompat} rather than features and requirements;
- suggestions on erofs_grab_bio:
https://lore.kernel.org/r/[email protected]/
- use compact/extended instead of erofs_inode_v1/v2 and i_format instead of i_advise;
- avoid chained if/else if/else if statements in erofs_read_inode;
- avoid erofs_vmap/vunmap wrappers;
- use read_cache_page_gfp for erofs_get_meta_page;
Gao Xiang (25):
erofs: remove all the byte offset comments
erofs: on-disk format should have explicitly assigned numbers
erofs: some macros are much more readable as a function
erofs: kill __packed for on-disk structures
erofs: update erofs_inode_is_data_compressed helper
erofs: use feature_incompat rather than requirements
erofs: better naming for erofs inode related stuffs
erofs: kill erofs_{init,exit}_inode_cache
erofs: use erofs_inode naming
erofs: update erofs_fs.h comments
erofs: update comments in inode.c
erofs: better erofs symlink stuffs
erofs: use dsb instead of layout for ondisk super_block
erofs: kill verbose debug info in erofs_fill_super
erofs: localize erofs_grab_bio()
erofs: kill prio and nofail of erofs_get_meta_page()
erofs: kill __submit_bio()
erofs: add "erofs_" prefix for common and short functions
erofs: kill all erofs specific fault injection
erofs: kill use_vmap module parameter
erofs: save one level of indentation
erofs: rename errln/infoln/debugln to erofs_{err,info,dbg}
erofs: use read_mapping_page instead of sb_bread
erofs: always use iget5_locked
erofs: use read_cache_page_gfp for erofs_get_meta_page
Documentation/filesystems/erofs.txt | 9 - fs/erofs/Kconfig | 7 - fs/erofs/data.c | 118 +++-------- fs/erofs/decompressor.c | 76 +++---- fs/erofs/dir.c | 17 +- fs/erofs/erofs_fs.h | 197 +++++++++--------- fs/erofs/inode.c | 297 ++++++++++++++-------------- fs/erofs/internal.h | 192 ++++-------------- fs/erofs/namei.c | 21 +- fs/erofs/super.c | 282 +++++++++++--------------- fs/erofs/xattr.c | 41 ++-- fs/erofs/xattr.h | 4 +- fs/erofs/zdata.c | 63 +++--- fs/erofs/zdata.h | 2 +- fs/erofs/zmap.c | 73 +++---- include/trace/events/erofs.h | 14 +- 16 files changed, 578 insertions(+), 835 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/25 erofs: remove all the byte offset comments
As Christoph suggested [1], "Please remove all the byte offset comments. that is something that can easily be checked with gdb or pahole."
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 105 +++++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 51 deletions(-)
fs/erofs/erofs_fs.h +54 -51
@@ -17,27 +17,28 @@#define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING +/* 128-byte erofs on-disk super block */ struct erofs_super_block { -/* 0 */ __le32 magic;/* in the little endian */ -/* 4 */ __le32 checksum;/* crc32c(super_block) */ -/* 8 */ __le32 features;/* (aka. feature_compat) */ -/* 12 */ __u8 blkszbits;/* support block_size == PAGE_SIZE only */ -/* 13 */ __u8 reserved; - -/* 14 */ __le16 root_nid; -/* 16 */ __le64 inos;/* total valid ino # (== f_files - f_favail) */ - -/* 24 */ __le64 build_time;/* inode v1 time derivation */ -/* 32 */ __le32 build_time_nsec; -/* 36 */ __le32 blocks;/* used for statfs */ -/* 40 */ __le32 meta_blkaddr; -/* 44 */ __le32 xattr_blkaddr; -/* 48 */ __u8 uuid[16 ];/* 128-bit uuid for volume */ -/* 64 */ __u8 volume_name[16 ];/* volume name */ -/* 80 */ __le32 requirements;/* (aka. feature_incompat) */ - -/* 84 */ __u8 reserved2[44 ]; -} __packed;/* 128 bytes */ +__le32 magic;/* file system magic number */ +__le32 checksum;/* crc32c(super_block) */ +__le32 features;/* (aka. feature_compat) */ +__u8 blkszbits;/* support block_size == PAGE_SIZE only */ +__u8 reserved; + +__le16 root_nid;/* nid of root directory */ +__le64 inos;/* total valid ino # (== f_files - f_favail) */ + +__le64 build_time;/* inode v1 time derivation */ +__le32 build_time_nsec;/* inode v1 time derivation in nano scale */ +__le32 blocks;/* used for statfs */ +__le32 meta_blkaddr;/* start block address of metadata area */ +__le32 xattr_blkaddr;/* start block address of shared xattr area */ +__u8 uuid[16 ];/* 128-bit uuid for volume */ +__u8 volume_name[16 ];/* volume name */ +__le32 requirements;/* (aka. feature_incompat) */ + +__u8 reserved2[44 ]; +} __packed;/* * erofs inode data mapping:
@@ -73,16 +74,17 @@ static inline bool erofs_inode_is_data_compressed(unsigned int datamode)#define EROFS_I_VERSION_BIT 0#define EROFS_I_DATA_MAPPING_BIT 1 +/* 32-byte reduced form of an ondisk inode */ struct erofs_inode_v1 { -/* 0 */ __le16 i_advise; +__le16 i_advise;/* inode hints */ /* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ -/* 2 */ __le16 i_xattr_icount; -/* 4 */ __le16 i_mode; -/* 6 */ __le16 i_nlink; -/* 8 */ __le32 i_size; -/* 12 */ __le32 i_reserved; -/* 16 */ union { +__le16 i_xattr_icount; +__le16 i_mode; +__le16 i_nlink; +__le32 i_size; +__le32 i_reserved; +union {/* file total compressed blocks for data mapping 1 */ __le32 compressed_blocks;__le32 raw_blkaddr;
@@ -90,10 +92,10 @@ struct erofs_inode_v1 {/* for device files, used to indicate old/new device # */ __le32 rdev; }i_u __packed; -/* 20 */ __le32 i_ino;/* only used for 32-bit stat compatibility */ -/* 24 */ __le16 i_uid; -/* 26 */ __le16 i_gid; -/* 28 */ __le32 i_reserved2; +__le32 i_ino;/* only used for 32-bit stat compatibility */ +__le16 i_uid; +__le16 i_gid; +__le32 i_reserved2; } __packed;/* 32 bytes on-disk inode */
@@ -101,15 +103,16 @@ struct erofs_inode_v1 {/* 64 bytes on-disk inode */ #define EROFS_INODE_LAYOUT_V2 1 +/* 64-byte complete form of an ondisk inode */ struct erofs_inode_v2 { -/* 0 */ __le16 i_advise; +__le16 i_advise;/* inode hints */ /* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ -/* 2 */ __le16 i_xattr_icount; -/* 4 */ __le16 i_mode; -/* 6 */ __le16 i_reserved; -/* 8 */ __le64 i_size; -/* 16 */ union { +__le16 i_xattr_icount; +__le16 i_mode; +__le16 i_reserved; +__le64 i_size; +union {/* file total compressed blocks for data mapping 1 */ __le32 compressed_blocks;__le32 raw_blkaddr;
@@ -119,15 +122,15 @@ struct erofs_inode_v2 { }i_u __packed;/* only used for 32-bit stat compatibility */ -/* 20 */ __le32 i_ino; - -/* 24 */ __le32 i_uid; -/* 28 */ __le32 i_gid; -/* 32 */ __le64 i_ctime; -/* 40 */ __le32 i_ctime_nsec; -/* 44 */ __le32 i_nlink; -/* 48 */ __u8 i_reserved2[16 ]; -} __packed;/* 64 bytes */ +__le32 i_ino; + +__le32 i_uid; +__le32 i_gid; +__le64 i_ctime; +__le32 i_ctime_nsec; +__le32 i_nlink; +__u8 i_reserved2[16 ]; +} __packed;#define EROFS_MAX_SHARED_XATTRS (128)/* h_shared_count between 129 ... 255 are special # */
@@ -264,7 +267,7 @@ struct z_erofs_vle_decompressed_index {* [1] - pointing to the tail cluster */ __le16 delta[2 ]; - }di_u __packed;/* 8 bytes */ + }di_u __packed; } __packed;#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN (size) \
@@ -273,10 +276,10 @@ struct z_erofs_vle_decompressed_index {/* dirent sorts in alphabet order, thus we can do binary search */ struct erofs_dirent { -__le64 nid;/* 0, node number */ -__le16 nameoff;/* 8, start offset of file name */ -__u8 file_type;/* 10, file type */ -__u8 reserved;/* 11, reserved */ +__le64 nid;/* node number */ +__le16 nameoff;/* start offset of file name */ +__u8 file_type;/* file type */ +__u8 reserved;/* reserved */ } __packed;/*
Signature · 1 line
2.17.1
PATCH v2 2/25 erofs: on-disk format should have explicitly assigned numbers
In reply to Gao XiangAs Christoph suggested [1], on-disk format should have explicitly assigned numbers.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
fs/erofs/erofs_fs.h +9 -9
@@ -53,10 +53,10 @@ struct erofs_super_block {* 4~7 - reserved */ enum { -EROFS_INODE_FLAT_PLAIN , -EROFS_INODE_FLAT_COMPRESSION_LEGACY , -EROFS_INODE_FLAT_INLINE , -EROFS_INODE_FLAT_COMPRESSION , +EROFS_INODE_FLAT_PLAIN =0 , +EROFS_INODE_FLAT_COMPRESSION_LEGACY =1 , +EROFS_INODE_FLAT_INLINE =2 , +EROFS_INODE_FLAT_COMPRESSION =3 ,EROFS_INODE_LAYOUT_MAX };
@@ -184,7 +184,7 @@ struct erofs_xattr_entry {/* available compression algorithm types */ enum { -Z_EROFS_COMPRESSION_LZ4 , +Z_EROFS_COMPRESSION_LZ4 =0 ,Z_EROFS_COMPRESSION_MAX };
@@ -242,10 +242,10 @@ struct z_erofs_map_header {* (di_advise could be 0, 1 or 2) */ enum { -Z_EROFS_VLE_CLUSTER_TYPE_PLAIN , -Z_EROFS_VLE_CLUSTER_TYPE_HEAD , -Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD , -Z_EROFS_VLE_CLUSTER_TYPE_RESERVED , +Z_EROFS_VLE_CLUSTER_TYPE_PLAIN =0 , +Z_EROFS_VLE_CLUSTER_TYPE_HEAD =1 , +Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD =2 , +Z_EROFS_VLE_CLUSTER_TYPE_RESERVED =3 ,Z_EROFS_VLE_CLUSTER_TYPE_MAX };
Signature · 1 line
2.17.1
PATCH v2 3/25 erofs: some macros are much more readable as a function
In reply to Gao XiangAs Christoph suggested [1], these macros are much
more readable as a function.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 22 ++++++++++++++-------- fs/erofs/inode.c | 4 ++-- fs/erofs/xattr.c | 2 +- 3 files changed, 17 insertions(+), 11 deletions(-)
fs/erofs/erofs_fs.h +14 -8
@@ -171,16 +171,22 @@ struct erofs_xattr_entry {char e_name[0 ];/* attribute name */ } __packed; -#define ondisk_xattr_ibody_size (count) ({\ - u32 __count = le16_to_cpu(count); \ - ((__count) == 0) ? 0 : \ - sizeof(struct erofs_xattr_ibody_header) + \ - sizeof(__u32) * ((__count) - 1); }) +static inline unsigned int erofs_xattr_ibody_size (__le16 i_xattr_icount) +{ +if (!i_xattr_icount) +return 0 ; + +return sizeof (struct erofs_xattr_ibody_header ) + +sizeof (__u32) * (le16_to_cpu (i_xattr_icount) -1 ); +}#define EROFS_XATTR_ALIGN (size) round_up(size, sizeof(struct erofs_xattr_entry)) -#define EROFS_XATTR_ENTRY_SIZE (entry) EROFS_XATTR_ALIGN( \ - sizeof(struct erofs_xattr_entry) + \ - (entry)->e_name_len + le16_to_cpu((entry)->e_value_size)) + +static inline unsigned int erofs_xattr_entry_size (struct erofs_xattr_entry *e) +{ +return EROFS_XATTR_ALIGN (sizeof (struct erofs_xattr_entry ) + + e->e_name_len +le16_to_cpu (e->e_value_size)); +}/* available compression algorithm types */ enum {
fs/erofs/inode.c +2 -2
@@ -29,7 +29,7 @@ static int read_inode(struct inode *inode, void *data)struct erofs_inode_v2 *v2 = data; vi->inode_isize =sizeof (struct erofs_inode_v2 ); - vi->xattr_isize =ondisk_xattr_ibody_size (v2->i_xattr_icount); + vi->xattr_isize =erofs_xattr_ibody_size (v2->i_xattr_icount); inode->i_mode =le16_to_cpu (v2->i_mode); if (S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
@@ -62,7 +62,7 @@ static int read_inode(struct inode *inode, void *data)struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); vi->inode_isize =sizeof (struct erofs_inode_v1 ); - vi->xattr_isize =ondisk_xattr_ibody_size (v1->i_xattr_icount); + vi->xattr_isize =erofs_xattr_ibody_size (v1->i_xattr_icount); inode->i_mode =le16_to_cpu (v1->i_mode); if (S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
fs/erofs/xattr.c +1 -1
@@ -231,7 +231,7 @@ static int xattr_foreach(struct xattr_iter *it,*/ entry = *(struct erofs_xattr_entry *)(it->kaddr + it->ofs);if (tlimit) { -unsigned int entry_sz =EROFS_XATTR_ENTRY_SIZE (&entry); +unsigned int entry_sz =erofs_xattr_entry_size (&entry);/* xattr on-disk corruption: xattr entry beyond xattr_isize */ if (*tlimit < entry_sz) {
Signature · 1 line
2.17.1
PATCH v2 4/25 erofs: kill __packed for on-disk structures
In reply to Gao XiangAs Christoph suggested "Please don't add __packed" [1], remove all __packed except struct erofs_dirent here.
Note that all on-disk fields except struct erofs_dirent (12 bytes with a 8-byte nid) in EROFS are naturally aligned.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
fs/erofs/erofs_fs.h +9 -9
@@ -38,7 +38,7 @@ struct erofs_super_block {__le32 requirements;/* (aka. feature_incompat) */ __u8 reserved2[44 ]; -} __packed; +};/* * erofs inode data mapping:
@@ -91,12 +91,12 @@ struct erofs_inode_v1 {/* for device files, used to indicate old/new device # */ __le32 rdev; - }i_u __packed; + } i_u;__le32 i_ino;/* only used for 32-bit stat compatibility */ __le16 i_uid;__le16 i_gid;__le32 i_reserved2; -} __packed; +};/* 32 bytes on-disk inode */ #define EROFS_INODE_LAYOUT_V1 0
@@ -119,7 +119,7 @@ struct erofs_inode_v2 {/* for device files, used to indicate old/new device # */ __le32 rdev; - }i_u __packed; + } i_u;/* only used for 32-bit stat compatibility */ __le32 i_ino;
@@ -130,7 +130,7 @@ struct erofs_inode_v2 {__le32 i_ctime_nsec;__le32 i_nlink;__u8 i_reserved2[16 ]; -} __packed; +};#define EROFS_MAX_SHARED_XATTRS (128)/* h_shared_count between 129 ... 255 are special # */
@@ -152,7 +152,7 @@ struct erofs_xattr_ibody_header {__u8 h_shared_count;__u8 h_reserved2[7 ];__le32 h_shared_xattrs[0 ];/* shared xattr id array */ -} __packed; +};/* Name indexes */ #define EROFS_XATTR_INDEX_USER 1
@@ -169,7 +169,7 @@ struct erofs_xattr_entry {__le16 e_value_size;/* size of attribute value */ /* followed by e_name and e_value */ char e_name[0 ];/* attribute name */ -} __packed; +};static inline unsigned int erofs_xattr_ibody_size (__le16 i_xattr_icount) {
@@ -273,8 +273,8 @@ struct z_erofs_vle_decompressed_index {* [1] - pointing to the tail cluster */ __le16 delta[2 ]; - }di_u __packed; -} __packed; + } di_u; +};#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN (size) \ (round_up(size, sizeof(struct z_erofs_vle_decompressed_index)) + \
Signature · 1 line
2.17.1
PATCH v2 5/25 erofs: update erofs_inode_is_data_compressed helper
In reply to Gao XiangAs Christoph said, "This looks like a really obsfucated way to write:
return datamode == EROFS_INODE_FLAT_COMPRESSION || datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY; "
Although I had my own consideration, it's the right way for now.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/erofs_fs.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
fs/erofs/erofs_fs.h +2 -3
@@ -62,9 +62,8 @@ enum {static inline bool erofs_inode_is_data_compressed (unsigned int datamode) { -if (datamode ==EROFS_INODE_FLAT_COMPRESSION ) -return true; -return datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; +return datamode ==EROFS_INODE_FLAT_COMPRESSION || + datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ; }/* bit definitions of inode i_advise */
Signature · 1 line
2.17.1
PATCH v2 6/25 erofs: use feature_incompat rather than requirements
In reply to Gao XiangAs Christoph said [1], "This is only cosmetic, why not stick to feature_compat and feature_incompat?"
In my thought, requirements means "incompatible" instead of "feature" though.
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/decompressor.c | 3 ++- fs/erofs/erofs_fs.h | 12 ++++++------ fs/erofs/internal.h | 2 +- fs/erofs/super.c | 10 +++++----- 4 files changed, 14 insertions(+), 13 deletions(-)
fs/erofs/decompressor.c +2 -1
@@ -129,7 +129,8 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) support_0padding = false;/* decompression inplace is only safe when 0padding is enabled */ -if (EROFS_SB (rq->sb)->requirements &EROFS_REQUIREMENT_LZ4_0PADDING ) { +if (EROFS_SB (rq->sb)->feature_incompat & +EROFS_FEATURE_INCOMPAT_LZ4_0PADDING ) { support_0padding = true;while (!src[inputmargin & ~PAGE_MASK ])
fs/erofs/erofs_fs.h +6 -6
@@ -11,17 +11,17 @@#define EROFS_SUPER_OFFSET 1024/* -* Any bits that aren't in EROFS_ALL_REQUIREMENTS should be -* incompatible with this kernel version. +* Any bits that aren't in EROFS_ALL_FEATURE_INCOMPAT should +* be incompatible with this kernel version. */ -#define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001 -#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING +#define EROFS_FEATURE_INCOMPAT_LZ4_0PADDING 0x00000001 +#define EROFS_ALL_FEATURE_INCOMPAT EROFS_FEATURE_INCOMPAT_LZ4_0PADDING/* 128-byte erofs on-disk super block */ struct erofs_super_block {__le32 magic;/* file system magic number */ __le32 checksum;/* crc32c(super_block) */ -__le32 features;/* (aka. feature_compat) */ +__le32 feature_compat;__u8 blkszbits;/* support block_size == PAGE_SIZE only */ __u8 reserved;
@@ -35,7 +35,7 @@ struct erofs_super_block {__le32 xattr_blkaddr;/* start block address of shared xattr area */ __u8 uuid[16 ];/* 128-bit uuid for volume */ __u8 volume_name[16 ];/* volume name */ -__le32 requirements;/* (aka. feature_incompat) */ +__le32 feature_incompat;__u8 reserved2[44 ]; };
fs/erofs/internal.h +1 -1
@@ -96,7 +96,7 @@ struct erofs_sb_info {u8 uuid[16 ];/* 128-bit uuid for volume */ u8 volume_name[16 ];/* volume name */ -u32 requirements; +u32 feature_incompat;unsigned int mount_opt;
fs/erofs/super.c +5 -5
@@ -67,14 +67,14 @@ static void free_inode(struct inode *inode)static bool check_layout_compatibility (struct super_block *sb,struct erofs_super_block *layout) { -const unsigned int requirements =le32_to_cpu (layout->requirements); +const unsigned int feature =le32_to_cpu (layout->feature_incompat); -EROFS_SB (sb)->requirements = requirements; +EROFS_SB (sb)->feature_incompat = feature;/* check if current kernel meets all mandatory requirements */ -if (requirements & (~EROFS_ALL_REQUIREMENTS )) { -errln ("unidentified requirements %x, please upgrade kernel version" , - requirements & ~EROFS_ALL_REQUIREMENTS ); +if (feature & (~EROFS_ALL_FEATURE_INCOMPAT )) { +errln ("unidentified incompatible feature %x, please upgrade kernel version" , + feature & ~EROFS_ALL_FEATURE_INCOMPAT );return false; }return true;
Signature · 1 line
2.17.1
PATCH v2 7/25 erofs: better naming for erofs inode related stuffs
In reply to Gao Xiangupdates inode naming
- kill is_inode_layout_compression [1]
- kill magic underscores [2] [3]
- better naming for datamode & data_mapping_mode [3]
- better naming erofs_inode_{compact, extended} [4]
[1] https://lore.kernel.org/r/[email protected]/
[2] https://lore.kernel.org/r/[email protected]/
[3] https://lore.kernel.org/r/[email protected]/
[4] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/data.c | 9 ++-- fs/erofs/erofs_fs.h | 24 ++++----- fs/erofs/inode.c | 126 +++++++++++++++++++++++++------------------- fs/erofs/internal.h | 31 ++++++----- fs/erofs/super.c | 2 +- fs/erofs/zmap.c | 6 +-- 6 files changed, 108 insertions(+), 90 deletions(-)
fs/erofs/data.c +5 -4
@@ -113,11 +113,12 @@ static int erofs_map_blocks_flatmode(struct inode *inode,erofs_blk_t nblocks, lastblk;u64 offset = map->m_la;struct erofs_vnode *vi =EROFS_V (inode); +bool tailendpacking = (vi->datalayout ==EROFS_INODE_FLAT_INLINE );trace_erofs_map_blocks_flatmode_enter (inode, map, flags); nblocks =DIV_ROUND_UP (inode->i_size,PAGE_SIZE ); - lastblk = nblocks -is_inode_flat_inline (inode); + lastblk = nblocks - tailendpacking;if (offset >= inode->i_size) {/* leave out-of-bound access unmapped */
@@ -132,7 +133,7 @@ static int erofs_map_blocks_flatmode(struct inode *inode,if (offset <blknr_to_addr (lastblk)) { map->m_pa =blknr_to_addr (vi->raw_blkaddr) + map->m_la; map->m_plen =blknr_to_addr (lastblk) - offset; - }else if (is_inode_flat_inline (inode)) { + }else if (tailendpacking) {/* 2 - inode inline B: inode, [xattrs], inline last blk... */ struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb);
@@ -169,7 +170,7 @@ static int erofs_map_blocks_flatmode(struct inode *inode,int erofs_map_blocks (struct inode *inode,struct erofs_map_blocks *map,int flags) { -if (is_inode_layout_compression (inode)) { +if (erofs_inode_is_data_compressed (EROFS_V (inode)->datalayout)) {int err =z_erofs_map_blocks_iter (inode, map, flags);if (map->mpage) {
@@ -403,7 +404,7 @@ static sector_t erofs_bmap(struct address_space *mapping, sector_t block) {struct inode *inode = mapping->host; -if (is_inode_flat_inline (inode)) { +if (EROFS_V (inode)->datalayout ==EROFS_INODE_FLAT_INLINE ) {erofs_blk_t blks =i_size_read (inode) >>LOG_BLOCK_SIZE ;if (block >>LOG_SECTORS_PER_BLOCK >= blks)
fs/erofs/erofs_fs.h +12 -12
@@ -41,7 +41,7 @@ struct erofs_super_block { };/* -* erofs inode data mapping: +* erofs inode datalayout: * 0 - inode plain without inline data A: * inode, [xattrs], ... | ... | no-holed data * 1 - inode VLE compression B (legacy):
@@ -57,7 +57,7 @@ enum {EROFS_INODE_FLAT_COMPRESSION_LEGACY =1 ,EROFS_INODE_FLAT_INLINE =2 ,EROFS_INODE_FLAT_COMPRESSION =3 , - EROFS_INODE_LAYOUT_MAX + EROFS_INODE_DATALAYOUT_MAX };static inline bool erofs_inode_is_data_compressed (unsigned int datamode)
@@ -68,14 +68,14 @@ static inline bool erofs_inode_is_data_compressed(unsigned int datamode)/* bit definitions of inode i_advise */ #define EROFS_I_VERSION_BITS 1 -#define EROFS_I_DATA_MAPPING_BITS 3 +#define EROFS_I_DATALAYOUT_BITS 3#define EROFS_I_VERSION_BIT 0 -#define EROFS_I_DATA_MAPPING_BIT 1 +#define EROFS_I_DATALAYOUT_BIT 1/* 32-byte reduced form of an ondisk inode */ -struct erofs_inode_v1 { -__le16 i_advise;/* inode hints */ +struct erofs_inode_compact { +__le16 i_format;/* inode format hints */ /* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ __le16 i_xattr_icount;
@@ -98,13 +98,13 @@ struct erofs_inode_v1 { };/* 32 bytes on-disk inode */ -#define EROFS_INODE_LAYOUT_V1 0 +#define EROFS_INODE_LAYOUT_COMPACT 0/* 64 bytes on-disk inode */ -#define EROFS_INODE_LAYOUT_V2 1 +#define EROFS_INODE_LAYOUT_EXTENDED 1/* 64-byte complete form of an ondisk inode */ -struct erofs_inode_v2 { -__le16 i_advise;/* inode hints */ +struct erofs_inode_extended { +__le16 i_format;/* inode format hints */ /* 1 header + n-1 * 4 bytes inline xattr to keep continuity */ __le16 i_xattr_icount;
@@ -299,8 +299,8 @@ struct erofs_dirent {static inline void erofs_check_ondisk_layout_definitions (void ) {BUILD_BUG_ON (sizeof (struct erofs_super_block ) !=128 ); -BUILD_BUG_ON (sizeof (struct erofs_inode_v1 ) !=32 ); -BUILD_BUG_ON (sizeof (struct erofs_inode_v2 ) !=64 ); +BUILD_BUG_ON (sizeof (struct erofs_inode_compact ) !=32 ); +BUILD_BUG_ON (sizeof (struct erofs_inode_extended ) !=64 );BUILD_BUG_ON (sizeof (struct erofs_xattr_ibody_header ) !=12 );BUILD_BUG_ON (sizeof (struct erofs_xattr_entry ) !=4 );BUILD_BUG_ON (sizeof (struct z_erofs_map_header ) !=8 );
fs/erofs/inode.c +72 -54
179 lines not on this page · raw message
fs/erofs/internal.h +15 -16
@@ -285,7 +285,7 @@ struct erofs_vnode {/* atomic flags (including bitlocks) */ unsigned long flags; -unsigned char datamode; +unsigned char datalayout;unsigned char inode_isize;unsigned short xattr_isize;
@@ -310,31 +310,30 @@ struct erofs_vnode {#define EROFS_V (ptr) \ container_of(ptr, struct erofs_vnode, vfs_inode) -#define __inode_advise (x, bit, bits) \ - (((x) >> (bit)) & ((1 << (bits)) - 1)) - -#define __inode_version (advise) \ - __inode_advise(advise, EROFS_I_VERSION_BIT, \ - EROFS_I_VERSION_BITS) - -#define __inode_data_mapping (advise) \ - __inode_advise(advise, EROFS_I_DATA_MAPPING_BIT,\ - EROFS_I_DATA_MAPPING_BITS) -static inline unsigned long inode_datablocks (struct inode *inode) {/* since i_size cannot be changed */ return DIV_ROUND_UP (inode->i_size,EROFS_BLKSIZ ); } -static inline bool is_inode_layout_compression (struct inode *inode) +static inline unsigned int erofs_bitrange (unsigned int value,unsigned int bit, +unsigned int bits) +{ + +return (value >> bit) & ((1 << bits) -1 ); +} + + +static inline unsigned int erofs_inode_version (unsigned int value) { -return erofs_inode_is_data_compressed (EROFS_V (inode)->datamode); +return erofs_bitrange (value,EROFS_I_VERSION_BIT , +EROFS_I_VERSION_BITS ); } -static inline bool is_inode_flat_inline (struct inode *inode) +static inline unsigned int erofs_inode_datalayout (unsigned int value) { -return EROFS_V (inode)->datamode ==EROFS_INODE_FLAT_INLINE ; +return erofs_bitrange (value,EROFS_I_DATALAYOUT_BIT , +EROFS_I_DATALAYOUT_BITS ); }extern const struct super_operations erofs_sops;
fs/erofs/super.c +1 -1
@@ -121,7 +121,7 @@ static int superblock_read(struct super_block *sb)#ifdef CONFIG_EROFS_FS_XATTR sbi->xattr_blkaddr =le32_to_cpu (layout->xattr_blkaddr);#endif - sbi->islotbits =ffs (sizeof (struct erofs_inode_v1 )) -1 ; + sbi->islotbits =ilog2 (sizeof (struct erofs_inode_compact )); sbi->root_nid =le16_to_cpu (layout->root_nid); sbi->inos =le64_to_cpu (layout->inos);
fs/erofs/zmap.c +3 -3
@@ -12,7 +12,7 @@ int z_erofs_fill_inode(struct inode *inode) {struct erofs_vnode *const vi =EROFS_V (inode); -if (vi->datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ) { +if (vi->datalayout ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ) { vi->z_advise =0 ; vi->z_algorithmtype[0 ] =0 ; vi->z_algorithmtype[1 ] =0 ;
@@ -46,7 +46,7 @@ static int fill_inode_lazy(struct inode *inode)if (test_bit (EROFS_V_Z_INITED_BIT , &vi->flags)) goto out_unlock; -DBG_BUGON (vi->datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ); +DBG_BUGON (vi->datalayout ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ); pos =ALIGN (iloc (EROFS_SB (sb), vi->nid) + vi->inode_isize + vi->xattr_isize,8 );
@@ -314,7 +314,7 @@ static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,static int vle_load_cluster_from_disk (struct z_erofs_maprecorder *m,unsigned int lcn) { -const unsigned int datamode =EROFS_V (m->inode)->datamode; +const unsigned int datamode =EROFS_V (m->inode)->datalayout;if (datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY )return vle_legacy_load_cluster_from_disk (m, lcn);
Signature · 1 line
2.17.1
PATCH v2 8/25 erofs: kill erofs_{init,exit}_inode_cache
In reply to Gao XiangAs Christoph said [1] "having this function seems
entirely pointless", let's kill those.
filesystem function name
ext2,f2fs,ext4,isofs,squashfs,cifs,... init_inodecache
In addition, add a necessary "rcu_barrier()" on exit_fs();
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/super.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-)
fs/erofs/super.c +12 -19
@@ -23,21 +23,6 @@ static void init_once(void *ptr)inode_init_once (&vi->vfs_inode); } -static int __init erofs_init_inode_cache (void ) -{ - erofs_inode_cachep =kmem_cache_create ("erofs_inode" , -sizeof (struct erofs_vnode ),0 , -SLAB_RECLAIM_ACCOUNT , - init_once); - -return erofs_inode_cachep ?0 : -ENOMEM ; -} - -static void erofs_exit_inode_cache (void ) -{ -kmem_cache_destroy (erofs_inode_cachep); -} -static struct inode *alloc_inode (struct super_block *sb) { struct erofs_vnode *vi =
@@ -531,9 +516,14 @@ static int __init erofs_module_init(void)erofs_check_ondisk_layout_definitions ();infoln ("initializing erofs " EROFS_VERSION ); - err =erofs_init_inode_cache (); -if (err) + erofs_inode_cachep =kmem_cache_create ("erofs_inode" , +sizeof (struct erofs_vnode ),0 , +SLAB_RECLAIM_ACCOUNT , + init_once); +if (!erofs_inode_cachep) { + err = -ENOMEM ; goto icache_err; + } err =erofs_init_shrinker ();if (err)
@@ -555,7 +545,7 @@ static int __init erofs_module_init(void) zip_err:erofs_exit_shrinker (); shrinker_err: -erofs_exit_inode_cache (); +kmem_cache_destroy (erofs_inode_cachep); icache_err:return err; }
@@ -565,7 +555,10 @@ static void __exit erofs_module_exit(void)unregister_filesystem (&erofs_fs_type);z_erofs_exit_zip_subsystem ();erofs_exit_shrinker (); -erofs_exit_inode_cache (); + +/* Ensure all RCU free inodes are safe before cache is destroyed. */ +rcu_barrier (); +kmem_cache_destroy (erofs_inode_cachep);infoln ("successfully finalize erofs" ); }
Signature · 1 line
2.17.1
PATCH v2 9/25 erofs: use erofs_inode naming
In reply to Gao XiangAs Christoph suggested [1], "Why is this called vnode instead of inode? That seems like a rather odd naming for a Linux file system."
[1] https://lore.kernel.org/r/[email protected]/
--- fs/erofs/data.c | 8 ++++---- fs/erofs/dir.c | 6 +++--- fs/erofs/inode.c | 12 ++++++------ fs/erofs/internal.h | 14 +++++++------- fs/erofs/namei.c | 2 +- fs/erofs/super.c | 10 +++++----- fs/erofs/xattr.c | 18 +++++++++--------- fs/erofs/xattr.h | 4 ++-- fs/erofs/zdata.c | 9 +++------ fs/erofs/zmap.c | 28 ++++++++++++++-------------- include/trace/events/erofs.h | 14 +++++++------- 11 files changed, 61 insertions(+), 64 deletions(-)
fs/erofs/data.c +4 -4
@@ -112,7 +112,7 @@ static int erofs_map_blocks_flatmode(struct inode *inode,int err =0 ;erofs_blk_t nblocks, lastblk;u64 offset = map->m_la; -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);bool tailendpacking = (vi->datalayout ==EROFS_INODE_FLAT_INLINE );trace_erofs_map_blocks_flatmode_enter (inode, map, flags);
@@ -170,7 +170,7 @@ static int erofs_map_blocks_flatmode(struct inode *inode,int erofs_map_blocks (struct inode *inode,struct erofs_map_blocks *map,int flags) { -if (erofs_inode_is_data_compressed (EROFS_V (inode)->datalayout)) { +if (erofs_inode_is_data_compressed (EROFS_I (inode)->datalayout)) {int err =z_erofs_map_blocks_iter (inode, map, flags);if (map->mpage) {
@@ -365,7 +365,7 @@ static int erofs_raw_access_readpages(struct file *filp,if (IS_ERR (bio)) {pr_err ("%s, readahead error at page %lu of nid %llu\n" , __func__, page->index, -EROFS_V (mapping->host)->nid); +EROFS_I (mapping->host)->nid); bio =NULL ; }
@@ -404,7 +404,7 @@ static sector_t erofs_bmap(struct address_space *mapping, sector_t block) {struct inode *inode = mapping->host; -if (EROFS_V (inode)->datalayout ==EROFS_INODE_FLAT_INLINE ) { +if (EROFS_I (inode)->datalayout ==EROFS_INODE_FLAT_INLINE ) {erofs_blk_t blks =i_size_read (inode) >>LOG_BLOCK_SIZE ;if (block >>LOG_SECTORS_PER_BLOCK >= blks)
fs/erofs/dir.c +3 -3
@@ -47,7 +47,7 @@ static int erofs_fill_dentries(struct inode *dir, struct dir_context *ctx,/* a corrupted entry is found */ if (nameoff + de_namelen > maxsize || de_namelen >EROFS_NAME_LEN ) { -errln ("bogus dirent @ nid %llu" ,EROFS_V (dir)->nid); +errln ("bogus dirent @ nid %llu" ,EROFS_I (dir)->nid);DBG_BUGON (1 );return -EFSCORRUPTED ; }
@@ -85,7 +85,7 @@ static int erofs_readdir(struct file *f, struct dir_context *ctx)break ; }else if (IS_ERR (dentry_page)) {errln ("fail to readdir of logical block %u of nid %llu" , - i,EROFS_V (dir)->nid); + i,EROFS_I (dir)->nid); err = -EFSCORRUPTED ;break ; }
@@ -97,7 +97,7 @@ static int erofs_readdir(struct file *f, struct dir_context *ctx)if (nameoff <sizeof (struct erofs_dirent ) || nameoff >=PAGE_SIZE ) {errln ("%s, invalid de[0].nameoff %u @ nid %llu" , - __func__, nameoff,EROFS_V (dir)->nid); + __func__, nameoff,EROFS_I (dir)->nid); err = -EFSCORRUPTED ; goto skip_this; }
fs/erofs/inode.c +6 -6
@@ -11,7 +11,7 @@/* no locking */ static int read_inode (struct inode *inode,void *data) { -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);struct erofs_inode_compact *dic = data;struct erofs_inode_extended *die;
@@ -140,7 +140,7 @@ static int read_inode(struct inode *inode, void *data)static int fill_inline_data (struct inode *inode,void *data,unsigned int m_pofs) { -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);struct erofs_sb_info *sbi =EROFS_I_SB (inode);/* should be tail-packing data inline */
@@ -178,7 +178,7 @@ static int fill_inline_data(struct inode *inode, void *data,static int fill_inode (struct inode *inode,int isdir) {struct erofs_sb_info *sbi =EROFS_SB (inode->i_sb); -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);struct page *page;void *data;int err;
@@ -260,7 +260,7 @@ static int erofs_ilookup_test_actor(struct inode *inode, void *opaque) {const erofs_nid_t nid = *(erofs_nid_t *)opaque; -return EROFS_V (inode)->nid == nid; +return EROFS_I (inode)->nid == nid; }static int erofs_iget_set_actor (struct inode *inode,void *opaque)
@@ -297,7 +297,7 @@ struct inode *erofs_iget(struct super_block *sb,if (inode->i_state &I_NEW ) {int err; -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode); vi->nid = nid;
@@ -317,7 +317,7 @@ int erofs_getattr(const struct path *path, struct kstat *stat, {struct inode *const inode =d_inode (path->dentry); -if (erofs_inode_is_data_compressed (EROFS_V (inode)->datalayout)) +if (erofs_inode_is_data_compressed (EROFS_I (inode)->datalayout)) stat->attributes |=STATX_ATTR_COMPRESSED ; stat->attributes |=STATX_ATTR_IMMUTABLE ;
fs/erofs/internal.h +7 -7
@@ -272,14 +272,14 @@ static inline erofs_off_t iloc(struct erofs_sb_info *sbi, erofs_nid_t nid) }/* atomic flag definitions */ -#define EROFS_V_EA_INITED_BIT 0 -#define EROFS_V_Z_INITED_BIT 1 +#define EROFS_I_EA_INITED_BIT 0 +#define EROFS_I_Z_INITED_BIT 1/* bitlock definitions (arranged in reverse order) */ -#define EROFS_V_BL_XATTR_BIT (BITS_PER_LONG - 1) -#define EROFS_V_BL_Z_BIT (BITS_PER_LONG - 2) +#define EROFS_I_BL_XATTR_BIT (BITS_PER_LONG - 1) +#define EROFS_I_BL_Z_BIT (BITS_PER_LONG - 2) -struct erofs_vnode { +struct erofs_inode {erofs_nid_t nid;/* atomic flags (including bitlocks) */
@@ -307,8 +307,8 @@ struct erofs_vnode {struct inode vfs_inode; }; -#define EROFS_V (ptr) \ - container_of(ptr, struct erofs_vnode, vfs_inode) +#define EROFS_I (ptr) \ + container_of(ptr, struct erofs_inode, vfs_inode)static inline unsigned long inode_datablocks (struct inode *inode) {
fs/erofs/namei.c +1 -1
@@ -117,7 +117,7 @@ static struct page *find_target_block_classic(struct inode *dir,kunmap_atomic (de);put_page (page);errln ("corrupted dir block %d @ nid %llu" , - mid,EROFS_V (dir)->nid); + mid,EROFS_I (dir)->nid);DBG_BUGON (1 ); page =ERR_PTR (-EFSCORRUPTED ); goto out;
fs/erofs/super.c +5 -5
@@ -18,27 +18,27 @@ static struct kmem_cache *erofs_inode_cachep __read_mostly;static void init_once (void *ptr) { -struct erofs_vnode *vi = ptr; +struct erofs_inode *vi = ptr;inode_init_once (&vi->vfs_inode); }static struct inode *alloc_inode (struct super_block *sb) { -struct erofs_vnode *vi = +struct erofs_inode *vi =kmem_cache_alloc (erofs_inode_cachep,GFP_KERNEL );if (!vi)return NULL ;/* zero out everything except vfs_inode */ -memset (vi,0 , offsetof(struct erofs_vnode , vfs_inode)); +memset (vi,0 , offsetof(struct erofs_inode , vfs_inode));return &vi->vfs_inode; }static void free_inode (struct inode *inode) { -struct erofs_vnode *vi =EROFS_V (inode); +struct erofs_inode *vi =EROFS_I (inode);/* be careful RCU symlink path (see ext4_inode_info->i_data)! */ if (is_inode_fast_symlink (inode))
@@ -517,7 +517,7 @@ static int __init erofs_module_init(void)infoln ("initializing erofs " EROFS_VERSION ); erofs_inode_cachep =kmem_cache_create ("erofs_inode" , -sizeof (struct erofs_vnode ),0 , +sizeof (struct erofs_inode ),0 ,SLAB_RECLAIM_ACCOUNT , init_once);if (!erofs_inode_cachep) {
fs/erofs/xattr.c +9 -9
@@ -38,7 +38,7 @@ static inline void xattr_iter_end_final(struct xattr_iter *it)static int init_inode_xattrs (struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct xattr_iter it;unsigned int i;struct erofs_xattr_ibody_header *ih;
@@ -48,14 +48,14 @@ static int init_inode_xattrs(struct inode *inode)int ret =0 ;/* the most case is that xattrs of this inode are initialized. */ -if (test_bit (EROFS_V_EA_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_EA_INITED_BIT , &vi->flags))return 0 ; -if (wait_on_bit_lock (&vi->flags,EROFS_V_BL_XATTR_BIT ,TASK_KILLABLE )) +if (wait_on_bit_lock (&vi->flags,EROFS_I_BL_XATTR_BIT ,TASK_KILLABLE ))return -ERESTARTSYS ;/* someone has initialized xattrs for us? */ -if (test_bit (EROFS_V_EA_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_EA_INITED_BIT , &vi->flags)) goto out_unlock;/*
@@ -136,10 +136,10 @@ static int init_inode_xattrs(struct inode *inode) }xattr_iter_end (&it, atomic_map); -set_bit (EROFS_V_EA_INITED_BIT , &vi->flags); +set_bit (EROFS_I_EA_INITED_BIT , &vi->flags); out_unlock: -clear_and_wake_up_bit (EROFS_V_BL_XATTR_BIT , &vi->flags); +clear_and_wake_up_bit (EROFS_I_BL_XATTR_BIT , &vi->flags);return ret; }
@@ -184,7 +184,7 @@ static inline int xattr_iter_fixup(struct xattr_iter *it)static int inline_xattr_iter_begin (struct xattr_iter *it,struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct erofs_sb_info *const sbi =EROFS_SB (inode->i_sb);unsigned int xattr_header_sz, inline_xattr_ofs;
@@ -385,7 +385,7 @@ static int inline_getxattr(struct inode *inode, struct getxattr_iter *it)static int shared_getxattr (struct inode *inode,struct getxattr_iter *it) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct super_block *const sb = inode->i_sb;struct erofs_sb_info *const sbi =EROFS_SB (sb);unsigned int i;
@@ -608,7 +608,7 @@ static int inline_listxattr(struct listxattr_iter *it)static int shared_listxattr (struct listxattr_iter *it) {struct inode *const inode =d_inode (it->dentry); -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct super_block *const sb = inode->i_sb;struct erofs_sb_info *const sbi =EROFS_SB (sb);unsigned int i;
fs/erofs/xattr.h +2 -2
@@ -16,8 +16,8 @@static inline unsigned int inlinexattr_header_size (struct inode *inode) { -return sizeof (struct erofs_xattr_ibody_header ) - +sizeof (u32) *EROFS_V (inode)->xattr_shared_count; +return sizeof (struct erofs_xattr_ibody_header ) + +sizeof (u32) *EROFS_I (inode)->xattr_shared_count; } static inline erofs_blk_t xattrblock_addr(struct erofs_sb_info *sbi,
fs/erofs/zdata.c +3 -6
@@ -421,7 +421,7 @@ static struct z_erofs_collection *clregister(struct z_erofs_collector *clt,else pcl->algorithmformat =Z_EROFS_COMPRESSION_SHIFTED ; - pcl->clusterbits =EROFS_V (inode)->z_physical_clusterbits[0 ]; + pcl->clusterbits =EROFS_I (inode)->z_physical_clusterbits[0 ]; pcl->clusterbits -=PAGE_SHIFT ;/* new pclusters should be claimed as type 1, primary and followed */
@@ -1404,12 +1404,9 @@ static int z_erofs_vle_normalaccess_readpages(struct file *filp, head = (void *)page_private (page); err =z_erofs_do_read_page (&f, page, &pagepool); -if (err) { -struct erofs_vnode *vi =EROFS_V (inode); - +if (err)errln ("%s, readahead error at page %lu of nid %llu" , - __func__, page->index, vi->nid); - } + __func__, page->index,EROFS_I (inode)->nid);put_page (page); }
fs/erofs/zmap.c +14 -14
@@ -10,7 +10,7 @@int z_erofs_fill_inode (struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);if (vi->datalayout ==EROFS_INODE_FLAT_COMPRESSION_LEGACY ) { vi->z_advise =0 ;
@@ -19,7 +19,7 @@ int z_erofs_fill_inode(struct inode *inode) vi->z_logical_clusterbits =LOG_BLOCK_SIZE ; vi->z_physical_clusterbits[0 ] = vi->z_logical_clusterbits; vi->z_physical_clusterbits[1 ] = vi->z_logical_clusterbits; -set_bit (EROFS_V_Z_INITED_BIT , &vi->flags); +set_bit (EROFS_I_Z_INITED_BIT , &vi->flags); } inode->i_mapping->a_ops = &z_erofs_vle_normalaccess_aops;
@@ -28,7 +28,7 @@ int z_erofs_fill_inode(struct inode *inode)static int fill_inode_lazy (struct inode *inode) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct super_block *const sb = inode->i_sb;int err;erofs_off_t pos;
@@ -36,14 +36,14 @@ static int fill_inode_lazy(struct inode *inode)void *kaddr;struct z_erofs_map_header *h; -if (test_bit (EROFS_V_Z_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_Z_INITED_BIT , &vi->flags))return 0 ; -if (wait_on_bit_lock (&vi->flags,EROFS_V_BL_Z_BIT ,TASK_KILLABLE )) +if (wait_on_bit_lock (&vi->flags,EROFS_I_BL_Z_BIT ,TASK_KILLABLE ))return -ERESTARTSYS ; err =0 ; -if (test_bit (EROFS_V_Z_INITED_BIT , &vi->flags)) +if (test_bit (EROFS_I_Z_INITED_BIT , &vi->flags)) goto out_unlock;DBG_BUGON (vi->datalayout ==EROFS_INODE_FLAT_COMPRESSION_LEGACY );
@@ -83,13 +83,13 @@ static int fill_inode_lazy(struct inode *inode) vi->z_physical_clusterbits[1 ] = vi->z_logical_clusterbits + ((h->h_clusterbits >>5 ) &7 ); -set_bit (EROFS_V_Z_INITED_BIT , &vi->flags); +set_bit (EROFS_I_Z_INITED_BIT , &vi->flags); unmap_done:kunmap_atomic (kaddr);unlock_page (page);put_page (page); out_unlock: -clear_and_wake_up_bit (EROFS_V_BL_Z_BIT , &vi->flags); +clear_and_wake_up_bit (EROFS_I_BL_Z_BIT , &vi->flags);return err; }
@@ -142,7 +142,7 @@ static int vle_legacy_load_cluster_from_disk(struct z_erofs_maprecorder *m, unsigned long lcn) {struct inode *const inode = m->inode; -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);const erofs_off_t ibase =iloc (EROFS_I_SB (inode), vi->nid);const erofs_off_t pos = Z_EROFS_VLE_LEGACY_INDEX_ALIGN(ibase + vi->inode_isize +
@@ -196,7 +196,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m, unsigned int amortizedshift, unsigned int eofs) { -struct erofs_vnode *const vi =EROFS_V (m->inode); +struct erofs_inode *const vi =EROFS_I (m->inode);const unsigned int lclusterbits = vi->z_logical_clusterbits;const unsigned int lomask = (1 << lclusterbits) -1 ;unsigned int vcnt, base, lo, encodebits, nblk;
@@ -260,7 +260,7 @@ static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m, unsigned long lcn) {struct inode *const inode = m->inode; -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);const unsigned int lclusterbits = vi->z_logical_clusterbits;const erofs_off_t ebase =ALIGN (iloc (EROFS_I_SB (inode), vi->nid) + vi->inode_isize + vi->xattr_isize, 8) +
@@ -314,7 +314,7 @@ static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,static int vle_load_cluster_from_disk (struct z_erofs_maprecorder *m,unsigned int lcn) { -const unsigned int datamode =EROFS_V (m->inode)->datalayout; +const unsigned int datamode =EROFS_I (m->inode)->datalayout;if (datamode ==EROFS_INODE_FLAT_COMPRESSION_LEGACY )return vle_legacy_load_cluster_from_disk (m, lcn);
@@ -328,7 +328,7 @@ static int vle_load_cluster_from_disk(struct z_erofs_maprecorder *m,static int vle_extent_lookback (struct z_erofs_maprecorder *m,unsigned int lookback_distance) { -struct erofs_vnode *const vi =EROFS_V (m->inode); +struct erofs_inode *const vi =EROFS_I (m->inode);struct erofs_map_blocks *const map = m->map;const unsigned int lclusterbits = vi->z_logical_clusterbits;unsigned long lcn = m->lcn;
@@ -374,7 +374,7 @@ int z_erofs_map_blocks_iter(struct inode *inode, struct erofs_map_blocks *map, int flags) { -struct erofs_vnode *const vi =EROFS_V (inode); +struct erofs_inode *const vi =EROFS_I (inode);struct z_erofs_maprecorder m = { .inode = inode, .map = map,
include/trace/events/erofs.h +7 -7
@@ -41,7 +41,7 @@ TRACE_EVENT(erofs_lookup,TP_fast_assign ( __entry->dev = dir->i_sb->s_dev; - __entry->nid =EROFS_V (dir)->nid; + __entry->nid =EROFS_I (dir)->nid; __entry->name = dentry->d_name.name; __entry->flags = flags; ),
@@ -66,7 +66,7 @@ TRACE_EVENT(erofs_fill_inode,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->blkaddr =erofs_blknr (iloc (EROFS_I_SB (inode), __entry->nid)); __entry->ofs =erofs_blkoff (iloc (EROFS_I_SB (inode), __entry->nid)); __entry->isdir = isdir;
@@ -95,7 +95,7 @@ TRACE_EVENT(erofs_readpage,TP_fast_assign ( __entry->dev = page->mapping->host->i_sb->s_dev; - __entry->nid =EROFS_V (page->mapping->host)->nid; + __entry->nid =EROFS_I (page->mapping->host)->nid; __entry->dir =S_ISDIR (page->mapping->host->i_mode); __entry->index = page->index; __entry->uptodate =PageUptodate (page);
@@ -128,7 +128,7 @@ TRACE_EVENT(erofs_readpages,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->start = page->index; __entry->nrpage = nrpage; __entry->raw = raw;
@@ -157,7 +157,7 @@ DECLARE_EVENT_CLASS(erofs__map_blocks_enter,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->la = map->m_la; __entry->llen = map->m_llen; __entry->flags = flags;
@@ -203,7 +203,7 @@ DECLARE_EVENT_CLASS(erofs__map_blocks_exit,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; __entry->flags = flags; __entry->la = map->m_la; __entry->pa = map->m_pa;
@@ -247,7 +247,7 @@ TRACE_EVENT(erofs_destroy_inode,TP_fast_assign ( __entry->dev = inode->i_sb->s_dev; - __entry->nid =EROFS_V (inode)->nid; + __entry->nid =EROFS_I (inode)->nid; ),TP_printk ("dev = (%d,%d), nid = %llu" ,show_dev_nid (__entry))
Signature · 1 line
2.17.1