overlayfs + casefolding
Subsystems (3)
cover overlayfs + casefolding
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used _at all_ if a filesystem even supports casefolding, which is really nasty for users.
Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Kent Overstreet (6):
bcachefs: BCH_INODE_has_case_insensitive
darray: lift from bcachefs
fs: SB_CASEFOLD
fs: dcache locking for exlusion between overlayfs, casefolding
bcachefs: Hook up d_casefold_enable()
overlayfs: Support casefolded filesystems
MAINTAINERS | 7 +
fs/bcachefs/Makefile | 1 -
fs/bcachefs/bcachefs_format.h | 3 +-
fs/bcachefs/btree_node_scan_types.h | 2 +-
fs/bcachefs/btree_types.h | 2 +-
fs/bcachefs/btree_update.c | 1 +
fs/bcachefs/btree_write_buffer_types.h | 2 +-
fs/bcachefs/disk_accounting_types.h | 2 +-
fs/bcachefs/fs.c | 45 +++++-
fs/bcachefs/fsck.c | 12 +-
fs/bcachefs/inode.c | 8 +-
fs/bcachefs/inode.h | 2 +-
fs/bcachefs/inode_format.h | 7 +-
fs/bcachefs/journal_io.h | 2 +-
fs/bcachefs/journal_sb.c | 2 +-
fs/bcachefs/namei.c | 166 +++++++++++++++++++++-
fs/bcachefs/namei.h | 5 +
fs/bcachefs/rcu_pending.c | 3 +-
fs/bcachefs/sb-downgrade.c | 9 +-
fs/bcachefs/sb-errors_format.h | 4 +-
fs/bcachefs/sb-errors_types.h | 2 +-
fs/bcachefs/sb-members.h | 3 +-
fs/bcachefs/snapshot_types.h | 3 +-
fs/bcachefs/subvolume.h | 1 -
fs/bcachefs/thread_with_file_types.h | 2 +-
fs/bcachefs/util.h | 28 +---
fs/dcache.c | 177 ++++++++++++++++++++++++
fs/libfs.c | 1 +
fs/overlayfs/params.c | 20 ++-
fs/overlayfs/util.c | 19 ++-
{fs/bcachefs => include/linux}/darray.h | 70 +++++-----
include/linux/darray_types.h | 33 +++++
include/linux/dcache.h | 10 ++
include/linux/fs.h | 4 +
lib/Makefile | 2 +-
{fs/bcachefs => lib}/darray.c | 9 +-
36 files changed, 571 insertions(+), 98 deletions(-)
rename {fs/bcachefs => include/linux}/darray.h (64%)
create mode 100644 include/linux/darray_types.h
rename {fs/bcachefs => lib}/darray.c (75%)Signature · 1 line
2.49.0
PATCH 1/6 bcachefs: BCH_INODE_has_case_insensitive
Add a flag for tracking whether a directory has case-insensitive descendents - so that overlayfs can disallow mounting, even though the filesystem supports case insensitivity.
This is a new on disk format version, with a (cheap) upgrade to ensure the flag is correctly set on existing inodes.
Create, rename and fssetxattr are all plumbed to ensure the new flag is set, and we've got new fsck code that hooks into check_inode(0.
--- fs/bcachefs/bcachefs_format.h | 3 +- fs/bcachefs/fs.c | 4 + fs/bcachefs/fsck.c | 10 +- fs/bcachefs/inode.c | 8 +- fs/bcachefs/inode.h | 2 +- fs/bcachefs/inode_format.h | 7 +- fs/bcachefs/namei.c | 166 ++++++++++++++++++++++++++++++++- fs/bcachefs/namei.h | 5 + fs/bcachefs/sb-downgrade.c | 6 +- fs/bcachefs/sb-errors_format.h | 4 +- 10 files changed, 202 insertions(+), 13 deletions(-)
fs/bcachefs/bcachefs_format.h +2 -1
@@ -699,7 +699,8 @@ struct bch_sb_field_ext { x(casefolding, BCH_VERSION(1, 24)) \ x(extent_flags, BCH_VERSION(1, 25)) \ x(snapshot_deletion_v2, BCH_VERSION(1, 26)) \ - x(fast_device_removal, BCH_VERSION(1, 27)) + x(fast_device_removal, BCH_VERSION(1, 27)) \ + x(inode_has_case_insensitive, BCH_VERSION(1, 28))enum bcachefs_metadata_version { bcachefs_metadata_version_min =9 ,
fs/bcachefs/fs.c +4 -0
@@ -1681,6 +1681,10 @@ static int fssetxattr_inode_update_fn(struct btree_trans *trans, bi->bi_casefold = s->casefold +1 ; bi->bi_fields_set |=BIT (Inode_opt_casefold); + ret =bch2_maybe_propagate_has_case_insensitive (trans,inode_inum (inode), bi); +if (ret) +return ret; +#else printk (KERN_ERR "Cannot use casefolding on a kernel without CONFIG_UNICODE\n" );return -EOPNOTSUPP ;
fs/bcachefs/fsck.c +7 -3
@@ -264,7 +264,7 @@ static int lookup_lostfound(struct btree_trans *trans, u32 snapshot,u64 cpu =raw_smp_processor_id ();bch2_inode_init_early (c, lostfound); -bch2_inode_init_late (lostfound, now,0 ,0 ,S_IFDIR |0700 ,0 , &root_inode); +bch2_inode_init_late (c, lostfound, now,0 ,0 ,S_IFDIR |0700 ,0 , &root_inode); lostfound->bi_dir = root_inode.bi_inum; lostfound->bi_snapshot =le32_to_cpu (st.root_snapshot);
@@ -543,7 +543,7 @@ static int reconstruct_subvol(struct btree_trans *trans, u32 snapshotid, u32 subu64 cpu =raw_smp_processor_id ();bch2_inode_init_early (c, &new_inode); -bch2_inode_init_late (&new_inode,bch2_current_time (c),0 ,0 ,S_IFDIR |0755 ,0 ,NULL ); +bch2_inode_init_late (c, &new_inode,bch2_current_time (c),0 ,0 ,S_IFDIR |0755 ,0 ,NULL ); new_inode.bi_subvol = subvolid;
@@ -633,7 +633,7 @@ static int reconstruct_inode(struct btree_trans *trans, enum btree_id btree, u32struct bch_inode_unpacked new_inode;bch2_inode_init_early (c, &new_inode); -bch2_inode_init_late (&new_inode,bch2_current_time (c),0 ,0 , i_mode|0600 ,0 ,NULL ); +bch2_inode_init_late (c, &new_inode,bch2_current_time (c),0 ,0 , i_mode|0600 ,0 ,NULL ); new_inode.bi_size = i_size; new_inode.bi_inum = inum; new_inode.bi_snapshot = snapshot;
@@ -1135,6 +1135,10 @@ static int check_inode(struct btree_trans *trans, goto err; } + ret =bch2_check_inode_has_case_insensitive (trans, &u, &s->ids, &do_update); +if (ret) + goto err; +if (u.bi_dir || u.bi_dir_offset) { ret =check_inode_dirent_inode (trans, &u, &do_update);if (ret)
fs/bcachefs/inode.c +6 -2
@@ -907,7 +907,8 @@ void bch2_inode_init_early(struct bch_fs *c,get_random_bytes (&inode_u->bi_hash_seed,sizeof (inode_u->bi_hash_seed)); } -void bch2_inode_init_late (struct bch_inode_unpacked *inode_u,u64 now, +void bch2_inode_init_late (struct bch_fs *c, +struct bch_inode_unpacked *inode_u,u64 now,uid_t uid,gid_t gid,umode_t mode,dev_t rdev,struct bch_inode_unpacked *parent) {
@@ -934,6 +935,9 @@ void bch2_inode_init_late(struct bch_inode_unpacked *inode_u, u64 now,if (!S_ISDIR (mode)) inode_u->bi_casefold =0 ; + +if (bch2_inode_casefold (c, inode_u)) + inode_u->bi_flags |= BCH_INODE_has_case_insensitive; } void bch2_inode_init(struct bch_fs *c, struct bch_inode_unpacked *inode_u,
@@ -941,7 +945,7 @@ void bch2_inode_init(struct bch_fs *c, struct bch_inode_unpacked *inode_u, struct bch_inode_unpacked *parent) {bch2_inode_init_early (c, inode_u); -bch2_inode_init_late (inode_u,bch2_current_time (c), +bch2_inode_init_late (c, inode_u,bch2_current_time (c), uid, gid, mode, rdev, parent); }
fs/bcachefs/inode.h +1 -1
@@ -164,7 +164,7 @@ int bch2_fsck_write_inode(struct btree_trans *, struct bch_inode_unpacked *);void bch2_inode_init_early (struct bch_fs *,struct bch_inode_unpacked *); -void bch2_inode_init_late (struct bch_inode_unpacked *,u64 , +void bch2_inode_init_late (struct bch_fs *,struct bch_inode_unpacked *,u64 ,uid_t ,gid_t ,umode_t ,dev_t ,struct bch_inode_unpacked *); void bch2_inode_init(struct bch_fs *, struct bch_inode_unpacked *,
fs/bcachefs/inode_format.h +6 -1
@@ -129,6 +129,10 @@ enum inode_opt_id { Inode_opt_nr, }; +/* +* BCH_INODE_has_case_insensitive is set if any descendent is case insensitive - +* for overlayfs +*/ #define BCH_INODE_FLAGS () \ x(sync, 0) \ x(immutable, 1) \
@@ -139,7 +143,8 @@ enum inode_opt_id { x(i_sectors_dirty, 6) \ x(unlinked, 7) \ x(backptr_untrusted, 8) \ -x (has_child_snapshot,9 ) + x(has_child_snapshot, 9) \ +x (has_case_insensitive,10 )/* bits 20+ reserved for packed fields below: */
fs/bcachefs/namei.c +163 -3
193 lines not on this page · raw message
fs/bcachefs/namei.h +5 -0
@@ -71,4 +71,9 @@ static inline int bch2_check_dirent_target(struct btree_trans *trans, return __bch2_check_dirent_target(trans, dirent_iter, d, target, in_fsck); } +int bch2_maybe_propagate_has_case_insensitive (struct btree_trans *,subvol_inum , +struct bch_inode_unpacked *); +int bch2_check_inode_has_case_insensitive (struct btree_trans *,struct bch_inode_unpacked *, +snapshot_id_list *,bool *); +#endif /* _BCACHEFS_NAMEI_H */
fs/bcachefs/sb-downgrade.c +5 -1
@@ -100,7 +100,11 @@ BCH_FSCK_ERR_ptr_to_missing_backpointer) \ x(stripe_backpointers, \ BIT_ULL(BCH_RECOVERY_PASS_check_extents_to_backpointers),\ - BCH_FSCK_ERR_ptr_to_missing_backpointer) + BCH_FSCK_ERR_ptr_to_missing_backpointer) \ + x(inode_has_case_insensitive, \ + BIT_ULL(BCH_RECOVERY_PASS_check_inodes), \ + BCH_FSCK_ERR_inode_has_case_insensitive_not_set, \ + BCH_FSCK_ERR_inode_parent_has_case_insensitive_not_set) #define DOWNGRADE_TABLE() \ x(bucket_stripe_sectors, \
fs/bcachefs/sb-errors_format.h +3 -1
@@ -239,6 +239,8 @@ enum bch_fsck_flags { x(inode_unreachable, 210, FSCK_AUTOFIX) \ x(inode_journal_seq_in_future, 299, FSCK_AUTOFIX) \ x(inode_i_sectors_underflow, 312, FSCK_AUTOFIX) \ + x(inode_has_case_insensitive_not_set, 316, FSCK_AUTOFIX) \ + x(inode_parent_has_case_insensitive_not_set, 317, FSCK_AUTOFIX) \ x(vfs_inode_i_blocks_underflow, 311, FSCK_AUTOFIX) \ x(vfs_inode_i_blocks_not_zero_at_truncate, 313, FSCK_AUTOFIX) \x (deleted_inode_but_clean,211 ,FSCK_AUTOFIX ) \
@@ -325,7 +327,7 @@ enum bch_fsck_flags { x(dirent_stray_data_after_cf_name, 305, 0) \ x(rebalance_work_incorrectly_set, 309, FSCK_AUTOFIX) \ x(rebalance_work_incorrectly_unset, 310, FSCK_AUTOFIX) \ - x(MAX, 316, 0) + x(MAX, 318, 0)enum bch_sb_error_id {#define x(t, n, ...) BCH_FSCK_ERR_##t = n,
Signature · 1 line
2.49.0
PATCH 2/6 darray: lift from bcachefs
In reply to Kent Overstreetdynamic arrays - inspired from CCAN darrays, basically c++ stl vectors.
Open coding these is unwieldy. In this series they're used by the new dcache code for exclusion between case insensitive directories and overlayfs, and we've got a lot of open coded dynamic arrays that could be cleaned up with this.
---
MAINTAINERS | 7 +++
fs/bcachefs/Makefile | 1 -
fs/bcachefs/btree_node_scan_types.h | 2 +-
fs/bcachefs/btree_types.h | 2 +-
fs/bcachefs/btree_update.c | 1 +
fs/bcachefs/btree_write_buffer_types.h | 2 +-
fs/bcachefs/disk_accounting_types.h | 2 +-
fs/bcachefs/fsck.c | 2 +-
fs/bcachefs/journal_io.h | 2 +-
fs/bcachefs/journal_sb.c | 2 +-
fs/bcachefs/rcu_pending.c | 3 +-
fs/bcachefs/sb-downgrade.c | 3 +-
fs/bcachefs/sb-errors_types.h | 2 +-
fs/bcachefs/sb-members.h | 3 +-
fs/bcachefs/snapshot_types.h | 3 +-
fs/bcachefs/subvolume.h | 1 -
fs/bcachefs/thread_with_file_types.h | 2 +-
fs/bcachefs/util.h | 28 +---------
{fs/bcachefs => include/linux}/darray.h | 70 ++++++++++++++-----------
include/linux/darray_types.h | 33 ++++++++++++
lib/Makefile | 2 +-
{fs/bcachefs => lib}/darray.c | 9 +++-
22 files changed, 106 insertions(+), 76 deletions(-)
rename {fs/bcachefs => include/linux}/darray.h (64%)
create mode 100644 include/linux/darray_types.h
rename {fs/bcachefs => lib}/darray.c (75%)MAINTAINERS +7 -0
@@ -6515,6 +6515,13 @@ F: net/ax25/ax25_out.c F: net/ax25/ax25_timer.c F: net/ax25/sysctl_net_ax25.c +DARRAY +M: Kent Overstreet <[email protected]> +L: [email protected] +S: Maintained +F: include/linux/darray.h +F: include/linux/darray_types.h + DATA ACCESS MONITOR M: SeongJae Park <[email protected]> L: [email protected]
fs/bcachefs/Makefile +0 -1
@@ -28,7 +28,6 @@ bcachefs-y := \ checksum.o \ clock.o \ compress.o \ - darray.o \ data_update.o \ debug.o \ dirent.o \
fs/bcachefs/btree_node_scan_types.h +1 -1
@@ -2,7 +2,7 @@ #ifndef _BCACHEFS_BTREE_NODE_SCAN_TYPES_H #define _BCACHEFS_BTREE_NODE_SCAN_TYPES_H -#include "darray.h" +#include <linux/darray.h> struct found_btree_node { bool range_updated:1;
fs/bcachefs/btree_types.h +1 -1
@@ -2,13 +2,13 @@#ifndef _BCACHEFS_BTREE_TYPES_H#define _BCACHEFS_BTREE_TYPES_H +#include <linux/darray_types.h> #include <linux/list.h> #include <linux/rhashtable.h> #include "bbpos_types.h" #include "btree_key_cache_types.h" #include "buckets_types.h" -#include "darray.h" #include "errcode.h" #include "journal_types.h" #include "replicas_types.h"
fs/bcachefs/btree_update.c +1 -0
@@ -14,6 +14,7 @@#include "snapshot.h" #include "trace.h" +#include <linux/darray.h> #include <linux/string_helpers.h> static inline int btree_insert_entry_cmp(const struct btree_insert_entry *l,
fs/bcachefs/btree_write_buffer_types.h +1 -1
@@ -2,7 +2,7 @@#ifndef _BCACHEFS_BTREE_WRITE_BUFFER_TYPES_H#define _BCACHEFS_BTREE_WRITE_BUFFER_TYPES_H -#include "darray.h" +#include <linux/darray_types.h> #include "journal_types.h" #define BTREE_WRITE_BUFERED_VAL_U64s_MAX 4
fs/bcachefs/disk_accounting_types.h +1 -1
@@ -2,7 +2,7 @@ #ifndef _BCACHEFS_DISK_ACCOUNTING_TYPES_H #define _BCACHEFS_DISK_ACCOUNTING_TYPES_H -#include "darray.h" +#include <linux/darray.h> struct accounting_mem_entry { struct bpos pos;
fs/bcachefs/fsck.c +1 -1
@@ -6,7 +6,6 @@#include "btree_cache.h" #include "btree_update.h" #include "buckets.h" -#include "darray.h" #include "dirent.h" #include "error.h" #include "fs.h"
@@ -21,6 +20,7 @@#include "xattr.h" #include <linux/bsearch.h> +#include <linux/darray.h> #include <linux/dcache.h> /* struct qstr */ static int dirent_points_to_inode_nowarn(struct bkey_s_c_dirent d,
fs/bcachefs/journal_io.h +1 -1
@@ -2,7 +2,7 @@#ifndef _BCACHEFS_JOURNAL_IO_H#define _BCACHEFS_JOURNAL_IO_H -#include "darray.h" +#include <linux/darray_types.h> void bch2_journal_pos_from_member_info_set (struct bch_fs *);void bch2_journal_pos_from_member_info_resume (struct bch_fs *);
fs/bcachefs/journal_sb.c +1 -1
@@ -2,8 +2,8 @@#include "bcachefs.h" #include "journal_sb.h" -#include "darray.h" +#include <linux/darray.h> #include <linux/sort.h> /* BCH_SB_FIELD_journal: */
fs/bcachefs/rcu_pending.c +1 -2
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 #define pr_fmt (fmt) "%s() " fmt "\n", __func__ +#include <linux/darray.h> #include <linux/generic-radix-tree.h> #include <linux/mm.h> #include <linux/percpu.h>
@@ -9,8 +10,6 @@#include <linux/vmalloc.h> #include "rcu_pending.h" -#include "darray.h" -#include "util.h" #define static_array_for_each (_a, _i) \ for (typeof(&(_a)[0]) _i = _a; \
fs/bcachefs/sb-downgrade.c +2 -1
@@ -6,12 +6,13 @@*/ #include "bcachefs.h" -#include "darray.h" #include "recovery_passes.h" #include "sb-downgrade.h" #include "sb-errors.h" #include "super-io.h" +#include <linux/darray.h> +#define RECOVERY_PASS_ALL_FSCK BIT_ULL(63)/*
fs/bcachefs/sb-errors_types.h +1 -1
@@ -2,7 +2,7 @@ #ifndef _BCACHEFS_SB_ERRORS_TYPES_H #define _BCACHEFS_SB_ERRORS_TYPES_H -#include "darray.h" +#include <linux/darray_types.h> struct bch_sb_error_entry_cpu { u64 id:16,
fs/bcachefs/sb-members.h +2 -1
@@ -2,10 +2,11 @@#ifndef _BCACHEFS_SB_MEMBERS_H#define _BCACHEFS_SB_MEMBERS_H -#include "darray.h" #include "bkey_types.h" #include "enumerated_ref.h" +#include <linux/darray.h> +extern char *const bch2_member_error_strs[]; static inline struct bch_member *
fs/bcachefs/snapshot_types.h +2 -1
@@ -3,9 +3,10 @@#define _BCACHEFS_SNAPSHOT_TYPES_H#include "bbpos_types.h" -#include "darray.h" #include "subvolume_types.h" +#include <linux/darray_types.h> +typedef DARRAY (u32)snapshot_id_list ;#define IS_ANCESTOR_BITMAP 128
fs/bcachefs/subvolume.h +0 -1
@@ -2,7 +2,6 @@#ifndef _BCACHEFS_SUBVOLUME_H#define _BCACHEFS_SUBVOLUME_H -#include "darray.h" #include "subvolume_types.h" int bch2_check_subvols (struct bch_fs *);
fs/bcachefs/thread_with_file_types.h +1 -1
@@ -2,7 +2,7 @@ #ifndef _BCACHEFS_THREAD_WITH_FILE_TYPES_H #define _BCACHEFS_THREAD_WITH_FILE_TYPES_H -#include "darray.h" +#include <linux/darray_types.h> struct stdio_buf { spinlock_t lock;
fs/bcachefs/util.h +2 -26
@@ -5,24 +5,24 @@#include <linux/bio.h> #include <linux/blkdev.h> #include <linux/closure.h> +#include <linux/darray.h> #include <linux/errno.h> #include <linux/freezer.h> #include <linux/kernel.h> #include <linux/min_heap.h> -#include <linux/sched/clock.h> #include <linux/llist.h> #include <linux/log2.h> #include <linux/percpu.h> #include <linux/preempt.h> #include <linux/random.h> #include <linux/ratelimit.h> +#include <linux/sched/clock.h> #include <linux/slab.h> #include <linux/vmalloc.h> #include <linux/workqueue.h> #include "mean_and_variance.h" -#include "darray.h" #include "time_stats.h" struct closure ;
@@ -552,30 +552,6 @@ static inline void memset_u64s_tail(void *s, int c, unsigned bytes)memset (s + bytes, c, rem); } -/* just the memmove, doesn't update @_nr */ -#define __array_insert_item (_array, _nr, _pos) \ - memmove(&(_array)[(_pos) + 1], \ - &(_array)[(_pos)], \ - sizeof((_array)[0]) * ((_nr) - (_pos))) - -#define array_insert_item (_array, _nr, _pos, _new_item) \ -do { \ - __array_insert_item(_array, _nr, _pos); \ - (_nr)++; \ - (_array)[(_pos)] = (_new_item); \ -} while (0) - -#define array_remove_items (_array, _nr, _pos, _nr_to_remove) \ -do { \ - (_nr) -= (_nr_to_remove); \ - memmove(&(_array)[(_pos)], \ - &(_array)[(_pos) + (_nr_to_remove)], \ - sizeof((_array)[0]) * ((_nr) - (_pos))); \ -} while (0) - -#define array_remove_item (_array, _nr, _pos) \ - array_remove_items(_array, _nr, _pos, 1) -static inline void __move_gap (void *array,size_t element_size,size_t nr,size_t size,size_t old_gap,size_t new_gap)
include/linux/darray.h renamed from fs/bcachefs/darray.h +39 -31
104 lines not on this page · raw message
include/linux/darray_types.h new file +33 -0
@@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* (C) 2022-2024 Kent Overstreet <[email protected]> +*/ +#ifndef _LINUX_DARRAY_TYPES_H +#define _LINUX_DARRAY_TYPES_H + +#include <linux/types.h> + +#define DARRAY_PREALLOCATED (_type, _nr) \ +struct { \ + size_t nr, size; \ + _type *data; \ + _type preallocated[_nr]; \ +} + +#define DARRAY (_type) DARRAY_PREALLOCATED(_type, 0) + +typedef DARRAY (char )darray_char ; +typedef DARRAY (char *)darray_str ; +typedef DARRAY (const char *)darray_const_str ; + +typedef DARRAY (u8)darray_u8 ; +typedef DARRAY (u16)darray_u16 ; +typedef DARRAY (u32)darray_u32 ; +typedef DARRAY (u64)darray_u64 ; + +typedef DARRAY (s8)darray_s8 ; +typedef DARRAY (s16)darray_s16 ; +typedef DARRAY (s32)darray_s32 ; +typedef DARRAY (s64)darray_s64 ; + +#endif /* _LINUX_DARRAY_TYPES_H */
lib/Makefile +1 -1
@@ -56,7 +56,7 @@ obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \ bsearch.o find_bit.o llist.o lwq.o memweight.o kfifo.o \ percpu-refcount.o rhashtable.o base64.o \ once.o refcount.o rcuref.o usercopy.o errseq.o bucket_locks.o \ - generic-radix-tree.o bitmap-str.o + generic-radix-tree.o bitmap-str.o darray.o obj-y += string_helpers.o obj-y += hexdump.o obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
lib/darray.c renamed from fs/bcachefs/darray.c +7 -2
@@ -1,11 +1,15 @@// SPDX-License-Identifier: GPL-2.0 +/* +* (C) 2022-2024 Kent Overstreet <[email protected]> +*/ +#include <linux/darray.h> +#include <linux/export.h> #include <linux/log2.h> #include <linux/slab.h> #include <linux/vmalloc.h> -#include "darray.h" -int __bch2_darray_resize_noprof (darray_char *d,size_t element_size,size_t new_size,gfp_t gfp) +int __darray_resize_slowpath (darray_char *d,size_t element_size,size_t new_size,gfp_t gfp) {if (new_size > d->size) { new_size =roundup_pow_of_two (new_size);
@@ -36,3 +40,4 @@ int __bch2_darray_resize_noprof(darray_char *d, size_t element_size, size_t new_return 0 ; } +EXPORT_SYMBOL_GPL (__darray_resize_slowpath);
Signature · 1 line
2.49.0
PATCH 3/6 fs: SB_CASEFOLD
In reply to Kent OverstreetAdd a new flag indicating that a filesystem supports casefolding.
This is better than overlayfs's current method of checking for sb->s_encoding, which isn't reliable - XFS implements ASCII casefolding, so it won't be set there.
It's needed for overlayfs and the new dcache exclusion code to check "should we allow union mounts on this filesystem even though the dcache hash/compare ops are set? and do we need the new exclusion?".
--- fs/libfs.c | 1 + include/linux/fs.h | 1 + 2 files changed, 2 insertions(+)
fs/libfs.c +1 -0
@@ -1952,6 +1952,7 @@ void generic_set_sb_d_ops(struct super_block *sb) {#if IS_ENABLED (CONFIG_UNICODE )if (sb->s_encoding) { + sb->s_flags |=SB_CASEFOLD ; sb->s_d_op = &generic_ci_dentry_ops;return ; }
include/linux/fs.h +1 -0
@@ -1228,6 +1228,7 @@ extern int send_sigurg(struct file *file);#define SB_SYNCHRONOUS BIT(4)/* Writes are synced at once */ #define SB_MANDLOCK BIT(6)/* Allow mandatory locks on an FS */ #define SB_DIRSYNC BIT(7)/* Directory modifications are synchronous */ +#define SB_CASEFOLD BIT(8)/* Superblock supports casefolding */ #define SB_NOATIME BIT(10)/* Do not update access times. */ #define SB_NODIRATIME BIT(11)/* Do not update directory access times */ #define SB_SILENT BIT(15)
Signature · 1 line
2.49.0
PATCH 4/6 fs: dcache locking for exlusion between overlayfs, casefolding
In reply to Kent OverstreetAllow casefolding to be used on the same filesystem as overlayfs.
Overlayfs cannot handle a directory that has been casefolding, so this implements locking between enabling casefolding and overlayfs.
The underlying filesystem mus also track, for each directory, whether casefolding has been enabled on any subdirectory, and provide this information to the VFS with the S_NO_CASEFOLD flag.
Since we can't inflate struct dentry for this, add a separate rhashtable that functions as a per-dentry "casefolding disabled" ref.
New helpers, for overlayfs:
- d_casefold_disabled_put()
- d_casefold_disabled_get()
These acquire and release refs that check that S_NO_CASEFOLD is set on a directory, and prevent it from being cleared while the ref exists.
For the underlying filesystem:
- d_casefolding_enable()
- d_casefolding_enable_commit()
This is for the underlying filesystem that implements casefolding, in settar and rename. d_casefolding_enable() checks for conflicting refs held by overlayfs and acquires refs while the rename or setatr is being done, d_casefolding_enable_commit() releases those refs and clears the S_NO_CASEFOLD flag.
Implementation -
We can't inflate struct dentry for this, so references are tracked in a separate rhashtable.
To guard against races with rename(), d_casefolding_enable() must take refs (which in turn call dget()) on all dentries with inodes that need S_NO_CASEFOLD flipped; these dget() refs are released by commit().
d_casefold_disabled_get/put take dget() refs for the duration of the union mount, but this is unchanged because unionfs already holds a path ref for the duration of the mount.
--- fs/dcache.c | 177 +++++++++++++++++++++++++++++++++++++++++ include/linux/dcache.h | 10 +++ include/linux/fs.h | 3 + 3 files changed, 190 insertions(+)
fs/dcache.c +177 -0
@@ -32,6 +32,9 @@#include <linux/bit_spinlock.h> #include <linux/rculist_bl.h> #include <linux/list_lru.h> +#include <linux/rhashtable.h> +#include <linux/darray.h> +#include <linux/errname.h> #include "internal.h" #include "mount.h"
@@ -3141,6 +3144,176 @@ ino_t d_parent_ino(struct dentry *dentry) }EXPORT_SYMBOL (d_parent_ino); +static DEFINE_MUTEX (no_casefold_dentries_lock); +struct rhashtable no_casefold_dentries; + +enum no_casefold_dentry_ref { + ref_casefold_disable, + ref_casefold_enable, +}; + +struct no_casefold_dentry { +struct rhash_head hash; +struct dentry *dentry; +unsigned long ref[2 ]; +}; + +static const struct rhashtable_params no_casefold_dentries_params = { + .head_offset = offsetof(struct no_casefold_dentry , hash), + .key_offset = offsetof(struct no_casefold_dentry , dentry), + .key_len =sizeof (struct dentry *), + .automatic_shrinking = true, +}; + +static int no_casefold_dentry_get (struct dentry *dentry, +enum no_casefold_dentry_ref ref) +{ +struct no_casefold_dentry *n = +rhashtable_lookup_fast (&no_casefold_dentries, + &dentry, + no_casefold_dentries_params); +if (n) { +if (n->ref[!ref]) +return -EINVAL ; + + n->ref[ref]++; +return 0 ; + } + + n =kzalloc (sizeof (*n),GFP_KERNEL ); +if (!n) +return -ENOMEM ; + + n->dentry =dget (dentry); + n->ref[ref]++; + +int ret =rhashtable_lookup_insert_fast (&no_casefold_dentries, + &n->hash, no_casefold_dentries_params); +if (WARN_ON (ret)) { +kfree (n); +return ret; + } + +return 0 ; +} + +static void no_casefold_dentry_put (struct dentry *dentry, +enum no_casefold_dentry_ref ref) +{ +struct no_casefold_dentry *n = +rhashtable_lookup_fast (&no_casefold_dentries, + &dentry, + no_casefold_dentries_params); +if (WARN_ON (!n)) +return ; + +if (--n->ref[ref]) +return ; + +dput (n->dentry); +int ret =rhashtable_remove_fast (&no_casefold_dentries, + &n->hash, no_casefold_dentries_params); +WARN_ON (ret); +} + +/** +* d_casefold_disabled_put - drop a "casefold disabled" ref +* +* Only for overlayfs. +*/ +void d_casefold_disabled_put (struct dentry *dentry) +{ +if (!(dentry->d_sb->s_flags &SB_CASEFOLD )) +return ; + +guard (mutex)(&no_casefold_dentries_lock); +no_casefold_dentry_put (dentry, ref_casefold_disable); +} +EXPORT_SYMBOL_GPL (d_casefold_disabled_put); + +/** +* d_casefold_disabled_get - attempt to disable casefold on a tree +* +* Only for overlayfs. +* +* Returns -EINVAL if casefolding is in use on any subdirectory; this must be +* tracked by the filesystem. +* +* On success, returns with a reference held that must be released by +* d_casefold_disabled_put(); this ref blocks casefold from being enabled +* by d_casefold_enable(). +*/ +int d_casefold_disabled_get (struct dentry *dentry) +{ +if (!(dentry->d_sb->s_flags &SB_CASEFOLD )) +return 0 ; + +guard (mutex)(&no_casefold_dentries_lock); + +if (!(dentry->d_inode->i_flags &S_NO_CASEFOLD )) +return -EINVAL ; + +return no_casefold_dentry_get (dentry, ref_casefold_disable); +} +EXPORT_SYMBOL_GPL (d_casefold_disabled_get); + +/** +* d_casefold_enable - check if casefolding may be enabled on a dentry +* +* Returns -EINVAL if casefolding has been disabled on any parent directory (by +* overlayfs). +* +* On success the inode will hase S_NO_CASEFOLD cleared. +*/ +int d_casefold_enable (struct dentry *dentry,struct d_casefold_enable *e) +{ +struct dentry *root = dentry->d_sb->s_root; +int ret =0 ; + +guard (mutex)(&no_casefold_dentries_lock); + +for (struct dentry *i = dentry; + i && i->d_inode->i_flags &S_NO_CASEFOLD ; + i = i != root ? i->d_parent :NULL ) { + ret =darray_push (&e->refs, i); +if (ret) + goto err; + + ret =no_casefold_dentry_get (i, ref_casefold_enable); +if (ret) { +darray_pop (&e->refs); + goto err; + } + } + +return 0 ; +err: +darray_for_each (e->refs, i) +no_casefold_dentry_put (*i, ref_casefold_enable); +darray_exit (&e->refs); +return ret; +} +EXPORT_SYMBOL_GPL (d_casefold_enable); + +void d_casefold_enable_commit (struct d_casefold_enable *e,int ret) +{ +guard (mutex)(&no_casefold_dentries_lock); + +darray_for_each (e->refs, i) { +if (!ret) { +struct inode *inode = (*i)->d_inode; + +spin_lock (&inode->i_lock); + inode->i_flags &= ~S_NO_CASEFOLD ; +spin_unlock (&inode->i_lock); + } + +no_casefold_dentry_put (*i, ref_casefold_enable); + } +darray_exit (&e->refs); +} +EXPORT_SYMBOL_GPL (d_casefold_enable_commit); +static __initdata unsigned long dhash_entries;static int __init set_dhash_entries (char *str) {
@@ -3186,6 +3359,10 @@ static void __init dcache_init(void)SLAB_RECLAIM_ACCOUNT |SLAB_PANIC |SLAB_ACCOUNT , d_shortname.string); +int ret =rhashtable_init (&no_casefold_dentries, &no_casefold_dentries_params); +if (ret) +panic ("error initializing no_casefold_dentries: %s\n" ,errname (ret)); +/* Hash may have been set up in dcache_init_early */ if (!hashdist)return ;
include/linux/dcache.h +10 -0
@@ -3,6 +3,7 @@#define __LINUX_DCACHE_H#include <linux/atomic.h> +#include <linux/darray_types.h> #include <linux/list.h> #include <linux/math.h> #include <linux/rculist.h>
@@ -604,4 +605,13 @@ static inline struct dentry *d_next_sibling(const struct dentry *dentry) return hlist_entry_safe(dentry->d_sib.next, struct dentry, d_sib); } +void d_casefold_disabled_put (struct dentry *dentry); +int d_casefold_disabled_get (struct dentry *dentry); + +struct d_casefold_enable { +DARRAY (struct dentry *) refs; +}; +int d_casefold_enable (struct dentry *dentry,struct d_casefold_enable *e); +void d_casefold_enable_commit (struct d_casefold_enable *e,int ret); +#endif /* __LINUX_DCACHE_H */
include/linux/fs.h +3 -0
@@ -48,6 +48,7 @@#include <linux/rw_hint.h> #include <linux/file_ref.h> #include <linux/unicode.h> +#include <linux/rhashtable-types.h> #include <asm/byteorder.h> #include <uapi/linux/fs.h>
@@ -1440,6 +1441,7 @@ struct super_block {struct mutex s_sync_lock;/* sync serialisation lock */ +/* * Indicates how deep in a filesystem stack this SB is */
@@ -2345,6 +2347,7 @@ struct super_operations {#define S_CASEFOLD (1 << 15)/* Casefolded file */ #define S_VERITY (1 << 16)/* Verity file (using fs/verity/) */ #define S_KERNEL_FILE (1 << 17)/* File is in use by the kernel (eg. fs/cachefiles) */ +#define S_NO_CASEFOLD (1 << 18)/* Directory, and all descendents, are not casefolded */ /* * Note that nosuid etc flags are inode-specific: setting some file-system
Signature · 1 line
2.49.0
Al Viro
maintainerRe: [PATCH 4/6] fs: dcache locking for exlusion between overlayfs, casefolding
Quoted 5 lines · Kent Overstreet
fs/dcache.c +5
-0+int d_casefold_enable (struct dentry *dentry,struct d_casefold_enable *e) +{ +struct dentry *root = dentry->d_sb->s_root; +int ret =0 ; +
fs/dcache.c ino_t d_parent_ino(struct dentry *dentry) · line 3282
Quoted 10 lines
fs/dcache.c +10
-0+guard (mutex)(&no_casefold_dentries_lock); + +for (struct dentry *i = dentry; + i && i->d_inode->i_flags &S_NO_CASEFOLD ; + i = i != root ? i->d_parent :NULL ) { + ret =darray_push (&e->refs, i); +if (ret) + goto err; + + ret =no_casefold_dentry_get (i, ref_casefold_enable);
Beyond being fucking ugly, this is outright broken. Lose
the timeslice (e.g. on allocation in that thing), and there's nothing to prevent your 'i' from pointing to freed memory.
Re: [PATCH 4/6] fs: dcache locking for exlusion between overlayfs, casefolding
Quoted 13 lines · Al Viro
Quoted 11 lines · Kent Overstreet
fs/dcache.c +11
-0+int d_casefold_enable (struct dentry *dentry,struct d_casefold_enable *e) +{ +struct dentry *root = dentry->d_sb->s_root; +int ret =0 ; + +guard (mutex)(&no_casefold_dentries_lock); + +for (struct dentry *i = dentry; + i && i->d_inode->i_flags &S_NO_CASEFOLD ; + i = i != root ? i->d_parent :NULL ) { + ret =darray_push (&e->refs, i);
Quoted 8 lines
fs/dcache.c ino_t d_parent_ino(struct dentry *dentry) · line 3282
Quoted 4 lines
fs/dcache.c +4
-0+if (ret) + goto err; + + ret =no_casefold_dentry_get (i, ref_casefold_enable);Beyond being fucking ugly, this is outright broken. Lose
the timeslice (e.g. on allocation in that thing), and there's nothing to prevent your 'i' from pointing to freed memory.
I was under the impression that dentries couldn't be freed while a child is pinned, and we have a dget() on the start of the chain.
But no, rename would break that, of course.
PATCH v2 1/1 fs: dcache exlusion between overlayfs, casefolding
In reply to Kent OverstreetAllow casefolding to be used on the same filesystem as overlayfs.
Overlayfs cannot handle a directory that has been casefolding, so this implements locking between enabling casefolding and overlayfs.
The underlying filesystem mus also track, for each directory, whether casefolding has been enabled on any subdirectory, and provide this information to the VFS with the S_NO_CASEFOLD flag.
Since we can't inflate struct dentry for this, add a separate rhashtable that functions as a per-dentry "casefolding disabled" ref.
New helpers, for overlayfs:
- d_casefold_disabled_put()
- d_casefold_disabled_get()
These acquire and release refs that check that S_NO_CASEFOLD is set on a directory, and prevent it from being cleared while the ref exists.
For the underlying filesystem:
- d_casefolding_enable()
- d_casefolding_enable_commit()
This is for the underlying filesystem that implements casefolding, in settar and rename. d_casefolding_enable() checks for conflicting refs held by overlayfs and acquires refs while the rename or setatr is being done, d_casefolding_enable_commit() releases those refs and clears the S_NO_CASEFOLD flag.
Implementation -
We can't inflate struct dentry for this, so references are tracked in a separate rhashtable.
To guard against races with rename(), d_casefolding_enable() must take refs (which in turn call dget()) on all dentries with inodes that need S_NO_CASEFOLD flipped; these dget() refs are released by commit().
d_casefold_disabled_get/put take dget() refs for the duration of the union mount, but this is unchanged because unionfs already holds a path ref for the duration of the mount.
Changes since v1:
- Fix "walk up the tree" synchronization
s_vfs_rename_mutex is required: we can't allow an operation enabling casefolding (via fileattr_set) to race with a rename moving it into a tree exported by overlayfs.
d_casefold_enable() has a new @rename parameter: if set, we check that s_vfs_rename_mutex is held, if not (called from fileattr_set), we acquire it and release it in d_casefold_enable_commit().
Also, use dget_parent().
- no_casefold_dentries_lock -> sb->s_casefold_enable_lock
- d_casefold_enable() now bails out early if called on non-directory, or if S_NO_CASEFOLD is not set.
--- fs/dcache.c | 266 +++++++++++++++++++++++++++++++++++++++++ fs/super.c | 1 + include/linux/dcache.h | 12 ++ include/linux/fs.h | 4 + 4 files changed, 283 insertions(+)
fs/dcache.c +266 -0
@@ -32,6 +32,8 @@#include <linux/bit_spinlock.h> #include <linux/rculist_bl.h> #include <linux/list_lru.h> +#include <linux/rhashtable.h> +#include <linux/darray.h> #include "internal.h" #include "mount.h"
@@ -3141,6 +3143,266 @@ ino_t d_parent_ino(struct dentry *dentry) }EXPORT_SYMBOL (d_parent_ino); +static struct rhashtable no_casefold_dentries; + +enum no_casefold_dentry_ref { + ref_casefold_disable, + ref_casefold_enable, +}; + +struct no_casefold_dentry { +struct rhash_head hash; +struct dentry *dentry; +unsigned long ref[2 ]; +}; + +static const struct rhashtable_params no_casefold_dentries_params = { + .head_offset = offsetof(struct no_casefold_dentry , hash), + .key_offset = offsetof(struct no_casefold_dentry , dentry), + .key_len =sizeof (struct dentry *), + .automatic_shrinking = true, +}; + +static int no_casefold_dentry_get (struct dentry *dentry, +enum no_casefold_dentry_ref ref) +{ +struct no_casefold_dentry *n = +rhashtable_lookup_fast (&no_casefold_dentries, + &dentry, + no_casefold_dentries_params); +if (n) { +if (n->ref[!ref]) +return -EINVAL ; + + n->ref[ref]++; +return 0 ; + } + + n =kzalloc (sizeof (*n),GFP_KERNEL ); +if (!n) +return -ENOMEM ; + + n->dentry =dget (dentry); + n->ref[ref]++; + +int ret =rhashtable_lookup_insert_fast (&no_casefold_dentries, + &n->hash, no_casefold_dentries_params); +if (WARN_ON (ret)) { +kfree (n); +return ret; + } + +return 0 ; +} + +static void no_casefold_dentry_put (struct dentry *dentry, +enum no_casefold_dentry_ref ref) +{ +struct no_casefold_dentry *n = +rhashtable_lookup_fast (&no_casefold_dentries, + &dentry, + no_casefold_dentries_params); +if (WARN_ON (!n)) +return ; + +if (--n->ref[ref]) +return ; + +dput (n->dentry); +int ret =rhashtable_remove_fast (&no_casefold_dentries, + &n->hash, no_casefold_dentries_params); +WARN_ON (ret); +} + +/** +* d_casefold_disabled_put - drop a "casefold disabled" ref +* +* Only for overlayfs. +*/ +void d_casefold_disabled_put (struct dentry *dentry) +{ +struct super_block *sb = dentry->d_sb; + +if (!(sb->s_flags &SB_CASEFOLD )) +return ; + +guard (mutex)(&sb->s_casefold_enable_lock); +no_casefold_dentry_put (dentry, ref_casefold_disable); +} +EXPORT_SYMBOL_GPL (d_casefold_disabled_put); + +/** +* d_casefold_disabled_get - attempt to disable casefold on a tree +* +* Only for overlayfs. +* +* Returns -EINVAL if casefolding is in use on any subdirectory; this must be +* tracked by the filesystem. +* +* On success, returns with a reference held that must be released by +* d_casefold_disabled_put(); this ref blocks casefold from being enabled +* by d_casefold_enable(). +*/ +int d_casefold_disabled_get (struct dentry *dentry) +{ +struct super_block *sb = dentry->d_sb; + +if (!(sb->s_flags &SB_CASEFOLD )) +return 0 ; + +guard (mutex)(&sb->s_casefold_enable_lock); + +if (!(dentry->d_inode->i_flags &S_NO_CASEFOLD )) +return -EINVAL ; + +return no_casefold_dentry_get (dentry, ref_casefold_disable); +} +EXPORT_SYMBOL_GPL (d_casefold_disabled_get); + +/* Crabwalk: releases @dentry after getting ref on parent */ +static struct dentry *dget_parent_this_sb (struct dentry *dentry) +{ +struct dentry *parent = dentry != dentry->d_sb->s_root + ?dget_parent (dentry) + :NULL ; +dput (dentry); +return parent; +} + +/** +* d_casefold_enable - check if casefolding may be enabled on a dentry +* +* @dentry: dentry to enable casefolding on +* @e: state object, released by d_casefold_enable_commit() +* @rename: Are we in the rename path? +* If so, we expect s_vfs_rename_mutex to be held, if not (called +* from setflags), we aquire it if necessary, and release in +* commit. +* +* The rename mutex is required because we're operating on a whole path, +* potentially up to the filesystem root, and we need it to be stable until +* commit (i.e. we don't want to be renamed into a tree overlayfs is exporting +* after we've returned success). +* +* For rename, this should only be called for cross-directory rename. +* S_NO_CASEFOLD doesn't need to change on rename within a directory, and +* s_vfs_rename_mutex won't be held on non cross-directory rename. +* +* Returns -EINVAL if casefolding has been disabled on any parent directory (by +* overlayfs). +* +* On success, the d_casefold_enable object must be committed with +* d_casefold_enable_commit(), after the filesystem has updated its internal +* state. +* +* Commit will clear S_NO_CASEFOLD on all inodes up to the filesystem root, +* informing overlayfs that this tree has casefolding enabled somewhere in it. +*/ +int d_casefold_enable (struct dentry *dentry,struct d_casefold_enable *e, +bool rename) +{ +int ret =0 ; + +memset (e,0 ,sizeof (*e)); + e->sb = dentry->d_sb; + +if (!(e->sb->s_flags &SB_CASEFOLD )) +return 0 ; + +if (!S_ISDIR (dentry->d_inode->i_mode)) +return 0 ; + +/* +* On rename, we're passed the dentry being renamed (the filesystem is +* not passed the dentry of the directory we're renaming to), but it's +* the parent that may need to have S_NO_CASEFOLD cleared: +*/ + dentry = rename + ?dget_parent (dentry) + :dget (dentry); + +if (!(dentry->d_inode->i_flags &S_NO_CASEFOLD )) { +dput (dentry); +return 0 ; + } + +if (rename) { +lockdep_assert_held (&e->sb->s_vfs_rename_mutex); + }else { +mutex_lock (&e->sb->s_vfs_rename_mutex); + e->rename_mutex_held = true; + } + +guard (mutex)(&e->sb->s_casefold_enable_lock); + +for (struct dentry *i = dentry; i; i =dget_parent_this_sb (i)) { +if (!(i->d_inode->i_flags &S_NO_CASEFOLD )) { +dput (i); +break ; + } + + ret =darray_push (&e->refs, i); +if (ret) { +dput (i); + goto err; + } + + ret =no_casefold_dentry_get (i, ref_casefold_enable); +if (ret) { +dput (i); + --e->refs.nr; + goto err; + } + } + +return 0 ; +err: +darray_for_each (e->refs, i) +no_casefold_dentry_put (*i, ref_casefold_enable); +darray_exit (&e->refs); + +if (e->rename_mutex_held) +mutex_unlock (&e->sb->s_vfs_rename_mutex); + e->rename_mutex_held = false; +return ret; +} +EXPORT_SYMBOL_GPL (d_casefold_enable); + +/** +* d_casefold_enable_commit - finish operation started by d_casefold_enable() +* +* @e: state object, started by d_casefold_enable_commit() +* @ret: Success or failure of the operation, from the filesystem +* +* On success (@ret == 0), clear S_NO_CASEFOLD on all inodes up to the +* filesystem root that have it set, which d_casefold_enable() previously took +* references to. +*/ +void d_casefold_enable_commit (struct d_casefold_enable *e,int ret) +{ +if (e->refs.nr) { +guard (mutex)(&e->sb->s_casefold_enable_lock); + +darray_for_each (e->refs, i) { +if (!ret) { +struct inode *inode = (*i)->d_inode; + +spin_lock (&inode->i_lock); + inode->i_flags &= ~S_NO_CASEFOLD ; +spin_unlock (&inode->i_lock); + } + +no_casefold_dentry_put (*i, ref_casefold_enable); + } +darray_exit (&e->refs); + } + +if (e->rename_mutex_held) +mutex_unlock (&e->sb->s_vfs_rename_mutex); + e->rename_mutex_held = false; +} +EXPORT_SYMBOL_GPL (d_casefold_enable_commit); +static __initdata unsigned long dhash_entries;static int __init set_dhash_entries (char *str) {
@@ -3186,6 +3448,10 @@ static void __init dcache_init(void)SLAB_RECLAIM_ACCOUNT |SLAB_PANIC |SLAB_ACCOUNT , d_shortname.string); +int ret =rhashtable_init (&no_casefold_dentries, &no_casefold_dentries_params); +if (ret) +panic ("error initializing no_casefold_dentries: %i\n" , ret); +/* Hash may have been set up in dcache_init_early */ if (!hashdist)return ;
fs/super.c +1 -0
@@ -368,6 +368,7 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags,atomic_set (&s->s_active,1 );mutex_init (&s->s_vfs_rename_mutex);lockdep_set_class (&s->s_vfs_rename_mutex, &type->s_vfs_rename_key); +mutex_init (&s->s_casefold_enable_lock);init_rwsem (&s->s_dquot.dqio_sem); s->s_maxbytes =MAX_NON_LFS ; s->s_op = &default_op;
include/linux/dcache.h +12 -0
@@ -3,6 +3,7 @@#define __LINUX_DCACHE_H#include <linux/atomic.h> +#include <linux/darray_types.h> #include <linux/list.h> #include <linux/math.h> #include <linux/rculist.h>
@@ -604,4 +605,15 @@ static inline struct dentry *d_next_sibling(const struct dentry *dentry) return hlist_entry_safe(dentry->d_sib.next, struct dentry, d_sib); } +void d_casefold_disabled_put (struct dentry *dentry); +int d_casefold_disabled_get (struct dentry *dentry); + +struct d_casefold_enable { +DARRAY (struct dentry *) refs; +struct super_block *sb; +bool rename_mutex_held; +}; +int d_casefold_enable (struct dentry *dentry,struct d_casefold_enable *e,bool ); +void d_casefold_enable_commit (struct d_casefold_enable *e,int ret); +#endif /* __LINUX_DCACHE_H */
include/linux/fs.h +4 -0
@@ -48,6 +48,7 @@#include <linux/rw_hint.h> #include <linux/file_ref.h> #include <linux/unicode.h> +#include <linux/rhashtable-types.h> #include <asm/byteorder.h> #include <uapi/linux/fs.h>
@@ -1397,6 +1398,7 @@ struct super_block {* even looking at it. You had been warned. */ struct mutex s_vfs_rename_mutex;/* Kludge */ +struct mutex s_casefold_enable_lock;/* * Filesystem subtype. If non-empty the filesystem type field
@@ -1440,6 +1442,7 @@ struct super_block {struct mutex s_sync_lock;/* sync serialisation lock */ +/* * Indicates how deep in a filesystem stack this SB is */
@@ -2345,6 +2348,7 @@ struct super_operations {#define S_CASEFOLD (1 << 15)/* Casefolded file */ #define S_VERITY (1 << 16)/* Verity file (using fs/verity/) */ #define S_KERNEL_FILE (1 << 17)/* File is in use by the kernel (eg. fs/cachefiles) */ +#define S_NO_CASEFOLD (1 << 18)/* Directory, and all descendents, are not casefolded */ /* * Note that nosuid etc flags are inode-specific: setting some file-system
Signature · 1 line
2.49.0
PATCH 5/6 bcachefs: Hook up d_casefold_enable()
In reply to Kent OverstreetHook up BCH_INODE_has_case_insensitive -> S_NO_CASEFOLD, and call the new dcache methods for exclusion with overlayfs when the flag is changing.
--- fs/bcachefs/fs.c | 41 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-)
fs/bcachefs/fs.c +39 -2
@@ -68,6 +68,11 @@ static inline void bch2_inode_flags_to_vfs(struct bch_fs *c, struct bch_inode_in inode->v.i_flags |=S_CASEFOLD ; else inode->v.i_flags &= ~S_CASEFOLD; + +if (inode->ei_inode.bi_flags & BCH_INODE_has_case_insensitive) + inode->v.i_flags &= ~S_NO_CASEFOLD ; +else + inode->v.i_flags |=S_NO_CASEFOLD ; } void bch2_inode_update_after_write(struct btree_trans *trans,
@@ -916,6 +921,8 @@ static int bch2_rename2(struct mnt_idmap *idmap,struct bch_inode_info *dst_inode =to_bch_ei (dst_dentry->d_inode);struct bch_inode_unpacked dst_dir_u, src_dir_u;struct bch_inode_unpacked src_inode_u, dst_inode_u, *whiteout_inode_u; +struct d_casefold_enable casefold_enable_src = {}; +struct d_casefold_enable casefold_enable_dst = {};struct btree_trans *trans;enum bch_rename_mode mode = flags &RENAME_EXCHANGE ? BCH_RENAME_EXCHANGE
@@ -940,6 +947,21 @@ static int bch2_rename2(struct mnt_idmap *idmap, src_inode, dst_inode); +if (src_dir != dst_dir) { +if (bch2_inode_casefold (c, &src_inode->ei_inode)) { + ret =d_casefold_enable (dst_dentry, &casefold_enable_dst); +if (ret) + goto err; + } + +if (mode ==BCH_RENAME_EXCHANGE && +bch2_inode_casefold (c, &dst_inode->ei_inode)) { + ret =d_casefold_enable (src_dentry, &casefold_enable_src); +if (ret) + goto err; + } + } + trans =bch2_trans_get (c); ret = bch2_subvol_is_ro_trans(trans, src_dir->ei_inum.subvol) ?:
@@ -1044,6 +1066,9 @@ static int bch2_rename2(struct mnt_idmap *idmap, src_inode, dst_inode); +d_casefold_enable_commit (&casefold_enable_dst, ret); +d_casefold_enable_commit (&casefold_enable_src, ret); +return bch2_err_class (ret); }
@@ -1710,6 +1735,7 @@ static int bch2_fileattr_set(struct mnt_idmap *idmap,struct bch_inode_info *inode =to_bch_ei (d_inode (dentry));struct bch_fs *c = inode->v.i_sb->s_fs_info;struct flags_set s = {}; +struct d_casefold_enable casefold_enable = {};int ret;if (fa->fsx_valid) {
@@ -1742,9 +1768,17 @@ static int bch2_fileattr_set(struct mnt_idmap *idmap, s.casefold = (fa->flags &FS_CASEFOLD_FL ) !=0 ; fa->flags &= ~FS_CASEFOLD_FL ; +if (s.casefold) { + ret =d_casefold_enable (dentry, &casefold_enable); +if (ret) + goto err; + } + s.flags |=map_flags_rev (bch_flags_to_uflags, fa->flags); -if (fa->flags) -return -EOPNOTSUPP ; +if (fa->flags) { + ret = -EOPNOTSUPP ; + goto err; + } }mutex_lock (&inode->ei_update_lock);
@@ -1755,6 +1789,9 @@ static int bch2_fileattr_set(struct mnt_idmap *idmap,bch2_write_inode (c, inode, fssetxattr_inode_update_fn, &s,ATTR_CTIME );mutex_unlock (&inode->ei_update_lock); +err: +d_casefold_enable_commit (&casefold_enable, ret); +return ret; }
Signature · 1 line
2.49.0
PATCH 6/6 overlayfs: Support casefolded filesystems
In reply to Kent OverstreetOverlayfs can now work on filesystems that support casefolding, provided the specific subtree overlayfs is using as layers don't have casefolding enabled.
d_casefold_disabled_get() and put() are used, which check that casefolding is enabled nowhere on a given subtree, and get and release a reference that prevents the filesystem from enabling casefolding on that tree while overlayfs is in use.
We also now check the new SB_CASEFOLD superblock flag; if it's set we allow for dcache hash and compare ops to be set, relying instead on the new dcache methods.
--- fs/overlayfs/params.c | 20 +++++++++++++++++--- fs/overlayfs/util.c | 19 +++++++++++++++---- 2 files changed, 32 insertions(+), 7 deletions(-)
fs/overlayfs/params.c +17 -3
@@ -287,7 +287,8 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,* with overlayfs. Check explicitly to prevent post-mount * failures. */ -if (sb_has_encoding (path->mnt->mnt_sb)) +if ((path->mnt->mnt_sb->s_flags &SB_CASEFOLD ) && + !(path->dentry->d_inode->i_flags &S_NO_CASEFOLD ))return invalfc (fc,"case-insensitive capable filesystem on %s not supported" , name);if (ovl_dentry_weird (path->dentry))
@@ -411,20 +412,32 @@ static int ovl_do_parse_layer(struct fs_context *fc, const char *layer_name,if (!name)return -ENOMEM ; +if (layer != Opt_workdir && + layer != Opt_upperdir) { + err =d_casefold_disabled_get (layer_path->dentry); +if (err) +return err; + } + upper =is_upper_layer (layer); err =ovl_mount_dir_check (fc, layer_path, layer, name, upper);if (err) -return err; + goto err_put;if (!upper) { err =ovl_ctx_realloc_lower (fc);if (err) -return err; + goto err_put; }/* Store the user provided path string in ctx to show in mountinfo */ ovl_add_layer (fc, layer, layer_path, &name);return err; +err_put: +if (layer != Opt_workdir && + layer != Opt_upperdir) +d_casefold_disabled_put (layer_path->dentry); +return err; } static int ovl_parse_layer(struct fs_context *fc, struct fs_parameter *param,
@@ -475,6 +488,7 @@ static void ovl_reset_lowerdirs(struct ovl_fs_context *ctx) ctx->lowerdir_all =NULL ;for (size_t nr =0 ; nr < ctx->nr; nr++, l++) { +d_casefold_disabled_put (l->path.dentry);path_put (&l->path);kfree (l->name); l->name =NULL ;
fs/overlayfs/util.c +15 -4
@@ -205,10 +205,21 @@ bool ovl_dentry_weird(struct dentry *dentry)if (!d_can_lookup (dentry) && !d_is_file (dentry) && !d_is_symlink (dentry))return true; -return dentry->d_flags & (DCACHE_NEED_AUTOMOUNT | -DCACHE_MANAGE_TRANSIT | -DCACHE_OP_HASH | -DCACHE_OP_COMPARE ); +if (dentry->d_flags & (DCACHE_NEED_AUTOMOUNT | +DCACHE_MANAGE_TRANSIT )) +return true; + +/* +* The filesystem might support casefolding, but we've already checked +* that casefolding isn't present on this tree: we only need to check +* for non-casefolding hash/compare ops +*/ +if (!(dentry->d_sb->s_flags &SB_CASEFOLD ) && + (dentry->d_flags & (DCACHE_OP_HASH | +DCACHE_OP_COMPARE ))) +return true; + +return false; }enum ovl_path_type ovl_path_type (struct dentry *dentry)
Signature · 1 line
2.49.0
Amir Goldstein
maintainerQuoted 13 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
Quoted 9 lines
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
in ovl_lookup() that returns a merged directory, ovl_dentry_weird() would result in -EIO if any of the real directories have casefolding and we can add another sanotify in ovl_lookup_single() that the 'base' dentry is not weird() to cover the case of casefolder changed on an underlying reference directory.
Obviously, if any of the overlayfs layer root dirs have casefolding enabled the mount would fail.
w.r.t enabling casefolding underneath overlayfs, overlayfs documentation says:
"Changes to underlying filesystems
---------------------------------
Changes to the underlying filesystems while part of a mounted overlay filesystem are not allowed. If the underlying filesystem is changed, the behavior of the overlay is undefined, though it will not result in a crash or deadlock."
So why is enabling casefolding on underlying layers so special that we should have specific protection for that?
From what I remember in ext4, enabling casefolding is only allowed on empty directories.
Is this also the case for bcachefs?
If that is the case, then the situation is even simpler -
If filesystem can singal to vfs/ovl that directory is empty (i.e. S_EMPTYDIR) then overlayfs can ignore this dir altogether when composing
the merged directory.
But again, I don't think there is a good reason to treat this case of changing the underlying layer specially.
Please explain if I missed anything.
Thanks,
Amir.
Quoted 25 lines · Amir Goldstein
Quoted 13 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
Quoted 9 lines
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
Amir Goldstein
maintainerQuoted 27 lines · Kent Overstreet
Quoted 25 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 8 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
Thanks,
Amir.
Quoted 36 lines · Amir Goldstein
Quoted 34 lines · Kent Overstreet
Quoted 31 lines · Amir Goldstein
Quoted 13 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
Quoted 9 lines
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
Quoted 7 lines
Quoted 1 line
overlayfs works on trees, not directories...
I know how overlayfs works...
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
Amir Goldstein
maintainerQuoted 42 lines · Kent Overstreet
Quoted 40 lines · Amir Goldstein
Quoted 27 lines · Kent Overstreet
Quoted 25 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 8 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 8 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
That's one way of putting it.
Please explain the use case.
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Thanks,
Amir.
Quoted 53 lines · Amir Goldstein
Quoted 42 lines · Kent Overstreet
Quoted 40 lines · Amir Goldstein
Quoted 27 lines · Kent Overstreet
Quoted 25 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 8 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 8 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
Quoted 7 lines
That's one way of putting it.
Please explain the use case.
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
That's _broken_.
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Amir Goldstein
maintainerQuoted 59 lines · Kent Overstreet
Quoted 57 lines · Amir Goldstein
Quoted 42 lines · Kent Overstreet
Quoted 40 lines · Amir Goldstein
Quoted 27 lines · Kent Overstreet
Quoted 25 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 8 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 8 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
That's one way of putting it.
Please explain the use case.
Quoted 8 lines
Quoted 3 lines
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
That's _broken_.
Correct.
I am saying that IMO a smaller impact (and less user friendly) fix is more appropriate way to deal with this problem.
Quoted 6 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Right, I am not expecting users to partition by application, but my question was this:
When is overlayfs created over a subtree that is only partially case-folded?
Obviously, docker would create overlayfs on parts of the fs and smbd/cygwin could create a case folder subtree on another part of the fs.
I just don't see a common use case when these sections overlap.
Perhaps I am wrong (please present real world use cases), but my claim is that this case is not common enough and therefore, a suboptimal EIO error from lookup is good enough to prevert crossing over into the case folded zone by mistake, just as EIO on lookup is enough to deal with the unsupported use case of modifying overlayfs underlying layers with overlay is mounted.
BTW, it is not enough to claim that there is no case folding for the entire subtree to allow the mount.
For overlayfs to allow d_hash()/d_compare() fs must claim that these implementations are the default implementation in all subtree or at least that all layers share the same implementation.
Thanks,
Amir.
Quoted 10 lines · Amir Goldstein
On Tue, May 20, 2025 at 4:12 PM Kent Overstreet
Quoted 4 lines
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
That's _broken_.
Correct.
I am saying that IMO a smaller impact (and less user friendly) fix is more appropriate way to deal with this problem.
Less user friendly is an understatement.
Obscure errors that only get reported via overloaded standard error codes is a massive problem today, for _developers_ - have you never had a day of swearing over trying to track down where in a massive subsystem an -EINVAL is coming from?
It's even worse for end users that don't know to check the dmesg log.
And I support my code, so these would turn into bug reports coming across my desk - no thanks; I already get enough weird shit from other subsystems that I have to look at and at least triage.
Quoted 9 lines
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Right, I am not expecting users to partition by application, but my question was this:
Quoted 6 lines
When is overlayfs created over a subtree that is only partially case-folded?
Obviously, docker would create overlayfs on parts of the fs and smbd/cygwin could create a case folder subtree on another part of the fs.
I just don't see a common use case when these sections overlap.
Today, you cannot user docker and casefolding on _different parts of_ the same filesystem.
So yees, today users do have to partition by application, or only use one feature or the other.
This isn't about allowing casefolding and overlayfs to fix on the same subtree, that would be a bigger project.
Quoted 7 lines
Perhaps I am wrong (please present real world use cases), but my claim is that this case is not common enough and therefore, a suboptimal EIO error from lookup is good enough to prevert crossing over into the case folded zone by mistake, just as EIO on lookup is enough to deal with the unsupported use case of modifying overlayfs underlying layers with overlay is mounted.
Quoted 5 lines
BTW, it is not enough to claim that there is no case folding for the entire subtree to allow the mount.
For overlayfs to allow d_hash()/d_compare() fs must claim that these implementations are the default implementation in all subtree or at least that all layers share the same implementation.
I honestly don't know what you're claiming here.
Amir Goldstein
maintainerQuoted 42 lines · Kent Overstreet
Quoted 10 lines · Amir Goldstein
On Tue, May 20, 2025 at 4:12 PM Kent Overstreet
Quoted 4 lines
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
That's _broken_.
Correct.
I am saying that IMO a smaller impact (and less user friendly) fix is more appropriate way to deal with this problem.
Less user friendly is an understatement.
Obscure errors that only get reported via overloaded standard error codes is a massive problem today, for _developers_ - have you never had a day of swearing over trying to track down where in a massive subsystem an
-EINVALis coming from?It's even worse for end users that don't know to check the dmesg log.
And I support my code, so these would turn into bug reports coming across my desk - no thanks; I already get enough weird shit from other subsystems that I have to look at and at least triage.
Quoted 9 lines
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Right, I am not expecting users to partition by application, but my question was this:
Quoted 6 lines
When is overlayfs created over a subtree that is only partially case-folded?
Obviously, docker would create overlayfs on parts of the fs and smbd/cygwin could create a case folder subtree on another part of the fs.
I just don't see a common use case when these sections overlap.
Quoted 6 lines
Today, you cannot user docker and casefolding on
_differentpartsof_the same filesystem.So yees, today users do have to partition by application, or only use one feature or the other.
Didn't say there was no problem.
Argued that your fix is a big gun and not worth the added complexity.
Let's see what Miklos thinks.
Quoted 16 lines
This isn't about allowing casefolding and overlayfs to fix on the same subtree, that would be a bigger project.
Quoted 12 lines
Perhaps I am wrong (please present real world use cases), but my claim is that this case is not common enough and therefore, a suboptimal EIO error from lookup is good enough to prevert crossing over into the case folded zone by mistake, just as EIO on lookup is enough to deal with the unsupported use case of modifying overlayfs underlying layers with overlay is mounted.
BTW, it is not enough to claim that there is no case folding for the entire subtree to allow the mount.
For overlayfs to allow d_hash()/d_compare() fs must claim that these implementations are the default implementation in all subtree or at least that all layers share the same implementation.
Nevermind. Misread patch 6.
Thanks,
Amir.
Quoted 67 lines · Amir Goldstein
Quoted 42 lines · Kent Overstreet
Quoted 10 lines · Amir Goldstein
On Tue, May 20, 2025 at 4:12 PM Kent Overstreet
Quoted 4 lines
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
That's _broken_.
Correct.
I am saying that IMO a smaller impact (and less user friendly) fix is more appropriate way to deal with this problem.
Less user friendly is an understatement.
Obscure errors that only get reported via overloaded standard error codes is a massive problem today, for _developers_ - have you never had a day of swearing over trying to track down where in a massive subsystem an
-EINVALis coming from?It's even worse for end users that don't know to check the dmesg log.
And I support my code, so these would turn into bug reports coming across my desk - no thanks; I already get enough weird shit from other subsystems that I have to look at and at least triage.
Quoted 9 lines
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Right, I am not expecting users to partition by application, but my question was this:
Quoted 6 lines
When is overlayfs created over a subtree that is only partially case-folded?
Obviously, docker would create overlayfs on parts of the fs and smbd/cygwin could create a case folder subtree on another part of the fs.
I just don't see a common use case when these sections overlap.Quoted 6 lines
Today, you cannot user docker and casefolding on
_differentpartsof_the same filesystem.So yees, today users do have to partition by application, or only use one feature or the other.
Didn't say there was no problem.
Argued that your fix is a big gun and not worth the added complexity.
Let's see what Miklos thinks.
Quoted 10 lines
This isn't about allowing casefolding and overlayfs to fix on the same subtree, that would be a bigger project.
Quoted 7 lines
Perhaps I am wrong (please present real world use cases), but my claim is that this case is not common enough and therefore, a suboptimal EIO error from lookup is good enough to prevert crossing over into the case folded zone by mistake, just as EIO on lookup is enough to deal with the unsupported use case of modifying overlayfs underlying layers with overlay is mounted.
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
BTW, it is not enough to claim that there is no case folding for the entire subtree to allow the mount.
For overlayfs to allow d_hash()/d_compare() fs must claim that these implementations are the default implementation in all subtree or at least that all layers share the same implementation.Nevermind. Misread patch 6.
Since you were asking for use cases - docker & related are pretty widely used for deploying things that are "unwieldy" within the normal packgae manager ecosystem - and wine is case study #1 in that, where these days people want to ship a specific version of wine with applications being emulated (that's been tested with that application).
But wine wants casefolding, so - hapless user deploys docker image where casefolding is enabled _but only on the subdir that holds Windows data_, not the whole image.
Docker mounts the image, but then everything explodes when you try to use it with what look to the user like impenetrable IO errors.
That's a bad day for someone, or more likely a lot of someones.
Miklos Szeredi
maintainerQuoted 4 lines · Kent Overstreet
Docker mounts the image, but then everything explodes when you try to use it with what look to the user like impenetrable IO errors.
That's a bad day for someone, or more likely a lot of someones.
Wouldn't it be docker's responsibility to know that that won't work with overlayfs?
Any error, whether at startup or during operation is not something the user will like.
What am I missing?
Thanks,
Miklos
Quoted 9 lines · Miklos Szeredi
Quoted 4 lines · Kent Overstreet
Docker mounts the image, but then everything explodes when you try to use it with what look to the user like impenetrable IO errors.
That's a bad day for someone, or more likely a lot of someones.
Wouldn't it be docker's responsibility to know that that won't work with overlayfs?
Why would it be?
Quoted 4 lines
Any error, whether at startup or during operation is not something the user will like.
What am I missing?
A _mount_ error due to misconfiguration is expected operation, and we've built mechanisms for reporting those errors in a way that users can understand what's going on.
That's not true for normal file accesses post mount.
Quoted 2 lines · Amir Goldstein
I am saying that IMO a smaller impact (and less user friendly) fix is more appropriate way to deal with this problem.
What do you think about doing your approach as a stopgap?
It seems this is hitting a lot of people, something we can backport to 6.15 would be good to have.
Amir Goldstein
maintainerQuoted 9 lines · Kent Overstreet
Quoted 2 lines · Amir Goldstein
I am saying that IMO a smaller impact (and less user friendly) fix is more appropriate way to deal with this problem.
What do you think about doing your approach as a stopgap?
It seems this is hitting a lot of people, something we can backport to 6.15 would be good to have.
Yes, I think I can do that.
Will try to get to it next week.
Thanks,
Amir.
Amir Goldstein
maintainerQuoted 8 lines · Amir Goldstein
Quoted 5 lines · Kent Overstreet
Quoted 2 lines · Amir Goldstein
I am saying that IMO a smaller impact (and less user friendly) fix is more appropriate way to deal with this problem.
Quoted 8 lines
Quoted 4 lines
What do you think about doing your approach as a stopgap?
It seems this is hitting a lot of people, something we can backport to 6.15 would be good to have.
Yes, I think I can do that.
Will try to get to it next week.
On second look, here is a compile-basic-sanity-tested patch.
Care to test if it does the job for you?
Thanks,
Amir.
From 9fd9ce0e2b6de84b3665ed743b36348006739cd3 Mon Sep 17 00:00:00 2001
From: Amir Goldstein <[email protected]>
Date: Fri, 23 May 2025 22:13:18 +0200
Subject: [PATCH] ovl: support casefolded filesystems
Case folding is often applied to subtrees and not on an entire filesystem.
Disallowing layers from filesystems that support case folding is over limiting.
Replace the rule that case-folding capable are not allowed as layers with a rule that case folded directories are not allowed in a merged directory stack.
Should case folding be enabled on an underlying directory while overlayfs is mounted the outcome is generally undefined.
Specifically in ovl_lookup(), we check the base underlying directory and fail with -ESTALE if an underlying directory case folding is enabled.
--- fs/overlayfs/namei.c | 10 ++++++++++ fs/overlayfs/params.c | 10 ++++------ fs/overlayfs/util.c | 15 +++++++++++---- 3 files changed, 25 insertions(+), 10 deletions(-)
fs/overlayfs/namei.c +10 -0
@@ -237,6 +237,16 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,bool is_upper = d->layer->idx ==0 ;char val; +/* +* We allow filesystems that are case-folding capable but deny composing +* ovl stack from case-folded directories. If someone has enabled case +* folding on a directory on underlying layer, the warranty of the ovl +* stack is voided. +*/ + err = -ESTALE ; +if (sb_has_encoding (base->d_sb) &&IS_CASEFOLDED (d_inode (base))) + goto out; + this =ovl_lookup_positive_unlocked (d, name, base, namelen, drop_negative);if (IS_ERR (this)) { err =PTR_ERR (this);
fs/overlayfs/params.c +4 -6
@@ -282,13 +282,11 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,return invalfc (fc,"%s is not a directory" , name);/* -* Root dentries of case-insensitive capable filesystems might -* not have the dentry operations set, but still be incompatible -* with overlayfs. Check explicitly to prevent post-mount -* failures. +* Allow filesystems that are case-folding capable but deny composing +* ovl stack from case-folded directories. */ -if (sb_has_encoding (path->mnt->mnt_sb)) -return invalfc (fc,"case-insensitive capable filesystem on %s not supported" , name); +if (sb_has_encoding (path->mnt->mnt_sb) &&IS_CASEFOLDED (d_inode (path->dentry))) +return invalfc (fc,"case-insensitive directory on %s not supported" , name);if (ovl_dentry_weird (path->dentry))return invalfc (fc,"filesystem on %s not supported" , name);
fs/overlayfs/util.c +11 -4
@@ -206,10 +206,17 @@ bool ovl_dentry_weird(struct dentry *dentry)if (!d_can_lookup (dentry) && !d_is_file (dentry) && !d_is_symlink (dentry))return true; -return dentry->d_flags & (DCACHE_NEED_AUTOMOUNT | -DCACHE_MANAGE_TRANSIT | -DCACHE_OP_HASH | -DCACHE_OP_COMPARE ); +if (dentry->d_flags & (DCACHE_NEED_AUTOMOUNT |DCACHE_MANAGE_TRANSIT )) +return true; + +/* +* Allow filesystems that are case-folding capable but deny composing +* ovl stack from case-folded directories. +*/ +if (sb_has_encoding (dentry->d_sb)) +return IS_CASEFOLDED (d_inode (dentry)); + +return dentry->d_flags & (DCACHE_OP_HASH |DCACHE_OP_COMPARE ); }enum ovl_path_type ovl_path_type (struct dentry *dentry)
Signature · 1 line
2.34.1
On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
That makes fstests generic/631 pass.
We really should be logging something in dmesg on error due to casefolded directory, though.
Amir Goldstein
maintainerQuoted 4 lines · Kent Overstreet
On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
That makes fstests generic/631 pass.
Yes, that is not very surprising.
I meant if you could help test that:
1. mounting case folder upperdir/lowerdir fails
2. lookup a case folder subdir fails
3. lookup in a dir that was empty and became case folder while ovl was mounted fails
For me, I do not have any setup with case folding subtrees
so testing those use cases would take me time and
I think that you must have tested all those scenarios with your patch set? and maybe already have some fstests for them?
Quoted 3 lines
We really should be logging something in dmesg on error due to casefolded directory, though.
No problem.
Attached v2 with warnings.
For example, here is the new warning in test overlay/065:
[ 131.815110] overlayfs: failed lookup in lower (/lower,
name='upper', err=-40): overlapping layers
Thanks,
Amir.
From 4f730811b80670b65e4edd818621b988c59e150b Mon Sep 17 00:00:00 2001
From: Amir Goldstein <[email protected]>
Date: Fri, 23 May 2025 22:13:18 +0200
Subject: [PATCH v2] ovl: support layers on case-folding capable filesystems
Case folding is often applied to subtrees and not on an entire filesystem.
Disallowing layers from filesystems that support case folding is over limiting.
Replace the rule that case-folding capable are not allowed as layers with a rule that case folded directories are not allowed in a merged directory stack.
Should case folding be enabled on an underlying directory while overlayfs is mounted the outcome is generally undefined.
Specifically in ovl_lookup(), we check the base underlying directory and fail with -ESTALE and write a warning to kmsg if an underlying directory case folding is enabled.
--- fs/overlayfs/namei.c | 25 ++++++++++++++++++++++--- fs/overlayfs/params.c | 11 +++++------ fs/overlayfs/util.c | 15 +++++++++++---- 3 files changed, 38 insertions(+), 13 deletions(-)
fs/overlayfs/namei.c +22 -3
@@ -230,13 +230,26 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d, struct dentry **ret, bool drop_negative) {struct ovl_fs *ofs =OVL_FS (d->sb); -struct dentry *this; +struct dentry *this =NULL ; +const char *warn;struct path path;int err;bool last_element = !post[0 ];bool is_upper = d->layer->idx ==0 ;char val; +/* +* We allow filesystems that are case-folding capable but deny composing +* ovl stack from case-folded directories. If someone has enabled case +* folding on a directory on underlying layer, the warranty of the ovl +* stack is voided. +*/ +if (sb_has_encoding (base->d_sb) &&IS_CASEFOLDED (d_inode (base))) { + warn ="case folded parent" ; + err = -ESTALE ; + goto out_warn; + } + this =ovl_lookup_positive_unlocked (d, name, base, namelen, drop_negative);if (IS_ERR (this)) { err =PTR_ERR (this);
@@ -248,8 +261,9 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d,if (ovl_dentry_weird (this)) {/* Don't support traversing automounts and other weirdness */ + warn ="unsupported object type" ; err = -EREMOTE ; - goto out_err; + goto out_warn; } path.dentry = this;
@@ -283,8 +297,9 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d, }else {if (ovl_lookup_trap_inode (d->sb, this)) {/* Caught in a trap of overlapping layers */ + warn ="overlapping layers" ; err = -ELOOP ; - goto out_err; + goto out_warn; }if (last_element)
@@ -316,6 +331,10 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d, this =NULL ; goto out; +out_warn: +pr_warn_ratelimited ("failed lookup in %s (%pd2, name='%.*s', err=%i): %s\n" , + is_upper ?"upper" :"lower" , base, + namelen, name, err, warn); out_err:dput (this);return err;
fs/overlayfs/params.c +5 -6
@@ -282,13 +282,12 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,return invalfc (fc,"%s is not a directory" , name);/* -* Root dentries of case-insensitive capable filesystems might -* not have the dentry operations set, but still be incompatible -* with overlayfs. Check explicitly to prevent post-mount -* failures. +* Allow filesystems that are case-folding capable but deny composing +* ovl stack from case-folded directories. */ -if (sb_has_encoding (path->mnt->mnt_sb)) -return invalfc (fc,"case-insensitive capable filesystem on %s not supported" , name); +if (sb_has_encoding (path->mnt->mnt_sb) && +IS_CASEFOLDED (d_inode (path->dentry))) +return invalfc (fc,"case-insensitive directory on %s not supported" , name);if (ovl_dentry_weird (path->dentry))return invalfc (fc,"filesystem on %s not supported" , name);
fs/overlayfs/util.c +11 -4
@@ -206,10 +206,17 @@ bool ovl_dentry_weird(struct dentry *dentry)if (!d_can_lookup (dentry) && !d_is_file (dentry) && !d_is_symlink (dentry))return true; -return dentry->d_flags & (DCACHE_NEED_AUTOMOUNT | -DCACHE_MANAGE_TRANSIT | -DCACHE_OP_HASH | -DCACHE_OP_COMPARE ); +if (dentry->d_flags & (DCACHE_NEED_AUTOMOUNT |DCACHE_MANAGE_TRANSIT )) +return true; + +/* +* Allow filesystems that are case-folding capable but deny composing +* ovl stack from case-folded directories. +*/ +if (sb_has_encoding (dentry->d_sb)) +return IS_CASEFOLDED (d_inode (dentry)); + +return dentry->d_flags & (DCACHE_OP_HASH |DCACHE_OP_COMPARE ); }enum ovl_path_type ovl_path_type (struct dentry *dentry)
Signature · 1 line
2.34.1
Quoted 15 lines · Amir Goldstein
Quoted 4 lines · Kent Overstreet
On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
That makes fstests generic/631 pass.
Yes, that is not very surprising.
I meant if you could help test that:1. mounting case folder upperdir/lowerdir fails
2. lookup a case folder subdir fails
3. lookup in a dir that was empty and became case folder while ovl was mounted fails
Quoted 4 lines
For me, I do not have any setup with case folding subtrees
so testing those use cases would take me time and
I think that you must have tested all those scenarios with your patch set? and maybe already have some fstests for them?
Unmount fauls after I test an overlayfs with a casefold subdir:
Testing an overlayfs on a casefold fs with non-casefolded dirs
Test using casefolded dir - should fail
overlayfs: failed to resolve '/mnt/casefold': -2
mount: /mnt/merged: special device overlay does not exist.
dmesg(1) may have more information after failed mount system call.
Test using a dir with a casefold subdir - should mount
overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
overlayfs: workdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
ls: cannot access '/mnt/merged/dir/casefold': No such file or directory
umount: /mnt: target is busy.
https://evilpiepirate.org/git/ktest.git/commit/?id=47d1f2a04d79bc4cbc843f81e71eb7d821fb8384
Amir Goldstein
maintainerQuoted 22 lines · Kent Overstreet
Quoted 15 lines · Amir Goldstein
Quoted 4 lines · Kent Overstreet
On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
That makes fstests generic/631 pass.
Yes, that is not very surprising.
I meant if you could help test that:1. mounting case folder upperdir/lowerdir fails
2. lookup a case folder subdir fails
3. lookup in a dir that was empty and became case folder while ovl was mounted failsQuoted 4 lines
For me, I do not have any setup with case folding subtrees
so testing those use cases would take me time and
I think that you must have tested all those scenarios with your patch set? and maybe already have some fstests for them?
Quoted 7 lines
Unmount fauls after I test an overlayfs with a casefold subdir:
Testing an overlayfs on a casefold fs with non-casefolded dirs Test using casefolded dir - should fail
overlayfs: failed to resolve '/mnt/casefold': -2
mount:/mnt/merged: special device overlay does not exist.
dmesg(1) may have more information after failed mount system call.
Test is using the wrong path:
+ echo "Test using casefolded dir - should fail"
+ ! mount -t overlay -o
lowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay /mnt/merged
+ ! mount -t overlay -o
lowerdir=/mnt/casefold,upperdir=/mnt/casefold,workdir=/mnt/work overlay /mnt/merged
There is no "/mnt/casefold"
Quoted 3 lines
Test using a dir with a casefold subdir - should mount
overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
overlayfs: workdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
Those warnings are because you have a stray mount command above:
+ echo "Test using casefolded dir - should fail"
+ ! mount -t overlay -o
lowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay /mnt/merged
So a mount already exists. leftover?
Quoted 2 lines
ls: cannot access '
/mnt/merged/dir/casefold': No such file or directory
umount: /mnt: target is busy.
Not sure about that, but could be due to the aforementioned stay mount.
Quoted 2 lines
https://evilpiepirate.org/git/ktest.git/commit/?id=47d1f2a04d79bc4cbc843f81e71eb7d821fb8384
Please fix the test and report Tested-by if all works as expected. Please include dmesg so we can see the new warnings.
Thanks,
Amir.
Quoted 43 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
Quoted 15 lines · Amir Goldstein
Quoted 4 lines · Kent Overstreet
On Fri, May 23, 2025 at 10:30:16PM +0200, Amir Goldstein wrote:
That makes fstests generic/631 pass.
Yes, that is not very surprising.
I meant if you could help test that:1. mounting case folder upperdir/lowerdir fails
2. lookup a case folder subdir fails
3. lookup in a dir that was empty and became case folder while ovl was mounted failsQuoted 4 lines
For me, I do not have any setup with case folding subtrees
so testing those use cases would take me time and
I think that you must have tested all those scenarios with your patch set? and maybe already have some fstests for them?Quoted 7 lines
Unmount fauls after I test an overlayfs with a casefold subdir:
Testing an overlayfs on a casefold fs with non-casefolded dirs Test using casefolded dir - should fail
overlayfs: failed to resolve '/mnt/casefold': -2
mount:/mnt/merged: special device overlay does not exist.
dmesg(1) may have more information after failed mount system call.Test is using the wrong path:
(quoted diff) +2
-0+ echo "Test using casefolded dir - should fail" + ! mount -t overlay -olowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay
/mnt/merged
+ ! mount -t overlay -o
lowerdir=/mnt/casefold,upperdir=/mnt/casefold,workdir=/mnt/work overlay/mnt/merged
Quoted 1 line
There is no "
/mnt/casefold"
*nod*
Quoted 11 lines
Quoted 3 lines
Test using a dir with a casefold subdir - should mount
overlayfs: upperdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
overlayfs: workdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.Those warnings are because you have a stray mount command above:
(quoted diff) +2
-0+ echo "Test using casefolded dir - should fail" + ! mount -t overlay -olowerdir=/mnt/lower,upperdir=/mnt/upper,workdir=/mnt/work overlay
/mnt/mergedSo a mount already exists. leftover?
*snort* - -ENOCAFFEINE, I presume.
The new warning doesn't fire after the last mount.
ls: cannot access '/mnt/merged/casefold': Object is remote
But nothing in dmesg. Adding a printk to ovl_mount_dir_check() shows that it's never called for the 'ls /mnt/merged/casefold' call.
Quoted 58 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 55 lines · Amir Goldstein
Quoted 42 lines · Kent Overstreet
Quoted 40 lines · Amir Goldstein
Quoted 27 lines · Kent Overstreet
Quoted 25 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 8 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 8 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
That's one way of putting it.
Quoted 8 lines
Quoted 5 lines
Please explain the use case.
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
Wait, what? How many people use casefolding, on a per-directory basis? It's stupid. Unix/Linux has used case-sensitive filesystems for years. Yes, linux supports other OSes which did do casefolding, but yikes... per-directory support is just insane. It should be per-filesystem only at BEST.
Quoted 1 line
That's _broken_.
So? what about my cross mounting of VMS filesystems with "foo.txt;3" version control so I can go back to previous versions? Why can't I do that from my Linux systems that's mounting that VMS image?
Just because it's done doesn't mean it's not dumb.
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Sure I can, I don't run windows applications to screw casefolding. :-)
And I personally LIKE having a seperate /boot and /home, because it gives isolation. The world is not just single user laptops with everything all on one disk or spread across a couple of disks using LVM or RAID or all of the above.
I also don't see any updates for the XFS tests, or any other filesystem tests, that actually checks and confirms this decidedly obtuse and dumb to implement idea.
John
Quoted 64 lines · John Stoffel
Quoted 63 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 60 lines · Amir Goldstein
Quoted 42 lines · Kent Overstreet
Quoted 40 lines · Amir Goldstein
Quoted 27 lines · Kent Overstreet
Quoted 25 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 8 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 8 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
That's one way of putting it.
Please explain the use case.
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Quoted 8 lines
Quoted 2 lines
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
Wait, what? How many people use casefolding, on a per-directory basis? It's stupid. Unix/Linux has used case-sensitive filesystems for years. Yes, linux supports other OSes which did do casefolding, but yikes... per-directory support is just insane. It should be per-filesystem only at BEST.
Quite a lot.
You may not realize this, but Valve has, quietly, behind the scenes, been intelligently funding a ton of Linux work with an eye towards not just gaming, but improving Linus on the desktop. And they've been deploying it too, you can buy a Steam deck today.
And a significant fraction of desktop users like to play games - we're not just all work. Windows ports need casefolding - alternatives have been discussed and they're non viable.
(I fondly remember the days when I had time for such things).
Samba fileservers are a thing, too.
And for all us desktop/workstation users, not being able to use the same filesystem for wine and docker is the kind of jankiness that makes people say "maybe Linux isn't ready for the desktop after all".
Put aside your feelings on casefolding - this is about basic attention to detail.
Quoted 17 lines
Quoted 1 line
That's _broken_.
So? what about my cross mounting of VMS filesystems with "foo.txt;3" version control so I can go back to previous versions? Why can't I do that from my Linux systems that's mounting that VMS image?
Just because it's done doesn't mean it's not dumb.
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Sure I can, I don't run windows applications to screw casefolding. :-)
Quoted 8 lines
And I personally LIKE having a seperate /boot and /home, because it gives isolation. The world is not just single user laptops with everything all on one disk or spread across a couple of disks using LVM or RAID or all of the above.
I also don't see any updates for the XFS tests, or any other filesystem tests, that actually checks and confirms this decidedly obtuse and dumb to implement idea.
Well, you certainly are making your personal feelings known :)
But for me, as the engineer designing and implementing this stuff, I don't let my personal feelings dictate.
That's not my place.
My job is to write code that works, works reliably, solves real problems, and lets users do the things they want with their machines.
All this drama over casefolding has been pure distraction, and I would appreciate if you and everyone else could tone it down now.
Quoted 70 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 67 lines · John Stoffel
Quoted 66 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 53 lines · Amir Goldstein
Quoted 42 lines · Kent Overstreet
Quoted 40 lines · Amir Goldstein
Quoted 27 lines · Kent Overstreet
Quoted 25 lines · Amir Goldstein
Quoted 22 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 8 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 8 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
Quoted 7 lines
That's one way of putting it.
Please explain the use case.
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
Quoted 7 lines
Quoted 5 lines
Wait, what? How many people use casefolding, on a per-directory basis? It's stupid. Unix/Linux has used case-sensitive filesystems for years. Yes, linux supports other OSes which did do casefolding, but yikes... per-directory support is just insane. It should be per-filesystem only at BEST.
Quite a lot.
As I'm learning! :-) And sorry for the late reply... life got in
the way...
Quoted 4 lines
You may not realize this, but Valve has, quietly, behind the scenes, been intelligently funding a ton of Linux work with an eye towards not just gaming, but improving Linus on the desktop. And they've been deploying it too, you can buy a Steam deck today.
So... this arguement that you're improving the desktop by doing case-folding on filesystems smells bogus to me. I think it's more correct to say "Valve wants windows application support on top of linux, and since windows does case-folding, linux should too."
Which is more an arguement for supporting legacy backwards implementations. But for the future? God, why?
Quoted 5 lines
And a significant fraction of desktop users like to play games - we're not just all work. Windows ports need casefolding - alternatives have been discussed and they're non viable.
(I fondly remember the days when I had time for such things).
Heh heh. I'm not a Windows gamer at all, which is why I haven't run into this at all.
Quoted 1 line
Samba fileservers are a thing, too.
Sure. And I understand that Windows (even Windows 11!) is still does casefolding by default. Sigh... which sucks.
Quoted 3 lines
And for all us desktop/workstation users, not being able to use the same filesystem for wine and docker is the kind of jankiness that makes people say "maybe Linux isn't ready for the desktop after all".
I dunno... I just wonder why overlayfs (or some other level) can't just hide this and leave the lower levels alone. Yes, when you have 'foo' and 'Foo' on Linux, how do you handle it? It's painful.
Quoted 2 lines
Put aside your feelings on casefolding - this is about basic attention to detail.
Heh. I'm getting an education here. And disliking all the hoops filesystems have to go through to support this (IMHO) dumb feature of windows.
Quoted 38 lines
Quoted 17 lines
Quoted 1 line
That's _broken_.
So? what about my cross mounting of VMS filesystems with "foo.txt;3" version control so I can go back to previous versions? Why can't I do that from my Linux systems that's mounting that VMS image?
Just because it's done doesn't mean it's not dumb.
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Sure I can, I don't run windows applications to screw casefolding. :-)
Quoted 8 lines
And I personally LIKE having a seperate /boot and /home, because it gives isolation. The world is not just single user laptops with everything all on one disk or spread across a couple of disks using LVM or RAID or all of the above.
I also don't see any updates for the XFS tests, or any other filesystem tests, that actually checks and confirms this decidedly obtuse and dumb to implement idea.
Well, you certainly are making your personal feelings known :)
But for me, as the engineer designing and implementing this stuff, I don't let my personal feelings dictate.
That's not my place.
My job is to write code that works, works reliably, solves real problems, and lets users do the things they want with their machines.
All this drama over casefolding has been pure distraction, and I would appreciate if you and everyone else could tone it down now.
Quoted 82 lines · John Stoffel
Quoted 49 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 47 lines · Amir Goldstein
Quoted 36 lines · Kent Overstreet
Quoted 35 lines · Amir Goldstein
Quoted 24 lines · Kent Overstreet
Quoted 23 lines · Amir Goldstein
Quoted 21 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 7 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 7 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
That's one way of putting it.
Quoted 7 lines
Quoted 5 lines
Please explain the use case.
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
Wait, what? How many people use casefolding, on a per-directory basis? It's stupid. Unix/Linux has used case-sensitive filesystems for years. Yes, linux supports other OSes which did do casefolding, but yikes... per-directory support is just insane. It should be per-filesystem only at BEST.
Quoted 1 line
That's _broken_.
So? what about my cross mounting of VMS filesystems with "foo.txt;3" version control so I can go back to previous versions? Why can't I do that from my Linux systems that's mounting that VMS image?
Just because it's done doesn't mean it's not dumb.
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Sure I can, I don't run windows applications to screw casefolding. :-)
And I personally LIKE having a seperate /boot and /home, because it gives isolation. The world is not just single user laptops with everything all on one disk or spread across a couple of disks using LVM or RAID or all of the above.
Quoted 7 lines
I also don't see any updates for the XFS tests, or any other filesystem tests, that actually checks and confirms this decidedly obtuse and dumb to implement idea.
John
Hi there,
would you partition different subdirs of your /home? So there is .local/share/containers where users put their container-stuff (at least podman does). Then there is .wine where case-folding-craziness lives. And then there is the mess that is Steam, which does all kinds of containery case-foldy stuff. As much as I would like to keep these things apart, it is not feasible. Not for me as a "power user", and certainly far out of reach for average Joe user.
Just my 2 ct, greets
/Malte
Quoted 100 lines · Malte Schröder
Quoted 82 lines · John Stoffel
Quoted 49 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kent" == Kent Overstreet <[email protected]> writes:
Quoted 47 lines · Amir Goldstein
Quoted 36 lines · Kent Overstreet
Quoted 35 lines · Amir Goldstein
Quoted 24 lines · Kent Overstreet
Quoted 23 lines · Amir Goldstein
Quoted 21 lines · Kent Overstreet
This series allows overlayfs and casefolding to safely be used on the same filesystem by providing exclusion to ensure that overlayfs never has to deal with casefolded directories.
Currently, overlayfs can't be used
_atall_if a filesystem even supports casefolding, which is really nasty for users.Components:
- filesystem has to track, for each directory, "does any _descendent_ have casefolding enabled"
- new inode flag to pass this to VFS layer
- new dcache methods for providing refs for overlayfs, and filesystem methods for safely clearing this flag
- new superblock flag for indicating to overlayfs & dcache "filesystem supports casefolding, it's safe to use provided new dcache methods are used"
Quoted 7 lines
Quoted 6 lines
I don't think that this is really needed.
Too bad you did not ask before going through the trouble of this implementation.
I think it is enough for overlayfs to know the THIS directory has no casefolding.
overlayfs works on trees, not directories...
I know how overlayfs works...
Quoted 7 lines
Quoted 3 lines
I've explained why I don't think that sanitizing the entire tree is needed for creating overlayfs over a filesystem that may enable casefolding on some of its directories.
So, you want to move error checking from mount time, where we _just_ did a massive API rework so that we can return errors in a way that users will actually see them - to open/lookup, where all we have are a small fixed set of error codes?
That's one way of putting it.
Quoted 7 lines
Quoted 5 lines
Please explain the use case.
When is overlayfs created over a subtree that is only partially case folded? Is that really so common that a mount time error justifies all the vfs infrastructure involved?
Amir, you've got two widely used filesystem features that conflict and can't be used on the same filesystem.
Wait, what? How many people use casefolding, on a per-directory basis? It's stupid. Unix/Linux has used case-sensitive filesystems for years. Yes, linux supports other OSes which did do casefolding, but yikes... per-directory support is just insane. It should be per-filesystem only at BEST.
Quoted 1 line
That's _broken_.
So? what about my cross mounting of VMS filesystems with "foo.txt;3" version control so I can go back to previous versions? Why can't I do that from my Linux systems that's mounting that VMS image?
Just because it's done doesn't mean it's not dumb.
Quoted 5 lines
Users hate partitioning just for separate /boot and /home, having to partition for different applications is horrible. And since overlay fs is used under the hood by docker, and casefolding is used under the hood for running Windows applications, this isn't something people can predict in advance.
Sure I can, I don't run windows applications to screw casefolding. :-)
And I personally LIKE having a seperate /boot and /home, because it gives isolation. The world is not just single user laptops with everything all on one disk or spread across a couple of disks using LVM or RAID or all of the above.
Quoted 7 lines
I also don't see any updates for the XFS tests, or any other filesystem tests, that actually checks and confirms this decidedly obtuse and dumb to implement idea.
John
Hi there,
would you partition different subdirs of your /home? So there is .local/share/containers where users put their container-stuff (at least podman does). Then there is .wine where case-folding-craziness lives. And then there is the mess that is Steam, which does all kinds of containery case-foldy stuff. As much as I would like to keep these things apart, it is not feasible. Not for me as a "power user", and certainly far out of reach for average Joe user.
Quoted 1 line
Just my 2 ct, greets
"But just disable it globally" How about no. Sure, ext4 has that flag, but bcachefs by design does not. And a change that already made it into 6.15 has made it trip overlayfs as it is now, unconditionally. The purpose of this new implementation is to make it work, and satisfy the condition that overlayfs is guaranteed no casefolding in its tree, and that nobody may create a new folder inside that tree and suddenly turn on casefolding on it.
And this change also makes it possible to use it with ext4 with the global casefolding flag enabled. It shouldn't be necessary to have a global killswitch, these features should be able to live together on the same filesystem as long as they're not touching each other, and aren't allowed to touch each other.
Quoted 2 lines
/Malte