dcache: allow word-at-a-time name hashing with big-endian CPUs
Subsystems (2)
PATCH v2 1/2 dcache: allow word-at-a-time name hashing with big-endian CPUs
When explicitly hashing the end of a string with the word-at-a-time interface, we have to be careful which end of the word we pick up.
On big-endian CPUs, the upper-bits will contain the data we're after, so ensure we generate our masks accordingly (and avoid hashing whatever random junk may have been sitting after the string).
This patch adds a new dcache helper, bytemask_from_count, which creates a mask appropriate for the CPU endianness.
---
v1 -> v2: moved shifting out into a macro to avoid inline the #ifdefs
I didn't bother checking for CONFIG_DCACHE_WORD_ACCESS, since the macros are harmless enough as they are.
fs/dcache.c | 2 +- fs/namei.c | 7 +------ include/linux/dcache.h | 2 ++ 3 files changed, 4 insertions(+), 7 deletions(-)
fs/dcache.c +1 -1
@@ -192,7 +192,7 @@ static inline int dentry_string_cmp(const unsigned char *cs, const unsigned charif (!tcount)return 0 ; } - mask = ~(~0ul << tcount*8 ); + mask =bytemask_from_count (tcount);return unlikely (!!((a ^ b) & mask)); }
fs/namei.c +1 -6
@@ -1598,11 +1598,6 @@ static inline int nested_symlink(struct path *path, struct nameidata *nd)* do a "get_unaligned()" if this helps and is sufficiently * fast. * -* - Little-endian machines (so that we can generate the mask -* of low bytes efficiently). Again, we *could* do a byte -* swapping load on big-endian architectures if that is not -* expensive enough to make the optimization worthless. -* * - non-CONFIG_DEBUG_PAGEALLOC configurations (so that we * do not trap on the (extremely unlikely) case of a page * crossing operation.
@@ -1646,7 +1641,7 @@ unsigned int full_name_hash(const unsigned char *name, unsigned int len)if (!len) goto done; } - mask = ~(~0ul << len*8 ); + mask =bytemask_from_count (len); hash += mask & a; done:return fold_hash (hash);
include/linux/dcache.h +2 -0
@@ -29,8 +29,10 @@ struct vfsmount;/* The hash is always the low bits of hash_len */ #ifdef __LITTLE_ENDIAN#define HASH_LEN_DECLARE u32 hash; u32 len; +#define bytemask_from_count (cnt) (~(~0ul << (cnt)*8))#else #define HASH_LEN_DECLARE u32 len; u32 hash; +#define bytemask_from_count (cnt) (~(~0ul >> (cnt)*8))#endif /*
Signature · 1 line
1.8.2.2
PATCH v2 2/2 word-at-a-time: provide generic big-endian zero_bytemask implementation
Whilst architectures may be able to do better than this (which they can, by simply defining their own macro), this is a generic stab at a zero_bytemask implementation for the asm-generic, big-endian word-at-a-time implementation.
On arm64, a clz instruction is used to implement the fls efficiently.
--- include/asm-generic/word-at-a-time.h | 8 ++++++++ 1 file changed, 8 insertions(+)
include/asm-generic/word-at-a-time.h +8 -0
@@ -49,4 +49,12 @@ static inline bool has_zero(unsigned long val, unsigned long *data, const struct return (val + c->high_bits) & ~rhs; } +#ifndef zero_bytemask +#ifdef CONFIG_64BIT +#define zero_bytemask (mask) (~0ul << fls64(mask)) +#else +#define zero_bytemask (mask) (~0ul << fls(mask)) +#endif /* CONFIG_64BIT */ +#endif /* zero_bytemask */ +#endif /* _ASM_WORD_AT_A_TIME_H */
Signature · 1 line
1.8.2.2
Ok, looks good. So I'm taking these even though it's outside the merge window - they're small, and until you enable DCACHE_WORD_ACCESS on big-endian they don't actually do anything.
Ben - you might want to check if this makes DCACHE_WORD_ACCESS useful on PowerPC too. Right now that requires an efficient fls() implementation, but I think you have "cntlz" like ARM does.
Linus
Quoted 7 lines · Linus Torvalds
Ok, looks good. So I'm taking these even though it's outside the merge window - they're small, and until you enable
DCACHE_WORD_ACCESSon big-endian they don't actually do anything.Ben - you might want to check if this makes
DCACHE_WORD_ACCESSuseful on PowerPC too. Right now that requires an efficientfls()implementation, but I think you have "cntlz" like ARM does.
Thanks. I'll see if I can get somebody to measure it. It should be good though.
Cheers,
Ben.