Skip to contentSkip to conversation

dcache: allow word-at-a-time name hashing with big-endian CPUs

PATCH In review Will Deacon 3 participants 4 messages Dec 12, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 dcache: allow word-at-a-time name hashing with big-endian CPUs

When explicitly hashing the end of a string with the word-at-a-time interface, we have to be careful which end of the word we pick up.

On big-endian CPUs, the upper-bits will contain the data we're after, so ensure we generate our masks accordingly (and avoid hashing whatever random junk may have been sitting after the string).

This patch adds a new dcache helper, bytemask_from_count, which creates a mask appropriate for the CPU endianness.

---

v1 -> v2: moved shifting out into a macro to avoid inline the #ifdefs
I didn't bother checking for CONFIG_DCACHE_WORD_ACCESS, since the macros are harmless enough as they are.

 fs/dcache.c            | 2 +-
 fs/namei.c             | 7 +------
 include/linux/dcache.h | 2 ++
 3 files changed, 4 insertions(+), 7 deletions(-)
fs/dcache.c +1 -1
index 4bdb300b16e2..6055d61811d3 100644
@@ -192,7 +192,7 @@ static inline int dentry_string_cmp(const unsigned char *cs, const unsigned char
 		if (!tcount)
 			return 0;
 	}
-	mask = ~(~0ul << tcount*8);
+	mask = bytemask_from_count(tcount);
 	return unlikely(!!((a ^ b) & mask));
 }
 
fs/namei.c +1 -6
index c53d3a9547f9..3531deebad30 100644
@@ -1598,11 +1598,6 @@ static inline int nested_symlink(struct path *path, struct nameidata *nd)
  *   do a "get_unaligned()" if this helps and is sufficiently
  *   fast.
  *
- * - Little-endian machines (so that we can generate the mask
- *   of low bytes efficiently). Again, we *could* do a byte
- *   swapping load on big-endian architectures if that is not
- *   expensive enough to make the optimization worthless.
- *
  * - non-CONFIG_DEBUG_PAGEALLOC configurations (so that we
  *   do not trap on the (extremely unlikely) case of a page
  *   crossing operation.
@@ -1646,7 +1641,7 @@ unsigned int full_name_hash(const unsigned char *name, unsigned int len)
 		if (!len)
 			goto done;
 	}
-	mask = ~(~0ul << len*8);
+	mask = bytemask_from_count(len);
 	hash += mask & a;
 done:
 	return fold_hash(hash);
include/linux/dcache.h +2 -0
index 57e87e749a48..bf72e9ac6de0 100644
@@ -29,8 +29,10 @@ struct vfsmount;
 /* The hash is always the low bits of hash_len */
 #ifdef __LITTLE_ENDIAN
  #define HASH_LEN_DECLARE u32 hash; u32 len;
+ #define bytemask_from_count(cnt)	(~(~0ul << (cnt)*8))
 #else
  #define HASH_LEN_DECLARE u32 len; u32 hash;
+ #define bytemask_from_count(cnt)	(~(~0ul >> (cnt)*8))
 #endif
 
 /*
Signature · 1 line
1.8.2.2
#1 lore raw

PATCH v2 2/2 word-at-a-time: provide generic big-endian zero_bytemask implementation

Whilst architectures may be able to do better than this (which they can, by simply defining their own macro), this is a generic stab at a zero_bytemask implementation for the asm-generic, big-endian word-at-a-time implementation.

On arm64, a clz instruction is used to implement the fls efficiently.

---
 include/asm-generic/word-at-a-time.h | 8 ++++++++
 1 file changed, 8 insertions(+)
include/asm-generic/word-at-a-time.h +8 -0
index 3f21f1b72e45..d3909effd725 100644
@@ -49,4 +49,12 @@ static inline bool has_zero(unsigned long val, unsigned long *data, const struct
 	return (val + c->high_bits) & ~rhs;
 }
 
+#ifndef zero_bytemask
+#ifdef CONFIG_64BIT
+#define zero_bytemask(mask)	(~0ul << fls64(mask))
+#else
+#define zero_bytemask(mask)	(~0ul << fls(mask))
+#endif /* CONFIG_64BIT */
+#endif /* zero_bytemask */
+
 #endif /* _ASM_WORD_AT_A_TIME_H */
Signature · 1 line
1.8.2.2
#2 lore raw
In reply to Will Deacon

Ok, looks good. So I'm taking these even though it's outside the merge window - they're small, and until you enable DCACHE_WORD_ACCESS on big-endian they don't actually do anything.

Ben - you might want to check if this makes DCACHE_WORD_ACCESS useful on PowerPC too. Right now that requires an efficient fls() implementation, but I think you have "cntlz" like ARM does.

Linus

#3 lore raw
Quoted 7 lines · Linus Torvalds

Ok, looks good. So I'm taking these even though it's outside the merge window - they're small, and until you enable DCACHE_WORD_ACCESS on big-endian they don't actually do anything.

Ben - you might want to check if this makes DCACHE_WORD_ACCESS useful on PowerPC too. Right now that requires an efficient fls() implementation, but I think you have "cntlz" like ARM does.

Thanks. I'll see if I can get somebody to measure it. It should be good though.

Cheers,
Ben.