Skip to contentSkip to conversation

RDSEED support for the Linux kernel

PATCH Applied H. Peter Anvin 3 participants 8 messages Mar 17, 2014 – Mar 18, 2014

Subsystems (3)
x86 PowerPC Random number driver
lore
Share & tools
lore

H. Peter Anvin

author
#0 lore raw

v2 cover RDSEED support for the Linux kernel

Upcoming Intel silicon adds a new RDSEED instruction. Whereas RDRAND returns output from a PRNG, the RDSEED instruction returns fully conditioned entropy that is suitable for use as seeds to a PRNG.

This patchset adds support for RDSEED in the Linux kernel in three places:

1. During bootup, use RDSEED to initialize the entropy pool if available (we already use RDRAND for this). We don't add any credit at this point, but it will give much better starting point.

2. In the slow path to add_interrupt_randomness, executed once per second, we take a single RDSEED sample and mix it into the entropy pool, crediting it at 50% of its rated entropy. This was suggested by Linus.

3. If we are about to block on /dev/random due to lack of entropy, attempt an "emergency pool refill" using RDSEED.

Changes since version 1:

a. Rebased on top of random.git:dev.
b. Unbreak the PowerPC build (I had managed to miss that PowerPC had
grown archrandom.h support.)
c. Remove duplicate dummy function definitions in <linux/random.h>. d. Add a fourth patch containing a microoptimization: avoid the loop
in arch_random_refill() if arch_get_random_seed*() is unavailable.

Comments are, of course, appreciated.

Ted, if you are OK with this could you add this to random.git:dev so linux-next can pick it up?

-hpa

author
#1 lore raw

PATCH v2 1/4 x86, random: Enable the RDSEED instruction

Upcoming Intel silicon adds a new RDSEED instruction, which is similar to RDRAND but provides a stronger guarantee: unlike RDRAND, RDSEED will always reseed the PRNG from the true random number source between each read. Thus, the output of RDSEED is guaranteed to be 100% entropic, unlike RDRAND which is only architecturally guaranteed to be 1/512 entropic (although in practice is much more.)

The RDSEED instruction takes the same time to execute as RDRAND, but RDSEED unlike RDRAND can legitimately return failure (CF=0) due to entropy exhaustion if too many threads on too many cores are hammering the RDSEED instruction at the same time. Therefore, we have to be more conservative and only use it in places where we can tolerate failures.

This patch introduces the primitives arch_get_random_seed_{int,long}() but does not use it yet.

Signed-off-by H. Peter Anvin <[email protected]>
Reviewed-by Ingo Molnar <[email protected]>
Cc Benjamin Herrenschmidt <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Michael Ellerman <[email protected]>
---
 arch/powerpc/include/asm/archrandom.h |  9 +++++++++
 arch/x86/include/asm/archrandom.h     | 34 +++++++++++++++++++++++++++++++++-
 include/linux/random.h                |  8 ++++++++
 3 files changed, 50 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/archrandom.h +9 -0
index d853d16..801beba 100644
@@ -27,6 +27,15 @@ static inline int arch_get_random_int(unsigned int *v)
 
 int powernv_get_random_long(unsigned long *v);
 
+static inline int arch_get_random_seed_long(unsigned long *v)
+{
+	return 0;
+}
+static inline int arch_get_random_seed_int(unsigned int *v)
+{
+	return 0;
+}
+
 #endif /* CONFIG_ARCH_RANDOM */
 
 #endif /* _ASM_POWERPC_ARCHRANDOM_H */
arch/x86/include/asm/archrandom.h +33 -1
index 0d9ec77..ba064d5 100644
@@ -1,7 +1,7 @@
 /*
  * This file is part of the Linux kernel.
  *
- * Copyright (c) 2011, Intel Corporation
+ * Copyright (c) 2011-2014, Intel Corporation
  * Authors: Fenghua Yu <[email protected]>,
  *          H. Peter Anvin <[email protected]>
  *

@@ -31,14 +31,27 @@
 #define RDRAND_RETRY_LOOPS	10
 
 #define RDRAND_INT	".byte 0x0f,0xc7,0xf0"
+#define RDSEED_INT	".byte 0x0f,0xc7,0xf8"
 #ifdef CONFIG_X86_64
 # define RDRAND_LONG	".byte 0x48,0x0f,0xc7,0xf0"
+# define RDSEED_LONG	".byte 0x48,0x0f,0xc7,0xf8"
 #else
 # define RDRAND_LONG	RDRAND_INT
+# define RDSEED_LONG	RDSEED_INT
 #endif
 
 #ifdef CONFIG_ARCH_RANDOM
 
+/* A single attempt at RDSEED */
+static inline bool rdseed_long(unsigned long *v)
+{
+	unsigned char ok;
+	asm volatile(RDSEED_LONG "\n\t"
+		     "setc %0"
+		     : "=qm" (ok), "=a" (*v));
+	return ok;
+}
+
 #define GET_RANDOM(name, type, rdrand, nop)			\
 static inline int name(type *v)					\
 {								\

@@ -56,16 +69,35 @@ static inline int name(type *v)					\
 	return ok;						\
 }
 
+#define GET_SEED(name, type, rdseed, nop)			\
+static inline int name(type *v)					\
+{								\
+	unsigned char ok;					\
+	alternative_io("movb $0, %0\n\t"			\
+		       nop,					\
+		       rdseed "\n\t"				\
+		       "setc %0",				\
+		       X86_FEATURE_RDSEED,                      \
+		       ASM_OUTPUT2("=q" (ok), "=a" (*v)));	\
+	return ok;						\
+}
+
 #ifdef CONFIG_X86_64
 
 GET_RANDOM(arch_get_random_long, unsigned long, RDRAND_LONG, ASM_NOP5);
 GET_RANDOM(arch_get_random_int, unsigned int, RDRAND_INT, ASM_NOP4);
 
+GET_SEED(arch_get_random_seed_long, unsigned long, RDSEED_LONG, ASM_NOP5);
+GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
+
 #else
 
 GET_RANDOM(arch_get_random_long, unsigned long, RDRAND_LONG, ASM_NOP3);
 GET_RANDOM(arch_get_random_int, unsigned int, RDRAND_INT, ASM_NOP3);
 
+GET_SEED(arch_get_random_seed_long, unsigned long, RDSEED_LONG, ASM_NOP4);
+GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
+
 #endif /* CONFIG_X86_64 */
 
 #endif  /* CONFIG_ARCH_RANDOM */
include/linux/random.h +8 -0
index 4002b3d..ddaddec 100644
@@ -72,6 +72,14 @@ static inline int arch_get_random_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_get_random_seed_long(unsigned long *v)
+{
+	return 0;
+}
+static inline int arch_get_random_seed_int(unsigned int *v)
+{
+	return 0;
+}
 #endif
 
 /* Pseudo random number generator from numerical recipes. */
Signature · 1 line
1.8.5.3
author
#2 lore raw

PATCH v2 2/4 random: Use arch_get_random_seed*() at init time and once a second

In reply to H. Peter Anvin

Use arch_get_random_seed*() in two places in the Linux random driver (drivers/char/random.c):

1. During entropy pool initialization, use RDSEED in favor of RDRAND, with a fallback to the latter. Entropy exhaustion is unlikely to happen there on physical hardware as the machine is single-threaded at that point, but could happen in a virtual machine. In that case, the fallback to RDRAND will still provide more than adequate entropy pool initialization.

2. Once a second, issue RDSEED and, if successful, feed it to the entropy pool. To ensure an extra layer of security, only credit half the entropy just in case.

Suggested-by Linus Torvalds <[email protected]>
Reviewed-by Ingo Molnar <[email protected]>
Signed-off-by H. Peter Anvin <[email protected]>
---
 drivers/char/random.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)
drivers/char/random.c +20 -4
index d07575c..a4bea77 100644
@@ -844,6 +844,8 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	cycles_t		cycles = random_get_entropy();
 	__u32			input[4], c_high, j_high;
 	__u64			ip;
+	unsigned long		seed;
+	int			credit;
 
 	c_high = (sizeof(cycles) > 4) ? cycles >> 32 : 0;
 	j_high = (sizeof(now) > 4) ? now >> 32 : 0;

@@ -862,20 +864,33 @@ void add_interrupt_randomness(int irq, int irq_flags)
 
 	r = nonblocking_pool.initialized ? &input_pool : &nonblocking_pool;
 	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool), NULL);
+
 	/*
 	 * If we don't have a valid cycle counter, and we see
 	 * back-to-back timer interrupts, then skip giving credit for
-	 * any entropy.
+	 * any entropy, otherwise credit 1 bit.
 	 */
+	credit = 1;
 	if (cycles == 0) {
 		if (irq_flags & __IRQF_TIMER) {
 			if (fast_pool->last_timer_intr)
-				return;
+				credit = 0;
 			fast_pool->last_timer_intr = 1;
 		} else
 			fast_pool->last_timer_intr = 0;
 	}
-	credit_entropy_bits(r, 1);
+
+	/*
+	 * If we have architectural seed generator, produce a seed and
+	 * add it to the pool.  For the sake of paranoia count it as
+	 * 50% entropic.
+	 */
+	if (arch_get_random_seed_long(&seed)) {
+		__mix_pool_bytes(r, &seed, sizeof(seed), NULL);
+		credit += sizeof(seed) * 4;
+	}
+
+	credit_entropy_bits(r, credit);
 }
 
 #ifdef CONFIG_BLOCK

@@ -1235,7 +1250,8 @@ static void init_std_data(struct entropy_store *r)
 	r->last_pulled = jiffies;
 	mix_pool_bytes(r, &now, sizeof(now), NULL);
 	for (i = r->poolinfo->poolbytes; i > 0; i -= sizeof(rv)) {
-		if (!arch_get_random_long(&rv))
+		if (!arch_get_random_seed_long(&rv) &&
+		    !arch_get_random_long(&rv))
 			rv = random_get_entropy();
 		mix_pool_bytes(r, &rv, sizeof(rv), NULL);
 	}
Signature · 1 line
1.8.5.3
author
#3 lore raw

PATCH v2 3/4 random: If we have arch_get_random_seed*(), try it before blocking

In reply to H. Peter Anvin

If we have arch_get_random_seed*(), try to use it for emergency refill of the entropy pool before giving up and blocking on /dev/random. It may or may not work in the moment, but if it does work, it will give the user better service than blocking will.

Reviewed-by Ingo Molnar <[email protected]>
Signed-off-by H. Peter Anvin <[email protected]>
---
 drivers/char/random.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
drivers/char/random.c +33 -0
index a4bea77..c35cee2 100644
@@ -1294,6 +1294,34 @@ void rand_initialize_disk(struct gendisk *disk)
 }
 #endif
 
+/*
+ * Attempt an emergency refill using arch_get_random_seed_long().
+ *
+ * As with add_interrupt_randomness() be paranoid and only
+ * credit the output as 50% entropic.
+ */
+static int arch_random_refill(void)
+{
+	const unsigned int nlongs = 64;	/* Arbitrary number */
+	unsigned int n = 0;
+	unsigned int i;
+	unsigned long buf[nlongs];
+
+	for (i = 0; i < nlongs; i++) {
+		if (arch_get_random_seed_long(&buf[n]))
+			n++;
+	}
+
+	if (n) {
+		unsigned int rand_bytes = n * sizeof(unsigned long);
+
+		mix_pool_bytes(&input_pool, buf, rand_bytes, NULL);
+		credit_entropy_bits(&input_pool, rand_bytes*4);
+	}
+
+	return n;
+}
+
 static ssize_t
 random_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
 {

@@ -1312,8 +1340,13 @@ random_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
 				  ENTROPY_BITS(&input_pool));
 		if (n > 0)
 			return n;
+
 		/* Pool is (near) empty.  Maybe wait and retry. */
 
+		/* First try an emergency refill */
+		if (arch_random_refill())
+			continue;
+
 		if (file->f_flags & O_NONBLOCK)
 			return -EAGAIN;
Signature · 1 line
1.8.5.3
author
#4 lore raw

PATCH v2 4/4 random: Add arch_has_random[_seed]()

In reply to H. Peter Anvin

Add predicate functions for having arch_get_random[_seed]*(). The only current use is to avoid the loop in arch_random_refill() when arch_get_random_seed_long() is unavailable.

Signed-off-by H. Peter Anvin <[email protected]>
Cc Benjamin Herrenschmidt <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Michael Ellerman <[email protected]>
---
 arch/powerpc/include/asm/archrandom.h | 9 +++++++++
 arch/x86/include/asm/archrandom.h     | 3 +++
 drivers/char/random.c                 | 3 +++
 include/linux/random.h                | 8 ++++++++
 4 files changed, 23 insertions(+)
arch/powerpc/include/asm/archrandom.h +9 -0
index 801beba..bde5311 100644
@@ -25,6 +25,11 @@ static inline int arch_get_random_int(unsigned int *v)
 	return rc;
 }
 
+static inline int arch_has_random(void)
+{
+	return !!ppc_md.get_random_long;
+}
+
 int powernv_get_random_long(unsigned long *v);
 
 static inline int arch_get_random_seed_long(unsigned long *v)

@@ -35,6 +40,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random_seed(void)
+{
+	return 0;
+}
 
 #endif /* CONFIG_ARCH_RANDOM */
arch/x86/include/asm/archrandom.h +3 -0
index ba064d5..c7ed4a6 100644
@@ -100,6 +100,9 @@ GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
 
 #endif /* CONFIG_X86_64 */
 
+#define arch_has_random()	static_cpu_has(X86_FEATURE_RDRAND)
+#define arch_has_random_seed()	static_cpu_has(X86_FEATURE_RDSEED)
+
 #endif  /* CONFIG_ARCH_RANDOM */
 
 extern void x86_init_rdrand(struct cpuinfo_x86 *c);
drivers/char/random.c +3 -0
index c35cee2..6b75713 100644
@@ -1307,6 +1307,9 @@ static int arch_random_refill(void)
 	unsigned int i;
 	unsigned long buf[nlongs];
 
+	if (!arch_has_random_seed())
+		return 0;
+
 	for (i = 0; i < nlongs; i++) {
 		if (arch_get_random_seed_long(&buf[n]))
 			n++;
include/linux/random.h +8 -0
index ddaddec..e7a221a 100644
@@ -72,6 +72,10 @@ static inline int arch_get_random_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random(void)
+{
+	return 0;
+}
 static inline int arch_get_random_seed_long(unsigned long *v)
 {
 	return 0;

@@ -80,6 +84,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random_seed(void)
+{
+	return 0;
+}
 #endif
 
 /* Pseudo random number generator from numerical recipes. */
Signature · 1 line
1.8.5.3
#5 lore raw

Re: [PATCH v2 4/4] random: Add arch_has_random[_seed]()

Quoted 3 lines · H. Peter Anvin

Add predicate functions for having arch_get_random[_seed]*(). The only current use is to avoid the loop in arch_random_refill() when arch_get_random_seed_long() is unavailable.

Paul, I think our HW rng on P7+ and later would qualify no ?

Cheers,
Ben.

Quoted 92 lines
Signed-off-by H. Peter Anvin <[email protected]>
Cc Benjamin Herrenschmidt <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Michael Ellerman <[email protected]>
---
 arch/powerpc/include/asm/archrandom.h | 9 +++++++++
 arch/x86/include/asm/archrandom.h     | 3 +++
 drivers/char/random.c                 | 3 +++
 include/linux/random.h                | 8 ++++++++
 4 files changed, 23 insertions(+)
arch/powerpc/include/asm/archrandom.h +9 -0
index 801beba..bde5311 100644
@@ -25,6 +25,11 @@ static inline int arch_get_random_int(unsigned int *v)
 	return rc;
 }
 
+static inline int arch_has_random(void)
+{
+	return !!ppc_md.get_random_long;
+}
+
 int powernv_get_random_long(unsigned long *v);
 
 static inline int arch_get_random_seed_long(unsigned long *v)

@@ -35,6 +40,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random_seed(void)
+{
+	return 0;
+}
 
 #endif /* CONFIG_ARCH_RANDOM */
arch/x86/include/asm/archrandom.h +3 -0
index ba064d5..c7ed4a6 100644
@@ -100,6 +100,9 @@ GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
 
 #endif /* CONFIG_X86_64 */
 
+#define arch_has_random()	static_cpu_has(X86_FEATURE_RDRAND)
+#define arch_has_random_seed()	static_cpu_has(X86_FEATURE_RDSEED)
+
 #endif  /* CONFIG_ARCH_RANDOM */
 
 extern void x86_init_rdrand(struct cpuinfo_x86 *c);
drivers/char/random.c +3 -0
index c35cee2..6b75713 100644
@@ -1307,6 +1307,9 @@ static int arch_random_refill(void)
 	unsigned int i;
 	unsigned long buf[nlongs];
 
+	if (!arch_has_random_seed())
+		return 0;
+
 	for (i = 0; i < nlongs; i++) {
 		if (arch_get_random_seed_long(&buf[n]))
 			n++;
include/linux/random.h +8 -0
index ddaddec..e7a221a 100644
@@ -72,6 +72,10 @@ static inline int arch_get_random_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random(void)
+{
+	return 0;
+}
 static inline int arch_get_random_seed_long(unsigned long *v)
 {
 	return 0;

@@ -80,6 +84,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random_seed(void)
+{
+	return 0;
+}
 #endif
 
 /* Pseudo random number generator from numerical recipes. */
author
#6 lore raw

Re: [PATCH v2 4/4] random: Add arch_has_random[_seed]()

Quoted 9 lines · Benjamin Herrenschmidt
Quoted 3 lines · H. Peter Anvin

Add predicate functions for having arch_get_random[_seed]*(). The only current use is to avoid the loop in arch_random_refill() when arch_get_random_seed_long() is unavailable.

Paul, I think our HW rng on P7+ and later would qualify no ?

Cheers,
Ben.

Just to be clear: since I don't know the parameters of the PowerPC hwrng I just put in the stub functions, leaving it up to you to analyze if it is suitable and analyze it. Do let me know if I can be of any assistance, of course.

-hpa

maintainer Applied
#7 lore raw
In reply to H. Peter Anvin
Quoted 9 lines · H. Peter Anvin

Changes since version 1:

a. Rebased on top of random.git:dev.
b. Unbreak the PowerPC build (I had managed to miss that PowerPC had
grown archrandom.h support.)
c. Remove duplicate dummy function definitions in <linux/random.h>. d. Add a fourth patch containing a microoptimization: avoid the loop
in arch_random_refill() if arch_get_random_seed*() is unavailable.

Quoted 4 lines

Comments are, of course, appreciated.

Ted, if you are OK with this could you add this to random.git:dev so linux-next can pick it up?

Thanks, applied to the random.git tree.

- Ted