Skip to contentSkip to conversation

Reading entropy_avail file appears to consume entropy

DISCUSSION Jan Ceuleers 2 participants 2 messages Apr 5, 2010 2 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): b7822cd7-97f3-489e-a16b-686b0419d3f2@k19g2000yqn.googlegroups.com

#2 lore raw
Quoted 33 lines · Alexander Konovalenko
Quoted 8 lines · Jan Ceuleers

I'm using the 2.6.31 kernel that comes with Ubuntu 9.10.

If I

   # watch cat /proc/sys/kernel/random/entropy_avail

then the size of the entropy pool falls rapidly (by more than 200 bytes per 2s interval). It settles down around 160 bytes.

Jan, did you find out anything interesting about this issue?

I have a wild guess, although I have no idea whether it can be correct. I couldn't catch any user-space /dev/random or /dev/urandom readers with fuser, so I think something in the kernel is using up the entropy. If I remember correctly, recent Ubuntu releases were supposed to include a security feature that randomizes memory layout in order to mitigate some kinds of security vulnerabilities. What if each time a new process is started the kernel needs to obtain a significant number of random bytes? Here is some supporting evidence. I can reproduce the behavior you describe on a vanilla Ubuntu 9.10 system (without latest updates). But if I watch entropy_avail using a Python script that does not start a new process every time, then the available entropy amount won't decrease. (The system is otherwise idle.) Give it a try:

$ python
import sys, time
while True:
  sys.stdout.write(open('/proc/sys/kernel/random/entropy_avail',
'r').read())
  time.sleep(1)
Quoted 5 lines

It would be great if someone with knowledge about this could confirm or refute my guess.

-- Alexander

Alexander,

Thanks, this never made it out to LKML so I'm forwarding it now.

This does sound quite plausible; does anyone from Ubuntu want to chip in?

Thanks, Jan

#3 lore raw
Quoted 37 lines · Jan Ceuleers
Quoted 36 lines · Alexander Konovalenko
Quoted 8 lines · Jan Ceuleers

I'm using the 2.6.31 kernel that comes with Ubuntu 9.10.

If I

   # watch cat /proc/sys/kernel/random/entropy_avail

then the size of the entropy pool falls rapidly (by more than 200 bytes per 2s interval). It settles down around 160 bytes.

Jan, did you find out anything interesting about this issue?

I have a wild guess, although I have no idea whether it can be correct. I couldn't catch any user-space /dev/random or /dev/urandom readers with fuser, so I think something in the kernel is using up the entropy. If I remember correctly, recent Ubuntu releases were supposed to include a security feature that randomizes memory layout in order to mitigate some kinds of security vulnerabilities. What if each time a new process is started the kernel needs to obtain a significant number of random bytes? Here is some supporting evidence. I can reproduce the behavior you describe on a vanilla Ubuntu 9.10 system (without latest updates). But if I watch entropy_avail using a Python script that does not start a new process every time, then the available entropy amount won't decrease. (The system is otherwise idle.) Give it a try:

$ python
import sys, time
while True:
  sys.stdout.write(open('/proc/sys/kernel/random/entropy_avail',
'r').read())
  time.sleep(1)

It would be great if someone with knowledge about this could confirm or refute my guess.

Quoted 9 lines
Quoted 2 lines

-- Alexander

Alexander,

Thanks, this never made it out to LKML so I'm forwarding it now.

This does sound quite plausible; does anyone from Ubuntu want to chip in?

Nothing related to Ubuntu I guess

commit f06295b44c296c8fb08823a3118468ae343b60f2

Author: Kees Cook <[email protected]>
Date:   Wed Jan 7 18:08:52 2009 -0800

ELF: implement AT_RANDOM for glibc PRNG seeding

While discussing[1] the need for glibc to have access to random bytes during program load, it seems that an earlier attempt to implement AT_RANDOM got stalled. This implements a random 16 byte string, available to every ELF program via a new auxv AT_RANDOM vector.

[1] http://sourceware.org/ml/libc-alpha/2008-10/msg00006.html

Ulrich said:

glibc needs right after startup a bit of random data for internal protections (stack canary etc). What is now in upstream glibc is that we always unconditionally open /dev/urandom, read some data, and use it. For every process startup. That's slow.

...

The solution is to provide a limited amount of random data to the starting process in the aux vector. I suggested 16 bytes and this is what the patch implements. If we need only 16 bytes or less we use the data directly. If we need more we'll use the 16 bytes to see a PRNG. This avoids the costly /dev/urandom use and it allows the kernel to use the most adequate source of random data for this purpose. It might not be the same pool as that for /dev/urandom.

Concerns were expressed about the depletion of the randomness pool. But this patch doesn't make the situation worse, it doesn't deplete entropy more than happens now.