SSH server starts too late
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Omar,
On 2020-02-21 13:00, Hammami Omar <[email protected]> wrote
Quoted 5 lines
I am using the buildroot version "2019.11.1" and I have noticed that my SSH
server starts too late.
I was using an old version "2017.08" and the SSH server was starting normally.
This issue is not related to Buildroot, but caused by changes in libopenssl (which in turn is used by OpenSSH), starting around the middle of last year. You will find a lot of background information when you search the www for 'boot-time entropy starvation'; the cause is, reading random data from /dev/urandom blocks until the kernel entropy pool has been initialized.
Workarounds:
(a) Patch libopenssl (not recommended for nodes directly connected to the WAN).
(b) Use a more recent kernel; version 5.4.y introduced a temporary fix for this issue.
(c) Use other sources of extra randomness, as proposed by Peter Seiderer.
Kind regards,
Andreas
Hello Andreas,
Thank you for your response.
In fact, I have tried to add extra randomnes by enabling "haveged". The entropy has increased but the problem is the same.
Is it possible that the entropy value is always less than the wanted one ?
Kind regards,
Omar
Le ven. 21 f?vr. 2020 ? 13:51, Andreas Ziegler <[email protected]> a ?crit :
Quoted 25 lines
Hi Omar,
On 2020-02-21 13:00, Hammami Omar <[email protected]> wrote
Quoted 5 lines
I am using the buildroot version "2019.11.1" and I have noticed that my SSH
server starts too late.
I was using an old version "2017.08" and the SSH server was starting normally.This issue is not related to Buildroot, but caused by changes in libopenssl (which in turn is used by OpenSSH), starting around the middle of last year. You will find a lot of background information when you search the www for 'boot-time entropy starvation'; the cause is, reading random data from
/dev/urandomblocks until the kernel entropy pool has been initialized.Workarounds:
(a) Patch libopenssl (not recommended for nodes directly connected to the WAN).
(b) Use a more recent kernel; version 5.4.y introduced a temporary fix for this issue.
(c) Use other sources of extra randomness, as proposed by Peter Seiderer.
Quoted 3 lines
Kind regards,
Andreas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200224/28027cac/attachment.html>
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines
Message: 31
Date: Mon, 24 Feb 2020 10:29:42 +0100
From: Hammami Omar <[email protected]>
To: Andreas Ziegler <[email protected]>
Cc: buildroot at busybox.net, Peter Seiderer <[email protected]>
Subject: Re: [Buildroot] SSH server starts too lateHello Andreas,
Quoted 8 lines
Thank you for your response.
In fact, I have tried to add extra randomnes by enabling "haveged". The entropy has increased but the problem is the same.
Is it possible that the entropy value is always less than the wanted one ?Kind regards,
Omar
Hi Omar,
The kernel entropy pool needs to be "seeded"; after that it works as intended. Until the seeding is finished, calls to getentropy() or reads from /dev/random block; reads from /dev/urandom work, but print warnings in the kernel log. Thus it seems that there is always less entropy than needed, because successful reads do not produce warning messages:
# dmesg | grep random [ 0.070842] 000: random: get_random_u32 called from 0x8b299601 with crng_init=0 [ 9.553915] 000: random: fast init done [ 11.522913] 000: random: dd: uninitialized urandom read (512 bytes read) [ 14.271888] 000: random: wpa_supplicant: uninitialized urandom read (32 bytes read) [ 14.307673] 000: random: mktemp: uninitialized urandom read (6 bytes read) [ 23.668125] 000: random: mktemp: uninitialized urandom read (6 bytes read) [ 23.863680] 000: random: mktemp: uninitialized urandom read (6 bytes read) [ 24.003307] 000: random: sshd: uninitialized urandom read (32 bytes read) [ 221.067499] 000: random: sshd: uninitialized urandom read (32 bytes read) [ 221.092863] 000: random: sshd: uninitialized urandom read (32 bytes read) [ 221.405090] 000: random: sshd: uninitialized urandom read (32 bytes read) [ 327.117294] 000: random: crng init done [ 327.117305] 000: random: 1 urandom warning(s) missed due to ratelimiting
Changes in libopenssh, starting with version 1.1.1c, try to enforce a blocking behaviour (regardless of the device used) until the kernel pool is ready.
Increasing entropy can be achieved by typing on the keyboard, generating I/O from physical disks, or by using hardware devices (RNG). Another source of randomness is the patch that was introduced by Linus Torvalds in kernel 5.4.y:
https://lkml.org/lkml/2019/9/18/1078
Kind regards,
Andreas
A message that is not in the archive (sent off-list or not yet received): CAGSpp9ng2yxCK=RZeq1Z_Bce4pwij9FQED23SpGuvjqm5awHhg@mail.gmail.com
On 2020-02-24 13:33, Hammami Omar wrote:
Hi Omar,
Quoted 4 lines
The problem is that, in kernel logs, I cannot see that my ssh server was blocked.
That is why, I am thinking if my problem is really du to entropy or no.
You should experience a time lag in kernel log time stamps when that happens. If this lag is caused by an entropy issue, starting to type randomly on the keyboard should resolve it quite fast. This will speed up the time until you see the final random output:
Quoted 1 line
[ 77.847665] random: nonblocking pool is initialized
The SSH server will definitely not be operative until this pool is initialized.
Kind regards,
Andreas
Hi,
In fact, my problem is not seen in my PC. Rather, it is seen in my embedded board which has a Linux OS.
That is why I cannot use a keyboard.
Le lun. 24 f?vr. 2020 ? 14:22, Andreas Ziegler <[email protected]> a ?crit :
Quoted 14 lines · Hammami Omar
Hi Omar,
Quoted 4 lines
The problem is that, in kernel logs, I cannot see that my ssh server was blocked.
That is why, I am thinking if my problem is really du to entropy or no.You should experience a time lag in kernel log time stamps when that happens. If this lag is caused by an entropy issue, starting to type randomly on the keyboard should resolve it quite fast. This will speed up the time until you see the final random output:
Quoted 8 lines
Quoted 1 line
[ 77.847665] random: nonblocking pool is initializedThe SSH server will definitely not be operative until this pool is initialized.
Kind regards,
Andreas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200224/32f5e76c/attachment.html>
Hello,
Does anyone know the modification or the patch that was applied to resolve the problem in the kernel version 5.4.y ?
Regards,
Omar
Le lun. 24 f?vr. 2020 ? 15:07, Hammami Omar <[email protected]> a ?crit :
Quoted 6 lines
Hi,
In fact, my problem is not seen in my PC. Rather, it is seen in my embedded board which has a Linux OS.
That is why I cannot use a keyboard.
Quoted 26 lines
Le lun. 24 f?vr. 2020 ? 14:22, Andreas Ziegler <[email protected]> a ?crit :
Quoted 22 lines · Hammami Omar
Hi Omar,
Quoted 4 lines
The problem is that, in kernel logs, I cannot see that my ssh server was blocked.
That is why, I am thinking if my problem is really du to entropy or no.You should experience a time lag in kernel log time stamps when that happens. If this lag is caused by an entropy issue, starting to type randomly on the keyboard should resolve it quite fast. This will speed up the time until you see the final random output:
Quoted 1 line
[ 77.847665] random: nonblocking pool is initializedThe SSH server will definitely not be operative until this pool is initialized.
Kind regards,
Andreas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200224/3782cf87/attachment.html>