Skip to contentSkip to conversation

Race condition? /tmp/net.ifaces and pre-pivot

DISCUSSION Warren Togami 3 participants 4 messages Jul 23, 2009 – Aug 7, 2009 2 messages not in the archive

lore
Share & tools
lore
#0 lore raw

I am running into an odd issue that is supposedto be impossible.

pre-pivot/20write-ifcfg.sh is often being run but fails to write net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment.
The mount was otherwise successful, and rdbreak before switch_root
sees that /tmp/net.ifaces exists.

With plain "root=dhcp" with a NFS rootfs, it seems to never happen. However with "root=dhcp bridge", perhaps 25-75% of the time it is failing to write ifcfg files.

Some kind of race going on? An inspection of the code seems to me that /tmp/net.ifaces should have already been created prior to pre-pivot?

Warren Togami
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe initramfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
In reply to a message not in the archive
Quoted 7 lines · Warren Togami

I am running into an odd issue that is supposedto be impossible.

pre-pivot/20write-ifcfg.sh is often being run but fails to write net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment. The mount was otherwise successful, and rdbreak before switch_root sees that /tmp/net.ifaces exists.

Quoted 7 lines

With plain "root=dhcp" with a NFS rootfs, it seems to never happen. However with "root=dhcp bridge", perhaps 25-75% of the time it is failing to write ifcfg files.

Some kind of race going on? An inspection of the code seems to me that /tmp/net.ifaces should have already been created prior to pre-pivot?

<dillow> warren: if you don't have any ip= lines on the command line, /tmp/net.ifaces gets created by netroot after the handler successfully completes
<dillow> warren: I think that races with the check loop in init
<dillow> warren: single processor box?
<dillow> warren: it looks like you could get scheduled away from netroot and the init command loop could notice that root is mounted and make it to the pre-pivot hook during a single quantum <dillow> warren: as a test, you could move the line '[ ! -f /tmp/net.ifaces ] && echo $netif > /tmp/net.ifaces' in netroot in front of the handler call <dillow> warren: that should work around your issue. I don't think it is a long term fix, though -- the multiple NIC case would need some work to be clean <dillow> warren: this would only affect NFS, as the other devices would create /dev/root and exit that way to the mount loop
<dillow> so they wouldn't sail through the mount loop due to the existence of $NEWROOT/proc
<warren> dillow: single processor qemu VM
<warren> dillow: so this race was happening before, only we didn't notice it since we weren't relying on anything
<warren> dillow: any suggestions of a substitution that wont break dash?
<dillow> warren: I think it got introduced with the initqueue stuff or a combination of that and changing the loops

Harald, any ideas?

Warren Togami
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe initramfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
In reply to a message not in the archive
Quoted 41 lines · Warren Togami
Quoted 7 lines · Warren Togami

I am running into an odd issue that is supposedto be impossible.

pre-pivot/20write-ifcfg.sh is often being run but fails to write net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment. The mount was otherwise successful, and rdbreak before switch_root sees that /tmp/net.ifaces exists.

Quoted 7 lines

With plain "root=dhcp" with a NFS rootfs, it seems to never happen. However with "root=dhcp bridge", perhaps 25-75% of the time it is failing to write ifcfg files.

Some kind of race going on? An inspection of the code seems to me that /tmp/net.ifaces should have already been created prior to pre-pivot?

<dillow> warren: if you don't have any ip= lines on the command line, /tmp/net.ifaces gets created by netroot after the handler successfully completes
<dillow> warren: I think that races with the check loop in init <dillow> warren: single processor box?
<dillow> warren: it looks like you could get scheduled away from netroot and the init command loop could notice that root is mounted and make it to the pre-pivot hook during a single quantum
<dillow> warren: as a test, you could move the line '[ ! -f /tmp/net.ifaces ] && echo $netif > /tmp/net.ifaces' in netroot in front of the handler call
<dillow> warren: that should work around your issue. I don't think it is a long term fix, though -- the multiple NIC case would need some work to be clean
<dillow> warren: this would only affect NFS, as the other devices would create /dev/root and exit that way to the mount loop <dillow> so they wouldn't sail through the mount loop due to the existence of $NEWROOT/proc
<warren> dillow: single processor qemu VM
<warren> dillow: so this race was happening before, only we didn't notice it since we weren't relying on anything
<warren> dillow: any suggestions of a substitution that wont break dash? <dillow> warren: I think it got introduced with the initqueue stuff or a combination of that and changing the loops

Quoted 1 line

Harald, any ideas?

if you rely on a udev event to complete the solution is to place a

udevadm settle

before doing anything
--
To unsubscribe from this list: send the line "unsubscribe initramfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw
In reply to a message not in the archive
Quoted 9 lines · Warren Togami

I am running into an odd issue that is supposedto be impossible.

pre-pivot/20write-ifcfg.sh is often being run but fails to write net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment.
The mount was otherwise successful, and rdbreak before switch_root sees
that /tmp/net.ifaces exists.

With plain "root=dhcp" with a NFS rootfs, it seems to never happen. However with "root=dhcp bridge", perhaps 25-75% of the time it is failing to write ifcfg files.

Some kind of race going on? An inspection of the code seems to me that /tmp/net.ifaces should have already been created prior to pre-pivot?

This is not entirely unexpected due to the asynchronousness of udev-events and the new initqueue/mount-loop.

The correct solution would be to move the whole call of netroot to the initqueue, which we'll have to do anyway when we start handling interface timeouts with arp.

Quoted 7 lines

Warren Togami
[email protected]

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe initramfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe initramfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html