Skip to contentSkip to conversation

oe-init-build-env slowdown: chmod -R

DISCUSSION Patrick Ohly 3 participants 3 messages Sep 4, 2015

lore
Share & tools
lore
#0 lore raw

Hello!

The following commit introduced a "chmod -R -st" on the build dir:

commit f0d3587706fffdcc1c708013f6d6ce296dfc5f24

Author: Alex Franco <[email protected]>
Date:   Fri Aug 28 17:34:04 2015 -0500

Fix mode +st on TMPDIR when creating it

A sanity check fails when TMPDIR has setuid, setgid set. It was proposed to fix this on TMPDIR creation instead of failing with a sanity test only. This patch handles removal of those special bits (and additonally, the sticky bit) from BUILDDIR and TMPDIR when these directories are created.

In my case (*), that chmod takes about 2 minutes to complete. strace shows that it does a fchmodat on all files, even if nothing needs to be changed. atop then shows that this keeps the disk 100% busy with writes. I don't know what gets written (can't be the access time, I tried mounting with noatime).

Are others seeing the same behavior? I can't rule out that my disk is just slow due to fragmentation.

The following commands work much better for me. I can submit a patch if others see the same thing.

        find . -perm /+st -print0 | xargs -0 --no-run-if-empty chmod -st

(*) Debian 8, Linux 3.16.0-4-amd64, coreutils 8.23-4, ext4 on a SATA hard disk.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#1 lore raw
Quoted 16 lines · Patrick Ohly

Hello!

The following commit introduced a "chmod -R -st" on the build dir:

commit f0d3587706fffdcc1c708013f6d6ce296dfc5f24

Author: Alex Franco <[email protected]>
Date:   Fri Aug 28 17:34:04 2015 -0500

Fix mode +st on TMPDIR when creating it

A sanity check fails when TMPDIR has setuid, setgid set. It was proposed to fix this on TMPDIR creation instead of failing with a sanity test only. This patch handles removal of those special bits (and additonally, the sticky bit) from BUILDDIR and TMPDIR when these directories are created.

Quoted 9 lines

In my case (*), that chmod takes about 2 minutes to complete. strace shows that it does a fchmodat on all files, even if nothing needs to be changed. atop then shows that this keeps the disk 100% busy with writes. I don't know what gets written (can't be the access time, I tried mounting with noatime).

Are others seeing the same behavior? I can't rule out that my disk is just slow due to fragmentation.

d8caef4f6 in master should fix this (and yes it was a problem elsewhere
too -- an autobuilder was seen spending 12 minutes on that).

Jussi

#2 lore raw
In reply to Patrick Ohly
Quoted 6 lines · Patrick Ohly

In my case (*), that chmod takes about 2 minutes to complete. strace shows that it does a fchmodat on all files, even if nothing needs to be changed. atop then shows that this keeps the disk 100% busy with writes. I don't know what gets written (can't be the access time, I tried mounting with noatime).

The autobuilder has been seen to take 14 minutes at this stage before being killed by buildbot. :) There's a patch on the list for this, and will be merged soon.

Ross