Skip to contentSkip to conversation

namespace: fix namespace.pl script to support relative paths

PATCH In review Jacob Keller 3 participants 12 messages Jan 29, 2019 – Oct 25, 2019 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 namespace: fix namespace.pl script to support relative paths

The namespace.pl script does not work properly if objtree is not set to an absolute path. The do_nm function is run from within the find function, which changes directories.

Because of this, appending objtree, $File::Find::dir, and $source, will return a path which is not valid from the current directory.

This used to work when objtree was set to an absolute path when using "make namespacecheck". It appears to have not worked when calling ./scripts/namespace.pl directly.

This behavior was changed in 7e1c04779efd ("kbuild: Use relative path for $(objtree)", 2014-05-14)

Rather than fixing the Makefile to set objtree to an absolute path, just fix namespace.pl to work when srctree and objtree are relative. Also fix the script to use an absolute path for these by default.

Use the File::Spec module for this purpose. It's been part of perl 5 since 5.005.

The curdir() function is used to get the current directory when the objtree and srctree aren't set in the environment.

rel2abs() is used to convert possibly relative objtree and srctree environment variables to absolute paths.

Finally, the catfile() function is used instead of string appending paths together, since this is more robust when joining paths together.

---
 scripts/namespace.pl | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
scripts/namespace.pl +7 -6
index 6135574a6f39..1da7bca201a4 100755
@@ -65,13 +65,14 @@
 use warnings;
 use strict;
 use File::Find;
+use File::Spec;
 
 my $nm = ($ENV{'NM'} || "nm") . " -p";
 my $objdump = ($ENV{'OBJDUMP'} || "objdump") . " -s -j .comment";
-my $srctree = "";
-my $objtree = "";
-$srctree = "$ENV{'srctree'}/" if (exists($ENV{'srctree'}));
-$objtree = "$ENV{'objtree'}/" if (exists($ENV{'objtree'}));
+my $srctree = File::Spec->curdir();
+my $objtree = File::Spec->curdir();
+$srctree = File::Spec->rel2abs($ENV{'srctree'}) if (exists($ENV{'srctree'}));
+$objtree = File::Spec->rel2abs($ENV{'objtree'}) if (exists($ENV{'objtree'}));
 
 if ($#ARGV != -1) {
 	print STDERR "usage: $0 takes no parameters\n";
@@ -231,9 +232,9 @@ sub do_nm
 	}
 	($source = $basename) =~ s/\.o$//;
 	if (-e "$source.c" || -e "$source.S") {
-		$source = "$objtree$File::Find::dir/$source";
+		$source = File::Spec->catfile($objtree, $File::Find::dir, $source)
 	} else {
-		$source = "$srctree$File::Find::dir/$source";
+		$source = File::Spec->catfile($srctree, $File::Find::dir, $source)
 	}
 	if (! -e "$source.c" && ! -e "$source.S") {
 		# No obvious source, exclude the object if it is conglomerate
Signature · 1 line
2.18.0.219.gaf81d287a9da
Acked-by Tested-by on v1 1/1
#1 lore raw

re: https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller at intel.com/

Did anything happen with this patch?

Please send it to linux-kbuild at vger.kernel.org and
Cc: Masahiro Yamada <[email protected]>

You can also add:

I was just about to fix this script but I decided to first see if anyone else had already done so. Thanks.

Signature · 1 line
~Randy
#2 lore raw
Quoted 9 lines

-----Original Message-----
From: Randy Dunlap [mailto:rdunlap at infradead.org]
Sent: Friday, September 27, 2019 4:12 PM
To: Keller, Jacob E <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <linux- kbuild at vger.kernel.org>; Masahiro Yamada <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

Quoted 4 lines

re: https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller at intel.com/

Did anything happen with this patch?

I haven't heard anything or seen it get applied.

Quoted 3 lines

Please send it to linux-kbuild at vger.kernel.org and
Cc: Masahiro Yamada <[email protected]>

Sure, I can forward it.

Quoted 8 lines

You can also add:

I was just about to fix this script but I decided to first see if anyone else had already done so. Thanks.

Thanks for the review/notice here.

Quoted 2 lines

--
~Randy

-Jake

#3 lore raw
In reply to Randy Dunlap
Quoted 21 lines

-----Original Message-----
From: Randy Dunlap [mailto:rdunlap at infradead.org]
Sent: Friday, September 27, 2019 4:12 PM
To: Keller, Jacob E <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <linux- kbuild at vger.kernel.org>; Masahiro Yamada <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

re: https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller at intel.com/

Did anything happen with this patch?

Please send it to linux-kbuild at vger.kernel.org and
Cc: Masahiro Yamada <[email protected]>

You can also add:

Quoted 5 lines

I was just about to fix this script but I decided to first see if anyone else had already done so. Thanks.

--
~Randy

Done, thanks.

Regards,
Jake

#4 lore raw
Quoted 25 lines · Jacob E
Quoted 25 lines

-----Original Message-----
From: Randy Dunlap [mailto:rdunlap at infradead.org]
Sent: Friday, September 27, 2019 4:12 PM
To: Keller, Jacob E <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <linux- kbuild at vger.kernel.org>; Masahiro Yamada <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

re: https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller at intel.com/

Did anything happen with this patch?

Please send it to linux-kbuild at vger.kernel.org and
Cc: Masahiro Yamada <[email protected]>

You can also add:

I was just about to fix this script but I decided to first see if anyone else had already done so. Thanks.

Quoted 7 lines
Quoted 2 lines

--
~Randy

Done, thanks.

Regards,
Jake

Applied to linux/kbuild. Thanks.

Signature · 2 lines
Best Regards
Masahiro Yamada
#5 lore raw
Quoted 40 lines

-----Original Message-----
From: Masahiro Yamada [mailto:yamada.masahiro at socionext.com]
Sent: Saturday, September 28, 2019 5:21 PM
To: Keller, Jacob E <[email protected]>
Cc: Randy Dunlap <[email protected]>; intel-wired-lan at lists.osuosl.org; linux- kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

Quoted 6 lines · Jacob E
Quoted 6 lines

-----Original Message-----
From: Randy Dunlap [mailto:rdunlap at infradead.org]
Sent: Friday, September 27, 2019 4:12 PM
To: Keller, Jacob E <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild

<linux-

Quoted 5 lines
Quoted 5 lines

kbuild at vger.kernel.org>; Masahiro Yamada <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

re: https://lore.kernel.org/lkml/20190129204319.15238-1-

jacob.e.keller at intel.com/

Quoted 17 lines
Quoted 16 lines

Did anything happen with this patch?

Please send it to linux-kbuild at vger.kernel.org and
Cc: Masahiro Yamada <[email protected]>

You can also add:

I was just about to fix this script but I decided to first see if anyone else had already done so. Thanks.

--
~Randy

Quoted 8 lines
Quoted 4 lines

Done, thanks.

Regards,
Jake

Applied to linux/kbuild. Thanks.

Great, thanks!

Quoted 3 lines

--
Best Regards
Masahiro Yamada

#6 lore raw
Quoted 43 lines · Jacob E
Quoted 43 lines

-----Original Message-----
From: Masahiro Yamada [mailto:yamada.masahiro at socionext.com]
Sent: Saturday, September 28, 2019 5:21 PM
To: Keller, Jacob E <[email protected]>
Cc: Randy Dunlap <[email protected]>; intel-wired-lan at lists.osuosl.org; linux- kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

Quoted 6 lines · Jacob E
Quoted 6 lines

-----Original Message-----
From: Randy Dunlap [mailto:rdunlap at infradead.org]
Sent: Friday, September 27, 2019 4:12 PM
To: Keller, Jacob E <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild

<linux-

Quoted 5 lines
Quoted 5 lines

kbuild at vger.kernel.org>; Masahiro Yamada <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

re: https://lore.kernel.org/lkml/20190129204319.15238-1-

jacob.e.keller at intel.com/

Quoted 19 lines
Quoted 11 lines

Did anything happen with this patch?

Please send it to linux-kbuild at vger.kernel.org and
Cc: Masahiro Yamada <[email protected]>

You can also add:

Quoted 5 lines

I was just about to fix this script but I decided to first see if anyone else had already done so. Thanks.

--
~Randy

Done, thanks.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Regards,
Jake

Applied to linux/kbuild. Thanks.

Great, thanks!

This scripts has been 5-year broken,
and I did not see any complaint except from you.
So, I wonder how many people are using this.

Nor, do I understand how to use it.

Could you teach me a bit more about this script?

Something might be missing in my mind, but
I do not know how to use this script in a useful way.

It provides three checks.

[1] list_multiply_defined()

This warns multiple definition of functions.

The compiler would fail if it saw any multiple definition, so the reports from this check are all false-positive.

[2] resolve_external_references()

This warns unresolved symbols.

The compiler would fail if it saw any unresolved symbol, so the reports from this check are all false-positive, too.

[3] list_extra_externals

This warns symbols with no reference.

This potentially contains lots of false-positives.
For example, the core framework provides APIs, but if all drivers are disabled, there is no user of those APIs.

I built the kernel with x86_64_defconfig,
and namespacecheck provides

1400 line reports for [1].
200 line reports for [2].
6800 line reports for [3].

Most of these seem false-positives.

How can I use it for the code improvement?

[3] might be still useful to find 'static' candidates, but it would be difficult given the amount of the report.

Signature · 2 lines
Best Regards
Masahiro Yamada
#7 lore raw
Quoted 35 lines

-----Original Message-----
From: Masahiro Yamada <[email protected]>
Sent: Tuesday, October 22, 2019 10:22 PM
To: Keller, Jacob E <[email protected]>; Randy Dunlap <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

This scripts has been 5-year broken,
and I did not see any complaint except from you.
So, I wonder how many people are using this.

Nor, do I understand how to use it.

Could you teach me a bit more about this script?

Something might be missing in my mind, but
I do not know how to use this script in a useful way.

It provides three checks.

[1] list_multiply_defined()

This warns multiple definition of functions.

The compiler would fail if it saw any multiple definition, so the reports from this check are all false-positive.

Quoted 8 lines

[2] resolve_external_references()

This warns unresolved symbols.

The compiler would fail if it saw any unresolved symbol, so the reports from this check are all false-positive, too.

The compiler won't necessarily fail when building modules, because the symbol might be in another loadable module.

Quoted 10 lines

[3] list_extra_externals

This warns symbols with no reference.

This potentially contains lots of false-positives.
For example, the core framework provides APIs, but if all drivers are disabled, there is no user of those APIs.

We use this to help verify that driver modules do not expose symbols.

Thanks,
Jake

#8 lore raw
Quoted 38 lines · Jacob E
Quoted 38 lines

-----Original Message-----
From: Masahiro Yamada <[email protected]>
Sent: Tuesday, October 22, 2019 10:22 PM
To: Keller, Jacob E <[email protected]>; Randy Dunlap <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

This scripts has been 5-year broken,
and I did not see any complaint except from you.
So, I wonder how many people are using this.

Nor, do I understand how to use it.

Could you teach me a bit more about this script?

Something might be missing in my mind, but
I do not know how to use this script in a useful way.

It provides three checks.

[1] list_multiply_defined()

This warns multiple definition of functions.

The compiler would fail if it saw any multiple definition, so the reports from this check are all false-positive.

[2] resolve_external_references()

Quoted 8 lines
Quoted 6 lines

This warns unresolved symbols.

The compiler would fail if it saw any unresolved symbol, so the reports from this check are all false-positive, too.

The compiler won't necessarily fail when building modules, because the symbol might be in another loadable module.

Right, but this is already checked by modpost, isn't it?

Quoted 4 lines
Quoted 4 lines

[3] list_extra_externals

Quoted 8 lines
Quoted 6 lines

This warns symbols with no reference.

This potentially contains lots of false-positives.
For example, the core framework provides APIs, but if all drivers are disabled, there is no user of those APIs.

We use this to help verify that driver modules do not expose symbols.

Ah, the output is quite large, so
you search for only modules in your interest. Right?

If you want to detect missing 'static',
have you tried 'sparse'?

Quoted 2 lines

Thanks,
Jake

Signature · 2 lines
Best Regards
Masahiro Yamada
#9 lore raw
Quoted 76 lines

-----Original Message-----
From: Masahiro Yamada <[email protected]>
Sent: Wednesday, October 23, 2019 6:22 PM
To: Keller, Jacob E <[email protected]>
Cc: Randy Dunlap <[email protected]>; intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

Quoted 7 lines · Jacob E
Quoted 7 lines

-----Original Message-----
From: Masahiro Yamada <[email protected]>
Sent: Tuesday, October 22, 2019 10:22 PM
To: Keller, Jacob E <[email protected]>; Randy Dunlap <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-

kbuild

Quoted 31 lines
Quoted 31 lines

<[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

This scripts has been 5-year broken,
and I did not see any complaint except from you.
So, I wonder how many people are using this.

Nor, do I understand how to use it.

Could you teach me a bit more about this script?

Something might be missing in my mind, but
I do not know how to use this script in a useful way.

It provides three checks.

[1] list_multiply_defined()

This warns multiple definition of functions.

The compiler would fail if it saw any multiple definition, so the reports from this check are all false-positive.

[2] resolve_external_references()

Quoted 8 lines
Quoted 6 lines

This warns unresolved symbols.

The compiler would fail if it saw any unresolved symbol, so the reports from this check are all false-positive, too.

The compiler won't necessarily fail when building modules, because the symbol

might be in another loadable module.

Right, but this is already checked by modpost, isn't it?

Quoted 11 lines
Quoted 10 lines

[3] list_extra_externals

This warns symbols with no reference.

This potentially contains lots of false-positives.
For example, the core framework provides APIs, but if all drivers are disabled, there is no user of those APIs.

Quoted 5 lines
Quoted 1 line

We use this to help verify that driver modules do not expose symbols.

Ah, the output is quite large, so
you search for only modules in your interest. Right?

We run it on only one module at a time, yes.

Quoted 4 lines

If you want to detect missing 'static',
have you tried 'sparse'?

We've used that as well.

To be fair, I agree that it covers similar functionality as other tools. I haven't looked directly at namespace.pl output in a while, and the fix here is multiple years old that took a long time to get picked up.

If it's agreed that the tool has no value, and especially if it results in false indications of a problem, then maybe removing it to prevent someone from mis-reading its output makes sense?

Thanks,
Jake

Quoted 10 lines
Quoted 4 lines

Thanks,
Jake

--
Best Regards
Masahiro Yamada

#10 lore raw
Quoted 89 lines · Jacob E
Quoted 76 lines

-----Original Message-----
From: Masahiro Yamada <[email protected]>
Sent: Wednesday, October 23, 2019 6:22 PM
To: Keller, Jacob E <[email protected]>
Cc: Randy Dunlap <[email protected]>; intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

Quoted 7 lines · Jacob E
Quoted 7 lines

-----Original Message-----
From: Masahiro Yamada <[email protected]>
Sent: Tuesday, October 22, 2019 10:22 PM
To: Keller, Jacob E <[email protected]>; Randy Dunlap <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-

kbuild

Quoted 31 lines
Quoted 31 lines

<[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

This scripts has been 5-year broken,
and I did not see any complaint except from you.
So, I wonder how many people are using this.

Nor, do I understand how to use it.

Could you teach me a bit more about this script?

Something might be missing in my mind, but
I do not know how to use this script in a useful way.

It provides three checks.

[1] list_multiply_defined()

This warns multiple definition of functions.

The compiler would fail if it saw any multiple definition, so the reports from this check are all false-positive.

[2] resolve_external_references()

Quoted 8 lines
Quoted 6 lines

This warns unresolved symbols.

The compiler would fail if it saw any unresolved symbol, so the reports from this check are all false-positive, too.

The compiler won't necessarily fail when building modules, because the symbol

might be in another loadable module.

Right, but this is already checked by modpost, isn't it?

Quoted 11 lines
Quoted 10 lines

[3] list_extra_externals

This warns symbols with no reference.

This potentially contains lots of false-positives.
For example, the core framework provides APIs, but if all drivers are disabled, there is no user of those APIs.

Quoted 5 lines
Quoted 1 line

We use this to help verify that driver modules do not expose symbols.

Ah, the output is quite large, so
you search for only modules in your interest. Right?

We run it on only one module at a time, yes.

Quoted 4 lines

If you want to detect missing 'static',
have you tried 'sparse'?

Quoted 5 lines

We've used that as well.

To be fair, I agree that it covers similar functionality as other tools. I haven't looked directly at namespace.pl output in a while, and the fix here is multiple years old that took a long time to get picked up.

If it's agreed that the tool has no value, and especially if it results in false indications of a problem, then maybe removing it to prevent someone from mis-reading its output makes sense?

If there is a satisfactory alternative, I expect that namespace.pl is old, unmaintained, and unneeded, and should go away.

Signature · 1 line
~Randy
#11 lore raw
Quoted 32 lines

-----Original Message-----
From: Randy Dunlap <[email protected]>
Sent: Friday, October 25, 2019 12:30 PM
To: Keller, Jacob E <[email protected]>; Masahiro Yamada <[email protected]>
Cc: intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

Quoted 10 lines · Jacob E
Quoted 10 lines

-----Original Message-----
From: Masahiro Yamada <[email protected]>
Sent: Wednesday, October 23, 2019 6:22 PM
To: Keller, Jacob E <[email protected]>
Cc: Randy Dunlap <[email protected]>; intel-wired-lan at lists.osuosl.org; linux-kernel at vger.kernel.org; linux-kbuild <[email protected]>
Subject: Re: [PATCH] namespace: fix namespace.pl script to support relative paths

Quoted 7 lines
Quoted 3 lines

If you want to detect missing 'static',
have you tried 'sparse'?

We've used that as well.

To be fair, I agree that it covers similar functionality as other tools. I haven't

looked directly at namespace.pl output in a while, and the fix here is multiple years old that took a long time to get picked up.

Quoted 2 lines

If it's agreed that the tool has no value, and especially if it results in false

Quoted 8 lines

indications of a problem, then maybe removing it to prevent someone from mis- reading its output makes sense?

If there is a satisfactory alternative, I expect that namespace.pl is old, unmaintained, and unneeded, and should go away.

--
~Randy

Given Yamada's comments and my experience, modpost and sparse are both good at detecting the issues that namespace.pl handles.

I am not sure if there's any other output that would be valuable from namespace.pl, but if not.. I don't see a reason to keep it compared to the other tools.

Thanks,
Jake