Skip to contentSkip to conversation

Fuzz/x86emual and x86emul/test fixes

PATCH Superseded Wei Liu 3 participants 13 messages Feb 16, 2017 – Feb 20, 2017

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Wei Liu

author
#0 lore raw

cover Fuzz/x86emual and x86emul/test fixes

Wei Liu (3):
fuzz/x86emul: avoid race in link farm rune x86emul/test: avoid race in link farm rune
gitignore: ignore asm soft link in fuzz and x86emul test

 .gitignore                                   | 2 ++
 tools/fuzz/x86_instruction_emulator/Makefile | 8 ++++++--
 tools/tests/x86_emulator/Makefile            | 8 ++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
author
#1 lore raw

PATCH 1/3 fuzz/x86emul: avoid race in link farm rune

Several ln -sf can race with each other and cause error like:

14:43:56 00:07:06 O: ln: cannot remove 'asm': No such file or directory

Provide dedicated targets for soft-linking directories.

---
 tools/fuzz/x86_instruction_emulator/Makefile | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/fuzz/x86_instruction_emulator/Makefile +6 -2
index fede7e9afd..0cd49753cf 100644
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate
+
+asm:
 	[ -L asm ] || ln -sf $(XEN_ROOT)/xen/include/asm-x86 asm
 
+asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h: asm
+
 x86_emulate.c x86_emulate.h: %:
 	[ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*
 
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
#2 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

Quoted 17 lines · Wei Liu

Several ln -sf can race with each other and cause error like:

14:43:56 00:07:06 O: ln: cannot remove 'asm': No such file or directory

Provide dedicated targets for soft-linking directories.

---
 tools/fuzz/x86_instruction_emulator/Makefile | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/fuzz/x86_instruction_emulator/Makefile +0 -0
index fede7e9afd..0cd49753cf 100644

tools/fuzz/x86_instruction_emulator/Makefile else · line 14

Quoted 9 lines
tools/fuzz/x86_instruction_emulator/Makefile +2 -2
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate

You should be able to do this:

x86_emulate/%: x86_emulate

tools/fuzz/x86_instruction_emulator/Makefile else · line 19

Quoted 5 lines
tools/fuzz/x86_instruction_emulator/Makefile +3 -0
+
+asm:
 	[ -L asm ] || ln -sf $(XEN_ROOT)/xen/include/asm-x86 asm
 
+asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h: asm

And this:

asm/%: asm

Otherwise, Reviewed-by: Andrew Cooper <[email protected]>

tools/fuzz/x86_instruction_emulator/Makefile else · line 23

Quoted 4 lines
tools/fuzz/x86_instruction_emulator/Makefile +1 -0
+
 x86_emulate.c x86_emulate.h: %:
 	[ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*
 

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

author
#3 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

Quoted 25 lines · Andrew Cooper
Quoted 24 lines · Wei Liu

Several ln -sf can race with each other and cause error like:

14:43:56 00:07:06 O: ln: cannot remove 'asm': No such file or directory

Provide dedicated targets for soft-linking directories.

---
 tools/fuzz/x86_instruction_emulator/Makefile | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/fuzz/x86_instruction_emulator/Makefile +1 -1
index fede7e9afd..0cd49753cf 100644
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
Quoted 7 lines

tools/fuzz/x86_instruction_emulator/Makefile else · line 14

Quoted 2 lines
tools/fuzz/x86_instruction_emulator/Makefile +1 -1
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate

You should be able to do this:

x86_emulate/%: x86_emulate

Using % won't work. Jan made a similar comment during the first iteration of the fuzzer series. It turned out that we needed to list explicitly every file because of the other rules explicitly listed those files.

Quoted 10 lines

tools/fuzz/x86_instruction_emulator/Makefile else · line 19

Quoted 5 lines
tools/fuzz/x86_instruction_emulator/Makefile +3 -0
+
+asm:
 	[ -L asm ] || ln -sf $(XEN_ROOT)/xen/include/asm-x86 asm
 
+asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h: asm

And this:

asm/%: asm

This can be made work by eliminating the explicit listing of individual files because we're linking asm-x86 wholesale.

Wei.

Quoted 7 lines

Otherwise, Reviewed-by: Andrew Cooper <[email protected]>

Quoted 4 lines
tools/fuzz/x86_instruction_emulator/Makefile +1 -0
+
 x86_emulate.c x86_emulate.h: %:
 	[ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*
 

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

#4 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

Quoted 18 lines
Quoted 1 line
Quoted 1 line · On 17.02.17 at 11:12, <[email protected]> wrote:
Quoted 16 lines · Andrew Cooper

tools/fuzz/x86_instruction_emulator/Makefile else · line 14

Quoted 12 lines · Wei Liu
tools/fuzz/x86_instruction_emulator/Makefile +2 -2
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate

You should be able to do this:

Quoted 7 lines
Quoted 2 lines

x86_emulate/%: x86_emulate

Using % won't work. Jan made a similar comment during the first iteration of the fuzzer series. It turned out that we needed to list explicitly every file because of the other rules explicitly listed those files.

There was something in that area, but explicitly mentioning files elsewhere should not prevent this dependency to work as long as the files don't exist without the rule's commands having run.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

author
#5 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

Quoted 21 lines · Jan Beulich
Quoted 21 lines
Quoted 1 line
Quoted 1 line · On 17.02.17 at 11:12, <[email protected]> wrote:
Quoted 18 lines · Andrew Cooper

tools/fuzz/x86_instruction_emulator/Makefile else · line 14

Quoted 12 lines · Wei Liu
tools/fuzz/x86_instruction_emulator/Makefile +2 -2
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate

You should be able to do this:

x86_emulate/%: x86_emulate

Quoted 9 lines
Quoted 4 lines

Using % won't work. Jan made a similar comment during the first iteration of the fuzzer series. It turned out that we needed to list explicitly every file because of the other rules explicitly listed those files.

There was something in that area, but explicitly mentioning files elsewhere should not prevent this dependency to work as long as the files don't exist without the rule's commands having run.

I didn't dig out the old email so the description of the cause might be inaccurate.

Do you want me to do anything other than explicitly listing every file as we did before? I'm not sure I get your request.

Wei.

Quoted 2 lines

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

#6 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

Quoted 32 lines
Quoted 1 line
Quoted 1 line · On 17.02.17 at 12:36, <[email protected]> wrote:
Quoted 21 lines · Jan Beulich
Quoted 21 lines
Quoted 1 line
Quoted 1 line · On 17.02.17 at 11:12, <[email protected]> wrote:
Quoted 18 lines · Andrew Cooper

tools/fuzz/x86_instruction_emulator/Makefile else · line 14

Quoted 12 lines · Wei Liu
tools/fuzz/x86_instruction_emulator/Makefile +2 -2
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate

You should be able to do this:

x86_emulate/%: x86_emulate

Quoted 8 lines
Quoted 4 lines

Using % won't work. Jan made a similar comment during the first iteration of the fuzzer series. It turned out that we needed to list explicitly every file because of the other rules explicitly listed those files.

There was something in that area, but explicitly mentioning files elsewhere should not prevent this dependency to work as long as the files don't exist without the rule's commands having run.

Quoted 5 lines

I didn't dig out the old email so the description of the cause might be inaccurate.

Do you want me to do anything other than explicitly listing every file as we did before? I'm not sure I get your request.

Well, I'd prefer the shorter variant Andrew did suggest, so if that's not possible to be used here, I think it would help if the reason was recorded (in the commit message) now, to avoid later having the discussion yet another time.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

author
#7 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

Quoted 35 lines · Jan Beulich
Quoted 35 lines
Quoted 1 line
Quoted 1 line · On 17.02.17 at 12:36, <[email protected]> wrote:
Quoted 21 lines · Jan Beulich
Quoted 21 lines
Quoted 1 line
Quoted 1 line · On 17.02.17 at 11:12, <[email protected]> wrote:
Quoted 18 lines · Andrew Cooper

tools/fuzz/x86_instruction_emulator/Makefile else · line 14

Quoted 12 lines · Wei Liu
tools/fuzz/x86_instruction_emulator/Makefile +2 -2
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate

You should be able to do this:

x86_emulate/%: x86_emulate

Quoted 8 lines
Quoted 4 lines

Using % won't work. Jan made a similar comment during the first iteration of the fuzzer series. It turned out that we needed to list explicitly every file because of the other rules explicitly listed those files.

There was something in that area, but explicitly mentioning files elsewhere should not prevent this dependency to work as long as the files don't exist without the rule's commands having run.

I didn't dig out the old email so the description of the cause might be inaccurate.

Quoted 7 lines
Quoted 2 lines

Do you want me to do anything other than explicitly listing every file as we did before? I'm not sure I get your request.

Well, I'd prefer the shorter variant Andrew did suggest, so if that's not possible to be used here, I think it would help if the reason was recorded (in the commit message) now, to avoid later having the discussion yet another time.

I think I figure out how to make it work. New version of patches coming soon.

Wei.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

#8 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

In reply to Wei Liu

tools/fuzz/x86_instruction_emulator/Makefile else · line 12

Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 16.02.17 at 19:56, <[email protected]> wrote:
tools/fuzz/x86_instruction_emulator/Makefile +1 -1
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .

In addition to what Andrew has asked for, I would then also appreciate if you used $@ here and ...

tools/fuzz/x86_instruction_emulator/Makefile else · line 17

Quoted 5 lines
tools/fuzz/x86_instruction_emulator/Makefile +3 -1
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate
+
+asm:
 	[ -L asm ] || ln -sf $(XEN_ROOT)/xen/include/asm-x86 asm

... here along the lines of ...

tools/fuzz/x86_instruction_emulator/Makefile else · line 22

Quoted 4 lines
tools/fuzz/x86_instruction_emulator/Makefile +2 -0
+asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h: asm
+
 x86_emulate.c x86_emulate.h: %:
 	[ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*

the use of $* here. Looking at this last one makes me notice, btw, that the second argument to ln is missing? Is it portable to assume that it'll use . in that case?

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

author
#9 lore raw

Re: [PATCH 1/3] fuzz/x86emul: avoid race in link farm rune

Quoted 7 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 16.02.17 at 19:56, <[email protected]> wrote:
tools/fuzz/x86_instruction_emulator/Makefile +0 -0
@@ -8,12 +8,16 @@ else
 x86-instruction-emulator-fuzzer-all:
 endif
 
Quoted 7 lines

tools/fuzz/x86_instruction_emulator/Makefile else · line 12

Quoted 3 lines
tools/fuzz/x86_instruction_emulator/Makefile +1 -1
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .

In addition to what Andrew has asked for, I would then also appreciate if you used $@ here and ...

Using $@ is OK.

Quoted 8 lines

tools/fuzz/x86_instruction_emulator/Makefile else · line 17

Quoted 5 lines
tools/fuzz/x86_instruction_emulator/Makefile +3 -1
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate
+
+asm:
 	[ -L asm ] || ln -sf $(XEN_ROOT)/xen/include/asm-x86 asm

... here along the lines of ...

Quoted 9 lines

tools/fuzz/x86_instruction_emulator/Makefile else · line 22

Quoted 4 lines
tools/fuzz/x86_instruction_emulator/Makefile +2 -0
+asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h: asm
+
 x86_emulate.c x86_emulate.h: %:
 	[ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*

the use of $* here. Looking at this last one makes me notice, btw, that the second argument to ln is missing? Is it portable to assume that it'll use . in that case?

Yes. From manpage:

ln [OPTION]... TARGET (2nd form)
... in the 2nd form, create a link to TARGET in the current directory.

Wei.

Quoted 2 lines

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel

author
#10 lore raw

PATCH 2/3 x86emul/test: avoid race in link farm rune

In reply to Wei Liu

Several ln -sf can race with each other. Provide dedicated targets for soft-linking directories.

---
 tools/tests/x86_emulator/Makefile | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/tests/x86_emulator/Makefile +6 -2
index 9bf36947c0..860e3867d6 100644
@@ -40,12 +40,16 @@ distclean: clean
 .PHONY: install
 install:
 
-x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h:
+x86_emulate:
 	[ -L x86_emulate ] || ln -sf $(XEN_ROOT)/xen/arch/x86/x86_emulate .
 
-asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h:
+x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h: x86_emulate
+
+asm:
 	[ -L asm ] || ln -sf $(XEN_ROOT)/xen/include/asm-x86 asm
 
+asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h: asm
+
 HOSTCFLAGS += $(CFLAGS_xeninclude) -I.
 
 x86.h := asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel
author
#11 lore raw

PATCH 3/3 gitignore: ignore asm soft link in fuzz and x86emul test

In reply to Wei Liu
---
 .gitignore | 2 ++
 1 file changed, 2 insertions(+)
.gitignore +2 -0
index 8810c6975a..c8d56d1bdb 100644
@@ -147,6 +147,7 @@ tools/flask/utils/flask-setenforce
 tools/flask/utils/flask-set-bool
 tools/flask/utils/flask-label-pci
 tools/fuzz/libelf/afl-libelf-fuzzer
+tools/fuzz/x86_instruction_emulator/asm
 tools/fuzz/x86_instruction_emulator/x86_emulate*
 tools/fuzz/x86_instruction_emulator/afl-x86-insn-emulator-fuzzer
 tools/helpers/_paths.h
@@ -209,6 +210,7 @@ tools/python/build/*
 tools/security/secpol_tool
 tools/security/xen/*
 tools/security/xensec_tool
+tools/tests/x86_emulator/asm
 tools/tests/x86_emulator/blowfish.bin
 tools/tests/x86_emulator/blowfish.h
 tools/tests/x86_emulator/test_x86_emulator
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xen.org/xen-devel