Skip to contentSkip to conversation

Support Xen versions up to xen-4.1

PATCH In review Petr Tesařík 4 participants 7 messages Aug 24, 2012 – Sep 18, 2012

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover Support Xen versions up to xen-4.1

Hello everybody,

this is version 3 of my patch set. I'm very much indebted to Norbert Trapp for his extensive testing of my previous patch set, which revealed that Xen hypervisor static data was not kept with the previous version of the patch set.

I could also add the performance improvements, but I'm afraid it would make for an incremental patch, so let's wait until the filtering is finally right.

I have re-tested this patch set with the same result as v2:

- Xen-3.2.3 - WORKS
- Xen-4.0.1 - WORKS (only with --xen-syms)
- Xen-4.0.2 - WORKS
- Xen-4.1.2 - WORKS

Filtering free pages (-d16 and above) doesn't work (even after I manually provided the missing "max_pfn" symbol), but that seems unrelated, so I'm going to address it in a subsequent patch set.

Norbert, if you test these patches, please send your comments to the mailing list.

Atsushi-san, if the patch set passes Norbert's testing, what else do you need to accept the changes in upstream?

Regards,
Petr Tesarik
SUSE Linux

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#1 lore raw
Quoted 26 lines

-----Original Message-----
From: Petr Tesařík [mailto:[email protected]]
Sent: Friday, August 24, 2012 5:41 PM
To: [email protected]
Cc: Trapp, Norbert
Subject: [PATCHv3 0/9] Support Xen versions up to xen-4.1

Hello everybody,

this is version 3 of my patch set. I'm very much indebted to Norbert Trapp for his extensive testing of my previous patch set, which revealed that Xen hypervisor static data was not kept with the previous version of the patch set.

I could also add the performance improvements, but I'm afraid it would make for an incremental patch, so let's wait until the filtering is finally right.

I have re-tested this patch set with the same result as v2:

- Xen-3.2.3 - WORKS
- Xen-4.0.1 - WORKS (only with --xen-syms)
- Xen-4.0.2 - WORKS
- Xen-4.1.2 - WORKS

Quoted 6 lines

Filtering free pages (-d16 and above) doesn't work (even after I manually provided the missing "max_pfn" symbol), but that seems unrelated, so I'm going to address it in a subsequent patch set.

Norbert, if you test these patches, please send your comments to the mailing list.

Hello Petr et al.,

after testing Petr's patches PATCHv3 plus "support for x86_64 1GB pages", "add missing return statement" and "keep dumpfile pages in a cache" on a machine with Xen 4.0.3 and 1GB pages there is only one comment I have:
It all worked nicely!

With kind regards,

Norbert

Norbert Trapp
PBG PDG ES&S SWE OS 6

FUJITSU
Fujitsu Technology Solutions GmbH
Domagkstraße 28, D-80807 München, Germany
E-mail: [email protected]
Web: ts.fujitsu.com
Company details: ts.fujitsu.com/imprint

Quoted 7 lines

Atsushi-san, if the patch set passes Norbert's testing, what else do you need to accept the changes in upstream?

Regards,
Petr Tesarik
SUSE Linux

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#2 lore raw

Hello Norbert and Petr,

Quoted 33 lines · Norbert
Quoted 26 lines

-----Original Message-----
From: Petr Tesařík [mailto:[email protected]]
Sent: Friday, August 24, 2012 5:41 PM
To: [email protected]
Cc: Trapp, Norbert
Subject: [PATCHv3 0/9] Support Xen versions up to xen-4.1

Hello everybody,

this is version 3 of my patch set. I'm very much indebted to Norbert Trapp for his extensive testing of my previous patch set, which revealed that Xen hypervisor static data was not kept with the previous version of the patch set.

I could also add the performance improvements, but I'm afraid it would make for an incremental patch, so let's wait until the filtering is finally right.

I have re-tested this patch set with the same result as v2:

- Xen-3.2.3 - WORKS
- Xen-4.0.1 - WORKS (only with --xen-syms)
- Xen-4.0.2 - WORKS
- Xen-4.1.2 - WORKS

Quoted 6 lines

Filtering free pages (-d16 and above) doesn't work (even after I manually provided the missing "max_pfn" symbol), but that seems unrelated, so I'm going to address it in a subsequent patch set.

Norbert, if you test these patches, please send your comments to the mailing list.

Quoted 6 lines

Hello Petr et al.,

after testing Petr's patches PATCHv3 plus "support for x86_64 1GB pages", "add missing return statement" and "keep dumpfile pages in a cache" on a machine with Xen 4.0.3 and 1GB pages there is only one comment I have:
It all worked nicely!

Thank you for your report, that's good.
So, I'll review these patches for makedumpfile-1.5.1.

Thanks
Atsushi Kumagai

Quoted 23 lines

With kind regards,

Norbert

Norbert Trapp
PBG PDG ES&S SWE OS 6

FUJITSU
Fujitsu Technology Solutions GmbH
Domagkstraße 28, D-80807 München, Germany
E-mail: [email protected]
Web: ts.fujitsu.com
Company details: ts.fujitsu.com/imprint

Quoted 7 lines

Atsushi-san, if the patch set passes Norbert's testing, what else do you need to accept the changes in upstream?

Regards,
Petr Tesarik
SUSE Linux

Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#3 lore raw

Hello Atsushi, Petr, Stefan et al.,

When applying the makedumpfile Xen4 patches to the makedumpfile 1.5.0 version and using it for a crash file to save just the dom0 and xen pages it again worked splendid in that makedumpfile was much faster than without the Xen4 patches. Alas I had to remove the OFFSET_IN_UNION_INIT(page._domain, "page_info", "domain") call because OFFSET_IN_UNION_INIT was no longer present. It didn't matter much because the call is only needed for makedumpfile -g or when using the /boot/xen-syms file. In the current 1.5.0 makedumpfile.c version you see

    OFFSET_INIT(page_info._domain, "page_info", "u").

The Xen4 patch modified this because the _domain structure member is in different unions for different version (x86, ia64) and may also not be the first entry in the union depending on the Xen version.

    OFFSET_IN_UNION_INIT(page_info._domain, "page_info", "_domain");

As an experiment I tried to use

OFFSET_INIT(page_info._domain, "page_info", "domain")

with makedumpfile 1.5.0. To get the correct offset I had to modify the dwarf_info.c code:

dwarf_info.c 2012-09-12 15:49:23.000000000 +0200 renamed from makedumpfile-1.5.0/dwarf_info.c 2012-09-06 07:30:23.000000000 +0200 +3 -3
@@ -449,8 +449,8 @@
 static int
 is_anonymous_container(Dwarf_Die *die)
 {
-       if (dwarf_diename(die))
-               return FALSE;
+       //if (dwarf_diename(die))
+       //      return FALSE;
        if (dwarf_tag(die) == DW_TAG_structure_type)
                return TRUE;
        if (dwarf_info.cmd != DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION
@@ -495,7 +495,7 @@
                 * Descend into anonymous members and search for member
                 * there.
                 */
-               if (!name) {
+               if ((!name) || (strcmp(name, dwarf_info.member_name) != 0)) {
                        if (!get_die_type(walker, &die_type))
                                continue;
                        if (is_anonymous_container(&die_type))

This is not a patch but just a description of what I modified for a test. The underlying question is why only the anonymous unions are used and not the ones with a name.

With kind regards

Norbert

Norbert Trapp
PDG ES&S SWE OS 6

FUJITSU
Fujitsu Technology Solutions GmbH
Domagkstraße 28, D-80807 München, Germany
E-mail: Norbert.Trapp at ts.fujitsu.com
Web: ts.fujitsu.com
Company details: ts.fujitsu.com/imprint

Quoted 7 lines

-----Original Message-----
From: Atsushi Kumagai [mailto:[email protected]]
Sent: Monday, September 03, 2012 7:01 AM
To: Trapp, Norbert; [email protected]
Cc: [email protected]
Subject: Re: [PATCHv3 0/9] Support Xen versions up to xen-4.1

...

Quoted 7 lines

Thank you for your report, that's good.
So, I'll review these patches for makedumpfile-1.5.1.

Thanks
Atsushi Kumagai

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#4 lore raw
Quoted 2 lines · Norbert

Hello Atsushi, Petr, Stefan et al.,

Hi Norbert,

Quoted 15 lines

When applying the makedumpfile Xen4 patches to the makedumpfile 1.5.0 version and using it for a crash file to save just the dom0 and xen pages it again worked splendid in that makedumpfile was much faster than without the Xen4 patches. Alas I had to remove the OFFSET_IN_UNION_INIT(page._domain, "page_info", "domain") call because OFFSET_IN_UNION_INIT was no longer present. It didn't matter much because the call is only needed for makedumpfile -g or when using the /boot/xen-syms file. In the current 1.5.0 makedumpfile.c version you see

    OFFSET_INIT(page_info._domain, "page_info", "u").

The Xen4 patch modified this because the _domain structure member is in different unions for different version (x86, ia64) and may also not be the first entry in the union depending on the Xen version.

Quoted 6 lines
    OFFSET_IN_UNION_INIT(page_info._domain, "page_info", "_domain");

As an experiment I tried to use

OFFSET_INIT(page_info._domain, "page_info", "domain")

Just to make sure and since I am a bit lazy: with the Xen patch the page_info structure has a member (somewhere) named domain? (or _domain)?

Quoted 28 lines

with makedumpfile 1.5.0. To get the correct offset I had to modify the dwarf_info.c code:

dwarf_info.c 2012-09-12 15:49:23.000000000 +0200 renamed from makedumpfile-1.5.0/dwarf_info.c 2012-09-06 07:30:23.000000000 +0200 +3 -3
@@ -449,8 +449,8 @@
 static int
 is_anonymous_container(Dwarf_Die *die)
 {
-       if (dwarf_diename(die))
-               return FALSE;
+       //if (dwarf_diename(die))
+       //      return FALSE;
        if (dwarf_tag(die) == DW_TAG_structure_type)
                return TRUE;
        if (dwarf_info.cmd != DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION
@@ -495,7 +495,7 @@
                 * Descend into anonymous members and search for member
                 * there.
                 */
-               if (!name) {
+               if ((!name) || (strcmp(name, dwarf_info.member_name) != 0)) {
                        if (!get_die_type(walker, &die_type))
                                continue;
                        if (is_anonymous_container(&die_type))
Quoted 3 lines

This is not a patch but just a description of what I modified for a test. The underlying question is why only the anonymous unions are used and not the ones with a name.

I made the patch because one element in some anonymous structure/union combo was not found in newer kernels and the previous exceptions for such a thing were hard coded. But as far as the dwarf info I was looking at the named structures/unions were handled and I did not want to modify that case without a way to test.

-Stefan

Quoted 15 lines

With kind regards

Norbert

Norbert Trapp
PDG ES&S SWE OS 6

FUJITSU
Fujitsu Technology Solutions GmbH
Domagkstraße 28, D-80807 München, Germany
E-mail: Norbert.Trapp at ts.fujitsu.com
Web: ts.fujitsu.com
Company details: ts.fujitsu.com/imprint

Quoted 16 lines
Quoted 7 lines

-----Original Message-----
From: Atsushi Kumagai [mailto:[email protected]]
Sent: Monday, September 03, 2012 7:01 AM
To: Trapp, Norbert; [email protected]
Cc: [email protected]
Subject: Re: [PATCHv3 0/9] Support Xen versions up to xen-4.1

...

Quoted 7 lines

Thank you for your report, that's good.
So, I'll review these patches for makedumpfile-1.5.1.

Thanks
Atsushi Kumagai

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#5 lore raw

Hello Stefan,

Quoted 30 lines

-----Original Message-----
From: Stefan Bader [mailto:[email protected]]
Sent: Monday, September 17, 2012 2:38 PM
To: Trapp, Norbert
Cc: Atsushi Kumagai; [email protected]; [email protected]
Subject: Re: [PATCHv3 0/9] Support Xen versions up to xen-4.1

Quoted 2 lines · Norbert

Hello Atsushi, Petr, Stefan et al.,

Hi Norbert,

Quoted 17 lines

When applying the makedumpfile Xen4 patches to the makedumpfile 1.5.0 version and using it for a crash file to save just the dom0 and xen pages it again worked splendid in that makedumpfile was much faster than without the Xen4 patches. Alas I had to remove the OFFSET_IN_UNION_INIT(page._domain, "page_info", "domain") call because OFFSET_IN_UNION_INIT was no longer present. It didn't matter much because the call is only needed for makedumpfile -g or when using the /boot/xen-syms file. In the current 1.5.0 makedumpfile.c version you see

    OFFSET_INIT(page_info._domain, "page_info", "u").

The Xen4 patch modified this because the _domain structure member is in different unions for different version (x86, ia64) and may also not be the first entry in the union depending on the Xen version.

    OFFSET_IN_UNION_INIT(page_info._domain, "page_info", "_domain");
Quoted 8 lines
Quoted 4 lines

As an experiment I tried to use

OFFSET_INIT(page_info._domain, "page_info", "domain")

Just to make sure and since I am a bit lazy: with the Xen patch the page_info structure has a member (somewhere) named domain? (or _domain)?

Just to get you on the right track, the struct page_info in the Xen code is meant. Not the one in the makedumpfile code.

For example in xen-4.1.3/xen/common/kexec.c:

#ifdef __ia64__
    VMCOREINFO_OFFSET_SUB(page_info, u.inuse, _domain);
#else
    VMCOREINFO_OFFSET_SUB(page_info, v.inuse, _domain);

#endif

And the different page_info structures you find in:
xen-4.1.3/xen/include/asm-ia64/mm.h xen-4.1.3/xen/include/asm-x86/mm.h

And also the structures can be different depending on the Xen version.

Quoted 38 lines
Quoted 28 lines

with makedumpfile 1.5.0. To get the correct offset I had to modify the dwarf_info.c code:

dwarf_info.c 2012-09-12 15:49:23.000000000 +0200 renamed from makedumpfile-1.5.0/dwarf_info.c 2012-09-06 07:30:23.000000000 +0200 +3 -3
@@ -449,8 +449,8 @@
 static int
 is_anonymous_container(Dwarf_Die *die)
 {
-       if (dwarf_diename(die))
-               return FALSE;
+       //if (dwarf_diename(die))
+       //      return FALSE;
        if (dwarf_tag(die) == DW_TAG_structure_type)
                return TRUE;
        if (dwarf_info.cmd != DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION
@@ -495,7 +495,7 @@
                 * Descend into anonymous members and search for member
                 * there.
                 */
-               if (!name) {
+               if ((!name) || (strcmp(name, dwarf_info.member_name) != 0)) {
                        if (!get_die_type(walker, &die_type))
                                continue;
                        if (is_anonymous_container(&die_type))
Quoted 3 lines

This is not a patch but just a description of what I modified for a test. The underlying question is why only the anonymous unions are used and not the ones with a name.

I made the patch because one element in some anonymous structure/union combo was not found in newer kernels and the previous exceptions for such a thing were hard coded. But as far as the dwarf info I was looking at the named structures/unions were handled and I did not want to modify that case without a way to test.

Quoted 1 line

-Stefan

With kind regards

Norbert

Norbert Trapp
PBG PDG ES&S SWE OS 6

FUJITSU
Fujitsu Technology Solutions GmbH
Domagkstraße 28, D-80807 München, Germany
E-mail: [email protected]
Web: ts.fujitsu.com
Company details: ts.fujitsu.com/imprint

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#6 lore raw

Hello Norbert,

Quoted 17 lines · Norbert
Quoted 17 lines

Hi Norbert,

Quoted 15 lines

When applying the makedumpfile Xen4 patches to the makedumpfile 1.5.0 version and using it for a crash file to save just the dom0 and xen pages it again worked splendid in that makedumpfile was much faster than without the Xen4 patches. Alas I had to remove the OFFSET_IN_UNION_INIT(page._domain, "page_info", "domain") call because OFFSET_IN_UNION_INIT was no longer present. It didn't matter much because the call is only needed for makedumpfile -g or when using the /boot/xen-syms file. In the current 1.5.0 makedumpfile.c version you see

    OFFSET_INIT(page_info._domain, "page_info", "u").

The Xen4 patch modified this because the _domain structure member is in different unions for different version (x86, ia64) and may also not be the first entry in the union depending on the Xen version.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
    OFFSET_IN_UNION_INIT(page_info._domain, "page_info", "_domain");

As an experiment I tried to use

OFFSET_INIT(page_info._domain, "page_info", "domain")

Sorry for my carelessness.
However, I still think that to integrate OFFSET_IN_UNION_INIT into OFFSET_INIT is reasonable. So, please use OFFSET_INIT in v1.5.0 or later.

Quoted 13 lines
Quoted 4 lines

Just to make sure and since I am a bit lazy: with the Xen patch the page_info structure has a member (somewhere) named domain? (or _domain)?

Just to get you on the right track, the struct page_info in the Xen code is meant. Not the one in the makedumpfile code.

For example in xen-4.1.3/xen/common/kexec.c:

#ifdef __ia64__
    VMCOREINFO_OFFSET_SUB(page_info, u.inuse, _domain);
#else
    VMCOREINFO_OFFSET_SUB(page_info, v.inuse, _domain);
Quoted 45 lines

#endif

And the different page_info structures you find in:
xen-4.1.3/xen/include/asm-ia64/mm.h xen-4.1.3/xen/include/asm-x86/mm.h

And also the structures can be different depending on the Xen version.

Quoted 37 lines
Quoted 28 lines

with makedumpfile 1.5.0. To get the correct offset I had to modify the dwarf_info.c code:

dwarf_info.c 2012-09-12 15:49:23.000000000 +0200 renamed from makedumpfile-1.5.0/dwarf_info.c 2012-09-06 07:30:23.000000000 +0200 +3 -3
@@ -449,8 +449,8 @@
 static int
 is_anonymous_container(Dwarf_Die *die)
 {
-       if (dwarf_diename(die))
-               return FALSE;
+       //if (dwarf_diename(die))
+       //      return FALSE;
        if (dwarf_tag(die) == DW_TAG_structure_type)
                return TRUE;
        if (dwarf_info.cmd != DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION
@@ -495,7 +495,7 @@
                 * Descend into anonymous members and search for member
                 * there.
                 */
-               if (!name) {
+               if ((!name) || (strcmp(name, dwarf_info.member_name) != 0)) {
                        if (!get_die_type(walker, &die_type))
                                continue;
                        if (is_anonymous_container(&die_type))
Quoted 3 lines

This is not a patch but just a description of what I modified for a test. The underlying question is why only the anonymous unions are used and not the ones with a name.

I made the patch because one element in some anonymous structure/union combo was not found in newer kernels and the previous exceptions for such a thing were hard coded. But as far as the dwarf info I was looking at the named structures/unions were handled and I did not want to modify that case without a way to test.

As for getting the offset, it is necessary to descend into structures/unions even which are named.
So, I'll modify dwarf_info.c as below in the next version.

dwarf_info.c +9 -7
index fb11e49..0c18dd1 100644
@@ -447,10 +447,8 @@ get_dwarf_base_type(Dwarf_Die *die)
 }
 
 static int
-is_anonymous_container(Dwarf_Die *die)
+is_container(Dwarf_Die *die)
 {
-       if (dwarf_diename(die))
-               return FALSE;
        if (dwarf_tag(die) == DW_TAG_structure_type)
                return TRUE;
        if (dwarf_info.cmd != DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION
@@ -492,13 +490,13 @@ search_member(Dwarf_Die *die)
                        continue;
 
                /*
-                * Descend into anonymous members and search for member
+                * Descend into structures/unions and search for member
                 * there.
                 */
-               if (!name) {
+               if ((!name) || (strcmp(name, dwarf_info.member_name) != 0)) {
                        if (!get_die_type(walker, &die_type))
                                continue;
-                       if (is_anonymous_container(&die_type))
+                       if (is_container(&die_type))
                                if (search_member(&die_type)) {
                                        adjust_member_offset(walker);
                                        return TRUE;
@@ -1047,9 +1045,13 @@ get_member_offset(char *structname, char *membername, int cmd)
        dwarf_info.cmd = cmd;
        dwarf_info.struct_name = structname;
        dwarf_info.struct_size = NOT_FOUND_STRUCTURE;
-       dwarf_info.member_name = membername;
        dwarf_info.member_offset = NOT_FOUND_STRUCTURE;
 
+       if (dwarf_info.cmd == DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION)
+               dwarf_info.member_name = "";
+       else
+               dwarf_info.member_name = membername;
+
        if (!get_debug_info())
                return FAILED_DWARFINFO;
 

Thanks
Atsushi Kumagai

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec