Skip to contentSkip to conversation

kdump, vmcoreinfo: Export sme_me_mask value to vmcoreinfo

PATCH In review Lianbo Jiang 6 participants 33 messages Oct 26, 2018 – Nov 1, 2018

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kdump, vmcoreinfo: Export sme_me_mask value to vmcoreinfo

For AMD machine with SME feature, makedumpfile tools need to know whether the crash kernel was encrypted or not. So it is necessary to write the sme_me_mask to vmcoreinfo.

---
 arch/x86/kernel/machine_kexec_64.c | 2 ++
 1 file changed, 2 insertions(+)
arch/x86/kernel/machine_kexec_64.c +2 -0
index 4c8acdfdc5a7..dcfdb64d1097 100644
@@ -357,6 +357,8 @@ void arch_crash_save_vmcoreinfo(void)
 	vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
 			pgtable_l5_enabled());
 
+	VMCOREINFO_NUMBER(sme_me_mask);
+
 #ifdef CONFIG_NUMA
 	VMCOREINFO_SYMBOL(node_data);
 	VMCOREINFO_LENGTH(node_data, MAX_NUMNODES);
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 2 lines · Lianbo Jiang

For AMD machine with SME feature, makedumpfile tools need to know whether the crash kernel was encrypted or not.

Why?

Quoted 14 lines

So it is necessary
to write the sme_me_mask to vmcoreinfo.

---
arch/x86/kernel/machine_kexec_64.c | 2 ++

1 file changed, 2 insertions(+)
arch/x86/kernel/machine_kexec_64.c +0 -0
index 4c8acdfdc5a7..dcfdb64d1097 100644

arch/x86/kernel/machine_kexec_64.c void arch_crash_save_vmcoreinfo(void) · line 360

Quoted 4 lines
arch/x86/kernel/machine_kexec_64.c +1 -0
@@ -357,6 +357,8 @@ void arch_crash_save_vmcoreinfo(void)
 	vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
 			pgtable_l5_enabled());
 
+	VMCOREINFO_NUMBER(sme_me_mask);

No we're not going to expose a kernel-internal mask to userspace.

If at all needed, add functions to kexec which figure out whether we are encrypted or not and export that result as a kexec variable.

Signature · 1 line
Sent from a small device: formatting sux and brevity is inevitable. 
author
#2 lore raw
Quoted 6 lines · Boris Petkov
Quoted 2 lines · Lianbo Jiang

For AMD machine with SME feature, makedumpfile tools need to know whether the crash kernel was encrypted or not.

Why?

If SME is enabled in the first kernel, the crash kernel's page table(pgd/pud/pmd/pte) contains the memory encryption mask, so i have to remove the sme mask to obtain the true physical address when dump vmcore.

Quoted 17 lines
Quoted 17 lines

So it is necessary
to write the sme_me_mask to vmcoreinfo.

---
arch/x86/kernel/machine_kexec_64.c | 2 ++

1 file changed, 2 insertions(+)
arch/x86/kernel/machine_kexec_64.c +0 -0
index 4c8acdfdc5a7..dcfdb64d1097 100644
@@ -357,6 +357,8 @@ void arch_crash_save_vmcoreinfo(void)
 	vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
 			pgtable_l5_enabled());
 
Quoted 4 lines

arch/x86/kernel/machine_kexec_64.c void arch_crash_save_vmcoreinfo(void) · line 360

Quoted 1 line
arch/x86/kernel/machine_kexec_64.c +1 -0
+	VMCOREINFO_NUMBER(sme_me_mask);

No we're not going to expose a kernel-internal mask to userspace.

If so, can i set a variable flag for the 'sme_me_mask' and export the variable flag? For example:

void arch_crash_save_vmcoreinfo(void) {

    ....

    if (sme_active())
        sme_enabled = 1;

    VMCOREINFO_NUMBER(sme_enabled);

....
}

Quoted 3 lines

If at all needed, add functions to kexec which figure out whether we are encrypted or not and export that result as a kexec variable.

For AMD machine with the SME feature, the msr 'MSR_K8_SYSCFG' can examine whether SME is enabled in kernel, but the kexec is also userspace tool, it has no permission to access the msr.

Furthermore, i also tried to read the "/dev/cpu/cpu[number]/msr", but the value depends on BIOS's configuration. That is to say, if SME is set in BIOS, the value of msr is always 0xF40000 whatever the kernel commandline parameter is "mem_encrypt=on" or "mem_encrypt=off".

If i made a mistake, please help to point it out.

Thanks.
Lianbo

#3 lore raw
Quoted 4 lines · On Fri, 26 Oct 2018 20:32:11 +0800 lijiang <[email protected]> wrote:

[...]
For AMD machine with the SME feature, the msr 'MSR_K8_SYSCFG' can examine whether SME is enabled in kernel, but the kexec is also userspace tool, it has no permission to access the msr.

But we need the MSR value from the panic kernel environment, not while the production kernel is still running, right?

If so, then this reminds me that I have wanted for a long time to store more of the hardware state in a vmcore NOTE after a kernel crash ... control registers, MSRs and whatnot. Of course, this would be a long-term project, but I wonder what other people think about it in general.

Just my 2 cents,
Petr T

Quoted 14 lines

Furthermore, i also tried to read the "/dev/cpu/cpu[number]/msr", but the value depends on BIOS's configuration. That is to say, if SME is set in BIOS, the value of msr is always 0xF40000 whatever the kernel commandline parameter is "mem_encrypt=on" or "mem_encrypt=off".

If i made a mistake, please help to point it out.

Thanks.
Lianbo

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

maintainer
#4 lore raw
Quoted 2 lines · Petr Tesarik

But we need the MSR value from the panic kernel environment, not while the production kernel is still running, right?

Actually, we need only the encryption bit number (and it should be 0 otherwise to denote SME wasn't enabled).

I guess something like

	VMCOREINFO_NUMBER(sme_mask);

which gets written by the kexec-ed kernel.

Quoted 5 lines

If so, then this reminds me that I have wanted for a long time to store more of the hardware state in a vmcore NOTE after a kernel crash ... control registers, MSRs and whatnot. Of course, this would be a long-term project, but I wonder what other people think about it in general.

I guess that sounds like a good idea - the more relevant hw info for debugging, the better. Determining the important MSRs to save would need a bit of a pondering over though. For example, some MSRs are per-core, some per-socket, etc...

Signature · 5 lines
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
author
#5 lore raw
Quoted 7 lines · Borislav Petkov
Quoted 2 lines · Petr Tesarik

But we need the MSR value from the panic kernel environment, not while the production kernel is still running, right?

Actually, we need only the encryption bit number (and it should be 0 otherwise to denote SME wasn't enabled).

Thanks for your comment.
For this patch, it really needs only the encryption bit number.

For the AMD machine with SME feature, the OS or HV sets bit 47 of a physical address to 1 in the page table entry to indicate the page should be encrypted.

Thanks.
Lianbo

Quoted 12 lines

I guess something like

	VMCOREINFO_NUMBER(sme_mask);

which gets written by the kexec-ed kernel.

Quoted 5 lines

If so, then this reminds me that I have wanted for a long time to store more of the hardware state in a vmcore NOTE after a kernel crash ... control registers, MSRs and whatnot. Of course, this would be a long-term project, but I wonder what other people think about it in general.

Quoted 5 lines

I guess that sounds like a good idea - the more relevant hw info for debugging, the better. Determining the important MSRs to save would need a bit of a pondering over though. For example, some MSRs are per-core, some per-socket, etc...

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#6 lore raw
In reply to Borislav Petkov
Quoted 10 lines · Borislav Petkov
Quoted 2 lines · Petr Tesarik

But we need the MSR value from the panic kernel environment, not while the production kernel is still running, right?

Actually, we need only the encryption bit number (and it should be 0 otherwise to denote SME wasn't enabled).

I guess something like

	VMCOREINFO_NUMBER(sme_mask);

Yes, agree. So sme_me_mask itself or the encryption bit number, both is fine.

We need read and parse the memory content of crashed kernel which is mapped to /prov/vmcore by user space makedumpfile. Not only is it because user space makedumpfile can't access and get if it's sme enabled from system, we may use cp to copy /proc/vmcore to a file directly, then analyze it in another compupter. This often happen when there's something wrong with makedumpfile, we need debug makedumpfile with the complete copied file.

And only telling whether sme is enabled might be not enough. Since AMD CPU might extend to support 5-level, then the current fixed bit positon 47 would need be changed.

Thanks
Baoquan

maintainer
#7 lore raw
Quoted 2 lines · Baoquan He

Yes, agree. So sme_me_mask itself or the encryption bit number, both is fine.

You need the encryption bit position and it better be properly formatted and extracted into a vmcoreinfo-specific variable because we don't expose arch-specific details like sme_me_mask to the outside.

Quoted 4 lines

we may use cp to copy /proc/vmcore to a file directly, then analyze
it in another compupter. This often happen when there's something wrong with makedumpfile, we need debug makedumpfile with the complete copied file.

So for the analyze-on-another-computer scenario you absolutely must copy anything from the first kernel decrypted because you can't decrypt it on the other machine.

Which means, in a sensitive environment, you probably should copy and *encrypt* the dump again with gpg or so.

Signature · 5 lines
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
#8 lore raw
Quoted 7 lines · Borislav Petkov
Quoted 2 lines · Baoquan He

Yes, agree. So sme_me_mask itself or the encryption bit number, both is fine.

You need the encryption bit position and it better be properly formatted and extracted into a vmcoreinfo-specific variable because we don't expose arch-specific details like sme_me_mask to the outside.

Not very sure about this, we have arch_crash_save_vmcoreinfo() in arch/x86/kernel/machine_kexec_64.c to export arch-specific stuffs outside. Is there any special reason about a mask in one architecture when expose it out? In fact it's only that bit position set mask, no other information. Surely it's also fine if we export encryption bit position, then convert it to mask in makedumpfile.

Quoted 6 lines
Quoted 5 lines

we may use cp to copy /proc/vmcore to a file directly, then analyze
it in another compupter. This often happen when there's something wrong with makedumpfile, we need debug makedumpfile with the complete copied file.

Quoted 6 lines

So for the analyze-on-another-computer scenario you absolutely must copy anything from the first kernel decrypted because you can't decrypt it on the other machine.

Which means, in a sensitive environment, you probably should copy and *encrypt* the dump again with gpg or so.

Hmm, no matter it's makedumpfile or cp directly, when we read /proc/vmcore in kdump kernel, it will call __read_vmcore() or related functions in fs/proc/vmcore.c. With regarding to SME memory reading, Lianbo should have handle it in previous SME support in kdump patches. Just those page tables in crashed kernel are also memory content, they are decrypted and copied out, while with SME bit position enabled to indicate encryption, that bit position is added by kernel, now the 1st kernel is dead, we need wipe it out in makedumpfile when parsing. So this 'cp', it's still need be done in kdump kernel by 'cp' /proc/vmcore.

Thanks
Baoquan

maintainer
#9 lore raw
Quoted 4 lines · Baoquan He

Not very sure about this, we have arch_crash_save_vmcoreinfo() in arch/x86/kernel/machine_kexec_64.c to export arch-specific stuffs outside. Is there any special reason about a mask in one architecture when expose it out?

Yes, we don't export random arch-specific details to the outside which we then cannot change later. So vmcoreinfo needs to define its own.

Signature · 10 lines
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#10 lore raw
Quoted 8 lines · Borislav Petkov
Quoted 4 lines · Baoquan He

Not very sure about this, we have arch_crash_save_vmcoreinfo() in arch/x86/kernel/machine_kexec_64.c to export arch-specific stuffs outside. Is there any special reason about a mask in one architecture when expose it out?

Yes, we don't export random arch-specific details to the outside which we then cannot change later. So vmcoreinfo needs to define its own.

OK, then it's fine to get the bit number, e.g calling find_first_bit(sme_me_mask, BITS_PER_LONG), and export it to vmcoreinfo. Thanks.

maintainer
#11 lore raw
Quoted 3 lines · Baoquan He

OK, then it's fine to get the bit number, e.g calling find_first_bit(sme_me_mask, BITS_PER_LONG), and export it to vmcoreinfo. Thanks.

You can simply assign sme_me_mask for now...

Signature · 1 line
Sent from a small device: formatting sux and brevity is inevitable. 
author
#12 lore raw
Quoted 7 lines · Boris Petkov
Quoted 3 lines · Baoquan He

OK, then it's fine to get the bit number, e.g calling find_first_bit(sme_me_mask, BITS_PER_LONG), and export it to vmcoreinfo. Thanks.

You can simply assign sme_me_mask for now...

Thank you, Boris and Baoquan.

Actually, the value of 'sme_me_mask' is 0x800000000000 when SME is enabled, otherwise it is 0. That is to say, if the bit 47 is set, the bit number is also 0x800000000000 (1 << 47UL);

At present, they are both the same value.

Regards,
Lianbo

maintainer
#13 lore raw
Quoted 3 lines · lijiang

Actually, the value of 'sme_me_mask' is 0x800000000000 when SME is enabled, otherwise it is 0. That is to say, if the bit 47 is set, the bit number is also 0x800000000000 (1 << 47UL);

Yes, and you can simply copy the mask into your variable and export that. Thinking about it more, though, it might be better if you instead export a smaller value - not an u64 - and construct the mask in userspace. I.e., a u8 which should be enough for your current purposes.

I say current because if it turns out we need to export more SME-related info to userspace, exporting an u64 and then OR-ing in more information in it would allow that. u8 not so much. So doing something like:

[ misc ][ enc bit ][ other misc SME info ]
0000_0000_0000_0000_1000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000
63 59 55 51 47 43 39 35 31 27 23 19 15 11 7 3

would allow for exporting that additional info.

Especially if we want to use VMCOREINFO for more than kexec things.

Signature · 5 lines
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
author
#14 lore raw
Quoted 7 lines · Borislav Petkov
Quoted 3 lines · lijiang

Actually, the value of 'sme_me_mask' is 0x800000000000 when SME is enabled, otherwise it is 0. That is to say, if the bit 47 is set, the bit number is also 0x800000000000 (1 << 47UL);

Yes, and you can simply copy the mask into your variable and export that.

Ok, if you agree with the following patch, i will post it again.

arch/x86/kernel/machine_kexec_64.c +6 -0
index 4c8acdfdc5a7..de363796ed20 100644
@@ -29,6 +29,9 @@
 #include <asm/setup.h>
 #include <asm/set_memory.h>
 
+u64 sme_mask;
+EXPORT_SYMBOL(sme_mask);
+
 #ifdef CONFIG_KEXEC_FILE
 const struct kexec_file_ops * const kexec_file_loaders[] = {
                &kexec_bzImage64_ops,
@@ -357,6 +360,9 @@ void arch_crash_save_vmcoreinfo(void)
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());
 
+       sme_mask = sme_me_mask;
+       VMCOREINFO_NUMBER(sme_mask);
+
 #ifdef CONFIG_NUMA
        VMCOREINFO_SYMBOL(node_data);
        VMCOREINFO_LENGTH(node_data, MAX_NUMNODES);
Quoted 8 lines

Thinking about it more, though, it might be better if you instead export a smaller value - not an u64 - and construct the mask in userspace. I.e., a u8 which should be enough for your current purposes.

I say current because if it turns out we need to export more SME-related info to userspace, exporting an u64 and then OR-ing in more information in it would allow that. u8 not so much. So doing something like:

Quoted 8 lines

[ misc ][ enc bit ][ other misc SME info ]
0000_0000_0000_0000_1000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000
63 59 55 51 47 43 39 35 31 27 23 19 15 11 7 3

would allow for exporting that additional info.

Especially if we want to use VMCOREINFO for more than kexec things.

As you mentioned, if need, the definition of this variable(u64 sme_mask) can be extended conveniently in the future(The bit might be redefined).

Thanks.
Lianbo

#15 lore raw

arch/x86/kernel/machine_kexec_64.c line 33

Quoted 10 lines · lijiang
arch/x86/kernel/machine_kexec_64.c +2 -0
index 4c8acdfdc5a7..de363796ed20 100644
@@ -29,6 +29,9 @@
 #include <asm/setup.h>
 #include <asm/set_memory.h>
 
+u64 sme_mask;
+EXPORT_SYMBOL(sme_mask);

Wondering why it is global. Who else will use it?

Quoted 14 lines
arch/x86/kernel/machine_kexec_64.c +4 -0
+
 #ifdef CONFIG_KEXEC_FILE
 const struct kexec_file_ops * const kexec_file_loaders[] = {
                &kexec_bzImage64_ops,
@@ -357,6 +360,9 @@ void arch_crash_save_vmcoreinfo(void)
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());
 
+       sme_mask = sme_me_mask;
+       VMCOREINFO_NUMBER(sme_mask);
+
 #ifdef CONFIG_NUMA
        VMCOREINFO_SYMBOL(node_data);
        VMCOREINFO_LENGTH(node_data, MAX_NUMNODES);
author
#16 lore raw
Quoted 9 lines · Baoquan He
Quoted 8 lines · lijiang
arch/x86/kernel/machine_kexec_64.c +0 -0
index 4c8acdfdc5a7..de363796ed20 100644
@@ -29,6 +29,9 @@
 #include <asm/setup.h>
 #include <asm/set_memory.h>
 
Quoted 5 lines

arch/x86/kernel/machine_kexec_64.c line 33

Quoted 2 lines
arch/x86/kernel/machine_kexec_64.c +2 -0
+u64 sme_mask;
+EXPORT_SYMBOL(sme_mask);

Wondering why it is global. Who else will use it?

In general, no need to export the symbol of local variable.

Ok, i will change it to a local variable and export it.

Thanks.
Lianbo

Quoted 14 lines
Quoted 14 lines
arch/x86/kernel/machine_kexec_64.c +4 -0
+
 #ifdef CONFIG_KEXEC_FILE
 const struct kexec_file_ops * const kexec_file_loaders[] = {
                &kexec_bzImage64_ops,
@@ -357,6 +360,9 @@ void arch_crash_save_vmcoreinfo(void)
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());
 
+       sme_mask = sme_me_mask;
+       VMCOREINFO_NUMBER(sme_mask);
+
 #ifdef CONFIG_NUMA
        VMCOREINFO_SYMBOL(node_data);
        VMCOREINFO_LENGTH(node_data, MAX_NUMNODES);
maintainer
#17 lore raw
Quoted 1 line · lijiang

Ok, i will change it to a local variable and export it.

Why do you have to export it at all?!

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
author
#18 lore raw
Quoted 5 lines · Borislav Petkov
Quoted 1 line · lijiang

Ok, i will change it to a local variable and export it.

Why do you have to export it at all?!

I mean that i will write the value of local variable to the vmcoreinfo. For example:

arch/x86/kernel/machine_kexec_64.c +5 -0
index 4c8acdfdc5a7..3cc975e7ff8f 100644
@@ -352,11 +352,16 @@ void machine_kexec(struct kimage *image)
 
 void arch_crash_save_vmcoreinfo(void)
 {
+       u64 sme_mask = 0;
+
        VMCOREINFO_NUMBER(phys_base);
        VMCOREINFO_SYMBOL(init_top_pgt);
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());
 
+       sme_mask = sme_me_mask;
+       VMCOREINFO_NUMBER(sme_mask);
+
 #ifdef CONFIG_NUMA
        VMCOREINFO_SYMBOL(node_data);
        VMCOREINFO_LENGTH(node_data, MAX_NUMNODES);
Signature · 7 lines
Because the crash kernel's page table(pgd/pud/pmd/pte) contains
the memory encryption mask(bit 47), makedumpfile needs to remove
the sme mask to obtain the true physical address.


Thanks.
Lianbo
#19 lore raw

arch/x86/kernel/machine_kexec_64.c void machine_kexec(struct kimage *image) · line 352

Quoted 6 lines · lijiang
arch/x86/kernel/machine_kexec_64.c +0 -0
index 4c8acdfdc5a7..3cc975e7ff8f 100644
@@ -352,11 +352,16 @@ void machine_kexec(struct kimage *image)
 

void arch_crash_save_vmcoreinfo(void)
{
+ u64 sme_mask = sme_me_mask;
+

This may looks better.

        VMCOREINFO_NUMBER(phys_base);
        VMCOREINFO_SYMBOL(init_top_pgt);
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());

+ VMCOREINFO_NUMBER(sme_mask);
+

 #ifdef CONFIG_NUMA
        VMCOREINFO_SYMBOL(node_data);
        VMCOREINFO_LENGTH(node_data, MAX_NUMNODES);
author
#20 lore raw
Quoted 12 lines · Baoquan He

arch/x86/kernel/machine_kexec_64.c void machine_kexec(struct kimage *image) · line 352

Quoted 6 lines · lijiang
arch/x86/kernel/machine_kexec_64.c +0 -0
index 4c8acdfdc5a7..3cc975e7ff8f 100644
@@ -352,11 +352,16 @@ void machine_kexec(struct kimage *image)
 

void arch_crash_save_vmcoreinfo(void)
{

(quoted diff) +2 -0
+       u64 sme_mask = sme_me_mask;
+
Quoted 2 lines

This may looks better.

Ok, thanks for your comment.

Quoted 11 lines
        VMCOREINFO_NUMBER(phys_base);
        VMCOREINFO_SYMBOL(init_top_pgt);
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());
arch/x86/kernel/machine_kexec_64.c +2 -0
+       VMCOREINFO_NUMBER(sme_mask);
+
 #ifdef CONFIG_NUMA
        VMCOREINFO_SYMBOL(node_data);
        VMCOREINFO_LENGTH(node_data, MAX_NUMNODES);
maintainer
#21 lore raw
Quoted 6 lines · lijiang
Quoted 6 lines
        VMCOREINFO_NUMBER(phys_base);
        VMCOREINFO_SYMBOL(init_top_pgt);
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());

+ VMCOREINFO_NUMBER(sme_mask);

What I'm still missing from the whole fun is where are we documenting the contents of that vmcoreinfo thing?

If not, we need it documented.

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
author
#22 lore raw
Quoted 11 lines · Borislav Petkov
Quoted 6 lines · lijiang
Quoted 6 lines
        VMCOREINFO_NUMBER(phys_base);
        VMCOREINFO_SYMBOL(init_top_pgt);
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());

+ VMCOREINFO_NUMBER(sme_mask);

What I'm still missing from the whole fun is where are we documenting the contents of that vmcoreinfo thing?

I'm not sure whether the following document is what you need. Documentation/ABI/testing/sysfs-kernel-vmcoreinfo

Thanks.
Lianbo

Quoted 2 lines

If not, we need it documented.

#23 lore raw
Quoted 7 lines · lijiang
Quoted 6 lines · Borislav Petkov
Quoted 5 lines · lijiang
Quoted 5 lines
        VMCOREINFO_NUMBER(phys_base);
        VMCOREINFO_SYMBOL(init_top_pgt);
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line

+ VMCOREINFO_NUMBER(sme_mask);

What I'm still missing from the whole fun is where are we documenting the contents of that vmcoreinfo thing?

I'm not sure whether the following document is what you need. Documentation/ABI/testing/sysfs-kernel-vmcoreinfo

It is not the content, I think it is a good catch from Boris, it would be good to document the exported things in somewhere eg. Documentation/kdump/vmcoreinfo.txt

Quoted 5 lines

Thanks.
Lianbo

Quoted 2 lines

If not, we need it documented.

Thanks
Dave

maintainer
#24 lore raw
Quoted 3 lines · Dave Young

It is not the content, I think it is a good catch from Boris, it would be good to document the exported things in somewhere eg. Documentation/kdump/vmcoreinfo.txt

Yes.

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#25 lore raw

Hi Boris, DaveY and Lianbo,

Quoted 4 lines · Borislav Petkov
Quoted 3 lines · Dave Young

It is not the content, I think it is a good catch from Boris, it would be good to document the exported things in somewhere eg. Documentation/kdump/vmcoreinfo.txt

For the vmcoreinfo variables document, I personally think it might be not necessary. The reason is that all the old varialbles are exported with the name of themselves. We know what they are or what they represent since they are all kernel symbols or macro. Only this me_mask, it's a local variable and store the value of sme_me_mask for now, may store more info later like Petr mentioned, and also will store the memory encryption information of TME (which is intel's transparent memory encryption). We can add code comment around to tell these.

Personal opinion.

Thanks
Baoquan

author
#26 lore raw
Quoted 9 lines · Baoquan He

Hi Boris, DaveY and Lianbo,

Quoted 4 lines · Borislav Petkov
Quoted 3 lines · Dave Young

It is not the content, I think it is a good catch from Boris, it would be good to document the exported things in somewhere eg. Documentation/kdump/vmcoreinfo.txt

Quoted 9 lines

For the vmcoreinfo variables document, I personally think it might be not necessary. The reason is that all the old varialbles are exported with the name of themselves. We know what they are or what they represent since they are all kernel symbols or macro. Only this me_mask, it's a local variable and store the value of sme_me_mask for now, may store more info later like Petr mentioned, and also will store the memory encryption information of TME (which is intel's transparent memory encryption). We can add code comment around to tell these.

Thank you, everyone.

I personally agree with Baoquan's opinion. What do you think about? Boris and other reviewer?

If the vmcoreinfo document is necessary, would you like to help me to provide an outline? I can try my best to write the document.

Anyway, we should make a choice.

Regards,
Lianbo

Quoted 5 lines

Personal opinion.

Thanks
Baoquan

#27 lore raw

Add Kazu, the makedumpfile maintainer in cc.

Quoted 19 lines · lijiang
Quoted 9 lines · Baoquan He

Hi Boris, DaveY and Lianbo,

Quoted 4 lines · Borislav Petkov
Quoted 3 lines · Dave Young

It is not the content, I think it is a good catch from Boris, it would be good to document the exported things in somewhere eg. Documentation/kdump/vmcoreinfo.txt

Quoted 9 lines

For the vmcoreinfo variables document, I personally think it might be not necessary. The reason is that all the old varialbles are exported with the name of themselves. We know what they are or what they represent since they are all kernel symbols or macro. Only this me_mask, it's a local variable and store the value of sme_me_mask for now, may store more info later like Petr mentioned, and also will store the memory encryption information of TME (which is intel's transparent memory encryption). We can add code comment around to tell these.

Quoted 6 lines

Thank you, everyone.

I personally agree with Baoquan's opinion. What do you think about? Boris and other reviewer?

If the vmcoreinfo document is necessary, would you like to help me to provide an outline? I can try my best to write the document.

It is true like what Baoquan said, these information are mainly used by makedumpfile tool for creating a filtered vmcore. But these vmcoreinfo hide in a lot of different code paths:
kernel/crash_core.c, printk code, arch code etc.

It is a mist only a few kdump people know them, documenting them will help people to understand and review. It will also be clearer instead of digging into code?

The document can briefly explain what is vmcoreinfo, why we need it, and some background info. Then list the exported values with some classifying by core kernel, arch related, string or number etc. For most of them like Baoquan said no need more explanation, but add explanations for something if needed like this sme mask.

But I think this can be done separately instead of blocking this patch. Maybe Kazu knows more about it, so I would like to ask Kazu about his opinion and if he want to do it.

Quoted 11 lines

Anyway, we should make a choice.

Regards,
Lianbo

Quoted 5 lines

Personal opinion.

Thanks
Baoquan

Thanks
Dave

author
#28 lore raw
Quoted 38 lines · Dave Young

Add Kazu, the makedumpfile maintainer in cc.

Quoted 19 lines · lijiang
Quoted 9 lines · Baoquan He

Hi Boris, DaveY and Lianbo,

Quoted 4 lines · Borislav Petkov
Quoted 3 lines · Dave Young

It is not the content, I think it is a good catch from Boris, it would be good to document the exported things in somewhere eg. Documentation/kdump/vmcoreinfo.txt

Quoted 9 lines

For the vmcoreinfo variables document, I personally think it might be not necessary. The reason is that all the old varialbles are exported with the name of themselves. We know what they are or what they represent since they are all kernel symbols or macro. Only this me_mask, it's a local variable and store the value of sme_me_mask for now, may store more info later like Petr mentioned, and also will store the memory encryption information of TME (which is intel's transparent memory encryption). We can add code comment around to tell these.

Quoted 6 lines

Thank you, everyone.

I personally agree with Baoquan's opinion. What do you think about? Boris and other reviewer?

If the vmcoreinfo document is necessary, would you like to help me to provide an outline? I can try my best to write the document.

It is true like what Baoquan said, these information are mainly used by makedumpfile tool for creating a filtered vmcore. But these vmcoreinfo hide in a lot of different code paths:
kernel/crash_core.c, printk code, arch code etc.

It is a mist only a few kdump people know them, documenting them will help people to understand and review. It will also be clearer instead of digging into code?

Quoted 7 lines

The document can briefly explain what is vmcoreinfo, why we need it, and some background info. Then list the exported values with some classifying by core kernel, arch related, string or number etc. For most of them like Baoquan said no need more explanation, but add explanations for something if needed like this sme mask.

But I think this can be done separately instead of blocking this patch.

For this patch, i think that it had been reached an agreement. So i will update my patch log and post v2 if there is no objection.

arch/x86/kernel/machine_kexec_64.c +13 -0
index 4c8acdfdc5a7..ca9bdf46b8e7 100644
@@ -352,10 +352,23 @@ void machine_kexec(struct kimage *image)
 
 void arch_crash_save_vmcoreinfo(void)
 {
+       u64 sme_mask = sme_me_mask;
+
        VMCOREINFO_NUMBER(phys_base);
        VMCOREINFO_SYMBOL(init_top_pgt);
        vmcoreinfo_append_str("NUMBER(pgtable_l5_enabled)=%d\n",
                        pgtable_l5_enabled());
+       /*
+        * Currently, the local variable 'sme_mask' stores the value of
+        * sme_me_mask(bit 47), and also write the value of sme_mask to
+        * the vmcoreinfo.
+        * If need, the bit(sme_mask) might be redefined in the future.
+        * For example:
+        * [ misc          ][ enc bit  ][ other misc SME info       ]
+        * 0000_0000_0000_0000_1000_0000_0000_0000_0000_0000_..._0000
+        * 63   59   55   51   47   43   39   35   31   27   ... 3
+        */
+       VMCOREINFO_NUMBER(sme_mask);
 
 #ifdef CONFIG_NUMA
        VMCOREINFO_SYMBOL(node_data);
Quoted 3 lines

Maybe Kazu knows more about it, so I would like to ask Kazu about his opinion and if he want to do it.
Ok, thank you, Dave Young.

About the vmcoreinfo document issue, once make a decision, i will try my best to do this based on your outline.

Regards,
Lianbo

Quoted 15 lines
Quoted 11 lines

Anyway, we should make a choice.

Regards,
Lianbo

Quoted 5 lines

Personal opinion.

Thanks
Baoquan

Thanks
Dave

maintainer
#29 lore raw
In reply to Dave Young
Quoted 3 lines · Dave Young

It is a mist only a few kdump people know them, documenting them will help people to understand and review. It will also be clearer instead of digging into code?

Wholeheartedly agreed. Especially if people start using vmcoreinfo for other stuff, like live debugging:

https://lkml.kernel.org/r/[email protected]

Quoted 5 lines

The document can briefly explain what is vmcoreinfo, why we need it, and some background info. Then list the exported values with some classifying by core kernel, arch related, string or number etc. For most of them like Baoquan said no need more explanation, but add explanations for something if needed like this sme mask.

And add explanations for *all* of them!

If this becomes an API, then it better be documented.

Quoted 1 line

But I think this can be done separately instead of blocking this patch.

We have three months to the next merge window - I'm sure you guys can do both. :-)

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#30 lore raw
Quoted 9 lines · Borislav Petkov
Quoted 3 lines · Dave Young

It is a mist only a few kdump people know them, documenting them will help people to understand and review. It will also be clearer instead of digging into code?

Wholeheartedly agreed. Especially if people start using vmcoreinfo for other stuff, like live debugging:

https://lkml.kernel.org/r/[email protected]

I also agree. If it can help reviewers and other users to understand vmcoreinfo and can help itself to become more standard, it would be better to write it.

One small thing as a vmcoreinfo user (not about the documentation), I think it might be better to export each information to each variable separately, not OR-ing them into a variable, because of code simpleness of both kernel and tools, if there is no limitation in kernel.

Thanks,
Kazu

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

maintainer
#31 lore raw
In reply to lijiang
Quoted 3 lines · lijiang

If SME is enabled in the first kernel, the crash kernel's page table(pgd/pud/pmd/pte) contains the memory encryption mask, so i have to remove the sme mask to obtain the true physical address when dump vmcore.

Sorry, I have no clue what makedumpfile does exactly so you'd have to be more detailed (or wait until I look at it :)). Which kernel accesses which kernel's pagetable?

/me goes and looks at the makedumpfile's manpage...

Ok, it uses vmcoreinfo to exclude pages which would mean, it accesses the first kernel's pagetable and traverses it.

Am I close?

Signature · 5 lines
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
author
#32 lore raw
Quoted 11 lines · Borislav Petkov
Quoted 3 lines · lijiang

If SME is enabled in the first kernel, the crash kernel's page table(pgd/pud/pmd/pte) contains the memory encryption mask, so i have to remove the sme mask to obtain the true physical address when dump vmcore.

Sorry, I have no clue what makedumpfile does exactly so you'd have to be more detailed (or wait until I look at it :)). Which kernel accesses which kernel's pagetable?

/me goes and looks at the makedumpfile's manpage...

So sorry that i should provide more detail about this. Thanks a lot for spending time reading the manpage.

Quoted 5 lines

Ok, it uses vmcoreinfo to exclude pages which would mean, it accesses the first kernel's pagetable and traverses it.

Am I close?

Yes, your explanation is perfect.

Thanks.
Lianbo