x86, kdump: crashkernel=X try to reserve below 896M first, then try below 4G, then MAXMEM
Subsystems (3)
PATCH 1/1 x86, kdump: crashkernel=X try to reserve below 896M first, then try below 4G, then MAXMEM
Now crashkernel=X will fail out if there's not enough memory at low (below 896M). What makes sense for crashkernel=X would be:
- First try to reserve X below 896M (for being compatible with old kexec-tools).
- If fails, try to reserve X below 4G (swiotlb need to stay below 4G).
- If fails, try to reserve X from MAXMEM top down.
So that user can easily reserve large memory with crashkernel=X instead of crashkernel=X,high. It's more transparent and user-friendly.
If crashkernel is large and the reserved is beyond 896M, old kexec-tools won't be compatible with new kernel for most of time.
kexec will fail out immediately in this case. But the failure could be expected, because old kexec users should not try to reserve that large amount of memory at the first place.
On the other hand, old kexec also will fail on old kernel when there's not enough low memory to reserve a large crash kernel area. So the failure of old kexec is consistent between old kernel and new kernel.
--- arch/x86/kernel/setup.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
arch/x86/kernel/setup.c +14 -0
@@ -593,6 +593,20 @@ static void __init reserve_crashkernel(void) high ?CRASH_KERNEL_ADDR_HIGH_MAX :CRASH_KERNEL_ADDR_LOW_MAX , crash_size, alignment); +/* +* crashkernel=X reserve below 896M fails? Try below 4G +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, + (1ULL <<32 ), + crash_size, alignment); +/* +* crashkernel=X reserve below 4G fails? Try MAXMEM +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, +CRASH_KERNEL_ADDR_HIGH_MAX , + crash_size, alignment);if (!crash_base) {pr_info ("crashkernel reservation failed - No suitable area found.\n" );
Signature · 7 lines
1.8.3.1 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 43 lines · WANG Chao
Now crashkernel=X will fail out if there's not enough memory at low (below 896M). What makes sense for crashkernel=X would be:
- First try to reserve X below 896M (for being compatible with old kexec-tools).
- If fails, try to reserve X below 4G (swiotlb need to stay below 4G).
- If fails, try to reserve X from MAXMEM top down.So that user can easily reserve large memory with crashkernel=X instead of crashkernel=X,high. It's more transparent and user-friendly.
If crashkernel is large and the reserved is beyond 896M, old kexec-tools won't be compatible with new kernel for most of time.
kexec will fail out immediately in this case. But the failure could be expected, because old kexec users should not try to reserve that large amount of memory at the first place.
On the other hand, old kexec also will fail on old kernel when there's not enough low memory to reserve a large crash kernel area. So the failure of old kexec is consistent between old kernel and new kernel.
Signed-off-by WANG Chao <[email protected]>--- arch/x86/kernel/setup.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)arch/x86/kernel/setup.c +8
-0@@ -593,6 +593,20 @@ static void __init reserve_crashkernel(void) high ?CRASH_KERNEL_ADDR_HIGH_MAX :CRASH_KERNEL_ADDR_LOW_MAX , crash_size, alignment); +/* +* crashkernel=X reserve below 896M fails? Try below 4G +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, + (1ULL <<32 ), + crash_size, alignment); +/*
arch/x86/kernel/setup.c static void __init reserve_crashkernel(void)
Quoted 10 lines
arch/x86/kernel/setup.c +6
-1+* crashkernel=X reserve below 4G fails? Try MAXMEM +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, +CRASH_KERNEL_ADDR_HIGH_MAX , + crash_size, alignment);if (!crash_base) {pr_info ("crashkernel reservation failed - No suitable area found.\n" ); --
User should change crashkernel=X to crashkernel=X,high.
As user could forget to update kexec-tools to utilize ",high" feature, and get kdump later fail later.
Yinghai
Quoted 51 lines · Yinghai Lu
Quoted 50 lines · WANG Chao
Now crashkernel=X will fail out if there's not enough memory at low (below 896M). What makes sense for crashkernel=X would be:
- First try to reserve X below 896M (for being compatible with old kexec-tools).
- If fails, try to reserve X below 4G (swiotlb need to stay below 4G).
- If fails, try to reserve X from MAXMEM top down.So that user can easily reserve large memory with crashkernel=X instead of crashkernel=X,high. It's more transparent and user-friendly.
If crashkernel is large and the reserved is beyond 896M, old kexec-tools won't be compatible with new kernel for most of time.
kexec will fail out immediately in this case. But the failure could be expected, because old kexec users should not try to reserve that large amount of memory at the first place.
On the other hand, old kexec also will fail on old kernel when there's not enough low memory to reserve a large crash kernel area. So the failure of old kexec is consistent between old kernel and new kernel.
Signed-off-by WANG Chao <[email protected]>--- arch/x86/kernel/setup.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)arch/x86/kernel/setup.c +14
-0@@ -593,6 +593,20 @@ static void __init reserve_crashkernel(void) high ?CRASH_KERNEL_ADDR_HIGH_MAX :CRASH_KERNEL_ADDR_LOW_MAX , crash_size, alignment); +/* +* crashkernel=X reserve below 896M fails? Try below 4G +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, + (1ULL <<32 ), + crash_size, alignment); +/* +* crashkernel=X reserve below 4G fails? Try MAXMEM +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, +CRASH_KERNEL_ADDR_HIGH_MAX , + crash_size, alignment);
Quoted 5 lines
arch/x86/kernel/setup.c static void __init reserve_crashkernel(void)
Quoted 3 lines
arch/x86/kernel/setup.c +0
-1if (!crash_base) {pr_info ("crashkernel reservation failed - No suitable area found.\n" ); --User should change crashkernel=X to crashkernel=X,high.
crashkernel=X is more straightforward. IMHO, It should be a more general crash kernel reservation parameter and it shouldn't reserve at low only.
Quoted 3 lines
As user could forget to update kexec-tools to utilize ",high" feature, and get kdump later fail later.
old kexec-tools should fail to load kernel high anyway, right?
Thanks
WANG Chao
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 51 lines · Yinghai Lu
Quoted 50 lines · WANG Chao
Now crashkernel=X will fail out if there's not enough memory at low (below 896M). What makes sense for crashkernel=X would be:
- First try to reserve X below 896M (for being compatible with old kexec-tools).
- If fails, try to reserve X below 4G (swiotlb need to stay below 4G).
- If fails, try to reserve X from MAXMEM top down.So that user can easily reserve large memory with crashkernel=X instead of crashkernel=X,high. It's more transparent and user-friendly.
If crashkernel is large and the reserved is beyond 896M, old kexec-tools won't be compatible with new kernel for most of time.
kexec will fail out immediately in this case. But the failure could be expected, because old kexec users should not try to reserve that large amount of memory at the first place.
On the other hand, old kexec also will fail on old kernel when there's not enough low memory to reserve a large crash kernel area. So the failure of old kexec is consistent between old kernel and new kernel.
Signed-off-by WANG Chao <[email protected]>--- arch/x86/kernel/setup.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)arch/x86/kernel/setup.c +14
-0@@ -593,6 +593,20 @@ static void __init reserve_crashkernel(void) high ?CRASH_KERNEL_ADDR_HIGH_MAX :CRASH_KERNEL_ADDR_LOW_MAX , crash_size, alignment); +/* +* crashkernel=X reserve below 896M fails? Try below 4G +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, + (1ULL <<32 ), + crash_size, alignment); +/* +* crashkernel=X reserve below 4G fails? Try MAXMEM +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, +CRASH_KERNEL_ADDR_HIGH_MAX , + crash_size, alignment);
Quoted 5 lines
arch/x86/kernel/setup.c static void __init reserve_crashkernel(void)
Quoted 3 lines
arch/x86/kernel/setup.c +0
-1if (!crash_base) {pr_info ("crashkernel reservation failed - No suitable area found.\n" ); --User should change crashkernel=X to crashkernel=X,high.
I think if we can extend old syntax of crashkernel=X, then it makes life really easy for users.
Quoted 3 lines
As user could forget to update kexec-tools to utilize ",high" feature, and get kdump later fail later.
Previously high reservation (reservation above 896M) will anyway fail. So instead of failing, if we try reservation in higher memory areas why that would break old kexec-tools?
IOW, previously anyway kexec-tools will not work as no memory will be reserved in higher memory area. Now memory will be reserved but old kexec-tools should fail as it can't load in that area.
If that works, then one would use crashkernel=X,high only if he is particualr that memory reservation comes from area above 4G (despite the fact that same memory could have been reserved below 4G too).
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 8 lines · Vivek Goyal
Quoted 2 lines · Yinghai Lu
On Mon, Oct 14, 2013 at 4:46 AM, WANG Chao <[email protected]> wrote:
User should change crashkernel=X to crashkernel=X,high.I think if we can extend old syntax of crashkernel=X, then it makes life really easy for users.
Quoted 1 line
Quoted 6 lines
Quoted 2 lines
As user could forget to update kexec-tools to utilize ",high" feature, and get kdump later fail later.
Previously high reservation (reservation above 896M) will anyway fail. So instead of failing, if we try reservation in higher memory areas why that would break old kexec-tools?
If thel old kexec-tools would fail, we should let them fail early as possible, do not reserve at first point.
Quoted 8 lines
IOW, previously anyway kexec-tools will not work as no memory will be reserved in higher memory area. Now memory will be reserved but old kexec-tools should fail as it can't load in that area.
If that works, then one would use crashkernel=X,high only if he is particualr that memory reservation comes from area above 4G (despite the fact that same memory could have been reserved below 4G too).
My point:
Push user to use ",high" as more as possible, so we only to handle one path eventually.
for old kernel, leave them to use old grammer. do not need to change it.
Also boot loader should always have different entry for old kernel and new kernel.
Thanks
Yinghai
On Thu, Oct 17, 2013 at 08:50:07PM -0700, Yinghai Lu wrote:
[..]
Quoted 6 lines
Quoted 3 lines
Previously high reservation (reservation above 896M) will anyway fail. So instead of failing, if we try reservation in higher memory areas why that would break old kexec-tools?
If thel old kexec-tools would fail, we should let them fail early as possible, do not reserve at first point.
A user does not care if they get the message "memory nor reserved" or if they get the message that "could not find a suitable memory hole at address X".
Quoted 6 lines
Quoted 6 lines
IOW, previously anyway kexec-tools will not work as no memory will be reserved in higher memory area. Now memory will be reserved but old kexec-tools should fail as it can't load in that area.
Quoted 8 lines
Quoted 3 lines
If that works, then one would use crashkernel=X,high only if he is particualr that memory reservation comes from area above 4G (despite the fact that same memory could have been reserved below 4G too).
My point:
Push user to use ",high" as more as possible, so we only to handle one path eventually.
for old kernel, leave them to use old grammer. do not need to change it.
I don't understand this. Why we should push users to use ",high" syntax. That is an option. Those who want to use it, should use it.
We have been using crashkernel=XM for a long time now. And it makes sense to extend this option to be able to reserve memory at higher addresses if asked memory is not available at lower addresses.
You are not thinking about ease of use here for existing users.
Quoted 3 lines
Also boot loader should always have different entry for old kernel and new kernel.
What does memory reservation location has to do with kernel entry point? If it is a 64bit bzImage, we will use 64bit entry point by default, isn't it? Does not matter whether memory is reserved above 4G or not.
I think it makes sense that existing crashkernel=XM usres be able to reserve memory in higher memory area if sufficient memory is not available below 896M or below 4G. Those who always want memory reservation above 4G they should use ",high" syntax and enforce memory allocation above 4G.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 7 lines · Vivek Goyal
On Thu, Oct 17, 2013 at 08:50:07PM -0700, Yinghai Lu wrote:
[..]
Quoted 4 lines
Quoted 3 lines
Previously high reservation (reservation above 896M) will anyway fail. So instead of failing, if we try reservation in higher memory areas why that would break old kexec-tools?
Quoted 6 lines
Quoted 2 lines
If thel old kexec-tools would fail, we should let them fail early as possible, do not reserve at first point.
A user does not care if they get the message "memory nor reserved" or if they get the message that "could not find a suitable memory hole at address X".
you are asking for trouble.
Now we have two paths:
1. old kernel with old kexec tools. crashkernel=XM, work,
the new kernel with old kexec tools still working with crashkernel=XM
2. old kernel with old kernel tools, crashkernel=XM, not working. as X is too big.
then user update to new kernel AND new kexec-tools, and crashkernel=XM,high
with this patch, you will need to test new kernel all old kexec tools to make sure
it will fail later instead of fail early to remind them to update kexec tools. Also would make user to guess and try to make new kernel to work with old kexec-tools
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
IOW, previously anyway kexec-tools will not work as no memory will be reserved in higher memory area. Now memory will be reserved but old kexec-tools should fail as it can't load in that area.
Quoted 8 lines
Quoted 3 lines
If that works, then one would use crashkernel=X,high only if he is particualr that memory reservation comes from area above 4G (despite the fact that same memory could have been reserved below 4G too).
My point:
Push user to use ",high" as more as possible, so we only to handle one path eventually.
for old kernel, leave them to use old grammer. do not need to change it.
Quoted 8 lines
I don't understand this. Why we should push users to use ",high" syntax. That is an option. Those who want to use it, should use it.
We have been using crashkernel=XM for a long time now. And it makes sense to extend this option to be able to reserve memory at higher addresses if asked memory is not available at lower addresses.
You are not thinking about ease of use here for existing users.
most existing user don't need to do anything. just with new kernel and old kexec tools.
those system that did not work kexec before because XM is too big, they have to update kexec tools, and use ",high"
Make it simple, less error.
Quoted 5 lines
Quoted 4 lines
Also boot loader should always have different entry for old kernel and new kernel.
Quoted 8 lines
What does memory reservation location has to do with kernel entry point? If it is a 64bit bzImage, we will use 64bit entry point by default, isn't it? Does not matter whether memory is reserved above 4G or not.
I think it makes sense that existing crashkernel=XM usres be able to reserve memory in higher memory area if sufficient memory is not available below 896M or below 4G. Those who always want memory reservation above 4G they should use ",high" syntax and enforce memory allocation above 4G.
We already support above 4G, what is point for trying below 4G?
You could get more bug report about new kernel with old kexec-tools, as old kexec-tools could work with range between 896M and 4G in some case, but not all.
Thanks
Yinghai
On Fri, Oct 18, 2013 at 10:45:43PM -0700, Yinghai Lu wrote:
[..]
Quoted 13 lines
Quoted 3 lines
A user does not care if they get the message "memory nor reserved" or if they get the message that "could not find a suitable memory hole at address X".
you are asking for trouble.
Now we have two paths:
1. old kernel with old kexec tools. crashkernel=XM, work,
the new kernel with old kexec tools still working with crashkernel=XM
2. old kernel with old kernel tools, crashkernel=XM, not working. as X is too big.
then user update to new kernel AND new kexec-tools, and crashkernel=XM,high
Quoted 5 lines
with this patch, you will need to test new kernel all old kexec tools to make sure
it will fail later instead of fail early to remind them to update kexec tools. Also would make user to guess and try to make new kernel to work with old kexec-tools
Seriously, I don't understand what are you trying to say above. You will need to explain your concern more clearly.
IIUC, you are trying to say that with new kernel old kexec-tools will fail at a different failure point. I don't see why that is a problem. It still fails.
[..]
Quoted 9 lines
Quoted 1 line
You are not thinking about ease of use here for existing users.
most existing user don't need to do anything. just with new kernel and old kexec tools.
those system that did not work kexec before because XM is too big, they have to update kexec tools, and use ",high"
Make it simple, less error.
No, it is not that simple. Think from a distribution's perspective also. We have the logic to scale reserved memory based on physical memory present in the system. Now we are seeing bigger memory systems (which would not have worked in the past). We still want to retain the existing logic and not switch to crashkernel=x,high. One does not have to. It makes life simpler.
Same logic working both with smaller memory systems as well as large memory systems. One should not have to choose a different command line because there is more physical RAM present in the system.
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Also boot loader should always have different entry for old kernel and new kernel.
What does memory reservation location has to do with kernel entry point? If it is a 64bit bzImage, we will use 64bit entry point by default, isn't it? Does not matter whether memory is reserved above 4G or not.
Quoted 6 lines
Quoted 4 lines
I think it makes sense that existing crashkernel=XM usres be able to reserve memory in higher memory area if sufficient memory is not available below 896M or below 4G. Those who always want memory reservation above 4G they should use ",high" syntax and enforce memory allocation above 4G.
We already support above 4G, what is point for trying below 4G?
Because it is not *required* to reserve memory above 4G. Because we want same command line to work with both small memory systems as well as large memory systems and we don't care whether memory is reserved below 4G or above 4G. What does matter though that we don't have to worry about switching command line option if it is large memory system.
Quoted 4 lines
You could get more bug report about new kernel with old kexec-tools, as old kexec-tools could work with range between 896M and 4G in some case, but not all.
So you are worried that if we lift the artificial cap, and old kexec-tools start working that will be a problem. I don't think that is an issue. People worry about something existing broken. It does not break existing kexec-tools. If in some cases they work with memory reserved at higher addresses, where is the problem. Even in the past one could do crashkernel=X@Y and if Y is above 896M and if kexec-tools works with that what's wrong. I have not seen people complaining about it.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 6 lines · Vivek Goyal
On Fri, Oct 18, 2013 at 10:45:43PM -0700, Yinghai Lu wrote:
IIUC, you are trying to say that with new kernel old kexec-tools will fail at a different failure point. I don't see why that is a problem. It still fails.
Yes, that could cause confusion.
We already knew it would fail possible at most later, we should make it skip allocation during first kernel booting.
Quoted 10 lines
[..]
Quoted 8 lines
Quoted 1 line
You are not thinking about ease of use here for existing users.
most existing user don't need to do anything. just with new kernel and old kexec tools.
those system that did not work kexec before because XM is too big, they have to update kexec tools, and use ",high"
Quoted 8 lines
Quoted 1 line
Make it simple, less error.
No, it is not that simple. Think from a distribution's perspective also. We have the logic to scale reserved memory based on physical memory present in the system. Now we are seeing bigger memory systems (which would not have worked in the past). We still want to retain the existing logic and not switch to crashkernel=x,high. One does not have to. It makes life simpler.
distribution should go with ",high" for 64 bit kernel and new kexec-tools. for 32bit kernel, you still can have ",high" or not, as ",high" is ignored.
Quoted 4 lines
Same logic working both with smaller memory systems as well as large memory systems. One should not have to choose a different command line because there is more physical RAM present in the system.
",high" is working even on smaller memory sytem.
Quoted 9 lines
Quoted 3 lines
We already support above 4G, what is point for trying below 4G?
Because it is not *required* to reserve memory above 4G. Because we want same command line to work with both small memory systems as well as large memory systems and we don't care whether memory is reserved below 4G or above 4G. What does matter though that we don't have to worry about switching command line option if it is large memory system.
",high" will work smaller or large memory system after you install new kexec tools.
Again, for distribution, when new kernel is added, new kernel will all have ",high"
and new kexec-tools get installed.
Even we want to extend crashkernel=XM, then i would like to have it identical to crashkernel=XM,high instead.
Thanks
Yinghai
Quoted 4 lines · Yinghai Lu
Quoted 3 lines · Vivek Goyal
On Fri, Oct 18, 2013 at 10:45:43PM -0700, Yinghai Lu wrote:
Quoted 8 lines
Quoted 3 lines
IIUC, you are trying to say that with new kernel old kexec-tools will fail at a different failure point. I don't see why that is a problem. It still fails.
Yes, that could cause confusion.
We already knew it would fail possible at most later, we should make it skip allocation during first kernel booting.
It's said:
crashkernel=size[KMG][@offset[KMG]]
[KNL] Using kexec, Linux can switch to a 'crash kernel' upon panic. This parameter reserves the physical memory region [offset, offset + size] for that kernel image. If '@offset' is omitted, then a suitable offset is selected automatically. Check Documentation/kdump/kdump.txt for further details.
Nothing mentions that crashkernel=X is restricted to reserve at low.
It could be more confusing that crashkernel=X (say 1G) can not find a suitable area while actually there's enough memory out there.
Current situation is the combination of crashkernel X/high/low is quite complicated, it's more like supposed to be used by the pros.
Truth is, most user, after upgrading to new kernel, would want reserve large by just changing to crashkernel=1G, not be forced to use a new format craskernel=1G,high.
I think crashkernel=XM,high is really supposed to be used when user indeed want to reserve from high.
Like Vivek said, failing at different point shouldn't be a problem. That's an incorrect configuration. When crashkernel=1G,high, old kexec-tools still fails the same way. That could cause confusion, in your word.
Let me put it in an example, a user want to utilize this new kernel feature to reserve 1G for crash kernel but not upgrade kexec-tools,
- W/o this patch:
First he would try crashkernel=1G, but failed to reserve. Second time, he goes with crashkernel=1G,high, reservation is fine but kexec fails to load. Upgrading kexec-tools is essential to him.
- W/ this patch:
First he would try crashkernel=1G, reservation is ok but kexec fails to load the same way as the case of crashkernel=1G,high. Upgrading kexec-tools is essential to him.
The point is old kexec-tools can't load high, no matter by what kind of crashkernel cmdline to reserve at high.
Quoted 20 lines
Quoted 11 lines
[..]
Quoted 8 lines
Quoted 1 line
You are not thinking about ease of use here for existing users.
most existing user don't need to do anything. just with new kernel and old kexec tools.
those system that did not work kexec before because XM is too big, they have to update kexec tools, and use ",high"
Quoted 8 lines
Quoted 1 line
Make it simple, less error.
No, it is not that simple. Think from a distribution's perspective also. We have the logic to scale reserved memory based on physical memory present in the system. Now we are seeing bigger memory systems (which would not have worked in the past). We still want to retain the existing logic and not switch to crashkernel=x,high. One does not have to. It makes life simpler.
Quoted 2 lines
distribution should go with ",high" for 64 bit kernel and new kexec-tools. for 32bit kernel, you still can have ",high" or not, as ",high" is ignored.
Why ",high" is becoming a default one instead of "XM"?
Isn't that the kernel's job to find a most suitable area at best guess in this case? By this case, I mean kernel have no idea kexec-tools is new or old.
Quoted 8 lines
Quoted 6 lines
Same logic working both with smaller memory systems as well as large memory systems. One should not have to choose a different command line because there is more physical RAM present in the system.
",high" is working even on smaller memory sytem.
Yeah, you're right.
Quoted 14 lines
Quoted 10 lines
Quoted 4 lines
We already support above 4G, what is point for trying below 4G?
Because it is not *required* to reserve memory above 4G. Because we want same command line to work with both small memory systems as well as large memory systems and we don't care whether memory is reserved below 4G or above 4G. What does matter though that we don't have to worry about switching command line option if it is large memory system.
",high" will work smaller or large memory system after you install new kexec tools.
Quoted 6 lines
Again, for distribution, when new kernel is added, new kernel will all have ",high"
and new kexec-tools get installed.Even we want to extend crashkernel=XM, then i would like to have it identical to crashkernel=XM,high instead.
Why would you like crashkernel=XM is identical to crashkernel=XM,high?
That's not an option if you want new kernel to be compatible with old kexec-tools.
Thanks
WANG Chao
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 3 lines · WANG Chao
I think crashkernel=XM,high is really supposed to be used when user indeed want to reserve from high.
No. Keep the all 64bit to stay high, make thing simple. instead of some low and some use high.
Quoted 5 lines
Like Vivek said, failing at different point shouldn't be a problem. That's an incorrect configuration. When crashkernel=1G,high, old kexec-tools still fails the same way. That could cause confusion, in your word.
If it would fail later, we should let it fail early as possible.
Quoted 9 lines
Let me put it in an example, a user want to utilize this new kernel feature to reserve 1G for crash kernel but not upgrade kexec-tools,
- W/o this patch:
First he would try crashkernel=1G, but failed to reserve. Second time, he goes with crashkernel=1G,high, reservation is fine but kexec fails to load. Upgrading kexec-tools is essential to him.
Quoted 7 lines
- W/ this patch:
First he would try crashkernel=1G, reservation is ok but kexec fails to load the same way as the case of crashkernel=1G,high. Upgrading kexec-tools is essential to him.The point is old kexec-tools can't load high, no matter by what kind of crashkernel cmdline to reserve at high.
old kexec-tools could work cross 892M in some case.
That will confuse the user, as it works some time on some setup, but does not work on other setup.
Thanks
Yinghai
On Thu, Oct 24, 2013 at 12:13:41PM -0700, Yinghai Lu wrote:
[..]
Quoted 2 lines
old kexec-tools could work cross 892M in some case. That will confuse the user,
If it could work beyong 892MB, then why are you limiting user to 896MB only. That sounds wrong.
Quoted 2 lines
as it works some time on some setup, but does
not work on other setup.
If it can work above 896MB, then those other setup should be fixed.
All the existing users of crashkernel=xM *will not* be affected by this change. Memory reservation will still be below 892MB.
Memory reservation will gove above 896MB, only if user chooses to reserve more memory. If kexec works in that case, its a win-win situation. If it does not and fails, then it is status quo is maintained.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 4 lines · Yinghai Lu
Quoted 3 lines · Vivek Goyal
On Fri, Oct 18, 2013 at 10:45:43PM -0700, Yinghai Lu wrote:
Quoted 8 lines
Quoted 3 lines
IIUC, you are trying to say that with new kernel old kexec-tools will fail at a different failure point. I don't see why that is a problem. It still fails.
Yes, that could cause confusion.
We already knew it would fail possible at most later, we should make it skip allocation during first kernel booting.
One could upgrade kexec-tools later. So Kernel does not know whether user space is able to handle higher memory regions or not. So forcing kernel to skip memory allocation does not sound right to me.
Also even with memory reserved high, old kexec-tools which can't deal with it will fail anyway. So I don't agree that it is a problem.
[..]
Quoted 9 lines
Quoted 6 lines
No, it is not that simple. Think from a distribution's perspective also. We have the logic to scale reserved memory based on physical memory present in the system. Now we are seeing bigger memory systems (which would not have worked in the past). We still want to retain the existing logic and not switch to crashkernel=x,high. One does not have to. It makes life simpler.
distribution should go with ",high" for 64 bit kernel and new kexec-tools. for 32bit kernel, you still can have ",high" or not, as ",high" is ignored.
Just because we have the facility to allocate memory starting from top, does not mean that we should kill other options and force user to use this option.
With crashkernel=X,high, a low memory will be reserved for swiotlb. And I don't want that extra memory reservation. I am more than happy to reserve memory below 4G and not do low memory reservation.
[..]
Quoted 2 lines
Even we want to extend crashkernel=XM, then i would like to have it identical to crashkernel=XM,high instead.
You are assuming that crashkernel=xM,high is always good. I am arguing that because it requires low memory reservation for swiotlb, when IOMMU is not around, it will end up reserving more memory. So it is not necessarily better than reserving memory below 4G.
Hence both crashkernel=xM and crashkernel=XM,high have their own usage. We have been using crashkernel=xM and we know it works. So extending it to be able to allocate memory from higher regions, if sufficient memory is not available in lower regions makes sense. Memory reservation below 4G is more efficient due to not requiring swiotlb. And crashkernel=xM has been working for us and users are familiar with it.
So I don't see a point that why would you try to block any move to extend crashkernel=xM semantics.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 11 lines · Vivek Goyal
On Wed, Oct 23, 2013 at 11:11:51PM -0700, Yinghai Lu wrote:
Hence both crashkernel=xM and crashkernel=XM,high have their own usage. We have been using crashkernel=xM and we know it works. So extending it to be able to allocate memory from higher regions, if sufficient memory is not available in lower regions makes sense. Memory reservation below 4G is more efficient due to not requiring swiotlb. And crashkernel=xM has been working for us and users are familiar with it.
So I don't see a point that why would you try to block any move to extend crashkernel=xM semantics.
Make the thing simple.
Keep them separately, leave crashkernel=xM to old kexec-tools mostly and keep crashkernel=xM,high to newer kexec-tools as needed.
Thanks
Yinghai
Quoted 10 lines · Yinghai Lu
Quoted 9 lines · Vivek Goyal
On Wed, Oct 23, 2013 at 11:11:51PM -0700, Yinghai Lu wrote:
Hence both crashkernel=xM and crashkernel=XM,high have their own usage. We have been using crashkernel=xM and we know it works. So extending it to be able to allocate memory from higher regions, if sufficient memory is not available in lower regions makes sense. Memory reservation below 4G is more efficient due to not requiring swiotlb. And crashkernel=xM has been working for us and users are familiar with it.
Quoted 6 lines
Quoted 2 lines
So I don't see a point that why would you try to block any move to extend crashkernel=xM semantics.
Make the thing simple.
Keep them separately, leave crashkernel=xM to old kexec-tools mostly and keep crashkernel=xM,high to newer kexec-tools as needed.
I am keeping things simple by making sure that both old kexec-tools and new kexec-tools can use crashkernel=xM and one does not have to choose between two based on what kexec-tools version you are using.
Also keeping things simple by not trying to *impose* a new crashkernel= syntax on existing crashkernel=xM users.
Hence extending the semantics of crashkernel=xM makes sense to me.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 4 lines · Vivek Goyal
On Thu, Oct 24, 2013 at 12:15:25PM -0700, Yinghai Lu wrote:
Also keeping things simple by not trying to *impose* a new crashkernel= syntax on existing crashkernel=xM users.
Existing user that have crashkernel=xM working with their old kernel and old kexec-tools, they still could keep their old command line and old kexec-tools
with new updated kernel.
We should not change semantics to surprise them.
Thanks
Yinghai
Quoted 11 lines · Yinghai Lu
Quoted 4 lines · Vivek Goyal
On Thu, Oct 24, 2013 at 12:15:25PM -0700, Yinghai Lu wrote:
Also keeping things simple by not trying to *impose* a new crashkernel= syntax on existing crashkernel=xM users.
Existing user that have crashkernel=xM working with their old kernel and old kexec-tools, they still could keep their old command line and old kexec-tools
with new updated kernel.
We should not change semantics to surprise them.
Old users will get reservation still below 896MB.
It will go above 896MB only if memory could not be allocated below 896MB.
In the past reservation will fail and kexec-tools will fail. Now reservation will succeed but kexec-tools will fail.
So end result a user sees is that kexec-tools fails. So I don't see how we are breaking existing installations or user setups.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 13 lines · Vivek Goyal
Quoted 11 lines · Yinghai Lu
Quoted 4 lines · Vivek Goyal
On Thu, Oct 24, 2013 at 12:15:25PM -0700, Yinghai Lu wrote:
Also keeping things simple by not trying to *impose* a new crashkernel= syntax on existing crashkernel=xM users.
Existing user that have crashkernel=xM working with their old kernel and old kexec-tools, they still could keep their old command line and old kexec-tools
with new updated kernel.
We should not change semantics to surprise them.
Quoted 6 lines
Old users will get reservation still below 896MB.
It will go above 896MB only if memory could not be allocated below 896MB.
In the past reservation will fail and kexec-tools will fail. Now reservation will succeed but kexec-tools will fail.
Only corner case I can see is where memory is reserved higher, old kexec-tools is not smart enough to figure out in advance that kernel will not boot at this high address and still goes ahead and loads the kernel and crash dump fails at run time.
But this will happen only if memory reservation below 896MB fails. And in most of the cases after kernel upgrade, memory should still come from memory region below 896MB.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 15 lines · Vivek Goyal
Quoted 11 lines · Yinghai Lu
Quoted 4 lines · Vivek Goyal
On Thu, Oct 24, 2013 at 12:15:25PM -0700, Yinghai Lu wrote:
Also keeping things simple by not trying to *impose* a new crashkernel= syntax on existing crashkernel=xM users.
Existing user that have crashkernel=xM working with their old kernel and old kexec-tools, they still could keep their old command line and old kexec-tools
with new updated kernel.
We should not change semantics to surprise them.Old users will get reservation still below 896MB.
Quoted 7 lines
It will go above 896MB only if memory could not be allocated below 896MB.
In the past reservation will fail and kexec-tools will fail. Now reservation will succeed but kexec-tools will fail.
So end result a user sees is that kexec-tools fails. So I don't see how we are breaking existing installations or user setups.
case could be: if user add more memory and put more pcie cards, and second kernel will need more ram and OOM there.
so user could just increase crashkernel=512M to crashkernel=1G.
without Cong's patch, kernel will fail to reserve, and user would dig to change it
to crashkernel=1G,high, and update kexec-tools.
with Cong's patch, kernel will reserve other range like between 896 and 4G, old kexec-tools either
fail to load second kernel or hang in purgatory or early stage of second kernel, or other unknown behavior.
I would think first path is much clear and predicted.
If my memory is right, HPA did not like idea that we try below 896M, and then under 4G and then above 4G.
and want us to have ",high" solution. Not sure if he would change his mind. Also you will need and #ifdef CONFIG_X86_64 for 896M/4G searching code so it will not confuse the
32bit arch.
Thanks
Yinghai
Quoted 1 line
without Cong's patch, kernel will fail to reserve, and user would dig
s/Chao/Cong ;)
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 3 lines · Dave Young
Quoted 1 line
without Cong's patch, kernel will fail to reserve, and user would dig
s/Chao/Cong ;)
s/Cong/Chao in fact :(
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 18 lines · Yinghai Lu
Quoted 17 lines · Vivek Goyal
Quoted 11 lines · Yinghai Lu
Quoted 4 lines · Vivek Goyal
On Thu, Oct 24, 2013 at 12:15:25PM -0700, Yinghai Lu wrote:
Also keeping things simple by not trying to *impose* a new crashkernel= syntax on existing crashkernel=xM users.
Existing user that have crashkernel=xM working with their old kernel and old kexec-tools, they still could keep their old command line and old kexec-tools
with new updated kernel.
We should not change semantics to surprise them.Old users will get reservation still below 896MB.
It will go above 896MB only if memory could not be allocated below 896MB.
Quoted 8 lines
Quoted 5 lines
In the past reservation will fail and kexec-tools will fail. Now reservation will succeed but kexec-tools will fail.
So end result a user sees is that kexec-tools fails. So I don't see how we are breaking existing installations or user setups.
case could be: if user add more memory and put more pcie cards, and second kernel will need more ram and OOM there.
Now makedumpfile supports cyclic mode by default. So one does not have to necessarily linearly scale reserved memory based on physical memory present in the box.
Quoted 1 line
so user could just increase crashkernel=512M to crashkernel=1G.
If user has new makedumpfile, OOM should not happen and one should not have to increase memory reservation.
Quoted 9 lines
without Cong's patch, kernel will fail to reserve, and user would dig to change it
to crashkernel=1G,high, and update kexec-tools.with Cong's patch, kernel will reserve other range like between 896 and 4G, old kexec-tools either
fail to load second kernel or hang in purgatory or early stage of second kernel, or other unknown behavior.
I understand your concern about memory being reserved high and kexec just hanging. Only thing I am arguing is that the number of cases where it will happen is small.
- First of all for all old existing kexec-tools memory will stil come from 896MB.
- Old kexec-tools enforced that purgatory is loaded below 2G. So if memory is reserved above 2G, kexec-tools will fail.
So only problematic case seems to be that if user increased crashkernel=X value and memory got reserved between 896M and 2G. But this is not same as breaking old setups as old setups anyway never worked with this configuration.
You argument that user will research and upgrade kexec-tools and use crashkernel=X,high, then it holds true for the case where memory is reserved between 896M and 2G.
I personally think that it is easier for a user to not change any kernel parameters with kernel and kexec-tools upgrade and still be able to work with large memory systems. So the benefit of extending the semantics of existing parameter seems to be outweighing the downside of side, IMHO.
Quoted 5 lines
I would think first path is much clear and predicted.
If my memory is right, HPA did not like idea that we try below 896M, and then under 4G and then above 4G.
hpa, I know you did not like the idea in the past. Is it still the case.
IMHO, I like the fact that existing users still be able to work with crashkernel=X and not forced to switch to crashkernel=x,high and also incur the penalty of reserving extra memory for software iotlb.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 72 lines · Vivek Goyal
Quoted 18 lines · Yinghai Lu
Quoted 17 lines · Vivek Goyal
Quoted 11 lines · Yinghai Lu
Quoted 4 lines · Vivek Goyal
On Thu, Oct 24, 2013 at 12:15:25PM -0700, Yinghai Lu wrote:
Also keeping things simple by not trying to *impose* a new crashkernel= syntax on existing crashkernel=xM users.
Existing user that have crashkernel=xM working with their old kernel and old kexec-tools, they still could keep their old command line and old kexec-tools
with new updated kernel.
We should not change semantics to surprise them.Old users will get reservation still below 896MB.
It will go above 896MB only if memory could not be allocated below 896MB.
Quoted 8 lines
Quoted 5 lines
In the past reservation will fail and kexec-tools will fail. Now reservation will succeed but kexec-tools will fail.
So end result a user sees is that kexec-tools fails. So I don't see how we are breaking existing installations or user setups.
case could be: if user add more memory and put more pcie cards, and second kernel will need more ram and OOM there.
Now makedumpfile supports cyclic mode by default. So one does not have to necessarily linearly scale reserved memory based on physical memory present in the box.
Quoted 1 line
so user could just increase crashkernel=512M to crashkernel=1G.
If user has new makedumpfile, OOM should not happen and one should not have to increase memory reservation.
Quoted 9 lines
without Cong's patch, kernel will fail to reserve, and user would dig to change it
to crashkernel=1G,high, and update kexec-tools.with Cong's patch, kernel will reserve other range like between 896 and 4G, old kexec-tools either
fail to load second kernel or hang in purgatory or early stage of second kernel, or other unknown behavior.I understand your concern about memory being reserved high and kexec just hanging. Only thing I am arguing is that the number of cases where it will happen is small.
- First of all for all old existing kexec-tools memory will stil come from 896MB.
- Old kexec-tools enforced that purgatory is loaded below 2G. So if memory is reserved above 2G, kexec-tools will fail.So only problematic case seems to be that if user increased crashkernel=X value and memory got reserved between 896M and 2G. But this is not same as breaking old setups as old setups anyway never worked with this configuration.
You argument that user will research and upgrade kexec-tools and use crashkernel=X,high, then it holds true for the case where memory is reserved between 896M and 2G.
I personally think that it is easier for a user to not change any kernel parameters with kernel and kexec-tools upgrade and still be able to work with large memory systems. So the benefit of extending the semantics of existing parameter seems to be outweighing the downside of side, IMHO.
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
I would think first path is much clear and predicted.
If my memory is right, HPA did not like idea that we try below 896M, and then under 4G and then above 4G.
hpa, I know you did not like the idea in the past. Is it still the case.
Hi, hpa
Do you still not like this idea? Could you make it more clear to us please?
Thanks
Chao
Quoted 7 lines
IMHO, I like the fact that existing users still be able to work with crashkernel=X and not forced to switch to crashkernel=x,high and also incur the penalty of reserving extra memory for software iotlb.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 6 lines
Again, for distribution, when new kernel is added, new kernel will all have ",high"
and new kexec-tools get installed.Even we want to extend crashkernel=XM, then i would like to have it identical to crashkernel=XM,high instead.
My points: I can accept the approach of extending crashkernel=XM to use high mem firstly if we could remove the extra 72M in low mem, but apparently it's not doable now. So I would vote for Chao's patch.
Also newer kernel use more complex ",high" then older kernel looks not good.
Thanks
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 13 lines · WANG Chao
Now crashkernel=X will fail out if there's not enough memory at low (below 896M). What makes sense for crashkernel=X would be:
- First try to reserve X below 896M (for being compatible with old kexec-tools).
- If fails, try to reserve X below 4G (swiotlb need to stay below 4G).
- If fails, try to reserve X from MAXMEM top down.arch/x86/kernel/setup.c +0
-0
arch/x86/kernel/setup.c static void __init reserve_crashkernel(void) · line 602
Quoted 10 lines
arch/x86/kernel/setup.c +7
-0@@ -593,6 +593,20 @@ static void __init reserve_crashkernel(void) high ?CRASH_KERNEL_ADDR_HIGH_MAX :CRASH_KERNEL_ADDR_LOW_MAX , crash_size, alignment); +/* +* crashkernel=X reserve below 896M fails? Try below 4G +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, + (1ULL <<32 ), + crash_size, alignment);
Another problem, it would allocate range in [0,4g) for 32bit, if the user have crashkernel=512M or plus.
arch/x86/kernel/setup.c static void __init reserve_crashkernel(void) · line 612
Quoted 10 lines
arch/x86/kernel/setup.c +7
-0+/* +* crashkernel=X reserve below 4G fails? Try MAXMEM +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, +CRASH_KERNEL_ADDR_HIGH_MAX , + crash_size, alignment);if (!crash_base) {pr_info ("crashkernel reservation failed - No suitable area found.\n" );
Yinghai
Quoted 19 lines · Yinghai Lu
Quoted 18 lines · WANG Chao
Now crashkernel=X will fail out if there's not enough memory at low (below 896M). What makes sense for crashkernel=X would be:
- First try to reserve X below 896M (for being compatible with old kexec-tools).
- If fails, try to reserve X below 4G (swiotlb need to stay below 4G).
- If fails, try to reserve X from MAXMEM top down.arch/x86/kernel/setup.c +2
-0@@ -593,6 +593,20 @@ static void __init reserve_crashkernel(void) high ?CRASH_KERNEL_ADDR_HIGH_MAX :CRASH_KERNEL_ADDR_LOW_MAX , crash_size, alignment); +/* +* crashkernel=X reserve below 896M fails? Try below 4G
Quoted 8 lines
arch/x86/kernel/setup.c static void __init reserve_crashkernel(void) · line 602
Quoted 5 lines
arch/x86/kernel/setup.c +5
-0+*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, + (1ULL <<32 ), + crash_size, alignment);Another problem, it would allocate range in [0,4g) for 32bit, if the user have crashkernel=512M or plus.
Yes, should use #ifdef CONFIG_X86_64 .. #endif.
Thanks for pointing this out.
WANG Chao
Quoted 9 lines
Quoted 9 lines
arch/x86/kernel/setup.c +7
-0+/* +* crashkernel=X reserve below 4G fails? Try MAXMEM +*/ +if (!high && !crash_base) + crash_base =memblock_find_in_range (alignment, +CRASH_KERNEL_ADDR_HIGH_MAX , + crash_size, alignment);
Quoted 5 lines
arch/x86/kernel/setup.c static void __init reserve_crashkernel(void) · line 612
Quoted 2 lines
arch/x86/kernel/setup.c +0
-0if (!crash_base) {pr_info ("crashkernel reservation failed - No suitable area found.\n" );Yinghai
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec