Why we echo a invalid start_address_of_new_memory succeeded ?

DISCUSSION Zhang Zhen 2 participants 4 messages Jun 20, 2014 – Jun 23, 2014 Memory management

Share & tools

Zhang Zhen

author

2014-06-20 07:07 UTC #0 lore raw

Hi,

I am testing mem-hotplug on a qemu virtual machine. I executed the following command to notify memory hot-add event by hand.

% echo start_address_of_new_memory > /sys/devices/system/memory/probe

To a different start_address_of_new_memory I got different results. The results are as follows:

MBSC-x86_64 /sys/devices/system/memory # ls

block_size_bytes  memory2           memory5           power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent

MBSC-x86_64 /sys/devices/system/memory # echo 0x70000000 > probe MBSC-x86_64 /sys/devices/system/memory # echo 0x78000000 > probe -sh: echo: write error: File exists
MBSC-x86_64 /sys/devices/system/memory # echo 0x80000000 > probe -sh: echo: write error: File exists
MBSC-x86_64 /sys/devices/system/memory # echo 0x88000000 > probe -sh: echo: write error: File exists
MBSC-x86_64 /sys/devices/system/memory # echo 0x8f000000 > probe -sh: echo: write error: Invalid argument
MBSC-x86_64 /sys/devices/system/memory # echo 0x90000000 > probe -sh: echo: write error: File exists
MBSC-x86_64 /sys/devices/system/memory # echo 0xff0000000 > probe MBSC-x86_64 /sys/devices/system/memory # ls

block_size_bytes  memory2           memory510         probe
memory0           memory3           memory6           uevent
memory1           memory4           memory7
memory14          memory5           power
MBSC-x86_64 /sys/devices/system/memory # echo 0xfff0000000 > probe
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory510         power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent
memory14          memory5           memory8190

The qemu virtual machine's physical memory size is 2048M, and the boot memory is 1024M.

MBSC-x86_64 / # cat /proc/meminfo
MemTotal: 1018356 kB
MBSC-x86_64 / # cat /sys/devices/system/memory/block_size_bytes 8000000

Three questions:
1. The machine's physical memory size is 2048M, why echo 0x78000000 as the start_address_of_new_memory failed ?

2. Why echo 0x8f000000 as the start_address_of_new_memory, the error message is different ?

3. Why echo 0xfff0000000 as the start_address_of_new_memory succeeded ? 0xfff0000000 has exceeded the machine's physical memory size.

Best regards!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

David Rientjes

2014-06-20 10:30 UTC #1 lore raw

Quoted 7 lines · Zhang Zhen

Hi,
I am testing mem-hotplug on a qemu virtual machine. I executed the following command to notify memory hot-add event by hand.
% echo start_address_of_new_memory > /sys/devices/system/memory/probe

Quoted 8 lines

To a different start_address_of_new_memory I got different results. The results are as follows:
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory5           power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent
MBSC-x86_64 /sys/devices/system/memory # echo 0x70000000 > probe

Since block_size_bytes is 0x8000000 == 128MB, this is 0x70000000 / 0x8000000 = section number 14. Successfully hot added. Presumably you're reporting that there is no physical memory there, so this would default to the online node of the first memory block, probably node 0.

Quoted 2 lines

MBSC-x86_64 /sys/devices/system/memory # echo 0x78000000 > probe -sh: echo: write error: File exists

EEXIST gets returned when the resource already exists, mostly likely system RAM or reserved memory as reported by your BIOS. You report this is a 2GB machine, no reason to believe memory at 1920MB isn't already online (including reserved).

Quoted 4 lines

MBSC-x86_64 /sys/devices/system/memory # echo 0x80000000 > probe -sh: echo: write error: File exists
MBSC-x86_64 /sys/devices/system/memory # echo 0x88000000 > probe -sh: echo: write error: File exists

Same.

Quoted 2 lines

MBSC-x86_64 /sys/devices/system/memory # echo 0x8f000000 > probe -sh: echo: write error: Invalid argument

Returns EINVAL because it's not a multiple of block_size_bytes, it's not aligned properly.

Quoted 2 lines

MBSC-x86_64 /sys/devices/system/memory # echo 0x90000000 > probe -sh: echo: write error: File exists

See above, the resoure already exists. Check your e820 your dmesg, which is missing from this report, to determine what already exists and may be already online or reserved.

Quoted 1 line

MBSC-x86_64 /sys/devices/system/memory # echo 0xff0000000 > probe

0xff0000000 / 0x8000000 is section 510, successfully onlined.

Quoted 5 lines

MBSC-x86_64 /sys/devices/system/memory # ls

block_size_bytes  memory2           memory510         probe
memory0           memory3           memory6           uevent
memory1           memory4           memory7
memory14          memory5           power

Looks good, you onlined sections 14 and 510 above.

Quoted 1 line

MBSC-x86_64 /sys/devices/system/memory # echo 0xfff0000000 > probe

Same for section 8190.

Quoted 6 lines

MBSC-x86_64 /sys/devices/system/memory # ls

block_size_bytes  memory2           memory510         power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent
memory14          memory5           memory8190

Confirmed it's onlined.

Quoted 7 lines

The qemu virtual machine's physical memory size is 2048M, and the boot memory is 1024M.
MBSC-x86_64 / # cat /proc/meminfo
MemTotal: 1018356 kB
MBSC-x86_64 / # cat /sys/devices/system/memory/block_size_bytes 8000000

That's irrelevant, you've explicitly onlined memory that doesn't exist. Not sure why you're using the probe interface unless you need it for x86, is ACPI not registering it correctly?

Quoted 3 lines

Three questions:
1. The machine's physical memory size is 2048M, why echo 0x78000000 as the start_address_of_new_memory failed ?

Copy your e820 map from your dmesg, it's probably reserved or already online, this is lower than 2048M.

Quoted 2 lines

2. Why echo 0x8f000000 as the start_address_of_new_memory, the error message is different ?

Not properly aligned to block_size_bytes. It's a nuance, but block_size_bytes is exported in hex, not decimal.

Quoted 2 lines

3. Why echo 0xfff0000000 as the start_address_of_new_memory succeeded ? 0xfff0000000 has exceeded the machine's physical memory size.

You're telling the kernel differently.

Zhang Zhen

author

2014-06-23 10:50 UTC #2 lore raw

Quoted 88 lines · David Rientjes

Quoted 7 lines · Zhang Zhen
Hi,
I am testing mem-hotplug on a qemu virtual machine. I executed the following command to notify memory hot-add event by hand.
% echo start_address_of_new_memory > /sys/devices/system/memory/probe
Quoted 8 lines
To a different start_address_of_new_memory I got different results. The results are as follows:
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory5           power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent
MBSC-x86_64 /sys/devices/system/memory # echo 0x70000000 > probe
Since block_size_bytes is 0x8000000 == 128MB, this is 0x70000000 / 0x8000000 = section number 14. Successfully hot added. Presumably you're reporting that there is no physical memory there, so this would default to the online node of the first memory block, probably node 0.
Quoted 2 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x78000000 > probe -sh: echo: write error: File exists
EEXIST gets returned when the resource already exists, mostly likely system RAM or reserved memory as reported by your BIOS. You report this is a 2GB machine, no reason to believe memory at 1920MB isn't already online (including reserved).
Quoted 4 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x80000000 > probe -sh: echo: write error: File exists
MBSC-x86_64 /sys/devices/system/memory # echo 0x88000000 > probe -sh: echo: write error: File exists
Same.
Quoted 2 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x8f000000 > probe -sh: echo: write error: Invalid argument
Returns EINVAL because it's not a multiple of block_size_bytes, it's not aligned properly.
Quoted 2 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x90000000 > probe -sh: echo: write error: File exists
See above, the resoure already exists. Check your e820 your dmesg, which is missing from this report, to determine what already exists and may be already online or reserved.
Quoted 1 line
MBSC-x86_64 /sys/devices/system/memory # echo 0xff0000000 > probe
0xff0000000 / 0x8000000 is section 510, successfully onlined.
Quoted 5 lines
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory510         probe
memory0           memory3           memory6           uevent
memory1           memory4           memory7
memory14          memory5           power
Looks good, you onlined sections 14 and 510 above.
Quoted 1 line
MBSC-x86_64 /sys/devices/system/memory # echo 0xfff0000000 > probe
Same for section 8190.
Quoted 6 lines
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory510         power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent
memory14          memory5           memory8190
Confirmed it's onlined.
Quoted 7 lines
The qemu virtual machine's physical memory size is 2048M, and the boot memory is 1024M.
MBSC-x86_64 / # cat /proc/meminfo
MemTotal: 1018356 kB
MBSC-x86_64 / # cat /sys/devices/system/memory/block_size_bytes 8000000
That's irrelevant, you've explicitly onlined memory that doesn't exist. Not sure why you're using the probe interface unless you need it for x86, is ACPI not registering it correctly?

Quoted 7 lines

Quoted 3 lines
Three questions:
1. The machine's physical memory size is 2048M, why echo 0x78000000 as the start_address_of_new_memory failed ?
Copy your e820 map from your dmesg, it's probably reserved or already online, this is lower than 2048M.

Hi David,

You are right, if we echo 0x78000000 as the start_address_of_new_memory, the end_address_of_new_memory is exceeded the usable range.
Thank you for your comments.

My e820 map as follows:

[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007fffdfff] usable
[    0.000000] BIOS-e820: [mem 0x000000007fffe000-0x000000007fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] e820: remove [mem 0x40000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] e820: user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] user: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000003fffffff] usable
[    0.000000] user: [mem 0x000000007fffe000-0x000000007fffffff] reserved
[    0.000000] user: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved

Quoted 5 lines

Quoted 2 lines
2. Why echo 0x8f000000 as the start_address_of_new_memory, the error message is different ?
Not properly aligned to block_size_bytes. It's a nuance, but block_size_bytes is exported in hex, not decimal.

You are right, it's not properly aligned to block_size_bytes. I have made a mistake.

Quoted 6 lines

Quoted 3 lines
3. Why echo 0xfff0000000 as the start_address_of_new_memory succeeded ? 0xfff0000000 has exceeded the machine's physical memory size.
You're telling the kernel differently.

I'm not clearly here, 0xfff0000000 is exceeded the usable range [mem 0x0000000000100000-0x000000007fffdfff] usable. So i think here should return "File exists", but it succeeded.

Is it properly ?

Best regards!

Quoted 2 lines

.

Zhang Zhen

author

2014-06-23 11:18 UTC #3 lore raw

Quoted 134 lines · Zhang Zhen

Quoted 88 lines · David Rientjes
Quoted 7 lines · Zhang Zhen
Hi,
I am testing mem-hotplug on a qemu virtual machine. I executed the following command to notify memory hot-add event by hand.
% echo start_address_of_new_memory > /sys/devices/system/memory/probe
Quoted 8 lines
To a different start_address_of_new_memory I got different results. The results are as follows:
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory5           power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent
MBSC-x86_64 /sys/devices/system/memory # echo 0x70000000 > probe
Since block_size_bytes is 0x8000000 == 128MB, this is 0x70000000 / 0x8000000 = section number 14. Successfully hot added. Presumably you're reporting that there is no physical memory there, so this would default to the online node of the first memory block, probably node 0.
Quoted 2 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x78000000 > probe -sh: echo: write error: File exists
EEXIST gets returned when the resource already exists, mostly likely system RAM or reserved memory as reported by your BIOS. You report this is a 2GB machine, no reason to believe memory at 1920MB isn't already online (including reserved).
Quoted 4 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x80000000 > probe -sh: echo: write error: File exists
MBSC-x86_64 /sys/devices/system/memory # echo 0x88000000 > probe -sh: echo: write error: File exists
Same.
Quoted 2 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x8f000000 > probe -sh: echo: write error: Invalid argument
Returns EINVAL because it's not a multiple of block_size_bytes, it's not aligned properly.
Quoted 2 lines
MBSC-x86_64 /sys/devices/system/memory # echo 0x90000000 > probe -sh: echo: write error: File exists
See above, the resoure already exists. Check your e820 your dmesg, which is missing from this report, to determine what already exists and may be already online or reserved.
Quoted 1 line
MBSC-x86_64 /sys/devices/system/memory # echo 0xff0000000 > probe
0xff0000000 / 0x8000000 is section 510, successfully onlined.
Quoted 5 lines
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory510         probe
memory0           memory3           memory6           uevent
memory1           memory4           memory7
memory14          memory5           power
Looks good, you onlined sections 14 and 510 above.
Quoted 1 line
MBSC-x86_64 /sys/devices/system/memory # echo 0xfff0000000 > probe
Same for section 8190.
Quoted 6 lines
MBSC-x86_64 /sys/devices/system/memory # ls
block_size_bytes  memory2           memory510         power
memory0           memory3           memory6           probe
memory1           memory4           memory7           uevent
memory14          memory5           memory8190
Confirmed it's onlined.
Quoted 7 lines
The qemu virtual machine's physical memory size is 2048M, and the boot memory is 1024M.
MBSC-x86_64 / # cat /proc/meminfo
MemTotal: 1018356 kB
MBSC-x86_64 / # cat /sys/devices/system/memory/block_size_bytes 8000000
That's irrelevant, you've explicitly onlined memory that doesn't exist. Not sure why you're using the probe interface unless you need it for x86, is ACPI not registering it correctly?
Quoted 7 lines
Quoted 3 lines
Three questions:
1. The machine's physical memory size is 2048M, why echo 0x78000000 as the start_address_of_new_memory failed ?
Copy your e820 map from your dmesg, it's probably reserved or already online, this is lower than 2048M.
Hi David,
You are right, if we echo 0x78000000 as the start_address_of_new_memory, the end_address_of_new_memory is exceeded the usable range.
Thank you for your comments.
My e820 map as follows:
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007fffdfff] usable
[    0.000000] BIOS-e820: [mem 0x000000007fffe000-0x000000007fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] e820: remove [mem 0x40000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] e820: user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] user: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000003fffffff] usable
[    0.000000] user: [mem 0x000000007fffe000-0x000000007fffffff] reserved
[    0.000000] user: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
Quoted 5 lines
Quoted 2 lines
2. Why echo 0x8f000000 as the start_address_of_new_memory, the error message is different ?
Not properly aligned to block_size_bytes. It's a nuance, but block_size_bytes is exported in hex, not decimal.
You are right, it's not properly aligned to block_size_bytes. I have made a mistake.
Quoted 4 lines
Quoted 3 lines
3. Why echo 0xfff0000000 as the start_address_of_new_memory succeeded ? 0xfff0000000 has exceeded the machine's physical memory size.

Quoted 8 lines

Quoted 2 lines
You're telling the kernel differently.
I'm not clearly here, 0xfff0000000 is exceeded the usable range [mem 0x0000000000100000-0x000000007fffdfff] usable. So i think here should return "File exists", but it succeeded.
Is it properly ?

I got it, the address's validity should be guaranteed by me. Right? Thank you.

Quoted 6 lines

Best regards!
Quoted 2 lines
.

Quoted 8 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
.