get_user_pages() still broken in 2.6

DISCUSSION Timur Tabi 6 participants 10 messages Sep 28, 2004 – Sep 29, 2004 Memory management

Share & tools

Timur Tabi

author

2004-09-28 22:40 UTC #0 lore raw

I was hoping that this bug would be fixed in the 2.6 kernels, but apparently it hasn't been.

Function get_user_pages() is supposed to lock user memory. However, under extreme memory constraints, the kernel will swap out the "locked" memory.

I have a test app which does this:

1) Calls our driver, which issues a get_user_pages() call for one page.
2) Calls our driver again to get the physical address of that page (the driver uses pgd/pmd/pte_offset).
3) Tries allocate 1GB of memory (this system has 1GB of physical RAM).
4) Tries to get the physical address again.

In step 4, the physical address is usually zero, which means either pgd_offset or pmd_offset failed. This indicates the page was swapped out.

I don't understand how this bug can continue to exist after all this time. get_user_pages() is supposed to lock the memory, because drivers use it for DMA'ing directly into user memory.

Signature · 8 lines

Timur Tabi
Staff Software Engineer
[email protected]
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Christoph Hellwig

2004-09-28 23:03 UTC #1 lore raw

Quoted 15 lines · Timur Tabi

I was hoping that this bug would be fixed in the 2.6 kernels, but apparently it hasn't been.
Function get_user_pages() is supposed to lock user memory. However, under extreme memory constraints, the kernel will swap out the "locked" memory.
I have a test app which does this:
1) Calls our driver, which issues a get_user_pages() call for one page.
2) Calls our driver again to get the physical address of that page (the driver uses pgd/pmd/pte_offset).
3) Tries allocate 1GB of memory (this system has 1GB of physical RAM).
4) Tries to get the physical address again.

Quoted 6 lines

In step 4, the physical address is usually zero, which means either pgd_offset or pmd_offset failed. This indicates the page was swapped out.
I don't understand how this bug can continue to exist after all this time. get_user_pages() is supposed to lock the memory, because drivers use it for DMA'ing directly into user memory.

get_user_pages locks the page in memory. It doesn't do anything about ptes.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Dave Hansen

2004-09-28 23:21 UTC #2 lore raw

Quoted 1 line · Christoph Hellwig

get_user_pages locks the page in memory. It doesn't do anything about ptes.

You probably want mlock(2) to keep the kernel from messing with the ptes at all. But, you should probably really be thinking about why you're accessing the page tables at all. I count *ONE* instance in drivers/ where page tables are accessed directly.

-- Dave

Timur Tabi

author

2004-09-29 14:46 UTC #3 lore raw

Quoted 2 lines · Dave Hansen

You probably want mlock(2) to keep the kernel from messing with the ptes at all.

mlock() can only be called via sys_mlock(), which is a user-space call.
Not only that, but only root can call sys_mlock(). This is not
compatible with our needs.

Quoted 3 lines

But, you should probably really be thinking about why you're
accessing the page tables at all. I count *ONE* instance in drivers/ where page tables are accessed directly.

I access PTEs to get the physical addresses of a user-space buffer, so that we can DMA to/from it directly.

Signature · 8 lines

Timur Tabi
Staff Software Engineer
[email protected]
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Timur Tabi

author

2004-09-29 14:48 UTC #4 lore raw

In reply to Christoph Hellwig

Quoted 1 line · Christoph Hellwig

get_user_pages locks the page in memory. It doesn't do anything about ptes.

I don't understand the difference. I thought a locked page is one that stays in memory (i.e. isn't swapped out) and whose physical address never changes. Is that wrong? All I need to do is keep a page in memory at the same physical address until I'm done with it.

Signature · 8 lines

Timur Tabi
Staff Software Engineer
[email protected]
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Christoph Hellwig

2004-09-29 15:01 UTC #5 lore raw

Quoted 7 lines · Timur Tabi

Quoted 1 line · Christoph Hellwig
get_user_pages locks the page in memory. It doesn't do anything about ptes.
I don't understand the difference. I thought a locked page is one that stays in memory (i.e. isn't swapped out) and whose physical address never changes. Is that wrong?

Yes. But if you're walking ptes you're looking at virtual addresses somehow. Can you send me a pointer to your code please? I suspect it's doing something terribly stupid.

Rakesh Jagota

2004-09-29 04:47 UTC #6 lore raw

opening a file inside the kernel module

In reply to Timur Tabi

Hi all,
I am working in linux, i would like to know abt whether can I open a file inside the kernel module without using any application. If so how how the files_struct will be maintained. Does a kernel module has this struct?

Waiting for any suggestion from the list.

Thanks in advance,
rakesh

Jeff Garzik

2004-09-29 05:00 UTC #7 lore raw

Re: opening a file inside the kernel module

Quoted 4 lines · Rakesh Jagota

Hi all,
I am working in linux, i would like to know abt whether can I open a file inside the kernel module without using any application. If so how how the files_struct will be maintained. Does a kernel module has this struct?

Don't do this. It's incompatible with namespaces.

Instead, figure out some way to pass the file contents to the kernel module.

Jeff

Rakesh Jagota

2004-09-29 05:54 UTC #8 lore raw

Re: opening a file inside the kernel module

Hi,
Thnx.

I want to implement socket from the module. I won't be having any user process running to handle the descriptors coming from socket. Could you pl tell me how to handle the socket descriptor from the kernel module.

Thanks,
rakesh
----- Original Message -----
From: "Jeff Garzik" <[email protected]>
To: "Rakesh Jagota" <[email protected]>
Cc: <[email protected]>; <[email protected]>; <[email protected]>
Sent: Wednesday, September 29, 2004 10:30 AM
Subject: Re: opening a file inside the kernel module

Quoted 3 lines

Quoted 2 lines · Rakesh Jagota
Hi all,
I am working in linux, i would like to know abt whether can I open a

file

Quoted 1 line

Quoted 1 line
inside the kernel module without using any application. If so how how

the

Quoted 5 lines

Quoted 1 line
files_struct will be maintained. Does a kernel module has this struct?
Don't do this. It's incompatible with namespaces.
Instead, figure out some way to pass the file contents to the kernel

module.

Quoted 11 lines

Jeff

Kernelnewbies: Help each other learn about the Linux kernel.
Archive:       http://mail.nl.linux.org/kernelnewbies/
FAQ:           http://kernelnewbies.org/faq/

Stuart MacDonald

2004-09-29 15:52 UTC #9 lore raw

RE: opening a file inside the kernel module

Quoted 4 lines

I want to implement socket from the module. I won't be having any user process running to handle the descriptors coming from socket. Could you pl
tell me how to handle the socket descriptor from the kernel module.

Check out fs/smbfs/sock.c.

..Stu