Example filesystem fail to init on parisc
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Miklos Szeredi
maintainerQuoted 8 lines · Miklos Szeredi
Quoted 5 lines
Can you try the following patch to the library?
It initializes the memory to some random value (
0x28) before theread(), to check if the kernel sets the memory to zero or it doesn't touch the buffer at all.Interesting, the kernel is not setting the memory.
OK. I'm out of ideas, CC-ing parisc-linux, maybe they can help shed some light on this.
A short summary of the problem:
(for the full thread see:
http://thread.gmane.org/gmane.comp.file-systems.fuse.devel/2833)
FUSE doesn't seem to work on PA-RISC (2.6.15 + pa4 patchset)
The reason appears to be that when the userspace filesystem reads from the FUSE device, the kernel doesn't copy any data to the userspace read buffer, though the correct size is returned by read().
FUSE uses a combination of get_user_pages(), kmap_atomic() and memcpy(). After kunmap_atomic(), flush_dcache_page() is called to avoid virtual aliasing. The data in the read buffer is totally untouched.
Can anyone confirm the above behavior of FUSE on PA-RISC?
Any ideas why this would behave differently on PA-RISC as on other archs?
Thanks,
Miklos
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Miklos Szeredi
maintainerJick, Chris,
Can you please try the following module containing the simplified versions of the fuse device read/write routines?
Copy the attached files into an empty directory, then do 'make' and 'insmod copytest.ko'.
Then do
cat /proc/fs/test
echo hello > /proc/fs/test
dmesg | tail
I get something like this:
~$ cat /proc/fs/test
Hello!
~$ echo hello > /proc/fs/test
~$ dmesg | tailtest_read 0804d848 1024 0
dst: 0804d848 page: c13a9f40 offset: 2120 mapaddr: dd4fa000 ptr: dd4fa848
test_read: 7test_read 0804d848 1024 7
test_read: 0test_write 080e9008 6 0
src: 080e9008 page: c1167560 offset: 8 mapaddr: cb3ab000 ptr: cb3ab008 <hello
Quoted 1 line
test_write: 6
~$
You could also try changing the calls to my_copy_{to,from}_user2() to my_copy_..._user1() and see if it makes a difference.
Thanks,
Miklos
#include <linux/fs.h>
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/pagemap.h>
#include <asm/uaccess.h>
MODULE_LICENSE("GPL");
static int my_copy_to_user1(char __user *dst, const char *src, size_t size)
{
return copy_to_user(dst, src, size) ? -EFAULT : 0;
}static int my_copy_from_user1(char *dst, const char __user *src, size_t size)
{
return copy_from_user(dst, src, size) ? -EFAULT : 0;
}static int my_copy_to_user2(char __user *dst, const char *src, size_t size)
{
int err;
struct page *page;
unsigned long addr = (unsigned long) dst;
unsigned offset = addr % PAGE_SIZE;
void *mapaddr;
void *ptr;
if (offset + size > PAGE_SIZE)
return -EINVAL;
down_read(¤t->mm->mmap_sem);
err = get_user_pages(current, current->mm, addr, 1, 1, 0, &page, NULL);
up_read(¤t->mm->mmap_sem);
if (err < 0)
return err;
BUG_ON(err != 1);
mapaddr = kmap_atomic(page, KM_USER0);
ptr = mapaddr + offset;
printk("dst: %p page: %p offset: %u mapaddr: %p ptr: %p\n",
dst, page, offset, mapaddr, ptr);
memcpy(ptr, src, size);
kunmap_atomic(mapaddr, KM_USER0);
flush_dcache_page(page);
set_page_dirty_lock(page);
put_page(page);
return 0;
}static int my_copy_from_user2(char *dst, const char __user *src, size_t size)
{
int err;
struct page *page;
unsigned long addr = (unsigned long) src;
unsigned offset = addr % PAGE_SIZE;
void *mapaddr;
void *ptr;
if (offset + size > PAGE_SIZE)
return -EINVAL;
down_read(¤t->mm->mmap_sem);
err = get_user_pages(current, current->mm, addr, 1, 0, 0, &page, NULL);
up_read(¤t->mm->mmap_sem);
if (err < 0)
return err;
BUG_ON(err != 1);
mapaddr = kmap_atomic(page, KM_USER0);
ptr = mapaddr + offset;
printk("src: %p page: %p offset: %u mapaddr: %p ptr: %p\n",
src, page, offset, mapaddr, ptr);
memcpy(dst, ptr, size);
kunmap_atomic(mapaddr, KM_USER0);
put_page(page);
return 0;
}static ssize_t test_read(struct file *file, char __user *buf, size_t nbytes,
loff_t *off)
{
ssize_t res = 0;
char hello[] = "Hello!\n";
size_t hellolen = strlen(hello);
printk("test_read %p %lu %llu\n", buf, (unsigned long) nbytes,
(unsigned long long) *off);
if (!*off && nbytes > hellolen) {
res = my_copy_to_user2(buf, hello, hellolen);
if (!res) {
*off += hellolen;
res = hellolen;
}
}
printk(" test_read: %li\n", (long) res);
return res;
}static ssize_t test_write(struct file *file, const char __user *buf,
size_t nbytes, loff_t *off)
{
ssize_t res = nbytes;
char hello[32];
printk("test_write %p %lu %llu\n", buf, (unsigned long) nbytes,
(unsigned long long) *off);
if (!*off && nbytes < sizeof(hello) - 1) {
res = my_copy_from_user2(hello, buf, nbytes);
if (!res) {
hello[nbytes] = '\0';
printk("<%s>\n", hello);
*off += nbytes;
res = nbytes;
}
}
printk(" test_write: %li\n", (long) res);
return res;
}static struct file_operations test_ops = {
.owner = THIS_MODULE,
.read = test_read,
.write = test_write,
};static int test_init(void)
{
struct proc_dir_entry *ent;
ent = create_proc_entry("test", S_IFREG | 0666, proc_root_fs);
if (ent)
ent->proc_fops = &test_ops;
return 0;
}static void test_exit(void)
{
remove_proc_entry("test", proc_root_fs);
}module_init(test_init);module_exit(test_exit);
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
Quoted 2 lines · Miklos Szeredi
OK. I'm out of ideas, CC-ing parisc-linux, maybe they can help shed some light on this.
Miklos,
I took a quick look last week but didn't see anything obvious. I was suspicious of the memcopy() calls to copy stuff to user space. But willy me told those are fine if using the result of kmap() (and then kunmap'ing when done).
In a private conversation someone else said they would look at it on the weekend.
thanks,
grant
Quoted 22 lines
A short summary of the problem:
(for the full thread see:
http://thread.gmane.org/gmane.comp.file-systems.fuse.devel/2833)FUSE doesn't seem to work on PA-RISC (2.6.15 + pa4 patchset)
The reason appears to be that when the userspace filesystem reads from the FUSE device, the kernel doesn't copy any data to the userspace read buffer, though the correct size is returned by
read().FUSE uses a combination of
get_user_pages(),kmap_atomic()andmemcpy(). Afterkunmap_atomic(),flush_dcache_page()is called to avoid virtual aliasing. The data in the read buffer is totally untouched.Can anyone confirm the above behavior of FUSE on PA-RISC?
Any ideas why this would behave differently on PA-RISC as on other archs?
Quoted 6 lines
Thanks,
Miklos
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Grant Grundler
Quoted 2 lines · Miklos Szeredi
OK. I'm out of ideas, CC-ing parisc-linux, maybe they can help shed some light on this.
Miklos,
I took a quick look last week but didn't see anything obvious. I was suspicious of thememcopy()calls to copy stuff to user space. But willy me told those are fine if using the result ofkmap()(and then kunmap'ing when done).
Sorry, meant to reply earlier.
Quoted 8 lines
Quoted 8 lines
The reason appears to be that when the userspace filesystem reads from the FUSE device, the kernel doesn't copy any data to the userspace read buffer, though the correct size is returned by
read().FUSE uses a combination of
get_user_pages(),kmap_atomic()andmemcpy(). Afterkunmap_atomic(),flush_dcache_page()is called to avoid virtual aliasing. The data in the read buffer is totally untouched.
This is wrong. A VIPT cache requires a mapping to flush on. If you kunmap, you've lost the mapping. What you should do is
kmap()
operate on dataflush_kernel_dcache_page()kunmap()
James
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Miklos Szeredi
maintainerQuoted 2 lines
This is wrong. A VIPT cache requires a mapping to flush on. If you kunmap, you've lost the mapping.
Ahh, I knew I was missing something fundamental.
Quoted 6 lines
What you should do is
kmap()
operate on dataflush_kernel_dcache_page()kunmap()
Chris, can you try the below patch?
Thanks,
Miklos
fs/fuse/dev.c +7 -5
@@ -432,11 +432,11 @@ static void fuse_copy_init(struct fuse_cstatic void fuse_copy_finish (struct fuse_copy_state *cs) {if (cs->mapaddr) { -kunmap_atomic (cs->mapaddr,KM_USER0 ); -if (cs->write) { +if (cs->write)flush_dcache_page (cs->pg); +kunmap_atomic (cs->mapaddr,KM_USER0 ); +if (cs->write)set_page_dirty_lock (cs->pg); - }put_page (cs->pg); cs->mapaddr =NULL ; }
@@ -504,6 +504,8 @@ static int fuse_copy_page(struct fuse_coif (page && zeroing && count <PAGE_SIZE ) {void *mapaddr =kmap_atomic (page,KM_USER1 );memset (mapaddr,0 ,PAGE_SIZE ); +if (!cs->write) +flush_dcache_page (page);kunmap_atomic (mapaddr,KM_USER1 ); }while (count) {
@@ -514,12 +516,12 @@ static int fuse_copy_page(struct fuse_covoid *mapaddr =kmap_atomic (page,KM_USER1 );void *buf = mapaddr + offset; offset +=fuse_copy_do (cs, &buf, &count); +if (!cs->write) +flush_dcache_page (page);kunmap_atomic (mapaddr,KM_USER1 ); }else offset +=fuse_copy_do (cs,NULL , &count); } -if (page && !cs->write) -flush_dcache_page (page);return 0 ; }
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
Miklos Szeredi
maintainerQuoted 5 lines
Quoted 4 lines
Quoted 4 lines
FUSE uses a combination of
get_user_pages(),kmap_atomic()andmemcpy(). Afterkunmap_atomic(),flush_dcache_page()is called to avoid virtual aliasing. The data in the read buffer is totally untouched.
Quoted 7 lines
This is wrong. A VIPT cache requires a mapping to flush on. If you kunmap, you've lost the mapping. What you should do is
kmap()
operate on dataflush_kernel_dcache_page()kunmap()
OK, this didn't help. Looking closer it didn't have a chance of helping, since kunmap()/kunmap_atomic() are no-ops unless CONFIG_HIGHMEM is set, but on parisc there's no highmem support.
Any other ideas?
Thanks,
Miklos
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
The kernel output on my parisc machine using copy_to/from_user2() is a bit different:
woozle:/home/frost/fuse_test# insmod copytest.ko woozle:/home/frost/fuse_test# cat /proc/fs/test woozle:/home/frost/fuse_test# echo hello > /proc/fs/test woozle:/home/frost/fuse_test# dmesg | tail program smartd is using a deprecated SCSI ioctl, please convert it to SG_IO test_read 000258a0 1024 0 dst: 000258a0 page: 10acb460 offset: 2208 mapaddr: 3b523000 ptr: 3b5238a0 test_read: 7 test_read 000258a0 1024 7 test_read: 0 test_write 000d6c08 6 0 src: 000d6c08 page: 108e9060 offset: 3080 mapaddr: 2c403000 ptr: 2c403c08 <������> test_write: 6 woozle:/home/frost/fuse_test#
Changing the copy_to/from_user2() calls to copy_to/from_user1() changes the output to include the "Hello!" and "<hello\n>" you see on an x86 (x86, I presume);
woozle:/home/frost/fuse_test# insmod copytest.ko woozle:/home/frost/fuse_test# cat /proc/fs/test Hello! woozle:/home/frost/fuse_test# echo hello > /proc/fs/test woozle:/home/frost/fuse_test# dmesg | tail -9 program smartd is using a deprecated SCSI ioctl, please convert it to SG_IO test_read 000258a0 1024 0 test_read: 7 test_read 000258a0 1024 7 test_read: 0 test_write 000d6c08 6 0 <hello
Quoted 1 line
test_write: 6
woozle:/home/frost/fuse_test#
Signature · 9 lines
Chris Frost | <http://www.frostnet.net/chris/> -------------+---------------------------------- Public PGP Key: Email [email protected] with the subject "retrieve pgp key" or visit <http://www.frostnet.net/chris/about/pgp_key.phtml> _______________________________________________ parisc-linux mailing list [email protected] http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines · Chris Frost
The kernel output on my parisc machine using copy_to/from_user2() is a bit different:
OK, I need Hugh's help on this one.
First of all, I've modified your copypage.c to be correct. You need a flush_kernel_dcache_page() within the kmap/kunmap (not flush_dcache_page).
However, the problem appears to be that the user pointer is coming from malloc'd memory, which is of type PAGE_MAPPING_ANON. This flag ensures that page_mapping(page) returns NULL and thus that flush_dcache_page() on parisc never actually does any user space flushes.
The reason you see nothing when you cat /proc/fs/test is that these PAGE_MAPPING_ANON pages for malloc() have a nice trick where the clear_user_highpage() function blasts a string of zeros over them via the cache in user space, so they're cache incoherent; you can modify the underlying page as much as you like, but the data the user sees is the zeros until the page is flushed. This is the essence of the problem:flush_dcache_page() isn't bringing these user pages back into coherency.
Just as proof of this, if you uncomment the asm in my_copy_to_user2, you'll see everything work correctly, because this asm statement forces coherency on the user page.
So, Hugh, you were the one who introduced this anonymous page behaviour; how are we supposed to obtain coherency for these pages?
James
#include <linux/fs.h>
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/pagemap.h>
#include <asm/uaccess.h>
MODULE_LICENSE("GPL");
static int my_copy_to_user1(char __user *dst, const char *src, size_t size)
{
return copy_to_user(dst, src, size) ? -EFAULT : 0;
}static int my_copy_from_user1(char *dst, const char __user *src, size_t size)
{
return copy_from_user(dst, src, size) ? -EFAULT : 0;
}static int my_copy_to_user2(char __user *dst, const char *src, size_t size)
{
int err;
struct page *page;
unsigned long addr = (unsigned long) dst;
unsigned offset = addr % PAGE_SIZE;
void *mapaddr;
void *ptr;
if (offset + size > PAGE_SIZE)
return -EINVAL;
down_read(¤t->mm->mmap_sem);
err = get_user_pages(current, current->mm, addr, 1, 1, 0, &page, NULL);
up_read(¤t->mm->mmap_sem);
if (err < 0)
return err;
//__asm__ __volatile("\tfdc\t0(%%sr3, %0)\n" : : "r" (addr) : "memory");
BUG_ON(err != 1);
mapaddr = kmap_atomic(page, KM_USER0);
ptr = mapaddr + offset;
printk("dst: %p page: %p offset: %u mapaddr: %p ptr: %p\n",
dst, page, offset, mapaddr, ptr);
printk("mapping %p mapping_mapped %d\n", page->mapping,
page->mapping ? mapping_mapped(page->mapping): 0);
memcpy(ptr, src, size);
flush_kernel_dcache_page(mapaddr);
kunmap_atomic(mapaddr, KM_USER0);
set_page_dirty_lock(page);
page_cache_release(page);
return 0;
}static int my_copy_from_user2(char *dst, const char __user *src, size_t size)
{
int err;
struct page *page;
unsigned long addr = (unsigned long) src;
unsigned offset = addr % PAGE_SIZE;
void *mapaddr;
void *ptr;
if (offset + size > PAGE_SIZE)
return -EINVAL;
down_read(¤t->mm->mmap_sem);
err = get_user_pages(current, current->mm, addr, 1, 0, 0, &page, NULL);
up_read(¤t->mm->mmap_sem);
if (err < 0)
return err;
BUG_ON(err != 1);
mapaddr = kmap_atomic(page, KM_USER0);
ptr = mapaddr + offset;
printk("src: %p page: %p offset: %u mapaddr: %p ptr: %p\n",
src, page, offset, mapaddr, ptr);
memcpy(dst, ptr, size);
kunmap_atomic(mapaddr, KM_USER0);
put_page(page);
return 0;
}static ssize_t test_read(struct file *file, char __user *buf, size_t nbytes,
loff_t *off)
{
ssize_t res = 0;
char hello[] = "Hello!\n";
size_t hellolen = strlen(hello);
printk("test_read %p %lu %llu\n", buf, (unsigned long) nbytes,
(unsigned long long) *off);
if (!*off && nbytes > hellolen) {
res = my_copy_to_user2(buf, hello, hellolen);
if (!res) {
*off += hellolen;
res = hellolen;
}
}
printk(" test_read: %li\n", (long) res);
return res;
}static ssize_t test_write(struct file *file, const char __user *buf,
size_t nbytes, loff_t *off)
{
ssize_t res = nbytes;
char hello[32];
printk("test_write %p %lu %llu\n", buf, (unsigned long) nbytes,
(unsigned long long) *off);
if (!*off && nbytes < sizeof(hello) - 1) {
res = my_copy_from_user2(hello, buf, nbytes);
if (!res) {
hello[nbytes] = '\0';
printk("<%s>\n", hello);
*off += nbytes;
res = nbytes;
}
}
printk(" test_write: %li\n", (long) res);
return res;
}static struct file_operations test_ops = {
.owner = THIS_MODULE,
.read = test_read,
.write = test_write,
};static int test_init(void)
{
struct proc_dir_entry *ent;
ent = create_proc_entry("test", S_IFREG | 0666, proc_root_fs);
if (ent)
ent->proc_fops = &test_ops;
return 0;
}static void test_exit(void)
{
remove_proc_entry("test", proc_root_fs);
}module_init(test_init);module_exit(test_exit);
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 14 lines · James Bottomley
However, the problem appears to be that the user pointer is coming from malloc'd memory, which is of type
PAGE_MAPPING_ANON. This flag ensures thatpage_mapping(page)returns NULL and thus thatflush_dcache_page()on parisc never actually does any user space flushes.The reason you see nothing when you cat
/proc/fs/testis that thesePAGE_MAPPING_ANONpages formalloc()have a nice trick where theclear_user_highpage()function blasts a string of zeros over them via the cache in user space, so they're cache incoherent; you can modify the underlying page as much as you like, but the data the user sees is the zeros until the page is flushed. This is the essence of the problem:flush_dcache_page()isn't bringing these user pages back into coherency.
Quoted 6 lines
Just as proof of this, if you uncomment the asm in
my_copy_to_user2, you'll see everything work correctly, because this asm statement forces coherency on the user page.So, Hugh, you were the one who introduced this anonymous page behaviour; how are we supposed to obtain coherency for these pages?
I don't think I changed any behaviour here, just implemention details (what you might expect to find stored in "page->mapping" in different circumstances). Comparing current parisc source against 2.6.5 source (several releases before PAGE_MAPPING_ANON) bears that out, doesn't it? page->mapping on anonymous pages at that time would have been NULL (or &swapper_space if the page happened to have some swap assigned to it).
Isn't the point simply that anonymous pages are never writably shared, and so don't require the coherency measures that shared writable pages do? Anonymous pages may be shared readonly between children and parent of fork (and any modification requires Copy-On-Write at either end). There are MAP_SHARED|MAP_ANONYMOUS pages, but those are implemented like shared memory, as pages of an underlying tmpfs object (to which page->mapping will point as for a normal file).
Whereas now Chris is working on a /proc file which gives access to anonymous pages by a backdoor? That's new territory, and may indeed need something special such as you appended. It might be something very useful for local use, but I doubt it would be accepted into a mainline kernel.
Hugh
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Hugh Dickins
Quoted 2 lines
So, Hugh, you were the one who introduced this anonymous page behaviour; how are we supposed to obtain coherency for these pages?
I don't think I changed any behaviour here, just implemention details (what you might expect to find stored in "
page->mapping" in different circumstances). Comparing current parisc source against 2.6.5 source (several releases beforePAGE_MAPPING_ANON) bears that out, doesn't it?page->mappingon anonymous pages at that time would have been NULL (or&swapper_spaceif the page happened to have some swap assigned to it).
OK ... I just remember you introducing the page_mapping() macro ...
Quoted 7 lines
Isn't the point simply that anonymous pages are never writably shared, and so don't require the coherency measures that shared writable pages do? Anonymous pages may be shared readonly between children and parent of fork (and any modification requires Copy-On-Write at either end). There are MAP_SHARED|MAP_ANONYMOUS pages, but those are implemented like shared memory, as pages of an underlying tmpfs object (to which
page->mappingwill point as for a normal file).
Yes, this is precisely the point. While it is true that anonymous pages shouldn't be writeably shared and therefore no coherence measures should be necessary between user space processes, anonymous pages are still not congruently mapped in our kernel, so we have to take coherency measures when trying to view them from the kernel.
Quoted 5 lines
Whereas now Chris is working on a /proc file which gives access to anonymous pages by a backdoor? That's new territory, and may indeed need something special such as you appended. It might be something very useful for local use, but I doubt it would be accepted into a mainline kernel.
Well, yes, I think fuse is guilty to some degree. However, the same situation would prevail if anonymous memory were sent down to our other kernel<->user mapping routines, like SG_IO for instance. Perhaps we should be doing early cow breaking for anonymous pages we know we're going to write to?
James
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 6 lines · James Bottomley
Well, yes, I think fuse is guilty to some degree. However, the same situation would prevail if anonymous memory were sent down to our other kernel<->user mapping routines, like
SG_IOfor instance. Perhaps we should be doing early cow breaking for anonymous pages we know we're going to write to?
Certainly, that is part of what copy_to_user and get_user_pages(,,,,1,,,) (and ordinary userspace writing) guarantee. So if there's some new route by which userspace pages may be modified on behalf of the user, never mind the architecture-specific cache coherency issues, COWs must be broken to preserve the semantics of fork.
drivers/scsi/sg.c looks like it makes all the necessary calls to __copy_to_user or get_user_pages... but it's not hard to fool me.
Hugh
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 5 lines · Hugh Dickins
Certainly, that is part of what
copy_to_userandget_user_pages(,,,,1,,,)(and ordinary userspace writing) guarantee. So if there's some new route by which userspace pages may be modified on behalf of the user, never mind the architecture-specific cache coherency issues, COWs must be broken to preserve the semantics of fork.
Yes, that's where I think the problem lies. For an anonymous unshared page, the user view of the cache may still be dirty. get_user_pages() only seems to use flush_dcache_page() to restore coherency, which won't work on anon pages. Thus, get_user_pages() for an anon page is returning a page to the kernel that still has an incoherent user cache, which is what causes the issues in the copytest module.
Quoted 2 lines
drivers/scsi/sg.clooks like it makes all the necessary calls to__copy_to_userorget_user_pages... but it's not hard to fool me.
Yes ... and the copytest module has an almost exact duplicate of that code ... it's the actual get_user_pages() where I think we have coherency issues ... I've verified that it is returning an incoherent page if the underlying user page is PAGE_MAPPING_ANON.
James
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 14 lines · James Bottomley
Quoted 5 lines · Hugh Dickins
Certainly, that is part of what
copy_to_userandget_user_pages(,,,,1,,,)(and ordinary userspace writing) guarantee. So if there's some new route by which userspace pages may be modified on behalf of the user, never mind the architecture-specific cache coherency issues, COWs must be broken to preserve the semantics of fork.Yes, that's where I think the problem lies. For an anonymous unshared page, the user view of the cache may still be dirty.
get_user_pages()only seems to useflush_dcache_page()to restore coherency, which won't work on anon pages. Thus,get_user_pages()for an anon page is returning a page to the kernel that still has an incoherent user cache, which is what causes the issues in the copytest module.
Quoted 7 lines
Quoted 2 lines
drivers/scsi/sg.clooks like it makes all the necessary calls to__copy_to_userorget_user_pages... but it's not hard to fool me.Yes ... and the copytest module has an almost exact duplicate of that code ... it's the actual
get_user_pages()where I think we have coherency issues ... I've verified that it is returning an incoherent page if the underlying user page isPAGE_MAPPING_ANON.
It sounds like an issue you need to raise on linux-arch? I've no idea whether it's the parisc implementation of flush_dcache_page which is at fault (skipping anonymous pages), or that get_user_pages ought to be calling something stronger (something new?). Remember, I'm one of those x86 people with only a nursery-school grasp of coherency issues.
I do see that when flush_dcache_page was added into get_user_pages in 2.5.46, the parisc flush_dcache_page looked to be doing something else, something which would have caught the anonymous pages. And I just looked up the ChangeLog-2.5.46 entry, below: akpm and davem CC'ed.
<[email protected]>
[PATCH] flush_dcache_page in get_user_pages()
Davem said:
"Ho hum, it is tricky :-)))
At bio_map_user() you need to see the user's most recent write to the page if you are going "user --> device". So if "user --> device" bio_map_user() must flush_dcache_page().
I find the write_to_vm condition confusion which is probably why I am sitting here spelling this out :-)
At bio_unmap_user(), if we are going "device --> user" you have to flush_dcache_page(). And actually, this flush could just as legitimately occur at bio_map_user() time.
Therefore, the easiest thing to do is always flush_dcache_page() at bio_map_user().
All the other cases are going to be like this, so we might as well cut to the chase and flush_dcache_page() for all the pages inside of get_user_pages()."
_______________________________________________
parisc-linux mailing list [email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 1 line · Hugh Dickins
It sounds like an issue you need to raise on linux-arch? I've no idea
Yes ... I think so. I think the issue is that get_user_pages() has to guarantee that every page it gets has no dirty user cache above it.
Quoted 4 lines
whether it's the parisc implementation of
flush_dcache_pagewhich is at fault (skipping anonymous pages), or thatget_user_pagesought to be calling something stronger (something new?). Remember, I'm one of those x86 people with only a nursery-school grasp of coherency issues.
It seems to me that flush_dcache_page() should sweep up anonymous pages as well ... as long as we have this pre-emptive coherency model where we only call flush_dcache_page() once on entry so that as long as we exit to user level having modified the underlying page memory (and the user hasn't done anything to bring the page into their cache) then everything works.
Quoted 4 lines
I do see that when
flush_dcache_pagewas added intoget_user_pagesin 2.5.46, the pariscflush_dcache_pagelooked to be doing something else, something which would have caught the anonymous pages. And I just looked up the ChangeLog-2.5.46 entry, below: akpm and davem CC'ed.
Yes, I suspect our implementation should just sweep up anon pages again ... However, I'll also compose a mailing for linux-arch just to be sure we're all on the same page (no pun intended).
James
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Miklos Szeredi
maintainerQuoted 3 lines
Yes, I suspect our implementation should just sweep up anon pages again ... However, I'll also compose a mailing for linux-arch just to be sure we're all on the same page (no pun intended).
Hi James,
I've seen you submitted the needed API changes on lkml. Do you also have the parisc implementation for Chris to try out?
Thanks,
Miklos
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 5 lines · Miklos Szeredi
Hi James,
I've seen you submitted the needed API changes on lkml. Do you also have the parisc implementation for Chris to try out?
He committed it to cvs... I should have it divvied up and sent to Linus by the weekend. In the meantime, try this patch against 2.6.16 (which is the parisc patchset)
http://cvs.parisc-linux.org/download/linux-2.6/patch-2.6.16-pa8.gz
Cheers,
Kyle
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux