PAGE_RW Should be added to PAGE_COPY ?
The current kernel set PAGE_COPY without write bit. This will cause intermittent non-cosistent data for user-level network drivers such as Infiniband, Quadrics and Myrinet. Which has also be mentioned by Costin Iancu in the paper "HUNTing the Overlap " (PACT'05).
An example of such phenomena is the following sequences:
register a memory space BUFF for receive message,
receive message,
call mprotect(...PROT_NONE...) and mprotect(...PROT_READ|PROT_WRITE) one by one,
write into BUFF,
then receive again.
The second time received data will perhaps not be the data sent by the peer machine but the data written by itself in the 4th step.
The reson is that :
1) User-level network driver locks phy pages when memory space is registered;
2) 2 calls to mprotect change ptes in the space to PAGE_COPY, so write any page in the space will cause a page fault;
3) In the page fault handler, it goes to do_wp_page, and in it if Page Is Locked, a new page is generated and filled into the pte. So the physical page seen by the host is not the same one by the NIC.
Adding PAGE_RW to PAGE_COPY will resolve this problem.
In my option, the reason for absense of RW is to save memory by mapping all those only read pages into ZERO_PAGE. But is there really programs which make many read-ops in memory space without even initialize them?
___________________________________________________
_ Yingchao Zhou _ _ ICT, CAS _ _ (86)010-62601009 _
___________________________________________________
Quoted 9 lines · Yingchao Zhou
The current kernel set
PAGE_COPYwithout write bit. This will cause intermittent non-cosistent data for user-level network drivers such as Infiniband, Quadrics and Myrinet. Which has also be mentioned by Costin Iancu in the paper "HUNTing the Overlap " (PACT'05).
An example of such phenomena is the following sequences:
register a memory space BUFF for receive message,
receive message,
callmprotect(...PROT_NONE...)andmprotect(...PROT_READ|PROT_WRITE)one by one,
write into BUFF,
then receive again.
The second time received data will perhaps not be the data sent by the peer machine but the data written by itself in the 4th step.
PAGE_COPY (without the write bit) is used when the area was mmap'edMAP_PRIVATE: which indeed is asking for private copies of pages to be made - which will be left containing the data written there by the application, rather than shared data received later by the driver.
You want to mmap MAP_SHARED, which will use PAGE_SHARED instead, including the write bit, both before and after the mprotects. There should be no problem then: do you actually see a problem when MAP_SHARED is used?
(You don't mention which release you're describing, and some of the details may vary: the not-yet-started 2.6.19 is likely to use PAGE_COPY even when MAP_SHARED, to help it keep track of the number of dirty pages; but in that case, do_wp_page() won't make a copy.)
Quoted 4 lines
The reson is that :
1) User-level network driver locks phy pages when memory space is registered;
2) 2 calls to mprotect change ptes in the space toPAGE_COPY, so write any page in the space will cause a page fault;
Not if PROT_WRITE, MAP_SHARED I think.
Quoted 1 line
3) In the page fault handler, it goes to
do_wp_page, and in it if Page Is Locked, a new page is generated and filled into the pte. So the physical page seen by the host is not the same one by the NIC.
When MAP_PRIVATE, it's not the page being locked that causes the copy (it's not normally locked there, is it?), it's that it's not PageAnon; or if you're looking at 2.6.12 or older, that page_count is raised.
Quoted 2 lines
Adding
PAGE_RWtoPAGE_COPYwill resolve this problem.
No! That would give every user write access to shared files they should have no write access to.
Quoted 1 line
In my option, the reason for absense of RW is to save memory by mapping all those only read pages into
ZERO_PAGE. But is there really programs which make many read-ops in memory space without even initialize them?
Not just the ZERO_PAGE: initial program data is another common example.
Hugh