Skip to contentSkip to conversation

fscache fixes for ceph

PATCH New Milosz Tanski 1 participant 3 messages Sep 3, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover fscache fixes for ceph

On the whole the current iteration of FSCache on Ceph has been quite stable for me. We've had it deployed on 4 clients running for a week in a row. I did run into two small issues.

First one, I wasn't checking if the page was marked with private_2 before asking FSCache to write to it. I've never seen this before and I only seen this issue once.

The second issue is similar to what I was seeing before with pages marked private_2 when we return them to the page cache. In this case the start_read() in addr.c was returning the pages in the error path of the code. I have plugged that issue. I only seen this one a handful of times across all clients.

As usual please apply the changes from repo located at:

https://bitbucket.org/adfin/linux-fs.git branch: wip-ceph-fscache

Cheers,
- Milosz

Milosz Tanski (2):
ceph: ceph_readpage_to_fscache didn't check if marked
ceph: page still marked private_2

 fs/ceph/addr.c  |  1 +
 fs/ceph/cache.c |  3 +++
 fs/ceph/cache.h | 14 +++++++++++++-
 3 files changed, 17 insertions(+), 1 deletion(-)
Signature · 1 line
1.8.1.2
#1 lore raw

PATCH 1/2 ceph: ceph_readpage_to_fscache didn't check if marked

Previously ceph_readpage_to_fscache did not call if page was marked as cached before calling fscache_write_page resulting in a BUG inside of fscache.

FS-Cache: Assertion failed

------------[ cut here ]------------
kernel BUG at fs/fscache/page.c:874!
invalid opcode: 0000 [#1] SMP
Call Trace:
 [<ffffffffa02e6566>] __ceph_readpage_to_fscache+0x66/0x80 [ceph]
 [<ffffffffa02caf84>] readpage_nounlock+0x124/0x210 [ceph]
 [<ffffffffa02cb08d>] ceph_readpage+0x1d/0x40 [ceph]
 [<ffffffff81126db6>] generic_file_aio_read+0x1f6/0x700
 [<ffffffffa02c6fcc>] ceph_aio_read+0x5fc/0xab0 [ceph]
---
 fs/ceph/cache.c | 3 +++
 1 file changed, 3 insertions(+)
fs/ceph/cache.c +3 -0
index 5c413ec..c737ae9 100644
@@ -311,6 +311,9 @@ void ceph_readpage_to_fscache(struct inode *inode, struct page *page)
 	struct ceph_inode_info *ci = ceph_inode(inode);
 	int ret;
 
+	if (!PageFsCache(page))
+		return;
+
 	if (!cache_valid(ci))
 		return;
 
Signature · 1 line
1.8.1.2
#2 lore raw

PATCH 2/2 ceph: page still marked private_2

In reply to Milosz Tanski

Previous patch that allowed us to cleanup most of the issues with pages marked as private_2 when calling ceph_readpages. However, there seams to be a case in the error case clean up in start read that still trigers this from time to time. I've only seen this one a couple times.

BUG: Bad page state in process petabucket  pfn:335b82
page:ffffea000cd6e080 count:0 mapcount:0 mapping:          (null) index:0x0
page flags: 0x200000000001000(private_2)
Call Trace:
 [<ffffffff81563442>] dump_stack+0x46/0x58
 [<ffffffff8112c7f7>] bad_page+0xc7/0x120
 [<ffffffff8112cd9e>] free_pages_prepare+0x10e/0x120
 [<ffffffff8112e580>] free_hot_cold_page+0x40/0x160
 [<ffffffff81132427>] __put_single_page+0x27/0x30
 [<ffffffff81132d95>] put_page+0x25/0x40
 [<ffffffffa02cb409>] ceph_readpages+0x2e9/0x6f0 [ceph]
 [<ffffffff811313cf>] __do_page_cache_readahead+0x1af/0x260
---
 fs/ceph/addr.c  |  1 +
 fs/ceph/cache.h | 14 +++++++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)
fs/ceph/addr.c +1 -0
index 1fda9cf..6df8bd4 100644
@@ -336,6 +336,7 @@ static int start_read(struct inode *inode, struct list_head *page_list, int max)
 		     page->index);
 		if (add_to_page_cache_lru(page, &inode->i_data, page->index,
 					  GFP_NOFS)) {
+			ceph_fscache_uncache_page(inode, page);
 			page_cache_release(page);
 			dout("start_read %p add_to_page_cache failed %p\n",
 			     inode, page);
fs/ceph/cache.h +13 -1
index fb326fd..bf48695 100644
@@ -51,6 +51,13 @@ static inline void ceph_fscache_invalidate(struct inode *inode)
 	fscache_invalidate(ceph_inode(inode)->fscache);
 }
 
+static inline void ceph_fscache_uncache_page(struct inode *inode,
+					     struct page *page)
+{
+	struct ceph_inode_info *ci = ceph_inode(inode);
+	return fscache_uncache_page(ci->fscache, page);
+}
+
 static inline int ceph_release_fscache_page(struct page *page, gfp_t gfp)
 {
 	struct inode* inode = page->mapping->host;
@@ -94,7 +101,8 @@ static inline void ceph_fscache_register_inode_cookie(struct ceph_fs_client* par
 {
 }
 
-static inline void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info* ci)
+static inline void ceph_fscache_uncache_page(struct inode *inode,
+					     struct page *pages)
 {
 }
 
@@ -126,6 +134,10 @@ static inline void ceph_invalidate_fscache_page(struct inode *inode,
 {
 }
 
+static inline void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info* ci)
+{
+}
+
 static inline int ceph_release_fscache_page(struct page *page, gfp_t gfp)
 {
 	return 1;
Signature · 1 line
1.8.1.2