memcg: add support for hwpoison testing

Hi all,

In hardware poison testing, we want to inject hwpoison errors to pages of a collection of selected tasks, so that random tasks (eg. init) won't be killed in stress tests and lead to test failure.

Memory cgroup provides an ideal tool for tracking and testing these target process pages. All we have to do is to
- export the memory cgroup id via cgroupfs
- export two functions/structs for hwpoison_inject.c

This might be an unexpected usage of memory cgroup. The last patch and this script demonstrates how the exported interfaces are to be used to limit the scope of hwpoison injection.

	test -d /cgroup/hwpoison && rmdir /cgroup/hwpoison
	mkdir /cgroup/hwpoison

	usemem -m 100 -s 100 &   # eat 100MB and sleep 100s
	echo `pidof usemem` > /cgroup/hwpoison/tasks

==> memcg_id=$(</cgroup/hwpoison/memory.id)
==> echo $memcg_id > /debug/hwpoison/corrupt-filter-memcg

	# hwpoison all pfn
	pfn=0
	while true
	do      
		let pfn=pfn+1
		echo $pfn > /debug/hwpoison/corrupt-pfn
		if [ $? -ne 0 ]; then
			break
		fi
	done

Comments are welcome, thanks!

Cheers,
Fengguang

Quoted 35 lines · Wu Fengguang

Hi all,

In hardware poison testing, we want to inject hwpoison errors to pages of a collection of selected tasks, so that random tasks (eg. init) won't be killed in stress tests and lead to test failure.

Memory cgroup provides an ideal tool for tracking and testing these target process pages. All we have to do is to
- export the memory cgroup id via cgroupfs
- export two functions/structs for hwpoison_inject.c

This might be an unexpected usage of memory cgroup. The last patch and this script demonstrates how the exported interfaces are to be used to limit the scope of hwpoison injection.

	test -d /cgroup/hwpoison && rmdir /cgroup/hwpoison
	mkdir /cgroup/hwpoison

	usemem -m 100 -s 100 &   # eat 100MB and sleep 100s
	echo `pidof usemem` > /cgroup/hwpoison/tasks

==> memcg_id=$(</cgroup/hwpoison/memory.id)
==> echo $memcg_id > /debug/hwpoison/corrupt-filter-memcg

	# hwpoison all pfn
	pfn=0
	while true
	do      
		let pfn=pfn+1
		echo $pfn > /debug/hwpoison/corrupt-pfn
		if [ $? -ne 0 ]; then
			break
		fi
	done

Quoted 2 lines

Comments are welcome, thanks!

I took a quick look and the patches seem OKAY to me, but I have question, can't we do all of this from user space? The bits about id export and import the ids look like they can be replaced by names in user space.

Hi all,

In hardware poison testing, we want to inject hwpoison errors to pages of a collection of selected tasks, so that random tasks (eg. init) won't be killed in stress tests and lead to test failure.

Memory cgroup provides an ideal tool for tracking and testing these target process pages. All we have to do is to
- export the memory cgroup id via cgroupfs
- export two functions/structs for hwpoison_inject.c

This might be an unexpected usage of memory cgroup. The last patch and this script demonstrates how the exported interfaces are to be used to limit the scope of hwpoison injection.

	test -d /cgroup/hwpoison && rmdir /cgroup/hwpoison
	mkdir /cgroup/hwpoison

	usemem -m 100 -s 100 &   # eat 100MB and sleep 100s
	echo `pidof usemem` > /cgroup/hwpoison/tasks

==> memcg_id=$(</cgroup/hwpoison/memory.id)
==> echo $memcg_id > /debug/hwpoison/corrupt-filter-memcg

	# hwpoison all pfn
	pfn=0
	while true
	do      
		let pfn=pfn+1
		echo $pfn > /debug/hwpoison/corrupt-pfn
		if [ $? -ne 0 ]; then
			break
		fi
	done

Comments are welcome, thanks!

Cheers,
Fengguang

Signature · 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Quoted 24 lines · On Mon, 31 Aug 2009 18:26:40 +0800 Wu Fengguang <[email protected]> wrote:

Hi all,

In hardware poison testing, we want to inject hwpoison errors to pages of a collection of selected tasks, so that random tasks (eg. init) won't be killed in stress tests and lead to test failure.

Memory cgroup provides an ideal tool for tracking and testing these target process pages. All we have to do is to
- export the memory cgroup id via cgroupfs
- export two functions/structs for hwpoison_inject.c

This might be an unexpected usage of memory cgroup. The last patch and this script demonstrates how the exported interfaces are to be used to limit the scope of hwpoison injection.

	test -d /cgroup/hwpoison && rmdir /cgroup/hwpoison
	mkdir /cgroup/hwpoison

	usemem -m 100 -s 100 &   # eat 100MB and sleep 100s
	echo `pidof usemem` > /cgroup/hwpoison/tasks

==> memcg_id=$(</cgroup/hwpoison/memory.id)
==> echo $memcg_id > /debug/hwpoison/corrupt-filter-memcg

Quoted 11 lines

	# hwpoison all pfn
	pfn=0
	while true
	do      
		let pfn=pfn+1
		echo $pfn > /debug/hwpoison/corrupt-pfn
		if [ $? -ne 0 ]; then
			break
		fi
	done

I don't like this.

1. plz put all under CONFIG_DEBUG_HWPOISON or some
2. plz don't export memcg's id. you can do it without it.

3. If I was you, just adds following file

memory.hwpoison_test
Then, if you allow test
#echo 1 > memory.hwpoison_test

4. I can't understand why you need this. I wonder you can get pfn via /proc/<pid>/????. And this may insert HWPOISON to page-cache of shared library and "unexpected" process will be poisoned.

Quoted 3 lines

4. I can't understand why you need this. I wonder you can get pfn via /proc/<pid>/????. And this may insert HWPOISON to page-cache of shared library and "unexpected" process will be poisoned.

Sorry I should have explained this. It's mainly for correctness. When a user space tool queries the task PFNs in /proc/pid/pagemap and then send to /debug/hwpoison/corrupt-pfn, there is a racy window that the page could be reclaimed and allocated by some one else. It would be awkward to try to pin the pages in user space. So we need the guarantees provided by /debug/hwpoison/corrupt-filter-memcg, which will be checked inside the page lock with elevated reference count.

Quoted 4 lines · On Tue, 1 Sep 2009 10:25:14 +0800 Wu Fengguang <[email protected]> wrote:

Quoted 3 lines

4. I can't understand why you need this. I wonder you can get pfn via /proc/<pid>/????. And this may insert HWPOISON to page-cache of shared library and "unexpected" process will be poisoned.

Quoted 8 lines

Sorry I should have explained this. It's mainly for correctness. When a user space tool queries the task PFNs in /proc/pid/pagemap and then send to /debug/hwpoison/corrupt-pfn, there is a racy window that the page could be reclaimed and allocated by some one else. It would be awkward to try to pin the pages in user space. So we need the guarantees provided by /debug/hwpoison/corrupt-filter-memcg, which will be checked inside the page lock with elevated reference count.

memcg never holds refcnt for a page and the kernel::vmscan.c can reclaim any pages under memcg whithout checking anything related to memcg. *And*, your code has no "pin" code.
This patch sed does no jobs for your concern.

I recommend you to add
/debug/hwpoizon/pin-pfn

Then,
echo pfn > /debug/hwpoizon/pin-pfn
# add pfn for hwpoison debug's watch list. and elevate refcnt check 'pfn' is still used.
echo pfn > /debug/hwpoison/corrupt-pfn

	# check 'watch list' and make it corrupt and release refcnt.
or some.

Quoted 15 lines · KAMEZAWA Hiroyuki

Quoted 4 lines · On Tue, 1 Sep 2009 10:25:14 +0800 Wu Fengguang <[email protected]> wrote:

Quoted 3 lines

4. I can't understand why you need this. I wonder you can get pfn via /proc/<pid>/????. And this may insert HWPOISON to page-cache of shared library and "unexpected" process will be poisoned.

Quoted 8 lines

Sorry I should have explained this. It's mainly for correctness. When a user space tool queries the task PFNs in /proc/pid/pagemap and then send to /debug/hwpoison/corrupt-pfn, there is a racy window that the page could be reclaimed and allocated by some one else. It would be awkward to try to pin the pages in user space. So we need the guarantees provided by /debug/hwpoison/corrupt-filter-memcg, which will be checked inside the page lock with elevated reference count.

Quoted 4 lines

memcg never holds refcnt for a page and the kernel::vmscan.c can reclaim any pages under memcg whithout checking anything related to memcg. *And*, your code has no "pin" code.
This patch sed does no jobs for your concern.

We grabbed page here, which is not in the scope of this patchset:

        static int try_memory_failure(unsigned long pfn)
        {      
                struct page *p;
                int res = -EINVAL;

                if (!pfn_valid(pfn))
                        return res;

                p = pfn_to_page(pfn);
                if (!get_page_unless_zero(compound_head(p)))
                        return res;

                lock_page_nosync(compound_head(p));

                if (hwpoison_filter(p))
                        goto out;

                res = __memory_failure(pfn, 18,
                                       MEMORY_FAILURE_FLAG_COUNTED |
                                       MEMORY_FAILURE_FLAG_LOCKED);
        out:
                unlock_page(p);
                return res;
        }

Quoted 10 lines

I recommend you to add
/debug/hwpoizon/pin-pfn

Then,
echo pfn > /debug/hwpoizon/pin-pfn
# add pfn for hwpoison debug's watch list. and elevate refcnt check 'pfn' is still used.
echo pfn > /debug/hwpoison/corrupt-pfn

	# check 'watch list' and make it corrupt and release refcnt.
or some.

Looks like a good alternative. At least no more memcg dependency..

Quoted 40 lines · On Tue, 1 Sep 2009 14:46:52 +0800 Wu Fengguang <[email protected]> wrote:

Quoted 15 lines · KAMEZAWA Hiroyuki

Quoted 4 lines · On Tue, 1 Sep 2009 10:25:14 +0800 Wu Fengguang <[email protected]> wrote:

Quoted 3 lines

4. I can't understand why you need this. I wonder you can get pfn via /proc/<pid>/????. And this may insert HWPOISON to page-cache of shared library and "unexpected" process will be poisoned.

Quoted 8 lines

Sorry I should have explained this. It's mainly for correctness. When a user space tool queries the task PFNs in /proc/pid/pagemap and then send to /debug/hwpoison/corrupt-pfn, there is a racy window that the page could be reclaimed and allocated by some one else. It would be awkward to try to pin the pages in user space. So we need the guarantees provided by /debug/hwpoison/corrupt-filter-memcg, which will be checked inside the page lock with elevated reference count.

Quoted 4 lines

memcg never holds refcnt for a page and the kernel::vmscan.c can reclaim any pages under memcg whithout checking anything related to memcg. *And*, your code has no "pin" code.
This patch sed does no jobs for your concern.

We grabbed page here, which is not in the scope of this patchset:

        static int try_memory_failure(unsigned long pfn)
        {      
                struct page *p;
                int res = -EINVAL;

                if (!pfn_valid(pfn))
                        return res;

                p = pfn_to_page(pfn);
                if (!get_page_unless_zero(compound_head(p)))
                        return res;

                lock_page_nosync(compound_head(p));

                if (hwpoison_filter(p))
                        goto out;

Quoted 7 lines

                res = __memory_failure(pfn, 18,
                                       MEMORY_FAILURE_FLAG_COUNTED |
                                       MEMORY_FAILURE_FLAG_LOCKED);
        out:
                unlock_page(p);
                return res;
        }

Hmm. maybe off-topic but why lock_page() is necessary ?

Quoted 12 lines

Quoted 11 lines

I recommend you to add
/debug/hwpoizon/pin-pfn

Then,
echo pfn > /debug/hwpoizon/pin-pfn
# add pfn for hwpoison debug's watch list. and elevate refcnt check 'pfn' is still used.
echo pfn > /debug/hwpoison/corrupt-pfn

	# check 'watch list' and make it corrupt and release refcnt.
or some.

Quoted 2 lines

Looks like a good alternative. At least no more memcg dependency..

My point is that memcg can show 'owner' of pages but the page may be shared with something important task _and_ if a task is migrated, its pages' memcg information is not updated now. Then, you can kill a task which is not in memcg.

Then, I don't recommend to use memcg. I think you'll see too much pitfalls.

Quoted 4 lines

My point is that memcg can show 'owner' of pages but the page may be shared with something important task _and_ if a task is migrated, its pages' memcg information is not updated now. Then, you can kill a task which is not in memcg.

Ah thanks! I'm not aware of that tricky fact, and it does make a very good reason not to use memcg, although I guess locked page won't be migrated.

Quoted 5 lines · Wu Fengguang

Quoted 4 lines

My point is that memcg can show 'owner' of pages but the page may be shared with something important task _and_ if a task is migrated, its pages' memcg information is not updated now. Then, you can kill a task which is not in memcg.

Quoted 4 lines

Ah thanks! I'm not aware of that tricky fact, and it does make a very good reason not to use memcg, although I guess locked page won't be migrated.

I think what Kamezawa-San is pointing to is that the task can migrate, leaving behind the page in the memcg and poisioning those pages can kill a task outside the memcg.