Latest AIO patchset
The latest set of AIO patches are being maintained at:
http://www.kernel.org/pub/linux/kernel/people/suparna/aio/
The patches have been reduced to a minimal set that addresses the most relevant blocking points. Please let me know if you think there is a case for bringing in any of the additional patches.
(The patches need to be applied in the order mention in the 'series' file)
A new addition to the patchset is Chris Mason's nice and simple implementation of AIO support for pipes using the retry infrastructure. He also fixed some problems in the AIO cancel logic to make it play well with retries.
Besides this and some re-organization and cleaning up, there are a couple of changes since the last set of patches in -mm, that are worth a mention:
- Upfront readahead is now clipped to the readahead limit for the device (ra_pages) and happens only for AIO. This helps address the sendfile regression seen by Felix von Leitner. If your AIO read requests are likely to exceed the default readahead size, then use hdparm -a <new size> to tune it. The patchset also currently includes Ram Pai's adaptive lazy readahead code which is required for good streaming AIO read performance.
- David Brownell's suggestion of enabling the fops to set up their own retry methods. This should make his USB gadgetfs AIO patch co-exist smoothly with fsaio.
Some basic results are up comparing streaming non-cached random AIO read/write throughputs for a single ext3 file using aio-stress for various io sizes with and without these patches, and also comparsions with O_DIRECT AIO throughputs. The short summary has been a doubling of throughput using the fsaio patches, which is also close to the results seen with O_DIRECT AIO.
As usual feedback, bug fixes, test results etc are welcome.
Regards
Suparna
Signature · 3 lines
Suparna Bhattacharya ([email protected]) Linux Technology Center IBM Software Lab, India
kexec "problem"
hi guys,
i have just compiled a kernel with the kexec patch. compiled kexec-tools and when i try to load a kernel, it gives me this:
# ./do-kexec.sh /boot/bzImage-2.6.2-g kexec_load failed: Invalid argument entry = 0x91764 nr_segments = 2 segment[0].buf = 0x80b3480 segment[0].bufsz = 1880 segment[0].mem = 0x90000 segment[0].memsz = 1880 segment[1].buf = 0x40001008 segment[1].bufsz = 19795a segment[1].mem = 0x100000 segment[1].memsz = 19795a
anyone tried to run kexec and actually did it? i'm trying with kernel 2.6.3
Re: kexec "problem"
On Tue, 24 Feb 2004 11:02:21 -0000 (WET) Carlos Silva wrote:
| hi guys, | | i have just compiled a kernel with the kexec patch. compiled kexec-tools | and when i try to load a kernel, it gives me this: | # ./do-kexec.sh /boot/bzImage-2.6.2-g | kexec_load failed: Invalid argument | entry = 0x91764 | nr_segments = 2 | segment[0].buf = 0x80b3480 | segment[0].bufsz = 1880 | segment[0].mem = 0x90000 | segment[0].memsz = 1880 | segment[1].buf = 0x40001008 | segment[1].bufsz = 19795a | segment[1].mem = 0x100000 | segment[1].memsz = 19795a | | anyone tried to run kexec and actually did it? i'm trying with kernel 2.6.3
I haven't updated for 2.6.3 yet, or even tested it... but I'll get to it soon.
--
~Randy
Re: kexec "problem"
Quoted 20 lines · On Tue, 24 Feb 2004 11:02:21 -0000 (WET) Carlos Silva wrote:
| hi guys, | | i have just compiled a kernel with the kexec patch. compiled kexec-tools | and when i try to load a kernel, it gives me this: | # ./do-kexec.sh /boot/bzImage-2.6.2-g | kexec_load failed: Invalid argument | entry = 0x91764 | nr_segments = 2 | segment[0].buf = 0x80b3480 | segment[0].bufsz = 1880 | segment[0].mem = 0x90000 | segment[0].memsz = 1880 | segment[1].buf = 0x40001008 | segment[1].bufsz = 19795a | segment[1].mem = 0x100000 | segment[1].memsz = 19795a | | anyone tried to run kexec and actually did it? i'm trying with kernel
Quoted 8 lines
2.6.3
I haven't updated for 2.6.3 yet, or even tested it... but I'll get to it soon.
--
~Randy
just changed line 13 on kexec.h from:
"long kexec_load(void *entry, unsigned long nr_segments, struct kexec_segment *segments, unsigned long);"
to:
"long kexec_load(void *entry, unsigned long nr_segments, struct kexec_segment *segments, unsigned long flags);"
and it gives me this error:
# ./do-kexec.sh /boot/bzImage-2.6.2-g kexec_load failed: Function not implemented entry = 0x91764 nr_segments = 2 segment[0].buf = 0x80b3480 segment[0].bufsz = 1880 segment[0].mem = 0x90000 segment[0].memsz = 1880 segment[1].buf = 0x40001008 segment[1].bufsz = 19795a segment[1].mem = 0x100000 segment[1].memsz = 19795a
Re: kexec "problem" [and patch updates]
In reply to Carlos SilvaOn Tue, 24 Feb 2004 11:02:21 -0000 (WET) Carlos Silva wrote:
| hi guys, | | i have just compiled a kernel with the kexec patch. compiled kexec-tools | and when i try to load a kernel, it gives me this: | # ./do-kexec.sh /boot/bzImage-2.6.2-g | kexec_load failed: Invalid argument | entry = 0x91764 | nr_segments = 2 | segment[0].buf = 0x80b3480 | segment[0].bufsz = 1880 | segment[0].mem = 0x90000 | segment[0].memsz = 1880 | segment[1].buf = 0x40001008 | segment[1].bufsz = 19795a | segment[1].mem = 0x100000 | segment[1].memsz = 19795a | | anyone tried to run kexec and actually did it? i'm trying with kernel 2.6.3 | -
I updated the kexec patch for 2.6.2 and 2.6.3.
It works fine on 2.6.2. It works for me on 2.6.3 if not SMP. If the kernel is built for SMP, when running kexec, I get a BUG in arch/i386/kernel/smp.c at line 359.
I'm testing various workarounds for that BUG now.
--
~Randy
kexec updates are at:
http://developer.osdl.org/rddunlap/kexec/2.6.2/
and
http://developer.osdl.org/rddunlap/kexec/2.6.3/
Re: [Fastboot] Re: kexec "problem" [and patch updates]
Quoted 22 lines · Randy.Dunlap
On Tue, 24 Feb 2004 11:02:21 -0000 (WET) Carlos Silva wrote:
| hi guys, | | i have just compiled a kernel with the kexec patch. compiled kexec-tools | and when i try to load a kernel, it gives me this: | # ./do-kexec.sh /boot/bzImage-2.6.2-g | kexec_load failed: Invalid argument | entry = 0x91764 | nr_segments = 2 | segment[0].buf = 0x80b3480 | segment[0].bufsz = 1880 | segment[0].mem = 0x90000 | segment[0].memsz = 1880 | segment[1].buf = 0x40001008 | segment[1].bufsz = 19795a | segment[1].mem = 0x100000 | segment[1].memsz = 19795a | | anyone tried to run kexec and actually did it? i'm trying with kernel 2.6.3 | -
Quoted 5 lines
I updated the kexec patch for 2.6.2 and 2.6.3.
It works fine on 2.6.2. It works for me on 2.6.3 if not SMP. If the kernel is built for SMP, when running kexec, I get a BUG inarch/i386/kernel/smp.cat line 359.
I'm testing various workarounds for that BUG now.
I will eyeball it...
Is it the kernel that is shutting down, or the kernel that is being brought up that has problems?
The back trace from the BUG would be interesting.
As I see it flush_tlb_others is being called when we have shutdown cpus and the kernel still thinks we have the mm present on foreign cpus.
So it appears we simply have a case that was not anticipated by the authors of that code. So we need to adjust either the code we are calling or cpu_vm_mask so it does not list other cpus after we have shut them down.
At least that it what it looks like at first glance.
Eric
Re: [Fastboot] Re: kexec "problem" [and patch updates]
On 27 Feb 2004 01:00:04 -0700 Eric W. Biederman wrote:
| "Randy.Dunlap" <[email protected]> writes: | | > On Tue, 24 Feb 2004 11:02:21 -0000 (WET) Carlos Silva wrote: | > | > | hi guys, | > | | > | i have just compiled a kernel with the kexec patch. compiled kexec-tools | > | and when i try to load a kernel, it gives me this: | > | # ./do-kexec.sh /boot/bzImage-2.6.2-g | > | kexec_load failed: Invalid argument | > | entry = 0x91764 | > | nr_segments = 2 | > | segment[0].buf = 0x80b3480 | > | segment[0].bufsz = 1880 | > | segment[0].mem = 0x90000 | > | segment[0].memsz = 1880 | > | segment[1].buf = 0x40001008 | > | segment[1].bufsz = 19795a | > | segment[1].mem = 0x100000 | > | segment[1].memsz = 19795a | > | | > | anyone tried to run kexec and actually did it? i'm trying with kernel 2.6.3 | > | - | > | > I updated the kexec patch for 2.6.2 and 2.6.3. | > It works fine on 2.6.2. It works for me on 2.6.3 if not SMP. | > If the kernel is built for SMP, when running kexec, I get a | > BUG in arch/i386/kernel/smp.c at line 359. | > I'm testing various workarounds for that BUG now. | | I will eyeball it... | | Is it the kernel that is shutting down, or the kernel that is being | brought up that has problems?
the kernel that is shutting down.
| The back trace from the BUG would be interesting.
see below. my bad. i should have included it.
| As I see it flush_tlb_others is being called when we have shutdown | cpus and the kernel still thinks we have the mm present on foreign | cpus.
Martin Bligh thinks that there is a tlb race here. I printed the 2 cpu masks on my dual-proc macine and saw 0 in one of them and 0xc in the other one.
| So it appears we simply have a case that was not anticipated | by the authors of that code. So we need to adjust either | the code we are calling or cpu_vm_mask so it does not list | other cpus after we have shut them down. | | At least that it what it looks like at first glance.
Thanks,
--
~Randy
Feb 25 15:52:21 gargoyle kernel: ------------[ cut here ]------------ Feb 25 15:52:21 gargoyle kernel: kernel BUG at arch/i386/kernel/smp.c:359! Feb 25 15:52:21 gargoyle kernel: invalid operand: 0000 [#1] Feb 25 15:52:21 gargoyle kernel: CPU: 1 Feb 25 15:52:21 gargoyle kernel: EIP: 0060:[<c011673d>] Not tainted Feb 25 15:52:21 gargoyle kernel: EFLAGS: 00010206 Feb 25 15:52:21 gargoyle kernel: EIP is at flush_tlb_others+0x141/0x15c Feb 25 15:52:21 gargoyle kernel: eax: 00000000 ebx: c043c9e0 ecx: c043c9e0 edx: 0000000c Feb 25 15:52:21 gargoyle kernel: esi: f53effc4 edi: 00851da8 ebp: f5449ebc esp: f5449ea8 Feb 25 15:52:21 gargoyle kernel: ds: 007b es: 007b ss: 0068 Feb 25 15:52:21 gargoyle kernel: Process kexec (pid: 1095, threadinfo=f5448000 task=f54719b0) Feb 25 15:52:21 gargoyle kernel: Stack: f5449ed4 c014eae3 c1851d58 353f0000 00000000 f5449ed4 c01167e9 0000000c Feb 25 15:52:21 gargoyle kernel: c043c9e0 ffffffff 0000000c f5449f20 c0150084 c043c9e0 f61d7610 003f1000 Feb 25 15:52:21 gargoyle kernel: 003f1000 35000000 35000000 00400000 c0101354 c043ca14 c043c9e0 353f1000 Feb 25 15:52:21 gargoyle kernel: Call Trace: Feb 25 15:52:21 gargoyle kernel: [<c014eae3>] pte_alloc_map+0xd9/0x12e Feb 25 15:52:21 gargoyle kernel: [<c01167e9>] flush_tlb_mm+0x47/0x8c Feb 25 15:52:21 gargoyle kernel: [<c0150084>] remap_page_range+0x1ae/0x218 Feb 25 15:52:21 gargoyle kernel: [<c013dae7>] identity_map_pages+0xf7/0x130 Feb 25 15:52:21 gargoyle kernel: [<c013dbd4>] kimage_alloc_reboot_code_pages+0xb4/0x164 Feb 25 15:52:21 gargoyle kernel: [<c013d928>] kimage_alloc+0x100/0x186 Feb 25 15:52:21 gargoyle kernel: [<c013e496>] sys_kexec_load+0x9c/0xff Feb 25 15:52:21 gargoyle kernel: [<c0109637>] syscall_call+0x7/0xb Feb 25 15:52:21 gargoyle kernel: Feb 25 15:52:21 gargoyle kernel: Code: 0f 0b 67 01 ee df 3d c0 e9 d7 fe ff ff 0f 0b 64 01 ee df 3d
Re: [Fastboot] Re: kexec "problem" [and patch updates]
Quoted 18 lines · Randy.Dunlap
On 27 Feb 2004 01:00:04 -0700 Eric W. Biederman wrote:
| > It works fine on 2.6.2. It works for me on 2.6.3 if not SMP. | > If the kernel is built for SMP, when running kexec, I get a | > BUG in arch/i386/kernel/smp.c at line 359. | > I'm testing various workarounds for that BUG now. | | I will eyeball it... | | Is it the kernel that is shutting down, or the kernel that is being | brought up that has problems?the kernel that is shutting down.
| The back trace from the BUG would be interesting.
see below. my bad. i should have included it.
Quoted 7 lines
| As I see it flush_tlb_others is being called when we have shutdown | cpus and the kernel still thinks we have the mm present on foreign | cpus.Martin Bligh thinks that there is a tlb race here. I printed the 2 cpu masks on my dual-proc macine and saw 0 in one of them and
0xcin the other one.
Ouch we have both cpus running when this happens, and we have not started any shutdown whatsoever. This is the bit that sets up the page tables for later use...
I think identity_map_pages will have problems with a kernel that does the 4G/4G split, and it has known issues on some other architectures, because they treat init_mm specially. So the proper solution may be to simply rewrite identity_map_pages.
Before we do that in the short term we need to see if identity_map_pages is actually doing anything bad. You are not using the 4G/4G split so that is not the cause. So either init_mm is now special in some way, or we have hit a generic kernel bug.
So this may indeed be a tlb race. But it is init_mm->cpu_vm_mask and cpu_online map that are different. With the implication being that init_mm->cpu_vm_mask has cpus set that are not in cpu_online_map? Very weird especially on SMP.
Without attribution I have a hard time making sense of which cpumask is which so I can't draw any conclusions. But I find it very interesting that it is bits 2 and 3 that are set. I wonder if there is any mixup between logical cpu identities and apic ids.
Eric
Re: [Fastboot] Re: kexec "problem" [and patch updates]
Hello,
I recreated this on a UNI system running an SMP kernel as well.
The problem is because we now initialize cpu_vm_mask for init_mm with CPU_MASK_ALL (from 2.6.3 onwards) which makes all bits in cpumask 1. Hence BUG_ON(!cpus_equal(cpumask,tmp) fails. The change to set cpu_vm_mask to CPU_MASK_ALL was done to remove tlb flush optimizations for ppc64. On UNI kernels, CPU_MASK_ALL is 1 and hence the problem does not occur.
I made a small patch which fixes this problem. The change is, essentially, to use "tmp" instead of "cpumask". This ensures that only the (other) online cpus are sent the IPI.
I have done some testing with this patch. Kexec loads fine and I haven't seen anything untoward.
Comments please.
Regards, Hari
arch/i386/kernel/smp.c +5 -4
@@ -356,7 +356,8 @@BUG_ON (cpus_empty (cpumask));cpus_and (tmp, cpumask, cpu_online_map); -BUG_ON (!cpus_equal (cpumask, tmp)); +if (cpus_empty (tmp)) +return ;BUG_ON (cpu_isset (smp_processor_id (), cpumask));BUG_ON (!mm);
@@ -371,12 +372,12 @@ flush_mm = mm; flush_va = va;#if NR_CPUS <=BITS_PER_LONG -atomic_set_mask (cpumask, &flush_cpumask); +atomic_set_mask (tmp, &flush_cpumask);#else {int k;unsigned long *flush_mask = (unsigned long *)&flush_cpumask; -unsigned long *cpu_mask = (unsigned long *)&cpumask; +unsigned long *cpu_mask = (unsigned long *)&tmp;for (k =0 ; k <BITS_TO_LONGS (NR_CPUS ); ++k)atomic_set_mask (cpu_mask[k], &flush_mask[k]); }
@@ -385,7 +386,7 @@* We have to send the IPI only to * CPUs affected. */ -send_IPI_mask (cpumask,INVALIDATE_TLB_VECTOR ); +send_IPI_mask (tmp,INVALIDATE_TLB_VECTOR );while (!cpus_empty (flush_cpumask))/* nothing. lockup detection does not belong here */
Quoted 58 lines · Eric W. Biederman
Quoted 18 lines · Randy.Dunlap
On 27 Feb 2004 01:00:04 -0700 Eric W. Biederman wrote:
| > It works fine on 2.6.2. It works for me on 2.6.3 if not SMP. | > If the kernel is built for SMP, when running kexec, I get a | > BUG in arch/i386/kernel/smp.c at line 359. | > I'm testing various workarounds for that BUG now. | | I will eyeball it... | | Is it the kernel that is shutting down, or the kernel that is being | brought up that has problems?the kernel that is shutting down.
| The back trace from the BUG would be interesting.
see below. my bad. i should have included it.
Quoted 7 lines
| As I see it flush_tlb_others is being called when we have shutdown | cpus and the kernel still thinks we have the mm present on foreign | cpus.Martin Bligh thinks that there is a tlb race here. I printed the 2 cpu masks on my dual-proc macine and saw 0 in one of them and
0xcin the other one.Ouch we have both cpus running when this happens, and we have not started any shutdown whatsoever. This is the bit that sets up the page tables for later use...
I think
identity_map_pageswill have problems with a kernel that does the 4G/4G split, and it has known issues on some other architectures, because they treatinit_mmspecially. So the proper solution may be to simply rewriteidentity_map_pages.Before we do that in the short term we need to see if
identity_map_pagesis actually doing anything bad. You are not using the 4G/4G split so that is not the cause. So eitherinit_mmis now special in some way, or we have hit a generic kernel bug.So this may indeed be a tlb race. But it is
init_mm->cpu_vm_maskandcpu_onlinemap that are different. With the implication being thatinit_mm->cpu_vm_maskhas cpus set that are not incpu_online_map? Very weird especially on SMP.Without attribution I have a hard time making sense of which cpumask is which so I can't draw any conclusions. But I find it very interesting that it is bits 2 and 3 that are set. I wonder if there is any mixup between logical cpu identities and apic ids.
Eric
_______________________________________________
fastboot mailing list
[email protected]
http://lists.osdl.org/mailman/listinfo/fastboot
Signature · 4 lines
Hariprasad Nellitheertha Linux Technology Center India Software Labs IBM India, Bangalore
Re: [Fastboot] Re: kexec "problem" [and patch updates]
Quoted 10 lines · Hariprasad Nellitheertha
Hello,
I recreated this on a UNI system running an SMP kernel as well.
The problem is because we now initialize
cpu_vm_maskforinit_mmwithCPU_MASK_ALL(from 2.6.3 onwards) which makes all bits in cpumask 1. Hence BUG_ON(!cpus_equal(cpumask,tmp) fails. The change to setcpu_vm_masktoCPU_MASK_ALLwas done to remove tlb flush optimizations for ppc64. On UNI kernels,CPU_MASK_ALLis 1 and hence the problem does not occur.
So the problem is that CPU_MASK_ALL includes cpus that are not currently online. So it has gone from being wrong by including too few cpus to being wrong by including too many cpus.
Quoted 8 lines
I made a small patch which fixes this problem. The change is, essentially, to use "tmp" instead of "cpumask". This ensures that only the (other) online cpus are sent the IPI.
I have done some testing with this patch. Kexec loads fine and I haven't seen anything untoward.
Comments please.
Any chance we can fix this right and get a proper value in cpu_vm_mask for init_mm? All that needs to happen is that each cpu as it is started up is included in cpu_vm_mask.
The reason kexec sees this is that it is possibly the only generic modifier of init_mm.
If fixing this needs to be kexec specific we need to simply remove using init_mm.
Eric
Re: [Fastboot] Re: kexec "problem" [and patch updates]
In reply to Hariprasad NellitheerthaOn Thu, 4 Mar 2004 18:33:10 +0530 Hariprasad Nellitheertha wrote:
| Hello, | | I recreated this on a UNI system running an SMP kernel as well. | | The problem is because we now initialize cpu_vm_mask for init_mm with | CPU_MASK_ALL (from 2.6.3 onwards) which makes all bits in cpumask 1. | Hence BUG_ON(!cpus_equal(cpumask,tmp) fails. The change to set | cpu_vm_mask to CPU_MASK_ALL was done to remove tlb flush optimizations | for ppc64. On UNI kernels, CPU_MASK_ALL is 1 and hence the problem | does not occur. | | I made a small patch which fixes this problem. The change is, essentially, | to use "tmp" instead of "cpumask". This ensures that only the (other) online | cpus are sent the IPI. | | I have done some testing with this patch. Kexec loads fine and I haven't seen | anything untoward.
Yes, that does work well... Thanks for the patch.
Is this satisfactory for pushing into the mainline kernel, or should kexec use another method to solve this problem?
--
~Randy
| Comments please.
|
| Regards, Hari
|
|
| diff -Naur linux-2.6.3-before/arch/i386/kernel/smp.c linux-2.6.3/arch/i386/kernel/smp.c
| --- linux-2.6.3-before/arch/i386/kernel/smp.c 2004-02-18 09:27:15.000000000 +0530
| +++ linux-2.6.3/arch/i386/kernel/smp.c 2004-03-04 14:16:43.000000000 +0530
| @@ -356,7 +356,8 @@
| BUG_ON(cpus_empty(cpumask));
|
| cpus_and(tmp, cpumask, cpu_online_map);
| - BUG_ON(!cpus_equal(cpumask, tmp));
| + if(cpus_empty(tmp))
| + return;
| BUG_ON(cpu_isset(smp_processor_id(), cpumask));
| BUG_ON(!mm);
|
| @@ -371,12 +372,12 @@
| flush_mm = mm;
| flush_va = va;
| #if NR_CPUS <= BITS_PER_LONG
| - atomic_set_mask(cpumask, &flush_cpumask);
| + atomic_set_mask(tmp, &flush_cpumask);
| #else
| {
| int k;
| unsigned long *flush_mask = (unsigned long *)&flush_cpumask;
| - unsigned long *cpu_mask = (unsigned long *)&cpumask;
| + unsigned long *cpu_mask = (unsigned long *)&tmp;
| for (k = 0; k < BITS_TO_LONGS(NR_CPUS); ++k)
| atomic_set_mask(cpu_mask[k], &flush_mask[k]);
| }
| @@ -385,7 +386,7 @@
| * We have to send the IPI only to
| * CPUs affected.
| */
| - send_IPI_mask(cpumask, INVALIDATE_TLB_VECTOR);
| + send_IPI_mask(tmp, INVALIDATE_TLB_VECTOR);
|
| while (!cpus_empty(flush_cpumask))
| /* nothing. lockup detection does not belong here */
|
|
| On Sat, Feb 28, 2004 at 03:41:33AM -0700, Eric W. Biederman wrote:
| > "Randy.Dunlap" <[email protected]> writes:
| >
| > > On 27 Feb 2004 01:00:04 -0700 Eric W. Biederman wrote:
| > >
| > > | > It works fine on 2.6.2. It works for me on 2.6.3 if not SMP.
| > > | > If the kernel is built for SMP, when running kexec, I get a
| > > | > BUG in arch/i386/kernel/smp.c at line 359.
| > > | > I'm testing various workarounds for that BUG now.
| > > |
| > > | I will eyeball it...
| > > |
| > > | Is it the kernel that is shutting down, or the kernel that is being
| > > | brought up that has problems?
| > >
| > > the kernel that is shutting down.
| > >
| > > | The back trace from the BUG would be interesting.
| > >
| > > see below. my bad. i should have included it.
| > >
| > > | As I see it flush_tlb_others is being called when we have shutdown
| > > | cpus and the kernel still thinks we have the mm present on foreign
| > > | cpus.
| > >
| > > Martin Bligh thinks that there is a tlb race here.
| > > I printed the 2 cpu masks on my dual-proc macine and saw
| > > 0 in one of them and 0xc in the other one.
| >
| > Ouch we have both cpus running when this happens, and we have not
| > started any shutdown whatsoever. This is the bit that sets up
| > the page tables for later use...
| >
| > I think identity_map_pages will have problems with a kernel that does
| > the 4G/4G split, and it has known issues on some other architectures,
| > because they treat init_mm specially. So the proper solution may be
| > to simply rewrite identity_map_pages.
| >
| > Before we do that in the short term we need to see if
| > identity_map_pages is actually doing anything bad. You are
| > not using the 4G/4G split so that is not the cause. So either
| > init_mm is now special in some way, or we have hit a generic kernel
| > bug.
| >
| > So this may indeed be a tlb race. But it is init_mm->cpu_vm_mask and
| > cpu_online map that are different. With the implication being
| > that init_mm->cpu_vm_mask has cpus set that are not in cpu_online_map?
| > Very weird especially on SMP.
| >
| > Without attribution I have a hard time making sense of which cpumask
| > is which so I can't draw any conclusions. But I find it very
| > interesting that it is bits 2 and 3 that are set. I wonder if
| > there is any mixup between logical cpu identities and apic ids.
| >
| > Eric
| > _______________________________________________| | -- | Hariprasad Nellitheertha | Linux Technology Center | India Software Labs | IBM India, Bangalore | -
Quoted 10 lines
Some basic results are up comparing streaming non-cached random AIO read/write throughputs for a single ext3 file using aio-stress for various io sizes with and without these patches, and also comparsions with
O_DIRECTAIO throughputs. The short summary has been a doubling of throughput using the fsaio patches, which is also close to the results seen withO_DIRECTAIO.As usual feedback, bug fixes, test results etc are welcome.
What exactly is the O_DIRECT flag? When I add this flag to the open func it fails.
More specificaly, this function fails
open("filename", O_RDWR | O_DIRECT | O_LARGEFILE | O_CREAT, S_IRWXU);
but this one succeeds
open("filename", O_RDWR | O_LARGEFILE | O_CREAT, S_IRWXU);
I'm running linux 2.6.0 with libaio 0.3.92.
Hayim.
Quoted 10 lines · Hayim Shaul
What exactly is the
O_DIRECTflag? When I add this flag to the open func it fails.More specificaly, this function fails
open("filename", O_RDWR | O_DIRECT | O_LARGEFILE | O_CREAT, S_IRWXU);but this one succeeds
open("filename", O_RDWR | O_LARGEFILE | O_CREAT, S_IRWXU);I'm running linux 2.6.0 with libaio 0.3.92.
Which filesystem? Not all support O_DIRECT.
-ben
Quoted 7 lines · Benjamin LaHaise
Quoted 6 lines · Hayim Shaul
What exactly is the
O_DIRECTflag? When I add this flag to the open func it fails.More specificaly, this function fails
open("filename", O_RDWR | O_DIRECT | O_LARGEFILE | O_CREAT, S_IRWXU);
Quoted 7 lines
Quoted 4 lines
but this one succeeds
open("filename", O_RDWR | O_LARGEFILE | O_CREAT, S_IRWXU);I'm running linux 2.6.0 with libaio 0.3.92.
Which filesystem? Not all support
O_DIRECT.
ext3
I'm think it does support ext3.
Actually, I was wrong. open does succeed. It return a valid fd but after writing and exiting, the file is still zero size.
removing the O_DIRECT with the same prog writes quite alot to the file.
Hayim.
Are you checking the return value of write()?
O_DIRECT has alignment requirements since it writes directly from user-space to disk by passing the page cache. Also the size of the write has to be a multiple of 512 (for 2.6).
Try using posix_memalign() with pagesize as the alignment arg to allocated the data buffer. O_DIRECT does work on ext3.
Daniel
Quoted 32 lines · Hayim Shaul
Quoted 7 lines · Benjamin LaHaise
Quoted 6 lines · Hayim Shaul
What exactly is the
O_DIRECTflag? When I add this flag to the open func it fails.More specificaly, this function fails
open("filename", O_RDWR | O_DIRECT | O_LARGEFILE | O_CREAT, S_IRWXU);Quoted 7 lines
Quoted 4 lines
but this one succeeds
open("filename", O_RDWR | O_LARGEFILE | O_CREAT, S_IRWXU);I'm running linux 2.6.0 with libaio 0.3.92.
Which filesystem? Not all support
O_DIRECT.ext3
I'm think it does support ext3.Actually, I was wrong. open does succeed. It return a valid fd but after writing and exiting, the file is still zero size.
removing the
O_DIRECTwith the same prog writes quite alot to the file.Hayim.
--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>