Fastboot revisited: Asynchronous function calls
cover Fastboot revisited: Asynchronous function calls
Hi,
This patch series is based on Linus' original reaction to some of the fastboot patches which introduced an "asynchronous initcall level". Linus did not like the initcall level approach much, and wanted a much more finegrained kind of thing.
This patch series introduces asynchronous function calls. The goal is still the same (boot faster), the method is entirely different.
Asynchronous function calls allow drivers and subsystems to do operations asynchronously, while providing a mechanism to serialize the asynchronous operations at "externally visible" locations, such as device number registrations.
As such, this approach is NOT a "full parallel kernel init". For one, it keeps device numbers stable. For another, it's very much opt-in for any code using this.
Rather, it is more comparable to an out-of-order cpu, where computations are done asynchronously, but are retired in program order.
This series first introduces the infrastructure in patch 1, and then adds users of this infrastructure (scsi in patch2, libata in patch 3 and acpi in patch 4).
To see this working, I uploaded a "before" and "after" kernel bootchart for one of my testboxes at
http://www.fenrus.org/linux/before.svg
http://www.fenrus.org/linux/after.svg
As you can see with inkscape (or gimp or any other svg viewer), there is still room for doing more things in parallel, but the 3 example usage patches are a big win already.
One comment about the API:
Originally I had functionality in mind where the caller of async_schedule() could get back the cookie value of the scheduled task. However I had no users of this, so I removed the code to do this. If someone finds a place where this is useful we can easily add this back.
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
PATCH 1/4 fastboot: Asynchronous function calls to speed up kernel boot
Quoted 1 line
From
97ade4deb2124aebe73d0094da76fd136fc5b082Mon Sep 17 00:00:00 2001
From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: Asynchronous function calls to speed up kernel boot
Right now, most of the kernel boot is strictly synchronous, such that various hardware delays are done sequentially.
In order to make the kernel boot faster, this patch introduces infrastructure to allow doing some of the initialization steps asynchronously, which will hide significant portions of the hardware delays in practice.
In order to not change device order and other similar observables, this patch does NOT do full parallel initialization.
Rather, it operates more in the way an out of order CPU does; the work may be done out of order and asynchronous, but the observable effects (instruction retiring for the CPU) are still done in the original sequence.
--- include/linux/async.h | 21 ++++ init/main.c | 5 +- kernel/Makefile | 3 +- kernel/async.c | 305 +++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 332 insertions(+), 2 deletions(-) create mode 100644 include/linux/async.h create mode 100644 kernel/async.c
include/linux/async.h new file +21 -0
@@ -0,0 +1,21 @@ +/* +* async.h: Asynchronous function calls for boot performance +* +* (C) Copyright 2009 Intel Corporation +* Author: Arjan van de Ven <[email protected]> +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License +* as published by the Free Software Foundation; version 2 +* of the License. +*/ + +#include <linux/types.h> + +typedef u64 async_cookie_t ; +typedef void (async_func_ptr ) (void *data,async_cookie_t cookie); + +extern void async_schedule (async_func_ptr *ptr,void *data); +extern void async_synchronize_full (void ); +extern void async_synchronize_cookie (async_cookie_t cookie); +
init/main.c +4 -1
@@ -63,6 +63,7 @@#include <linux/signal.h> #include <linux/idr.h> #include <linux/ftrace.h> +#include <linux/async.h> #include <trace/boot.h> #include <asm/io.h>
@@ -687,7 +688,7 @@ asmlinkage void __init start_kernel(void)rest_init (); } -static int initcall_debug; +int initcall_debug;core_param (initcall_debug, initcall_debug, bool,0644 );int do_one_initcall (initcall_t fn)
@@ -788,6 +789,8 @@ static void run_init_process(char *init_filename)*/ static int noinline init_post(void) { +/* need to finish all async __init code before freeing the memory */ +async_synchronize_full ();free_initmem ();unlock_kernel ();mark_rodata_ro ();
kernel/Makefile +2 -1
@@ -9,7 +9,8 @@ obj-y = sched.o fork.o exec_domain.o panic.o printk.o \ rcupdate.o extable.o params.o posix-timers.o \ kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \ hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \ - notifier.o ksysfs.o pm_qos_params.o sched_clock.o cred.o + notifier.o ksysfs.o pm_qos_params.o sched_clock.o cred.o \ + async.o ifdef CONFIG_FUNCTION_TRACER # Do not trace debug files and internal ftrace files
kernel/async.c new file +305 -0
305 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Re: [PATCH 1/4] fastboot: Asynchronous function calls to speed up kernel boot
include/linux/async.h line 20
Quoted 7 lines · Arjan van de Ven
include/linux/async.h new file +7
-0+ +typedef u64 async_cookie_t ; +typedef void (async_func_ptr ) (void *data,async_cookie_t cookie); + +extern void async_schedule (async_func_ptr *ptr,void *data); +extern void async_synchronize_full (void ); +extern void async_synchronize_cookie (async_cookie_t cookie);
Hmm. The cookie use doesn't seem to make much sense.
Why do you pass in the cookie to the async function, but don't return it to the caller? That seems backwards - you'd normally expect that it is the _caller_ that wants the cookie (to synchronise with a specific async call), not the callee. But now the only one who knows the cookie is the wrong entry - just the callee, not the caller.
Yes, yes, I read the explanation in the comments, and it says that the callee should do it to guarantee its own ordering, and your acpi port thing does that in order to apparently start a sequence that is asynchronous only wrt the synchronous code, but not wrt itself. That's a _very_ odd model, but whatever works. But wouldn't it still make sense to let the caller wait for individual events too?
IOW, I'd just suggest changing the interface so that "async_schedule()" also returns the cookie.
Linus
Re: [PATCH 1/4] fastboot: Asynchronous function calls to speed up kernel boot
Quoted 13 lines · On Sun, 4 Jan 2009 10:33:34 -0800 (PST) Linus Torvalds…
include/linux/async.h line 20
Quoted 7 lines · Arjan van de Ven
include/linux/async.h new file +7
-0+ +typedef u64 async_cookie_t ; +typedef void (async_func_ptr ) (void *data,async_cookie_t cookie); + +extern void async_schedule (async_func_ptr *ptr,void *data); +extern void async_synchronize_full (void ); +extern void async_synchronize_cookie (async_cookie_t cookie);Hmm. The cookie use doesn't seem to make much sense.
Quoted 5 lines
Why do you pass in the cookie to the async function, but don't return it to the caller? That seems backwards - you'd normally expect that it is the _caller_ that wants the cookie (to synchronise with a specific async call), not the callee. But now the only one who knows the cookie is the wrong entry - just the callee, not the caller.
in fact, either could need it.
the callee could need it when IT does a global registration (for example to get a device number) at the end of its sequence. We'd want that registration to happen sequential (it's basically the equivalent of a commit/retirement of the instruction in a CPU) the caller could need it when it wants to wait for the async function it kicked off.
so both make total sense to me.
Quoted 10 lines
Yes, yes, I read the explanation in the comments, and it says that the callee should do it to guarantee its own ordering, and your acpi port thing does that in order to apparently start a sequence that is asynchronous only wrt the synchronous code, but not wrt itself. That's a _very_ odd model, but whatever works. But wouldn't it still make sense to let the caller wait for individual events too?
IOW, I'd just suggest changing the interface so that "
async_schedule()" also returns the cookie.
I had that originally... (as I described in the first mail).. but had no users of it in the places I converted.
I'm happy to just return it; it does make sense (that's why I did this originally)....
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 1/4] fastboot: Asynchronous function calls to speed up kernel boot
In reply to Linus TorvaldsQuoted 5 lines · On Sun, 4 Jan 2009 10:33:34 -0800 (PST) Linus Torvalds…
IOW, I'd just suggest changing the interface so that "
async_schedule()" also returns the cookie.From
6b436c0fab92c50cae7ba3bcd9bcfebf2c9596f7Mon Sep 17 00:00:00 2001
From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 10:47:27 -0800
Subject: [PATCH] fastboot: return the cookie from async_schedule()
Put this code back as suggested by Linus; even if there's no users right now it makes conceptual sense and the cost is basically zero.
--- include/linux/async.h | 2 +- kernel/async.c | 10 ++++++---- 2 files changed, 7 insertions(+), 5 deletions(-)
include/linux/async.h +1 -1
@@ -15,7 +15,7 @@typedef u64 async_cookie_t ;typedef void (async_func_ptr ) (void *data,async_cookie_t cookie); -extern void async_schedule (async_func_ptr *ptr,void *data); +extern async_cookie_t async_schedule (async_func_ptr *ptr,void *data);extern void async_synchronize_full (void );extern void async_synchronize_cookie (async_cookie_t cookie);
kernel/async.c +6 -4
@@ -156,32 +156,34 @@ out: } -void async_schedule (async_func_ptr *ptr,void *data) +async_cookie_t async_schedule (async_func_ptr *ptr,void *data) {struct async_entry *entry;unsigned long flags; +async_cookie_t newcookie; +/* allow irq-off callers */ entry =kzalloc (sizeof (struct async_entry ),GFP_ATOMIC );if (!entry) { -async_cookie_t newcookie;spin_lock_irqsave (&async_lock, flags); newcookie = next_cookie++;spin_unlock_irqrestore (&async_lock, flags);/* low on memory.. run synchronously */ ptr (data, newcookie); -return ; +return newcookie; } entry->func = ptr; entry->data = data;spin_lock_irqsave (&async_lock, flags); - entry->cookie = next_cookie++; + newcookie = entry->cookie = next_cookie++;list_add_tail (&entry->list, &async_pending);atomic_inc (&entry_count);spin_unlock_irqrestore (&async_lock, flags);wake_up (&async_new); +return newcookie; }EXPORT_SYMBOL_GPL (async_schedule);
Signature · 7 lines
1.6.0.6 -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 1/4] fastboot: Asynchronous function calls to speed up kernel boot
In reply to Arjan van de VenQuoted 26 lines · Arjan van de Ven
kernel/async.c new file +26
-0+static int async_manager_thread (void *unused) +{ +int max_threads; +DECLARE_WAITQUEUE (wq, current); +add_wait_queue (&async_new, &wq); + +while (!kthread_should_stop ()) { +int tc, ec; + +/* +* Maximum number of worker threads. +* Even on the smallest machine we want 8 +* Scaling by 4 per logical CPU +* But no more than 256 to not overload stuff too much +* (and yes these are magic numbers that might need tuning) +* Calculated dynamic because early on the nr of online cpus +* is 1... +*/ + max_threads =4 +4 *num_online_cpus (); +if (max_threads >256 ) + max_threads =256 ; + + +set_current_state (TASK_INTERRUPTIBLE ); + + tc =atomic_read (&thread_count);
kernel/async.c line 292
Quoted 10 lines
kernel/async.c new file +10
-0+rmb (); + ec =atomic_read (&entry_count); + +while (tc < ec && tc < max_threads) { +kthread_run (async_thread,NULL ,"async/%i" , tc); +atomic_inc (&thread_count); + tc++; + } + +schedule ();
Surely the thread should die again boot up? On module load synchronisity is usually not a problem.
Personally I think it would be better to make this more generic. Various subsystems have thread pool implementations now, and this is just another variant that except for the sequence stuff isn't all that much different. So it would be better to have a generic worker thread manager that just supports these barriers too.
-andi
Signature · 1 line
[email protected]
Re: [PATCH 1/4] fastboot: Asynchronous function calls to speed up kernel boot
Quoted 2 lines · On Sun, 04 Jan 2009 20:05:26 +0100 Andi Kleen <[email protected]> wrote:
Surely the thread should die again boot up? On module load synchronisity is usually not a problem.
sadly that's not correct in practice based on the fast boot work we've done.
Quoted 3 lines
Personally I think it would be better to make this more generic. Various subsystems have thread pool implementations now,
sort of kinda. If a good one appears I'd be happy to build on top of that, assuming it's generic enough.
Quoted 5 lines
and this
is just another variant that except for the sequence stuff isn't all that much different. So it would be better to have a generic worker thread manager that just supports these barriers too.
... or maybe think about seeing this system as exactly that thread manager?
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 1/4] fastboot: Asynchronous function calls to speed up kernel boot
Quoted 8 lines · Arjan van de Ven
Quoted 2 lines · On Sun, 04 Jan 2009 20:05:26 +0100 Andi Kleen <[email protected]> wrote:
Surely the thread should die again boot up? On module load synchronisity is usually not a problem.
sadly that's not correct in practice based on the fast boot work we've done.
Hmm, but I'm not sure your current code is module safe, in particular against unloading again. You would likely need a barrier at the end of module load at least.
Quoted 7 lines
Quoted 4 lines
Personally I think it would be better to make this more generic. Various subsystems have thread pool implementations now,
sort of kinda. If a good one appears I'd be happy to build on top of that, assuming it's generic enough.
I think you can just create a separate barrier primitive which will work independently of any special thread managers.
Quoted 9 lines
Quoted 6 lines
and this
is just another variant that except for the sequence stuff isn't all that much different. So it would be better to have a generic worker thread manager that just supports these barriers too.... or maybe think about seeing this system as exactly that thread manager?
I'm not sure it's generic enough.
-Andi
Signature · 1 line
[email protected]
Re: [PATCH 1/4] fastboot: Asynchronous function calls to speed up kernel boot
Quoted 11 lines · On Sun, 4 Jan 2009 20:49:46 +0100 Andi Kleen <[email protected]> wrote:
Quoted 9 lines
Quoted 6 lines
and this
is just another variant that except for the sequence stuff isn't all that much different. So it would be better to have a generic worker thread manager that just supports these barriers too.... or maybe think about seeing this system as exactly that thread manager?
I'm not sure it's generic enough.
There's appreciable overlap with David Howells's "slow work" thread pool thingy. If either of those was generic enough, the other would go away.
PATCH 2/4 fastboot: make scsi probes asynchronous
In reply to Arjan van de VenQuoted 1 line
From
2087b546abddced50fa73ea9a71b5a040ae1026eMon Sep 17 00:00:00 2001
From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make scsi probes asynchronous
This patch makes part of the scsi probe (which is mostly device spin up and the partition scan) asynchronous. Only the part that runs after getting the device number allocated is asynchronous, ensuring that device numbering remains stable.
--- drivers/scsi/scsi_scan.c | 3 + drivers/scsi/sd.c | 109 ++++++++++++++++++++++++++++------------------ 2 files changed, 70 insertions(+), 42 deletions(-)
drivers/scsi/scsi_scan.c +3 -0
@@ -32,6 +32,7 @@#include <linux/delay.h> #include <linux/kthread.h> #include <linux/spinlock.h> +#include <linux/async.h> #include <scsi/scsi.h> #include <scsi/scsi_cmnd.h>
@@ -179,6 +180,8 @@ int scsi_complete_async_scans(void)spin_unlock (&async_scan_lock);kfree (data); +/* Synchronize async operations globally */ +async_synchronize_full ();return 0 ; }
drivers/scsi/sd.c +67 -42
127 lines not on this page · raw message
Signature · 9 lines
1.6.0.6 -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 2/4] fastboot: make scsi probes asynchronous
Quoted 8 lines · Arjan van de Ven
Quoted 1 line
From
2087b546abddced50fa73ea9a71b5a040ae1026eMon Sep 17 00:00:00 2001From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make scsi probes asynchronousThis patch makes part of the scsi probe (which is mostly device spin up and the partition scan) asynchronous. Only the part that runs after getting the device number allocated is asynchronous, ensuring that device numbering remains stable.
Is this really a big win? Were you using the async scsi scan code already?
Signature · 4 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Re: [PATCH 2/4] fastboot: make scsi probes asynchronous
Quoted 7 lines · On Sun, 4 Jan 2009 11:05:15 -0700 Matthew Wilcox <[email protected]> wrote:
Quoted 6 lines · Arjan van de Ven
Quoted 2 lines
From
2087b546abddced50fa73ea9a71b5a040ae1026eMon Sep 17 00:00:00 2001From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make scsi probes asynchronous
Quoted 7 lines
Quoted 4 lines
This patch makes part of the scsi probe (which is mostly device spin up and the partition scan) asynchronous. Only the part that runs after getting the device number allocated is asynchronous, ensuring that device numbering remains stable.
Is this really a big win? Were you using the async scsi scan code already?
just checked;
my box has CONFIG_SCSI_SCAN_ASYNC=y set for both cases
it seems to be about 0.1 seconds win still though...
sounds little, but a kernel boot should be less than a second, so that is still 10%. And it's obviously also disk dependent..
(I used libata connected disks obviously)
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 2/4] fastboot: make scsi probes asynchronous
Quoted 8 lines · Arjan van de Ven
just checked;
my box hasCONFIG_SCSI_SCAN_ASYNC=yset for both cases
it seems to be about 0.1 seconds win still though...
sounds little, but a kernel boot should be less than a second, so that is still 10%. And it's obviously also disk dependent..
(I used libata connected disks obviously)
Do you also have boot time numbers for RAM and/or flash based filesystem (disk-less)
JSR
Re: [PATCH 2/4] fastboot: make scsi probes asynchronous
Quoted 11 lines · On Mon, 5 Jan 2009 00:08:42 +0530 "Jaswinder Singh Rajput"…
Quoted 8 lines · Arjan van de Ven
just checked;
my box hasCONFIG_SCSI_SCAN_ASYNC=yset for both cases it seems to be about 0.1 seconds win still though... sounds little, but a kernel boot should be less than a second, so that is still 10%. And it's obviously also disk dependent.. (I used libata connected disks obviously)
Quoted 2 lines
Do you also have boot time numbers for RAM and/or flash based filesystem (disk-less)
my laptop with a pretty nice SSD shows a similar kind of behavior (bit shorter in time, but there the entire kernel boot is shorter as a whole)
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
PATCH 3/4 fastboot: make the libata port scan asynchronous
In reply to Arjan van de VenQuoted 2 lines
From
2eb80a6c7b7c33a9053cfd286169d5163027f0cdMon Sep 17 00:00:00 2001
From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make the libata port scan asynchronous
This patch makes the libata port scanning asynchronous (per device). There is a synchronization point before doing the actual disk scan so that device ordering is not affected.
--- drivers/ata/libata-core.c | 84 ++++++++++++++++++++++++-------------------- 1 files changed, 46 insertions(+), 38 deletions(-)
drivers/ata/libata-core.c +46 -38
@@ -56,6 +56,7 @@#include <linux/workqueue.h> #include <linux/scatterlist.h> #include <linux/io.h> +#include <linux/async.h> #include <scsi/scsi.h> #include <scsi/scsi_cmnd.h> #include <scsi/scsi_host.h>
@@ -5909,6 +5910,48 @@ void ata_host_init(struct ata_host *host, struct device *dev, host->ops = ops; } + +void async_port_probe (void *data,async_cookie_t cookie) +{ +int rc; +struct ata_port *ap = data; +/* probe */ +if (ap->ops->error_handler) { +struct ata_eh_info *ehi = &ap->link.eh_info; +unsigned long flags; + +ata_port_probe (ap); + +/* kick EH for boot probing */ +spin_lock_irqsave (ap->lock, flags); + + ehi->probe_mask |=ATA_ALL_DEVICES ; + ehi->action |=ATA_EH_RESET |ATA_EH_LPM ; + ehi->flags |=ATA_EHI_NO_AUTOPSY |ATA_EHI_QUIET ; + + ap->pflags &= ~ATA_PFLAG_INITIALIZING ; + ap->pflags |=ATA_PFLAG_LOADING ; +ata_port_schedule_eh (ap); + +spin_unlock_irqrestore (ap->lock, flags); + +/* wait for EH to finish */ +ata_port_wait_eh (ap); + }else { +DPRINTK ("ata%u: bus probe begin\n" , ap->print_id); + rc =ata_bus_probe (ap); +DPRINTK ("ata%u: bus probe end\n" , ap->print_id); + +if (rc) { +/* FIXME: do something useful here? +* Current libata behavior will +* tear down everything when +* the module is removed +* or the h/w is unplugged. +*/ + } + } +}/** * ata_host_register - register initialized ATA host * @host: ATA host to register
@@ -5988,45 +6031,9 @@ int ata_host_register(struct ata_host *host, struct scsi_host_template *sht)DPRINTK ("probe begin\n" );for (i =0 ; i < host->n_ports; i++) {struct ata_port *ap = host->ports[i]; - -/* probe */ -if (ap->ops->error_handler) { -struct ata_eh_info *ehi = &ap->link.eh_info; -unsigned long flags; - -ata_port_probe (ap); - -/* kick EH for boot probing */ -spin_lock_irqsave (ap->lock, flags); - - ehi->probe_mask |=ATA_ALL_DEVICES ; - ehi->action |=ATA_EH_RESET |ATA_EH_LPM ; - ehi->flags |=ATA_EHI_NO_AUTOPSY |ATA_EHI_QUIET ; - - ap->pflags &= ~ATA_PFLAG_INITIALIZING ; - ap->pflags |=ATA_PFLAG_LOADING ; -ata_port_schedule_eh (ap); - -spin_unlock_irqrestore (ap->lock, flags); - -/* wait for EH to finish */ -ata_port_wait_eh (ap); - }else { -DPRINTK ("ata%u: bus probe begin\n" , ap->print_id); - rc =ata_bus_probe (ap); -DPRINTK ("ata%u: bus probe end\n" , ap->print_id); - -if (rc) { -/* FIXME: do something useful here? -* Current libata behavior will -* tear down everything when -* the module is removed -* or the h/w is unplugged. -*/ - } - } +async_schedule (async_port_probe, ap); } - +async_synchronize_full ();/* probes are done, now scan each port's disk(s) */ DPRINTK ("host probe begin\n" );for (i =0 ; i < host->n_ports; i++) {
@@ -6034,6 +6041,7 @@ int ata_host_register(struct ata_host *host, struct scsi_host_template *sht)ata_scsi_scan_host (ap,1 ); } +DPRINTK ("host probe end\n" );return 0 ; }
Signature · 7 lines
1.6.0.6 -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
PATCH 4/4 fastboot: make ACPI bus drivers probe asynchronous
In reply to Arjan van de VenQuoted 2 lines
From
20af3efd07b775a27a997e3df1039e20ea18f62bMon Sep 17 00:00:00 2001
From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make ACPI bus drivers probe asynchronous
the various ACPI bus drivers have non-overlapping devices and can each be run asynchronous. Some of the ACPI drivers (especially the battery one, but others as well) can take quite a long time to probe.
--- drivers/acpi/scan.c | 23 ++++++++++++++++------- 1 files changed, 16 insertions(+), 7 deletions(-)
drivers/acpi/scan.c +16 -7
@@ -8,6 +8,7 @@#include <linux/acpi.h> #include <linux/signal.h> #include <linux/kthread.h> +#include <linux/async.h> #include <acpi/acpi_drivers.h> #include <acpi/acinterp.h> /* for acpi_ex_eisa_id_to_string() */
@@ -578,6 +579,19 @@ static int acpi_start_single_object(struct acpi_device *device)return result; } +static void acpi_bus_register_async (void *data,async_cookie_t cookie) +{ +int ret; +struct acpi_driver *driver = data; + driver->drv.name = driver->name; + driver->drv.bus = &acpi_bus_type; + driver->drv.owner = driver->owner; + +async_synchronize_cookie (cookie); + + ret =driver_register (&driver->drv); +WARN_ON (ret !=0 ); +}/** * acpi_bus_register_driver - register a driver with the ACPI bus * @driver: driver being registered
@@ -588,16 +602,11 @@ static int acpi_start_single_object(struct acpi_device *device)*/ int acpi_bus_register_driver (struct acpi_driver *driver) { -int ret;if (acpi_disabled)return -ENODEV ; - driver->drv.name = driver->name; - driver->drv.bus = &acpi_bus_type; - driver->drv.owner = driver->owner; - - ret =driver_register (&driver->drv); -return ret; +async_schedule (acpi_bus_register_async, driver); +return 0 ; }EXPORT_SYMBOL (acpi_bus_register_driver);
Signature · 7 lines
1.6.0.6 -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 4/4] fastboot: make ACPI bus drivers probe asynchronous
Quoted 8 lines · Arjan van de Ven
From
20af3efd07b775a27a997e3df1039e20ea18f62bMon Sep 17 00:00:00 2001
From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make ACPI bus drivers probe asynchronousthe various ACPI bus drivers have non-overlapping devices and can each be run asynchronous. Some of the ACPI drivers (especially the battery one, but others as well) can take quite a long time to probe.
It seems that all the ACPI drivers are registered asynchronously after this change.
But there exists some dependency between some ACPI device drivers. It
is not very reasonable that the device driver is loaded asynchronously if there exists the dependency between some drivers.
For example:
On some boxes the info of battery is related with Embedded
controller. The EC driver should be loaded completely before registering battery driver.
On some boxes there exist the devices of ACPI power resource. And the
device power state is transited by controlling power resource. In such case the power resource device driver should be loaded completely before transiting the device power state.
Best regards.
Quoted 61 lines
Signed-off-by Arjan van de Ven <[email protected]>--- drivers/acpi/scan.c | 23 ++++++++++++++++------- 1 files changed, 16 insertions(+), 7 deletions(-)drivers/acpi/scan.c +16
-7@@ -8,6 +8,7 @@#include <linux/acpi.h> #include <linux/signal.h> #include <linux/kthread.h> +#include <linux/async.h> #include <acpi/acpi_drivers.h> #include <acpi/acinterp.h> /* for acpi_ex_eisa_id_to_string() */ @@ -578,6 +579,19 @@ static int acpi_start_single_object(struct acpi_device *device)return result; } +static void acpi_bus_register_async (void *data,async_cookie_t cookie) +{ +int ret; +struct acpi_driver *driver = data; + driver->drv.name = driver->name; + driver->drv.bus = &acpi_bus_type; + driver->drv.owner = driver->owner; + +async_synchronize_cookie (cookie); + + ret =driver_register (&driver->drv); +WARN_ON (ret !=0 ); +}/** * acpi_bus_register_driver - register a driver with the ACPI bus * @driver: driver being registered @@ -588,16 +602,11 @@ static int acpi_start_single_object(struct acpi_device *device)*/ int acpi_bus_register_driver (struct acpi_driver *driver) { -int ret;if (acpi_disabled)return -ENODEV ; - driver->drv.name = driver->name; - driver->drv.bus = &acpi_bus_type; - driver->drv.owner = driver->owner; - - ret =driver_register (&driver->drv); -return ret; +async_schedule (acpi_bus_register_async, driver); +return 0 ; }EXPORT_SYMBOL (acpi_bus_register_driver);Signature · 1 line
1.6.0.6
Re: [PATCH 4/4] fastboot: make ACPI bus drivers probe asynchronous
Quoted 6 lines · On Mon, 05 Jan 2009 10:03:12 +0800 Zhao Yakui <[email protected]> wrote:
Quoted 5 lines · Arjan van de Ven
From
20af3efd07b775a27a997e3df1039e20ea18f62bMon Sep 17 00:00:00 2001 From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make ACPI bus drivers probe asynchronous
Quoted 6 lines
Quoted 4 lines
the various ACPI bus drivers have non-overlapping devices and can each be run asynchronous. Some of the ACPI drivers (especially the battery one, but others as well) can take quite a long time to probe.
It seems that all the ACPI drivers are registered asynchronously after this change.
they are registered asynchronously compared to the rest of the kernel, but in strict original sequence compared to eachother.
Quoted 5 lines
Quoted 5 lines
+static void
acpi_bus_register_async(void *data, async_cookie_t cookie)+{drivers/acpi/scan.c +3
-0+int ret; +struct acpi_driver *driver = data; + driver->drv.name = driver->name;
Quoted 8 lines
drivers/acpi/scan.c static int acpi_start_single_object(struct acpi_device *device) · line 594
Quoted 8 lines
drivers/acpi/scan.c +8
-0+ driver->drv.bus = &acpi_bus_type; + driver->drv.owner = driver->owner; + +async_synchronize_cookie (cookie); + + ret =driver_register (&driver->drv); +WARN_ON (ret !=0 ); +}
because of the async_synchronize_cookie() call!
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 4/4] fastboot: make ACPI bus drivers probe asynchronous
Quoted 20 lines · Arjan van de Ven
Quoted 6 lines · On Mon, 05 Jan 2009 10:03:12 +0800 Zhao Yakui <[email protected]> wrote:
Quoted 5 lines · Arjan van de Ven
From
20af3efd07b775a27a997e3df1039e20ea18f62bMon Sep 17 00:00:00 2001 From: Arjan van de Ven <[email protected]>
Date: Sun, 4 Jan 2009 05:32:28 -0800
Subject: [PATCH] fastboot: make ACPI bus drivers probe asynchronousQuoted 6 lines
Quoted 4 lines
the various ACPI bus drivers have non-overlapping devices and can each be run asynchronous. Some of the ACPI drivers (especially the battery one, but others as well) can take quite a long time to probe.
It seems that all the ACPI drivers are registered asynchronously after this change.
they are registered asynchronously compared to the rest of the kernel, but in strict original sequence compared to eachother.
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
+static void
acpi_bus_register_async(void *data, async_cookie_t cookie)+{drivers/acpi/scan.c +3
-0+int ret; +struct acpi_driver *driver = data; + driver->drv.name = driver->name;Quoted 8 lines
drivers/acpi/scan.c static int acpi_start_single_object(struct acpi_device *device) · line 594
Quoted 8 lines
drivers/acpi/scan.c +8
-0+ driver->drv.bus = &acpi_bus_type; + driver->drv.owner = driver->owner; + +async_synchronize_cookie (cookie); + + ret =driver_register (&driver->drv); +WARN_ON (ret !=0 ); +}because of the
async_synchronize_cookie()call!
After the async_synchronize_cookie is called, the register sequence will become strict.
But there exist multiple threads. It is possible that the function of acpi_bus_register_async is called on different threads. In such case the dependency can't be guaranteed.
Thanks.
Quoted 1 line
Re: [PATCH 4/4] fastboot: make ACPI bus drivers probe asynchronous
Quoted 6 lines · On Mon, 05 Jan 2009 10:51:50 +0800 Zhao Yakui <[email protected]> wrote:
Quoted 1 line
because of the
async_synchronize_cookie()call!After the
async_synchronize_cookieis called, the register sequence will become strict.
But there exist multiple threads. It is possible that the function ofacpi_bus_register_asyncis called on different threads. In such case the dependency can't be guaranteed.
this is not correct.
when the async_synchronize_cookie() call returns, all previous acpi_bus_register_async() calls have completed. So no older ones can be running.
there also can be no newer ones, because newer ones will wait for the current one to entirely complete before they will return from acpi_bus_register_async()
so.. since all older ones are complete, and no new ones can happen.. the sequence is strictly followed.
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 4/4] fastboot: make ACPI bus drivers probe asynchronous
Quoted 10 lines · Arjan van de Ven
Quoted 6 lines · On Mon, 05 Jan 2009 10:51:50 +0800 Zhao Yakui <[email protected]> wrote:
Quoted 1 line
because of the
async_synchronize_cookie()call!After the
async_synchronize_cookieis called, the register sequence will become strict.
But there exist multiple threads. It is possible that the function ofacpi_bus_register_asyncis called on different threads. In such case the dependency can't be guaranteed.
Quoted 4 lines
this is not correct.
when theasync_synchronize_cookie()call returns, all previousacpi_bus_register_async()calls have completed. So no older ones can be running.
Thanks for the detailed explanation. Understand it.
But I still have another question about the function of async_synchronize_cookie. I don't know whether my understanding is correct or not.
The function of async_synchronize_cookie is realized by comparing the variable of lowest_in_progress with the cookie. Only when the lowest_in_process is greater than or equal to the cookie value, it will return. When async_schedule is called, a new cookie will be assigned, which can also be used as the second argument of async_synchronize_cookie.
The value of lowest_in_progress is updated after calling the callback function (async_func_ptr). But the async_synchronize_cookie is called in the callback function (acpi_bus_register_async).
In such case it seems that there exists the dead lock. The value of lowest_in_progress should be updated before finishing the async callback function. But the value of lowest_in_progress is updated after calling the async callback function.
Quoted 8 lines
there also can be no newer ones, because newer ones will wait for the current one to entirely complete before they will return from
acpi_bus_register_async()so.. since all older ones are complete, and no new ones can happen.. the sequence is strictly followed.
Quoted 6 lines · Arjan van de Ven
To see this working, I uploaded a "before" and "after" kernel bootchart for one of my testboxes at
http://www.fenrus.org/linux/before.svg
http://www.fenrus.org/linux/after.svg
Ok, so why does the serial port init take so long? That's a quarter of a second for you, which is ridiculous.
I _think_ it's the irq auto-probing, but that's just a guess. The 8250 driver does some historical crud, like
probe_irq_off(probe_irq_on());
to get rid of any pending irq's, but that should be entirely pointless these days. I bet that line basically goes back to pre-history, before we made the auto-probing much stabler.
The irq auto-probing itself also has a few excessive delays, like waiting for 0.1 s just to wait for spurious interrupts to trigger. Doing the extra unnecessary probe makes that doubly expensive.
Does this patch make any difference to you? I'm not at all sure that it's the irq probing, but if it is, then this should make the serial probe go much faster.
Linus
--- drivers/serial/8250.c | 1 - kernel/irq/autoprobe.c | 4 ++-- 2 files changed, 2 insertions(+), 3 deletions(-)
drivers/serial/8250.c +0 -1
@@ -1259,7 +1259,6 @@ static void autoconfig_irq(struct uart_8250_port *up) }/* forget possible initially masked and pending IRQ */ -probe_irq_off (probe_irq_on ()); save_mcr =serial_inp (up,UART_MCR ); save_ier =serial_inp (up,UART_IER );serial_outp (up,UART_MCR ,UART_MCR_OUT1 |UART_MCR_OUT2 );
kernel/irq/autoprobe.c +2 -2
@@ -60,7 +60,7 @@ unsigned long probe_irq_on(void) }/* Wait for longstanding interrupts to trigger. */ -msleep (20 ); +msleep (10 );/* * enable any unassigned irqs
@@ -80,7 +80,7 @@ unsigned long probe_irq_on(void)/* * Wait for spurious interrupts to trigger */ -msleep (100 ); +msleep (10 );/* * Now filter out any obviously spurious interrupts
Quoted 4 lines · On Sun, 4 Jan 2009 10:16:57 -0800 (PST) Linus Torvalds…
Quoted 1 line · Arjan van de Ven
Quoted 8 lines
Quoted 5 lines
To see this working, I uploaded a "before" and "after" kernel bootchart for one of my testboxes at
http://www.fenrus.org/linux/before.svg
http://www.fenrus.org/linux/after.svgOk, so why does the serial port init take so long? That's a quarter of a second for you, which is ridiculous.
good question; I was about to start looking into that one (but wanted to get the patches out first before I did more than 3 to not waste time on some dead-end approach if it ends up being that)
Quoted 2 lines
I _think_ it's the irq auto-probing, but that's just a guess.
I kinda was hoping that in part it was a scaling thing with the number of ports (4 in my .config) so that I could do the ports asynchronous compared to eachother ;-)
Quoted 7 lines
The irq auto-probing itself also has a few excessive delays, like waiting for 0.1 s just to wait for spurious interrupts to trigger. Doing the extra unnecessary probe makes that doubly expensive.
Does this patch make any difference to you? I'm not at all sure that it's the irq probing, but if it is, then this should make the serial probe go much faster.
it turned it into a 25 msec deal .. pretty good improvement in my book. (now I still might want to try to do this async so that it becomes 25/4 ... but the pressure i
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 6 lines · Arjan van de Ven
Quoted 2 lines
I _think_ it's the irq auto-probing, but that's just a guess.
I kinda was hoping that in part it was a scaling thing with the number of ports (4 in my .config) so that I could do the ports asynchronous compared to eachother ;-)
Since it seems to be the irq auto-probing, then doing it asynchronously won't help. Not only will it happen only for ports that you actually have (rather than the number you have configured), but even if you actually have multiple physical ports, the irq autoprobing is all serialized (and has to be - it depends on).
It's serialized by the 'probing_active' mutex.
In fact, I suspect we should entirely disallow asynchronous irq probing, because while the probing itself is serialized wrt _other_ autoprobing, it's not necessarily safe wrt other drivers allocating interrupts in a non-probing manner.
So not only should we not bother to do irq autoprobing asynchronously (because it won't help), we also probably should make sure that there is no other driver doing any asynchronous "request_irq()" _while_ we probe. IOW, we should consider the "request_irq()" to be a globally ordered event, like requesting a major/minor number or registering a driver.
Of course, the way your asynchronous setup is done, there should never really be a reason for a driver to do the "request_irq()" asynchronously.
The driver _should_ have already requested the irq in its synchronous part, and then do only the actual IO probing asynchronously. So I think we're all fine, but it may be worth pointing this out.
It might also be worth it to have some debug facilities for this, and have things like request_irq(), device_register() and driver_register() etc all verify that they are not called from "async context".
Quoted 5 lines
Quoted 3 lines
Does this patch make any difference to you? I'm not at all sure that it's the irq probing, but if it is, then this should make the serial probe go much faster.
it turned it into a 25 msec deal .. pretty good improvement in my book.
Ok, that certainly makes it a non-issue. It's a scary change in the sense that it's touching code that we _never_ touch, and it's magic irq autoprobing stuff, but at the same time, 0.1 seconds for testing whether some spurious irq happens is a pretty ridiculous cost. Especially since we're only talking legacy ISA interrupts anyway. If we have screaming interrupts, they'll happen immediately, not after a tenth of a second.
That said, I also wonder if we really even need to autoprobe the interrupts on any modern hardware. Rather than trying to speed up irq probing, maybe we could figure it out some other way.. The only thing that matters on 99% of all machines are the one or two standard ports that normally show up on 2f8h/irq3 and 3f8h/irq4 or something like that.
Linus
Quoted 14 lines · On Sun, 4 Jan 2009 10:54:15 -0800 (PST) Linus Torvalds…
Since it seems to be the irq auto-probing, then doing it asynchronously won't help. Not only will it happen only for ports that you actually have (rather than the number you have configured), but even if you actually have multiple physical ports, the irq autoprobing is all serialized (and has to be - it depends on).
It's serialized by the '
probing_active' mutex.In fact, I suspect we should entirely disallow asynchronous irq probing, because while the probing itself is serialized wrt _other_ autoprobing, it's not necessarily safe wrt other drivers allocating interrupts in a non-probing manner.
Quoted 6 lines
So not only should we not bother to do irq autoprobing asynchronously (because it won't help), we also probably should make sure that there is no other driver doing any asynchronous "
request_irq()" _while_ we probe. IOW, we should consider the "request_irq()" to be a globally ordered event, like requesting a major/minor number or registering a driver.
well.
or we declare the irq probing stuff "rare" and just make THAT fully serializing....
do a full synchronization before starting a probe
(at that point no new async stuff can show up if it wouldn't show up already right now), and be done with it.
Quoted 15 lines
Quoted 6 lines
Quoted 3 lines
Does this patch make any difference to you? I'm not at all sure that it's the irq probing, but if it is, then this should make the serial probe go much faster.
it turned it into a 25 msec deal .. pretty good improvement in my book.
Ok, that certainly makes it a non-issue. It's a scary change in the sense that it's touching code that we _never_ touch, and it's magic irq autoprobing stuff, but at the same time, 0.1 seconds for testing whether some spurious irq happens is a pretty ridiculous cost. Especially since we're only talking legacy ISA interrupts anyway. If we have screaming interrupts, they'll happen immediately, not after a tenth of a second.
Quoted 6 lines
That said, I also wonder if we really even need to autoprobe the interrupts on any modern hardware. Rather than trying to speed up irq probing, maybe we could figure it out some other way.. The only thing that matters on 99% of all machines are the one or two standard ports that normally show up on 2f8h/irq3 and 3f8h/irq4 or something like that.
too bad this stuff isn't PCI enumerated.
but if someone really still maintains this code, it could probably be rewritten in a "we think it's likely irq 3. how about we test that. Oh no? then we do expensive probing" kind of way.
Right now I don't think I have time for it (this is going to take time.. there's so many weird things with serial ports that it's bound to break stuff in the beginning etc)
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 4 lines · Arjan van de Ven
or we declare the irq probing stuff "rare" and just make THAT fully serializing....
do a full synchronization before starting a probe
Yes. That's entirely possible. However, have you verified that if an async thread does a synchronization, it doesn't deadlock?
Quoted 7 lines
Quoted 6 lines
That said, I also wonder if we really even need to autoprobe the interrupts on any modern hardware. Rather than trying to speed up irq probing, maybe we could figure it out some other way.. The only thing that matters on 99% of all machines are the one or two standard ports that normally show up on 2f8h/irq3 and 3f8h/irq4 or something like that.
Quoted 8 lines
too bad this stuff isn't PCI enumerated.
but if someone really still maintains this code, it could probably be rewritten in a "we think it's likely irq 3. how about we test that. Oh no? then we do expensive probing" kind of way.Right now I don't think I have time for it (this is going to take time.. there's so many weird things with serial ports that it's bound to break stuff in the beginning etc)
Yeah. The problem is that that driver is used on such varied hardware. But we can probably pick it up from PnP/ACPI, and autoprobe only for things that don't get the info from there (either because they are not PC's, or becuase PnP/ACPI isn't configured in).
This is one case where I wouldn't be afraid of PnP information, because we could easily choose to only trust it if it makes sense (ie "if it's the standard 2f8/irq3 thing, then you might as well trust PnP").
Linus
Quoted 10 lines · On Sun, 4 Jan 2009 11:11:53 -0800 (PST) Linus Torvalds…
Quoted 4 lines · Arjan van de Ven
or we declare the irq probing stuff "rare" and just make THAT fully serializing....
do a full synchronization before starting a probeYes. That's entirely possible. However, have you verified that if an async thread does a synchronization, it doesn't deadlock?
an async thread can synchronize on its own cookie.
so yes if we make this a global serialize, we can't do irq autoprobing in async context. If we can use PNP/etc for serial.. I don't think that's a big deal.. nobody with sane hw would need to autoprobe ever.
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 3 lines
Right now I don't think I have time for it (this is going to take time.. there's so many weird things with serial ports that it's bound to break stuff in the beginning etc)
I can't duplicate this. Can you check you don't have
config SERIAL_8250_DETECT_IRQ
bool "Autodetect IRQ on standard ports (unsafe)" depends on SERIAL_8250_EXTENDED
help
Say Y here if you want the kernel to try to guess which IRQ to use for your serial port.
This is considered unsafe; it is far better to configure the
IRQ in a boot script using the setserial command.
If unsure, say N.
enabled
Quoted 13 lines · On Mon, 5 Jan 2009 16:21:25 +0000 Alan Cox <[email protected]> wrote:
Quoted 3 lines
Right now I don't think I have time for it (this is going to take time.. there's so many weird things with serial ports that it's bound to break stuff in the beginning etc)
I can't duplicate this. Can you check you don't have
config
SERIAL_8250_DETECT_IRQ
bool "Autodetect IRQ on standard ports (unsafe)" depends onSERIAL_8250_EXTENDED
help
Say Y here if you want the kernel to try to guess which IRQ to use for your serial port.
Quoted 8 lines
This is considered unsafe; it is far better to configure the
IRQ in a boot script using the setserial command.If unsure, say N.
enabled
ok I had that enabled somehow....
retesting now
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 2 lines · Arjan van de Ven
ok I had that enabled somehow....
Well, it's not just "somehow".
It's the x86 def_config value, and it's also the default in at least the Fedora kernels.
Probably for a very good - although these days probably historical -
reason: on older hardware, it really wasn't all that unusual to have the serial port at other interrupts.
And Alan may point to the help text, but that doesn't really matter since everybody clearly ignores it, and it's not what has been tested.
But yeah, we probably should disable it at least for modern machines. And even if it doesn't autodetect the irq, "setserial" hopefully still does work (and was always required for some machines anyway)
Linus
Quoted 3 lines · Linus Torvalds
That said, I also wonder if we really even need to autoprobe the interrupts on any modern hardware. Rather than trying to speed up irq probing, maybe we could figure it out some other way..
You'll hate that suggestion, but i8250_pnp should already discover it fine via ACPI on modern systems. ia64 has done it this way forever.
It probably won't work on really old system, but one could always use DMI year to distingush (not pretty, but works usually)
-Andi
Signature · 1 line
[email protected]
Quoted 9 lines · Andi Kleen
Quoted 1 line · Linus Torvalds
That said, I also wonder if we really even need to autoprobe the interrupts on any modern hardware. Rather than trying to speed up irq probing, maybe we could figure it out some other way..
You'll hate that suggestion, but
i8250_pnpshould already discover it fine via ACPI on modern systems. ia64 has done it this way forever.It probably won't work on really old system, but one could always use DMI year to distingush (not pretty, but works usually)
Well, we should be able to tell where we got the port information from, if it's somewhere expected to be reliable like PCI configuration, ISAPnP or PnPACPI, or whether we're just probing magical ports and hoping to find something.
The only cases on x86 hardware where I would think probing would have to happen would be where the machine was too old to do either ISA Plug & Play or PnPACPI, or if you added in a non-PnP ISA modem or serial card.
Quoted 3 lines · Linus Torvalds
Ok, so why does the serial port init take so long? That's a quarter of a second for you, which is ridiculous.
Hmm. I also note that you don't seem to have the IDE probing configured in at all. It's not _that_ uncommon, afaik, and even if you have no devices behind it (or perhaps _especially_ if you have no devices behind it), the IDE probe is actually pretty expensive. Hmm?
Also, I'm looking at the AHCI vs Marvell thing, and the problem seems to be that you do that libata port scan like this:
for (i = 0; i < host->n_ports; i++) {
struct ata_port *ap = host->ports[i];
async_schedule(async_port_probe, ap);
}
async_synchronize_full();
/* probes are done, now scan each port's disk(s) */
DPRINTK("host probe begin\n");
for (i = 0; i < host->n_ports; i++) {
struct ata_port *ap = host->ports[i];
ata_scsi_scan_host(ap, 1);
}which means that there is a full serialization between each controller.
Wouldn't is be possible to move the "ata_scsi_scan_host(ap, 1);" _into_ the async_port_probe(), and just do a
async_synchronize_cookie(cookie);
before it? Hmm?
And then not do any async_synchronize_full() at all, until much later. For example, we clearly do need that full synchronization before we try to mount the root filesystem, but we should have that particular synchronization regardless of any SATA issues.
I may be missing something obvious, of course. Maybe a simple cookie synchronization isn't sufficient for some reason (most likely reason:
other SCSI drivers that don't do it).
Linus
Quoted 11 lines · On Sun, 4 Jan 2009 11:27:08 -0800 (PST) Linus Torvalds…
Quoted 3 lines · Linus Torvalds
Ok, so why does the serial port init take so long? That's a quarter of a second for you, which is ridiculous.
Hmm. I also note that you don't seem to have the IDE probing configured in at all. It's not _that_ uncommon, afaik, and even if you have no devices behind it (or perhaps _especially_ if you have no devices behind it), the IDE probe is actually pretty expensive. Hmm?
actually pretty much all PC distros no longer ship with CONFIG_IDE, only libata.
And afaik the box that that bootchart came from only has SATA devices and the bios is also configured as such.. so not even libata sees IDE devices.
Quoted 18 lines
Also, I'm looking at the AHCI vs Marvell thing, and the problem seems to be that you do that libata port scan like this:
for (i = 0; i < host->n_ports; i++) { struct ata_port *ap = host->ports[i]; async_schedule(async_port_probe, ap); } async_synchronize_full(); /* probes are done, now scan each port's disk(s) */ DPRINTK("host probe begin\n"); for (i = 0; i < host->n_ports; i++) { struct ata_port *ap = host->ports[i]; ata_scsi_scan_host(ap, 1); }
Quoted 2 lines
which means that there is a full serialization between each controller.
yeah in the current patch that is exactly the case.
I don't know the libata code enough to know if it's safe to do otherwise. The code was structured such that it looked VERY deliberate to do these two things in separate steps.
Quoted 6 lines
Wouldn't is be possible to move the "
ata_scsi_scan_host(ap, 1);" _into_ theasync_port_probe(), and just do aasync_synchronize_cookie(cookie);before it? Hmm?
I would think so but someone from the libata/scsi world needs to say so.. I didn't want to break the world entirely yet.
Quoted 3 lines
And then not do any
async_synchronize_full()at all, until much later. For example, we clearly do need that full synchronization before we try to mount the root filesystem, but we should have that
actually we should try to mount, and if we fail, then synchronize, and try again.
I need that capability for something else anyway
(and have a patch to do this in general, I just need to dust it off and see if it's still any good)
[in case you're curious: if you boot without initrd, and have a touchpad, right now the kernel will wait for the touchpad probing to be done explicitly before trying to mount the rootfs; on many laptops that I have access to that touchpad probing can easily take 7 seconds. The patch I have gets rid of that delay by just trying, for most cases it'll just be there anyway]
Quoted 1 line
particular synchronization regardless of any SATA issues.
yeah likewise before freeing initmem... pretty much the same reason. I'm not worried about that too much; at some point you just have to serialize. no big deal.
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 8 lines
I _think_ it's the irq auto-probing, but that's just a guess. The 8250 driver does some historical crud, like
probe_irq_off(probe_irq_on());to get rid of any pending irq's, but that should be entirely pointless these days. I bet that line basically goes back to pre-history, before we made the auto-probing much stabler.
For non ISA boxes we shouldn't be - but this old code. The IRQ for the built in ports is defined by convention, in the BIOS PnP tables or with ACPI in the ACPI tables. So we shouldn't even be autoprobing the IRQ.
We *do* have to wait one transmission timeout or so to detect buggy uarts but in theory that could be moved down into the open path.
Alan
Quoted 22 lines · Linus Torvalds
Quoted 6 lines · Arjan van de Ven
To see this working, I uploaded a "before" and "after" kernel bootchart for one of my testboxes at
http://www.fenrus.org/linux/before.svg
http://www.fenrus.org/linux/after.svgOk, so why does the serial port init take so long? That's a quarter of a second for you, which is ridiculous.
I _think_ it's the irq auto-probing, but that's just a guess. The 8250 driver does some historical crud, like
probe_irq_off(probe_irq_on());to get rid of any pending irq's, but that should be entirely pointless these days. I bet that line basically goes back to pre-history, before we made the auto-probing much stabler.
Quoted 7 lines
The irq auto-probing itself also has a few excessive delays, like waiting for 0.1 s just to wait for spurious interrupts to trigger. Doing the extra unnecessary probe makes that doubly expensive.
Does this patch make any difference to you? I'm not at all sure that it's the irq probing, but if it is, then this should make the serial probe go much faster.
Do serial ports even matter for the netbooks this is targeted at? Serial ports are gone even on normal notebooks and getting rare at desktop...
drivers/serial/8250.c static void autoconfig_irq(struct uart_8250_port *up) · line 1262
Quoted 4 lines
drivers/serial/8250.c +0
-1}/* forget possible initially masked and pending IRQ */ -probe_irq_off (probe_irq_on ());
Delete the comment, too? :-).
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Hi Arjan.
Great work, and valuable too. I'm just wondering if the header file name "async.h" is a little too generic? Something like async_init.h perhaps?
Regards,
Nigel
Quoted 9 lines · Nigel Cunningham
Hi Arjan.
Great work, and valuable too. I'm just wondering if the header file name "
async.h" is a little too generic? Something likeasync_init.hperhaps?Regards,
Nigel
Hi Nigel,
That's what I thought too at first. But I guess there could be other
users of these async functions than the initcalls.
It looks like out of order workqueues that can synchronize themselves.
It seems a bit more costly than workqueues (since the concept comes along
a loop of creation/destroying of several threads) but more efficient for the same reason as the number of active threads adapts to the number of CPU and jobs to perform. The purpose is not the same. But I guess it's more reliable
than workqueues for pathes that require mutiple parallel jobs which need
to locally synchronize on critical pathes.
Quoted 6 lines · On Mon, 05 Jan 2009 17:01:34 +1100 Nigel Cunningham…
Hi Arjan.
Great work, and valuable too. I'm just wondering if the header file name "
async.h" is a little too generic? Something likeasync_init.hperhaps?
while the users I have in mind are during init, the concept and infrastructure of asynchronous function calls is by no means limited to that use case... I hope there are many other uses for this :)
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Hi.
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines · On Mon, 05 Jan 2009 17:01:34 +1100 Nigel Cunningham…
Hi Arjan.
Quoted 8 lines
Quoted 4 lines
Great work, and valuable too. I'm just wondering if the header file name "
async.h" is a little too generic? Something likeasync_init.hperhaps?while the users I have in mind are during init, the concept and infrastructure of asynchronous function calls is by no means limited to that use case... I hope there are many other uses for this :)
Okee doke. How about async_fn or something like that then? It's just that there's async I/O and no doubt other async stuff I haven't thought of. Still, it's a relatively minor point.
Night!
Nigel
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines · On Mon, 05 Jan 2009 17:01:34 +1100 Nigel Cunningham…
Hi Arjan.
Quoted 8 lines
Quoted 4 lines
Great work, and valuable too. I'm just wondering if the header file name "
async.h" is a little too generic? Something likeasync_init.hperhaps?while the users I have in mind are during init, the concept and infrastructure of asynchronous function calls is by no means limited to that use case... I hope there are many other uses for this :)
This can be used for shutdown too. Some drivers like sata and serio shutdown time is long.
Thanks,
Shaohua