x86/init: Linux linker tables
Subsystems (12)
cover x86/init: Linux linker tables
From: "Luis R. Rodriguez" <[email protected]>
A long time ago in a galaxy far,
far away...
Konrad Rzeszutek Wilk posted patches which eventually got merged to help with modularizing the IOMMUs we have on x86 [0]. This work was done due to the complex relationship that exists on IOMMUs and the requirements on careful execution. The solution also provided a mechanism which jettisoned unused IOMMUs during run-time.
During review, even though the code was merged, hpa did note that we tend to encounter this type of problem "often enough that we should implement a generic facility for it" [1], hpa acknowledged that it obviously has to be based on sections and even noted that perhaps we might be able in the future to automate its creation. He noted that the gPXE folks had done just this with linker tables and suggested that "presumably we'd need a few different flavors for init tables and so on, but this would make it a generic mechanism."
The IOMMU code got merged and this was left on someone's mental backburner. I've had an itch to scratch recently to try to avoid issues which are possible if one does not jettison other code carefully due to the large complexity of implicit dependencies of certain code on x86 in particular with possible dead code on x86 due to paravirtualization, and the IOMMU jettison strategy turned out to be my favorite solution so far. I've taken on hpa's suggestions from back in the day to review gPXE's solution to see if we could embrace it on Linux for a generic section solution to help jettison code carefully.
What this patch set does exactly:
This RFC patch set attempts to add support for such a generic solution. In the end, it turns out that the best solution possible was the best of both worlds: a combination of what Konrad had implemented in addition to what Michael Brown had implemented on the gXPE front. The IOMMU solution enables simple semantic annotations for dependency relationships, this however requires a run time sort. The gPXE solution grants the option to simply sort at build time. One of gPXE's solution primary goals however was also to help avoid bit-rot on code that's possible from #ifdef'ery. The Linux linker table solution enables developers to pick and choose what they need, with linker tables being the simplest solution. Contrary to gPXE which strives to force compilation of all linker table solutions we let developers pick *when* they want this as part of their solution. As can be seen from the suggested x86 init specific use of linker tables proposed you can also take advantage of both, linker sorting, optional compilation when needed (at developer's discretion), and even careful semantics annotation for dependency / relationship annotations. Although the x86 init solution here is heavily inspired by the IOMMU solution it diverges with strong semantics, and a new optional subarchitecture annotation. Sorting of init sequences is structure specific, as such each subsystem must defing their own solution unless semantics could be shared. I considered sharing semantics but in the end this proved pointless so this keeps things separate. A series of changes were made to the x86 init sequence in contrast to the IOMMU solution to be *extremely pedantic* on semantics, review of this changes can be studied on the table-init tree [2].
Quick review of gPXE's solution and prospects on further changes:
In my review from gPXE's solution it was not clear what hpa meant by gXPE folks having automated this process, they actually use linker tables all around, forcing compilation of *everything* and just do linking of enabled features at link time. You still need to build linker tables on your own. What I do see more potential for in the future is enabling to evolve stronger semantics over time, and this would also be subsystem specific. This will be evident in this patch set on the x86 init use of linker tables. I also see potential in strenghtening semantics for linker sorting, any of these types of features however would impose requiring newer binutils. For instance, gPXE's linker solution currently relies on SORT(), that defaults to SORT_BY_NAME(). This sorts lexicographically, gPXE's solution uses two digits to enable SORT_BY_NAME()'s lexicographical sort to sort orde by numeric priority. Since one is in control of order-level numbers one can provide guarantee that this sort should work as intended, however binutils also now has a SORT_BY_INIT_PRIORITY() which sorts specifically based on digits. SORT_BY_INIT_PRIORITY() was designed specifically for init_array sections though. Refer to the userspace mockup solution table-init git tree [2] commit 6deba47ee1ad461e90 for more details on this. One thing I can envision to help here further are prospects for future linker enhancements, these however must be carefully considered in light of possible requirements of newer binutils and also of compatibility with other toolchains. For now we resort to the good 'ol SORT() which even Linux has made use of for ages already. In that regards here, nothing new here. gPXE folks however did find some fun ICC compatiblity issues, and have also developed fixes for them, these were obviously carried over but should be reviewed carefully. Lastly, I should point out that essentially what we're developing are different forms loose and strong semantics -- in the most complex form what we really are after are feature graphs. Mauro explained to me that for media driver they needed to build a feature graphs, IMHO that could be a next level of strong semantics we might want to consider. For now though the combination of gPXE's linker-table solution based on linker order-level sort, and our old IOMMU init solution and its simple dependency map (and possible extensions, see subarchitecture) should likely suffice as a light weight solution for where we need semantics for at least on the x86 init path.
Motivation and possible enhancements:
To understand what made me tick to work on this feel free to read the dead code concerns with paravirtualization posts I've made, which also go into and detail Xen's alternative entry point [4] [5]. That's not the end of possibilities to help address to possible "dead code" on Linux, I have other ideas but I have a bit more work to do before publishing anything about it. If anything -- later work should likely supplement this solution further.
Currently the earliest I was able to make use of boot_params on x86-64 for linker tables was after load_idt(), so I've decided to stick with the earliest init call for x86 init sequences starting on x86_64_start_reservation(). However, if we're able to make use of boot_params earlier (help appreciated), in particular just boot_params.hdr.hardware_subarch we should be able to make clearer and careful annotations on early x86-64 init code. Using the boot_params.hdr.hardware_subarch and requiring clear subarchitecture support annotations on early init code should provide a *proactive* means to avoid issues such as the cr4 shadow oversight [6] which later caused crashes on Xen. The latest similar type of issue is when KAsan was introduced, after it was integrated I suspected KAsan would probably break if enabled on Xen, I reported this in March 2015 [7] and Andrey confirmed this might be the case but since he wanted it enabled for allyesconfig and allmodconfig he could not think of a clean way to disable it or address support for it then [8]. During the development of this patch set I confirmed KAsan crashes Xen dom0 and therefore should obviously crash Xen PV guests as well. Using the same kernel KAsan still worked on KVM and bare-metal. Provided we had early access to boot_params.hdr.hardware_subarch, in x86_64_start_kernel() we could annotate kasan_early_init() and friends not supported on Xen, however since we don't have a way to disable KAsan at run time at this time we wouldn't have any other option but to crash early. Because of this though we should perhaps just consider disabling KAsan for Xen configs and KAsan folks might just have to bite the bullet. If KAsan folks are gung-ho about not disabling KAsan when Xen is enabled and if its easier to add support to disable KAsan at run time than it is to add KAsan support for Xen one alternative might be to use linker tables to annotate this and disable KAsan at run time for Xen.
One of the points of the use of the x86 linker table solution and specific semantics there in is to *force* developers to think about requirements on x86 carefully. So for instance by requiring itemizing supported subarchitectures *early* on in development we should be able to avoid proactively issues such as the crash due to the cr4 shadow changes not added on the Xen entry path, and also missing the requirement to develop a solution for Kasan for Xen. The proposed x86 linker table solution is not the first to use boot_params.hdr.hardware_subarch, its first use was actually on i386, and for lguest. We take advantage of it to avoid further extending pv_ops, and *actually* to see if we could simplify pv_ops over time further. This patch set also includes a renaming of paravirt_enabled() to paravirt_legacy()
An unexpected long term side goal which *might* be possible due to linker tables on x86 is to help unify the different C entry points for Linux x86-64. I've taken a stab at it after these patches [9] but it fails on Xen, likely because the stack is not set up right due to the different calls / argument requirements. If this is desirable folks could take a look and perhaps help on that front.
Where to get code alternatively and testing:
All of these patches are RFCs, if anything the only patches worth consdering merging sooner rather than later might be "paravirt: rename paravirt_enabled to paravirt_legacy" and "x86/boot: add BIT() to boot/bitops.h". After review I can send those separately in patch form if agreeable. Since these are all just RFCs I've based these patches on top of Linus' v4.4-rc5. If you want all patches in one file you can get them here [10], and a respective linux-next version which applies on top of next-20151215 here [11]. I also have two git trees set up with branches for this code, one based on Linus' tree [12] and another based on linux-next next-20151215 [13]. I'll see what issues zero-day bot testing finds out. I've just run time tested this on x86-64 bare metal, KVM, and Xen dom0 so far.
[0] https://marc.info/?l=linux-kernel&m=128284562303565&w=2
[1] https://marc.info/?l=linux-kernel&m=128285216913266&w=2
[2] https://github.com/mcgrof/table-init/
[3] https://github.com/mcgrof/table-init/commit/6deba47ee1ad461e90e0fbba226a535cfc1c58f3
[4] http://www.do-not-panic.com/2015/12/avoiding-dead-code-pvops-not-silver-bullet.html
[5] http://www.do-not-panic.com/2015/12/xen-and-x86-linux-zero-page.html
[6] http://lists.xenproject.org/archives/html/xen-devel/2015-02/msg02742.html
[7] http://lkml.kernel.org/r/CAB=NE6Xs5fepzNtymzT4CueeJZ0KMPETpda114DpL4eMtDswtw@mail.gmail.com
[8] http://lkml.kernel.org/r/[email protected]
[9] http://drvbp1.linux-foundation.org/~mcgrof/patches/2015/12/15/x86-merge-x86-init-v1.patch
[10] http://drvbp1.linux-foundation.org/~mcgrof/patches/2015/12/15/pend-20151215-rfc-v1-linker-tables.patch
[11] http://drvbp1.linux-foundation.org/~mcgrof/patches/2015/12/15/pend-next-20151215-rfc-v1-linker-tables.patch
[12] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux.git/log/?h=20151215-rfc-v1-linker-tables
[13] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20151215-rfc-v1-linker-tables
Luis R. Rodriguez (8):
paravirt: rename paravirt_enabled to paravirt_legacy tables.h: add linker table support x86/boot: add BIT() to boot/bitops.h x86/init: add linker table support x86/init: move ebda reservations into linker table x86/init: use linker table for i386 early setup x86/init: user linker table for ce4100 early setup x86/init: use linker table for mid early setup
Documentation/kbuild/makefiles.txt | 19 ++
arch/x86/Kconfig.debug | 47 ++++
arch/x86/boot/bitops.h | 2 +
arch/x86/boot/boot.h | 2 +-
arch/x86/include/asm/bios_ebda.h | 2 -
arch/x86/include/asm/paravirt.h | 4 +-
arch/x86/include/asm/paravirt_types.h | 11 +-
arch/x86/include/asm/processor.h | 2 +-
arch/x86/include/asm/setup.h | 12 -
arch/x86/include/asm/x86_init.h | 1 +
arch/x86/include/asm/x86_init_fn.h | 267 ++++++++++++++++++++
arch/x86/kernel/Makefile | 4 +-
arch/x86/kernel/apm_32.c | 2 +-
arch/x86/kernel/asm-offsets.c | 2 +-
arch/x86/kernel/cpu/intel.c | 2 +-
arch/x86/kernel/cpu/microcode/core.c | 2 +-
arch/x86/kernel/dbg-tables/Makefile | 18 ++
arch/x86/kernel/dbg-tables/alpha.c | 10 +
arch/x86/kernel/dbg-tables/beta.c | 18 ++
arch/x86/kernel/dbg-tables/delta.c | 10 +
arch/x86/kernel/dbg-tables/gamma.c | 18 ++
arch/x86/kernel/dbg-tables/gamma.h | 3 +
arch/x86/kernel/{head.c => ebda.c} | 6 +-
arch/x86/kernel/head32.c | 22 +-
arch/x86/kernel/head64.c | 4 +-
arch/x86/kernel/init.c | 55 +++++
arch/x86/kernel/kvm.c | 9 +-
arch/x86/kernel/paravirt.c | 2 +-
arch/x86/kernel/sort-init.c | 114 +++++++++
arch/x86/kernel/tboot.c | 2 +-
arch/x86/kernel/vmlinux.lds.S | 25 ++
arch/x86/lguest/boot.c | 2 +-
arch/x86/mm/dump_pagetables.c | 2 +-
arch/x86/platform/ce4100/ce4100.c | 4 +-
arch/x86/platform/intel-mid/intel-mid.c | 4 +-
arch/x86/tools/relocs.c | 1 +
arch/x86/xen/enlighten.c | 2 +-
drivers/pnp/pnpbios/core.c | 2 +-
include/linux/tables.h | 421 ++++++++++++++++++++++++++++++++
scripts/Makefile.build | 4 +-
scripts/Makefile.clean | 1 +
scripts/Makefile.lib | 12 +
42 files changed, 1091 insertions(+), 61 deletions(-)
create mode 100644 arch/x86/include/asm/x86_init_fn.h
create mode 100644 arch/x86/kernel/dbg-tables/Makefile
create mode 100644 arch/x86/kernel/dbg-tables/alpha.c
create mode 100644 arch/x86/kernel/dbg-tables/beta.c
create mode 100644 arch/x86/kernel/dbg-tables/delta.c
create mode 100644 arch/x86/kernel/dbg-tables/gamma.c
create mode 100644 arch/x86/kernel/dbg-tables/gamma.h
rename arch/x86/kernel/{head.c => ebda.c} (94%)
create mode 100644 arch/x86/kernel/init.c
create mode 100644 arch/x86/kernel/sort-init.c
create mode 100644 include/linux/tables.hSignature · 1 line
2.6.2
RFC 1/8 paravirt: rename paravirt_enabled to paravirt_legacy
From: "Luis R. Rodriguez" <[email protected]>
paravirt_enabled conveys the idea that if this is set or if paravirt_enabled() returns true you are in a paravirtualized environment. This is not true, this is really only useful to determine if you have a paravirtualization hypervisor that uses supports legacy paravirtualized guests. At run time, this tells us if we've booted into a Linux guest with support for legacy paravirtualized guests.
Make emphasis about this by renaming the member and helper. While at it, make the type bool, and document it therefore avoiding special cased comments trying to explain it.
The rename is done using the following Coccinelle SmPL patch:
@ rename_paravirt_enabled @
@@
-paravirt_enabled()
+paravirt_legacy()
@ rename_pv_info_pv_enabled @
@@
-pv_info.paravirt_enabled
+pv_info.paravirt_legacy
@ is_pv @
identifier pv;
@@
struct pv_info pv = {
};
@ rename_struct_pv_enabled depends on is_pv @
identifier is_pv.pv;
expression val;
@@
struct pv_info pv = {
- .paravirt_enabled
+ .paravirt_legacy
= val, };
--- arch/x86/include/asm/paravirt.h | 4 ++-- arch/x86/include/asm/paravirt_types.h | 11 ++++++++++- arch/x86/include/asm/processor.h | 2 +- arch/x86/kernel/apm_32.c | 2 +- arch/x86/kernel/asm-offsets.c | 2 +- arch/x86/kernel/cpu/intel.c | 2 +- arch/x86/kernel/cpu/microcode/core.c | 2 +- arch/x86/kernel/head.c | 2 +- arch/x86/kernel/kvm.c | 9 +-------- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/tboot.c | 2 +- arch/x86/lguest/boot.c | 2 +- arch/x86/mm/dump_pagetables.c | 2 +- arch/x86/xen/enlighten.c | 2 +- drivers/pnp/pnpbios/core.c | 2 +- 15 files changed, 25 insertions(+), 23 deletions(-)
arch/x86/include/asm/paravirt.h +2 -2
@@ -14,9 +14,9 @@#include <linux/types.h> #include <linux/cpumask.h> -static inline int paravirt_enabled (void ) +static inline bool paravirt_legacy (void ) { -return pv_info.paravirt_enabled; +return pv_info.paravirt_legacy; } static inline void load_sp0(struct tss_struct *tss,
arch/x86/include/asm/paravirt_types.h +10 -1
@@ -61,6 +61,15 @@ struct paravirt_callee_save { };/* general info */ + +/** +* struct pv_info - hypervisor information +* +* @paravirt_legacy: true if this hypervisor supports legacy +* paravirtualized guests. Examples of features that +* characterize legacy paravirtualized guests are +* things such as the need for APM, PNP BIOS. +*/ struct pv_info {unsigned int kernel_rpl;int shared_kernel_pmd;
@@ -69,7 +78,7 @@ struct pv_info {u16 extra_user_64bit_cs;/* __USER_CS if none */ #endif -int paravirt_enabled; +bool paravirt_legacy;const char *name; };
arch/x86/include/asm/processor.h +1 -1
@@ -471,7 +471,7 @@ static inline unsigned long current_top_of_stack(void)#include <asm/paravirt.h> #else #define __cpuid native_cpuid -#define paravirt_enabled () 0 +#define paravirt_legacy () falsestatic inline void load_sp0 (struct tss_struct *tss,struct thread_struct *thread)
arch/x86/kernel/apm_32.c +1 -1
@@ -2267,7 +2267,7 @@ static int __init apm_init(void)dmi_check_system (apm_dmi_table); -if (apm_info.bios.version ==0 ||paravirt_enabled () ||machine_is_olpc ()) { +if (apm_info.bios.version ==0 ||paravirt_legacy () ||machine_is_olpc ()) {printk (KERN_INFO "apm: BIOS not found.\n" );return -ENODEV ; }
arch/x86/kernel/asm-offsets.c +1 -1
@@ -59,7 +59,7 @@ void common(void) {#ifdef CONFIG_PARAVIRT BLANK (); -OFFSET (PARAVIRT_enabled, pv_info, paravirt_enabled); +OFFSET (PARAVIRT_legacy, pv_info, paravirt_legacy);OFFSET (PARAVIRT_PATCH_pv_cpu_ops, paravirt_patch_template, pv_cpu_ops);OFFSET (PARAVIRT_PATCH_pv_irq_ops, paravirt_patch_template, pv_irq_ops);OFFSET (PV_IRQ_irq_disable, pv_irq_ops, irq_disable);
arch/x86/kernel/cpu/intel.c +1 -1
@@ -220,7 +220,7 @@ static void intel_workarounds(struct cpuinfo_x86 *c)* The Quark is also family 5, but does not have the same bug. */ clear_cpu_bug (c,X86_BUG_F00F ); -if (!paravirt_enabled () && c->x86 ==5 && c->x86_model <9 ) { +if (!paravirt_legacy () && c->x86 ==5 && c->x86_model <9 ) {static int f00f_workaround_enabled;set_cpu_bug (c,X86_BUG_F00F );
arch/x86/kernel/cpu/microcode/core.c +1 -1
@@ -630,7 +630,7 @@ int __init microcode_init(void)struct cpuinfo_x86 *c = &boot_cpu_data;int error; -if (paravirt_enabled () || dis_ucode_ldr) +if (paravirt_legacy () || dis_ucode_ldr)return -EINVAL ;if (c->x86_vendor ==X86_VENDOR_INTEL )
arch/x86/kernel/head.c +1 -1
@@ -38,7 +38,7 @@ void __init reserve_ebda_region(void)* that the paravirt case can handle memory setup * correctly, without our help. */ -if (paravirt_enabled ()) +if (paravirt_legacy ())return ;/* end of low (conventional) memory */
arch/x86/kernel/kvm.c +1 -8
@@ -283,14 +283,7 @@ NOKPROBE_SYMBOL(do_async_page_fault);static void __init paravirt_ops_setup (void ) { pv_info.name ="KVM" ; - -/* -* KVM isn't paravirt in the sense of paravirt_enabled. A KVM -* guest kernel works like a bare metal kernel with additional -* features, and paravirt_enabled is about features that are -* missing. -*/ - pv_info.paravirt_enabled =0 ; + pv_info.paravirt_legacy = false;if (kvm_para_has_feature (KVM_FEATURE_NOP_IO_DELAY )) pv_cpu_ops.io_delay = kvm_io_delay;
arch/x86/kernel/paravirt.c +1 -1
@@ -312,7 +312,7 @@ enum paravirt_lazy_mode paravirt_get_lazy_mode(void)struct pv_info pv_info = { .name ="bare hardware" , - .paravirt_enabled =0 , + .paravirt_legacy = false, .kernel_rpl =0 , .shared_kernel_pmd =1 ,/* Only used when CONFIG_X86_PAE is set */
arch/x86/kernel/tboot.c +1 -1
@@ -75,7 +75,7 @@ void __init tboot_probe(void) }/* only a natively booted kernel should be using TXT */ -if (paravirt_enabled ()) { +if (paravirt_legacy ()) {pr_warning ("non-0 tboot_addr but pv_ops is enabled\n" );return ; }
arch/x86/lguest/boot.c +1 -1
@@ -1409,7 +1409,7 @@ __init void lguest_init(void)/* We're under lguest. */ pv_info.name ="lguest" ;/* Paravirt is enabled. */ - pv_info.paravirt_enabled =1 ; + pv_info.paravirt_legacy = true;/* We're running at privilege level 1, not 0 as normal. */ pv_info.kernel_rpl =1 ;/* Everyone except Xen runs with this set. */
arch/x86/mm/dump_pagetables.c +1 -1
@@ -365,7 +365,7 @@ static inline bool is_hypervisor_range(int idx)* ffff800000000000 - ffff87ffffffffff is reserved for * the hypervisor. */ -return paravirt_enabled () && +return paravirt_legacy () && (idx >=pgd_index (__PAGE_OFFSET) -16 ) && (idx <pgd_index (__PAGE_OFFSET)); }
arch/x86/xen/enlighten.c +1 -1
@@ -1186,7 +1186,7 @@ static unsigned xen_patch(u8 type, u16 clobbers, void *insnbuf, }static const struct pv_info xen_info__initconst = { - .paravirt_enabled =1 , + .paravirt_legacy = true, .shared_kernel_pmd =0 , #ifdef CONFIG_X86_64
drivers/pnp/pnpbios/core.c +1 -1
@@ -521,7 +521,7 @@ static int __init pnpbios_init(void)int ret;if (pnpbios_disabled ||dmi_check_system (pnpbios_dmi_table) || -paravirt_enabled ()) { +paravirt_legacy ()) {printk (KERN_INFO "PnPBIOS: Disabled\n" );return -ENODEV ; }
Signature · 1 line
2.6.2
Re: [RFC v1 1/8] paravirt: rename paravirt_enabled to paravirt_legacy
Quoted 1 line · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
Hey Luis,
Sorry for the long time to respond..
Quoted 5 lines
paravirt_enabledconveys the idea that if this is set or ifparavirt_enabled()returns true you are in a paravirtualized environment. This is not true, this is really only useful to determine if you have a paravirtualization hypervisor that uses
s/users//
Quoted 3 lines
supports legacy paravirtualized guests. At run time, this tells us if we've booted into a Linux guest with support for legacy paravirtualized guests.
I know I suggested the name but when I look at the words 'paravirtualized legacy' immediately I thought about legacy PV drivers.. which we do not have in the code.
The description:
Quoted 3 lines
+ * paravirtualized guests. Examples of features that
+ * characterize legacy paravirtualized guests are
+ * things such as the need for APM, PNP BIOS.
Explains it very nicely.
Perhaps (And sorry for that) it should be just called
legacy_platform() ?
Orancient_platform() ?
Since that would lead folks to immediately think of things that existed long time ago - such as APM or PNP BIOS.
Other suggestions would be to piggyback on Microsoft pushing the option in the BIOS to have an "Legacy Free" option (PS/2, PnP, serial port, parallel port - all are disabled):
https://en.wikipedia.org/wiki/Legacy-free_PC
Perhaps:
legacy_free() ?
Thank you!
Quoted 282 lines
Make emphasis about this by renaming the member and helper. While at it, make the type bool, and document it therefore avoiding special cased comments trying to explain it.
The rename is done using the following Coccinelle SmPL patch:
@
rename_paravirt_enabled@
@@(quoted diff) +1
-1-paravirt_enabled() +paravirt_legacy()@
rename_pv_info_pv_enabled@
@@(quoted diff) +1
-1-pv_info.paravirt_enabled +pv_info.paravirt_legacy@
is_pv@
identifier pv;
@@
structpv_infopv = {
};@
rename_struct_pv_enableddepends onis_pv@
identifier is_pv.pv;
expression val;
@@struct
pv_infopv = {(quoted diff) +1
-1- .paravirt_enabled + .paravirt_legacy= val, };Generated-by Coccinelle SmPLSuggested-by Konrad Rzeszutek Wilk <[email protected]>cc Konrad Rzeszutek Wilk <[email protected]>Signed-off-by Luis R. Rodriguez <[email protected]>--- arch/x86/include/asm/paravirt.h | 4 ++-- arch/x86/include/asm/paravirt_types.h | 11 ++++++++++- arch/x86/include/asm/processor.h | 2 +- arch/x86/kernel/apm_32.c | 2 +- arch/x86/kernel/asm-offsets.c | 2 +- arch/x86/kernel/cpu/intel.c | 2 +- arch/x86/kernel/cpu/microcode/core.c | 2 +- arch/x86/kernel/head.c | 2 +- arch/x86/kernel/kvm.c | 9 +-------- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/tboot.c | 2 +- arch/x86/lguest/boot.c | 2 +- arch/x86/mm/dump_pagetables.c | 2 +- arch/x86/xen/enlighten.c | 2 +- drivers/pnp/pnpbios/core.c | 2 +- 15 files changed, 25 insertions(+), 23 deletions(-)arch/x86/include/asm/paravirt.h +2
-2@@ -14,9 +14,9 @@#include <linux/types.h> #include <linux/cpumask.h> -static inline int paravirt_enabled (void ) +static inline bool paravirt_legacy (void ) { -return pv_info.paravirt_enabled; +return pv_info.paravirt_legacy; } static inline void load_sp0(struct tss_struct *tss,arch/x86/include/asm/paravirt_types.h +10
-1@@ -61,6 +61,15 @@ struct paravirt_callee_save { };/* general info */ + +/** +* struct pv_info - hypervisor information +* +* @paravirt_legacy: true if this hypervisor supports legacy +* paravirtualized guests. Examples of features that +* characterize legacy paravirtualized guests are +* things such as the need for APM, PNP BIOS. +*/ struct pv_info {unsigned int kernel_rpl;int shared_kernel_pmd;@@ -69,7 +78,7 @@ struct pv_info {u16 extra_user_64bit_cs;/* __USER_CS if none */ #endif -int paravirt_enabled; +bool paravirt_legacy;const char *name; };arch/x86/include/asm/processor.h +1
-1@@ -471,7 +471,7 @@ static inline unsigned long current_top_of_stack(void)#include <asm/paravirt.h> #else #define __cpuid native_cpuid -#define paravirt_enabled () 0 +#define paravirt_legacy () falsestatic inline void load_sp0 (struct tss_struct *tss,struct thread_struct *thread)arch/x86/kernel/apm_32.c +1
-1@@ -2267,7 +2267,7 @@ static int __init apm_init(void)dmi_check_system (apm_dmi_table); -if (apm_info.bios.version ==0 ||paravirt_enabled () ||machine_is_olpc ()) { +if (apm_info.bios.version ==0 ||paravirt_legacy () ||machine_is_olpc ()) {printk (KERN_INFO "apm: BIOS not found.\n" );return -ENODEV ; }arch/x86/kernel/asm-offsets.c +1
-1@@ -59,7 +59,7 @@ void common(void) {#ifdef CONFIG_PARAVIRT BLANK (); -OFFSET (PARAVIRT_enabled, pv_info, paravirt_enabled); +OFFSET (PARAVIRT_legacy, pv_info, paravirt_legacy);OFFSET (PARAVIRT_PATCH_pv_cpu_ops, paravirt_patch_template, pv_cpu_ops);OFFSET (PARAVIRT_PATCH_pv_irq_ops, paravirt_patch_template, pv_irq_ops);OFFSET (PV_IRQ_irq_disable, pv_irq_ops, irq_disable);arch/x86/kernel/cpu/intel.c +1
-1@@ -220,7 +220,7 @@ static void intel_workarounds(struct cpuinfo_x86 *c)* The Quark is also family 5, but does not have the same bug. */ clear_cpu_bug (c,X86_BUG_F00F ); -if (!paravirt_enabled () && c->x86 ==5 && c->x86_model <9 ) { +if (!paravirt_legacy () && c->x86 ==5 && c->x86_model <9 ) {static int f00f_workaround_enabled;set_cpu_bug (c,X86_BUG_F00F );arch/x86/kernel/cpu/microcode/core.c +1
-1@@ -630,7 +630,7 @@ int __init microcode_init(void)struct cpuinfo_x86 *c = &boot_cpu_data;int error; -if (paravirt_enabled () || dis_ucode_ldr) +if (paravirt_legacy () || dis_ucode_ldr)return -EINVAL ;if (c->x86_vendor ==X86_VENDOR_INTEL )arch/x86/kernel/head.c +1
-1@@ -38,7 +38,7 @@ void __init reserve_ebda_region(void)* that the paravirt case can handle memory setup * correctly, without our help. */ -if (paravirt_enabled ()) +if (paravirt_legacy ())return ;/* end of low (conventional) memory */ arch/x86/kernel/kvm.c +1
-8@@ -283,14 +283,7 @@ NOKPROBE_SYMBOL(do_async_page_fault);static void __init paravirt_ops_setup (void ) { pv_info.name ="KVM" ; - -/* -* KVM isn't paravirt in the sense of paravirt_enabled. A KVM -* guest kernel works like a bare metal kernel with additional -* features, and paravirt_enabled is about features that are -* missing. -*/ - pv_info.paravirt_enabled =0 ; + pv_info.paravirt_legacy = false;if (kvm_para_has_feature (KVM_FEATURE_NOP_IO_DELAY )) pv_cpu_ops.io_delay = kvm_io_delay;arch/x86/kernel/paravirt.c +1
-1@@ -312,7 +312,7 @@ enum paravirt_lazy_mode paravirt_get_lazy_mode(void)struct pv_info pv_info = { .name ="bare hardware" , - .paravirt_enabled =0 , + .paravirt_legacy = false, .kernel_rpl =0 , .shared_kernel_pmd =1 ,/* Only used when CONFIG_X86_PAE is set */ arch/x86/kernel/tboot.c +1
-1@@ -75,7 +75,7 @@ void __init tboot_probe(void) }/* only a natively booted kernel should be using TXT */ -if (paravirt_enabled ()) { +if (paravirt_legacy ()) {pr_warning ("non-0 tboot_addr but pv_ops is enabled\n" );return ; }arch/x86/lguest/boot.c +1
-1@@ -1409,7 +1409,7 @@ __init void lguest_init(void)/* We're under lguest. */ pv_info.name ="lguest" ;/* Paravirt is enabled. */ - pv_info.paravirt_enabled =1 ; + pv_info.paravirt_legacy = true;/* We're running at privilege level 1, not 0 as normal. */ pv_info.kernel_rpl =1 ;/* Everyone except Xen runs with this set. */ arch/x86/mm/dump_pagetables.c +1
-1@@ -365,7 +365,7 @@ static inline bool is_hypervisor_range(int idx)* ffff800000000000 - ffff87ffffffffff is reserved for * the hypervisor. */ -return paravirt_enabled () && +return paravirt_legacy () && (idx >=pgd_index (__PAGE_OFFSET) -16 ) && (idx <pgd_index (__PAGE_OFFSET)); }arch/x86/xen/enlighten.c +1
-1@@ -1186,7 +1186,7 @@ static unsigned xen_patch(u8 type, u16 clobbers, void *insnbuf, }static const struct pv_info xen_info__initconst = { - .paravirt_enabled =1 , + .paravirt_legacy = true, .shared_kernel_pmd =0 , #ifdef CONFIG_X86_64drivers/pnp/pnpbios/core.c +1
-1@@ -521,7 +521,7 @@ static int __init pnpbios_init(void)int ret;if (pnpbios_disabled ||dmi_check_system (pnpbios_dmi_table) || -paravirt_enabled ()) { +paravirt_legacy ()) {printk (KERN_INFO "PnPBIOS: Disabled\n" );return -ENODEV ; }Signature · 1 line
2.6.2
Re: [RFC v1 1/8] paravirt: rename paravirt_enabled to paravirt_legacy
Quoted 12 lines · Konrad Rzeszutek Wilk
Quoted 1 line · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
Hey Luis,
Sorry for the long time to respond..
Quoted 5 lines
paravirt_enabledconveys the idea that if this is set or ifparavirt_enabled()returns true you are in a paravirtualized environment. This is not true, this is really only useful to determine if you have a paravirtualization hypervisor that uses
Quoted 8 lines
s/users//
Quoted 3 lines
supports legacy paravirtualized guests. At run time, this tells us if we've booted into a Linux guest with support for legacy paravirtualized guests.
I know I suggested the name but when I look at the words 'paravirtualized legacy' immediately I thought about legacy PV drivers.. which we do not have in the code.
OK...
Quoted 6 lines
The description:
Quoted 3 lines
+ * paravirtualized guests. Examples of features that + * characterize legacy paravirtualized guests are + * things such as the need for APM, PNP BIOS.
Quoted 8 lines
Explains it very nicely.
Perhaps (And sorry for that) it should be just called
legacy_platform()?Or
ancient_platform()?
No, we need to prefix this as its paravirt related, I've given this silly little rename quite a bit of thought and have decided to stick to paravirt_legacy() and just adding paravirt_legacy_free() as a separate helper. The clarification can simply be placed through kdoc form in documentation. More on this below.
Quoted 2 lines
Since that would lead folks to immediately think of things that existed long time ago - such as APM or PNP BIOS.
Right.
Quoted 4 lines
Other suggestions would be to piggyback on Microsoft pushing the option in the BIOS to have an "Legacy Free" option (PS/2, PnP, serial port, parallel port - all are disabled):
https://en.wikipedia.org/wiki/Legacy-free_PC
Interesting, yes I like this as it has stronger semantics. I looked further and it seems this concept of "legacy free" comes from the PC system design guide [0] by both Microsoft and Intel. The latest of such documents and the latest definitions of a PC 2001 contains the concept of "legacy free" and defines it well on Page 50 [1] under the "PC 2001 requirements". It notes:
----
The basic goal for the legacy-free system requirements is that the operating system and devices do not use any of the following
* ISA slots or devices
* Legacy FDC
* PS/2, serial, parallel, and game ports
Revisions to the ACPI specification provide a mechanism that allows the BIOS to report whether a system provides the services of these components and interfaces. If the BIOS reports that the system is legacy free, the system must meet the requirements provided in this section.
----
[0] http://tech-insider.org/windows/research/2000/1102.html
[1] http://tech-insider.org/windows/research/acrobat/001102/03sys-2001.pdf
Quoted 3 lines
Perhaps:
legacy_free()?
That's a negation of the question "is legacy?", so this would be a separate call. Also this doesn't have a prefix.
Note that what is legacy on x86 today may be very legacy tomorrow and later on there may be other notions of legacy... so sticking loosely to at least the "PC 2001" definition seems to make sense here and would limit scope and date the notion of "legacy" here for x86 and for this paravirt hook. For now I don't see a need to include a date check though as its a paravirt check anyway, but at least the definition needs to limit the scope in its documentation for now.
This particular check is *if your hypervisor* supports legacy PC 2001 systems, so we do in the end need a prefix given the question as to whether or not the kernel can determine you are on a legacy free system or not is a separate possible question. The above definition seems to indicate that ACPI should allow the BIOS to reoport if a system is legacy free or not, so if such mechanisms exist, code that currently has sprinkled paravirt_enabled() (we'll rename this soon) could also have such checks for legacy free as well. Once, and if, such system check is available perhaps we should prevent a legacy free hypervisor from booting on a legacy system.
Likewise with such system check present if a PV legacy capable hypervisor boots a legacy free system -- the code checks for subsystems that are legacy would avoid running given that although pavarit_supports_legacy() gives true, is_system_legacy() would give false and avoid running.
Anyway, unless I hear of better suggestions I'm going to stick to paravirt_legacy() and add paravirt_legacy_free(), and just expand on the documentation to help clarify this is not for PV legacy drivers.
Quoted 1 line
Thank you!
Luis
RFC 2/8 tables.h: add linker table support
In reply to Luis R. RodriguezFrom: "Luis R. Rodriguez" <[email protected]>
This adds Linux table support to Linux, based on Michael Brown's gPXE's linker table solution. Linker tables enable an extremely light weight linker build time solution for feature ordering and selection, this can help to both simplify init sequences in a generic fashion and helps avoiding code bit-rotting when desirable.
Bit rotting avoidance is enabled by *granting the option* to force compilation of code and only enable linking object code in when specific features have been enabled. It accomplishes this by using linker sections, the lightweight feature ordering of code is enabled by taking advantage of the old binutils ld SORT() on features specific sections.
Contrary to gPXE's solution, which strives to force compilation of all features, Linux' solution strives to encourage usage of linker tables to force compilation of code *only on designated code*. Linker tables can be used without requiring one to force code compilation of features that use it though, to take advantage of the simplification of init sequences.
Linux code that uses linker tables *and* wishes to always require compilation but selectively linking must use new kbuild target object that helps accomplishes this, table-y. Linux code that uses linker tables but does not want to require forcing compilation can use the good 'ol obj-y targets.
Used commit c23508d796 from gPXE upstream [0] as the starting point for development and evaluation of Linux's integration, further changes made and evaluation of different linker table options for Linux are available on the table-init userspace mockup solution [1].
[0] git://git.etherboot.org/scm/gpxe.git
[1] https://github.com/mcgrof/table-init.git
--- Documentation/kbuild/makefiles.txt | 19 ++ include/linux/tables.h | 421 +++++++++++++++++++++++++++++++++++++ scripts/Makefile.build | 4 +- scripts/Makefile.clean | 1 + scripts/Makefile.lib | 12 ++ 5 files changed, 455 insertions(+), 2 deletions(-) create mode 100644 include/linux/tables.h
Documentation/kbuild/makefiles.txt +19 -0
@@ -1088,6 +1088,25 @@ When kbuild executes, the following steps are followed (roughly): In this example, extra-y is used to list object files that shall be built, but shall not be linked as part of built-in.o. + table-y table-m and table- + + To avoid code bit-rot you may wish to avoid using #ifdefs on your + code. Bit-rot is possible if certain features may not be enabled + on certain build environments. The table-y, table-m specifies + targets which you always wish to force compilation on, but wish + to only enable linking in if the feature is enabled, these are + features taking advantage of Linux's linker tables. For details + on how to implement a feature using linker tablesrefer to + include/linux/tables.h. + + Example: + table-$(CONFIG-FEATURE_FOO) += foo.o + + An alternative to using table-y, table-m is to use extra-y followed + by the respective obj-y, obj-m: + + extra-y += foo.o + obj-$(CONFIG-FEATURE_FOO) += foo.o --- 6.7 Commands useful for building a boot image
include/linux/tables.h new file +421 -0
421 lines not on this page · raw message
scripts/Makefile.build +2 -2
@@ -91,7 +91,7 @@ modorder-target := $(obj)/modules.order # We keep a list of all modules in $(MODVERDIR) -__build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \ +__build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y) $(table-y)) \ $(if $(KBUILD_MODULES),$(obj-m) $(modorder-target)) \ $(subdir-ym) $(always) @:
@@ -294,7 +294,7 @@ $(obj)/%.o: $(src)/%.S FORCE $(call if_changed_dep,as_o_S) targets += $(real-objs-y) $(real-objs-m) $(lib-y) -targets += $(extra-y) $(MAKECMDGOALS) $(always) +targets += $(extra-y) $(table-y) $(MAKECMDGOALS) $(always) # Linker scripts preprocessor (.lds.S -> .lds) # ---------------------------------------------------------------------------
scripts/Makefile.clean +1 -0
@@ -36,6 +36,7 @@ subdir-ymn := $(addprefix $(obj)/,$(subdir-ymn)) # directory __clean-files := $(extra-y) $(extra-m) $(extra-) \ + $(table-y) $(table-m) $(table-) \ $(always) $(targets) $(clean-files) \ $(host-progs) \ $(hostprogs-y) $(hostprogs-m) $(hostprogs-)
scripts/Makefile.lib +12 -0
@@ -12,6 +12,16 @@ export KBUILD_SUBDIR_CCFLAGS := $(KBUILD_SUBDIR_CCFLAGS) $(subdir-ccflags-y) # Figure out what we need to build from the various variables # =========================================================================== +# Linker tables objects always wish to be built to avoid bit-rot in +# code, but only linked in *iff* they were enabled. We accomplish this +# using pegging linker table objects into extra-y, which forces +# compilation and then using the respective table-y and table-m as +# as hints for things we do want enabled. Objects which we want to +# avoid linking in will be in table-, not table-y and table-m. +extra-y += $(table-) +obj-m += $(table-m) +obj-y += $(table-y) + # When an object is listed to be built compiled-in and modular, # only build the compiled-in version
@@ -72,6 +82,8 @@ real-objs-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y) # Add subdir path extra-y := $(addprefix $(obj)/,$(extra-y)) +table-y := $(addprefix $(obj)/,$(table-y)) +table-m := $(addprefix $(obj)/,$(table-m)) always := $(addprefix $(obj)/,$(always)) targets := $(addprefix $(obj)/,$(targets)) modorder := $(addprefix $(obj)/,$(modorder))
Signature · 1 line
2.6.2
Re: [RFC v1 2/8] tables.h: add linker table support
Quoted 7 lines
Documentation/kbuild/makefiles.txt +0
-0@@ -1088,6 +1088,25 @@ When kbuild executes, the following steps are followed (roughly): In this example, extra-y is used to list object files that shall be built, but shall not be linked as part of built-in.o.
Documentation/kbuild/makefiles.txt When kbuild executes, the following steps are followed (roughly): · line 1099
Quoted 10 lines
Documentation/kbuild/makefiles.txt +9
-0+ table-y table-m and table- + + To avoid code bit-rot you may wish to avoid using #ifdefs on your + code. Bit-rot is possible if certain features may not be enabled + on certain build environments. The table-y, table-m specifies + targets which you always wish to force compilation on, but wish + to only enable linking in if the feature is enabled, these are + features taking advantage of Linux's linker tables. For details + on how to implement a feature using linker tablesrefer to
s/tablesrefer/tables refer/
Quoted 93 lines
Documentation/kbuild/makefiles.txt +10
-0+ include/linux/tables.h. + + Example: + table-$(CONFIG-FEATURE_FOO) += foo.o + + An alternative to using table-y, table-m is to use extra-y followed + by the respective obj-y, obj-m: + + extra-y += foo.o + obj-$(CONFIG-FEATURE_FOO) += foo.o --- 6.7 Commands useful for building a boot imageinclude/linux/tables.h new file +74
-0@@ -0,0 +1,421 @@ +#ifndef _LINUX_LINKER_TABLES_H +#define _LINUX_LINKER_TABLES_H +/* +* Linux linker tables +* +* Copyright (C) 2005-2009 Michael Brown <[email protected]> +* Copyright (C) 2015 Luis R. Rodriguez <[email protected]> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* Due to this file being licensed under the GPL there is controversy over +* whether this permits you to write a module that #includes this file +* without placing your module under the GPL. Please consult a lawyer for +* advice before doing this. +*/ + +/** +* DOC: linker tables - simplifying inits and for when #ifdefs are harmful +* +* Linker tables help you simplify init sequences by using ELF sections, linker +* build time selective sorting (disabled options get ignored), and can also be +* used to help you avoid bit-rot code due to #ifdery collateral. +* +* The code bit-rot prolem: +* +* Overuse of C #ifdefs can be problematic for certain types of code. Linux +* provides a rich array of features, but all these features take up valuable +* space in a kernel image. The traditional solution to this problem has been +* for each feature to have its own Kconfig entry and for the respective code +* to be wrapped around #ifdefs, allowing the feature to be compiled in only +* if desired. +* +* The problem with this is that over time it becomes very difficult and time +* consuming to compile, let alone test, all possible versions of Linux. Code +* that is not typically used tends to suffer from bit-rot over time. It can +* become difficult to predict which combinations of compile-time options will +* result in code that can compile and link correctly. +* +* To solve this problem linker tables can be used on Linux, it enables you to +* always force compiling of select features that one wishes to avoid bit-rot +* while still enabling you to disable linking feature code into the final +* kernel image or building certain modules if the features have been disabled +* via Kconfig. The code is derivative of gPXE linker table's solution. For +* more details on history and to see how this code evolved refer to the +* userspace mockup solution on github [0]. This repository can also be used to +* for ease of testing of extensions and sampling of changes prior to inclusion +* into Linux, it is intended to be kept up to date to match Linux's solution. +* Contrary to gPXE's solution, which stives to force compilation of +* *everything*, Linux's solution allows for developers to be selective over +* where one should use linker tables. +* +* [0] https://github.com/mcgrof/table-init/ +* +* To use linker tables, features should be implemented in separate C files, +* and should always be compiled -- they should not be guarded with a C code +* #ifdef CONFIG_FOO statements. By making code always compile, you avoid +* the problem of bit-rot in rarely-used code. The trick to linker tables +* is by forcing compilation but only linking if the feature has been enabled. +* This is accomplished with a new target for Makefiles documented below. +* +* Let's assume we want to always force compilation of feature FOO in the +* kernel but avoid linking it. When you enable the FOO feature via Kconfig +* you'd end up with: +* +* #define CONFIG_FOO 1 +* +* You typically would then just use this on your Makefile to selectively +* compile and link the feature: +* +* obj-$(CONFIG_FOO) += foo.o +*
include/linux/tables.h line 84
Quoted 10 lines
include/linux/tables.h new file +10
-0+* You'd instead use the new linker table object: +* +* table-$(CONFIG_FOO) += foo.o +* +* Alternatively this would be the equivalent of listing: +* +* extra += foo.o +* obj-$(CONFIG_FOO) += foo.o +* +* The reason why this works cleanly is due to how linker tables work.
I would add an \n here to make it clear you are talking about folding the initialization function in to the table array.
include/linux/tables.h line 85
Quoted 1 line
include/linux/tables.h new file +1
-0+* Traditionally, we would implement features in C code as follows:
s/implent/initialize/
Quoted 11 lines
include/linux/tables.h new file +11
-0+* +* foo_init(); +* +* You'd then have a foo.h which would have: +* +* #ifdef CONFIG_FOO +* void foo(void); +* #else +* static inline void foo(void) { } +* #endif +*
include/linux/tables.h line 106
Quoted 10 lines
include/linux/tables.h new file +10
-0+* Simplifying inits: +* +* With linker tables this is no longer necessary as your init routines would +* be implicit, you'd instead call: +* +* call_init_fns(); +* +* call_init_fns() would call all functions present in your init table and if +* and only if foo.o gets linked in, then its initialisation function will be +* called.
Q: How does that work for modules? The module code would also need the code to parse the table array and call the functions, right?
You may want to mention that in this paragraph.
Quoted 25 lines
include/linux/tables.h new file +25
-0+* +* The linker script takes care of assembling the tables for us. All of our +* table sections have names of the format .tbl.NAME.NN. NAME designates the +* data structure stored in the table. For instance if you had defined a struct +* init_fn as the data type for your init sequences you would have a respective +* init_fns table name declared as reference for these init sequences. NN is a +* two-digit decimal number used to impose an "order level" upon the tables if +* required. NN=00 is reserved for the symbol indicating "table start", and +* NN=99 is reserved for the symbol indicating "table end". In order for the +* call_init_fns() to work behind the scenes the custom linker script would +* need to define the beginning of the table, the end of the table, and in +* between it should use SORT() to give order-level effect. To account for all +* of your struct init_fn init sequences, in your linker script you would have: +* +* .tbl : { +* __tbl_start_init_fns = .; +* *(SORT(.tbl.init_fns.*)) +* __tbl_end_init_fns = .; +* +* Other init tables can go here as well for different +* structures. +* } +* +* The order-level is really only a helper, if only one order level is +* used, the next contributing factor to order is the order of the code
include/linux/tables.h line 141
Quoted 10 lines
include/linux/tables.h new file +10
-0+* in the C file, and the order of the objects in the Makefile. Using +* an order level then should not really be needed in most cases, its +* use however enables to compartamentalize code into tables where ordering +* through C file or through the Makefile would otherwise be very difficult +* or if one wanted to enable very specific initialization semantics. +* +* As an example, suppose that we want to create a "frobnicator" +* feature framework, and allow for several independent modules to +* provide frobnicating services. Then we would create a frob.h +* header file containing e.g.
frobnicator :-) Awesome!
include/linux/tables.h line 152
Quoted 11 lines
include/linux/tables.h new file +11
-0+* +* struct frobnicator { +* const char *name; +* void (*frob) (void); +* }; +* +* #define FROBNICATORS __table(struct frobnicator, "frobnicators") +* #define __frobnicator __table_entry(FROBNICATORS, 01) +* +* Any module providing frobnicating services would look something +* like
this:
Quoted 13 lines
include/linux/tables.h new file +13
-0+* +* #include "frob.h" +* +* static void my_frob(void) { +* ... Do my frobnicating +* } +* +* struct frob my_frobnicator __frobnicator = { +* .name = "my_frob", +* .frob = my_frob, +* }; +* +* The central frobnicator code (frob.c) would use the frobnicating
include/linux/tables.h line 175
Quoted 10 lines
include/linux/tables.h new file +10
-0+* modules as follows +* +* #include "frob.h" +* +* void frob_all(void) { +* struct frob *frob; +* +* for_each_table(frob, FROBNICATORS) { +* pr_info("Calling frobnicator \"%s\"\n", frob->name); +* frob->frob();
May want to align it on the same column.
Quoted 73 lines
include/linux/tables.h new file +73
-0+* } +* } +*/ + +/* +* __table - Declares a linker table +* +* @type: data type +* @name: table name +* +* Declares a linker table. +*/ +#define __table (type, name) (type, name) + +/* +* __table_type() - get linker table data type +* +* @table: linker table +* +* Gives you the linker table data type. +*/ +#define __table_type (table) __table_extract_type table +#define __table_extract_type (type, name) type + +/* +* __table_name - get linker table name +* +* @table: linker table +* +* Gives you the table name. +*/ +#define __table_name (table) __table_extract_name table +#define __table_extract_name (type, name) name + +/* +* __table_section - get linker table section name +* +* @table: linker table +* @idx: order level, this is the sub-table index +* +* Declares the section name. +*/ +#define __table_section (table, idx) \ + ".tbl." __table_name (table) "." __table_str (idx) +#define __table_str (x) #x + +/* +* __table_alignment - get linker table alignment +* +* @table: linker table +* +* Gives you the linker table alignment. +*/ +#define __table_alignment ( table ) __alignof__ (__table_type(table)) + +/* +* __table_entry - declare a linker table entry +* +* @table: linker table +* @idx: order level, the sub-table index +* +* Example usage: +* +* #define FROBNICATORS __table (struct frobnicator, "frobnicators") +* #define __frobnicator __table_entry(FROBNICATORS, 01) +* +* struct frobnicator my_frob __frobnicator = { +* ... +* }; +*/ +#define __table_entry (table, idx) \ + __attribute__ ((__section__(__table_section(table, idx)), \ + __aligned__(__table_alignment(table))))
include/linux/tables.h line 258
Quoted 10 lines
include/linux/tables.h new file +10
-0+ +/* +* __table_entries - get start of linker table entries +* +* @table: linker table +* @idx: order level, the sub-table index +* +* This gives you the start of the respective linker table entries +*/ +#define __table_entries (table, idx) ( { \
The \ is at an odd column.
Quoted 40 lines
include/linux/tables.h new file +40
-0+ static __table_type(table) __table_entries[0] \ +__table_entry (table, idx); \ + __table_entries; } ) + +/* +* table_start - get start of linker table +* +* @table: linker table +* +* This gives you the start of the respective linker table. +* +* Example usage: +* +* #define FROBNICATORS __table (struct frobnicator, "frobnicators") +* +* struct frobnicator *frobs = table_start(FROBNICATORS); +*/ +#define table_start (table) __table_entries(table, 00) + +/* +* table_end - get end of linker table +* +* @table: linker table +* +* This gives you the end of the respective linker table. +* +* Example usage: +* +* #define FROBNICATORS __table (struct frobnicator, "frobnicators") +* +* struct frobnicator *frobs_end = table_end(FROBNICATORS); +*/ +#define table_end (table) __table_entries(table, 99) + +/* +* table_num_entries - get number of entries in linker table +* +* @table: linker table +* +* This gives you the number of entries in linker table.
include/linux/tables.h line 308
Quoted 10 lines
include/linux/tables.h new file +10
-0+* +* Example usage: +* +* #define FROBNICATORS __table(struct frobnicator, "frobnicators") +* +* unsigned int num_frobs = table_num_entries(FROBNICATORS); +*/ +#define table_num_entries (table) \ + ((unsigned int) (table_end(table) - \ + table_start(table)))
Should you have a check in case the addresses in the image are not in sequence? (which would imply that sorting of the linker has gone awry, but hey, what if somebody messed with the linker script?)
Like:
unsigned int s = table_start(table); unsigned int e = table_end(table); BUG_ON(s > e); return e-s;
Quoted 8 lines
include/linux/tables.h new file +8
-0+ +/* +* for_each_table_entry - iterate through all entries within a linker table +* +* @pointer: entry pointer +* @table: linker table +* +* Example usage:
include/linux/tables.h line 326
Quoted 10 lines
include/linux/tables.h new file +10
-0+* +* #define FROBNICATORS __table(struct frobnicator, "frobnicators") +* +* struct frobnicator *frob; +* +* for_each_table_entry(frob, FROBNICATORS) { +* ... +* } +*/ +#define for_each_table_entry (pointer, table) \
Again, perhaps add an check to make sure that the table_start(table) > table_end(table) is not true.
Quoted 27 lines
include/linux/tables.h new file +27
-0+for (pointer =table_start (table); \ + pointer <table_end (table); \ + pointer++) + +/** +* for_each_table_entry_reverse - iterate through linker table in reverse order +* +* @pointer: entry pointer +* @table: linker table +* +* Example usage: +* +* #define FROBNICATORS __table(struct frobnicator, "frobnicators") +* +* struct frobnicator *frob; +* +* for_each_table_entry_reverse(frob, FROBNICATORS) { +* ... +* } + */ +#define for_each_table_entry_reverse (pointer, table) \ + for (pointer = (table_end(table) - 1 ); \ + pointer >= table_start(table); \ + pointer--) + +/* +*
include/linux/tables.h line 363
Quoted 10 lines
include/linux/tables.h new file +10
-0+* Intel's C compiler chokes on several of the constructs used in this +* file. The workarounds are ugly, so we use them only for an icc +* build. +*/ +#define ICC_ALIGN_HACK_FACTOR 128 +#ifdef __ICC + +/* +* icc miscompiles zero-length arrays by inserting padding to a length +* of two array elements. We therefore have to generate the
Do those two array elements have anything in them? Could the code:
call_init_fns()
check if they are empty and skip over them?
Quoted 46 lines
include/linux/tables.h new file +46
-0+* __table_entries() symbols by hand in asm. +*/ +#undef __table_entries +#define __table_entries (table, idx) ( { \ + extern __table_type(table) \ + __table_temp_sym(idx, __LINE__) [] \ + __table_entry(table, idx) \ + asm (__table_entries_sym(table, idx)); \ + __asm__( ".ifndef %c0\n\t" \ + ".section " __table_section(table, idx) "\n\t" \ + ".align %c1\n\t" \ + "\n%c0:\n\t" \ + ".previous\n\t" \ + ".endif\n\t" \ + : : "i" (__table_temp_sym(idx, __LINE__ )), \ + "i" (__table_alignment(table))); \ + __table_temp_sym ( idx, __LINE__ ); } ) +#define __table_entries_sym (table, idx) \ + "__tbl_" __table_name(table) "_" #idx +#define __table_temp_sym (a, b) \ + ___table_temp_sym(__table_, a, _, b) +#define ___table_temp_sym (a, b, c, d) a ## b ## c ## d + +/* +* icc ignores __attribute__ (( aligned (x) )) when it is used to +* decrease the compiler's default choice of alignment (which may be +* higher than the alignment actually required by the structure). We +* work around this by forcing the alignment to a large multiple of +* the required value (so that we are never attempting to decrease the +* default alignment) and then postprocessing the object file to +* reduce the alignment back down to the "real" value. +* +*/ +#undef __table_alignment +#define __table_alignment (table) \ + (ICC_ALIGN_HACK_FACTOR * __alignof__(__table_type(table))) + +/* +* Because of the alignment hack, we must ensure that the compiler +* never tries to place multiple objects within the same section, +* otherwise the assembler will insert padding to the (incorrect) +* alignment boundary. Do this by appending the line number to table +* section names. +* +* Note that we don't need to worry about padding between array +* elements, since the alignment is declared on the variable (i.e. the
include/linux/tables.h line 419
Quoted 10 lines
include/linux/tables.h new file +10
-0+* whole array) rather than on the type (i.e. on all individual array +* elements). +*/ +#undef __table_section +#define __table_section (table, idx) \ + ".tbl." __table_name(table) "." __table_str(idx) \ + "." __table_xstr(__LINE__) +#define __table_xstr (x) __table_str(x) + +#endif /* __ICC */
You kind of lost me with this Intel compiler hacks.
Could we structure to be specific size such that the aligment and such are always the same size?
Quoted 58 lines
include/linux/tables.h new file +2
-0+ +#endif /* _LINUX_LINKER_TABLES_H */ scripts/Makefile.build +2
-2@@ -91,7 +91,7 @@ modorder-target := $(obj)/modules.order # We keep a list of all modules in $(MODVERDIR) -__build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \ +__build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y) $(table-y)) \ $(if $(KBUILD_MODULES),$(obj-m) $(modorder-target)) \ $(subdir-ym) $(always) @:@@ -294,7 +294,7 @@ $(obj)/%.o: $(src)/%.S FORCE $(call if_changed_dep,as_o_S) targets += $(real-objs-y) $(real-objs-m) $(lib-y) -targets += $(extra-y) $(MAKECMDGOALS) $(always) +targets += $(extra-y) $(table-y) $(MAKECMDGOALS) $(always) # Linker scripts preprocessor (.lds.S -> .lds) # ---------------------------------------------------------------------------scripts/Makefile.clean +1
-0@@ -36,6 +36,7 @@ subdir-ymn := $(addprefix $(obj)/,$(subdir-ymn)) # directory __clean-files := $(extra-y) $(extra-m) $(extra-) \ + $(table-y) $(table-m) $(table-) \ $(always) $(targets) $(clean-files) \ $(host-progs) \ $(hostprogs-y) $(hostprogs-m) $(hostprogs-)scripts/Makefile.lib +10
-0@@ -12,6 +12,16 @@ export KBUILD_SUBDIR_CCFLAGS := $(KBUILD_SUBDIR_CCFLAGS) $(subdir-ccflags-y) # Figure out what we need to build from the various variables # =========================================================================== +# Linker tables objects always wish to be built to avoid bit-rot in +# code, but only linked in *iff* they were enabled. We accomplish this +# using pegging linker table objects into extra-y, which forces +# compilation and then using the respective table-y and table-m as +# as hints for things we do want enabled. Objects which we want to +# avoid linking in will be in table-, not table-y and table-m. +extra-y += $(table-) +obj-m += $(table-m) +obj-y += $(table-y) + # When an object is listed to be built compiled-in and modular, # only build the compiled-in version
scripts/Makefile.lib real-objs-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y) · line 89
Quoted 8 lines
scripts/Makefile.lib +2
-0@@ -72,6 +82,8 @@ real-objs-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y) # Add subdir path extra-y := $(addprefix $(obj)/,$(extra-y)) +table-y := $(addprefix $(obj)/,$(table-y)) +table-m := $(addprefix $(obj)/,$(table-m)) always := $(addprefix $(obj)/,$(always)) targets := $(addprefix $(obj)/,$(targets)) modorder := $(addprefix $(obj)/,$(modorder))
Thanks!
Quoted 3 lines
Signature · 1 line
2.6.2
Re: [RFC v1 2/8] tables.h: add linker table support
Quoted 5 lines
Quoted 5 lines
+ * To solve this problem linker tables can be used on Linux, it enables you to
+ * always force compiling of select features that one wishes to avoid bit-rot
+ * while still enabling you to disable linking feature code into the final
+ * kernel image or building certain modules if the features have been disabled
+ * viaKconfig. The code is derivative of gPXE linker table's solution.
I missed the start of this thread. However, asking as the author of the original Etherboot/gPXE/iPXE linker table solution: please change all references from "gPXE" to "iPXE".
Thanks,
Michael
H. Peter Anvin
reviewerRe: [RFC v1 2/8] tables.h: add linker table support
Quoted 11 lines · Michael Brown
Quoted 1 line
Quoted 1 line
+ * To solve this problem linker tables can be used on Linux, it
enables you to
Quoted 1 line
Quoted 1 line
+ * always force compiling of select features that one wishes to
avoid bit-rot
Quoted 1 line
Quoted 1 line
+ * while still enabling you to disable linking feature code into
the final
Quoted 1 line
Quoted 1 line
+ * kernel image or building certain modules if the features have
been disabled
Quoted 1 line
Quoted 1 line
+ * via
Kconfig. The code is derivative of gPXE linker table'ssolution.
Quoted 8 lines
I missed the start of this thread. However, asking as the author of the
original Etherboot/gPXE/iPXE linker table solution: please change all references from "gPXE" to "iPXE".Thanks,
Michael
Yes, that request has already been made :)
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.
RFC 3/8 x86/boot: add BIT() to boot/bitops.h
In reply to Luis R. RodriguezFrom: "Luis R. Rodriguez" <[email protected]>
The boot/bitops.h guards against included the regular bitops, it only implements what we need at early boot. We'll be making use of BIT() later so add it.
Users of boot/boot.h must include it prior to asm/setup.h otherwise the guard protection devise against the regular linux/bitops.h will not take effect.
--- arch/x86/boot/bitops.h | 2 ++ arch/x86/boot/boot.h | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/boot/bitops.h +2 -0
@@ -40,4 +40,6 @@ static inline void set_bit(int nr, void *addr) asm("btsl %1,%0" : "+m" (*(u32 *)addr) : "Ir" (nr)); } +#define BIT (x) (1 << x) +#endif /* BOOT_BITOPS_H */
arch/x86/boot/boot.h +1 -1
@@ -23,8 +23,8 @@#include <stdarg.h> #include <linux/types.h> #include <linux/edd.h> -#include <asm/setup.h> #include "bitops.h" +#include <asm/setup.h> #include "ctype.h" #include "cpuflags.h"
Signature · 1 line
2.6.2
Re: [RFC v1 3/8] x86/boot: add BIT() to boot/bitops.h
Quoted 3 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
The
boot/bitops.hguards against included the regular bitops,
It may sounds better this way:
The boot/bitops.h has guards against including the regular bitops (include/asm-generic/bitops.h)
Quoted 2 lines
it only implements what we need at early boot. We'll be making use of
BIT()later so add it.
Where? Could you paste in the name of the patch in the description?
Quoted 3 lines
Users of
boot/boot.hmust include it prior toasm/setup.hotherwise the guard protection devise against the regular
s/devise/devised/
Quoted 36 lines
linux/bitops.hwill not take effect.Signed-off-by Luis R. Rodriguez <[email protected]>--- arch/x86/boot/bitops.h | 2 ++ arch/x86/boot/boot.h | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)arch/x86/boot/bitops.h +2
-0@@ -40,4 +40,6 @@ static inline void set_bit(int nr, void *addr) asm("btsl %1,%0" : "+m" (*(u32 *)addr) : "Ir" (nr)); } +#define BIT (x) (1 << x) +#endif /* BOOT_BITOPS_H */ arch/x86/boot/boot.h +1
-1@@ -23,8 +23,8 @@#include <stdarg.h> #include <linux/types.h> #include <linux/edd.h> -#include <asm/setup.h> #include "bitops.h" +#include <asm/setup.h> #include "ctype.h" #include "cpuflags.h" Signature · 1 line
2.6.2
Re: [RFC v1 3/8] x86/boot: add BIT() to boot/bitops.h
Quoted 1 line · Konrad Rzeszutek Wilk
Where? Could you paste in the name of the patch in the description?
+x86_init_early(BIT(X86_SUBARCH_INTEL_MID), NULL, NULL,
+ x86_intel_mid_early_setup);
Is an example, so users of the subarch. Likewise the macro helpers to make these smaller, see x86_init_early_all() on the patch "x86/init:
add linker table support".
And then of course the checkers:
+static bool x86_init_fn_supports_subarch(struct x86_init_fn *fn) +{
+ if (!fn->supp_hardware_subarch) {
+ pr_err("Init sequence fails to declares any supported
subarchs: %pF\n", fn->early_init);
+ WARN_ON(1);
+ }
+ if (BIT(boot_params.hdr.hardware_subarch) & fn->supp_hardware_subarch)
+ return true;
+ return false;
+}
Luis
RFC 4/8 x86/init: add linker table support
In reply to Luis R. RodriguezFrom: "Luis R. Rodriguez" <[email protected]>
Any failure on the x86 init path can be catastrophic. A simple shift of a call from one place to another can easily break things. Likewise adding a new call to one path without considering all x86 requirements can make certain x86 run time environments crash. We currently account for these requirements through peer code review and run time testing. We could do much better if we had a clean and simple way to annotate strong semantics for run time requirements, init sequence dependencies, and detection mechanisms for additions of new x86 init sequences.
This adds support for linker tables for x86 in order to help define strong semantics for x86 init sequences. The defined struct x86_init_fn is inspired by both gPXE's sample definitions but is also also heavily inspired by Linux's own IOMMU initialiation solution which enables to extend initialization semantics to support init routine custom detection routines, and enables adding annotations for init routine dependency mapping. A newly featured solution in this design
is to build up on the hardware subarchitecture
added to the x86 boot protocol 2.07 in 2007 by Rusty but never really taken advantage of except for lguest, using it as a stop-gap for new x86 features which have failed to take into consideration the dual x86-64 entry points possible due to paravirtualization yielding requirements. The hardware subarchitecture could also potentially be used in the future to help unify Linux x86 entry points. The current disjoint entry points for x86-64 can be summarized as follows.
Bare metal, KVM, Xen HVM Xen PV / dom0
startup_64() startup_xen()
\ /
x86_64_start_kernel() xen_start_kernel()
\ /
x86_64_start_reservations()
|
start_kernel()
[ ... ]
[ setup_arch() ]
[ ... ]
initInherited by the nature of using linker tables we also gain the ability to sort init sequences using the linker through the use of order-level and linker SORT() and only optionally enable use specific init sequence sorting when init seqeuence dependencies semantics are needed. By using linker tables we can also avoid #ifdefer'y on select code when and if desirable, this is also completley optional but it is a feature we inherit by using linker tables.
For *new x86 features* this enables strong semantics to be considered. For *existing x86 feature code* this enables the opportunity to clarify requirements and dependencies, which ultimately also provides a new way to ensure that code that should not run never runs, that is, it provides one mechanism to help prevent dead code. For a more elaborate description on what dead code is exactly and how it can relate to init sequences in particular to Xen due to its init sequence refer to [0] and [1].
Four debug x86 features are included to help test x86 linker table support. You'll want CONFIG_X86_DEBUG_LINKER_TABLES to enable those. The features defined with table-$(CONFIG-x) will *always* be compiled but only linked in when enabled. The features defined with the good 'ol obj-y will only be compiled as we're used to. If CONFIG_X86_DEBUG_LINKER_TABLES is disabled nothing currently runs as there are no init sequences yet using any of this.
The goal is to evolve the semantics carefully as needed. We start with one basic callback, an early_init() called at the beginning of x86_64_start_reservations().
[0] http://www.do-not-panic.com/2015/12/avoiding-dead-code-pvops-not-silver-bullet.html
[1] http://www.do-not-panic.com/2015/12/xen-and-x86-linux-zero-page.html
--- arch/x86/Kconfig.debug | 47 +++++++ arch/x86/include/asm/x86_init.h | 1 + arch/x86/include/asm/x86_init_fn.h | 267 ++++++++++++++++++++++++++++++++++++ arch/x86/kernel/Makefile | 2 + arch/x86/kernel/dbg-tables/Makefile | 18 +++ arch/x86/kernel/dbg-tables/alpha.c | 10 ++ arch/x86/kernel/dbg-tables/beta.c | 18 +++ arch/x86/kernel/dbg-tables/delta.c | 10 ++ arch/x86/kernel/dbg-tables/gamma.c | 18 +++ arch/x86/kernel/dbg-tables/gamma.h | 3 + arch/x86/kernel/head32.c | 4 + arch/x86/kernel/head64.c | 4 + arch/x86/kernel/init.c | 55 ++++++++ arch/x86/kernel/sort-init.c | 114 +++++++++++++++ arch/x86/kernel/vmlinux.lds.S | 25 ++++ arch/x86/tools/relocs.c | 1 + 16 files changed, 597 insertions(+) create mode 100644 arch/x86/include/asm/x86_init_fn.h create mode 100644 arch/x86/kernel/dbg-tables/Makefile create mode 100644 arch/x86/kernel/dbg-tables/alpha.c create mode 100644 arch/x86/kernel/dbg-tables/beta.c create mode 100644 arch/x86/kernel/dbg-tables/delta.c create mode 100644 arch/x86/kernel/dbg-tables/gamma.c create mode 100644 arch/x86/kernel/dbg-tables/gamma.h create mode 100644 arch/x86/kernel/init.c create mode 100644 arch/x86/kernel/sort-init.c
arch/x86/Kconfig.debug +47 -0
@@ -400,4 +400,51 @@ config PUNIT_ATOM_DEBUG The current power state can be read from /sys/kernel/debug/punit_atom/dev_power_state +config X86_DEBUG_LINKER_TABLES + bool "x86 linker table debug" + depends on DEBUG_KERNEL + default n + ---help--- + Enabling this should enable debugging linker tables on x86 + with its own solution. To help debug it enables the "x86 beta" + and "x86 gamma" built-in feature. Both "x86 alpha" and "x86 delta" + built-in features are left disabled. You should see all tables + compiled, except delta, only the beta and gamma feature will be + linked in. The delta feature will only be compiled and linked in + if and only if its enabled the old way. + + For more details on these linker table refer to: + + include/linux/tables.h + + If unsure, say N. + +config X86_DEBUG_LINKER_TABLE_ALPHA + bool "x86 linker table alpha" + default n + depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 alpha" feature. + +config X86_DEBUG_LINKER_TABLE_BETA + bool "x86 linker table beta" + default y + depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 beta" feature. + +config X86_DEBUG_LINKER_TABLE_GAMMA + bool "x86 linker table gamma" + default y + depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 gamma" feature. + +config X86_DEBUG_LINKER_TABLE_DELTA + bool "x86 linker table delta" + default n + depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 delta" feature. + endmenu
arch/x86/include/asm/x86_init.h +1 -0
@@ -2,6 +2,7 @@#define _ASM_X86_PLATFORM_H#include <asm/bootparam.h> +#include <asm/x86_init_fn.h> struct mpc_bus ;struct mpc_cpu ;
arch/x86/include/asm/x86_init_fn.h new file +267 -0
267 lines not on this page · raw message
arch/x86/kernel/Makefile +2 -0
@@ -23,6 +23,8 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n CFLAGS_irq.o := -I$(src)/../include/asm/trace obj-y := process_$(BITS).o signal.o +obj-y += init.o sort-init.o +obj-$(CONFIG_X86_DEBUG_LINKER_TABLES) += dbg-tables/ obj-$(CONFIG_COMPAT) += signal_compat.o obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o dumpstack.o nmi.o
arch/x86/kernel/dbg-tables/Makefile new file +18 -0
@@ -0,0 +1,18 @@ +# You should see all these compile but at run time you'd +# only see the ones that were linked in. + +# This ensures we always compile but only link +# in alpha if it was enabled. This is typically what +# you are expected to use to avoid code bit-rot. +table-$(CONFIG_X86_DEBUG_LINKER_TABLE_ALPHA) += alpha.o + +# This accomplishes the same, but requires 2 lines. +extra-y += beta.o +obj-$(CONFIG_X86_DEBUG_LINKER_TABLE_BETA) += beta.o + +table-$(CONFIG_X86_DEBUG_LINKER_TABLE_GAMMA) += gamma.o + +# If you *know* you only want to enable compilation of +# a feature when its selected you can just use the good +# ol' obj +obj-$(CONFIG_X86_DEBUG_LINKER_TABLE_DELTA) += delta.o
arch/x86/kernel/dbg-tables/alpha.c new file +10 -0
@@ -0,0 +1,10 @@ +#define pr_fmt (fmt) "debug-alpha: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +static void early_init_dbg_alpha (void ) { +pr_info ("early_init triggered\n" ); +} + +x86_init_early_pc_simple (early_init_dbg_alpha);
arch/x86/kernel/dbg-tables/beta.c new file +18 -0
@@ -0,0 +1,18 @@ +#define pr_fmt (fmt) "debug-beta: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +#include "gamma.h" + +static bool x86_dbg_detect_beta (void ) +{ +return true; +} + +static void early_init_dbg_beta (void ) { +pr_info ("early_init triggered\n" ); +} +x86_init_early_pc (x86_dbg_detect_beta, + x86_dbg_detect_gamma, + early_init_dbg_beta);
arch/x86/kernel/dbg-tables/delta.c new file +10 -0
@@ -0,0 +1,10 @@ +#define pr_fmt (fmt) "debug-delta: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +static void early_init_dbg_delta (void ) { +pr_info ("early_init triggered\n" ); +} + +x86_init_early_pc_simple (early_init_dbg_delta);
arch/x86/kernel/dbg-tables/gamma.c new file +18 -0
@@ -0,0 +1,18 @@ +#define pr_fmt (fmt) "debug-gamma: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +bool x86_dbg_detect_gamma (void ) +{ +return true; +} + +static void early_init_dbg_gamma (void ) +{ +pr_info ("early_init triggered\n" ); +} + +x86_init_early_pc (x86_dbg_detect_gamma, +NULL , + early_init_dbg_gamma);
arch/x86/kernel/dbg-tables/gamma.h new file +3 -0
@@ -0,0 +1,3 @@ +#include <asm/x86_init.h> + +bool x86_dbg_detect_gamma (void );
arch/x86/kernel/head32.c +4 -0
@@ -19,6 +19,7 @@#include <asm/bios_ebda.h> #include <asm/tlbflush.h> #include <asm/bootparam_utils.h> +#include <asm/x86_init.h> static void __init i386_default_early_setup (void ) {
@@ -47,5 +48,8 @@ asmlinkage __visible void __init i386_start_kernel(void)break ; } +x86_init_fn_init_tables (); +x86_init_fn_early_init (); +start_kernel (); }
arch/x86/kernel/head64.c +4 -0
@@ -28,6 +28,7 @@#include <asm/bootparam_utils.h> #include <asm/microcode.h> #include <asm/kasan.h> +#include <asm/x86_init.h> /* * Manage page tables very early on.
@@ -190,6 +191,9 @@ void __init x86_64_start_reservations(char *real_mode_data)if (!boot_params.hdr.version)copy_bootdata (__va (real_mode_data)); +x86_init_fn_init_tables (); +x86_init_fn_early_init (); +reserve_ebda_region ();start_kernel ();
arch/x86/kernel/init.c new file +55 -0
55 lines not on this page · raw message
arch/x86/kernel/sort-init.c new file +114 -0
114 lines not on this page · raw message
arch/x86/kernel/vmlinux.lds.S +25 -0
@@ -265,6 +265,31 @@ SECTIONS __apicdrivers_end = .; } + /* + * x86 linker tables: this will sort all order levels + * lexicographically. Something like SORT_BY_INIT_PRIORITY() + * which sorts specifically on numeric order is desirable but + * that would impose a change on binutils to generalize + * SORT_BY_INIT_PRIORITY() for any section, SORT_BY_INIT_PRIORITY() + * is specific to the init sections, and that would also mean having + * a bump on requirments for building Linux. Using SORT() should + * suffice, * its what gpxe uses. For things in the same order + * level we rely on order in the C file, and also order on the + * Makefile. Since this solution also embraces the IOMMU init solution + * it has further sorting semantics implemented in C and its semantics + * are defined in x86_init_fn.h and implemented in sort-init.c. + */ + .tbl : { + __tbl_x86_start_init_fns = .; + *(SORT(.tbl.x86_init_fns.*)) + __tbl_x86_end_init_fns = .; + + /* ... etc ... */ + + /* You would add other tables here as needed */ + } + . = ALIGN(8); + . = ALIGN(8); /* * .exit.text is discard at runtime, not link time, to deal with
arch/x86/tools/relocs.c +1 -0
@@ -56,6 +56,7 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = {"__x86_cpu_dev_(start|end)|" "(__parainstructions|__alt_instructions)(|_end)|" "(__iommu_table|__apicdrivers|__smp_locks)(|_end)|" +"(__tbl_x86_(start|end)_init_fns|" "__(start|end)_pci_.*|" "__(start|end)_builtin_fw|" "__(start|stop)___ksymtab(|_gpl|_unused|_unused_gpl|_gpl_future)|"
Signature · 1 line
2.6.2
Re: [RFC v1 4/8] x86/init: add linker table support
Quoted 5 lines
arch/x86/Kconfig.debug +0
-0
arch/x86/Kconfig.debug config PUNIT_ATOM_DEBUG · line 409
Quoted 10 lines
arch/x86/Kconfig.debug +7
-0@@ -400,4 +400,51 @@ config PUNIT_ATOM_DEBUG The current power state can be read from /sys/kernel/debug/punit_atom/dev_power_state +config X86_DEBUG_LINKER_TABLES + bool "x86 linker table debug" + depends on DEBUG_KERNEL + default n + ---help--- + Enabling this should enable debugging linker tables on x86 + with its own solution. To help debug it enables the "x86 beta"
its own solution? Would it be good to just name what that solution is?
arch/x86/Kconfig.debug config PUNIT_ATOM_DEBUG · line 414
Quoted 5 lines
arch/x86/Kconfig.debug +5
-0+ and "x86 gamma" built-in feature. Both "x86 alpha" and "x86 delta" + built-in features are left disabled. You should see all tables + compiled, except delta, only the beta and gamma feature will be + linked in. The delta feature will only be compiled and linked in + if and only if its enabled the old way.
Whoa. Beta Alpha Delta Gamma? You lost me :-)
Quoted 17 lines
arch/x86/Kconfig.debug +17
-0+ + For more details on these linker table refer to: + + include/linux/tables.h + + If unsure, say N. + +config X86_DEBUG_LINKER_TABLE_ALPHA + bool "x86 linker table alpha" + default n + depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 alpha" feature. + +config X86_DEBUG_LINKER_TABLE_BETA + bool "x86 linker table beta" + default y
arch/x86/Kconfig.debug config PUNIT_ATOM_DEBUG · line 441
Quoted 10 lines
arch/x86/Kconfig.debug +10
-0+ depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 beta" feature. + +config X86_DEBUG_LINKER_TABLE_GAMMA + bool "x86 linker table gamma" + default y + depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 gamma" feature.
Which has nothing to with gamma radiation right :-)
Quoted 14 lines
arch/x86/Kconfig.debug +8
-0+ +config X86_DEBUG_LINKER_TABLE_DELTA + bool "x86 linker table delta" + default n + depends on X86_DEBUG_LINKER_TABLES + ---help--- + Enabling this should enable the linker table "x86 delta" feature. + endmenuarch/x86/include/asm/x86_init.h +1
-0+/**
arch/x86/include/asm/x86_init.h
Quoted 10 lines
arch/x86/include/asm/x86_init.h +10
-0+* enum x86_init_fn_flags: flags for init sequences +* +* X86_INIT_FINISH_IF_DETECTED: tells the core that once this init sequence +* has completed it can break out of the loop for init sequences on +* its own level. +* X86_INIT_DETECTED: private flag. Used by the x86 core to annotate that this +* init sequence has been detected and it all of its callbacks +* must be run during initialization. +*/ +enum x86_init_fn_flags {
Should you have a 'default' one? That is an default one? X86_INIT_IGNORE = 0,
arch/x86/include/asm/x86_init.h
Quoted 8 lines
arch/x86/include/asm/x86_init.h +8
-0+X86_INIT_FINISH_IF_DETECTED =BIT (0 ), + X86_INIT_DETECTED = BIT(1), +}; + +/* The x86 initialisation function table */ +#define X86_INIT_FNS __table(struct x86_init_fn, "x86_init_fns") + +/* Used to declares an x86 initialization table */
s/declares/declare/
Also missing stop.
arch/x86/include/asm/x86_init.h
Quoted 6 lines
arch/x86/include/asm/x86_init.h +6
-0+#define __x86_init_fn (order_level) __table_entry(X86_INIT_FNS, order_level) + +/* Init order levels, we can start at 01 but reserve 01-09 for now */ +#define X86_INIT_ORDER_EARLY 10 +#define X86_INIT_ORDER_NORMAL 30 +#define X86_INIT_ORDER_LATE 50
Something is odd with those tabs/spaces.
arch/x86/include/asm/x86_init.h
Quoted 4 lines
arch/x86/include/asm/x86_init.h +4
-0+ +/* +* Use LTO_REFERENCE_INITCALL just in case of issues with old versions of gcc. +* This might not be needed for linker tables due to how we compartamentalize
s/compartamentalize/compartmentalize/
Quoted 5 lines
+ * sections and then order them at linker time, but just in case.
+ */arch/x86/include/asm/x86_init.h +3
-0+ +#define x86_init(__level, \ + __supp_hardware_subarch, \
arch/x86/include/asm/x86_init.h
Quoted 10 lines
arch/x86/include/asm/x86_init.h +10
-0+ __detect, \ + __depend, \ + __early_init) \ + static struct x86_init_fn __x86_init_fn_##__early_init __used \ +__x86_init_fn (__level ) = { \ + .order_level = __level, \ + .supp_hardware_subarch = __supp_hardware_subarch, \ + .detect = __detect, \ + .depend = __depend, \ + .early_init = __early_init, \
.flags = 0, ? or X86_INIT_IGNORE ?
arch/x86/include/asm/x86_init.h
Quoted 4 lines
arch/x86/include/asm/x86_init.h +4
-0+ }; \ + LTO_REFERENCE_INITCALL(__x86_init_fn_##__early_init); + +#define x86_init_early(__supp_hardware_subarch, \
The first \ is at an odd column.
Quoted 20 lines
arch/x86/include/asm/x86_init.h +20
-0+ __detect, \ + __depend, \ + __early_init) \ +x86_init (X86_INIT_ORDER_EARLY ,__supp_hardware_subarch , \ +__detect ,__depend , \ +__early_init ); + +#define x86_init_normal (__supp_hardware_subarch, \ + __detect, \ + __depend, \ + __early_init) \ + x86_init(__name, X86_INIT_ORDER_NORMAL, __supp_hardware_subarch,\ + __detect, __depend, \ + __early_init); + +#define x86_init_early_all (__detect, \ + __depend, \ + __early_init) \ + x86_init_early(__name, X86_SUBARCH_ALL_SUBARCHS, \ + __detect, __depend, \
arch/x86/include/asm/x86_init.h
Quoted 10 lines
arch/x86/include/asm/x86_init.h +10
-0+ __early_init); + +#define x86_init_early_pc (__detect, \ + __depend, \ + __early_init) \ + x86_init_early(BIT(X86_SUBARCH_PC), \ + __detect, __depend, \ + __early_init); + +#define x86_init_early_pc_simple (__early_init) \
Ditto
arch/x86/include/asm/x86_init.h
Quoted 4 lines
arch/x86/include/asm/x86_init.h +4
-0+x86_init_early ((BIT (X86_SUBARCH_PC )),NULL ,NULL , \ + __early_init); + +#define x86_init_normal_all(__detect, \
Ditto
Quoted 5 lines
arch/x86/include/asm/x86_init.h +5
-0+ __depend, \ + __early_init) \ +x86_init_normal (X86_SUBARCH_ALL_SUBARCHS , \ + __detect, __depend, \ + __early_init);
arch/x86/include/asm/x86_init.h
Quoted 10 lines
arch/x86/include/asm/x86_init.h +10
-0+ +#define x86_init_normal_pc (__detect, \ + __depend, \ + __early_init) \ + x86_init_normal((BIT(X86_SUBARCH_PC)), \ + __detect, __depend, \ + __early_init); + + +#define x86_init_normal_xen(__detect, \
Ditto
Quoted 153 lines
arch/x86/include/asm/x86_init.h +30
-0+ __depend, \ + __early_init) \ +x86_init_normal ((BIT (X86_SUBARCH_XEN )), \ + __detect, __depend, \ + __early_init); + +/** +* x86_init_fn_early_init: call all early_init() callbacks +* +* This calls all early_init() callbacks on the x86_init_fns linker table. +*/ +void x86_init_fn_early_init (void ); + +/** +* x86_init_fn_init_tables - sort and check x86 linker table +* +* This sorts struct x86_init_fn init sequences in the x86_init_fns linker +* table by ensuring that init sequences that depend on other init sequences +* are placed later in the linker table. Init sequences that do not have +* dependencies are left in place. Circular dependencies are not allowed. +* The order-level of subordinate init sequences, that is of init sequences +* that depend on other init sequences, must have an order-level of lower +* or equal priority to the init sequence it depends on. +* +* This also validates semantics of all struct x86_init_fn init sequences +* on the x86_init_fns linker table. +*/ +void x86_init_fn_init_tables (void ); + +#endif /* __X86_INIT_TABLES_H */ arch/x86/kernel/Makefile +2
-0@@ -23,6 +23,8 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n CFLAGS_irq.o := -I$(src)/../include/asm/trace obj-y := process_$(BITS).o signal.o +obj-y += init.o sort-init.o +obj-$(CONFIG_X86_DEBUG_LINKER_TABLES) += dbg-tables/ obj-$(CONFIG_COMPAT) += signal_compat.o obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o dumpstack.o nmi.oarch/x86/kernel/dbg-tables/Makefile new file +18
-0@@ -0,0 +1,18 @@ +# You should see all these compile but at run time you'd +# only see the ones that were linked in. + +# This ensures we always compile but only link +# in alpha if it was enabled. This is typically what +# you are expected to use to avoid code bit-rot. +table-$(CONFIG_X86_DEBUG_LINKER_TABLE_ALPHA) += alpha.o + +# This accomplishes the same, but requires 2 lines. +extra-y += beta.o +obj-$(CONFIG_X86_DEBUG_LINKER_TABLE_BETA) += beta.o + +table-$(CONFIG_X86_DEBUG_LINKER_TABLE_GAMMA) += gamma.o + +# If you *know* you only want to enable compilation of +# a feature when its selected you can just use the good +# ol' obj +obj-$(CONFIG_X86_DEBUG_LINKER_TABLE_DELTA) += delta.oarch/x86/kernel/dbg-tables/alpha.c new file +10
-0@@ -0,0 +1,10 @@ +#define pr_fmt (fmt) "debug-alpha: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +static void early_init_dbg_alpha (void ) { +pr_info ("early_init triggered\n" ); +} + +x86_init_early_pc_simple (early_init_dbg_alpha);arch/x86/kernel/dbg-tables/beta.c new file +18
-0@@ -0,0 +1,18 @@ +#define pr_fmt (fmt) "debug-beta: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +#include "gamma.h" + +static bool x86_dbg_detect_beta (void ) +{ +return true; +} + +static void early_init_dbg_beta (void ) { +pr_info ("early_init triggered\n" ); +} +x86_init_early_pc (x86_dbg_detect_beta, + x86_dbg_detect_gamma, + early_init_dbg_beta);arch/x86/kernel/dbg-tables/delta.c new file +10
-0@@ -0,0 +1,10 @@ +#define pr_fmt (fmt) "debug-delta: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +static void early_init_dbg_delta (void ) { +pr_info ("early_init triggered\n" ); +} + +x86_init_early_pc_simple (early_init_dbg_delta);arch/x86/kernel/dbg-tables/gamma.c new file +18
-0@@ -0,0 +1,18 @@ +#define pr_fmt (fmt) "debug-gamma: " fmt + +#include <linux/kernel.h> +#include <asm/x86_init.h> + +bool x86_dbg_detect_gamma (void ) +{ +return true; +} + +static void early_init_dbg_gamma (void ) +{ +pr_info ("early_init triggered\n" ); +} + +x86_init_early_pc (x86_dbg_detect_gamma, +NULL , + early_init_dbg_gamma);arch/x86/kernel/dbg-tables/gamma.h new file +0
-0
arch/x86/kernel/dbg-tables/gamma.h line 3
Quoted 3 lines
arch/x86/kernel/dbg-tables/gamma.h new file +3
-0@@ -0,0 +1,3 @@ +#include <asm/x86_init.h> + +bool x86_dbg_detect_gamma (void );
OK, now that makes sense. I would split the debug functionality in a seperate patch. And update the description in the Kconfig to say what you would expect.
Quoted 86 lines
arch/x86/kernel/head32.c +4
-0@@ -19,6 +19,7 @@#include <asm/bios_ebda.h> #include <asm/tlbflush.h> #include <asm/bootparam_utils.h> +#include <asm/x86_init.h> static void __init i386_default_early_setup (void ) {@@ -47,5 +48,8 @@ asmlinkage __visible void __init i386_start_kernel(void)break ; } +x86_init_fn_init_tables (); +x86_init_fn_early_init (); +start_kernel (); }arch/x86/kernel/head64.c +4
-0@@ -28,6 +28,7 @@#include <asm/bootparam_utils.h> #include <asm/microcode.h> #include <asm/kasan.h> +#include <asm/x86_init.h> /* * Manage page tables very early on. @@ -190,6 +191,9 @@ void __init x86_64_start_reservations(char *real_mode_data)if (!boot_params.hdr.version)copy_bootdata (__va (real_mode_data)); +x86_init_fn_init_tables (); +x86_init_fn_early_init (); +reserve_ebda_region ();start_kernel ();arch/x86/kernel/init.c new file +37
-0@@ -0,0 +1,55 @@ +#define pr_fmt (fmt) "x86-init: " fmt + +#include <linux/bug.h> +#include <linux/kernel.h> + +#include <asm/x86_init_fn.h> +#include <asm/bootparam.h> +#include <asm/boot.h> +#include <asm/setup.h> + +extern struct x86_init_fn __tbl_x86_start_init_fns[], __tbl_x86_end_init_fns[]; + +static bool x86_init_fn_supports_subarch (struct x86_init_fn *fn) +{ +if (!fn->supp_hardware_subarch) { +pr_err ("Init sequence fails to declares any supported subarchs: %pF\n" , fn->early_init); +WARN_ON (1 ); + } +if (BIT (boot_params.hdr.hardware_subarch) & fn->supp_hardware_subarch) +return true; +return false; +} + +void x86_init_fn_early_init (void ) +{ +int ret; +struct x86_init_fn *init_fn; +unsigned int num_inits =table_num_entries (X86_INIT_FNS ); + +if (!num_inits) +return ; + +pr_debug ("Number of init entries: %d\n" , num_inits); + +for_each_table_entry (init_fn,X86_INIT_FNS ) { +if (!x86_init_fn_supports_subarch (init_fn)) +continue ;
arch/x86/kernel/init.c line 47
Quoted 10 lines
arch/x86/kernel/init.c new file +10
-0+if (!init_fn->detect) + init_fn->flags |=X86_INIT_DETECTED ; +else { + ret = init_fn->detect (); +if (ret >0 ) + init_fn->flags |=X86_INIT_DETECTED ; + } + +if (init_fn->flags &X86_INIT_DETECTED ) { + init_fn->flags |=X86_INIT_DETECTED ;
Huh? It is already set.
Quoted 151 lines
arch/x86/kernel/init.c new file +8
-0+pr_debug ("Running early init %pF ...\n" , init_fn->early_init); + init_fn->early_init (); +pr_debug ("Completed early init %pF\n" , init_fn->early_init); +if (init_fn->flags &X86_INIT_FINISH_IF_DETECTED ) +break ; + } + } +}arch/x86/kernel/sort-init.c new file +114
-0@@ -0,0 +1,114 @@ +#include <linux/types.h> +#include <linux/kernel.h> +#include <linux/string.h> +#include <asm/x86_init_fn.h> + +extern struct x86_init_fn __tbl_x86_start_init_fns[], __tbl_x86_end_init_fns[]; + +static struct x86_init_fn *x86_init_fn_find_dep (struct x86_init_fn *start, +struct x86_init_fn *finish, +struct x86_init_fn *q) +{ +struct x86_init_fn *p; + +if (!q) +return NULL ; + +for (p = start; p < finish; p++) +if (p->detect == q->depend) +return p; + +return NULL ; +} + +static void x86_init_fn_sort (struct x86_init_fn *start, +struct x86_init_fn *finish) +{ + +struct x86_init_fn *p, *q, tmp; + +for (p = start; p < finish; p++) { +again: + q =x86_init_fn_find_dep (start, finish, p); +/* +* We are bit sneaky here. We use the memory address to figure +* out if the node we depend on is past our point, if so, swap. +*/ +if (q > p) { + tmp = *p; +memmove (p, q,sizeof (*p)); + *q = tmp; + goto again; + } + } + +} + +static void x86_init_fn_check (struct x86_init_fn *start, +struct x86_init_fn *finish) +{ +struct x86_init_fn *p, *q, *x; + +/* Simple cyclic dependency checker. */ +for (p = start; p < finish; p++) { +if (!p->depend) +continue ; + q =x86_init_fn_find_dep (start, finish, p); + x =x86_init_fn_find_dep (start, finish, q); +if (p == x) { +pr_info ("CYCLIC DEPENDENCY FOUND! %pF depends on %pF and vice-versa. BREAKING IT.\n" , + p->early_init, q->early_init); +/* Heavy handed way..*/ + x->depend =0 ; + } + } + +/* +* Validate sorting semantics. +* +* p depends on q so: +* - q must run first, so q < p. If q > p that's an issue +* as its saying p must run prior to q. We already sorted +* this table, this is a problem. +* +* - q's order level must be <= than p's as it should run first +*/ +for (p = start; p < finish; p++) { +if (!p->depend) +continue ; +/* +* Be pedantic and do a full search on the entire table, +* if we need further validation, after this is called +* one could use an optimized version which just searches +* on x86_init_fn_find_dep(p, finish, p), as we would have +* guarantee on proper ordering both at the dependency level +* and by order level. +*/ + q =x86_init_fn_find_dep (start, finish, p); +if (q && q > p) { +pr_info ("EXECUTION ORDER INVALID! %pF should be called before %pF!\n" , + p->early_init, q->early_init); + } + +/* +* Technically this would still work as the memmove() would +* have forced the dependency to run first, however we want +* strong semantics, so lets avoid these. +*/ +if (q && q->order_level > p->order_level) { +pr_info ("INVALID ORDER LEVEL! %pF should have an order level <= be called before %pF!\n" , + p->early_init, q->early_init); + } + } +} + +void x86_init_fn_init_tables (void ) +{ +unsigned int num_inits =table_num_entries (X86_INIT_FNS ); + +if (!num_inits) +return ; + +x86_init_fn_sort (__tbl_x86_start_init_fns, __tbl_x86_end_init_fns); +x86_init_fn_check (__tbl_x86_start_init_fns, __tbl_x86_end_init_fns); +}arch/x86/kernel/vmlinux.lds.S +15
-0@@ -265,6 +265,31 @@ SECTIONS __apicdrivers_end = .; } + /* + * x86 linker tables: this will sort all order levels + * lexicographically. Something like SORT_BY_INIT_PRIORITY() + * which sorts specifically on numeric order is desirable but + * that would impose a change on binutils to generalize + * SORT_BY_INIT_PRIORITY() for any section, SORT_BY_INIT_PRIORITY() + * is specific to the init sections, and that would also mean having + * a bump on requirments for building Linux. Using SORT() should + * suffice, * its what gpxe uses. For things in the same order + * level we rely on order in the C file, and also order on the + * Makefile. Since this solution also embraces the IOMMU init solution + * it has further sorting semantics implemented in C and its semantics + * are defined in x86_init_fn.h and implemented in sort-init.c. + */ + .tbl : {
arch/x86/kernel/vmlinux.lds.S SECTIONS · line 292
Quoted 10 lines
arch/x86/kernel/vmlinux.lds.S +10
-0+ __tbl_x86_start_init_fns = .; + *(SORT(.tbl.x86_init_fns.*)) + __tbl_x86_end_init_fns = .; + + /* ... etc ... */ + + /* You would add other tables here as needed */ + } + . = ALIGN(8); +
There is an extra .= ALIGN(8) there ?
Quoted 18 lines
arch/x86/kernel/vmlinux.lds.S +0
-0. = ALIGN(8); /* * .exit.text is discard at runtime, not link time, to deal witharch/x86/tools/relocs.c +1
-0@@ -56,6 +56,7 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = {"__x86_cpu_dev_(start|end)|" "(__parainstructions|__alt_instructions)(|_end)|" "(__iommu_table|__apicdrivers|__smp_locks)(|_end)|" +"(__tbl_x86_(start|end)_init_fns|" "__(start|end)_pci_.*|" "__(start|end)_builtin_fw|" "__(start|stop)___ksymtab(|_gpl|_unused|_unused_gpl|_gpl_future)|" Signature · 1 line
2.6.2
Re: [RFC v1 4/8] x86/init: add linker table support
In reply to Luis R. RodriguezReview
Quoted 10 lines
(quoted diff) +10
-0+static bool x86_init_fn_supports_subarch(struct x86_init_fn *fn) +{ + if (!fn->supp_hardware_subarch) { + pr_err("Init sequence fails to declares any supported subarchs: %pF\n", fn->early_init); + WARN_ON(1); + } + if (BIT(boot_params.hdr.hardware_subarch) & fn->supp_hardware_subarch) + return true; + return false; +}
So the logic for this working is that boot_params.hdr.hardware_subarch
And the macros define two: BIT(X86_SUBARCH_PC) or BIT(X86_SUBARCH_XEN).
But hardware_subarch by default is set to zero. Which means if GRUB2, PXELinux, Xen multiboot1 don't set it - then the X86_SUBARCH_PC is choosen right?
1 << 0 & 1 << X86_SUBARCH_PC (which is zero).
For this to nicely work with Xen it ought to do this:
arch/x86/xen/enlighten.c +1 -0
@@ -1676,6 +1676,7 @@ asmlinkage __visible void __init xen_start_kernel(void) boot_params.hdr.ramdisk_image = initrd_start; boot_params.hdr.ramdisk_size = xen_start_info->mod_len; boot_params.hdr.cmd_line_ptr =__pa (xen_start_info->cmd_line); + boot_params.hdr.hardware_subarch =X86_SUBARCH_XEN ;if (!xen_initial_domain ()) {add_preferred_console ("xenboot" ,0 ,NULL );
?
Re: [RFC v1 4/8] x86/init: add linker table support
Quoted 36 lines · Konrad Rzeszutek Wilk
Review
Quoted 10 lines
(quoted diff) +10
-0+static bool x86_init_fn_supports_subarch(struct x86_init_fn *fn) +{ + if (!fn->supp_hardware_subarch) { + pr_err("Init sequence fails to declares any supported subarchs: %pF\n", fn->early_init); + WARN_ON(1); + } + if (BIT(boot_params.hdr.hardware_subarch) & fn->supp_hardware_subarch) + return true; + return false; +}So the logic for this working is that boot_params.hdr.hardware_subarch
And the macros define two:
BIT(X86_SUBARCH_PC)orBIT(X86_SUBARCH_XEN).But
hardware_subarchby default is set to zero. Which means if GRUB2, PXELinux, Xen multiboot1 don't set it - then theX86_SUBARCH_PCis choosen right?1 << 0 & 1 <<
X86_SUBARCH_PC(which is zero).For this to nicely work with Xen it ought to do this:
arch/x86/xen/enlighten.c +1
-0@@ -1676,6 +1676,7 @@ asmlinkage __visible void __init xen_start_kernel(void) boot_params.hdr.ramdisk_image = initrd_start; boot_params.hdr.ramdisk_size = xen_start_info->mod_len; boot_params.hdr.cmd_line_ptr =__pa (xen_start_info->cmd_line); + boot_params.hdr.hardware_subarch =X86_SUBARCH_XEN ;if (!xen_initial_domain ()) {add_preferred_console ("xenboot" ,0 ,NULL );
Quoted 1 line
?
That's correct for PV and PVH, likewise when qemu is required for HVM qemu could set it. I have the qemu change done but that should only cover HVM. A common place to set this as well could be the hypervisor, but currently the hypervisor doesn't set any boot_params, instead a generic struct is passed and the kernel code (for any OS) is expected to interpret this and then set the required values for the OS in the init path. Long term though if we wanted to merge init further one way could be to have the hypervisor just set the zero page cleanly for the different modes. If we needed more data other than the hardware_subarch we also have the hardware_subarch_data, that's a u64 , and how that is used would be up to the subarch. In Xen's case it could do what it wants with it. That would still mean perhaps defining as part of a Xen boot protocol a place where xen specific code can count on finding more Xen data passed by the hypervisor, the xen_start_info. That is, if we wanted to merge init paths this is something to consider.
One thing I considered on the question of who should set the zero page for Xen with the prospect of merging inits, or at least this subarch for both short term and long term are the obvious implications in terms of hypervisor / kernel / qemu combination requirements if the subarch is needed. Having it set in the kernel is an obvious immediate choice for PV / PVH but it means we can't merge init paths completely (down to asm inits), we'd still be able to merge some C init paths though, the first entry would still be different. Having the zero page set on the hypervisor would go long ways but it would mean a hypervisor change required.
These prospects are worth discussing, specially in light of Boris's hvmlite work.
Luis
H. Peter Anvin
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 29 lines · Luis R. Rodriguez
That's correct for PV and PVH, likewise when qemu is required for HVM qemu could set it. I have the qemu change done but that should only cover HVM. A common place to set this as well could be the hypervisor, but currently the hypervisor doesn't set any
boot_params, instead a generic struct is passed and the kernel code (for any OS) is expected to interpret this and then set the required values for the OS in the init path. Long term though if we wanted to merge init further one way could be to have the hypervisor just set the zero page cleanly for the different modes. If we needed more data other than thehardware_subarchwe also have thehardware_subarch_data, that's a u64 , and how that is used would be up to the subarch. In Xen's case it could do what it wants with it. That would still mean perhaps defining as part of a Xen boot protocol a place where xen specific code can count on finding more Xen data passed by the hypervisor, thexen_start_info. That is, if we wanted to merge init paths this is something to consider.One thing I considered on the question of who should set the zero page for Xen with the prospect of merging inits, or at least this subarch for both short term and long term are the obvious implications in terms of hypervisor / kernel / qemu combination requirements if the subarch is needed. Having it set in the kernel is an obvious immediate choice for PV / PVH but it means we can't merge init paths completely (down to asm inits), we'd still be able to merge some C init paths though, the first entry would still be different. Having the zero page set on the hypervisor would go long ways but it would mean a hypervisor change required.
Quoted 3 lines
These prospects are worth discussing, specially in light of Boris's hvmlite work.
The above doesn't make sense to me. hardware_subarch is really used when the boot sequence is somehow nonstandard. HVM probably doesn't need that.
-hpa
Re: [RFC v1 4/8] x86/init: add linker table support
Quoted 30 lines · H. Peter Anvin
Quoted 29 lines · Luis R. Rodriguez
That's correct for PV and PVH, likewise when qemu is required for HVM qemu could set it. I have the qemu change done but that should only cover HVM. A common place to set this as well could be the hypervisor, but currently the hypervisor doesn't set any
boot_params, instead a generic struct is passed and the kernel code (for any OS) is expected to interpret this and then set the required values for the OS in the init path. Long term though if we wanted to merge init further one way could be to have the hypervisor just set the zero page cleanly for the different modes. If we needed more data other than thehardware_subarchwe also have thehardware_subarch_data, that's a u64 , and how that is used would be up to the subarch. In Xen's case it could do what it wants with it. That would still mean perhaps defining as part of a Xen boot protocol a place where xen specific code can count on finding more Xen data passed by the hypervisor, thexen_start_info. That is, if we wanted to merge init paths this is something to consider.One thing I considered on the question of who should set the zero page for Xen with the prospect of merging inits, or at least this subarch for both short term and long term are the obvious implications in terms of hypervisor / kernel / qemu combination requirements if the subarch is needed. Having it set in the kernel is an obvious immediate choice for PV / PVH but it means we can't merge init paths completely (down to asm inits), we'd still be able to merge some C init paths though, the first entry would still be different. Having the zero page set on the hypervisor would go long ways but it would mean a hypervisor change required.
Quoted 6 lines
Quoted 3 lines
These prospects are worth discussing, specially in light of Boris's hvmlite work.
The above doesn't make sense to me.
hardware_subarchis really used when the boot sequence is somehow nonstandard.
Thanks for the feedback -- as it stands today hardware_subarch is only used by lguest, Moorestown, and CE4100 even though we had definitions for it for Xen -- this is not used yet. Its documentation does make references to differences for a paravirtualized environment, and uses a few examples but doesn't go into great depths about restrictions so its limitations in how we could use it were not clear to me.
Quoted 1 line
HVM probably doesn't need that.
Today HVM doesn't need it, but perhaps that is because it has not needed changes early on boot. Will it, or could it? I'd even invite us to consider the same for other hypervisors or PV hypervisors. I'll note that things like cpu_has_hypervisor() or derivatives (kvm_para_available() which is now used on drivers even, see sound/pci/intel8x0.c) requires init_hypervisor_platform() run, in terms of the x86 init sequence this is run pretty late at setup_arch(). Should code need to know hypervisor info anytime before that they have no generic option available.
I'm fine if we want to restrict hardware_subarch but I'll note the semantics were not that explicit to delineate clear differences and I just wanted to highlight the current early boot restriction of cpu_has_hypervisor().
Luis
H. Peter Anvin
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 68 lines · Luis R. Rodriguez
Quoted 3 lines · H. Peter Anvin
Quoted 2 lines · Luis R. Rodriguez
That's correct for PV and PVH, likewise when qemu is required for
HVM
Quoted 2 lines
Quoted 2 lines
qemu could set it. I have the qemu change done but that should only cover HVM. A common place to set this as well could be the
hypervisor,
Quoted 2 lines
Quoted 2 lines
but currently the hypervisor doesn't set any
boot_params, instead a generic struct is passed and the kernel code (for any OS) isexpected
Quoted 2 lines
Quoted 2 lines
to interpret this and then set the required values for the OS in the init path. Long term though if we wanted to merge init further one
way
Quoted 1 line
Quoted 1 line
could be to have the hypervisor just set the zero page cleanly for
the
Quoted 2 lines
Quoted 2 lines
different modes. If we needed more data other than the
hardware_subarchwe also have thehardware_subarch_data, that's au64
Quoted 2 lines
Quoted 2 lines
, and how that is used would be up to the subarch. In Xen's case it could do what it wants with it. That would still mean perhaps
defining
Quoted 6 lines
Quoted 6 lines
as part of a Xen boot protocol a place where xen specific code can count on finding more Xen data passed by the hypervisor, the
xen_start_info. That is, if we wanted to merge init paths this is something to consider.One thing I considered on the question of who should set the zero
page
Quoted 4 lines
Quoted 4 lines
for Xen with the prospect of merging inits, or at least this subarch for both short term and long term are the obvious implications in terms of hypervisor / kernel / qemu combination requirements if the subarch is needed. Having it set in the kernel is an obvious
immediate
Quoted 1 line
Quoted 1 line
choice for PV / PVH but it means we can't merge init paths
completely
Quoted 2 lines
Quoted 2 lines
(down to asm inits), we'd still be able to merge some C init paths though, the first entry would still be different. Having the zero
page
Quoted 9 lines
Quoted 6 lines
set on the hypervisor would go long ways but it would mean a hypervisor change required.
These prospects are worth discussing, specially in light of Boris's hvmlite work.
The above doesn't make sense to me.
hardware_subarchis really used when the boot sequence is somehow nonstandard.Thanks for the feedback -- as it stands today
hardware_subarchis only used by lguest, Moorestown, and CE4100 even though we had definitions for it for Xen -- this is not used yet. Its documentation does make references to differences for a paravirtualized environment, and uses a few examples but doesn't go into great depths about restrictions so its limitations in how we could use it were not clear to me.Quoted 1 line
HVM probably doesn't need that.
Today HVM doesn't need it, but perhaps that is because it has not needed changes early on boot. Will it, or could it? I'd even invite us to consider the same for other hypervisors or PV hypervisors. I'll note that things like
cpu_has_hypervisor()or derivatives (kvm_para_available()which is now used on drivers even, seesound/pci/intel8x0.c) requiresinit_hypervisor_platform()run, in terms of the x86 init sequence this is run pretty late atsetup_arch(). Should code need to know hypervisor info anytime before that they have no generic option available.
Quoted 6 lines
I'm fine if we want to restrict
hardware_subarchbut I'll note the semantics were not that explicit to delineate clear differences and I just wanted to highlight the current early boot restriction ofcpu_has_hypervisor().Luis
Basically, if the hardware is enumerable using standard PC mechanisms (PCI, ACPI) and doesn't need a special boot flow it should use type 0.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.
Re: [RFC v1 4/8] x86/init: add linker table support
Quoted 69 lines · H. Peter Anvin
Quoted 68 lines · Luis R. Rodriguez
Quoted 3 lines · H. Peter Anvin
Quoted 2 lines · Luis R. Rodriguez
That's correct for PV and PVH, likewise when qemu is required for
HVM
Quoted 2 lines
Quoted 2 lines
qemu could set it. I have the qemu change done but that should only cover HVM. A common place to set this as well could be the
hypervisor,
Quoted 2 lines
Quoted 2 lines
but currently the hypervisor doesn't set any
boot_params, instead a generic struct is passed and the kernel code (for any OS) isexpected
Quoted 2 lines
Quoted 2 lines
to interpret this and then set the required values for the OS in the init path. Long term though if we wanted to merge init further one
way
Quoted 1 line
Quoted 1 line
could be to have the hypervisor just set the zero page cleanly for
the
Quoted 2 lines
Quoted 2 lines
different modes. If we needed more data other than the
hardware_subarchwe also have thehardware_subarch_data, that's au64
Quoted 2 lines
Quoted 2 lines
, and how that is used would be up to the subarch. In Xen's case it could do what it wants with it. That would still mean perhaps
defining
Quoted 6 lines
Quoted 6 lines
as part of a Xen boot protocol a place where xen specific code can count on finding more Xen data passed by the hypervisor, the
xen_start_info. That is, if we wanted to merge init paths this is something to consider.One thing I considered on the question of who should set the zero
page
Quoted 4 lines
Quoted 4 lines
for Xen with the prospect of merging inits, or at least this subarch for both short term and long term are the obvious implications in terms of hypervisor / kernel / qemu combination requirements if the subarch is needed. Having it set in the kernel is an obvious
immediate
Quoted 1 line
Quoted 1 line
choice for PV / PVH but it means we can't merge init paths
completely
Quoted 2 lines
Quoted 2 lines
(down to asm inits), we'd still be able to merge some C init paths though, the first entry would still be different. Having the zero
page
Quoted 9 lines
Quoted 6 lines
set on the hypervisor would go long ways but it would mean a hypervisor change required.
These prospects are worth discussing, specially in light of Boris's hvmlite work.
The above doesn't make sense to me.
hardware_subarchis really used when the boot sequence is somehow nonstandard.Thanks for the feedback -- as it stands today
hardware_subarchis only used by lguest, Moorestown, and CE4100 even though we had definitions for it for Xen -- this is not used yet. Its documentation does make references to differences for a paravirtualized environment, and uses a few examples but doesn't go into great depths about restrictions so its limitations in how we could use it were not clear to me.Quoted 1 line
HVM probably doesn't need that.
Today HVM doesn't need it, but perhaps that is because it has not needed changes early on boot. Will it, or could it? I'd even invite us to consider the same for other hypervisors or PV hypervisors. I'll note that things like
cpu_has_hypervisor()or derivatives (kvm_para_available()which is now used on drivers even, seesound/pci/intel8x0.c) requiresinit_hypervisor_platform()run, in terms of the x86 init sequence this is run pretty late atsetup_arch(). Should code need to know hypervisor info anytime before that they have no generic option available.
Quoted 8 lines
Quoted 6 lines
I'm fine if we want to restrict
hardware_subarchbut I'll note the semantics were not that explicit to delineate clear differences and I just wanted to highlight the current early boot restriction ofcpu_has_hypervisor().Luis
Basically, if the hardware is enumerable using standard PC mechanisms (PCI, ACPI) and doesn't need a special boot flow it should use type 0.
I can extend the documentation as part of this to be clear.
I will note though that this still leaves a gap on code which might want to access the question "are we in a virt environment, and if so on which one" in between really early boot and right before init_hypervisor_platform(). Or rather, given subarch can be used by Xen and lguest but not KVM it means KVM doesn't get to use it. It may not need it, but its also rather trivial to set up on qemu, and I have a patch for that if we wanted one for KVM. That would extend the definition of subarch a bit more, but as it stands today its use was rather limited. Heck, subharch_data is to this day unused.
Luis
H. Peter Anvin
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 5 lines · Luis R. Rodriguez
Quoted 2 lines
Basically, if the hardware is enumerable using standard PC mechanisms (PCI, ACPI) and doesn't need a special boot flow it should use type 0.
I can extend the documentation as part of this to be clear.
Quoted 10 lines
I will note though that this still leaves a gap on code which might want to access the question "are we in a virt environment, and if so on which one" in between really early boot and right before
init_hypervisor_platform(). Or rather, given subarch can be used by Xen and lguest but not KVM it means KVM doesn't get to use it. It may not need it, but its also rather trivial to set up on qemu, and I have a patch for that if we wanted one for KVM. That would extend the definition of subarch a bit more, but as it stands today its use was rather limited. Heck,subharch_datais to this day unused.
KVM is not a subarch, and Xen HVM isn't either; the subarch was meant to be specifically to handle nonstandard boot entries; the CE4100 extension was itself kind of a hack.
If you have a genuine need for a "hypervisor type" then that is a separate thing and should be treated separately from subarch. However, you need to consider that some hypervisors can emulate other hypervisors and you may have more than one hypervisor API available.
-hpa
Re: [RFC v1 4/8] x86/init: add linker table support
Quoted 17 lines · H. Peter Anvin
Quoted 5 lines · Luis R. Rodriguez
Quoted 2 lines
Basically, if the hardware is enumerable using standard PC mechanisms (PCI, ACPI) and doesn't need a special boot flow it should use type 0.
I can extend the documentation as part of this to be clear.
Quoted 10 lines
I will note though that this still leaves a gap on code which might want to access the question "are we in a virt environment, and if so on which one" in between really early boot and right before
init_hypervisor_platform(). Or rather, given subarch can be used by Xen and lguest but not KVM it means KVM doesn't get to use it. It may not need it, but its also rather trivial to set up on qemu, and I have a patch for that if we wanted one for KVM. That would extend the definition of subarch a bit more, but as it stands today its use was rather limited. Heck,subharch_datais to this day unused.
Quoted 8 lines
KVM is not a subarch, and Xen HVM isn't either; the subarch was meant to be specifically to handle nonstandard boot entries; the CE4100 extension was itself kind of a hack.
If you have a genuine need for a "hypervisor type" then that is a separate thing and should be treated separately from subarch. However, you need to consider that some hypervisors can emulate other hypervisors and you may have more than one hypervisor API available.
Thanks, I've pegged this onto my series as a documentation extensions for boot params.
Luis
Re: [RFC v1 4/8] x86/init: add linker table support
In reply to Luis R. RodriguezEl 20/01/16 a les 22.33, Luis R. Rodriguez ha escrit:
Quoted 57 lines
Quoted 36 lines · Konrad Rzeszutek Wilk
Review
Quoted 10 lines
(quoted diff) +10
-0+static bool x86_init_fn_supports_subarch(struct x86_init_fn *fn) +{ + if (!fn->supp_hardware_subarch) { + pr_err("Init sequence fails to declares any supported subarchs: %pF\n", fn->early_init); + WARN_ON(1); + } + if (BIT(boot_params.hdr.hardware_subarch) & fn->supp_hardware_subarch) + return true; + return false; +}So the logic for this working is that boot_params.hdr.hardware_subarch
And the macros define two:
BIT(X86_SUBARCH_PC)orBIT(X86_SUBARCH_XEN).But
hardware_subarchby default is set to zero. Which means if GRUB2, PXELinux, Xen multiboot1 don't set it - then theX86_SUBARCH_PCis choosen right?1 << 0 & 1 <<
X86_SUBARCH_PC(which is zero).For this to nicely work with Xen it ought to do this:
arch/x86/xen/enlighten.c +1
-0@@ -1676,6 +1676,7 @@ asmlinkage __visible void __init xen_start_kernel(void) boot_params.hdr.ramdisk_image = initrd_start; boot_params.hdr.ramdisk_size = xen_start_info->mod_len; boot_params.hdr.cmd_line_ptr =__pa (xen_start_info->cmd_line); + boot_params.hdr.hardware_subarch =X86_SUBARCH_XEN ;if (!xen_initial_domain ()) {add_preferred_console ("xenboot" ,0 ,NULL );Quoted 1 line
?
That's correct for PV and PVH, likewise when qemu is required for HVM qemu could set it. I have the qemu change done but that should only cover HVM. A common place to set this as well could be the hypervisor, but currently the hypervisor doesn't set any
boot_params, instead a generic struct is passed and the kernel code (for any OS) is expected to interpret this and then set the required values for the OS in the init path. Long term though if we wanted to merge init further one way could be to have the hypervisor just set the zero page cleanly for the different modes. If we needed more data other than thehardware_subarchwe also have thehardware_subarch_data, that's a u64 , and how that is used would be up to the subarch. In Xen's case it could do what it wants with it. That would still mean perhaps defining as part of a Xen boot protocol a place where xen specific code can count on finding more Xen data passed by the hypervisor, thexen_start_info. That is, if we wanted to merge init paths this is something to consider.
Quoted 10 lines
One thing I considered on the question of who should set the zero page for Xen with the prospect of merging inits, or at least this subarch for both short term and long term are the obvious implications in terms of hypervisor / kernel / qemu combination requirements if the subarch is needed. Having it set in the kernel is an obvious immediate choice for PV / PVH but it means we can't merge init paths completely (down to asm inits), we'd still be able to merge some C init paths though, the first entry would still be different. Having the zero page set on the hypervisor would go long ways but it would mean a hypervisor change required.
I don't think the hypervisor should be setting Linux specific boot related parameters, the boot ABI should be OS agnostic. IMHO, a small shim should be added to Linux in order to set what Linux requires when entering from a Xen entry point.
Roger.
Boris Ostrovsky
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 66 lines · Roger Pau Monné
El 20/01/16 a les 22.33, Luis R. Rodriguez ha escrit:
Quoted 55 lines
Quoted 35 lines · Konrad Rzeszutek Wilk
Review
Quoted 10 lines
(quoted diff) +10
-0+static bool x86_init_fn_supports_subarch(struct x86_init_fn *fn) +{ + if (!fn->supp_hardware_subarch) { + pr_err("Init sequence fails to declares any supported subarchs: %pF\n", fn->early_init); + WARN_ON(1); + } + if (BIT(boot_params.hdr.hardware_subarch) & fn->supp_hardware_subarch) + return true; + return false; +}So the logic for this working is that boot_params.hdr.hardware_subarch
And the macros define two:
BIT(X86_SUBARCH_PC)orBIT(X86_SUBARCH_XEN).But
hardware_subarchby default is set to zero. Which means if GRUB2, PXELinux, Xen multiboot1 don't set it - then theX86_SUBARCH_PCis choosen right?1 << 0 & 1 <<
X86_SUBARCH_PC(which is zero).For this to nicely work with Xen it ought to do this:
arch/x86/xen/enlighten.c +1
-0@@ -1676,6 +1676,7 @@ asmlinkage __visible void __init xen_start_kernel(void) boot_params.hdr.ramdisk_image = initrd_start; boot_params.hdr.ramdisk_size = xen_start_info->mod_len; boot_params.hdr.cmd_line_ptr =__pa (xen_start_info->cmd_line); + boot_params.hdr.hardware_subarch =X86_SUBARCH_XEN ;if (!xen_initial_domain ()) {add_preferred_console ("xenboot" ,0 ,NULL );Quoted 1 line
?
That's correct for PV and PVH, likewise when qemu is required for HVM qemu could set it. I have the qemu change done but that should only cover HVM. A common place to set this as well could be the hypervisor, but currently the hypervisor doesn't set any
boot_params, instead a generic struct is passed and the kernel code (for any OS) is expected to interpret this and then set the required values for the OS in the init path. Long term though if we wanted to merge init further one way could be to have the hypervisor just set the zero page cleanly for the different modes. If we needed more data other than thehardware_subarchwe also have thehardware_subarch_data, that's a u64 , and how that is used would be up to the subarch. In Xen's case it could do what it wants with it. That would still mean perhaps defining as part of a Xen boot protocol a place where xen specific code can count on finding more Xen data passed by the hypervisor, thexen_start_info. That is, if we wanted to merge init paths this is something to consider.Quoted 10 lines
One thing I considered on the question of who should set the zero page for Xen with the prospect of merging inits, or at least this subarch for both short term and long term are the obvious implications in terms of hypervisor / kernel / qemu combination requirements if the subarch is needed. Having it set in the kernel is an obvious immediate choice for PV / PVH but it means we can't merge init paths completely (down to asm inits), we'd still be able to merge some C init paths though, the first entry would still be different. Having the zero page set on the hypervisor would go long ways but it would mean a hypervisor change required.
Quoted 4 lines
I don't think the hypervisor should be setting Linux specific boot related parameters, the boot ABI should be OS agnostic. IMHO, a small shim should be added to Linux in order to set what Linux requires when entering from a Xen entry point.
And that's exactly what HVMlite does. Most of this shim layer is setting up boot_params, after which we jump to standard x86 boot path (i.e. startup_{32|64}). With hardware_subarch set to zero.
-boris
H. Peter Anvin
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 4 lines · Boris Ostrovsky
Quoted 4 lines
I don't think the hypervisor should be setting Linux specific boot related parameters, the boot ABI should be OS agnostic. IMHO, a small shim should be added to Linux in order to set what Linux requires when entering from a Xen entry point.
For the record, this is exactly what hardware_subarch is meant to do:
revector to a stub to do this kind of setup, for a subarchitecture which doesn't have a natural stub like BIOS or EFI. In the case of Xen PV, or lguest, there are special care that has to be done very early in the path due to the nonstandard handling of page tables, which is another reason for this field.
Quoted 3 lines
And that's exactly what HVMlite does. Most of this shim layer is setting up
boot_params, after which we jump to standard x86 boot path (i.e. startup_{32|64}). Withhardware_subarchset to zero.
Which is the way to do it as long as the early code can be the same.
-hpa
Re: [RFC v1 4/8] x86/init: add linker table support
Quoted 5 lines · H. Peter Anvin
Quoted 3 lines
And that's exactly what HVMlite does. Most of this shim layer is setting up
boot_params, after which we jump to standard x86 boot path (i.e. startup_{32|64}). Withhardware_subarchset to zero.Which is the way to do it as long as the early code can be the same.
To be clear, with the subarchand linker table suggested in my patch series, it should be possible to have the same exact entry point, the Xen PV setup code could run early in the order. For instance in the linker table we could use the reserved order levels 01-09 for PV hypervisor code:
+/* Init order levels, we can start at 01 but reserve 01-09 for now */
+#define X86_INIT_ORDER_EARLY 10 +#define X86_INIT_ORDER_NORMAL 30 +#define X86_INIT_ORDER_LATE 50
So perhaps X86_INIT_ORDER_PV as 05 later.
The standard x86 init would just then be:
asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
{
x86_init_fn_init_tables();
x86_init_fn_early_init();
}The PV init code would kick in early and could parse the boot_params.hdr.hardware_subarch_data pointer as it sees fit.
Luis
H. Peter Anvin
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 22 lines · Luis R. Rodriguez
Quoted 5 lines · H. Peter Anvin
Quoted 3 lines
And that's exactly what HVMlite does. Most of this shim layer is setting up
boot_params, after which we jump to standard x86 boot path (i.e. startup_{32|64}). Withhardware_subarchset to zero.Which is the way to do it as long as the early code can be the same.
To be clear, with the subarchand linker table suggested in my patch series, it should be possible to have the same exact entry point, the Xen PV setup code could run early in the order. For instance in the linker table we could use the reserved order levels 01-09 for PV hypervisor code:
(quoted diff) +4
-0+/* Init order levels, we can start at 01 but reserve 01-09 for now */ +#define X86_INIT_ORDER_EARLY 10 +#define X86_INIT_ORDER_NORMAL 30 +#define X86_INIT_ORDER_LATE 50So perhaps
X86_INIT_ORDER_PVas 05 later.The standard x86 init would just then be:
Quoted 9 lines
asmlinkage
__visiblevoid__initx86_64_start_kernel(char * real_mode_data){ x86_init_fn_init_tables(); x86_init_fn_early_init(); }The PV init code would kick in early and could parse the boot_params.hdr.hardware_subarch_data pointer as it sees fit.
Right... we already do that.
I don't even think you need to initialize any tables. At least on i386, we have to do this in assembly code. However, it is just a simple table walk. :)
-hpa
H. Peter Anvin
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 7 lines · H. Peter Anvin
Right... we already do that.
I don't even think you need to initialize any tables. At least on i386, we have to do this in assembly code. However, it is just a simple table walk. :)
It might make more sense to make subarch its own table, though, although I haven't looked at your code in enough detail to say.
-hpa
Re: [RFC v1 4/8] x86/init: add linker table support
Quoted 11 lines · H. Peter Anvin
Quoted 7 lines · H. Peter Anvin
Right... we already do that.
I don't even think you need to initialize any tables. At least on i386, we have to do this in assembly code. However, it is just a simple table walk. :)
It might make more sense to make subarch its own table, though, although I haven't looked at your code in enough detail to say.
The gist of it:
void x86_init_fn_early_init() {
for_each_table_entry(init_fn, X86_INIT_FNS) {
if supported subarc
init_fn->early_init();
}
void x86_init_fn_setup_arch() {
for_each_table_entry(init_fn, X86_INIT_FNS) {
if supported subarc
init_fn->setup_archt();
}
Right now the code defines just an init routine at the stage for x86_64_start_reservations(), we call the callback early_init(), ie setup_arch() doesn't exist yet. Since certain routines can run on either Xen or bare-metal we allow a bitwise OR for the subarch as a bitmask. I'll think a bit more about how to use subarch for a table, but can't fit it yet. I'd like to later revise kfree'ing unused sections, and a table per subarch might be nice for that, but otherwise this flow seems easy to follow, so if we can kfree sections within a table we could still accomplish that later.
Luis
Re: [RFC v1 4/8] x86/init: add linker table support
In reply to H. Peter AnvinQuoted 7 lines · H. Peter Anvin
Quoted 3 lines
The PV init code would kick in early and could parse the boot_params.hdr.hardware_subarch_data pointer as it sees fit.
Right... we already do that.
I don't even think you need to initialize any tables.
The init above is for the sorting. Its not needed unless your subsystem uses it, its the same sort as with the IOMMU stuff only with stronger semantics. In theory one would *not* need to do this so early, it could wait, provided you at least are confident the linker order suffices for all routines called early. To be clear, the struct proposed defines a set of callbacks for calling feature's callbacks at different levels of the x86 init path. As this patch series is concerned it had one for x86_64_start_reservations(). This could easily be extended to also include one to be called for instance at setup_arch(). If we were able to solve the ability to make use of subarch so early as of we could then have a callback for x86_64_start_kernel(), then one at x86_64_start_reservations() and perhaps one at setup_arch() later -- Kasan is an example that could fit this example in the future. What I mean is that we could avoid the sort or wait for the run time sort later until x86_64_start_reservations() or even setup_arch() provided the deafult linker order suffices for the series of previous callbacks. Its up to us but I'm taken care to be pedantic about semantics here.
Quoted 2 lines
At least on i386,
we have to do this in assembly code.
Neat! How is that order kept?
Quoted 1 line
However, it is just a simple table walk. :)
Luis
H. Peter Anvin
reviewerRe: [RFC v1 4/8] x86/init: add linker table support
Quoted 6 lines · Luis R. Rodriguez
Quoted 3 lines
At least on i386,
we have to do this in assembly code.Neat! How is that order kept?
Right now subarch_entries[] is just an array indexed by subarch number hardcoded in head_32.S.
However, if you have a list of (id, target) then you could just iterate over it.
-hpa
RFC 5/8 x86/init: move ebda reservations into linker table
In reply to Luis R. RodriguezFrom: "Luis R. Rodriguez" <[email protected]>
This lets us annotate its requirements specifically for PC and lguest subarchitectures. While at it since head.c just has ebda data rename it. Since we're using linker tables and both x86 32-bit and 64-bit have them we don't need to declare a call for this separately. This lets us also keep this declared static now.
Since we're using linker tables and have support to annotate subarchitecture do that instead. pv_legacy() is incorrect as its really only for legacy PV guests. There's no need for pv_legacy() check anymore now.
---
arch/x86/include/asm/bios_ebda.h | 2 --
arch/x86/kernel/Makefile | 2 +-
arch/x86/kernel/{head.c => ebda.c} | 6 +++---
arch/x86/kernel/head32.c | 2 --
arch/x86/kernel/head64.c | 2 --
5 files changed, 4 insertions(+), 10 deletions(-)
rename arch/x86/kernel/{head.c => ebda.c} (94%)arch/x86/include/asm/bios_ebda.h +0 -2
@@ -38,8 +38,6 @@ static inline unsigned int get_bios_ebda_length(void)return length; } -void reserve_ebda_region (void ); -#ifdef CONFIG_X86_CHECK_BIOS_CORRUPTION /* * This is obviously not a great place for this, but we want to be
arch/x86/kernel/Makefile +1 -1
@@ -2,7 +2,7 @@ # Makefile for the linux kernel. # -extra-y := head_$(BITS).o head$(BITS).o head.o vmlinux.lds +extra-y := head_$(BITS).o head$(BITS).o ebda.o vmlinux.lds CPPFLAGS_vmlinux.lds += -U$(UTS_MACHINE)
arch/x86/kernel/ebda.c renamed from arch/x86/kernel/head.c +3 -3
@@ -4,6 +4,7 @@#include <asm/setup.h> #include <asm/bios_ebda.h> +#include <asm/x86_init.h> /* * The BIOS places the EBDA/XBDA at the top of conventional
@@ -26,7 +27,7 @@#define LOWMEM_CAP 0x9f000U/* Absolute maximum */ #define INSANE_CUTOFF 0x20000U/* Less than this = insane */ -void __init reserve_ebda_region (void ) +static void __init reserve_ebda_region (void ) {unsigned int lowmem, ebda_addr;
@@ -38,8 +39,6 @@ void __init reserve_ebda_region(void)* that the paravirt case can handle memory setup * correctly, without our help. */ -if (paravirt_legacy ()) -return ;/* end of low (conventional) memory */ lowmem = *(unsigned short *)__va (BIOS_LOWMEM_KILOBYTES );
@@ -69,3 +68,4 @@ void __init reserve_ebda_region(void)/* reserve all memory between lowmem and the 1MB mark */ memblock_reserve (lowmem,0x100000 - lowmem); } +x86_init_early_pc_simple (reserve_ebda_region);
arch/x86/kernel/head32.c +0 -2
@@ -26,8 +26,6 @@ static void __init i386_default_early_setup(void)/* Initialize 32bit specific setup functions */ x86_init.resources.reserve_resources = i386_reserve_resources; x86_init.mpparse.setup_ioapic_ids = setup_ioapic_ids_from_mpc; - -reserve_ebda_region (); } asmlinkage __visible void __init i386_start_kernel(void)
arch/x86/kernel/head64.c +0 -2
@@ -194,7 +194,5 @@ void __init x86_64_start_reservations(char *real_mode_data)x86_init_fn_init_tables ();x86_init_fn_early_init (); -reserve_ebda_region (); -start_kernel (); }
Signature · 1 line
2.6.2
Re: [RFC v1 5/8] x86/init: move ebda reservations into linker table
Quoted 9 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
This lets us annotate its requirements specifically for PC and lguest subarchitectures. While at it since
head.cjust has ebda data rename it. Since we're using linker tables and both x86 32-bit and 64-bit have them we don't need to declare a call for this separately. This lets us also keep this declared static now.
Quoted 4 lines
Since we're using linker tables and have support to annotate subarchitecture do that instead.
pv_legacy()is incorrect as its really only for legacy PV guests. There's no need forpv_legacy()check anymore now.
I'm entirely ignorant of anything going on in gPXE/iPXE.
Can you explain what a linker table *does*? It looks like all you've done in this patch is to move code around. What actually happens?
--Andy
H. Peter Anvin
reviewerRe: [RFC v1 5/8] x86/init: move ebda reservations into linker table
Quoted 6 lines · Andy Lutomirski
I'm entirely ignorant of anything going on in gPXE/iPXE.
Can you explain what a linker table *does*? It looks like all you've done in this patch is to move code around. What actually happens?
A linker table is a data structure that is stitched together from items in multiple object files.
We already have a *bunch* of linker tables in Linux, mostly the init tables, but they are all built in an ad hoc manner which requires linker script modifications, which are of course per architecture.
My desire would be to make a general linker table facility so that a new linker table can be implemented by changing C code only.
-hpa
Re: [RFC v1 5/8] x86/init: move ebda reservations into linker table
Quoted 11 lines · H. Peter Anvin
Quoted 6 lines · Andy Lutomirski
I'm entirely ignorant of anything going on in gPXE/iPXE.
Can you explain what a linker table *does*? It looks like all you've done in this patch is to move code around. What actually happens?
A linker table is a data structure that is stitched together from items in multiple object files.
Quoted 6 lines
We already have a *bunch* of linker tables in Linux, mostly the init tables, but they are all built in an ad hoc manner which requires linker script modifications, which are of course per architecture.
My desire would be to make a general linker table facility so that a new linker table can be implemented by changing C code only.
Sounds good to me.
--Andy
Re: [RFC v1 5/8] x86/init: move ebda reservations into linker table
Quoted 12 lines · Andy Lutomirski
Quoted 11 lines · H. Peter Anvin
Quoted 6 lines · Andy Lutomirski
I'm entirely ignorant of anything going on in gPXE/iPXE.
Can you explain what a linker table *does*? It looks like all you've done in this patch is to move code around. What actually happens?
A linker table is a data structure that is stitched together from items in multiple object files.
Quoted 8 lines
Quoted 6 lines
We already have a *bunch* of linker tables in Linux, mostly the init tables, but they are all built in an ad hoc manner which requires linker script modifications, which are of course per architecture.
My desire would be to make a general linker table facility so that a new linker table can be implemented by changing C code only.
Sounds good to me.
That's what this actually accomplishes, there are just a few caveats to consider, more on this shortly on the other thread in this patch series.
Luis
Re: [RFC v1 5/8] x86/init: move ebda reservations into linker table
In reply to Luis R. RodriguezQuoted 9 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
This lets us annotate its requirements specifically for PC and lguest subarchitectures. While at it since
head.cjust has ebda data rename it. Since we're using linker tables and both x86 32-bit and 64-bit have them we don't need to declare a call for this separately. This lets us also keep this declared static now.
Quoted 4 lines
Since we're using linker tables and have support to annotate subarchitecture do that instead.
pv_legacy()is incorrect as its really only for legacy PV guests. There's no need forpv_legacy()check anymore now.
<nods>
Quoted 93 lines
Signed-off-by Luis R. Rodriguez <[email protected]>--- arch/x86/include/asm/bios_ebda.h | 2 -- arch/x86/kernel/Makefile | 2 +- arch/x86/kernel/{head.c => ebda.c} | 6 +++--- arch/x86/kernel/head32.c | 2 -- arch/x86/kernel/head64.c | 2 -- 5 files changed, 4 insertions(+), 10 deletions(-) rename arch/x86/kernel/{head.c => ebda.c} (94%)arch/x86/include/asm/bios_ebda.h +0
-2@@ -38,8 +38,6 @@ static inline unsigned int get_bios_ebda_length(void)return length; } -void reserve_ebda_region (void ); -#ifdef CONFIG_X86_CHECK_BIOS_CORRUPTION /* * This is obviously not a great place for this, but we want to be arch/x86/kernel/Makefile +1
-1@@ -2,7 +2,7 @@ # Makefile for the linux kernel. # -extra-y := head_$(BITS).o head$(BITS).o head.o vmlinux.lds +extra-y := head_$(BITS).o head$(BITS).o ebda.o vmlinux.lds CPPFLAGS_vmlinux.lds += -U$(UTS_MACHINE)arch/x86/kernel/ebda.c renamed from arch/x86/kernel/head.c +3
-3@@ -4,6 +4,7 @@#include <asm/setup.h> #include <asm/bios_ebda.h> +#include <asm/x86_init.h> /* * The BIOS places the EBDA/XBDA at the top of conventional @@ -26,7 +27,7 @@#define LOWMEM_CAP 0x9f000U/* Absolute maximum */ #define INSANE_CUTOFF 0x20000U/* Less than this = insane */ -void __init reserve_ebda_region (void ) +static void __init reserve_ebda_region (void ) {unsigned int lowmem, ebda_addr;@@ -38,8 +39,6 @@ void __init reserve_ebda_region(void)* that the paravirt case can handle memory setup * correctly, without our help. */ -if (paravirt_legacy ()) -return ;/* end of low (conventional) memory */ lowmem = *(unsigned short *)__va (BIOS_LOWMEM_KILOBYTES );@@ -69,3 +68,4 @@ void __init reserve_ebda_region(void)/* reserve all memory between lowmem and the 1MB mark */ memblock_reserve (lowmem,0x100000 - lowmem); } +x86_init_early_pc_simple (reserve_ebda_region);arch/x86/kernel/head32.c +0
-2@@ -26,8 +26,6 @@ static void __init i386_default_early_setup(void)/* Initialize 32bit specific setup functions */ x86_init.resources.reserve_resources = i386_reserve_resources; x86_init.mpparse.setup_ioapic_ids = setup_ioapic_ids_from_mpc; - -reserve_ebda_region (); } asmlinkage __visible void __init i386_start_kernel(void)arch/x86/kernel/head64.c +0
-0
arch/x86/kernel/head64.c void __init x86_64_start_reservations(char *real_mode_data) · line 198
Quoted 5 lines
arch/x86/kernel/head64.c +0
-2@@ -194,7 +194,5 @@ void __init x86_64_start_reservations(char *real_mode_data)x86_init_fn_init_tables ();x86_init_fn_early_init (); -reserve_ebda_region (); -
Nice!
Quoted 5 lines
arch/x86/kernel/head64.c +0
-0start_kernel (); }Signature · 1 line
2.6.2
RFC 6/8 x86/init: use linker table for i386 early setup
In reply to Luis R. RodriguezFrom: "Luis R. Rodriguez" <[email protected]>
This also annotates this is only used for PC and
lguest hardware subarchitectures.
--- arch/x86/kernel/head32.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/kernel/head32.c +4 -4
@@ -21,12 +21,15 @@#include <asm/bootparam_utils.h> #include <asm/x86_init.h> -static void __init i386_default_early_setup (void ) +static void __init i386_set_setup_funcs (void ) {/* Initialize 32bit specific setup functions */ x86_init.resources.reserve_resources = i386_reserve_resources; x86_init.mpparse.setup_ioapic_ids = setup_ioapic_ids_from_mpc; } +x86_init_early (BIT (X86_SUBARCH_PC ) | +BIT (X86_SUBARCH_LGUEST ), +NULL ,NULL , i386_set_setup_funcs); asmlinkage __visible void __init i386_start_kernel(void) {
@@ -41,9 +44,6 @@ asmlinkage __visible void __init i386_start_kernel(void)case X86_SUBARCH_CE4100 :x86_ce4100_early_setup ();break ; -default : -i386_default_early_setup (); -break ; }x86_init_fn_init_tables ();
Signature · 1 line
2.6.2
Re: [RFC v1 6/8] x86/init: use linker table for i386 early setup
Quoted 6 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
This also annotates this is only used for PC and
lguest hardware subarchitectures.Signed-off-by Luis R. Rodriguez <[email protected]>
On Xen the code path (before this patch) was:xen_start_kernel->i386_start_kernel->i386_default_early_setup
and now you remove the i386_default_early_setup call?
The mpparse.setup_ioapic_ids is irrelevant as under Xen code it does:
709 /* Avoid searching for BIOS MP tables */
1710 x86_init.mpparse.find_smp_config = x86_init_noop;
1711 x86_init.mpparse.get_smp_config = x86_init_uint_noop;
Great, MP tables is completly ignored.
The next one : i386_reserve_resources
Looks to be reserving VGA RAM .. which is a bit pointless when you have no legacy devices.
The reserve_standard_io_resources means the /proc/ioport is smaller now.
Actually looking under an PV guest it is:
# cat /proc/ioports 0000-001f : dma1 0020-0021 : pic1 0040-0043 : timer0 0050-0053 : timer1 0060-0060 : keyboard 0064-0064 : keyboard 0070-0071 : rtc_cmos 0080-008f : dma page reg 00a0-00a1 : pic2 00c0-00df : dma2 00f0-00ff : fpu # cat /proc/iomem 00000000-00000fff : reserved 00001000-0009ffff : System RAM 000a0000-000fffff : reserved
Which means /proc/ioports won't have anything in it (with your change)
And /proc/iomem will have the 000a0000-000fffff region available (maybe?) Ah no. We mark it as reserved in the early part of boot:
[ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
I am curious - what does your kernel run as PV look like?
Thanks.
Quoted 38 lines
--- arch/x86/kernel/head32.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)arch/x86/kernel/head32.c +4
-4@@ -21,12 +21,15 @@#include <asm/bootparam_utils.h> #include <asm/x86_init.h> -static void __init i386_default_early_setup (void ) +static void __init i386_set_setup_funcs (void ) {/* Initialize 32bit specific setup functions */ x86_init.resources.reserve_resources = i386_reserve_resources; x86_init.mpparse.setup_ioapic_ids = setup_ioapic_ids_from_mpc; } +x86_init_early (BIT (X86_SUBARCH_PC ) | +BIT (X86_SUBARCH_LGUEST ), +NULL ,NULL , i386_set_setup_funcs); asmlinkage __visible void __init i386_start_kernel(void) {@@ -41,9 +44,6 @@ asmlinkage __visible void __init i386_start_kernel(void)case X86_SUBARCH_CE4100 :x86_ce4100_early_setup ();break ; -default : -i386_default_early_setup (); -break ; }x86_init_fn_init_tables ();Signature · 1 line
2.6.2
Re: [RFC v1 6/8] x86/init: use linker table for i386 early setup
Quoted 6 lines · Konrad Rzeszutek Wilk
Quoted 5 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
This also annotates this is only used for PC and
lguest hardware subarchitectures.
Quoted 6 lines
Quoted 1 line
Signed-off-by Luis R. Rodriguez <[email protected]>On Xen the code path (before this patch) was:
xen_start_kernel->i386_start_kernel->i386_default_early_setupand now you remove the
i386_default_early_setupcall?
No you're right, this should include the Xen subarch as well, thanks for that.
Luis
Re: [RFC v1 6/8] x86/init: use linker table for i386 early setup
Quoted 8 lines · Luis R. Rodriguez
Quoted 6 lines · Konrad Rzeszutek Wilk
Quoted 5 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
This also annotates this is only used for PC and
lguest hardware subarchitectures.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Signed-off-by Luis R. Rodriguez <[email protected]>On Xen the code path (before this patch) was:
xen_start_kernel->i386_start_kernel->i386_default_early_setupand now you remove the
i386_default_early_setupcall?No you're right, this should include the Xen subarch as well, thanks for that.
FYI - I've mended this fix into my upcoming v2 series, I've also made a separate patch to make annotate the XEN subarch on the PV guest boot path (xen_start_kernel()). While at it, I'll go and also add an extension of documentation to the subarch based on the conversations on this thread, and make lguest explicitly use LGUEST subarch rather than just the digit.
Luis
RFC 7/8 x86/init: user linker table for ce4100 early setup
In reply to Luis R. RodriguezFrom: "Luis R. Rodriguez" <[email protected]>
Using the linker table removes the need for the #ifdef'ery and clutter on head32.c.
--- arch/x86/include/asm/setup.h | 6 ------ arch/x86/kernel/head32.c | 3 --- arch/x86/platform/ce4100/ce4100.c | 4 +++- 3 files changed, 3 insertions(+), 10 deletions(-)
arch/x86/include/asm/setup.h +0 -6
@@ -51,12 +51,6 @@ extern void x86_intel_mid_early_setup(void);static inline void x86_intel_mid_early_setup (void ) { }#endif -#ifdef CONFIG_X86_INTEL_CE -extern void x86_ce4100_early_setup (void ); -#else -static inline void x86_ce4100_early_setup (void ) { } -#endif -#ifndef _SETUP#include <asm/espfix.h>
arch/x86/kernel/head32.c +0 -3
@@ -41,9 +41,6 @@ asmlinkage __visible void __init i386_start_kernel(void)case X86_SUBARCH_INTEL_MID :x86_intel_mid_early_setup ();break ; -case X86_SUBARCH_CE4100 : -x86_ce4100_early_setup (); -break ; }x86_init_fn_init_tables ();
arch/x86/platform/ce4100/ce4100.c +3 -1
@@ -24,6 +24,7 @@#include <asm/io.h> #include <asm/io_apic.h> #include <asm/emergency-restart.h> +#include <asm/x86_init.h> static int ce4100_i8042_detect (void ) {
@@ -144,7 +145,7 @@ static void sdv_pci_init(void)* CE4100 specific x86_init function overrides and early setup * calls. */ -void __init x86_ce4100_early_setup (void ) +static void __init x86_ce4100_early_setup (void ) { x86_init.oem.arch_setup = sdv_arch_setup; x86_platform.i8042_detect = ce4100_i8042_detect;
@@ -166,3 +167,4 @@ void __init x86_ce4100_early_setup(void) pm_power_off = ce4100_power_off; } +x86_init_early (BIT (X86_SUBARCH_CE4100 ),NULL ,NULL , x86_ce4100_early_setup);
Signature · 1 line
2.6.2
Re: [RFC v1 7/8] x86/init: user linker table for ce4100 early setup
Quoted 6 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
Using the linker table removes the need for the #ifdef'ery and clutter on
head32.c.Signed-off-by Luis R. Rodriguez <[email protected]>
Quoted 66 lines
--- arch/x86/include/asm/setup.h | 6 ------ arch/x86/kernel/head32.c | 3 --- arch/x86/platform/ce4100/ce4100.c | 4 +++- 3 files changed, 3 insertions(+), 10 deletions(-)arch/x86/include/asm/setup.h +0
-6@@ -51,12 +51,6 @@ extern void x86_intel_mid_early_setup(void);static inline void x86_intel_mid_early_setup (void ) { }#endif -#ifdef CONFIG_X86_INTEL_CE -extern void x86_ce4100_early_setup (void ); -#else -static inline void x86_ce4100_early_setup (void ) { } -#endif -#ifndef _SETUP#include <asm/espfix.h> arch/x86/kernel/head32.c +0
-3@@ -41,9 +41,6 @@ asmlinkage __visible void __init i386_start_kernel(void)case X86_SUBARCH_INTEL_MID :x86_intel_mid_early_setup ();break ; -case X86_SUBARCH_CE4100 : -x86_ce4100_early_setup (); -break ; }x86_init_fn_init_tables ();arch/x86/platform/ce4100/ce4100.c +3
-1@@ -24,6 +24,7 @@#include <asm/io.h> #include <asm/io_apic.h> #include <asm/emergency-restart.h> +#include <asm/x86_init.h> static int ce4100_i8042_detect (void ) {@@ -144,7 +145,7 @@ static void sdv_pci_init(void)* CE4100 specific x86_init function overrides and early setup * calls. */ -void __init x86_ce4100_early_setup (void ) +static void __init x86_ce4100_early_setup (void ) { x86_init.oem.arch_setup = sdv_arch_setup; x86_platform.i8042_detect = ce4100_i8042_detect;@@ -166,3 +167,4 @@ void __init x86_ce4100_early_setup(void) pm_power_off = ce4100_power_off; } +x86_init_early (BIT (X86_SUBARCH_CE4100 ),NULL ,NULL , x86_ce4100_early_setup);Signature · 1 line
2.6.2
RFC 8/8 x86/init: use linker table for mid early setup
In reply to Luis R. RodriguezFrom: "Luis R. Rodriguez" <[email protected]>
Using the linker table removes the need for the #ifdef'ery and clutter on head32.c.
--- arch/x86/include/asm/setup.h | 6 ------ arch/x86/kernel/head32.c | 7 ------- arch/x86/platform/intel-mid/intel-mid.c | 4 +++- 3 files changed, 3 insertions(+), 14 deletions(-)
arch/x86/include/asm/setup.h +0 -6
@@ -45,12 +45,6 @@ extern void reserve_standard_io_resources(void);extern void i386_reserve_resources (void );extern void setup_default_timer_irq (void ); -#ifdef CONFIG_X86_INTEL_MID -extern void x86_intel_mid_early_setup (void ); -#else -static inline void x86_intel_mid_early_setup (void ) { } -#endif -#ifndef _SETUP#include <asm/espfix.h>
arch/x86/kernel/head32.c +0 -7
@@ -36,13 +36,6 @@ asmlinkage __visible void __init i386_start_kernel(void)cr4_init_shadow ();sanitize_boot_params (&boot_params); -/* Call the subarch specific early setup function */ -switch (boot_params.hdr.hardware_subarch) { -case X86_SUBARCH_INTEL_MID : -x86_intel_mid_early_setup (); -break ; - } -x86_init_fn_init_tables ();x86_init_fn_early_init ();
arch/x86/platform/intel-mid/intel-mid.c +3 -1
@@ -167,7 +167,7 @@ static unsigned char intel_mid_get_nmi_reason(void)* Moorestown specific x86_init function overrides and early setup * calls. */ -void __init x86_intel_mid_early_setup (void ) +static void __init x86_intel_mid_early_setup (void ) { x86_init.resources.probe_roms = x86_init_noop; x86_init.resources.reserve_resources = x86_init_noop;
@@ -199,6 +199,8 @@ void __init x86_intel_mid_early_setup(void) x86_init.mpparse.get_smp_config = x86_init_uint_noop;set_bit (MP_BUS_ISA , mp_bus_not_pci); } +x86_init_early (BIT (X86_SUBARCH_INTEL_MID ),NULL ,NULL , + x86_intel_mid_early_setup);/* * if user does not want to use per CPU apb timer, just give it a lower rating
Signature · 1 line
2.6.2
Re: [RFC v1 8/8] x86/init: use linker table for mid early setup
Quoted 4 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
Using the linker table removes the need for the #ifdef'ery and clutter on
head32.c.
Quoted 68 lines
Signed-off-by Luis R. Rodriguez <[email protected]>--- arch/x86/include/asm/setup.h | 6 ------ arch/x86/kernel/head32.c | 7 ------- arch/x86/platform/intel-mid/intel-mid.c | 4 +++- 3 files changed, 3 insertions(+), 14 deletions(-)arch/x86/include/asm/setup.h +0
-6@@ -45,12 +45,6 @@ extern void reserve_standard_io_resources(void);extern void i386_reserve_resources (void );extern void setup_default_timer_irq (void ); -#ifdef CONFIG_X86_INTEL_MID -extern void x86_intel_mid_early_setup (void ); -#else -static inline void x86_intel_mid_early_setup (void ) { } -#endif -#ifndef _SETUP#include <asm/espfix.h> arch/x86/kernel/head32.c +0
-7@@ -36,13 +36,6 @@ asmlinkage __visible void __init i386_start_kernel(void)cr4_init_shadow ();sanitize_boot_params (&boot_params); -/* Call the subarch specific early setup function */ -switch (boot_params.hdr.hardware_subarch) { -case X86_SUBARCH_INTEL_MID : -x86_intel_mid_early_setup (); -break ; - } -x86_init_fn_init_tables ();x86_init_fn_early_init ();arch/x86/platform/intel-mid/intel-mid.c +3
-1@@ -167,7 +167,7 @@ static unsigned char intel_mid_get_nmi_reason(void)* Moorestown specific x86_init function overrides and early setup * calls. */ -void __init x86_intel_mid_early_setup (void ) +static void __init x86_intel_mid_early_setup (void ) { x86_init.resources.probe_roms = x86_init_noop; x86_init.resources.reserve_resources = x86_init_noop;@@ -199,6 +199,8 @@ void __init x86_intel_mid_early_setup(void) x86_init.mpparse.get_smp_config = x86_init_uint_noop;set_bit (MP_BUS_ISA , mp_bus_not_pci); } +x86_init_early (BIT (X86_SUBARCH_INTEL_MID ),NULL ,NULL , + x86_intel_mid_early_setup);/* * if user does not want to use per CPU apb timer, just give it a lower rating Signature · 1 line
2.6.2
H. Peter Anvin
reviewerQuoted 339 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
A long time ago in a galaxy far,
far away...Konrad Rzeszutek Wilk posted patches which eventually got merged to help
with modularizing the IOMMUs we have on x86 [0]. This work was done due to
the complex relationship that exists on IOMMUs and the requirements on careful execution. The solution also provided a mechanism which jettisoned
unused IOMMUs during run-time.During review, even though the code was merged, hpa did note that we tend
to encounter this type of problem "often enough that we should implement a
generic facility for it" [1], hpa acknowledged that it obviously has to be
based on sections and even noted that perhaps we might be able in the future to
automate its creation. He noted that the gPXE folks had done just this with
linker tables and suggested that "presumably we'd need a few different flavors
for init tables and so on, but this would make it a generic mechanism."The IOMMU code got merged and this was left on someone's mental backburner.
I've had an itch to scratch recently to try to avoid issues which are possible
if one does not jettison other code carefully due to the large complexity of
implicit dependencies of certain code on x86 in particular with possible dead
code on x86 due to paravirtualization, and the IOMMU jettison strategy turned
out to be my favorite solution so far. I've taken on hpa's suggestions from
back in the day to review gPXE's solution to see if we could embrace it on
Linux for a generic section solution to help jettison code carefully.What this patch set does exactly:
This RFC patch set attempts to add support for such a generic solution. In the end, it turns out that the best solution possible was the best of
both worlds: a combination of what Konrad had implemented in addition to
what Michael Brown had implemented on the gXPE front. The IOMMU solution
enables simple semantic annotations for dependency relationships, this however requires a run time sort. The gPXE solution grants the option to
simply sort at build time. One of gPXE's solution primary goals however was
also to help avoid bit-rot on code that's possible from #ifdef'ery. The Linux
linker table solution enables developers to pick and choose what they need, with linker tables being the simplest solution. Contrary to gPXE which strives to force compilation of all linker table solutions we let developers pick *when* they want this as part of their solution. As can be seen from the suggested x86 init specific use of linker tables
proposed you can also take advantage of both, linker sorting, optional compilation when needed (at developer's discretion), and even careful semantics annotation for dependency / relationship annotations. Although
the x86 init solution here is heavily inspired by the IOMMU solution it diverges with strong semantics, and a new optional subarchitecture annotation. Sorting of init sequences is structure specific, as such each subsystem must defing their own solution unless semantics could be shared. I considered sharing semantics but in the end this proved pointless so this keeps things separate. A series of changes were made to the x86 init sequence in contrast to the IOMMU solution to be *extremely
pedantic* on semantics, review of this changes can be studied on the table-init tree [2].Quick review of gPXE's solution and prospects on further changes:
In my review from gPXE's solution it was not clear what hpa meant by gXPE folks having automated this process, they actually use linker tables
all around, forcing compilation of *everything* and just do linking of enabled features at link time. You still need to build linker tables on your own. What I do see more potential for in the future is enabling to evolve stronger semantics over time, and this would also be subsystem specific.
This will be evident in this patch set on the x86 init use of linker tables.
I also see potential in strenghtening semantics for linker sorting, any of
these types of features however would impose requiring newer binutils. For
instance, gPXE's linker solution currently relies onSORT(), that defaults toSORT_BY_NAME(). This sorts lexicographically, gPXE's solution uses two digits
to enable SORT_BY_NAME()'s lexicographical sort to sort orde by numeric priority. Since one is in control of order-level numbers one can provide
guarantee that this sort should work as intended, however binutils also now has
aSORT_BY_INIT_PRIORITY()which sorts specifically based on digits.SORT_BY_INIT_PRIORITY()was designed specifically forinit_arraysections
though. Refer to the userspace mockup solution table-init git tree [2] commit6deba47ee1ad461e90for more details on this. One thing I can envision to help
here further are prospects for future linker enhancements, these however must
be carefully considered in light of possible requirements of newer binutils
and also of compatibility with other toolchains. For now we resort to the good
'olSORT()which even Linux has made use of for ages already. In that regards
here, nothing new here. gPXE folks however did find some fun ICC compatiblity
issues, and have also developed fixes for them, these were obviously carried
over but should be reviewed carefully. Lastly, I should point out that essentially what we're developing are different forms loose and strong semantics -- in the most complex form what we really are after are feature
graphs. Mauro explained to me that for media driver they needed to build a
feature graphs, IMHO that could be a next level of strong semantics we might
want to consider. For now though the combination of gPXE's linker-table solution based on linker order-level sort, and our old IOMMU init solution
and its simple dependency map (and possible extensions, see subarchitecture)
should likely suffice as a light weight solution for where we need semantics
for at least on the x86 init path.Motivation and possible enhancements:
To understand what made me tick to work on this feel free to read the dead
code concerns with paravirtualization posts I've made, which also go into
and detail Xen's alternative entry point [4] [5]. That's not the end of possibilities to help address to possible "dead code" on Linux, I have other
ideas but I have a bit more work to do before publishing anything about it.
If anything -- later work should likely supplement this solution further.Currently the earliest I was able to make use of
boot_paramson x86-64 for
linker tables was afterload_idt(), so I've decided to stick with the earliest
init call for x86 init sequences starting onx86_64_start_reservation().
However, if we're able to make use ofboot_paramsearlier (help appreciated),
in particular just boot_params.hdr.hardware_subarch we should be able to make
clearer and careful annotations on early x86-64 init code. Using the boot_params.hdr.hardware_subarch and requiring clear subarchitecture support
annotations on early init code should provide a *proactive* means to avoid issues
such as the cr4 shadow oversight [6] which later caused crashes on Xen. The
latest similar type of issue is when KAsan was introduced, after it was integrated I suspected KAsan would probably break if enabled on Xen, I reported
this in March 2015 [7] and Andrey confirmed this might be the case but since he
wanted it enabled for allyesconfig and allmodconfig he could not think of a
clean way to disable it or address support for it then [8]. During the development of this patch set I confirmed KAsan crashes Xen dom0 and therefore
should obviously crash Xen PV guests as well. Using the same kernel KAsan
still worked on KVM and bare-metal. Provided we had early access to boot_params.hdr.hardware_subarch, inx86_64_start_kernel()we could annotatekasan_early_init()and friends not supported on Xen, however since we don't
have a way to disable KAsan at run time at this time we wouldn't have any other
option but to crash early. Because of this though we should perhaps just consider
disabling KAsan for Xen configs and KAsan folks might just have to bite the
bullet. If KAsan folks are gung-ho about not disabling KAsan when Xen is
enabled and if its easier to add support to disable KAsan at run time than
it is to add KAsan support for Xen one alternative might be to use linker
tables to annotate this and disable KAsan at run time for Xen.One of the points of the use of the x86 linker table solution and specific
semantics there in is to *force* developers to think about requirements on x86
carefully. So for instance by requiring itemizing supported subarchitectures
*early* on in development we should be able to avoid proactively issues such as
the crash due to the cr4 shadow changes not added on the Xen entry path, and
also missing the requirement to develop a solution for Kasan for Xen. The
proposed x86 linker table solution is not the first to use boot_params.hdr.hardware_subarch, its first use was actually on i386, and for
lguest. We take advantage of it to avoid further extendingpv_ops, and *actually* to see if we could simplifypv_opsover time further. This patch set
also includes a renaming ofparavirt_enabled()toparavirt_legacy()An unexpected long term side goal which *might* be possible due to linker
tables on x86 is to help unify the different C entry points for Linux x86-64.
I've taken a stab at it after these patches [9] but it fails on Xen, likely
because the stack is not set up right due to the different calls / argument
requirements. If this is desirable folks could take a look and perhaps help
on that front.Where to get code alternatively and testing:
All of these patches are RFCs, if anything the only patches worth consdering
merging sooner rather than later might be "paravirt: renameparavirt_enabled
toparavirt_legacy" and "x86/boot: addBIT()toboot/bitops.h". After review
I can send those separately in patch form if agreeable. Since these are all
just RFCs I've based these patches on top of Linus' v4.4-rc5. If you want all
patches in one file you can get them here [10], and a respective linux-next
version which applies on top of next-20151215 here [11]. I also have two
git trees set up with branches for this code, one based on Linus' tree
[12]
and another based on linux-next next-20151215 [13]. I'll see what issues
zero-day bot testing finds out. I've just run time tested this on x86-64
bare metal, KVM, and Xen dom0 so far.[0] https://marc.info/?l=linux-kernel&m=128284562303565&w=2
[1] https://marc.info/?l=linux-kernel&m=128285216913266&w=2
[2] https://github.com/mcgrof/table-init/
[3]
https://github.com/mcgrof/table-init/commit/6deba47ee1ad461e90e0fbba226a535cfc1c58f3
[4]
http://www.do-not-panic.com/2015/12/avoiding-dead-code-pvops-not-silver-bullet.html
[5]
http://www.do-not-panic.com/2015/12/xen-and-x86-linux-zero-page.html
[6]
http://lists.xenproject.org/archives/html/xen-devel/2015-02/msg02742.html
[7]
http://lkml.kernel.org/r/CAB=NE6Xs5fepzNtymzT4CueeJZ0KMPETpda114DpL4eMtDswtw@mail.gmail.com
[8] http://lkml.kernel.org/r/[email protected]
[9]
http://drvbp1.linux-foundation.org/~mcgrof/patches/2015/12/15/x86-merge-x86-init-v1.patch
[10]
http://drvbp1.linux-foundation.org/~mcgrof/patches/2015/12/15/pend-20151215-rfc-v1-linker-tables.patch
[11]
http://drvbp1.linux-foundation.org/~mcgrof/patches/2015/12/15/pend-next-20151215-rfc-v1-linker-tables.patch
[12]
https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux.git/log/?h=20151215-rfc-v1-linker-tables
[13]
https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20151215-rfc-v1-linker-tablesLuis R. Rodriguez (8):
paravirt: renameparavirt_enabledtoparavirt_legacytables.h: add linker table support
x86/boot: addBIT()toboot/bitops.hx86/init: add linker table support
x86/init: move ebda reservations into linker table x86/init: use linker table for i386 early setup x86/init: user linker table for ce4100 early setup x86/init: use linker table for mid early setupDocumentation/kbuild/makefiles.txt | 19 ++ arch/x86/Kconfig.debug | 47 ++++ arch/x86/boot/bitops.h | 2 + arch/x86/boot/boot.h | 2 +- arch/x86/include/asm/bios_ebda.h | 2 - arch/x86/include/asm/paravirt.h | 4 +- arch/x86/include/asm/paravirt_types.h | 11 +- arch/x86/include/asm/processor.h | 2 +- arch/x86/include/asm/setup.h | 12 - arch/x86/include/asm/x86_init.h | 1 + arch/x86/include/asm/x86_init_fn.h | 267 ++++++++++++++++++++ arch/x86/kernel/Makefile | 4 +- arch/x86/kernel/apm_32.c | 2 +- arch/x86/kernel/asm-offsets.c | 2 +- arch/x86/kernel/cpu/intel.c | 2 +- arch/x86/kernel/cpu/microcode/core.c | 2 +- arch/x86/kernel/dbg-tables/Makefile | 18 ++ arch/x86/kernel/dbg-tables/alpha.c | 10 + arch/x86/kernel/dbg-tables/beta.c | 18 ++ arch/x86/kernel/dbg-tables/delta.c | 10 + arch/x86/kernel/dbg-tables/gamma.c | 18 ++ arch/x86/kernel/dbg-tables/gamma.h | 3 + arch/x86/kernel/{head.c => ebda.c} | 6 +- arch/x86/kernel/head32.c | 22 +- arch/x86/kernel/head64.c | 4 +- arch/x86/kernel/init.c | 55 +++++ arch/x86/kernel/kvm.c | 9 +- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/sort-init.c | 114 +++++++++ arch/x86/kernel/tboot.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 25 ++ arch/x86/lguest/boot.c | 2 +- arch/x86/mm/dump_pagetables.c | 2 +- arch/x86/platform/ce4100/ce4100.c | 4 +- arch/x86/platform/intel-mid/intel-mid.c | 4 +- arch/x86/tools/relocs.c | 1 + arch/x86/xen/enlighten.c | 2 +- drivers/pnp/pnpbios/core.c | 2 +- include/linux/tables.h | 421 ++++++++++++++++++++++++++++++++ scripts/Makefile.build | 4 +- scripts/Makefile.clean | 1 + scripts/Makefile.lib | 12 + 42 files changed, 1091 insertions(+), 61 deletions(-)
Quoted 11 lines
create mode 100644
arch/x86/include/asm/x86_init_fn.hcreate mode 100644arch/x86/kernel/dbg-tables/Makefilecreate mode 100644arch/x86/kernel/dbg-tables/alpha.ccreate mode 100644arch/x86/kernel/dbg-tables/beta.ccreate mode 100644arch/x86/kernel/dbg-tables/delta.ccreate mode 100644arch/x86/kernel/dbg-tables/gamma.ccreate mode 100644arch/x86/kernel/dbg-tables/gamma.hrenamearch/x86/kernel/{head.c=>ebda.c} (94%) create mode 100644arch/x86/kernel/init.ccreate mode 100644arch/x86/kernel/sort-init.ccreate mode 100644include/linux/tables.h
By the way, please refer to iPXE as gPXE is a dead project.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.
H. Peter Anvin
reviewerI think we can make this even more generic. In particular, I would love to see a solution for link tables that:
a) can be used for any kind of data structures, not just function pointers (the latter is a specialization of the former);
b) doesn't need any changes to the linker scripts once the initial enabling is done for any one architecture.
Key to this is to be able to define tables by name only, which is really why SORT_BY_NAME() is used: the name sorts before the priority simply by putting the name before the class.
Instead of .tbl.* naming of sections I think we should have the first component be the type of section (.rodata, .data, .init_rodata, .read_mostly etc.) which makes it easier to write a linker script that properly sorts it into the right section. The other thing is to take a clue from the implementation in iPXE, which uses priority levels 00 and 99 (or we could use non-integers which sort appropriately instead of using "real" levels) to contain the start and end symbols, which eliminates any need for linker script modifications to add new tables.
Making this a generic facility we could eventually eliminate a bunch of ad hoc hacks we currently have.
Oh, and the link table feature should NOT be x86-specific.
-hpa
Quoted 5 lines · H. Peter Anvin
I think we can make this even more generic. In particular, I would love to see a solution for link tables that:
a) can be used for any kind of data structures, not just function pointers (the latter is a specialization of the former);
Sure, I can give this a shot. I'll sync back to the userspace table-init.git tree what I had in this series and then build first there a solution to make it agnostic to the use case.
Quoted 2 lines
b) doesn't need any changes to the linker scripts once the initial enabling is done for any one architecture.
I'm glad you brought this up. I considered this as well, and sadly I forgot to document the resolution. So here it is:
iPXE uses *one* entry on their linker script, that's it. All else is just C code. That works well as all code in iPXE is happy to work with one linker script, and the goal and use case for the linker tables are *all* for initialization. iPXE also does *not* make use of any further semantics for dependency annotations.
For Linux we have different requirements and goals, and we also wanted to take advantage of other possible semantics when desirable. Consider the proposed use case of a linker table with struct x86_init_fn, for its init sequences. We need at the very least ***one linker table entry* to catch and peg these sections, just as iPXE has. If we wanted to we could live with that but for Linux on x86 that means vmlinux.lds.S, that's already architecture specific. Do we want a more generic place ? If so can anyone suggest one ?
So to be clear, the solution in place suggested here already enables this, but its current form does use vmlinux.lds.S. Although the current patch made this change:
.tbl : {
__tbl_x86_start_init_fns = .;
*(SORT(.tbl.x86_init_fns.*))
__tbl_x86_end_init_fns = .;
/* ... etc ... */
/* You would add other tables here as needed */
} This was on purpose, a generic form could have simply been used as well:
__tbl = .;
*(SORT(.tbl.*))
__tbl_end = .;I made this change on the mockup table-init through two commits here:
https://github.com/mcgrof/table-init/commit/1a330c1dadbf7fa255eadcbaf214452f7e9ea29c
https://github.com/mcgrof/table-init/commit/ad11b4010ab2a5ffae7be39cce0ed4d052fe18fd
I explain why I do that there but the gist of it is that on Linux we may also want stronger semantics for specific linker table solutions, and solutions such as those devised on the IOMMU init stuff do memmove() for sorting depending on semantics defined (in the simplest case here so far dependency between init sequences), this makes each set of sequences very subsystem specific. An issue with *one* subsystem could make things really bad for others. I thought about this quite a bit and figured its best left to the subsystem maintainers to decide.
So we have the freedom to do we wish still -- its just we need to understand the implications of using something like a run time sort, and also when we want to have the option to use different sized set of routines for different sections. Linker sorting should fortunately give us the freedom to sort without concern or regard to size of the structs / data used, but *iff* different sized data is used in that section then one has to be careful that the sort routine be very subsystem specific. So as I see it we'd need linker script annotations for start and end on subsystems that want to use run time sort. This would compartmentalize their sorting and bugs. If run time sorting is not needed a catch-it-all script section as the above simple one should catch it.
To try to avoid issues between these two types of cases we could just document the requirement. Now, it'd nice if we had a way to provide the section start/end through C code, and not modify anything on linker scripts even if you had run time sort requirements, I just can't think of a way to do that right now. Ideas welcomed. Maybe some Kconfig magic to custom linker script section ? Then the main linker script can just include this generated linker script section ?
Quoted 3 lines
Key to this is to be able to define tables by name only, which is really why
SORT_BY_NAME()is used: the name sorts before the priority simply by putting the name before the class.
Right, tbl.init_fns.01 for example would be one section if your scruct was .init_fns and 01 for the first "priority". That still puts a requirement if you want to use a run time sort so you can identify the start/end of the range.
Quoted 4 lines
Instead of .tbl.* naming of sections I think we should have the first component be the type of section (.rodata, .data, .init_rodata, .read_mostly etc.) which makes it easier to write a linker script that properly sorts it into the right section.
Sure, we can peg that as an argument to the __table(), so perhaps:
Perhaps a new sections.h file (you tell me) which documents the different section components:
/* document this *really* well */
#define SECTION_RODATA ".rodata"
#define SECTION_INIT ".init"
#define SECTION_INIT_RODATA ".init_rodata"
#define SECTION_READ_MOSTLY ".read_mostly"
Then on tables.h we add the section components support:
#define __table(component, type, name) (component, type, name)
#define __table_component(table) __table_extract_component table #define __table_extract_component(component, type, name) component
#define __table_type(table) __table_extract_type table #define __table_extract_type(component, type, name) type
#define __table_name(table) __table_extract_name table #define __table_extract_name(component, type, name) name
#define __table_str(x) #x
#define __table_section(table, idx) \
"." __table_component (table) ".tbl." __table_name (table) "." __table_str (idx) #define __table_entry(table, idx) \
__attribute__ ((__section__(__table_section(table, idx)), \
__aligned__(__table_alignment(table))))A user could then be something as follows:
#define X86_INIT_FNS __table(SECTION_INIT, struct x86_init_fn, "x86_init_fns") #define __x86_init_fn(order_level) __table_entry(X86_INIT_FNS, order_level)
If that's what you mean?
I'm a bit wary about having the linker sort any of the above SECTION_*'s, but if we're happy to do that perhaps a simple first step might be to see if 0-day but would be happy with just the sort without any consequences to any architecture. Thoughts?
Quoted 5 lines
The other thing is to take a
clue from the implementation in iPXE, which uses priority levels 00 and 99 (or we could use non-integers which sort appropriately instead of using "real" levels) to contain the start and end symbols, which eliminates any need for linker script modifications to add new tables.
This solution uses that as well. The only need for adding custom sections is when they have a requirement for a custom run time sort, and also to ensure they don't cause regressions on other subsystems if they have a buggy sort. The run time sorting is all subsystem specific and up to their own semantics.
Quoted 2 lines
Making this a generic facility we could eventually eliminate a bunch of ad hoc hacks we currently have.
Towards the end of my development I did look into seeing if I could use this sort of stuff also for the other section sorting hacks we have in place, I ran into an issue with the type stuff, if we make it agnostic to data structure, then perhaps indeed it might be possible to generalize this even to those scary cob-web corner infested places in the kernel.
Quoted 1 line
Oh, and the link table feature should NOT be x86-specific.
As far as I can tell tables.h is not architecture specific at this point, the way arch/x86/include/asm/x86_init_fn.h uses it is, but that's fine. Even with the above changes it would still be architecture agnostic. If there's some discrepancies on how some architecture linkers work -- now that's something to consider now. Its also why I've been mocking this up in userspace, we should be able to test that much easier than a full blown kernel.
Now would be a good time to get linker folks involved and eyeballing all this.
Luis
Quoted 4 lines · Luis R. Rodriguez
solutions such
as those devised on the IOMMU init stuff domemmove()for sorting depending on semantics defined (in the simplest case here so far dependency between init sequences), this makes each set of sequences very subsystem specific
I should also note -- since the sort uses memmove() we're constrained to using run time sort only once we can ensure the kernel can do that without issues. I'd hope x86_64_start_reservations() is a safe place, if not please let me know. Likewise, since one prospect here is to bring further strong subarchitecture semantics even earlier (help appreciated), say as early as possible in x86_64_start_kernel(), it'd be nice to know the earliest memmove() is safe there.
In this case the solution doesn't yet make use of calls in between x86_64_start_kernel() and x86_64_start_reservations() but if it could (we'd need a way for x86_init_fn_early_init() to access the subarchitecture, help appreciated) we *might* in the future want run time sort this early. It might still be possible to *not* do a run time sort until later so long as the priority level linker sort mechanism suffices for all early init routines. That's another possibility, should memmove() be a problem that early.
Luis
H. Peter Anvin
reviewerQuoted 10 lines · Luis R. Rodriguez
I explain why I do that there but the gist of it is that on Linux we may also want stronger semantics for specific linker table solutions, and solutions such as those devised on the IOMMU init stuff do
memmove()for sorting depending on semantics defined (in the simplest case here so far dependency between init sequences), this makes each set of sequences very subsystem specific. An issue with *one* subsystem could make things really bad for others. I thought about this quite a bit and figured its best left to the subsystem maintainers to decide.
A table that needs sorting or other runtime handling is just a read-write table for the purpose of the linker table construct. It presents to C as an array of initialized data.
Quoted 10 lines
Perhaps a new
sections.hfile (you tell me) which documents the different section components:/* document this *really* well */
#defineSECTION_RODATA".rodata"
#defineSECTION_INIT".init"
#defineSECTION_INIT_RODATA".init_rodata"
#defineSECTION_READ_MOSTLY".read_mostly"Then on
tables.hwe add the section components support:
Yes, something like that. How to macroize it cleanly is another matter; we may want to use slightly different conventions that iPXE to match our own codebase.
Quoted 16 lines
#define
__table(component, type, name)(component, type, name)#define
__table_component(table)__table_extract_componenttable #define__table_extract_component(component, type, name)component#define
__table_type(table)__table_extract_typetable #define__table_extract_type(component, type, name)type#define
__table_name(table)__table_extract_nametable #define__table_extract_name(component, type, name)name#define
__table_str(x)#x#define __table_section(table, idx) \ "." __table_component (table) ".tbl." __table_name (table) "." __table_str (idx)
Quoted 8 lines
#define __table_entry(table, idx) \ __attribute__ ((__section__(__table_section(table, idx)), \ __aligned__(__table_alignment(table))))A user could then be something as follows:
#define
X86_INIT_FNS__table(SECTION_INIT, struct x86_init_fn, "x86_init_fns")#define__x86_init_fn(order_level)__table_entry(X86_INIT_FNS, order_level)
Yes, but in particular the common case of function initialization tables should be generic.
I'm kind of thinking a syntax like this:
DECLARE_LINKTABLE_RO(struct foo, tablename);DEFINE_LINKTABLE_RO(struct foo, tablename);LINKTABLE_RO(tablename,level) = /* contents */;LINKTABLE_SIZE(tablename)
... which would turn into something like this once it goes through all the preprocessing phases
/* DECLARE_LINKTABLE_RO */
extern const struct foo tablename[], tablename__end[];
/* DEFINE_LINKTABLE_RO */DECLARE_LINKTABLE_RO(struct foo, tablename);
const struct foo__attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0];
const struct foo__attribute__((used,section(".rodata.tbl.tablename.999"))) tablename__end[0];
/* LINKTABLE_RO */
static const __typeof__(tablename) __attribute__((used,section(".rodata.tbl.tablename.50"))) __tbl_tablename_12345
/* LINKTABLE_SIZE */
((tablename__end) - (tablename))
... and so on for all the possible sections where we may want tables.
Note: I used 0 and 999 above since they sort before and after all possible 2-digit decimal numbers, but that's just cosmetic.
Quoted 6 lines
If that's what you mean?
I'm a bit wary about having the linker sort any of the above SECTION_*'s, but if we're happy to do that perhaps a simple first step might be to see if 0-day but would be happy with just the sort without any consequences to any architecture. Thoughts?
I don't see what is dangerous about it. The section names are such that a lexographical sort will do the right thing, and we can simply use SORT(.rodata.tbl.*) in the linker script, for example.
Quoted 11 lines
Quoted 5 lines
The other thing is to take a
clue from the implementation in iPXE, which uses priority levels 00 and 99 (or we could use non-integers which sort appropriately instead of using "real" levels) to contain the start and end symbols, which eliminates any need for linker script modifications to add new tables.This solution uses that as well. The only need for adding custom sections is when they have a requirement for a custom run time sort, and also to ensure they don't cause regressions on other subsystems if they have a buggy sort. The run time sorting is all subsystem specific and up to their own semantics.
Again, from a linker table POV this is nothing other than a read-write table; there is a runtime function that then operates on that read-write table.
-hpa
H. Peter Anvin
reviewerQuoted 19 lines · H. Peter Anvin
/*
DECLARE_LINKTABLE_RO*/
extern const struct foo tablename[], tablename__end[];/*
DEFINE_LINKTABLE_RO*/DECLARE_LINKTABLE_RO(struct foo, tablename);const struct foo__attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0];
const struct foo__attribute__((used,section(".rodata.tbl.tablename.999"))) tablename__end[0];
/*
LINKTABLE_RO*/
static const__typeof__(tablename)__attribute__((used,section(".rodata.tbl.tablename.50")))__tbl_tablename_12345
Quoted 5 lines
/*
LINKTABLE_SIZE*/
((tablename__end) - (tablename))... and so on for all the possible sections where we may want tables.
Come to think of it, we could even eliminate the need for a DEFINE entirely if we made the start and end symbols static. However, this would generate an awful lot of identical-but-local symbols which probably would make the linker slower and definitely would bloat the debug data.
-hpa
H. Peter Anvin
reviewerQuoted 8 lines · H. Peter Anvin
Quoted 8 lines
const struct foo__attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0];
const struct foo__attribute__((used,section(".rodata.tbl.tablename.999"))) tablename__end[0];
(Over)thinking about this some more, I suggest using the empty string for the start and "~" for the end. And, yes, I did check that ~ works as part of a section name.
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug. Worst case we have to use an assembly statement to create these sections; it isn't a big deal and shouldn't make it any more architecture-specific.
-hpa
Quoted 5 lines · H. Peter Anvin
Quoted 4 lines · H. Peter Anvin
Quoted 4 lines
const struct foo__attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0];
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
const struct foo__attribute__((used,section(".rodata.tbl.tablename.999"))) tablename__end[0];
(Over)thinking about this some more, I suggest using the empty string for the start and "~" for the end. And, yes, I did check that ~ works as part of a section name.
Sure, do we know if that ICC compatible? Do we care? There are a series of ICC hacks put in place on ipxe's original solution which I've folded in, it seems that works but if we care about ICC those folks should perhaps help review as well.
Quoted 5 lines
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug. Worst case we have to use an assembly statement to create these sections; it isn't a big deal and shouldn't make it any more architecture-specific.
OK!
Luis
Quoted 11 lines · Luis R. Rodriguez
Quoted 10 lines · H. Peter Anvin
Quoted 8 lines · H. Peter Anvin
Quoted 8 lines
const struct foo__attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0];
const struct foo__attribute__((used,section(".rodata.tbl.tablename.999"))) tablename__end[0];
Quoted 8 lines
Quoted 3 lines
(Over)thinking about this some more, I suggest using the empty string for the start and "~" for the end. And, yes, I did check that ~ works as part of a section name.
Sure, do we know if that ICC compatible? Do we care? There are a series of ICC hacks put in place on ipxe's original solution which I've folded in, it seems that works but if we care about ICC those folks should perhaps help review as well.
I didn't know the kernel could even be compiled with ICC? Thought only GCC worked?
Anyhow - it may be that those fixes were for quite old ICC versions. Does the latest one manifest these oddities?
Quoted 10 lines
Quoted 6 lines
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug. Worst case we have to use an assembly statement to create these sections; it isn't a big deal and shouldn't make it any more architecture-specific.
OK!
Luis
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 4 lines
Sure, do we know if that ICC compatible? Do we care? There are a series of ICC hacks put in place on ipxe's original solution which I've folded in, it seems that works but if we care about ICC those folks should perhaps help review as well.
I didn't know the kernel could even be compiled with ICC? Thought only GCC worked?
I'm happy with that, just wanted to make sure I raise the flag concern given the icc hacks on the linker tables.
Quoted 2 lines
Anyhow - it may be that those fixes were for quite old ICC versions. Does the latest one manifest these oddities?
I am not sure, I yield to Michael as the author of the original ICC compatibility pieces. If we don't care about ICC let me know and I'll just drop the stuff. In lack of such statements I'll just keep the work arounds in place, but I'm more than trilled to drop it.
Luis
H. Peter Anvin
reviewerQuoted 13 lines · Luis R. Rodriguez
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 4 lines
Sure, do we know if that ICC compatible? Do we care? There are a series of ICC hacks put in place on ipxe's original solution which I've folded in, it seems that works but if we care about ICC those folks should perhaps help review as well.
I didn't know the kernel could even be compiled with ICC? Thought only GCC worked?
I'm happy with that, just wanted to make sure I raise the flag concern given the icc hacks on the linker tables.
Quoted 8 lines
Quoted 2 lines
Anyhow - it may be that those fixes were for quite old ICC versions. Does the latest one manifest these oddities?
I am not sure, I yield to Michael as the author of the original ICC compatibility pieces. If we don't care about ICC let me know and I'll just drop the stuff. In lack of such statements I'll just keep the work arounds in place, but I'm more than trilled to drop it.
In general we let the ICC and Clang/LLVM teams communicate with out a post facto. We can't just guess what their requirements are, especially since they are likely to change between revisions.
-hpa
Quoted 23 lines · H. Peter Anvin
Quoted 13 lines · Luis R. Rodriguez
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 4 lines
Sure, do we know if that ICC compatible? Do we care? There are a series of ICC hacks put in place on ipxe's original solution which I've folded in, it seems that works but if we care about ICC those folks should perhaps help review as well.
I didn't know the kernel could even be compiled with ICC? Thought only GCC worked?
I'm happy with that, just wanted to make sure I raise the flag concern given the icc hacks on the linker tables.
Quoted 8 lines
Quoted 2 lines
Anyhow - it may be that those fixes were for quite old ICC versions. Does the latest one manifest these oddities?
I am not sure, I yield to Michael as the author of the original ICC compatibility pieces. If we don't care about ICC let me know and I'll just drop the stuff. In lack of such statements I'll just keep the work arounds in place, but I'm more than trilled to drop it.
Quoted 3 lines
In general we let the ICC and Clang/LLVM teams communicate with out a post facto. We can't just guess what their requirements are, especially since they are likely to change between revisions.
Great. I'm going to drop ICC hacks.
Luis
Quoted 11 lines · Konrad Rzeszutek Wilk
Quoted 4 lines · Luis R. Rodriguez
Sure, do we know if that ICC compatible? Do we care? There are a series of ICC hacks put in place on ipxe's original solution which I've folded in, it seems that works but if we care about ICC those folks should perhaps help review as well.
I didn't know the kernel could even be compiled with ICC? Thought only GCC worked?
Anyhow - it may be that those fixes were for quite old ICC versions. Does the latest one manifest these oddities?
I haven't tested building iPXE with icc for some time. (The support for icc was originally added with the plan to be able to compile to EFI Byte Code; this plan was swiftly abandoned.)
The most recent version of icc that I have personally used with that code was 10.1.018.
Michael
Hi,
Quoted 2 lines · H. Peter Anvin
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug.
Workaround: use an (possibly empty) intializer:
struct foo {int i;};
const struct foo __attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0] = {};
Ciao,
Michael.
H. Peter Anvin
reviewerQuoted 4 lines · Michael Matz
Hi,
On Thu, 21 Jan 2016, H. Peter Anvin wrote:
Quoted 9 lines
Quoted 2 lines
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug.
Workaround: use an (possibly empty) intializer:
struct foo {int i;};
const struct foo __attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0] = {};
And indeed that works. Awesome! Much better than having to do an assembly hack.
-hpa
Quoted 10 lines · H. Peter Anvin
Quoted 9 lines · Michael Matz
Hi,
Quoted 2 lines · H. Peter Anvin
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug.
Workaround: use an (possibly empty) intializer:
Quoted 7 lines
Quoted 4 lines
struct foo {int i;};
const struct foo __attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0] = {};And indeed that works. Awesome! Much better than having to do an assembly hack.
BTW before we set these in stone given that subarch (lguest, Xen, PC) really does provide a split in run time code we *know* we could technically free code for subarchs not needed. I don't expect this to happen now, but its possibility to do later seems worthy for us to consider on architecture here on the way we define the linker tables. As I have it linker tables are associated associated with a struct (which hpa asked to enable anyone to peg *anything* not just structs), these structs then have a priority which uses the linker later to sort things for us, and it also has a subarch bitmask which tells us the supported subarchs.
Should it be possible to resuse free_init_pages() and/or free_reserved_area() only for routines (members in the array in this case of a struct of fns) that don't meet our subarch once we're done iterating over the routies and know we can discard things we know we can drop? Through a cursory glance, *I think* its possible as-is, we would just need easy access to the respective start and end addresses and I guess there lies the challenge. Question is, is would that be clean enough for us? Or are there other things you can think of that perhaps might make this prospect cleaner later to add?
I figure better ask now for architectural purposes than later after merged.
Luis
Quoted 40 lines · Luis R. Rodriguez
Quoted 10 lines · H. Peter Anvin
Quoted 9 lines · Michael Matz
Hi,
Quoted 2 lines · H. Peter Anvin
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug.
Workaround: use an (possibly empty) intializer:
Quoted 7 lines
Quoted 4 lines
struct foo {int i;};
const struct foo __attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0] = {};And indeed that works. Awesome! Much better than having to do an assembly hack.
BTW before we set these in stone given that subarch (lguest, Xen, PC) really does provide a split in run time code we *know* we could technically free code for subarchs not needed. I don't expect this to happen now, but its possibility to do later seems worthy for us to consider on architecture here on the way we define the linker tables. As I have it linker tables are associated associated with a struct (which hpa asked to enable anyone to peg *anything* not just structs), these structs then have a priority which uses the linker later to sort things for us, and it also has a subarch bitmask which tells us the supported subarchs.
Should it be possible to resuse
free_init_pages()and/orfree_reserved_area()only for routines (members in the array in this case of a struct of fns) that don't meet our subarch once we're done iterating over the routies and know we can discard things we know we can drop? Through a cursory glance, *I think* its possible as-is, we would just need easy access to the respective start and end addresses and I guess there lies the challenge. Question is, is would that be clean enough for us? Or are there other things you can think of that perhaps might make this prospect cleaner later to add?
Quoted 1 line
I figure better ask now for architectural purposes than later after merged.
I don't think its needed we iron out in a solution *now* to be able to free code we know we won't need at run time but having a solid understanding adding this feature later without much impact to users might be worthy. As such I was pursuing a very basic proof of concept to ensure this is possible first given I didn't hear back if folks were sure this might be possible. I don't think a proof of concept should take long so just want to get fleshed out.
Current dev tree for making this proof of concept:
https://github.com/mcgrof/kbox
If you don't think being prudent about this in terms of architecture is necessary let me know and I'll just respin based on existing discussion and we can leave this for later and deal with any required changes later.
Luis
H. Peter Anvin
reviewerQuoted 13 lines · Luis R. Rodriguez
Quoted 11 lines
Should it be possible to resuse
free_init_pages()and/orfree_reserved_area()only for routines (members in the array in this case of a struct of fns) that don't meet our subarch once we're done iterating over the routies and know we can discard things we know we can drop? Through a cursory glance, *I think* its possible as-is, we would just need easy access to the respective start and end addresses and I guess there lies the challenge. Question is, is would that be clean enough for us? Or are there other things you can think of that perhaps might make this prospect cleaner later to add?Quoted 1 line
I figure better ask now for architectural purposes than later after merged.
Quoted 8 lines
I don't think its needed we iron out in a solution *now* to be able to free code we know we won't need at run time but having a solid understanding adding this feature later without much impact to users might be worthy. As such I was pursuing a very basic proof of concept to ensure this is possible first given I didn't hear back if folks were sure this might be possible. I don't think a proof of concept should take long so just want to get fleshed out.
This applies to the specific subarch use rather than generic linker tables, right?
-hpa
Quoted 23 lines · H. Peter Anvin
Quoted 13 lines · Luis R. Rodriguez
Quoted 11 lines
Should it be possible to resuse
free_init_pages()and/orfree_reserved_area()only for routines (members in the array in this case of a struct of fns) that don't meet our subarch once we're done iterating over the routies and know we can discard things we know we can drop? Through a cursory glance, *I think* its possible as-is, we would just need easy access to the respective start and end addresses and I guess there lies the challenge. Question is, is would that be clean enough for us? Or are there other things you can think of that perhaps might make this prospect cleaner later to add?Quoted 1 line
I figure better ask now for architectural purposes than later after merged.
Quoted 8 lines
I don't think its needed we iron out in a solution *now* to be able to free code we know we won't need at run time but having a solid understanding adding this feature later without much impact to users might be worthy. As such I was pursuing a very basic proof of concept to ensure this is possible first given I didn't hear back if folks were sure this might be possible. I don't think a proof of concept should take long so just want to get fleshed out.
Quoted 2 lines
This applies to the specific subarch use rather than generic linker tables, right?
Well both, given that for instance the kernel frees unused kernel code using the __init section, so a neat generic section solution for this perhaps in consideration for the subarch might be worthy to consider. You had also suggested perhaps the linker table could be pivoted based on the subarch at one point too, so I gave that a though. It would make the freeing much easier but for run time it wouldn't fit well into the current design yet.
Luis
Quoted 27 lines · Luis R. Rodriguez
Quoted 23 lines · H. Peter Anvin
On 02/02/2016 04:22 PM, Luis R. Rodriguez wrote:
s>>Quoted 12 lines
Quoted 11 lines
Should it be possible to resuse
free_init_pages()and/orfree_reserved_area()only for routines (members in the array in this case of a struct of fns) that don't meet our subarch once we're done iterating over the routies and know we can discard things we know we can drop? Through a cursory glance, *I think* its possible as-is, we would just need easy access to the respective start and end addresses and I guess there lies the challenge. Question is, is would that be clean enough for us? Or are there other things you can think of that perhaps might make this prospect cleaner later to add?I figure better ask now for architectural purposes than later after merged.
Quoted 8 lines
I don't think its needed we iron out in a solution *now* to be able to free code we know we won't need at run time but having a solid understanding adding this feature later without much impact to users might be worthy. As such I was pursuing a very basic proof of concept to ensure this is possible first given I didn't hear back if folks were sure this might be possible. I don't think a proof of concept should take long so just want to get fleshed out.
Quoted 2 lines
This applies to the specific subarch use rather than generic linker tables, right?
Quoted 7 lines
Well both, given that for instance the kernel frees unused kernel code using the
__initsection, so a neat generic section solution for this perhaps in consideration for the subarch might be worthy to consider. You had also suggested perhaps the linker table could be pivoted based on the subarch at one point too, so I gave that a though. It would make the freeing much easier but for run time it wouldn't fit well into the current design yet.
OK so I've decided that to make the Proof of Concept also immediately useful I could try to convert one of the *existing* ad hoc link tables to generic using this new solution. The POC will not be required as part of the patch series, just knowing it works is all that would be needed.
To be clear then what I'll do:
* I'll be respinning this series *now* to incorporate all feedback
given but *will not post* until I also have a proof of concept of a conversion done and tested that includes free unused code
* Since the renaming paravirt_enabled() patches are separate but
folks agree with them I'll submit that as an independent series, shortly
* Since the patch x86/boot: add BIT() to boot/bitops." is
independent I'll submit as independent single patch, shortly
Luis
H. Peter Anvin
reviewerQuoted 4 lines · Michael Matz
Hi,
On Thu, 21 Jan 2016, H. Peter Anvin wrote:
Quoted 9 lines
Quoted 2 lines
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug.
Workaround: use an (possibly empty) intializer:
struct foo {int i;};
const struct foo __attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0] = {};
Any forward progress on this?
-hpa
Quoted 8 lines · H. Peter Anvin
Quoted 7 lines · Michael Matz
Hi,
Quoted 2 lines · H. Peter Anvin
Something that confuses me is that gcc seems to give these sections the "aw" attributes which makes as complain. This might be a gcc bug.
Quoted 8 lines
Quoted 6 lines
Workaround: use an (possibly empty) intializer:
struct foo {int i;};
const struct foo __attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0] = {};Any forward progress on this?
Yes, I'll reply inline to my only concern.
Luis
Quoted 12 lines · H. Peter Anvin
Quoted 10 lines · Luis R. Rodriguez
I explain why I do that there but the gist of it is that on Linux we may also want stronger semantics for specific linker table solutions, and solutions such as those devised on the IOMMU init stuff do
memmove()for sorting depending on semantics defined (in the simplest case here so far dependency between init sequences), this makes each set of sequences very subsystem specific. An issue with *one* subsystem could make things really bad for others. I thought about this quite a bit and figured its best left to the subsystem maintainers to decide.
Quoted 3 lines
A table that needs sorting or other runtime handling is just a read-write table for the purpose of the linker table construct. It presents to C as an array of initialized data.
Sure but what I was getting at was that since some run time changes to the table *might* be desirable, depending on the subsystem, the subsystem table needs to be able to know the start and end address of its table, and that's a linker script change. But come to think of it, that was me just being pedantic and careful, I'll try even a run time sort with a few tables of different structure size to ensure its both possible to just declare them in C and also sort them without a linker script change.
Quoted 9 lines
Quoted 9 lines
Perhaps a new
sections.hfile (you tell me) which documents the different section components:/* document this *really* well */
#defineSECTION_RODATA".rodata"
#defineSECTION_INIT".init"
#defineSECTION_INIT_RODATA".init_rodata"
#defineSECTION_READ_MOSTLY".read_mostly"
Quoted 5 lines
Quoted 1 line
Then on
tables.hwe add the section components support:Yes, something like that. How to macroize it cleanly is another matter; we may want to use slightly different conventions that iPXE to match our own codebase.
Sure, the style below is from iPXE, the one in the patch set matches the Linux coding style. Other than that I'd welcome feedback on any issues or recommendations with style on our proposed version of tables.h Seems you provided some pointers below, so I'll go try to incorporate those suggestions.
Quoted 56 lines
Quoted 16 lines
#define
__table(component, type, name)(component, type, name)#define
__table_component(table)__table_extract_componenttable #define__table_extract_component(component, type, name)component#define
__table_type(table)__table_extract_typetable #define__table_extract_type(component, type, name)type#define
__table_name(table)__table_extract_nametable #define__table_extract_name(component, type, name)name#define
__table_str(x)#x#define __table_section(table, idx) \ "." __table_component (table) ".tbl." __table_name (table) "." __table_str (idx)Quoted 8 lines
#define __table_entry(table, idx) \ __attribute__ ((__section__(__table_section(table, idx)), \ __aligned__(__table_alignment(table))))A user could then be something as follows:
#define
X86_INIT_FNS__table(SECTION_INIT, struct x86_init_fn, "x86_init_fns")#define__x86_init_fn(order_level)__table_entry(X86_INIT_FNS, order_level)Yes, but in particular the common case of function initialization tables should be generic.
I'm kind of thinking a syntax like this:
DECLARE_LINKTABLE_RO(struct foo, tablename);DEFINE_LINKTABLE_RO(struct foo, tablename);LINKTABLE_RO(tablename,level)= /* contents */;LINKTABLE_SIZE(tablename)... which would turn into something like this once it goes through all the preprocessing phases
/*
DECLARE_LINKTABLE_RO*/
extern const struct foo tablename[], tablename__end[];/*
DEFINE_LINKTABLE_RO*/DECLARE_LINKTABLE_RO(struct foo, tablename);const struct foo__attribute__((used,section(".rodata.tbl.tablename.0"))) tablename[0];
const struct foo__attribute__((used,section(".rodata.tbl.tablename.999"))) tablename__end[0];
/*
LINKTABLE_RO*/
static const__typeof__(tablename)__attribute__((used,section(".rodata.tbl.tablename.50")))__tbl_tablename_12345
Quoted 4 lines
/*
LINKTABLE_SIZE*/
((tablename__end) - (tablename))... and so on for all the possible sections where we may want tables.
OK thanks so much! Will try working with that.
Quoted 2 lines
Note: I used 0 and 999 above since they sort before and after all possible 2-digit decimal numbers, but that's just cosmetic.
Ah neat, so we could still use the two digits 99 and 00 for order level if we wanted then.
Quoted 10 lines
Quoted 6 lines
If that's what you mean?
I'm a bit wary about having the linker sort any of the above SECTION_*'s, but if we're happy to do that perhaps a simple first step might be to see if 0-day but would be happy with just the sort without any consequences to any architecture. Thoughts?
I don't see what is dangerous about it. The section names are such that a lexographical sort will do the right thing, and we can simply use
SORT(.rodata.tbl.*)in the linker script, for example.
OK I'll leave it up to you, I'll go test sorting the sections broadly first.
Quoted 12 lines
Quoted 11 lines
Quoted 5 lines
The other thing is to take a
clue from the implementation in iPXE, which uses priority levels 00 and 99 (or we could use non-integers which sort appropriately instead of using "real" levels) to contain the start and end symbols, which eliminates any need for linker script modifications to add new tables.This solution uses that as well. The only need for adding custom sections is when they have a requirement for a custom run time sort, and also to ensure they don't cause regressions on other subsystems if they have a buggy sort. The run time sorting is all subsystem specific and up to their own semantics.
Quoted 3 lines
Again, from a linker table POV this is nothing other than a read-write table; there is a runtime function that then operates on that read-write table.
OK, I'm convinced, I'll go try these changes now.
Luis
H. Peter Anvin
reviewerQuoted 181 lines · Luis R. Rodriguez
Quoted 3 lines · H. Peter Anvin
Quoted 2 lines · Luis R. Rodriguez
I explain why I do that there but the gist of it is that on Linux
we may also
Quoted 1 line
Quoted 1 line
want stronger semantics for specific linker table solutions, and
solutions such
Quoted 1 line
Quoted 1 line
as those devised on the IOMMU init stuff do
memmove()for sortingdepending on
Quoted 1 line
Quoted 1 line
semantics defined (in the simplest case here so far dependency
between init
Quoted 1 line
Quoted 1 line
sequences), this makes each set of sequences very subsystem
specific. An issue
Quoted 1 line
Quoted 1 line
with *one* subsystem could make things really bad for others. I
thought about
Quoted 1 line
Quoted 1 line
this quite a bit and figured its best left to the subsystem
maintainers to
Quoted 6 lines
Quoted 2 lines
decide.
A table that needs sorting or other runtime handling is just a read-write table for the purpose of the linker table construct. It presents to C as an array of initialized data.
Sure but what I was getting at was that since some run time changes to the
table *might* be desirable, depending on the subsystem, the subsystem table needs
to be able to know the start and end address of its table, and that's a linker script change. But come to think of it, that was me just being pedantic
and careful, I'll try even a run time sort with a few tables of different
structure size to ensure its both possible to just declare them in C and also
sort them without a linker script change.Quoted 1 line
Quoted 1 line
Perhaps a new
sections.hfile (you tell me) which documents thedifferent
Quoted 11 lines
Quoted 9 lines
section components:
/* document this *really* well */
#defineSECTION_RODATA".rodata"
#defineSECTION_INIT".init"
#defineSECTION_INIT_RODATA".init_rodata"
#defineSECTION_READ_MOSTLY".read_mostly"Then on
tables.hwe add the section components support:Yes, something like that. How to macroize it cleanly is another
matter;
Quoted 1 line
we may want to use slightly different conventions that iPXE to match
our
Quoted 1 line
own codebase.
Sure, the style below is from iPXE, the one in the patch set matches the
Linux coding style. Other than that I'd welcome feedback on any issues or recommendations with style on our proposed version oftables.hSeems you provided some pointers below, so I'll go try to incorporate those suggestions.Quoted 12 lines
Quoted 11 lines
#define
__table(component, type, name)(component, type, name)#define
__table_component(table)__table_extract_componenttable#define
__table_extract_component(component, type, name)component#define
__table_type(table)__table_extract_typetable#define
__table_extract_type(component, type, name)type#define
__table_name(table)__table_extract_nametableQuoted 7 lines
Quoted 7 lines
#define
__table_extract_name(component, type, name)name#define
__table_str(x)#x#define __table_section(table, idx) \ "." __table_component (table) ".tbl." __table_name (table)"."
__table_str(idx)Quoted 2 lines
Quoted 2 lines
#define
__table_entry(table, idx)\
Quoted 1 line
Quoted 1 line
__attribute__ ((__section__(__table_section(table, idx)),\
Quoted 5 lines
Quoted 5 lines
__aligned__(__table_alignment(table))))A user could then be something as follows:
#define
X86_INIT_FNS__table(SECTION_INIT, structx86_init_fn,"
x86_init_fns")Quoted 1 line
Quoted 1 line
#define
__x86_init_fn(order_level)__table_entry(X86_INIT_FNS,
order_level)Quoted 2 lines
Yes, but in particular the common case of function initialization
tables
Quoted 10 lines
should be generic.
I'm kind of thinking a syntax like this:
DECLARE_LINKTABLE_RO(struct foo, tablename);DEFINE_LINKTABLE_RO(struct foo, tablename);LINKTABLE_RO(tablename,level)= /* contents */;LINKTABLE_SIZE(tablename)... which would turn into something like this once it goes through
all
Quoted 10 lines
the preprocessing phases
/*
DECLARE_LINKTABLE_RO*/
extern const struct foo tablename[], tablename__end[];/*
DEFINE_LINKTABLE_RO*/DECLARE_LINKTABLE_RO(struct foo, tablename);const struct foo__attribute__((used,section(".rodata.tbl.tablename.0")))
tablename[0];
Quoted 10 lines
const struct foo__attribute__((used,section(".rodata.tbl.tablename.999"))) tablename__end[0];
/*
LINKTABLE_RO*/
static const__typeof__(tablename)__attribute__((used,section(".rodata.tbl.tablename.50")))__tbl_tablename_12345Quoted 4 lines
/*
LINKTABLE_SIZE*/
((tablename__end) - (tablename))... and so on for all the possible sections where we may want tables.
OK thanks so much! Will try working with that.
Quoted 2 lines
Note: I used 0 and 999 above since they sort before and after all possible 2-digit decimal numbers, but that's just cosmetic.
Ah neat, so we could still use the two digits 99 and 00 for order level if we wanted then.
Quoted 3 lines
Quoted 3 lines
If that's what you mean?
I'm a bit wary about having the linker sort any of the above
SECTION_*'s, but
Quoted 1 line
Quoted 1 line
if we're happy to do that perhaps a simple first step might be to
see if 0-day
Quoted 1 line
Quoted 1 line
but would be happy with just the sort without any consequences to
any
Quoted 3 lines
Quoted 1 line
architecture. Thoughts?
I don't see what is dangerous about it. The section names are such
that
Quoted 2 lines
a lexographical sort will do the right thing, and we can simply use
SORT(.rodata.tbl.*)in the linker script, for example.OK I'll leave it up to you, I'll go test sorting the sections broadly first.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The other thing is to take a
clue from the implementation in iPXE, which uses priority levels00 and
Quoted 1 line
Quoted 1 line
Quoted 1 line
99 (or we could use non-integers which sort appropriately instead
of
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
using "real" levels) to contain the start and end symbols, which eliminates any need for linker script modifications to add new
tables.
Quoted 2 lines
Quoted 2 lines
This solution uses that as well. The only need for adding custom
sections
Quoted 1 line
Quoted 1 line
is when they have a requirement for a custom run time sort, and
also to
Quoted 1 line
Quoted 1 line
ensure they don't cause regressions on other subsystems if they
have a buggy
Quoted 1 line
Quoted 1 line
sort. The run time sorting is all subsystem specific and up to
their own
Quoted 3 lines
Quoted 1 line
semantics.
Again, from a linker table POV this is nothing other than a
Quoted 8 lines
read-write
Quoted 1 line
table; there is a runtime function that then operates on that
read-write
Quoted 1 line
table.
OK, I'm convinced, I'll go try these changes now.
Luis
No, start and end symbols are provided by having zero-length sections which sort at the very beginning and very end.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.
Quoted 4 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
A long time ago in a galaxy far,
far away...
:-)
You wouldn't have this on a git tree by any chance?
And thank you for doing this!
Quoted 9 lines · Konrad Rzeszutek Wilk
Quoted 4 lines · Luis R. Rodriguez
From: "Luis R. Rodriguez" <[email protected]>
A long time ago in a galaxy far,
far away...:-)
You wouldn't have this on a git tree by any chance?
You bet, refer to "Where to get code alternatively and testing:" on the original version of this e-mail. As you noted though the i386 change needs the bit for xen subarch, and also the PV enlighten.c code needs the subarch set too.
Quoted 1 line
And thank you for doing this!
You did the original nice work on the nice shiny dependency thing, this just combines it with what ipxe has defined. So thank you!
Lusi