Skip to contentSkip to conversation

2.6.30 Xen merge plan (what's in xen.git)

DISCUSSION Jeremy Fitzhardinge 4 participants 15 messages Mar 24, 2009 – Mar 29, 2009

lore
Share & tools
lore
#0 lore raw

Hi Ingo, Linus,

I have several topic branches which I'd like to merge in this merge window. They fall into two classes: those which make significant changes to arch/x86 files, and those which just touch Xen-specific files.

The x86-touching ones should definitely go via tip.git, just to keep things sane and orderly.

How should we handle the rest? Should I send them directly to Linus, or put them via x86.git too? Either way, they'll generally need to go after the non-Xen changes.

(I'll post full pull-requests with diffstats as followups to this mail.)

All branches in git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git

Branches which mostly or entirely Xen (only trivial non-Xen changes):

push2/xen/core

Updates to existing Xen support, features, optimisations, etc.

push2/xen/control

A set of usermode interfaces for Xen (/sys/hypervisor, /dev/evtchn, updates to xenbus and xenfs)

push2/xen/dom0/core

Core set of changes to add Xen dom0 functionality. (Mostly Xen-only, but adds a new Xen mtrr driver.)

Branches with non-trivial non-Xen changes:

push2/x86/paravirt

Allow preemption while doing a lazy mmu update.

push2/xen/dom0/apic

Changes to hook the Xen dom0 apic model into the standard platform apic code. HPA reviewed and OKed this.

push2/xen/dom0/pci

Add a Xen pci driver and dma_ops implementations. Small changes to x86 pci+swiotlb code to add some function calls.

push2/xen/dom0/agp

Change the AGP driver to allocate its memory using the pci_alloc_coherent to get guaranteed physically contiguous memory, and use proper phys<->bus functions so that Xen's pseudo-phys <-> machine mappings are handled properly.

Thanks,
J

#1 lore raw

[GIT PULL] xen.git 2.6.30: push2/x86/paravirt: preemption in lazy mmu updates

The following changes since commit d9937cb8df272909407552955e26f922945cb5aa:
Ingo Molnar (1):
Merge branch 'x86/apic'

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git push2/x86/paravirt

Jeremy Fitzhardinge (8):
mm: disable preemption in apply_to_pte_range
x86/paravirt: remove lazy mode in interrupts
x86/pvops: replace arch_enter_lazy_cpu_mode with arch_start_context_switch x86/paravirt: flush pending mmu updates on context switch x86/paravirt: finish change from lazy cpu to context switch start/end x86/paravirt: allow preemption with lazy mmu mode
mm: allow preemption in apply_to_pte_range
x86/paravirt: use percpu_ rather than __get_cpu_var

arch/x86/include/asm/paravirt.h    |   22 +++++++-------
arch/x86/include/asm/pgtable.h     |    2 +
arch/x86/include/asm/thread_info.h |    2 +
arch/x86/kernel/kvm.c              |    2 +-
arch/x86/kernel/paravirt.c         |   56 ++++++++++++++++-------------------
arch/x86/kernel/process_32.c       |    2 +-
arch/x86/kernel/process_64.c       |    2 +-
arch/x86/kernel/vmi_32.c           |   20 ++++++++----
arch/x86/lguest/boot.c             |   16 +++++++---
arch/x86/mm/fault.c                |    6 +--
arch/x86/mm/highmem_32.c           |    2 -
arch/x86/mm/iomap_32.c             |    2 -
arch/x86/mm/pageattr.c             |   14 ---------
arch/x86/xen/enlighten.c           |   10 ++----
arch/x86/xen/mmu.c                 |   20 +++++--------
arch/x86/xen/xen-ops.h             |    1 -
include/asm-frv/pgtable.h          |    4 +-
include/asm-generic/pgtable.h      |   21 +++++++------
kernel/sched.c                     |    2 +-
#2 lore raw

[GIT PULL] xen.git 2.6.30: core Xen updates

In reply to Jeremy Fitzhardinge

The following changes since commit 8d336ed83a87149bcae6b872fff1aed8be268342:
Jeremy Fitzhardinge (1):
x86/paravirt: use percpu_ rather than __get_cpu_var

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git push2/xen/core

Alex Nixon (1):
Xen: Add virt_to_pfn helper function

Hannes Eder (1):
NULL noise: arch/x86/xen/smp.c

Jeremy Fitzhardinge (15):
xen: disable preempt for leave_lazy_mmu
xen: separate p2m allocation from setting
xen: dynamically allocate p2m tables
xen: split construction of p2m mfn tables from registration
xen: clean up xen_load_gdt
xen: make xen_load_gdt simpler
xen: remove xen_load_gdt debug
xen: reserve i386 Xen pagetables
xen: mask XSAVE from cpuid
xen: add FIX_TEXT_POKE to fixmap
x86-64: remove PGE from must-have feature list
x86-64: non-paravirt systems always has PSE and PGE xen/mmu: some early pagetable cleanups
Revert "x86: create a non-zero sized bm_pte only when needed" xen/mmu: weaken flush_tlb_other test

arch/x86/include/asm/required-features.h |    8 ++-
arch/x86/include/asm/xen/page.h          |    3 +-
arch/x86/mm/ioremap.c                    |   19 +---
arch/x86/xen/enlighten.c                 |   76 +++++++++++---
arch/x86/xen/mmu.c                       |  160 ++++++++++++++++++++++--------
arch/x86/xen/mmu.h                       |    3 +
arch/x86/xen/smp.c                       |    4 +-
arch/x86/xen/xen-ops.h                   |    2 -
8 files changed, 196 insertions(+), 79 deletions(-)
#3 lore raw

[GIT PULL] xen.git 2.6.30: usermode control interfaces

In reply to Jeremy Fitzhardinge

The following changes since commit 627b11ca44b334e201c35bf1a1582dc10efaf4a8:
Jeremy Fitzhardinge (1):
xen/mmu: weaken flush_tlb_other test

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git push2/xen/control

Ian Campbell (6):
xen: add irq_from_evtchn
xen: add /dev/xen/evtchn driver
xen: export ioctl headers to userspace
xen: drop kexec bits from /sys/hypervisor since kexec isn't implemented yet
xen: remove suspend_cancel hook
xen: use device model for suspending xenbus devices

Jeremy Fitzhardinge (9):
xen/dev-evtchn: clean up locking in evtchn
xen: add "capabilities" file
xen: add /sys/hypervisor support xen/sys/hypervisor: change writable_pt to features xen/xenbus: export xenbus_dev_changed
Merge branch 'push2/xen/xenbus' into push2/xen/control Merge branch 'push2/xen/dev-evtchn' into push2/xen/control Merge branch 'push2/xen/sys-hypervisor' into push2/xen/control Merge branch 'push2/xen/xenfs' into push2/xen/control

drivers/xen/Kconfig               |   20 ++
drivers/xen/Makefile              |    4 +-
drivers/xen/events.c              |    6 +
drivers/xen/evtchn.c              |  507 +++++++++++++++++++++++++++++++++++++
drivers/xen/manage.c              |    9 +-
drivers/xen/sys-hypervisor.c      |  445 ++++++++++++++++++++++++++++++++
drivers/xen/xenbus/xenbus_probe.c |   61 +----
drivers/xen/xenbus/xenbus_xs.c    |    2 +
drivers/xen/xenfs/super.c         |   19 ++-
include/Kbuild                    |    1 +
include/xen/Kbuild                |    1 +
include/xen/events.h              |    3 +
include/xen/evtchn.h              |   88 +++++++
include/xen/interface/version.h   |    3 +
include/xen/xenbus.h              |    3 +-
15 files changed, 1112 insertions(+), 60 deletions(-)

create mode 100644 drivers/xen/evtchn.c
create mode 100644 drivers/xen/sys-hypervisor.c
create mode 100644 include/xen/Kbuild
create mode 100644 include/xen/evtchn.h

#4 lore raw

[GIT PULL] xen.git 2.6.30: xen dom0 core support

In reply to Jeremy Fitzhardinge

The following changes since commit 627b11ca44b334e201c35bf1a1582dc10efaf4a8:
Jeremy Fitzhardinge (1):
xen/mmu: weaken flush_tlb_other test

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git push2/xen/dom0/core

Christophe Saout (1):
paravirtualize IO permission bitmap

Ian Campbell (4):
xen: disable PAT
xen/dom0: Use host E820 map
xen: implement XENMEM_machphys_mapping
xen: clear reserved bits in l3 entries given in the initial pagetables

Jeremy Fitzhardinge (6):
xen dom0: Make hvc_xen console work for dom0. xen/dom0: use _PAGE_IOMAP in ioremap to do machine mappings
xen: allow enable use of VGA console on dom0 xen/dom0: add XEN_DOM0 config option
x86: make /dev/mem mappings _PAGE_IOMAP
x86: don't need "changed" parameter for set_io_bitmap()

Juan Quintela (2):
xen dom0: Initialize xenbus for dom0. xen dom0: Set up basic IO permissions for dom0.

Stephen Tweedie (2):
xen dom0: Add support for the platform_ops hypercall xen mtrr: Add mtrr_ops support for Xen mtrr

arch/x86/include/asm/paravirt.h         |    9 ++
arch/x86/include/asm/pat.h              |    5 +
arch/x86/include/asm/processor.h        |    4 +
arch/x86/include/asm/xen/hypercall.h    |    8 +
arch/x86/include/asm/xen/interface.h    |    6 +-
arch/x86/include/asm/xen/interface_32.h |    5 +
arch/x86/include/asm/xen/interface_64.h |   13 +--
arch/x86/include/asm/xen/page.h         |   15 +--
arch/x86/kernel/cpu/mtrr/Makefile       |    1 +
arch/x86/kernel/cpu/mtrr/amd.c          |    1 +
arch/x86/kernel/cpu/mtrr/centaur.c      |    1 +
arch/x86/kernel/cpu/mtrr/cyrix.c        |    1 +
arch/x86/kernel/cpu/mtrr/generic.c      |    1 +
arch/x86/kernel/cpu/mtrr/main.c         |   11 +-
arch/x86/kernel/cpu/mtrr/mtrr.h         |    5 +
arch/x86/kernel/cpu/mtrr/xen.c          |   59 ++++++++
arch/x86/kernel/ioport.c                |   29 +++-
arch/x86/kernel/paravirt.c              |    1 +
arch/x86/kernel/process.c               |   27 +---
arch/x86/mm/pat.c                       |    7 +-
arch/x86/xen/Kconfig                    |   26 ++++
arch/x86/xen/Makefile                   |    3 +-
arch/x86/xen/enlighten.c                |   51 ++++++-
arch/x86/xen/mmu.c                      |  123 ++++++++++++++++-
arch/x86/xen/setup.c                    |   51 ++++++-
arch/x86/xen/vga.c                      |   67 +++++++++
arch/x86/xen/xen-ops.h                  |   12 ++
drivers/char/hvc_xen.c                  |  101 +++++++++-----
drivers/xen/events.c                    |    2 +-
drivers/xen/xenbus/xenbus_probe.c       |   30 ++++-
include/xen/events.h                    |    2 +
include/xen/interface/memory.h          |   42 ++++++
include/xen/interface/platform.h        |  232 +++++++++++++++++++++++++++++++
include/xen/interface/xen.h             |   41 ++++++
34 files changed, 885 insertions(+), 107 deletions(-)

create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c create mode 100644 arch/x86/xen/vga.c create mode 100644 include/xen/interface/platform.h

#5 lore raw

[GIT PULL] xen.git 2.6.30: xen dom0 apic changes

In reply to Jeremy Fitzhardinge

The following changes since commit 6b36d6251e178636a3c032611948a5c7af44582e:
Jeremy Fitzhardinge (1):
x86: don't need "changed" parameter for set_io_bitmap()

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git push2/xen/dom0/apic

Gerd Hoffmann (2):
xen: set pirq name to something useful.
xen: fix legacy irq setup, make ioapic-less machines work.

Ian Campbell (1):
xen: pre-initialize legacy irqs early

Jeremy Fitzhardinge (13):
xen/dom0: handle acpi lapic parsing in Xen dom0
xen: hook io_apic read/write operations
xen: create dummy ioapic mapping
xen: implement pirq type event channels
x86/io_apic: add get_nr_irqs_gsi()
xen/apic: identity map gsi->irqs
xen: direct irq registration to pirq event channels
xen: bind pirq to vector and event channel
xen: don't setup acpi interrupt unless there is one
xen: use acpi_get_override_irq() to get triggering for legacy irqs
xen: initialize irq 0 too
xen: dynamically allocate irq & event structures
xen: disable MSI

arch/x86/include/asm/io_apic.h |    7 +
arch/x86/include/asm/xen/pci.h |   13 ++
arch/x86/kernel/acpi/boot.c    |   18 +++-
arch/x86/kernel/apic/io_apic.c |   37 +++++-
arch/x86/xen/Kconfig           |   11 ++
arch/x86/xen/Makefile          |    3 +-
arch/x86/xen/apic.c            |   60 +++++++++
arch/x86/xen/enlighten.c       |    2 +
arch/x86/xen/mmu.c             |   10 ++
arch/x86/xen/pci.c             |   86 +++++++++++++
arch/x86/xen/xen-ops.h         |    6 +
drivers/pci/pci.h              |    2 -
drivers/xen/events.c           |  273 ++++++++++++++++++++++++++++++++++++++--
include/linux/pci.h            |    6 +
include/xen/events.h           |   22 ++++
15 files changed, 541 insertions(+), 15 deletions(-)

create mode 100644 arch/x86/include/asm/xen/pci.h create mode 100644 arch/x86/xen/apic.c create mode 100644 arch/x86/xen/pci.c

#6 lore raw

[GIT PULL] xen.git 2.6.30: xen dom0 PCI device support

In reply to Jeremy Fitzhardinge

The following changes since commit 6b36d6251e178636a3c032611948a5c7af44582e:
Jeremy Fitzhardinge (1):
x86: don't need "changed" parameter for set_io_bitmap()

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git push2/xen/dom0/pci

Alex Nixon (7):
xen: Don't disable the I/O space
xen: Allow unprivileged Xen domains to create iomap pages
Xen: Rename the balloon lock
xen: Add xen_create_contiguous_region
x86/PCI: Clean up pci_cache_line_size
x86/PCI: Enable scanning of all pci functions Xen/x86/PCI: Add support for the Xen PCI subsytem

Ian Campbell (4):
xen swiotlb: fixup swiotlb is chunks smaller than MAX_CONTIG_ORDER
xen: add hooks for mapping phys<->bus addresses in swiotlb xen/swiotlb: add hook for swiotlb_arch_range_needs_mapping
xen: enable swiotlb for xen domain 0.

Jeremy Fitzhardinge (7):
x86/pci: make sure _PAGE_IOMAP it set on pci mappings xen/pci: clean up Kconfig a bit
xen: make sure swiotlb allocation is physically contigious xen/swiotlb: use dma_alloc_from_coherent to get device coherent memory
swiotlb: use swiotlb_alloc_boot to allocate emergency pool xen/swiotlb: improve comment on gfp flags in xen_alloc_coherent() xen/swiotlb: add sync functions

arch/x86/Kconfig                 |    4 +
arch/x86/include/asm/pci.h       |    8 +-
arch/x86/include/asm/pci_x86.h   |    2 +
arch/x86/include/asm/xen/iommu.h |   12 ++
arch/x86/kernel/pci-dma.c        |    4 +-
arch/x86/kernel/pci-swiotlb.c    |   28 +++-
arch/x86/pci/Makefile            |    1 +
arch/x86/pci/common.c            |   18 ++-
arch/x86/pci/i386.c              |    3 +
arch/x86/pci/init.c              |    6 +
arch/x86/pci/xen.c               |   52 ++++++
arch/x86/xen/Kconfig             |    3 +
arch/x86/xen/enlighten.c         |    6 +-
arch/x86/xen/mmu.c               |  225 +++++++++++++++++++++++++-
arch/x86/xen/setup.c             |    3 -
drivers/pci/Makefile             |    2 +
drivers/pci/xen-iommu.c          |  331 ++++++++++++++++++++++++++++++++++++++
drivers/xen/balloon.c            |   15 +--
include/xen/interface/memory.h   |   50 ++++++
include/xen/swiotlb.h            |   19 +++
include/xen/xen-ops.h            |    6 +
lib/swiotlb.c                    |    3 +-
22 files changed, 770 insertions(+), 31 deletions(-)

create mode 100644 arch/x86/include/asm/xen/iommu.h create mode 100644 arch/x86/pci/xen.c create mode 100644 drivers/pci/xen-iommu.c create mode 100644 include/xen/swiotlb.h

#7 lore raw
In reply to Jeremy Fitzhardinge
Quoted 21 lines · Jeremy Fitzhardinge

Hi Ingo, Linus,

I have several topic branches which I'd like to merge in this merge window. They fall into two classes: those which make significant changes to arch/x86 files, and those which just touch Xen-specific files.

The x86-touching ones should definitely go via tip.git, just to keep things sane and orderly.

How should we handle the rest? Should I send them directly to Linus, or put them via x86.git too? Either way, they'll generally need to go after the non-Xen changes.

(I'll post full pull-requests with diffstats as followups to this mail.)

All branches in git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git

Branches which mostly or entirely Xen (only trivial non-Xen changes):

Quoted 8 lines

push2/xen/core

Updates to existing Xen support, features, optimisations, etc.

push2/xen/control

A set of usermode interfaces for Xen (/sys/hypervisor, /dev/evtchn, updates to xenbus and xenfs)

Will take a look.

Quoted 4 lines

push2/xen/dom0/core

Core set of changes to add Xen dom0 functionality. (Mostly Xen-only, but adds a new Xen mtrr driver.)

I suspect this makes limited sense without the arch dom0 changes so they should go with this bucket:

Quoted 5 lines

Branches with non-trivial non-Xen changes:

push2/x86/paravirt

Allow preemption while doing a lazy mmu update.

That's probably OK, just a bit late. If Thomas signs off on them then i guess it might be OK later in the window - he's the one who was struggling with paravirt lazy mmu fallout in the past. Thomas?

Quoted 10 lines

push2/xen/dom0/apic

Changes to hook the Xen dom0 apic model into the standard platform apic code. HPA reviewed and OKed this.

push2/xen/dom0/pci

Add a Xen pci driver and dma_ops implementations. Small changes to x86 pci+swiotlb code to add some function calls.

Quoted 6 lines

push2/xen/dom0/agp

Change the AGP driver to allocate its memory using the pci_alloc_coherent to get guaranteed physically contiguous memory, and use proper phys<->bus functions so that Xen's pseudo-phys <-> machine mappings are handled properly.

hpa wanted dom0 changes in some cleaner fashion - he knows the details. (he'll be back in a few days)

Ingo

#8 lore raw
Quoted 6 lines · Ingo Molnar
Quoted 6 lines

Branches which mostly or entirely Xen (only trivial non-Xen changes):

push2/xen/core

Updates to existing Xen support, features, optimisations, etc.

Quoted 8 lines
Quoted 5 lines

push2/xen/control

A set of usermode interfaces for Xen (/sys/hypervisor, /dev/evtchn, updates to xenbus and xenfs)

Will take a look.

Thanks.

Quoted 8 lines
Quoted 5 lines

push2/xen/dom0/core

Core set of changes to add Xen dom0 functionality. (Mostly Xen-only, but adds a new Xen mtrr driver.)

I suspect this makes limited sense without the arch dom0 changes so they should go with this bucket:

Yes, you wouldn't get much beyond some console messages and a crash without the rest.

Quoted 9 lines
Quoted 6 lines

Branches with non-trivial non-Xen changes:

push2/x86/paravirt

Allow preemption while doing a lazy mmu update.

That's probably OK, just a bit late. If Thomas signs off on them then i guess it might be OK later in the window - he's the one who was struggling with paravirt lazy mmu fallout in the past. Thomas?

I've posted those patches several times over the last month. I needed to add a preempt_disable/enable pair in .29 to apply_to_page_range as a bugfix, which PeterZ, NickP and AKPM were not particularly happy about. This is the proper fix.

That said, they're not essential to the rest of the changes, and it would be fairly simple to drop them out (it would be a matter of clearing up some conflicts; no functional interactions).

Quoted 12 lines
Quoted 12 lines

push2/xen/dom0/apic

Changes to hook the Xen dom0 apic model into the standard platform apic code. HPA reviewed and OKed this.

push2/xen/dom0/pci

Add a Xen pci driver and dma_ops implementations. Small changes to x86 pci+swiotlb code to add some function calls.

push2/xen/dom0/agp

Quoted 8 lines
Quoted 5 lines

Change the AGP driver to allocate its memory using the pci_alloc_coherent to get guaranteed physically contiguous memory, and use proper phys<->bus functions so that Xen's pseudo-phys <-> machine mappings are handled properly.

hpa wanted dom0 changes in some cleaner fashion - he knows the details. (he'll be back in a few days)

When I spoke to him last week about this he said he was OK with it. He didn't make any specific (or even general) objections. Do you know which subsystem he was referring to?

J

#9 lore raw
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 2 lines

hpa wanted dom0 changes in some cleaner fashion - he knows the details. (he'll be back in a few days)

When I spoke to him last week about this he said he was OK with it. He didn't make any specific (or even general) objections. Do you know which subsystem he was referring to?

I was mostly objecting to the apic patches -- I'm okay with the latest version, as long as they aren't in hot code (I never got the time to measure it.)

Other than that: the x86-touching bits seemed okay, the pure xen code... well, it's not my problem.

I'll be back on Saturday.

-hpa

#10 lore raw
Quoted 3 lines · H. Peter Anvin

I was mostly objecting to the apic patches -- I'm okay with the latest version, as long as they aren't in hot code (I never got the time to measure it.)

Talking about performance measurements... do you have mmap-perf numbers (paravirt on native vs non-paravirt) for the latest -tip kernel? I've lost track of the numbers after all the changes.

-hpa

#11 lore raw
In reply to H. Peter Anvin
Quoted 1 line · H. Peter Anvin

I was mostly objecting to the apic patches -- I'm okay with the latest version, as long as they aren't in hot code (I never got the time to measure it.)

None of the changes are in the interrupt delivery path; they all just during apic discovery and interrupt routing/setup.

J

#12 lore raw
In reply to Ingo Molnar
Quoted 9 lines · Ingo Molnar
Quoted 6 lines

Branches with non-trivial non-Xen changes:

push2/x86/paravirt

Allow preemption while doing a lazy mmu update.

That's probably OK, just a bit late. If Thomas signs off on them then i guess it might be OK later in the window - he's the one who was struggling with paravirt lazy mmu fallout in the past. Thomas?

Those patches were actually in response to PeterZ's (and NickP+AKPM) concerns about adding preempt-disabled section to apply_to_page_range(). He's acked them now.

Thanks,
J

#14 lore raw
Quoted 10 lines · Andy Burns

2009/3/24 Jeremy Fitzhardinge <[email protected]>:

Quoted 3 lines

push2/xen/dom0/apic
push2/xen/dom0/pci

Does it mean that if these patchsets make it into 2.6.30 there will be paravirt_ops dom0 that supports PCI passthrough?

No, those are just the changes necessary to have native device drivers in dom0 work at all. In fact, I'm not proposing any backend device support for mainline in this merge window, so you'll still need some out of tree changes to get a useful system. But these changes do have a fair amount in common with what's needed for pci passthrough, so there is some progress (albeit abstract).

J