Skip to contentSkip to conversation

State of KVM bits in linux-headers

RFC PATCH New Jan Kiszka 5 participants 20 messages Jan 11, 2012 – Jan 12, 2012 KVM

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Alex, it looks to me like this is mostly PPC stuff. Can you comment on the origin and workflow? E.g. KVM_CAP_SW_TLB: This has been added half a year ago but is not in any Linux release around. Fishy...

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#1 lore raw
Quoted 9 lines · Jan Kiszka

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Yes, call me even more unhappy about it :(.

Quoted 3 lines

Alex, it looks to me like this is mostly PPC stuff. Can you comment on the origin and workflow? E.g. KVM_CAP_SW_TLB: This has been added half a year ago but is not in any Linux release around. Fishy...

Ok, here's my workflow:

* KVM: receive patches on the ML
* KVM: wait for reviews, review myself
* KVM: send out a pull request
-- this is the point in time where I assume the ABI can be considered stable --
* QEMU: run update on the headers, because in a perfect world things should hit kvm.git any day
* KVM: pull request gets reviews causing not-pulls or abi changes and lots of churn because i need forever to pullreq again ;)

I guess you see the problem. Hence I haven't pushed any kernel header updates since I realized how badly broken that process was. However even the stuff that's in qemu.git now hasn't managed to get upstream yet.

Quoted 5 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

I will reappear with ONE_REG semantics.

Alex

author
#19 lore raw

RFC PATCH 1/1 Update linux headers against kvm.git

In reply to Jan Kiszka
Quoted 20 lines · Jan Kiszka

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Alex, it looks to me like this is mostly PPC stuff. Can you comment on the origin and workflow? E.g. KVM_CAP_SW_TLB: This has been added half a year ago but is not in any Linux release around. Fishy...

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

Quoted 7 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

Jan

Just to underline this, not for merge (yet).

Is it clear that those PPC features will be merged upstream as-is now?

Jan

---8<---

This synchronizes our headers with kvm.git ff92e9b557 - and breaks PPC build. Fairly telling...

---
 linux-headers/asm-powerpc/kvm.h   |   37 -------------------------
 linux-headers/asm-x86/hyperv.h    |    1 +
 linux-headers/linux/kvm.h         |   54 ++----------------------------------
 linux-headers/linux/kvm_para.h    |    1 -
 linux-headers/linux/virtio_ring.h |    6 ++--
 5 files changed, 7 insertions(+), 92 deletions(-)
linux-headers/asm-powerpc/kvm.h +0 -37
index fb3fddc..f7727d9 100644
@@ -292,41 +292,4 @@ struct kvm_allocate_rma {
 	__u64 rma_size;
 };
 
-struct kvm_book3e_206_tlb_entry {
-	__u32 mas8;
-	__u32 mas1;
-	__u64 mas2;
-	__u64 mas7_3;
-};
-
-struct kvm_book3e_206_tlb_params {
-	/*
-	 * For mmu types KVM_MMU_FSL_BOOKE_NOHV and KVM_MMU_FSL_BOOKE_HV:
-	 *
-	 * - The number of ways of TLB0 must be a power of two between 2 and
-	 *   16.
-	 * - TLB1 must be fully associative.
-	 * - The size of TLB0 must be a multiple of the number of ways, and
-	 *   the number of sets must be a power of two.
-	 * - The size of TLB1 may not exceed 64 entries.
-	 * - TLB0 supports 4 KiB pages.
-	 * - The page sizes supported by TLB1 are as indicated by
-	 *   TLB1CFG (if MMUCFG[MAVN] = 0) or TLB1PS (if MMUCFG[MAVN] = 1)
-	 *   as returned by KVM_GET_SREGS.
-	 * - TLB2 and TLB3 are reserved, and their entries in tlb_sizes[]
-	 *   and tlb_ways[] must be zero.
-	 *
-	 * tlb_ways[n] = tlb_sizes[n] means the array is fully associative.
-	 *
-	 * KVM will adjust TLBnCFG based on the sizes configured here,
-	 * though arrays greater than 2048 entries will have TLBnCFG[NENTRY]
-	 * set to zero.
-	 */
-	__u32 tlb_sizes[4];
-	__u32 tlb_ways[4];
-	__u32 reserved[8];
-};
-
-#define KVM_ONE_REG_PPC_HIOR	KVM_ONE_REG_PPC | 0x100
-
 #endif /* __LINUX_KVM_POWERPC_H */
linux-headers/asm-x86/hyperv.h +1 -0
index 5df477a..b80420b 100644
@@ -189,5 +189,6 @@
 #define HV_STATUS_INVALID_HYPERCALL_CODE	2
 #define HV_STATUS_INVALID_HYPERCALL_INPUT	3
 #define HV_STATUS_INVALID_ALIGNMENT		4
+#define HV_STATUS_INSUFFICIENT_BUFFERS		19
 
 #endif
linux-headers/linux/kvm.h +3 -51

79 lines not on this page · raw message

linux-headers/linux/kvm_para.h +0 -1
index b315e27..7bdcf93 100644
@@ -26,4 +26,3 @@
 #include <asm/kvm_para.h>
 
 #endif /* __LINUX_KVM_PARA_H */
-
linux-headers/linux/virtio_ring.h +3 -3
index 78289ee..1b333e2 100644
@@ -135,13 +135,13 @@ static __inline__ void vring_init(struct vring *vr, unsigned int num, void *p,
 	vr->num = num;
 	vr->desc = p;
 	vr->avail = p + num*sizeof(struct vring_desc);
-	vr->used = (void *)(((unsigned long)&vr->avail->ring[num] + align-1)
-			    & ~(align - 1));
+	vr->used = (void *)(((unsigned long)&vr->avail->ring[num] + sizeof(__u16)
+		+ align-1) & ~(align - 1));
 }
 
 static __inline__ unsigned vring_size(unsigned int num, unsigned long align)
 {
-	return ((sizeof(struct vring_desc) * num + sizeof(__u16) * (2 + num)
+	return ((sizeof(struct vring_desc) * num + sizeof(__u16) * (3 + num)
 		 + align - 1) & ~(align - 1))
 		+ sizeof(__u16) * 3 + sizeof(struct vring_used_elem) * num;
 }
Signature · 1 line
1.7.3.4
#2 lore raw
In reply to Alexander Graf
Quoted 13 lines · Alexander Graf
Quoted 9 lines · Jan Kiszka

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Quoted 1 line

Yes, call me even more unhappy about it :(.

May I suggest the following:

1) Have the header syncing script take a commit hash that's stored in git. Make script ensure that this has is in Linus' tree.

2) Maintain a patch on top of Linus' tree in qemu.git that the script would apply before actually syncing header files.

That let's us track how we're differing from upstream in a more reliable fashion.

Quoted 6 lines
Quoted 3 lines

Alex, it looks to me like this is mostly PPC stuff. Can you comment on the origin and workflow? E.g. KVM_CAP_SW_TLB: This has been added half a year ago but is not in any Linux release around. Fishy...

Ok, here's my workflow:

Quoted 8 lines

* KVM: receive patches on the ML
* KVM: wait for reviews, review myself
* KVM: send out a pull request
-- this is the point in time where I assume the ABI can be considered stable --
* QEMU: run update on the headers, because in a perfect world things should hit kvm.git any day
* KVM: pull request gets reviews causing not-pulls or abi changes and lots of churn because i need forever to pullreq again ;)

I guess you see the problem. Hence I haven't pushed any kernel header updates since I realized how badly broken that process was. However even the stuff that's in qemu.git now hasn't managed to get upstream yet.

I don't think it's a broken process. I think you made a reasonable set of assumptions. I think it was just an exceptional circumstance.

Regards,

Anthony Liguori

author
#6 lore raw
In reply to Alexander Graf
Quoted 19 lines · Alexander Graf
Quoted 9 lines · Jan Kiszka

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Yes, call me even more unhappy about it :(.

Quoted 3 lines

Alex, it looks to me like this is mostly PPC stuff. Can you comment on the origin and workflow? E.g. KVM_CAP_SW_TLB: This has been added half a year ago but is not in any Linux release around. Fishy...

Quoted 8 lines

Ok, here's my workflow:

* KVM: receive patches on the ML
* KVM: wait for reviews, review myself
* KVM: send out a pull request
-- this is the point in time where I assume the ABI can be considered stable --
* QEMU: run update on the headers, because in a perfect world things should hit kvm.git any day
* KVM: pull request gets reviews causing not-pulls or abi changes and lots of churn because i need forever to pullreq again ;)

Likely, the last item has to be moved up by two steps...

Quoted 2 lines

I guess you see the problem. Hence I haven't pushed any kernel header updates since I realized how badly broken that process was. However even the stuff that's in qemu.git now hasn't managed to get upstream yet.

On the other hand, if I recall correctly, there were some complaint on the list recently about a header update patch again a Linux -rc version. Because it removed the limbo land stuff in the same run, of course. That's very bad. I see the problem: ppc targets will no longer build, at least with KVM enabled, right? But this needs to be resolved now.

Quoted 7 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

Quoted 9 lines

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

I will reappear with ONE_REG semantics.

OK.

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#7 lore raw
Quoted 14 lines · Jan Kiszka
Quoted 14 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

Quoted 7 lines
Quoted 2 lines

I will reappear with ONE_REG semantics.

OK.

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Before we did submodules and had a responsive BIOS maintainer, we maintained patches within qemu.git for our external dependencies. I think that's a good strategy here too. It's a little painful, but not entirely awful.

At least it makes it possible for you to (hopefully) trivial rebase a patch if something is still in limbo.

Regards,

Anthony Liguori

Quoted 3 lines

Jan

#3 lore raw
In reply to Anthony Liguori
Quoted 16 lines · Anthony Liguori
Quoted 13 lines · Alexander Graf
Quoted 9 lines · Jan Kiszka

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Quoted 1 line

Yes, call me even more unhappy about it :(.

Quoted 7 lines

May I suggest the following:

1) Have the header syncing script take a commit hash that's stored in git. Make script ensure that this has is in Linus' tree.

2) Maintain a patch on top of Linus' tree in qemu.git that the script would apply before actually syncing header files.

That let's us track how we're differing from upstream in a more reliable fashion.

Yeah, I guess the ultimate question it boils down to is: when is something upstream? The average time it takes for patches to trickle through to Linus right now is in the magnitude of half a year to a year.

Quoted 14 lines
Quoted 14 lines
Quoted 4 lines

Alex, it looks to me like this is mostly PPC stuff. Can you comment on the origin and workflow? E.g. KVM_CAP_SW_TLB: This has been added half a year ago but is not in any Linux release around. Fishy...

Ok, here's my workflow:

* KVM: receive patches on the ML
* KVM: wait for reviews, review myself
* KVM: send out a pull request
-- this is the point in time where I assume the ABI can be considered stable --
* QEMU: run update on the headers, because in a perfect world things should hit kvm.git any day
* KVM: pull request gets reviews causing not-pulls or abi changes and lots of churn because i need forever to pullreq again ;)

Quoted 3 lines
Quoted 1 line

I guess you see the problem. Hence I haven't pushed any kernel header updates since I realized how badly broken that process was. However even the stuff that's in qemu.git now hasn't managed to get upstream yet.

I don't think it's a broken process. I think you made a reasonable set of assumptions. I think it was just an exceptional circumstance.

Several times in a row? No, the assumptions were just wrong. In the kvm world, pull requests don't mean upstream, they mean the same as a patch set.

Alex

#8 lore raw
In reply to Anthony Liguori
Quoted 18 lines · Anthony Liguori
Quoted 17 lines · Jan Kiszka
Quoted 16 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

I will reappear with ONE_REG semantics.

Quoted 8 lines
Quoted 4 lines

OK.

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Before we did submodules and had a responsive BIOS maintainer, we maintained patches within qemu.git for our external dependencies. I think that's a good strategy here too. It's a little painful, but not entirely awful.

At least it makes it possible for you to (hopefully) trivial rebase a patch if something is still in limbo.

Yeah, that works. I can easily script that part. It doesn't solve the actual underlying problem though that we don't know when the abi is actually stable. I'm slowly starting to understand Pekka ;).

Alex

author
#4 lore raw
In reply to Anthony Liguori
Quoted 18 lines · Anthony Liguori
Quoted 13 lines · Alexander Graf
Quoted 9 lines · Jan Kiszka

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Quoted 1 line

Yes, call me even more unhappy about it :(.

May I suggest the following:

Quoted 7 lines

1) Have the header syncing script take a commit hash that's stored in git. Make script ensure that this has is in Linus' tree.

2) Maintain a patch on top of Linus' tree in qemu.git that the script would apply before actually syncing header files.

That let's us track how we're differing from upstream in a more reliable fashion.

That sounds fairly complicated for a simple problem: Do not merge ABI changes that aren't at least in kvm.git. There are also other reasons for this, beside making the sync harder.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#5 lore raw
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Anthony Liguori
Quoted 13 lines · Alexander Graf
Quoted 9 lines · Jan Kiszka

Hi,

I'm a bit unhappy about the current state of our supposed to be automatically sync'ed linux-headers directory in qemu. It has been updated several times against undefined kernel trees, means against neither a released version nor kvm.git. Now, if I run an update against kvm.git + some local change, I get a churn of removals. Same will happen when that local change ever goes upstream before the other stuff got finally committed.

Quoted 1 line

Yes, call me even more unhappy about it :(.

May I suggest the following:

1) Have the header syncing script take a commit hash that's stored in git. Make script ensure that this has is in Linus' tree.

Quoted 8 lines
Quoted 4 lines

2) Maintain a patch on top of Linus' tree in qemu.git that the script would apply before actually syncing header files.

That let's us track how we're differing from upstream in a more reliable fashion.

That sounds fairly complicated for a simple problem: Do not merge ABI changes that aren't at least in kvm.git. There are also other reasons for this, beside making the sync harder.

Let's just try to get my patch queue into kvm.git asap and then never to push linux-header updates before they hit kvm.git again. That's easier than setting up any complicated processes or scripts.

Alex

author
#9 lore raw
In reply to Alexander Graf
Quoted 23 lines · Alexander Graf
Quoted 20 lines · Anthony Liguori
Quoted 19 lines · Jan Kiszka
Quoted 7 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

Quoted 9 lines

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

I will reappear with ONE_REG semantics.

OK.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Before we did submodules and had a responsive BIOS maintainer, we maintained patches within qemu.git for our external dependencies. I think that's a good strategy here too. It's a little painful, but not entirely awful.

At least it makes it possible for you to (hopefully) trivial rebase a patch if something is still in limbo.

Yeah, that works. I can easily script that part. It doesn't solve the actual underlying problem though that we don't know when the abi is actually stable. I'm slowly starting to understand Pekka ;).

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#10 lore raw
Quoted 27 lines · Jan Kiszka
Quoted 26 lines · Alexander Graf
Quoted 23 lines · Anthony Liguori
Quoted 21 lines · Jan Kiszka
Quoted 7 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

Quoted 9 lines

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

I will reappear with ONE_REG semantics.

OK.

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Before we did submodules and had a responsive BIOS maintainer, we maintained patches within qemu.git for our external dependencies. I think that's a good strategy here too. It's a little painful, but not entirely awful.

At least it makes it possible for you to (hopefully) trivial rebase a patch if something is still in limbo.

Yeah, that works. I can easily script that part. It doesn't solve the actual underlying problem though that we don't know when the abi is actually stable. I'm slowly starting to understand Pekka ;).

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Are you suggesting that kvm header updates go through uq/master? That seems reasonable to me and is certainly the least amount of change.

Regards,

Anthony Liguori

Quoted 3 lines

Jan

#11 lore raw
Quoted 30 lines · Anthony Liguori
Quoted 29 lines · Jan Kiszka
Quoted 28 lines · Alexander Graf
Quoted 25 lines · Anthony Liguori
Quoted 14 lines · Jan Kiszka
Quoted 14 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

Quoted 7 lines
Quoted 2 lines

I will reappear with ONE_REG semantics.

OK.

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Before we did submodules and had a responsive BIOS maintainer, we maintained patches within qemu.git for our external dependencies. I think that's a good strategy here too. It's a little painful, but not entirely awful.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

At least it makes it possible for you to (hopefully) trivial rebase a patch if something is still in limbo.

Yeah, that works. I can easily script that part. It doesn't solve the actual underlying problem though that we don't know when the abi is actually stable. I'm slowly starting to understand Pekka ;).

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Are you suggesting that kvm header updates go through uq/master? That seems reasonable to me and is certainly the least amount of change.

So how about code that actually leverages the new headers?

Alex

author
#17 lore raw
In reply to Anthony Liguori
Quoted 32 lines · Anthony Liguori
Quoted 31 lines · Jan Kiszka
Quoted 30 lines · Alexander Graf
Quoted 26 lines · Anthony Liguori
Quoted 14 lines · Jan Kiszka
Quoted 14 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

Quoted 7 lines
Quoted 2 lines

I will reappear with ONE_REG semantics.

OK.

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Before we did submodules and had a responsive BIOS maintainer, we maintained patches within qemu.git for our external dependencies. I think that's a good strategy here too. It's a little painful, but not entirely awful.

At least it makes it possible for you to (hopefully) trivial rebase a patch if something is still in limbo.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Yeah, that works. I can easily script that part. It doesn't solve the actual underlying problem though that we don't know when the abi is actually stable. I'm slowly starting to understand Pekka ;).

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Are you suggesting that kvm header updates go through uq/master? That seems reasonable to me and is certainly the least amount of change.

Would be possible at least for changes that affect KVM bits. But we also use that headers for virtio and vhost. VFIO will surely join that group. So there is still coordination necessary.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#12 lore raw
In reply to Alexander Graf
Quoted 9 lines · Alexander Graf
Quoted 4 lines · Anthony Liguori
Quoted 2 lines

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Are you suggesting that kvm header updates go through uq/master? That seems reasonable to me and is certainly the least amount of change.

So how about code that actually leverages the new headers?

Shared KVM infrastructure should go through uq/master. So changes to kvm-all.c, linux-headers/* should go through uq/master.

Target specific kvm changes should go through the appropriate submaintainers tree.

Regards,

Anthony Liguori

Quoted 5 lines

Alex

#13 lore raw
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Alexander Graf
Quoted 3 lines · Anthony Liguori
Quoted 2 lines

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Are you suggesting that kvm header updates go through uq/master? That seems reasonable to me and is certainly the least amount of change.

So how about code that actually leverages the new headers?

Shared KVM infrastructure should go through uq/master. So changes to kvm-all.c, linux-headers/* should go through uq/master.

Target specific kvm changes should go through the appropriate submaintainers tree.

So then if I add some target specific stuff to KVM, I have to

* send pullreq to KVM
* wait for that to be applied
* post a patch to uq/master to update headers
* wait for that to merge back to qemu.git
* send a pull request to qemu.git

right? And then after about 3 months we'll have the feature available ;).

Alex

#14 lore raw
Quoted 12 lines · Alexander Graf
Quoted 9 lines · Anthony Liguori
Quoted 8 lines · Alexander Graf
Quoted 4 lines · Anthony Liguori
Quoted 2 lines

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Are you suggesting that kvm header updates go through uq/master? That seems reasonable to me and is certainly the least amount of change.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

So how about code that actually leverages the new headers?

Shared KVM infrastructure should go through uq/master. So changes to kvm-all.c, linux-headers/* should go through uq/master.

Target specific kvm changes should go through the appropriate submaintainers tree.

So then if I add some target specific stuff to KVM,

That requires a header update?

Quoted 5 lines

I have to

* send pullreq to KVM
* wait for that to be applied
* post a patch to uq/master to update headers

Strictly from a QEMU perspective, we can't depend on APIs that aren't committed upstream yet.

Quoted 2 lines

* wait for that to merge back to qemu.git
* send a pull request to qemu.git

Maybe we need to bring a stripped down version of Linux into qemu.git to make it easier to simultaneously update both trees... ;-)

Quoted 2 lines

right? And then after about 3 months we'll have the feature available ;).

You can always just get Acked-by's from the appropriate maintainers. That's just as good as going through the tree.

Regards,

Anthony Liguori

Quoted 4 lines

Alex

#15 lore raw
Quoted 15 lines · Anthony Liguori
Quoted 14 lines · Alexander Graf
Quoted 11 lines · Anthony Liguori
Quoted 9 lines · Alexander Graf
Quoted 4 lines · Anthony Liguori
Quoted 2 lines

IIRC, we never had this problem with qemu-kvm - as the merges were coordinated with the kernel (subsystem) tree.

Are you suggesting that kvm header updates go through uq/master? That seems reasonable to me and is certainly the least amount of change.

So how about code that actually leverages the new headers?

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Shared KVM infrastructure should go through uq/master. So changes to kvm-all.c, linux-headers/* should go through uq/master.

Target specific kvm changes should go through the appropriate submaintainers tree.

So then if I add some target specific stuff to KVM,

That requires a header update?

Almost all of the time, yes. The target is still rather incomplete. And even in places where it is, hardware evolves and we just get new information we need to pass back and forth.

Quoted 8 lines
Quoted 6 lines

I have to

* send pullreq to KVM
* wait for that to be applied
* post a patch to uq/master to update headers

Strictly from a QEMU perspective, we can't depend on APIs that aren't committed upstream yet.

The question again is: When do we consider something upstream?

Quoted 5 lines
Quoted 3 lines

* wait for that to merge back to qemu.git
* send a pull request to qemu.git

Maybe we need to bring a stripped down version of Linux into qemu.git to make it easier to simultaneously update both trees... ;-)

Nice one ;)

Quoted 5 lines
Quoted 3 lines

right? And then after about 3 months we'll have the feature available ;).

You can always just get Acked-by's from the appropriate maintainers. That's just as good as going through the tree.

So every time we change headers, I just require Avi's ack and then he can't complain on those patches later? Good idea! :)

Alex

#18 lore raw
In reply to Alexander Graf
Quoted 23 lines · Alexander Graf
Quoted 20 lines · Anthony Liguori
Quoted 19 lines · Jan Kiszka
Quoted 7 lines
Quoted 6 lines

I would like to see us avoiding this in the future. Headers update patches should mention the source and should not be merged until the ABI changes actually made it at least into kvm.git. Same applies, of course, to the functional changes related to that ABI. Otherwise we risk quite some mess on everyone's side.

Quoted 9 lines

I agree.

Quoted 4 lines

Another thing: KVM_CAP_PPC_HIOR has been removed again from the kernel and also the header. Is there real free space now or will the cap reappear? If there should better be a placeholder, let's add it (to the kernel).

I will reappear with ONE_REG semantics.

OK.

Quoted 10 lines
Quoted 6 lines
Quoted 2 lines

Then please clean up now so that update-linux-headers.sh can be used again by "normal" developers. :)

Before we did submodules and had a responsive BIOS maintainer, we maintained patches within qemu.git for our external dependencies. I think that's a good strategy here too. It's a little painful, but not entirely awful.

At least it makes it possible for you to (hopefully) trivial rebase a patch if something is still in limbo.

Yeah, that works. I can easily script that part. It doesn't solve the actual underlying problem though that we don't know when the abi is actually stable. I'm slowly starting to understand Pekka ;).

In my recent experience with submitting Joerg's patch series that touches both kernel and tools/perf I didn't see any advantages in having them in the same repository. Yes, the repository is the same, but maintainers are different and have their own timelines and priorities. Long story short userspace part was applied almost three month after the kernel part.

--
Gleb.

#16 lore raw
In reply to Alexander Graf
Quoted 2 lines · Alexander Graf
Quoted 2 lines

Strictly from a QEMU perspective, we can't depend on APIs that aren't committed upstream yet.

We can't release any qemu that depends on something not upstream.

Quoted 1 line

The question again is: When do we consider something upstream?

This far from a qemu release, we can consider anything in in kvm.git master (or slightly trailing that, kvm.git linux-next) as upstream. It's very rare that an ABI gets pulled out of that and not merged, and if it is, we can pull out the qemu feature. An ABI can change, but that just means we need to echo the change in qemu.

As qemu gets closer to release, we'll need to look at Linus' tree instead of kvm.git.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

right? And then after about 3 months we'll have the feature available ;).

You can always just get Acked-by's from the appropriate maintainers. That's just as good as going through the tree.

So every time we change headers, I just require Avi's ack and then he can't complain on those patches later? Good idea! :)

Of course I can complain about my patches later.

But yes, if your change involves multiple subsystem, get it through the most affected subsystem and get acks from the rest.

Signature · 1 line
error compiling committee.c: too many arguments to function