Skip to contentSkip to conversation

delayacct: align to 8 byte boundary on 64-bit systems

PATCH In review Jeff Mahoney 3 participants 7 messages Feb 12, 2010 – Feb 17, 2010 Taskstats statistics interface

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 delayacct: align to 8 byte boundary on 64-bit systems

prepare_reply sets up an skb for the response. If I understand it correctly, the payload contains:

 +--------------------------------+
 | genlmsghdr - 4 bytes           |
 +--------------------------------+
 | NLA header - 4 bytes           | /* Aggregate header */
 +-+------------------------------+
 | | NLA header - 4 bytes         | /* PID header */
 | +------------------------------+
 | | pid/tgid   - 4 bytes         |
 | +------------------------------+
 | | NLA header - 4 bytes         | /* stats header */
 | + -----------------------------+ <- oops. aligned on 4 byte boundary
 | | struct taskstats - 328 bytes |
 +-+------------------------------+

The start of the taskstats struct must be 8 byte aligned on IA64 (and other systems with 8 byte alignment rules for 64-bit types) or runtime alignment warnings will be issued.

This patch pads the pid/tgid field out to sizeof(long), which forces the alignment of taskstats. The getdelays userspace code is ok with this since it assumes 32-bit pid/tgid and then honors that header's length field.

An array is used to avoid exposing kernel memory contents to userspace in the response.

---
 kernel/taskstats.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/taskstats.c +7 -1
@@ -362,6 +362,12 @@ static struct taskstats *mk_reply(struct
 	struct nlattr *na, *ret;
 	int aggr;
 
+	/* If we don't pad, we end up with alignment on a 4 byte boundary.
+	 * This causes lots of runtime warnings on systems requiring 8 byte
+	 * alignment */
+	u32 pids[2] = { pid, 0 };
+	int pid_size = ALIGN(sizeof(pid), sizeof(long));
+
 	aggr = (type == TASKSTATS_TYPE_PID)
 			? TASKSTATS_TYPE_AGGR_PID
 			: TASKSTATS_TYPE_AGGR_TGID;
@@ -369,7 +375,7 @@ static struct taskstats *mk_reply(struct
 	na = nla_nest_start(skb, aggr);
 	if (!na)
 		goto err;
-	if (nla_put(skb, type, sizeof(pid), &pid) < 0)
+	if (nla_put(skb, type, pid_size, pids) < 0)
 		goto err;
 	ret = nla_reserve(skb, TASKSTATS_TYPE_STATS, sizeof(struct taskstats));
 	if (!ret)
Signature · 2 lines
Jeff Mahoney
SUSE Labs
#1 lore raw
Quoted 11 lines · On Fri, 12 Feb 2010 11:48:27 -0500 Jeff Mahoney <[email protected]> wrote:

prepare_reply sets up an skb for the response. If I understand it correctly, the payload contains:

 +--------------------------------+
 | genlmsghdr - 4 bytes           |
 +--------------------------------+
 | NLA header - 4 bytes           | /* Aggregate header */
 +-+------------------------------+
 | | NLA header - 4 bytes         | /* PID header */
 | +------------------------------+
 | | pid/tgid   - 4 bytes         |

So we put another four zero bytes in here and add four to the "PID header".

Quoted 38 lines
 | +------------------------------+
 | | NLA header - 4 bytes         | /* stats header */
 | + -----------------------------+ <- oops. aligned on 4 byte boundary
 | | struct taskstats - 328 bytes |

+-+------------------------------+

The start of the taskstats struct must be 8 byte aligned on IA64 (and other systems with 8 byte alignment rules for 64-bit types) or runtime alignment warnings will be issued.

This patch pads the pid/tgid field out to sizeof(long), which forces the alignment of taskstats. The getdelays userspace code is ok with this since it assumes 32-bit pid/tgid and then honors that header's length field.

An array is used to avoid exposing kernel memory contents to userspace in the response.

---
 kernel/taskstats.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/taskstats.c +6 -0
@@ -362,6 +362,12 @@ static struct taskstats *mk_reply(struct
 	struct nlattr *na, *ret;
 	int aggr;
 
+	/* If we don't pad, we end up with alignment on a 4 byte boundary.
+	 * This causes lots of runtime warnings on systems requiring 8 byte
+	 * alignment */
+	u32 pids[2] = { pid, 0 };
+	int pid_size = ALIGN(sizeof(pid), sizeof(long));
+
 	aggr = (type == TASKSTATS_TYPE_PID)
 			? TASKSTATS_TYPE_AGGR_PID
 			: TASKSTATS_TYPE_AGGR_TGID;

kernel/taskstats.c static struct taskstats *mk_reply(struct · line 381

Quoted 8 lines
kernel/taskstats.c +1 -1
@@ -369,7 +375,7 @@ static struct taskstats *mk_reply(struct
 	na = nla_nest_start(skb, aggr);
 	if (!na)
 		goto err;
-	if (nla_put(skb, type, sizeof(pid), &pid) < 0)
+	if (nla_put(skb, type, pid_size, pids) < 0)
 		goto err;
 	ret = nla_reserve(skb, TASKSTATS_TYPE_STATS, sizeof(struct taskstats));
 	if (!ret)

So any code which assumes that the pid/tgid field is four bytes long will break. Code which takes that length from the netlink message header will work OK.

32-bit architectures are unaltered.

Seems safe enough. We'd be safer still if we didn't do this on 64-bit architectures which don't need it. ie: x86_64. But if we do that we add a risk that people will develop shoddy code which works on x86_64 and doesn't work on ia64.

hmm.

#2 lore raw
Quoted 68 lines · Andrew Morton
Quoted 11 lines · On Fri, 12 Feb 2010 11:48:27 -0500 Jeff Mahoney <[email protected]> wrote:

prepare_reply sets up an skb for the response. If I understand it correctly, the payload contains:

 +--------------------------------+
 | genlmsghdr - 4 bytes           |
 +--------------------------------+
 | NLA header - 4 bytes           | /* Aggregate header */
 +-+------------------------------+
 | | NLA header - 4 bytes         | /* PID header */
 | +------------------------------+
 | | pid/tgid   - 4 bytes         |

So we put another four zero bytes in here and add four to the "PID header".

Quoted 38 lines
 | +------------------------------+
 | | NLA header - 4 bytes         | /* stats header */
 | + -----------------------------+ <- oops. aligned on 4 byte boundary
 | | struct taskstats - 328 bytes |

+-+------------------------------+

The start of the taskstats struct must be 8 byte aligned on IA64 (and other systems with 8 byte alignment rules for 64-bit types) or runtime alignment warnings will be issued.

This patch pads the pid/tgid field out to sizeof(long), which forces the alignment of taskstats. The getdelays userspace code is ok with this since it assumes 32-bit pid/tgid and then honors that header's length field.

An array is used to avoid exposing kernel memory contents to userspace in the response.

---
 kernel/taskstats.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/taskstats.c +6 -0
@@ -362,6 +362,12 @@ static struct taskstats *mk_reply(struct
 	struct nlattr *na, *ret;
 	int aggr;
 
+	/* If we don't pad, we end up with alignment on a 4 byte boundary.
+	 * This causes lots of runtime warnings on systems requiring 8 byte
+	 * alignment */
+	u32 pids[2] = { pid, 0 };
+	int pid_size = ALIGN(sizeof(pid), sizeof(long));
+
 	aggr = (type == TASKSTATS_TYPE_PID)
 			? TASKSTATS_TYPE_AGGR_PID
 			: TASKSTATS_TYPE_AGGR_TGID;

kernel/taskstats.c static struct taskstats *mk_reply(struct · line 381

Quoted 8 lines
kernel/taskstats.c +1 -1
@@ -369,7 +375,7 @@ static struct taskstats *mk_reply(struct
 	na = nla_nest_start(skb, aggr);
 	if (!na)
 		goto err;
-	if (nla_put(skb, type, sizeof(pid), &pid) < 0)
+	if (nla_put(skb, type, pid_size, pids) < 0)
 		goto err;
 	ret = nla_reserve(skb, TASKSTATS_TYPE_STATS, sizeof(struct taskstats));
 	if (!ret)

So any code which assumes that the pid/tgid field is four bytes long will break. Code which takes that length from the netlink message header will work OK.

Quoted 6 lines

32-bit architectures are unaltered.

Seems safe enough. We'd be safer still if we didn't do this on 64-bit architectures which don't need it. ie: x86_64. But if we do that we add a risk that people will develop shoddy code which works on x86_64 and doesn't work on ia64.

Is there a way to do that without needlessly complicating things? I didn't see any existing infrastructure to do that.

Another option was to put an empty attribute in with a garbage type, which would add a 4 byte header - but even the getdelays code included with the kernel can't deal with that.

It's ugly all the way around.

-Jeff

Signature · 2 lines
Jeff Mahoney
SUSE Labs
#3 lore raw
Quoted 7 lines · On Fri, 12 Feb 2010 14:20:23 -0500 Jeff Mahoney <[email protected]> wrote:
Quoted 4 lines

Seems safe enough. We'd be safer still if we didn't do this on 64-bit architectures which don't need it. ie: x86_64. But if we do that we add a risk that people will develop shoddy code which works on x86_64 and doesn't work on ia64.

Is there a way to do that without needlessly complicating things? I didn't see any existing infrastructure to do that.

ifdefs? I don't think it's worth doing, really. Probably anyone who wrote an application for this copied the getdelays.c code anyway.

#4 lore raw
Quoted 8 lines · Andrew Morton
Quoted 5 lines · On Fri, 12 Feb 2010 14:20:23 -0500 Jeff Mahoney <[email protected]> wrote:
Quoted 4 lines

Seems safe enough. We'd be safer still if we didn't do this on 64-bit architectures which don't need it. ie: x86_64. But if we do that we add a risk that people will develop shoddy code which works on x86_64 and doesn't work on ia64.

Quoted 5 lines
Quoted 2 lines

Is there a way to do that without needlessly complicating things? I didn't see any existing infrastructure to do that.

ifdefs? I don't think it's worth doing, really. Probably anyone who wrote an application for this copied the getdelays.c code anyway.

Yeah I didn't want to get into #if defined(LIST OF ARCHES) for this. I was hoping for a way to get the alignment rules for the arch. In the absence of that, this is good enough. You're probably right about people just lifting the getdelays.c code.

-Jeff

Signature · 2 lines
Jeff Mahoney
SUSE Labs
#5 lore raw
In reply to Andrew Morton
Quoted 15 lines · Andrew Morton
Quoted 11 lines · On Fri, 12 Feb 2010 11:48:27 -0500 Jeff Mahoney <[email protected]> wrote:

prepare_reply sets up an skb for the response. If I understand it correctly, the payload contains:

 +--------------------------------+
 | genlmsghdr - 4 bytes           |
 +--------------------------------+
 | NLA header - 4 bytes           | /* Aggregate header */
 +-+------------------------------+
 | | NLA header - 4 bytes         | /* PID header */
 | +------------------------------+
 | | pid/tgid   - 4 bytes         |
Quoted 2 lines

So we put another four zero bytes in here and add four to the "PID header".

Do you know if these breaks existing applications?

Quoted 6 lines
Quoted 6 lines
 | +------------------------------+
 | | NLA header - 4 bytes         | /* stats header */
 | + -----------------------------+ <- oops. aligned on 4 byte boundary
 | | struct taskstats - 328 bytes |

+-+------------------------------+

Quoted 8 lines
Quoted 8 lines

The start of the taskstats struct must be 8 byte aligned on IA64 (and other systems with 8 byte alignment rules for 64-bit types) or runtime alignment warnings will be issued.

This patch pads the pid/tgid field out to sizeof(long), which forces the alignment of taskstats. The getdelays userspace code is ok with this since it assumes 32-bit pid/tgid and then honors that header's length field.

Could you define OK above? Does the application work without recompiling? Have you checked for endianness issues?

Quoted 14 lines
Quoted 14 lines

An array is used to avoid exposing kernel memory contents to userspace in the response.

---
 kernel/taskstats.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/taskstats.c +0 -0
@@ -362,6 +362,12 @@ static struct taskstats *mk_reply(struct
 	struct nlattr *na, *ret;
 	int aggr;
 
Quoted 4 lines

kernel/taskstats.c static struct taskstats *mk_reply(struct · line 368

Quoted 4 lines
kernel/taskstats.c +4 -0
+	/* If we don't pad, we end up with alignment on a 4 byte boundary.
+	 * This causes lots of runtime warnings on systems requiring 8 byte
+	 * alignment */
+	u32 pids[2] = { pid, 0 };

Shouldn't this be endianness dependent?

Quoted 10 lines
Quoted 10 lines
kernel/taskstats.c +2 -1
+	int pid_size = ALIGN(sizeof(pid), sizeof(long));
+
 	aggr = (type == TASKSTATS_TYPE_PID)
 			? TASKSTATS_TYPE_AGGR_PID
 			: TASKSTATS_TYPE_AGGR_TGID;
@@ -369,7 +375,7 @@ static struct taskstats *mk_reply(struct
 	na = nla_nest_start(skb, aggr);
 	if (!na)
 		goto err;
-	if (nla_put(skb, type, sizeof(pid), &pid) < 0)
Quoted 9 lines

kernel/taskstats.c static struct taskstats *mk_reply(struct · line 381

Quoted 4 lines
kernel/taskstats.c +1 -0
+	if (nla_put(skb, type, pid_size, pids) < 0)
 		goto err;
 	ret = nla_reserve(skb, TASKSTATS_TYPE_STATS, sizeof(struct taskstats));
 	if (!ret)

So any code which assumes that the pid/tgid field is four bytes long will break. Code which takes that length from the netlink message header will work OK.

I think we assume that PID/TGID is 32 bits long, we use the length to extract the rest of the message. We cast NLA_DATA to int* in getdelays.c.

Quoted 7 lines

32-bit architectures are unaltered.

Seems safe enough. We'd be safer still if we didn't do this on 64-bit architectures which don't need it. ie: x86_64. But if we do that we add a risk that people will develop shoddy code which works on x86_64 and doesn't work on ia64.

May be, this deserves an ABI and version bump in taskstats. I'll test this patch soon.

Signature · 2 lines
	Three Cheers,
	Balbir
#6 lore raw
Quoted 17 lines · Balbir Singh
Quoted 15 lines · Andrew Morton
Quoted 11 lines · On Fri, 12 Feb 2010 11:48:27 -0500 Jeff Mahoney <[email protected]> wrote:

prepare_reply sets up an skb for the response. If I understand it correctly, the payload contains:

 +--------------------------------+
 | genlmsghdr - 4 bytes           |
 +--------------------------------+
 | NLA header - 4 bytes           | /* Aggregate header */
 +-+------------------------------+
 | | NLA header - 4 bytes         | /* PID header */
 | +------------------------------+
 | | pid/tgid   - 4 bytes         |
Quoted 4 lines
Quoted 2 lines

So we put another four zero bytes in here and add four to the "PID header".

Do you know if these breaks existing applications?

AFAIK it doesn't. The initial problem was reported by a partner and there have been no reports of breakage since the fix was shipped in a maintenance update for SLE11 in October. I think Andrew's guess about users just copying the getdelays.c code is probably pretty accurate.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
 | +------------------------------+
 | | NLA header - 4 bytes         | /* stats header */
 | + -----------------------------+ <- oops. aligned on 4 byte boundary
 | | struct taskstats - 328 bytes |

+-+------------------------------+

The start of the taskstats struct must be 8 byte aligned on IA64 (and other systems with 8 byte alignment rules for 64-bit types) or runtime alignment warnings will be issued.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

This patch pads the pid/tgid field out to sizeof(long), which forces the alignment of taskstats. The getdelays userspace code is ok with this since it assumes 32-bit pid/tgid and then honors that header's length field.

Could you define OK above? Does the application work without recompiling? Have you checked for endianness issues?

Yes, it seemed to work as before. I just used getdelays.c to test it.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

An array is used to avoid exposing kernel memory contents to userspace in the response.

---
 kernel/taskstats.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/taskstats.c +0 -0
@@ -362,6 +362,12 @@ static struct taskstats *mk_reply(struct
 	struct nlattr *na, *ret;
 	int aggr;
 
Quoted 6 lines
Quoted 4 lines

kernel/taskstats.c static struct taskstats *mk_reply(struct · line 368

Quoted 4 lines
kernel/taskstats.c +4 -0
+	/* If we don't pad, we end up with alignment on a 4 byte boundary.
+	 * This causes lots of runtime warnings on systems requiring 8 byte
+	 * alignment */
+	u32 pids[2] = { pid, 0 };

Shouldn't this be endianness dependent?

It could if the user casts to a 64-bit type instead of a 32-bit type, but that assumption was covered in the comment about the pids and tgids being 32-bit.

Quoted 26 lines
Quoted 10 lines
Quoted 10 lines
kernel/taskstats.c +2 -1
+	int pid_size = ALIGN(sizeof(pid), sizeof(long));
+
 	aggr = (type == TASKSTATS_TYPE_PID)
 			? TASKSTATS_TYPE_AGGR_PID
 			: TASKSTATS_TYPE_AGGR_TGID;
@@ -369,7 +375,7 @@ static struct taskstats *mk_reply(struct
 	na = nla_nest_start(skb, aggr);
 	if (!na)
 		goto err;
-	if (nla_put(skb, type, sizeof(pid), &pid) < 0)
Quoted 9 lines

kernel/taskstats.c static struct taskstats *mk_reply(struct · line 381

Quoted 4 lines
kernel/taskstats.c +1 -0
+	if (nla_put(skb, type, pid_size, pids) < 0)
 		goto err;
 	ret = nla_reserve(skb, TASKSTATS_TYPE_STATS, sizeof(struct taskstats));
 	if (!ret)

So any code which assumes that the pid/tgid field is four bytes long will break. Code which takes that length from the netlink message header will work OK.

I think we assume that PID/TGID is 32 bits long, we use the length to extract the rest of the message. We cast NLA_DATA to int* in getdelays.c.

Quoted 2 lines

32-bit architectures are unaltered.

Quoted 8 lines
Quoted 5 lines

Seems safe enough. We'd be safer still if we didn't do this on 64-bit architectures which don't need it. ie: x86_64. But if we do that we add a risk that people will develop shoddy code which works on x86_64 and doesn't work on ia64.

May be, this deserves an ABI and version bump in taskstats. I'll test this patch soon.

That could be too. Though if an ABI bump is in the works, then it's probably better to not fix it this way and have an empty attribute type instead.

-Jeff

Signature · 2 lines
Jeff Mahoney
SuSE Labs