Skip to contentSkip to conversation

ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux

PATCH In review Neil Brown 4 participants 9 messages May 26, 2006 – Jun 5, 2006

Subsystems (2)
lore
Share & tools
lore

Neil Brown

author
#0 lore raw

I am pleased to announce the availability of
mdadm version 2.5

It is available at the usual places:
http://www.cse.unsw.edu.au/~neilb/source/mdadm/
and
http://www.{countrycode}.kernel.org/pub/linux/utils/raid/mdadm/

mdadm is a tool for creating, managing and monitoring device arrays using the "md" driver in Linux, also known as Software RAID arrays.

Release 2.5 adds a host of minor updates and one major update.

The major update involves an "Auto Assemble" function which will, with certainly limits, scan all available devices for anything that looks like an md array, and will try to assemble it. This code should be treated with some caution as it is very new, and could be revised in future, though hopefully not very much.

The main problem I have always had will auto-assembly is that it is too easy for it to assemble thing that you don't want assembled, e.g. if you have moved some devices from a different computer. To remove this problem, arrays can now be tagged for the computer that is their home (homehost) and only arrays with the correct homehost will be auto-assembled.

Feedback on the effectiveness and usefulness of this feature and it's documentation is encouraged.

Changelog Entries:
- Support 'mailfrom' line in mdadm.conf so the From: line in alert
emails can be explicitly set.
- Arrange that SparesMissing (which is similar in import to
DegradedArray) generates an Email.
- Assume "DEVICE partitions" if no DEVICE line is given.
- Support new 'offset' layout for raid10.
- When creating a bitmap file, choose a chunksize to limit number
of bitmap chunks to 2 million. More than this can cause kmalloc failure.
- New 'CREATE' line in mdadm.conf for defaults such as owner, group,
mode and auto-flag
- --detail checks if array has been started or not and includes that
in report.
- When using --update=uuid on an array with a bitmap, update the
bitmap's uuid too.
- Add a copy of /proc/mdstat to the mail message sent by mdadm
--monitor.
- New flag --no-degraded to avoid starting arrays if there are
fewer devices available than last time the array was started. This is only needed with --scan, as with --scan, that behaviour is the default.
- Support for 'homehost' concept. This is a fairly major update.
It includes a configfile option and a command line option for specifying a homehost, records that host in the superblock, and reports the homehost where possible.
- Support for Auto Assembly. "mdadm -As" will, if provided with
the name of a homehost, try to assemble all arrays it can find that were created for that homehost. See man pages for more details.

Development of mdadm is sponsored by
SUSE Labs, Novell Inc.

NeilBrown 26th May 2006

#1 lore raw

PATCH 1/1 mdadm 2.5 (Was: ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux)

Quoted 5 lines · Neil Brown

I am pleased to announce the availability of
mdadm version 2.5

hello,
i tried rebuilding mdadm 2.5 on current mandriva cooker, which uses gcc-4.1.1, glibc-2.4 and dietlibc 0.29 and found the following issues addressed by patches attacched to this message
I would be glad if you could review these patches and include them in upcoming mdadm releases.

- mdadm-2.3.1-kernel-byteswap-include-fix.patch
reverts a change introduced with mdadm 2.3.1 for redhat compatibility asm/byteorder.h is an architecture dependent file and does more stuff than a call to the linux/byteorder/XXX_endian.h
the fact that not calling asm/byteorder.h does not define __BYTEORDER_HAS_U64__ is just an example of issues that might arise. if redhat is broken it should be worked around differently than breaking mdadm.

- mdadm-2.4-snprintf.patch
this is self commenting, just an error in the snprintf call

- mdadm-2.4-strict-aliasing.patch
fix for another srict-aliasing problem, you can typecast a reference to a void pointer to anything, you cannot typecast a reference to a struct.

- mdadm-2.5-mdassemble.patch
pass CFLAGS to mdassemble build, enabling -Wall -Werror showed some issues also fixed by the patch.

- mdadm-2.5-rand.patch
Posix dictates rand() versus bsd random() function, and dietlibc deprecated random(), so switch to srand()/rand() and make everybody happy.

- mdadm-2.5-unused.patch
glibc 2.4 is pedantic on ignoring return values from fprintf, fwrite and write, so now we check the rval and actually do something with it. in the Grow.c case i only print a warning, since i don't think we can do anithing in case we fail invalidating those superblocks (is should never happen, but then...)

Regards,
L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \

* Sat Feb 18 2006 Christiaan Welvaart <[email protected]> not including asm/byteorder.h will not define __BYTEORDER_HAS_U64__ causing __fswab64 to be undefined and failure compiling mdadm on big_endian architectures like PPC

mdadm.h renamed from mdadm.h.bak +1 -10
@@ -72,16 +72,7 @@
 #include	"bitmap.h"
 
 #include <endian.h>
-/* #include "asm/byteorder.h" Redhat don't like this so... */
-#if __BYTE_ORDER == __LITTLE_ENDIAN
-#  include <linux/byteorder/little_endian.h>
-#elif __BYTE_ORDER == __BIG_ENDIAN
-#  include <linux/byteorder/big_endian.h>
-#elif __BYTE_ORDER == __PDP_ENDIAN
-#  include <linux/byteorder/pdp_endian.h>
-#else
-#  error "unknown endianness."
-#endif
+#include <asm/byteorder.h>
 
 
 

* Sat May 27 2006 Luca Berra <[email protected]> snprintf size should be at most the size of the buffer

util.c renamed from util.c.snprintf +1 -1
@@ -439,7 +439,7 @@
 		}
 	if (create && !std && !nonstd) {
 		static char buf[30];
-		snprintf(buf, 1024, "%d:%d", major, minor);
+		snprintf(buf, 30, "%d:%d", major, minor);
 		nonstd = buf;
 	}
 

* Sat May 27 2006 Luca Berra <[email protected]>
This is to avoid gcc warnings when building with strict-aliasing optimization

dlink.h renamed from dlink.h.alias +4 -4
@@ -4,16 +4,16 @@
 
 struct __dl_head
 {
-    struct __dl_head *	dh_prev;
-    struct __dl_head *	dh_next;
+    void * dh_prev;
+    void * dh_next;
 };
 
 #define	dl_alloc(size)	((void*)(((char*)calloc(1,(size)+sizeof(struct __dl_head)))+sizeof(struct __dl_head)))
 #define	dl_new(t)	((t*)dl_alloc(sizeof(t)))
 #define	dl_newv(t,n)	((t*)dl_alloc(sizeof(t)*n))
 
-#define dl_next(p) *((void**)&(((struct __dl_head*)(p))[-1].dh_next))
-#define dl_prev(p) *((void**)&(((struct __dl_head*)(p))[-1].dh_prev))
+#define dl_next(p) *(&(((struct __dl_head*)(p))[-1].dh_next))
+#define dl_prev(p) *(&(((struct __dl_head*)(p))[-1].dh_prev))
 
 void *dl_head(void);
 char *dl_strdup(char *);

* Sat May 27 2006 Luca Berra <[email protected]>
add CFLAGS to mdassemble build and fix a couple of non-returning functions

mdadm.h renamed from mdadm.h.bluca +2 -4
@@ -44,10 +44,8 @@
 #include	<errno.h>
 #include	<string.h>
 #include	<syslog.h>
-#ifdef __dietlibc__NONO
-int strncmp(const char *s1, const char *s2, size_t n) __THROW __pure__;
-char *strncpy(char *dest, const char *src, size_t n) __THROW;
-#include    <strings.h>
+#ifdef __dietlibc__
+#include	<strings.h>
 #endif
 
 
mdassemble.c renamed from mdassemble.c.bluca +3 -2
@@ -54,7 +54,7 @@
 };
 
 #ifndef MDASSEMBLE_AUTO
-/* from mdadm.c */
+/* from mdopen.c */
 int open_mddev(char *dev, int autof/*unused */)
 {
 	int mdfd = open(dev, O_RDWR, 0);
@@ -79,7 +79,7 @@
 int verbose = 0;
 int force = 0;
 
-int main() {
+int main(int argc, char *argv[]) {
 	mddev_ident_t array_list =  conf_get_ident(configfile, NULL);
 	if (!array_list) {
 		fprintf(stderr, Name ": No arrays found in config file\n");
@@ -100,4 +100,5 @@
 					   NULL, NULL,
 					   readonly, runstop, NULL, NULL, verbose, force);
 		}
+	return rv;
 }
Makefile renamed from Makefile.bluca +1 -1
@@ -73,7 +73,7 @@
 	mdopen.c super0.c super1.c bitmap.c restripe.c sysfs.c
 
 ASSEMBLE_SRCS := mdassemble.c Assemble.c config.c dlink.c util.c super0.c super1.c
-ASSEMBLE_FLAGS:= -DMDASSEMBLE
+ASSEMBLE_FLAGS:= $(CFLAGS) -DMDASSEMBLE
 ifdef MDASSEMBLE_AUTO
 ASSEMBLE_SRCS += mdopen.c mdstat.c
 ASSEMBLE_FLAGS += -DMDASSEMBLE_AUTO
util.c renamed from util.c.bluca +1 -1
@@ -375,7 +375,7 @@
 }
 int nftw(const char *path, int (*han)(const char *name, const struct stat *stb, int flag, struct FTW *s), int nopenfd, int flags)
 {
-	ftw(path, add_dev_1, nopenfd);
+	return ftw(path, add_dev_1, nopenfd);
 }
 #endif
 

* Sat May 27 2006 Luca Berra <[email protected]> POSIX dictates rand/srand instead of BSD srandom/random

Assemble.c renamed from Assemble.c.rand +4 -4
@@ -412,10 +412,10 @@
 				int rfd;
 				if ((rfd = open("/dev/urandom", O_RDONLY)) < 0 ||
 				    read(rfd, ident->uuid, 16) != 16) {
-					*(__u32*)(ident->uuid) = random();
-					*(__u32*)(ident->uuid+1) = random();
-					*(__u32*)(ident->uuid+2) = random();
-					*(__u32*)(ident->uuid+3) = random();
+					*(__u32*)(ident->uuid) = rand();
+					*(__u32*)(ident->uuid+1) = rand();
+					*(__u32*)(ident->uuid+2) = rand();
+					*(__u32*)(ident->uuid+3) = rand();
 				}
 				if (rfd >= 0) close(rfd);
 			}
mdadm.c renamed from mdadm.c.rand +1 -1
@@ -107,7 +107,7 @@
 
 	int mdfd = -1;
 
-	srandom(time(0) ^ getpid());
+	srand(time(0) ^ getpid());
 
 	ident.uuid_set=0;
 	ident.level = UnSet;
super0.c renamed from super0.c.rand +4 -4
@@ -539,7 +539,7 @@
 	sb->patch_version = 0;
 	sb->gvalid_words = 0; /* ignored */
 	if (rfd < 0 || read(rfd, &sb->set_uuid0, 4) != 4)
-		sb->set_uuid0 = random();
+		sb->set_uuid0 = rand();
 	sb->ctime = time(0);
 	sb->level = info->level;
 	if (size != info->size)
@@ -550,9 +550,9 @@
 	sb->md_minor = info->md_minor;
 	sb->not_persistent = 0;
 	if (rfd < 0 || read(rfd, &sb->set_uuid1, 12) != 12) {
-		sb->set_uuid1 = random();
-		sb->set_uuid2 = random();
-		sb->set_uuid3 = random();
+		sb->set_uuid1 = rand();
+		sb->set_uuid2 = rand();
+		sb->set_uuid3 = rand();
 	}
 	if (rfd >= 0)
 		close(rfd);
super1.c renamed from super1.c.rand +8 -8
@@ -583,10 +583,10 @@
 
 	if ((rfd = open("/dev/urandom", O_RDONLY)) < 0 ||
 	    read(rfd, sb->set_uuid, 16) != 16) {
-		*(__u32*)(sb->set_uuid) = random();
-		*(__u32*)(sb->set_uuid+4) = random();
-		*(__u32*)(sb->set_uuid+8) = random();
-		*(__u32*)(sb->set_uuid+12) = random();
+		*(__u32*)(sb->set_uuid) = rand();
+		*(__u32*)(sb->set_uuid+4) = rand();
+		*(__u32*)(sb->set_uuid+8) = rand();
+		*(__u32*)(sb->set_uuid+12) = rand();
 	}
 	if (rfd >= 0) close(rfd);
 
@@ -751,10 +751,10 @@
 
 	if ((rfd = open("/dev/urandom", O_RDONLY)) < 0 ||
 	    read(rfd, sb->device_uuid, 16) != 16) {
-		*(__u32*)(sb->device_uuid) = random();
-		*(__u32*)(sb->device_uuid+4) = random();
-		*(__u32*)(sb->device_uuid+8) = random();
-		*(__u32*)(sb->device_uuid+12) = random();
+		*(__u32*)(sb->device_uuid) = rand();
+		*(__u32*)(sb->device_uuid+4) = rand();
+		*(__u32*)(sb->device_uuid+8) = rand();
+		*(__u32*)(sb->device_uuid+12) = rand();
 	}
 	if (rfd >= 0) close(rfd);
 	sb->events = 0;

* Sat May 27 2006 Luca Berra <[email protected]>
check return status of all write/fwrite functions as required by glibc 2.4

Monitor.c renamed from Monitor.c.unused +1 -1
@@ -521,7 +521,7 @@
 				int n;
 				fprintf(mp, "\nP.S. The /proc/mdstat file current contains the following:\n\n");
 				while ( (n=fread(buf, 1, sizeof(buf), mdstat)) > 0)
-					fwrite(buf, 1, n, mp);
+					n=fwrite(buf, 1, n, mp); /* yes, i don't care about the result */
 				fclose(mdstat);
 			}
 			fclose(mp);
super1.c renamed from super1.c.unused +2 -1
@@ -715,7 +715,8 @@
 			(((char*)sb)+1024);
 		if (__le32_to_cpu(bm->magic) == BITMAP_MAGIC) {
 			locate_bitmap1(st, fd, sbv);
-			write(fd, bm, sizeof(*bm));
+			if (write(fd, bm, sizeof(*bm)) != sizeof(*bm))
+			    return 5;
 		}
 	}
 	fsync(fd);
super0.c renamed from super0.c.unused +2 -1
@@ -625,7 +625,8 @@
 	if (super->state & (1<<MD_SB_BITMAP_PRESENT)) {
 		struct bitmap_super_s * bm = (struct bitmap_super_s*)(super+1);
 		if (__le32_to_cpu(bm->magic) == BITMAP_MAGIC)
-			write(fd, bm, sizeof(*bm));
+			if (write(fd, bm, sizeof(*bm)) != sizeof(*bm))
+			    return 5;
 	}
 
 	fsync(fd);
Grow.c renamed from Grow.c.unused +4 -1
@@ -801,7 +801,10 @@
 		memset(&bsb, 0, sizeof(bsb));
 		for (i=odisks; i<d ; i++) {
 			lseek64(fdlist[i], (offsets[i]+last_block)<<9, 0);
-			write(fdlist[i], &bsb, sizeof(bsb));
+			if (write(fdlist[i], &bsb, sizeof(bsb)) < 0) {
+				fprintf(stderr, Name ": %s: failed to invalidate metadata for raid disk %d\n",
+					devname, i);
+			}
 		}
 
 		/* unsuspend. */
bitmap.c renamed from bitmap.c.unused +12 -6
@@ -399,16 +399,22 @@
 	return rv;
 }
 
-void bitmap_update_uuid(int fd, int *uuid)
+int bitmap_update_uuid(int fd, int *uuid)
 {
 	struct bitmap_super_s bm;
-	lseek(fd, 0, 0);
+	if (lseek(fd, 0, 0) != 0)
+		return 1;
 	if (read(fd, &bm, sizeof(bm)) != sizeof(bm))
-		return;
+		return 1;
 	if (bm.magic != __cpu_to_le32(BITMAP_MAGIC))
-		return;
+		return 1;
 	memcpy(bm.uuid, uuid, 16);
+	if (lseek(fd, 0, 0) != 0)
+		return 2;
+	if (write(fd, &bm, sizeof(bm)) != sizeof(bm)) {
+		lseek(fd, 0, 0);
+		return 2;
+	}
 	lseek(fd, 0, 0);
-	write(fd, &bm, sizeof(bm));
-	lseek(fd, 0, 0);
+	return 0;
 }
mdadm.h renamed from mdadm.h.unused +1 -1
@@ -371,7 +371,7 @@
 			unsigned long long array_size,
 			int major);
 extern int ExamineBitmap(char *filename, int brief, struct supertype *st);
-extern void bitmap_update_uuid(int fd, int *uuid);
+extern int bitmap_update_uuid(int fd, int *uuid);
 
 extern int md_get_version(int fd);
 extern int get_linux_version(void);
Assemble.c renamed from Assemble.c.unused +3 -1
@@ -451,7 +451,9 @@
 
 			if (strcmp(update, "uuid")==0 &&
 			    ident->bitmap_fd)
-				bitmap_update_uuid(ident->bitmap_fd, info.uuid);
+				if (bitmap_update_uuid(ident->bitmap_fd, info.uuid) != 0)
+					fprintf(stderr, Name ": Could not update uuid on %s.\n",
+						devname);
 		} else
 #endif
 		{
#2 lore raw

Re: [PATCH] mdadm 2.5 (Was: ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux)

Quoted 4 lines · Luca Berra

- mdadm-2.5-rand.patch
Posix dictates rand() versus bsd random() function, and dietlibc deprecated random(), so switch to srand()/rand() and make everybody happy.

fwiw... lots of rand()s tend to suck... and RAND_MAX may not be large enough for this use. glibc rand() is the same as random(). do you know if dietlibc's rand() is good enough?

-dean

#3 lore raw

Re: [PATCH] mdadm 2.5 (Was: ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux)

Quoted 8 lines · dean gaudet
Quoted 4 lines · Luca Berra

- mdadm-2.5-rand.patch
Posix dictates rand() versus bsd random() function, and dietlibc deprecated random(), so switch to srand()/rand() and make everybody happy.

fwiw... lots of rand()s tend to suck... and RAND_MAX may not be large enough for this use. glibc rand() is the same as random(). do you know

the fact that glibc rand() is the same implementation as random() was one of the reason i believe we could switch to rand()

Quoted 1 line

if dietlibc's rand() is good enough?

dietlibc rand() and random() are the same function. but random will throw a warning saying it is deprecated.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#4 lore raw

Re: [PATCH] mdadm 2.5 (Was: ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux)

Quoted 2 lines · Luca Berra

dietlibc rand() and random() are the same function. but random will throw a warning saying it is deprecated.

that's terribly obnoxious... it's never going to be deprecated, there are only approximately a bazillion programs using random().

-dean

author
#5 lore raw

Re: [PATCH] mdadm 2.5 (Was: ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux)

In reply to Luca Berra
Quoted 7 lines · On Sunday May 28, [email protected] wrote:
Quoted 5 lines · Neil Brown

I am pleased to announce the availability of
mdadm version 2.5

Quoted 6 lines

hello,
i tried rebuilding mdadm 2.5 on current mandriva cooker, which uses gcc-4.1.1, glibc-2.4 and dietlibc 0.29 and found the following issues addressed by patches attacched to this message
I would be glad if you could review these patches and include them in upcoming mdadm releases.

Thanks for the patches. They are greatly appreciated.

Quoted 9 lines

- mdadm-2.3.1-kernel-byteswap-include-fix.patch
reverts a change introduced with mdadm 2.3.1 for redhat compatibility asm/byteorder.h is an architecture dependent file and does more stuff than a call to the linux/byteorder/XXX_endian.h
the fact that not calling asm/byteorder.h does not define __BYTEORDER_HAS_U64__ is just an example of issues that might arise. if redhat is broken it should be worked around differently than breaking mdadm.

I don't understand the problem here. What exactly breaks with the code currently in 2.5? mdadm doesn't need __BYTEORDER_HAS_U64__, so why does not having id defined break anything?
The coomment from the patch says:

Quoted 3 lines

not including asm/byteorder.h will not define __BYTEORDER_HAS_U64__ causing __fswab64 to be undefined and failure compiling mdadm on big_endian architectures like PPC

But again, mdadm doesn't use __fswab64 ....
More details please.

Quoted 3 lines

- mdadm-2.4-snprintf.patch
this is self commenting, just an error in the snprintf call

I wonder how that snuck in...
There was an odd extra tab in the patch, but no-matter. I changed it to use 'sizeof(buf)' to be consistent with other uses of snprint. Thanks.

Quoted 5 lines

- mdadm-2.4-strict-aliasing.patch
fix for another srict-aliasing problem, you can typecast a reference to a void pointer to anything, you cannot typecast a reference to a struct.

Why can't I typecast a reference to a struct??? It seems very unfair...
However I have no problem with the patch. Applied. Thanks. I should really change it to use 'list.h' type lists from the linux kernel.

Quoted 4 lines

- mdadm-2.5-mdassemble.patch
pass CFLAGS to mdassemble build, enabling -Wall -Werror showed some issues also fixed by the patch.

yep, thanks.

Quoted 5 lines

- mdadm-2.5-rand.patch
Posix dictates rand() versus bsd random() function, and dietlibc deprecated random(), so switch to srand()/rand() and make everybody happy.

Everybody?
'man 3 rand' tells me:

Do not use this function in applications intended to be portable when good randomness is needed.

Admittedly mdadm doesn't need to be portable - it only needs to run on Linux. But this line in the man page bothers me.

I guess
-Drandom=rand -Dsrandom=srand
might work.... no. stdlib.h doesn't like that. 'random' returns 'long int' while rand returns 'int'. Interestingly 'random_r' returns 'int' as does 'rand_r'.

#ifdef __dietlibc__
#include <strings.h>
/* dietlibc has deprecated random and srandom!! */ #define random rand
#define srandom srand
#endif

in mdadm.h. Will that do you?

Quoted 7 lines

- mdadm-2.5-unused.patch
glibc 2.4 is pedantic on ignoring return values from fprintf, fwrite and write, so now we check the rval and actually do something with it. in the Grow.c case i only print a warning, since i don't think we can do anithing in case we fail invalidating those superblocks (is should never happen, but then...)

Ok, thanks.

You can see these patches at
http://neil.brown.name/cgi-bin/gitweb.cgi?p=mdadm

more welcome :-)

Thanks again,
NeilBrown

#6 lore raw

Re: [PATCH] mdadm 2.5 (Was: ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux)

Quoted 2 lines · Neil Brown

On Sunday May 28, [email protected] wrote:
Thanks for the patches. They are greatly appreciated.

You're welcome

Quoted 14 lines
Quoted 9 lines

- mdadm-2.3.1-kernel-byteswap-include-fix.patch
reverts a change introduced with mdadm 2.3.1 for redhat compatibility asm/byteorder.h is an architecture dependent file and does more stuff than a call to the linux/byteorder/XXX_endian.h
the fact that not calling asm/byteorder.h does not define __BYTEORDER_HAS_U64__ is just an example of issues that might arise. if redhat is broken it should be worked around differently than breaking mdadm.

I don't understand the problem here. What exactly breaks with the code currently in 2.5? mdadm doesn't need __BYTEORDER_HAS_U64__, so why does not having id defined break anything?
The coomment from the patch says:

Quoted 6 lines
Quoted 3 lines

not including asm/byteorder.h will not define __BYTEORDER_HAS_U64__ causing __fswab64 to be undefined and failure compiling mdadm on big_endian architectures like PPC

But again, mdadm doesn't use __fswab64 ....
More details please.

you use __cpu_to_le64 (ie in super0.c line 987)

       bms->sync_size = __cpu_to_le64(size);

which in byteorder/big_endian.h is defined as

#define __cpu_to_le64(x) ((__force __le64)__swab64((x)))

but __swab64 is defined in byteorder/swab.h (included by byteorder/big_endian.h) as

#if defined(__GNUC__) && (__GNUC__ >= 2) && defined(__OPTIMIZE__) # define __swab64(x) \
(__builtin_constant_p((__u64)(x)) ? \
___swab64((x)) : \
__fswab64((x)))
#else
# define __swab64(x) __fswab64(x)
#endif /* OPTIMIZE */

and __fswab64 is defined further into byteorder/swab.h as

#ifdef __BYTEORDER_HAS_U64__
static __inline__ __attribute_const__ __u64 __fswab64(__u64 x)
.....
#endif /* __BYTEORDER_HAS_U64__ */

so building mdadm on a ppc (or i suppose a sparc) will break

now, if you look at /usr/src/linux/asm-*/byteorder.h you will notice they are very different files, this makes me believe it is not a good idea bypassing asm/byteorder.h
And no, just defining __BYTEORDER_HAS_U64__ will break on 32bit big-endian cpus (and if i do not misread it might just compile and give incorrect results)

Quoted 8 lines
Quoted 3 lines

- mdadm-2.4-snprintf.patch
this is self commenting, just an error in the snprintf call

I wonder how that snuck in...
There was an odd extra tab in the patch, but no-matter. I changed it to use 'sizeof(buf)' to be consistent with other uses of snprint. Thanks.

yes, that would be better.

Quoted 7 lines
Quoted 4 lines

- mdadm-2.4-strict-aliasing.patch
fix for another srict-aliasing problem, you can typecast a reference to a void pointer to anything, you cannot typecast a reference to a struct.

Why can't I typecast a reference to a struct??? It seems very unfair...

that's strict-aliasing optimization for you, i do agree it _is_ unfair.

Quoted 3 lines

However I have no problem with the patch. Applied. Thanks. I should really change it to use 'list.h' type lists from the linux kernel.

hopefull redhat would not object :)

Quoted 27 lines
Quoted 3 lines

- mdadm-2.5-mdassemble.patch
pass CFLAGS to mdassemble build, enabling -Wall -Werror showed some issues also fixed by the patch.

yep, thanks.

Quoted 5 lines

- mdadm-2.5-rand.patch
Posix dictates rand() versus bsd random() function, and dietlibc deprecated random(), so switch to srand()/rand() and make everybody happy.

Everybody?
'man 3 rand' tells me:

Do not use this function in applications intended to be portable when good randomness is needed.

Admittedly mdadm doesn't need to be portable - it only needs to run on Linux. But this line in the man page bothers me.

I guess
-Drandom=rand -Dsrandom=srand
might work.... no. stdlib.h doesn't like that. 'random' returns 'long int' while rand returns 'int'. Interestingly 'random_r' returns 'int' as does 'rand_r'.

Quoted 9 lines

#ifdef __dietlibc__
#include <strings.h>
/* dietlibc has deprecated random and srandom!! */ #define random rand
#define srandom srand
#endif

in mdadm.h. Will that do you?

yes, mdassemble will build, so it is ok for me.

Quoted 8 lines
Quoted 7 lines

- mdadm-2.5-unused.patch
glibc 2.4 is pedantic on ignoring return values from fprintf, fwrite and write, so now we check the rval and actually do something with it. in the Grow.c case i only print a warning, since i don't think we can do anithing in case we fail invalidating those superblocks (is should never happen, but then...)

Quoted 5 lines

Ok, thanks.

You can see these patches at
http://neil.brown.name/cgi-bin/gitweb.cgi?p=mdadm

Thanks.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
author
#7 lore raw

Re: [PATCH] mdadm 2.5 (Was: ANNOUNCE: mdadm 2.5 - A tool for managing Soft RAID under Linux)

Quoted 35 lines · On Monday May 29, [email protected] wrote:
Quoted 2 lines · Neil Brown

On Sunday May 28, [email protected] wrote:
Thanks for the patches. They are greatly appreciated.

You're welcome

Quoted 14 lines
Quoted 9 lines

- mdadm-2.3.1-kernel-byteswap-include-fix.patch
reverts a change introduced with mdadm 2.3.1 for redhat compatibility asm/byteorder.h is an architecture dependent file and does more stuff than a call to the linux/byteorder/XXX_endian.h
the fact that not calling asm/byteorder.h does not define __BYTEORDER_HAS_U64__ is just an example of issues that might arise. if redhat is broken it should be worked around differently than breaking mdadm.

I don't understand the problem here. What exactly breaks with the code currently in 2.5? mdadm doesn't need __BYTEORDER_HAS_U64__, so why does not having id defined break anything?
The coomment from the patch says:

Quoted 6 lines
Quoted 3 lines

not including asm/byteorder.h will not define __BYTEORDER_HAS_U64__ causing __fswab64 to be undefined and failure compiling mdadm on big_endian architectures like PPC

But again, mdadm doesn't use __fswab64 ....
More details please.

you use __cpu_to_le64 (ie in super0.c line 987)

        bms->sync_size = __cpu_to_le64(size);

which in byteorder/big_endian.h is defined as

#define __cpu_to_le64(x) ((__force __le64)__swab64((x)))

but __swab64 is defined in byteorder/swab.h (included by byteorder/big_endian.h) as

Quoted 8 lines

#if defined(__GNUC__) && (__GNUC__ >= 2) && defined(__OPTIMIZE__) # define __swab64(x) \
(__builtin_constant_p((__u64)(x)) ? \
___swab64((x)) : \
__fswab64((x)))
#else
# define __swab64(x) __fswab64(x)
#endif /* OPTIMIZE */

Grrr......

Thanks for the details. I think I'll just give up and do it myself. e.g.
short swap16(short in)

{
	int i;
	short out=0;
	for (i=0; i<4; i++) {
		out = out<<8 | (in&255);
		in = in >> 8;
	}
	return out;
}

I don't need top performance and at least this should be portable...

NeilBrown

#8 lore raw

Re: [PATCH] mdadm 2.5

In reply to Neil Brown

On 29 May 2006, Neil Brown suggested tentatively:

Quoted 8 lines · On Sunday May 28, [email protected] wrote:
Quoted 4 lines

- mdadm-2.4-strict-aliasing.patch
fix for another srict-aliasing problem, you can typecast a reference to a void pointer to anything, you cannot typecast a reference to a struct.

Why can't I typecast a reference to a struct??? It seems very unfair...

ISO C forbids it. Pretty much the only time you're allowed to cast a pointer to A to a pointer to B is if one or the other of them is a pointer to void or char.

The compiler can (and does) assume that all other pointers-to-T are in different `aliasing sets', i.e. that a modification through a pointer-to-T1 cannot affect variables of any other type (so they can be cached in registers, moved around without reference to each other, and so on).

It's *very* advantageous, and with the interprocedural optimizations in GCC 4.1+ is getting more so all the time (the compiler can now move code between *functions* if it needs to, or propagate constants consistently used as function arguments down int into the function themselves; pointer aliasing torpedoes both of these optimizations, and many more.)

(One particularly evil thing is the glibc printf handler extension. As a result of that, we must assume that any call to printf() or scanf() can potentially modify any global variable and any local to which a pointer may have escaped. This is decidedly suboptimal given that nobody sane ever uses those extensions!)

I think I'll dare to upgrade mdadm soon. I tried upgrading from 2.3.1 to 2.4 (with uClibc from SVN dated 2006-03-28) and it promptly segfaulted at array assembly time. If it happens with 2.5 I'll get a backtrace (hard though that is to get when you don't have any accessible storage to put the core dump on... I upgraded uClibc at the same time so it may well be an uClibc bug.)

Signature · 2 lines
`Voting for any American political party is fundamentally
 incomprehensible.' --- Vadik