mdadm: use static major/minor numbers.

Hi!

I recently updated mdadm-2.6.9 to mdadm-3.2.6 on a system (own busybox based distro) that had its kernel version updated from 2.6.35 to 3.4.

Everything works well apart from a feature I used that I seem to be unable to reactivate:

In the past, to ensure knowing the number of possible RAID devices that could be created on the system, I used static major:minor mappings by building the /dev/md/dXX and /dev/md/dXXpXX entries (with major 254, and three minors for each device for 3 partitions), allowing me to be certain that 64 RAID devices could be created at all times.

When creating an array, the /dev/md/dXX and dXXpXX devices "nodes" would be used (not recreated) and the major/minor number these devices point to would be used for the actual MD block device.

This was a very handy feature. With the latest mdadm, I simply cannot do that, as creating a /dev/md/dXX will in fact remove that device file and symlink it to /dev/md_dXX. I then created /dev/md_dXX devices prior to creating an array, and got the following error message: "mdadm:
/dev/md_d3 exists but looks wrong, please fix".

Looking through the source I can see that mdadm basically verifies if the file's major/minor matches the one that it had planned for the device, which in this case doesn't.

Is there any way to work around that? Or in fact, I don't actually *need* to use static major/minor numbers, but I need to know in advance how many RAIDs I'll be able to create provided I'll always use 2 partitions for them and can provide the -amdp2 argument to mdadm to make sure that only 2 partitions devices nodes are created.

Is there a way to know the maximum number of RAIDs that can be created, provided that nothing else uses major 9 and 254?

Thank you very much in advance for your help!

Quoted 34 lines · On Tue, 20 Nov 2012 12:36:52 +0000 Benjamin ESTRABAUD <[email protected]> wrote:

Hi!

I recently updated mdadm-2.6.9 to mdadm-3.2.6 on a system (own busybox based distro) that had its kernel version updated from 2.6.35 to 3.4.

Everything works well apart from a feature I used that I seem to be unable to reactivate:

In the past, to ensure knowing the number of possible RAID devices that could be created on the system, I used static major:minor mappings by building the /dev/md/dXX and /dev/md/dXXpXX entries (with major 254, and three minors for each device for 3 partitions), allowing me to be certain that 64 RAID devices could be created at all times.

When creating an array, the /dev/md/dXX and dXXpXX devices "nodes" would be used (not recreated) and the major/minor number these devices point to would be used for the actual MD block device.

This was a very handy feature. With the latest mdadm, I simply cannot do that, as creating a /dev/md/dXX will in fact remove that device file and symlink it to /dev/md_dXX. I then created /dev/md_dXX devices prior to creating an array, and got the following error message: "mdadm:
/dev/md_d3 exists but looks wrong, please fix".

Looking through the source I can see that mdadm basically verifies if the file's major/minor matches the one that it had planned for the device, which in this case doesn't.

Is there any way to work around that? Or in fact, I don't actually *need* to use static major/minor numbers, but I need to know in advance how many RAIDs I'll be able to create provided I'll always use 2 partitions for them and can provide the -amdp2 argument to mdadm to make sure that only 2 partitions devices nodes are created.

Quoted 4 lines

Is there a way to know the maximum number of RAIDs that can be created, provided that nothing else uses major 9 and 254?

Thank you very much in advance for your help!

I don't really understand what your problem is.

mdadm should create any devices it needs - unless it detects udev, in which case it leaves the device creation to udev.

So simply don't create any md devices in /dev and let mdadm do whatever is required.
Does that approach not suit your needs? If not, why not?

NeilBrown

Quoted 42 lines · NeilBrown

Quoted 34 lines · On Tue, 20 Nov 2012 12:36:52 +0000 Benjamin ESTRABAUD <[email protected]> wrote:

Hi!

I recently updated mdadm-2.6.9 to mdadm-3.2.6 on a system (own busybox based distro) that had its kernel version updated from 2.6.35 to 3.4.

Everything works well apart from a feature I used that I seem to be unable to reactivate:

In the past, to ensure knowing the number of possible RAID devices that could be created on the system, I used static major:minor mappings by building the /dev/md/dXX and /dev/md/dXXpXX entries (with major 254, and three minors for each device for 3 partitions), allowing me to be certain that 64 RAID devices could be created at all times.

When creating an array, the /dev/md/dXX and dXXpXX devices "nodes" would be used (not recreated) and the major/minor number these devices point to would be used for the actual MD block device.

This was a very handy feature. With the latest mdadm, I simply cannot do that, as creating a /dev/md/dXX will in fact remove that device file and symlink it to /dev/md_dXX. I then created /dev/md_dXX devices prior to creating an array, and got the following error message: "mdadm:
/dev/md_d3 exists but looks wrong, please fix".

Looking through the source I can see that mdadm basically verifies if the file's major/minor matches the one that it had planned for the device, which in this case doesn't.

Is there any way to work around that? Or in fact, I don't actually *need* to use static major/minor numbers, but I need to know in advance how many RAIDs I'll be able to create provided I'll always use 2 partitions for them and can provide the -amdp2 argument to mdadm to make sure that only 2 partitions devices nodes are created.

Quoted 4 lines

Is there a way to know the maximum number of RAIDs that can be created, provided that nothing else uses major 9 and 254?

Thank you very much in advance for your help!

I don't really understand what your problem is.

Quoted 8 lines

mdadm should create any devices it needs - unless it detects udev, in which case it leaves the device creation to udev.

So simply don't create any md devices in /dev and let mdadm do whatever is required.
Does that approach not suit your needs? If not, why not?

NeilBrown

Hi Neil,

mdadm creates the devices it needs quite well (there is no udev on this system), but how can I tell how many maximum RAID devices mdadm will be able to create before running out of major/minors, since their allocation is done on the fly?

Will mdadm use all of "md" registered major (9) minor numbers (9:0-254)?

Will it also use the "mdp" registered ones (254) when running out?

I need to know how many MD devices I'll be able to create in advance on that system. Creating the devices beforehand helped with that, since I had md_d0 254:0 up to md_d127 254:252, I knew that I could create a maximum of 128 devices.

What's the maximum number of MD devices that can be created on a udev-less system provided we specify 2 partitions in mdadm --create?

Thank you very much in advance for your help.

Regards,
Ben.

Quoted 58 lines · On Wed, 21 Nov 2012 12:26:00 +0000 Benjamin ESTRABAUD <[email protected]> wrote:

Quoted 42 lines · NeilBrown

Quoted 34 lines · On Tue, 20 Nov 2012 12:36:52 +0000 Benjamin ESTRABAUD <[email protected]> wrote:

Hi!

I recently updated mdadm-2.6.9 to mdadm-3.2.6 on a system (own busybox based distro) that had its kernel version updated from 2.6.35 to 3.4.

Everything works well apart from a feature I used that I seem to be unable to reactivate:

In the past, to ensure knowing the number of possible RAID devices that could be created on the system, I used static major:minor mappings by building the /dev/md/dXX and /dev/md/dXXpXX entries (with major 254, and three minors for each device for 3 partitions), allowing me to be certain that 64 RAID devices could be created at all times.

When creating an array, the /dev/md/dXX and dXXpXX devices "nodes" would be used (not recreated) and the major/minor number these devices point to would be used for the actual MD block device.

This was a very handy feature. With the latest mdadm, I simply cannot do that, as creating a /dev/md/dXX will in fact remove that device file and symlink it to /dev/md_dXX. I then created /dev/md_dXX devices prior to creating an array, and got the following error message: "mdadm:
/dev/md_d3 exists but looks wrong, please fix".

Looking through the source I can see that mdadm basically verifies if the file's major/minor matches the one that it had planned for the device, which in this case doesn't.

Is there any way to work around that? Or in fact, I don't actually *need* to use static major/minor numbers, but I need to know in advance how many RAIDs I'll be able to create provided I'll always use 2 partitions for them and can provide the -amdp2 argument to mdadm to make sure that only 2 partitions devices nodes are created.

Quoted 4 lines

Is there a way to know the maximum number of RAIDs that can be created, provided that nothing else uses major 9 and 254?

Thank you very much in advance for your help!

I don't really understand what your problem is.

Quoted 8 lines

mdadm should create any devices it needs - unless it detects udev, in which case it leaves the device creation to udev.

So simply don't create any md devices in /dev and let mdadm do whatever is required.
Does that approach not suit your needs? If not, why not?

NeilBrown

Hi Neil,

mdadm creates the devices it needs quite well (there is no udev on this system), but how can I tell how many maximum RAID devices mdadm will be able to create before running out of major/minors, since their allocation is done on the fly?

Quoted 1 line

Will mdadm use all of "md" registered major (9) minor numbers (9:0-254)?

Actually I think it is 9:0-4194303. There are 22 bits for minor numbers. And yes, md will use as many as needed (Though I haven't tested millions of arrays - or even hundreds - so there might be bugs).

Quoted 1 line

Will it also use the "mdp" registered ones (254) when running out?

mdp uses 4 bits for the partition, so 18 bits available for arrays. Though you don't really need mdp any more as 'md' devices can now be partitioned.

Quoted 4 lines

I need to know how many MD devices I'll be able to create in advance on that system. Creating the devices beforehand helped with that, since I had md_d0 254:0 up to md_d127 254:252, I knew that I could create a maximum of 128 devices.

Creating the device files in /dev doesn't really have any effect on whether the devices can be created. It is a bit like creating a whole bunch of symlinks to non-existent files, and assuming that means that the files can later be created.

Quoted 2 lines

What's the maximum number of MD devices that can be created on a udev-less system provided we specify 2 partitions in mdadm --create?

millions.

NeilBrown

Quoted 4 lines

Thank you very much in advance for your help.

Regards,
Ben.