About btrfs qgroup import/export command

Hi, everyone.

As we know, there is no backup function for qgroup. when the problem occurs, the users must recover qgroup configuration manually, it is not convenient. And besides that, some users might want to import an existed qgroup configuration into a new filesystem. Btrfs does not have such a function,it can only be done manually.

So we want to implement btrfs qgroup import/export commands. 1)'btrfs qgroup export' commands will export qgroup tree
into a user's specified file.(stdout by default)

2)user may modify the configuration file firstly and then
import it into the filesystem.(by 'btrfs qgroup import' command)

The file may be formated as the following:

Qgroupid is_compressed is_exclusive limited_size parent
----------------------------------------------------------------------

 0/1            0                 0              10G            1/0
 1/0            1                 1              20G            ---

If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, else max referenced size. Here 'parent' exclude ancestral qgroups.

Is there any comment about this idea?

Thanks
Miao

Quoted 25 lines · Miao Xie

Hi, everyone.

As we know, there is no backup function for qgroup. when the problem occurs, the users must recover qgroup configuration manually, it is not convenient. And besides that, some users might want to import an existed qgroup configuration into a new filesystem. Btrfs does not have such a function,it can only be done manually.

So we want to implement btrfs qgroup import/export commands. 1)'btrfs qgroup export' commands will export qgroup tree
into a user's specified file.(stdout by default)

2)user may modify the configuration file firstly and then
import it into the filesystem.(by 'btrfs qgroup import' command)

The file may be formated as the following:

Qgroupid is_compressed is_exclusive limited_size parent
----------------------------------------------------------------------

 0/1            0                 0              10G            1/0
 1/0            1                 1              20G            ---

If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, else max referenced size. Here 'parent' exclude ancestral qgroups.

Quoted 1 line

Is there any comment about this idea?

The configuration only really makes sense in combination with the existing subvolumes. Even if the target has subvolumes under the same name, they might have different internal IDs. So it might make more sense to address the level 0 qgroups by name.

Also it might be misleading to apply a configuration to an existing fs, as it currently is not possible get a correct accounting if the fs is not empty. Rescan is not yet implemented.
So instead of just saving and restoring the qgroup config, it might make more sense to create a new filesystem including all subvolumes and quota config from a config file.
But, I'm not completely convinced that this is a features that is needed frequently. If I want a standard deployment, I simple write a script that creates the fs + subvol + quota.

Quoted 23 lines · Miao Xie

As we know, there is no backup function for qgroup. when the problem occurs, the users must recover qgroup configuration manually, it is not convenient. And besides that, some users might want to import an existed qgroup configuration into a new filesystem. Btrfs does not have such a function,it can only be done manually.

So we want to implement btrfs qgroup import/export commands. 1)'btrfs qgroup export' commands will export qgroup tree
into a user's specified file.(stdout by default)

2)user may modify the configuration file firstly and then
import it into the filesystem.(by 'btrfs qgroup import' command)

The file may be formated as the following:

Qgroupid is_compressed is_exclusive limited_size parent
----------------------------------------------------------------------

 0/1            0                 0              10G            1/0
 1/0            1                 1              20G            ---

If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, else max referenced size. Here 'parent' exclude ancestral qgroups.

Quoted 1 line

Is there any comment about this idea?

The configuration only really makes sense in combination with the existing subvolumes. Even if the target has subvolumes under the same name, they might have different internal IDs. So it might make more sense to address the level 0 qgroups by name.

Also it might be misleading to apply a configuration to an existing fs, as it currently is not possible get a correct accounting if the fs is not empty. Rescan is not yet implemented.

So instead of just saving and restoring the qgroup config, it might make more sense to create a new filesystem including all subvolumes and quota config from a config file.
But, I'm not completely convinced that this is a features that is needed frequently. If I want a standard deployment, I simple write a script that creates the fs + subvol + quota.

Hi, Arne

Quoted 24 lines · Arne Jansen

Quoted 23 lines · Miao Xie

As we know, there is no backup function for qgroup. when the problem occurs, the users must recover qgroup configuration manually, it is not convenient. And besides that, some users might want to import an existed qgroup configuration into a new filesystem. Btrfs does not have such a function,it can only be done manually.

So we want to implement btrfs qgroup import/export commands. 1)'btrfs qgroup export' commands will export qgroup tree
into a user's specified file.(stdout by default)

2)user may modify the configuration file firstly and then
import it into the filesystem.(by 'btrfs qgroup import' command)

The file may be formated as the following:

Qgroupid is_compressed is_exclusive limited_size parent
----------------------------------------------------------------------

 0/1            0                 0              10G            1/0
 1/0            1                 1              20G            ---

If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, else max referenced size. Here 'parent' exclude ancestral qgroups.

Quoted 6 lines

Quoted 1 line

Is there any comment about this idea?

The configuration only really makes sense in combination with the existing subvolumes. Even if the target has subvolumes under the same name, they might have different internal IDs. So it might make more sense to address the level 0 qgroups by name.

Good idea.

Quoted 3 lines

Also it might be misleading to apply a configuration to an existing fs, as it currently is not possible get a correct accounting if the fs is not empty. Rescan is not yet implemented.

Rescan will be implemented in the future, so it is not a main problem to implement 'btrfs qgroup import/export' commands.

Quoted 6 lines

So instead of just saving and restoring the qgroup config, it might make more sense to create a new filesystem including all subvolumes and quota config from a config file.
But, I'm not completely convinced that this is a features that is needed frequently. If I want a standard deployment, I simple write a script that creates the fs + subvol + quota.

If users want to config some qgroups(reset the limited size, modify its ancestral qgroups),i think it is more convenient and flexible to use import/export commands than write a script.

Above all,our qgroup import/export commands will be implemented as follows:

qgroupid is_compressed is_exclusive limited_size parent full_path
------------------------------------------------------------------------

And we may specify matching degree when we import the qgroup information.

<1>strict matching
qgroup(level-0) matches a subvolume/snapshot 's objectid and full path exactly. If a qgroup fail to match, the process will exit.

<2>general matching
It only require qgroup(level-0) to match a subvolume/snapshot 's full path. If the corresponding subvolume/snapshot does not exist,skip it. Otherwise,apply modifications to the corresponding subvolume/snapshot qgroup.

<3>weak matching
It only require qgroup(level-0) to match a subvolume/snapshot 's full path. If the corresponding subvolume/snapshot does not exist,create the subvolume automatically(a tracking qgroup is also created automatically)and then apply modifications to the newly created tracking qgroup.

How do you think about the above idea?

Thanks
Miao
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html