Skip to contentSkip to conversation

zbd: remove reset_zone flag from fio_zone_info

PATCH Alexey Dobriyan 2 participants 5 messages Aug 19, 2020 – Sep 2, 2020

lore
Share & tools
lore
#0 lore raw
Quoted 4 lines

The reset_zone flag that is defined in fio_zone_info structure is only referenced in zbd_adjust_block() function. Convert this flag to a local variable and save some room in zbd_info array which can be pretty large when running fio against high capacity zoned devices.

This flag should be kept.

Test can crash or be interrupted leaving hw in indeterminate state, it must be restarted from clean state. Or test can be precondition and zone should not be reset.

Can't distinguish these two cases from write pointers alone.

Given that precondition and real workload can be in different fio instances, this information must be somehow given to fio process and either force or not force zone reset (config option obviously)

Review struct fio_zone_info { · line 44

Quoted 5 lines
(quoted diff) +0 -1
@@ -41,7 +40,6 @@ struct fio_zone_info {
 	enum zbd_zone_type	type:2;
 	enum zbd_zone_cond	cond:4;
 	unsigned int		open:1;
-	unsigned int		reset_zone:1;

Does it save space? Bitfields stick together after all.

#1 lore raw
Quoted 14 lines

-----Original Message-----
From: Alexey Dobriyan <[email protected]>
Sent: Wednesday, August 19, 2020 8:56 AM
To: Dmitry Fomichev <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] zbd: remove reset_zone flag from fio_zone_info

Quoted 4 lines

The reset_zone flag that is defined in fio_zone_info structure is only referenced in zbd_adjust_block() function. Convert this flag to a local variable and save some room in zbd_info array which can be pretty large when running fio against high capacity zoned devices.

Quoted 5 lines

This flag should be kept.

Test can crash or be interrupted leaving hw in indeterminate state, it must be restarted from clean state. Or test can be precondition and zone should not be reset.

This is beyond the scope of fio. Any preconditioning of zones is done by external scripts that have no access to this flag. Tools like blkzone, libzbc or nvme-cli are typically used to precondition zones before testing and to analyze zone state after test runs.

Quoted 6 lines

Can't distinguish these two cases from write pointers alone.

Given that precondition and real workload can be in different fio instances, this information must be somehow given to fio process and either force or not force zone reset (config option obviously)

The functionality you are describing in not a part of the current fio code. Perhaps you are talking about some proprietary code additions that you are developing. Please submit a patch that will implement what you are talking about and if it has merit you will have a solid case to reintroduce this flag.

For now, having this flag only makes the code less straightforward.

Quoted 8 lines

Review struct fio_zone_info { · line 44

Quoted 6 lines
(quoted diff) +0 -1
@@ -41,7 +40,6 @@ struct fio_zone_info {
 	enum zbd_zone_type	type:2;
 	enum zbd_zone_cond	cond:4;
 	unsigned int		open:1;
-	unsigned int		reset_zone:1;

Does it save space? Bitfields stick together after all.

It saves room for other stuff that is going be added to this struct in the future. Some new members will likely be added there to support zone append and there is some other development in the works that will likely add more bits there. Keep in mind that the number of zones on a 20TB DHSMR drive may exceed 130K and these numbers will only grow over time. Every bit counts in this structure :)

#2 lore raw
Quoted 16 lines · Dmitry Fomichev
Quoted 16 lines

-----Original Message-----
From: Alexey Dobriyan <[email protected]>
Sent: Wednesday, August 19, 2020 8:56 AM
To: Dmitry Fomichev <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] zbd: remove reset_zone flag from fio_zone_info

Quoted 4 lines

The reset_zone flag that is defined in fio_zone_info structure is only referenced in zbd_adjust_block() function. Convert this flag to a local variable and save some room in zbd_info array which can be pretty large when running fio against high capacity zoned devices.

This flag should be kept.

Quoted 8 lines
Quoted 3 lines

Test can crash or be interrupted leaving hw in indeterminate state, it must be restarted from clean state. Or test can be precondition and zone should not be reset.

This is beyond the scope of fio. Any preconditioning of zones is done by external scripts that have no access to this flag. Tools like blkzone, libzbc or nvme-cli are typically used to precondition zones before testing and to analyze zone state after test runs.

If it is a matter of policy that fio doesn't do preconditioning, then the patch is OK.

We do preconditioning in fio, it is neat:
* don't need external programs
* everything inside one job file
* not a lot of code to add

But to do this, flag must live in "struct fio_zone_info".

	[zra]
	zone_reset_all=1
	zone_finish_all=1
	rw=write

	[pre]
	stonewall
	zone_reset_all=0
	zone_finish_all=0
	job_max_open_zones=...
	rw=write
		...

	[j]
	stonewall
	zone_reset_all=0
	zone_finish_all=0
	rw=randread
		...
#3 lore raw
Quoted 43 lines

-----Original Message-----
From: Alexey Dobriyan <[email protected]>
Sent: Friday, August 21, 2020 1:07 PM
To: Dmitry Fomichev <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] zbd: remove reset_zone flag from fio_zone_info

Quoted 16 lines · Dmitry Fomichev
Quoted 16 lines

-----Original Message-----
From: Alexey Dobriyan <[email protected]>
Sent: Wednesday, August 19, 2020 8:56 AM
To: Dmitry Fomichev <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] zbd: remove reset_zone flag from fio_zone_info

Quoted 4 lines

The reset_zone flag that is defined in fio_zone_info structure is only referenced in zbd_adjust_block() function. Convert this flag to a local variable and save some room in zbd_info array which can be pretty large when running fio against high capacity zoned devices.

This flag should be kept.

Quoted 8 lines
Quoted 3 lines

Test can crash or be interrupted leaving hw in indeterminate state, it must be restarted from clean state. Or test can be precondition and zone should not be reset.

This is beyond the scope of fio. Any preconditioning of zones is done by external scripts that have no access to this flag. Tools like blkzone, libzbc or nvme-cli are typically used to precondition zones before testing and to analyze zone state after test runs.

If it is a matter of policy that fio doesn't do preconditioning, then the patch is OK.

We do preconditioning in fio, it is neat:
* don't need external programs
* everything inside one job file
* not a lot of code to add

Quoted 4 lines

But to do this, flag must live in "struct fio_zone_info".

	[zra]
	zone_reset_all=1

The way I understand this, this option makes ZBD code to set the reset_zone flag for all zones during init. This will force zone reset before the first write to a zone. This indeed is a neat idea and this feature looks to be pretty lightweight as you said.

Quoted 1 line

zone_finish_all=1

How does this option work? Seems like you have added a new finish_zone flag and you finish every zone for which the flag is set instead of performing the first write to the zone...

Quoted 8 lines
	rw=write

	[pre]
	stonewall
	zone_reset_all=0
	zone_finish_all=0
	job_max_open_zones=...
	rw=write
Quoted 8 lines
		...

	[j]
	stonewall
	zone_reset_all=0
	zone_finish_all=0
	rw=randread
		...

I guess we can hold off with removing the reset_zone flag if you are planning to send the patch for this.

Dmitry

#4 lore raw
Quoted 53 lines · Dmitry Fomichev
Quoted 43 lines

-----Original Message-----
From: Alexey Dobriyan <[email protected]>
Sent: Friday, August 21, 2020 1:07 PM
To: Dmitry Fomichev <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] zbd: remove reset_zone flag from fio_zone_info

Quoted 16 lines · Dmitry Fomichev
Quoted 16 lines

-----Original Message-----
From: Alexey Dobriyan <[email protected]>
Sent: Wednesday, August 19, 2020 8:56 AM
To: Dmitry Fomichev <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] zbd: remove reset_zone flag from fio_zone_info

Quoted 4 lines

The reset_zone flag that is defined in fio_zone_info structure is only referenced in zbd_adjust_block() function. Convert this flag to a local variable and save some room in zbd_info array which can be pretty large when running fio against high capacity zoned devices.

This flag should be kept.

Quoted 8 lines
Quoted 3 lines

Test can crash or be interrupted leaving hw in indeterminate state, it must be restarted from clean state. Or test can be precondition and zone should not be reset.

This is beyond the scope of fio. Any preconditioning of zones is done by external scripts that have no access to this flag. Tools like blkzone, libzbc or nvme-cli are typically used to precondition zones before testing and to analyze zone state after test runs.

If it is a matter of policy that fio doesn't do preconditioning, then the patch is OK.

We do preconditioning in fio, it is neat:
* don't need external programs
* everything inside one job file
* not a lot of code to add

Quoted 4 lines

But to do this, flag must live in "struct fio_zone_info".

	[zra]
	zone_reset_all=1

The way I understand this, this option makes ZBD code to set the reset_zone flag for all zones during init. This will force zone reset before the first write to a zone. This indeed is a neat idea and this feature looks to be pretty lightweight as you said.

Quoted 5 lines
Quoted 1 line

zone_finish_all=1

How does this option work? Seems like you have added a new finish_zone flag and you finish every zone for which the flag is set instead of performing the first write to the zone...

Sorry for delay.

ZRA marks every zone in [->min_zone, ->max_zone) for reset, forcing Zone Reset before first write.

ZFA sends Finish Zone with "all" flag set synchronously.

Some job orchestration is required but once it is done, jobs run reliably.

Quoted 17 lines
Quoted 8 lines
	rw=write

	[pre]
	stonewall
	zone_reset_all=0
	zone_finish_all=0
	job_max_open_zones=...
	rw=write
Quoted 8 lines
		...

	[j]
	stonewall
	zone_reset_all=0
	zone_finish_all=0
	rw=randread
		...
Quoted 2 lines

I guess we can hold off with removing the reset_zone flag if you are planning to send the patch for this.

Thanks, I'll try to extract something mainlineable.