Simulating bursts at different rates at irregular intervals?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi. I'm new to fio and though I've read the HOWTO from the git repo several times now, I'm unsure if it is possible to achieve what I want;
I can simulate a mix of sequential and random writes for a duration at an average rate or throughput of say, 5.4MB/s.
I presume that fio will attempt to sustain that rate? My real question is this - is it possible in either the same job
or a different job in the same file to simulate bursts of higher rates? I.e. on average attempt to write at nMB/s but
at random or irregular intervals during this period attempt a 'burst' of 138MB/s for 5 minutes and then return to the 'normal' rate. Then again at some other point a burst of 57MB/s for 30minutes etc.
Is this possible? Presently I've found that I cannot do this. Or rather, thinking about it, I'd have to specify n jobs for each different rate and then perhaps use 'ratecycle' and 'thinktime'? I can't see how to skew 'thinktime' randomly though?
Regards,
Jim Vanns
Signature · 3 lines
Jim Vanns Senior Software Developer Framestore
Quoted 12 lines · James Vanns
Hi. I'm new to fio and though I've read the HOWTO from the git repo several times now, I'm unsure if it is possible to achieve what I want;
I can simulate a mix of sequential and random writes for a duration at an average rate or throughput of say, 5.4MB/s. I presume that fio will attempt to sustain that rate? My real question is this - is it possible in either the same job or a different job in the same file to simulate bursts of higher rates? I.e. on average attempt to write at nMB/s but at random or irregular intervals during this period attempt a 'burst' of 138MB/s for 5 minutes and then return to the 'normal' rate. Then again at some other point a burst of 57MB/s for 30minutes etc.
Quoted 4 lines
Is this possible? Presently I've found that I cannot do this. Or rather, thinking about it, I'd have to specify n jobs for each different rate and then perhaps use 'ratecycle' and 'thinktime'? I can't see how to skew 'thinktime' randomly though?
That's currently not possible, fio only supports a fixed rate. But if you have a good idea on how to express your criteria as an option (or options), then it can surely be added :-)
Signature · 1 line
Jens Axboe
Just a thought that came to mind.
To work around the problem would the following work? Run fio to get you max MB/s
Write a script to generate you work/jobs file that has randomizes length of time for the job to run and % of max for it to run at.
Thanks,
Roger
Quoted 29 lines · Jens Axboe
Quoted 12 lines · James Vanns
Hi. I'm new to fio and though I've read the HOWTO from the git repo several times now, I'm unsure if it is possible to achieve what I want;
I can simulate a mix of sequential and random writes for a duration at an average rate or throughput of say, 5.4MB/s. I presume that fio will attempt to sustain that rate? My real question is this - is it possible in either the same job or a different job in the same file to simulate bursts of higher rates? I.e. on average attempt to write at nMB/s but at random or irregular intervals during this period attempt a 'burst' of 138MB/s for 5 minutes and then return to the 'normal' rate. Then again at some other point a burst of 57MB/s for 30minutes etc.
Quoted 4 lines
Is this possible? Presently I've found that I cannot do this. Or rather, thinking about it, I'd have to specify n jobs for each different rate and then perhaps use 'ratecycle' and 'thinktime'? I can't see how to skew 'thinktime' randomly though?
That's currently not possible, fio only supports a fixed rate. But if you have a good idea on how to express your criteria as an option (or options), then it can surely be added :-)
--
Jens Axboe--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · Roger Sibert
Just a thought that came to mind.
To work around the problem would the following work? Run fio to get you max MB/s
Write a script to generate you work/jobs file that has randomizes length of time for the job to run and % of max for it to run at.
That's a bit of a bother, though, since you have to collect data and output from each individual run. And not only that, shutdown/startup will make things less than smooth. It could potentially dirty your results.
We could have something ala:
rate_sequence={10s:20M},{20s:5M}
or whatever, which runs 10s at 20M/sec, then 20s at 5M/sec. Repeat. Would be fairly easy to do.
Signature · 1 line
Jens Axboe
Quoted 11 lines
That's a bit of a bother, though, since you have to collect data and output from each individual run. And not only that, shutdown/startup will make things less than smooth. It could potentially dirty your results.
We could have something ala:
rate_sequence={10s:20M},{20s:5M}
or whatever, which runs 10s at 20M/sec, then 20s at 5M/sec. Repeat. Would be fairly easy to do.
Yes, that would be rather useful I think and the parameter format is intuitive. Though perhaps another parameter to somehow skew or randomise the length at which the burst rate runs for? And to also modify the sequence itself? Perhaps something akin to the following;
; define a list of rates that occasionally override the default in 'rate' burst_rates={10s:20M},{20s:5M}
; rate_sequence could default to the order defined above burst_rate_sequence=random
; intervals (in ms) - defaults to random burst_rate_interval=2500
Just imagine a spiky chart of write (or read, for that matter) throughput over time ;)
I suppose if interval were to be random then there should be a minimum and maximum.
Regards,
Jim
Quoted 8 lines
--
Jens Axboe--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 3 lines
Jim Vanns Senior Software Developer Framestore