Stonewalled ?

Hi folks,

I'm running fio-2.2.5 on a Linux x86_64 platform. This is the first time I've had to create a job file with an extremely large number of job sections within the same file (hundreds of jobs). I need each job to run sequentially and have included "stonewall" within each section. When I execute the job file, I get this error:

error: maximum number of jobs (2048) reached.

When I reduce the number of job sections in the file to under 200, the job runs sequentially as expected.

My understanding of "stonewall" is that it should serialize the running of each job within a file (or files). The implication is that fio shouldn't be evaluating a subsequent job section until after the current job has fully completed. But in this case, fio appears to be looking ahead and ignoring the "stonewall" directive until it exhausts resources. This behavior also occurs within the current git commit.

Is this a feature or a bug, and is there another way to tell fio to execute each job sequentially (top to bottom) as it encounters them in the file?

Thanks!

(sorry about top-posting here)

For things like this, I'd use an external script specifying all params on each command line, and not use a job file at all. Order of execution is guaranteed and there's no limit to the number of jobs. (When fio evaluates the job file it creates all the jobs, but then doesn't let them all start at once.)

z!
________________________________________
From: [email protected] [[email protected]] on behalf of thoms [[email protected]]
Sent: Wednesday, February 25, 2015 8:04 PM
To: [email protected]
Subject: Stonewalled ?

Hi folks,

I'm running fio-2.2.5 on a Linux x86_64 platform. This is the first time I've had to create a job file with an extremely large number of job sections within the same file (hundreds of jobs). I need each job to run sequentially and have included "stonewall" within each section. When I execute the job file, I get this error:

error: maximum number of jobs (2048) reached.

When I reduce the number of job sections in the file to under 200, the job runs sequentially as expected.

My understanding of "stonewall" is that it should serialize the running of each job within a file (or files). The implication is that fio shouldn't be evaluating a subsequent job section until after the current job has fully completed. But in this case, fio appears to be looking ahead and ignoring the "stonewall" directive until it exhausts resources. This behavior also occurs within the current git commit.

Is this a feature or a bug, and is there another way to tell fio to execute each job sequentially (top to bottom) as it encounters them in the file?

Thanks!
--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm prepared to do that, it's just messy for my particular situation. Not having looked through the code, I was under the impression that "stonewall" changed the semantics of that behavior. Understood...

Thanks for the insight. Much appreciated...

The stonewall does do what you expect. The problem isn't related to how many jobs are running at the same time, it's how many jobs have been added. Basically it boils down to how big of an shm segment fio will need to run.

You could always just bump REAL_MAX_JOBS in fio.h to something larger than 2048, that should work on most platforms.

Jens Axboe