Skip to contentSkip to conversation

madvise failure

DISCUSSION Andries E. Brouwer 2 participants 3 messages May 7, 2009 – May 8, 2009

lore
Share & tools
lore
#0 lore raw

In an application something like

	p = mmap(0, sz, PROT_READ, MAP_PRIVATE, fd, 0);
	madvise(p, sz, MADV_SEQUENTIAL);

is done for a small number of files, each with a size of a few GB. A single sequential pass is done over these files - essentially a merge.

On an old machine the madvise is useful, and decreases total time. But on a more recent machine, with more memory, the madvise makes things worse. There, it seems better not to reveal to the kernel that the data will be read sequentially.

Timing (six files of 4GB each, quadcore Intel Q9550, 16GB memory, kernel 2.6.27 [Ubuntu], two other processes active):
with madvise, 7 runs: real time varying 9m10s - 37m29s, without madvise, 6 runs: real time fairly constant 5m45s - 5m54s.

Andries

#1 lore raw
Quoted 7 lines · On Thu, 7 May 2009 14:10:17 +0200 "Andries E. Brouwer"…

In an application something like

	p = mmap(0, sz, PROT_READ, MAP_PRIVATE, fd, 0);
	madvise(p, sz, MADV_SEQUENTIAL);

is done for a small number of files, each with a size of a few GB. A single sequential pass is done over these files - essentially a merge.

On an old machine the madvise is useful, and decreases total time.

With the same kernel (Ubuntu/2.6.27) ? old one ?

Quoted 10 lines

But on a more recent machine, with more memory, the madvise makes things worse. There, it seems better not to reveal to the kernel that the data will be read sequentially.

Timing (six files of 4GB each, quadcore Intel Q9550, 16GB memory, kernel 2.6.27 [Ubuntu], two other processes active):
with madvise, 7 runs: real time varying 9m10s - 37m29s, without madvise, 6 runs: real time fairly constant 5m45s - 5m54s.

Accessing each page one by one sequentially ? or Sequential but (may) skip some pages in each access ?

Thanks,
-Kame

#2 lore raw
Quoted 11 lines · KAMEZAWA Hiroyuki
Quoted 7 lines · Andries E. Brouwer

In an application something like

	p = mmap(0, sz, PROT_READ, MAP_PRIVATE, fd, 0);
	madvise(p, sz, MADV_SEQUENTIAL);

is done for a small number of files, each with a size of a few GB. A single sequential pass is done over these files - essentially a merge.

On an old machine the madvise is useful, and decreases total time.

With the same kernel (Ubuntu/2.6.27) ? old one ?

Both Ubuntu/2.6.27: 2.6.27-11-generic (new) vs 2.6.27-11-server (old)

Quoted 10 lines
Quoted 8 lines

But on a more recent machine, with more memory, the madvise makes things worse. There, it seems better not to reveal to the kernel that the data will be read sequentially.

Timing (six files of 4GB each, quadcore Intel Q9550, 16GB memory, kernel 2.6.27 [Ubuntu], two other processes active):
with madvise, 7 runs: real time varying 9m10s - 37m29s, without madvise, 6 runs: real time fairly constant 5m45s - 5m54s.

Accessing each page one by one sequentially ?

Yes. Essentially a merge.

Quoted 1 line

or Sequential but (may) skip some pages in each access ?

No.

Andries