nilfs_cleanerd some questions

I'm using nilfs2 on a debian (lenny machine) using kernel 2.6.26-1-686 and nilfs version 2.0.6. Currently I have a 250 Gb single partition drive connected via USB2, which I have formatted for nilfs2. I have noticed that when extracting a large archive onto the the file system that it seems to "burst", that is, it will extract for 5 seconds then halt for 1 and then repeat this cycle faithfully for the duration.

If I switch (kill) off the GC then the "bursting" effect disappears. I can remove checkpoints more efficiently using rmcp, however am I right in thinking that the GC does more than just remove checkpoints ?

Is the behaviour that I am witnessing typical of the nilfs GC or is being highlighted because I am using usb instead of directly attached ide / sata ?

I'm hoping to apply nilfs in production environment for our print unit at work because they would like self service recovery and because they have large amounts of graphical data that is updated daily but could only be safely backed up using a snaphot technology.

However I am concerned that any large file / archive extracts onto a nilfs file system with the GC switched on would impede efficiency.

It is safe to say that I wont be using USB attached storage in the production environment, but I thought it might be worth mentioning my findings as the behaviour may still be present in a very muted form on IDE / SATA based storage devices.

I guess I could run GC at off peak times as a workaround, but would it ever catch up ?

:-), to let other cases be handled, like running on battery, and

Hi,

Quoted 10 lines · tom

I'm using nilfs2 on a debian (lenny machine) using kernel 2.6.26-1-686 and nilfs version 2.0.6. Currently I have a 250 Gb single partition drive connected via USB2, which I have formatted for nilfs2. I have noticed that when extracting a large archive onto the the file system that it seems to "burst", that is, it will extract for 5 seconds then halt for 1 and then repeat this cycle faithfully for the duration.

If I switch (kill) off the GC then the "bursting" effect disappears. I can remove checkpoints more efficiently using rmcp, however am I right in thinking that the GC does more than just remove checkpoints ?

Yes, only GC can reclaim disk space; rmcp does not make free space.

Quoted 3 lines

Is the behaviour that I am witnessing typical of the nilfs GC or is being highlighted because I am using usb instead of directly attached ide / sata ?

I don't know why that happens, but a likely cause is bulky reads of blocks during GC.

In that case, the following workarounds may make a difference.

1) Decrease value of `nsegments_per_clean' in /etc/nilfs_cleanerd.conf e.g.
#nsegments_per_clean 2
nsegments_per_clean 1

2) Re-create the partition using mkfs.nilfs2 with a smaller segment size. e.g.

 # mkfs -t nilfs2 -B 1024 /dev/sdxx
 (This halves the number of blocks per segment)

Quoted 5 lines

I'm hoping to apply nilfs in production environment for our print unit at work because they would like self service recovery and because they have large amounts of graphical data that is updated daily but could only be safely backed up using a snaphot technology.

Quoted 7 lines

However I am concerned that any large file / archive extracts onto a nilfs file system with the GC switched on would impede efficiency.

It is safe to say that I wont be using USB attached storage in the production environment, but I thought it might be worth mentioning my findings as the behaviour may still be present in a very muted form on IDE / SATA based storage devices.

Thanks for reporting.

I feel, the current GC is indeed immature and needs more love especially for production use.

Quoted 2 lines

I guess I could run GC at off peak times as a workaround, but would it ever catch up ?

Not guaranteed. The speed of the GC is static unless you change nilfs_cleanerd.conf and send a HUP signal to the GC daemon. At least the speed should be adjusted adaptively, but not done yet.

Regards,
Ryusuke Konishi
_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users

Hello!

Sorry for my intrusion, but probably a way to pause the GC and resume it, would be nice, maybe with SIGUSR1 and SIGUSR2, or even trough dbus

Quoted 1 line

:-), to let other cases be handled, like running on battery, and

similiar. IMHO this is much more elegant way, then killing, and respawning the GC at every such change, and it can finish its work, before getting killed aswell.

Quoted 59 lines

Hi,

Quoted 10 lines · tom

I'm using nilfs2 on a debian (lenny machine) using kernel 2.6.26-1-686 and nilfs version 2.0.6. Currently I have a 250 Gb single partition drive connected via USB2, which I have formatted for nilfs2. I have noticed that when extracting a large archive onto the the file system that it seems to "burst", that is, it will extract for 5 seconds then halt for 1 and then repeat this cycle faithfully for the duration.

If I switch (kill) off the GC then the "bursting" effect disappears. I can remove checkpoints more efficiently using rmcp, however am I right in thinking that the GC does more than just remove checkpoints ?

Yes, only GC can reclaim disk space; rmcp does not make free space.

Quoted 3 lines

Is the behaviour that I am witnessing typical of the nilfs GC or is being highlighted because I am using usb instead of directly attached ide / sata ?

I don't know why that happens, but a likely cause is bulky reads of blocks during GC.

In that case, the following workarounds may make a difference.

1) Decrease value of `nsegments_per_clean' in /etc/nilfs_cleanerd.conf e.g.
#nsegments_per_clean 2
nsegments_per_clean 1

2) Re-create the partition using mkfs.nilfs2 with a smaller segment size. e.g.

 # mkfs -t nilfs2 -B 1024 /dev/sdxx
 (This halves the number of blocks per segment)

Quoted 5 lines

I'm hoping to apply nilfs in production environment for our print unit at work because they would like self service recovery and because they have large amounts of graphical data that is updated daily but could only be safely backed up using a snaphot technology.

Quoted 7 lines

However I am concerned that any large file / archive extracts onto a nilfs file system with the GC switched on would impede efficiency.

It is safe to say that I wont be using USB attached storage in the production environment, but I thought it might be worth mentioning my findings as the behaviour may still be present in a very muted form on IDE / SATA based storage devices.

Thanks for reporting.

I feel, the current GC is indeed immature and needs more love especially for production use.

Quoted 2 lines

I guess I could run GC at off peak times as a workaround, but would it ever catch up ?

Not guaranteed. The speed of the GC is static unless you change nilfs_cleanerd.conf and send a HUP signal to the GC daemon. At least the speed should be adjusted adaptively, but not done yet.

Quoted 7 lines

Regards,
Ryusuke Konishi
_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users

_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users