Skip to contentSkip to conversation

systemd "journalctl" is 1.89sec on ext4, 1.49 min on btrfs

DISCUSSION Szőts Ákos 4 participants 5 messages May 27, 2013 1 message not in the archive Btrfs file system

lore
Share & tools
lore
#0 lore raw

Dear list,

I have two openSUSE 12.3 systems with kernel 3.9. On one of them there's an ext4 partition, while on the other there's a btrfs.

I issued a "time journalctl -b --no-pager" command on both systems. This shows the logs from the current boot without passing them to "less".

On ext4 (3.9.3):

real    0m1.898s
user    0m0.291s
sys     0m0.105s

On btrfs (3.9.2):

real    1m49.698s
user    0m0.102s
sys     0m0.470s

Journalctl on btrfs was always this slow, some btrfsck were made on the file system too, but I don't think it was corrupted. On just the first run it's sluggish, after it's fast as the ext4 one.

Is it a known issue or can I help somehow debugging this further?

Best regards,

Ákos Szőts

#2 lore raw
In reply to a message not in the archive

Hi,

Quoted 3 lines
Quoted 3 lines

Journalctl on btrfs was always this slow, some btrfsck were made on the file system too, but I don't think it was corrupted. On just the first run it's sluggish, after it's fast as the ext4 one.

Most likely the file is heavily fragmented due to COW.
Have you tried to manually defrag the filesystem and turn on autodefrag?

Regards, Clemens

#3 lore raw
In reply to Szőts Ákos

I have gotten what appear to be large increases in speed out of btrfs by defragmentation of meta data. The manual defragmentation process takes forever as you have to defragment incrementally directory by directory. I was at the point where KDE startup times were getting abysmal (along with journalctl, etc) and the multiple drives would churn incessantly on startup. In the case of KDE, I found almost magical improvement with one operation: btrfs filesystem defrag /usr/share. I am currently going through the whole system deframenting directory by directory. Its amazing, it proceeds quite quickly and then hits a directory at random where it sits and plods away seemingly forever before moving on. I am convinced that there is something going on here with meta data fragmentation that, at times, is seriously affecting performance. I *think* that autodefrag, once its out the door will hopefully solve this, in the mean time I am trying to come up with some sort of way to schedule an anacron job to deal with this issue. But my suggestion would be that you try defragging your /var filesystem as thoroughly as possible on the meta data side.

Quoted 33 lines · Szőts Ákos

Dear list,

I have two openSUSE 12.3 systems with kernel 3.9. On one of them there's an ext4 partition, while on the other there's a btrfs.

I issued a "time journalctl -b --no-pager" command on both systems. This shows the logs from the current boot without passing them to "less".

On ext4 (3.9.3):

real    0m1.898s
user    0m0.291s
sys     0m0.105s

On btrfs (3.9.2):

real    1m49.698s
user    0m0.102s
sys     0m0.470s

Journalctl on btrfs was always this slow, some btrfsck were made on the file system too, but I don't think it was corrupted. On just the first run it's sluggish, after it's fast as the ext4 one.

Is it a known issue or can I help somehow debugging this further?

Best regards,

Ákos Szőts
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
In reply to Szőts Ákos
Quoted 18 lines · On Mon, 27 May 2013 18:21:21 +0200 Szőts Ákos <[email protected]> wrote:

Dear list,

I have two openSUSE 12.3 systems with kernel 3.9. On one of them there's an ext4 partition, while on the other there's a btrfs.

I issued a "time journalctl -b --no-pager" command on both systems. This shows the logs from the current boot without passing them to "less".

On ext4 (3.9.3):

real    0m1.898s
user    0m0.291s
sys     0m0.105s

On btrfs (3.9.2):

real    1m49.698s
user    0m0.102s
sys     0m0.470s
Quoted 5 lines

Journalctl on btrfs was always this slow, some btrfsck were made on the file system too, but I don't think it was corrupted. On just the first run it's sluggish, after it's fast as the ext4 one.

Is it a known issue or can I help somehow debugging this further?

Let's look at your

    /usr/sbin/filefrag /var/log/journal/*/*

on both systems

Signature · 1 line
  Sergei
#5 lore raw

Thank you very much for you suggestions!

I ran filefrag on /var/log/journal/ and the most fragmented files were:
/var/log/journal/[...]/faa9e41c6.journal: 11136 extents found /var/log/journal/[...]/e9f8e1346c.journal: 633 extents found /var/log/journal/[...]/59ec50b631.journal: 3585 extents found /var/log/journal/[...]/59ec51d1b4.journal: 710 extents found /var/log/journal/[...]/f74afadee9.journal: 484 extents found

Then I ran btrfs fi defrag -v -f -clzo * in the aforementioned directory three times and restarted the machine. Now, the journalctl -b --no-pager command runs in less than two seconds.

I tried to further optimize my file system (MySQL big storage files are highly affected), but somewhere (for me it looks like, on random places) there are kernel errors, so I suspended the further defragmentation until those errors are tracked down.

Thank you very much again.

Ákos