memory leak in <=3.11.6
Hello!
Is this output in slabtop normal after a few days uptime with a daily rsync (btrfs to btrfs) and some compiling (gentoo emerge):
Active / Total Objects (% used) : 1120836 / 1810907 (61,9%) Active / Total Slabs (% used) : 42492 / 42492 (100,0%) Active / Total Caches (% used) : 72 / 101 (71,3%) Active / Total Size (% used) : 253898,56K / 324674,92K (78,2%)
Minimum / Average / Maximum Object : 0,01K / 0,18K / 15,69K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
109220 99107 90% 0,98K 6830 16 109280K btrfs_inode
If I close all my desktop programs, my system stays at +12 GB RAM usage while after a fresh boot it has 12+ GB free (of 16 GB). Cache stays low at 2 GB while after a few minutes uptime cache is about 5 GB.
This looks like memory leaking somewhere. Top does not show where all this memory went (biggest process is 500 MB RES, next is 200 MB and lower).
Thanks,
Kai
Quoted 4 lines · Kai Krakow
If I close all my desktop programs, my system stays at +12 GB RAM usage while after a fresh boot it has 12+ GB free (of 16 GB). Cache stays low at 2 GB while after a few minutes uptime cache is about 5 GB.
I probably have the same problem over here, after about 2 weeks of random read/write it seems my memory and swap get almost full and even after killing all process and getting in single user mode, memory won't free up. Would you happen to have quota enabled too? Kernel is 3.11.4
Jérôme Poulin posted on Mon, 28 Oct 2013 23:24:10 -0400 as excerpted:
Quoted 11 lines
Quoted 4 lines · Kai Krakow
If I close all my desktop programs, my system stays at +12 GB RAM usage while after a fresh boot it has 12+ GB free (of 16 GB). Cache stays low at 2 GB while after a few minutes uptime cache is about 5 GB.
I probably have the same problem over here, after about 2 weeks of random read/write it seems my memory and swap get almost full and even after killing all process and getting in single user mode, memory won't free up. Would you happen to have quota enabled too? Kernel is 3.11.4
FWIW, seeing nothing like that here, no quota/qgroups enabled.
But there's definitely something not right with qgroups at this time. First, a lot of folks with it enabled are reporting negative numbers which shouldn't be there, and second, all or very nearly all the huge memory usage issues reported seem to be from qgroups-enabled people.
So I'd call the qgroups feature broken at this time. Don't use it if you can avoid it (but be aware that turning it off triggers the memory issue and ultimate crashing for some, so preferably turn it off when you're rebooting anyway), and if you really do need quotas for your use-case, you may be better off on a non-btrfs filesystem for the time being.
(Of course btrfs is still listed as experimental in any case, so one shouldn't be surprised if this sort of thing happens from time to time or with some features.)
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 2 lines · Duncan
So I'd call the qgroups feature broken at this time
I agree. I made a page about Quotas on the wiki so to document my findings as documentation is really sparse. If you consult the section Known issues, you will find those:
* To get accurate information, you must issue a sync before using the qgroup show command.
* The qgroup show command is missing some information, for example you cannot see which subvolume is part of a parent qgroup.
* Creating a qgroup from an existing directory will show a 0 usage until a full filesystem quota rescan is issued.
* Using btrfs subvolume delete will break qgroup unshared space usage.
* After deleting a subvolume, you must manually delete the associated qgroup.
Knowing this, if you create a qgroup after some files have been added to a subvolume, those never get accounted for until you rescan the whole volume. This means that if you manage to delete enough "not accounted for" files, you get negative numbers.
Quoted 8 lines · Jérôme Poulin
Quoted 1 line · Duncan
So I'd call the qgroups feature broken at this time
I agree. I made a page about Quotas on the wiki so to document my findings as documentation is really sparse. If you consult the section Known issues, you will find those:
* To get accurate information, you must issue a sync before using the qgroup show command.
This is true in some extent.
qgroup accounting info(stored in disk) only update when commting transaction.
Anyway, by triggering a sync is a little overhead. maybe we can implement an sync ioctl for quota sync.
Quoted 2 lines
* The qgroup show command is missing some information, for example you cannot see which subvolume is part of a parent qgroup.
This has been implemented in the latest chris's integration branch,you can use
btrfs qgroup show -p <mnt>
Quoted 2 lines
* Creating a qgroup from an existing directory will show a 0 usage until a full filesystem quota rescan is issued
In the latest kernel. if you enable quota, it will create qgroup for every existed subvolume, and then rescan will be triggered automatically.(only happens in rescanning)
When creating a subvolume, you can use
btrfs sub create -i <pid> <mnt>
this will add a parent qgroup for a subvolume the moment it was created, we can avoid rescanning for this case.
However, if you create a parent qgroup and then assign subvolume qgroup to this parent qgroup online, you have to rescan.
Quoted 3 lines
* Using btrfs subvolume delete will break qgroup unshared space usage.
* After deleting a subvolume, you must manually delete the associated qgroup.
This is true, for now, deleting a subvolume will cause wrong qgroup accounting. Jan sent a patch for this recently, hopely this problem can be solved as soon as possible.
I have planed to make a patch to drop associated qgroup when deleting a subvolume only if qgroup accounting problem has been fixed.
Quoted 12 lines
Knowing this, if you create a qgroup after some files have been added to a subvolume, those never get accounted for until you rescan the whole volume. This means that if you manage to delete enough "not accounted for" files, you get negative numbers.
Wiki: https://btrfs.wiki.kernel.org/index.php/Quota_support
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 9 lines · Jérôme Poulin
Quoted 3 lines
If I close all my desktop programs, my system stays at +12 GB RAM usage while after a fresh boot it has 12+ GB free (of 16 GB). Cache stays low at 2 GB while after a few minutes uptime cache is about 5 GB.
I probably have the same problem over here, after about 2 weeks of random read/write it seems my memory and swap get almost full and even after killing all process and getting in single user mode, memory won't free up. Would you happen to have quota enabled too? Kernel is 3.11.4
No, quota is off for me. This is a three-device btrfs desktop system. Quota has no real use for me (except maybe for subvolume accounting).
I first thought this may be related to having zcache enabled, turned it off. But anything that changed is when memory stress arises my system stays more responsible and does not go havoc on the CPU.
I switched to zswap instead, since then swap usage is much lower. But still it does not help if memory is mysteriously used somewhere. At least IO stress is lower now with that setting and I am better able to cleanly shut down programs and the system for reboot.
Regards,
Kai