No space left on device after many files creation

I have found issue with ext4 partition.
I created many many empty files with the following way:
[cy6ergn0m@cgmachine heap]$ seq 1 10000000 | xargs touch

But this command crashes asfter a while with a error 'no space left on device'. At the same time, df -h shows that I have 7Gb free on this file system. So, I can't create any file in any directory at this filesystem.

[cy6ergn0m@cgmachine heap]$ pwd
/e4/sandbox/heap
[cy6ergn0m@cgmachine heap]$ ls -1 | wc -l
1261144
[cy6ergn0m@cgmachine heap]$ df -h /e4
Файловая система Разм Исп Дост Исп% смонтирована на
(filesystem/size/used/available/used%/mounted)
/dev/sda4 28G 7.2G 19G 28% /e4

[cy6ergn0m@cgmachine heap]$ touch ttt
touch: невозможно выполнить touch для `ttt': На устройстве кончилось место (touch: failed to execute touch for 'ttt': No space left on device)

Quoted 8 lines · cy6erGn0m wrote:

I have found issue with ext4 partition.
I created many many empty files with the following way:
[cy6ergn0m@cgmachine heap]$ seq 1 10000000 | xargs touch

But this command crashes asfter a while with a error 'no space left on device'. At the same time, df -h shows that I have 7Gb free on this file system. So, I can't create any file in any directory at this filesystem.

Quoted 9 lines

[cy6ergn0m@cgmachine heap]$ pwd
/e4/sandbox/heap
[cy6ergn0m@cgmachine heap]$ ls -1 | wc -l
1261144
[cy6ergn0m@cgmachine heap]$ df -h /e4
Файловая система Разм Исп Дост Исп% смонтирована на
(filesystem/size/used/available/used%/mounted)
/dev/sda4 28G 7.2G 19G 28% /e4

try df -i

There is a limited number of inodes that you can create, as set at mkfs time.

-Eric

Quoted 6 lines

[cy6ergn0m@cgmachine heap]$ touch ttt
touch: невозможно выполнить touch для `ttt': На устройстве кончилось место (touch: failed to execute touch for 'ttt': No space left on device)

In the past I have done the same test for ext3 and it was created ~10M files in one firectory at working filesystem (/home) and it was no issues reached.

On ext4 i found inodes max size is limited as you wrote:
[root@cgmachine sandbox]# df -i /e4
Файловая система Инодов Испол Своб Исп % смонтирована на
(file system/inodes/used/free/used%/mounted on)
/dev/sda4 1,8M 1,8M 0 100% /e4
[root@cgmachine sandbox]#

Why inodes count limit is so small by default? Is it depends on partition size?

Quoted 27 lines · Eric Sandeen

Quoted 8 lines · cy6erGn0m wrote:

I have found issue with ext4 partition.
I created many many empty files with the following way:
[cy6ergn0m@cgmachine heap]$ seq 1 10000000 | xargs touch

But this command crashes asfter a while with a error 'no space left on device'. At the same time, df -h shows that I have 7Gb free on this file system. So, I can't create any file in any directory at this filesystem.

Quoted 9 lines

[cy6ergn0m@cgmachine heap]$ pwd
/e4/sandbox/heap
[cy6ergn0m@cgmachine heap]$ ls -1 | wc -l
1261144
[cy6ergn0m@cgmachine heap]$ df -h /e4
Файловая система Разм Исп Дост Исп% смонтирована на
(filesystem/size/used/available/used%/mounted)
/dev/sda4 28G 7.2G 19G 28% /e4

try df -i

There is a limited number of inodes that you can create, as set at mkfs time.

-Eric

Quoted 7 lines

Quoted 6 lines

[cy6ergn0m@cgmachine heap]$ touch ttt
touch: невозможно выполнить touch для `ttt': На устройстве кончилось место (touch: failed to execute touch for 'ttt': No space left on device)

Quoted 11 lines · cy6erGn0m wrote:

In the past I have done the same test for ext3 and it was created ~10M files in one firectory at working filesystem (/home) and it was no issues reached.

On ext4 i found inodes max size is limited as you wrote:
[root@cgmachine sandbox]# df -i /e4
Файловая система Инодов Испол Своб Исп % смонтирована на
(file system/inodes/used/free/used%/mounted on)
/dev/sda4 1,8M 1,8M 0 100% /e4
[root@cgmachine sandbox]#

Quoted 2 lines

Why inodes count limit is so small by default? Is it depends on partition size?

it does depend on partition size; the inode_ratio = 16384 in mke2fs.conf says 1 inode per 16k of filesystem space.

Quoted 13 lines · Eric Sandeen

Quoted 11 lines · cy6erGn0m wrote:

In the past I have done the same test for ext3 and it was created ~10M files in one firectory at working filesystem (/home) and it was no issues reached.

On ext4 i found inodes max size is limited as you wrote:
[root@cgmachine sandbox]# df -i /e4
Файловая система Инодов Испол Своб Исп % смонтирована на
(file system/inodes/used/free/used%/mounted on)
/dev/sda4 1,8M 1,8M 0 100% /e4
[root@cgmachine sandbox]#

Quoted 6 lines

Quoted 3 lines

Why inodes count limit is so small by default? Is it depends on partition size?

it does depend on partition size; the inode_ratio = 16384 in mke2fs.conf says 1 inode per 16k of filesystem space.

1.8 million inodes is hardly small..... most people complain that we're reserving too much space for inodes.

Cy6erGn0m, What is your use case where the average file size for your 20G partition is smaller that 16k?

- Ted

I know that filesystems is not targeted for this usecase, but inodes limit looks strange.. why they are can't be allocated dinamically? Can I

enlarge this limit on the fly?

I use filysystem as Set data structure to make fast checks from scripts. Of course I can make long long file and test it with grep but it's will have linear performance degradation (List), at the same time file system has efficient tree binary structures (smth. like TreeSet). So, using filesystem is the simplest way to do this efficient. Also I always have timestamps for every entry. In this usecase all files are always empty.