fbccf1df065: +276.3% boot-meminfo.KernelStack
FYI, we noticed the below changes on
git://git.kernel.org/pub/scm/linux/kernel/git/mlin/linux.git block-generic-req
commit fbccf1df06574add60bdb71103cc1662e477f02d ("block: make generic_make_request handle arbitrarily sized bios")
testcase/path_params/tbox_group: boot/1/vm-kbuild-1G
v4.1-rc4 fbccf1df06574add60bdb71103
---------------- --------------------------
%stddev %change %stddev
\ | \
1587 ± 7% +276.3% 5972 ± 1% boot-meminfo.KernelStack
152390 ± 0% +4.8% 159697 ± 0% boot-slabinfo.num_objs
10149 ± 0% +15.8% 11750 ± 0% boot-slabinfo.num_pages
35916 ± 5% +18.0% 42368 ± 9% boot-meminfo.DirectMap4k
478516 ± 0% -1.9% 469269 ± 0% boot-meminfo.MemFree
11477 ± 0% +49.1% 17111 ± 1% boot-meminfo.SUnreclaim
40551 ± 0% +15.7% 46910 ± 0% boot-meminfo.Slabvm-kbuild-1G: qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smap
Memory: 1G
boot-meminfo.KernelStack
7000 ++-------------------------------------------------------------------+
| O O O O |
6000 O+ O O O O O O O O O O O O O O O O O O
| |
5000 ++ |
| |
4000 ++ |
| |
3000 ++ |
| |
2000 ++ *.. |
* *..*..*..* : * |
1000 ++ .. + : |
| + . + : |0 ++-*--*--*---------------*-------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
To reproduce:
apt-get install ruby
git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
cd lkp-tests
bin/setup-local job.yaml # the job file attached in this email
bin/run-local job.yaml
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance.
Thanks,
Ying Huang
-------------------------------------
lkp(a)eclists.intel.com
https://eclists.intel.com/sympa/info/lkp
Unsubscribe by sending email to sympa(a)eclists.intel.com with subject "Unsubscribe lkp"
---LKP_SERVER: inn
testcase: boot
default-monitors:
default-watchdogs:
watch-oom:
watchdog:cpufreq_governor:
model: qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smapnr_vm: 16nr_cpu: 2
memory: 1Gdisk_type: virtio-scsi
rootfs: debian-x86_64-2015-02-07.cgzhdd_partitions: "/dev/sda /dev/sdb /dev/sdc /dev/sdd"swap_partitions: "/dev/sde"
boot-slabinfo:
boot-meminfo:
memmap:
boot-memory:
boot-time:
kernel-size:
timeout: 10m
sleep: 1
branch: linux-devel/devel-hourly-2015052515
commit: 9e46a1eba10500df620683e31742789ef2781e93repeat_to: 2
testbox: vm-kbuild-1G-6tbox_group: vm-kbuild-1G
kconfig: x86_64-lkpenqueue_time: 2015-05-25 16:11:14.888350830 +08:00
user: lkp
queue: boot
compiler: gcc-4.9
kernel: "/pkg/linux/x86_64-lkp/gcc-4.9/9e46a1eba10500df620683e31742789ef2781e93/vmlinuz-4.1.0-rc5-02510-g9e46a1e"result_root: "/result/boot/1/vm-kbuild-1G/debian-x86_64-2015-02-07.cgz/x86_64-lkp/gcc-4.9/9e46a1eba10500df620683e31742789ef2781e93/0"job_file: "/lkp/scheduled/vm-kbuild-1G-6/boot_boot-1-debian-x86_64.cgz-x86_64-lkp-9e46a1eba10500df620683e31742789ef2781e93-1-20150525-48539-iakefb.yaml"dequeue_time: 2015-05-25 16:11:19.987783209 +08:00max_uptime: 600
initrd: "/osimage/debian/debian-x86_64-2015-02-07.cgz"bootloader_append:
- root=/dev/ram0
- user=lkp
- job=/lkp/scheduled/vm-kbuild-1G-6/boot_boot-1-debian-x86_64.cgz-x86_64-lkp-9e46a1eba10500df620683e31742789ef2781e93-1-20150525-48539-iakefb.yaml
- ARCH=x86_64
- kconfig=x86_64-lkp
- branch=linux-devel/devel-hourly-2015052515
- commit=9e46a1eba10500df620683e31742789ef2781e93
- BOOT_IMAGE=/pkg/linux/x86_64-lkp/gcc-4.9/9e46a1eba10500df620683e31742789ef2781e93/vmlinuz-4.1.0-rc5-02510-g9e46a1e
- max_uptime=600
- RESULT_ROOT=/result/boot/1/vm-kbuild-1G/debian-x86_64-2015-02-07.cgz/x86_64-lkp/gcc-4.9/9e46a1eba10500df620683e31742789ef2781e93/0
- LKP_SERVER=inn
- |2-
earlyprintk=ttyS0,115200 systemd.log_level=err
debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal
rwlkp_initrd: "/lkp/lkp/lkp-x86_64.cgz"modules_initrd: "/pkg/linux/x86_64-lkp/gcc-4.9/9e46a1eba10500df620683e31742789ef2781e93/modules.cgz"bm_initrd: "/osimage/deps/debian-x86_64-2015-02-07.cgz/lkp.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/run-ipconfig.cgz"job_state: finished
loadavg: 0.13 0.03 0.01 1/373 2718start_time: '1432541537'end_time: '1432541538'
version: "/lkp/lkp/.src-20150525-131232"
Quoted 6 lines · Huang Ying
FYI, we noticed the below changes on
git://git.kernel.org/pub/scm/linux/kernel/git/mlin/linux.gitblock-generic-req
commitfbccf1df06574add60bdb71103cc1662e477f02d("block: makegeneric_make_requesthandle arbitrarily sized bios")
Quoted 7 lines
testcase/path_params/tbox_group: boot/1/vm-kbuild-1G
v4.1-rc4 fbccf1df06574add60bdb71103 ---------------- -------------------------- %stddev %change %stddev \ | \ 1587 ± 7% +276.3% 5972 ± 1% boot-meminfo.KernelStack
I tested with 5 virtual disks.
diff meminfo.v4.1-rc4 diff meminfo.fbccf1df0
-Slab: 16908 kB
-SReclaimable: 8708 kB -SUnreclaim: 8200 kB -KernelStack: 1456 kB -PageTables: 1496 kB +Slab: 17492 kB +SReclaimable: 8860 kB +SUnreclaim: 8632 kB +KernelStack: 2144 kB +PageTables: 1480 kB
The KernelStack increase is because now we create bio pool for each request_queue.
Quoted 9 lines
152390 ± 0% +4.8% 159697 ± 0% boot-slabinfo.num_objs 10149 ± 0% +15.8% 11750 ± 0% boot-slabinfo.num_pages 35916 ± 5% +18.0% 42368 ± 9% boot-meminfo.DirectMap4k 478516 ± 0% -1.9% 469269 ± 0% boot-meminfo.MemFree 11477 ± 0% +49.1% 17111 ± 1% boot-meminfo.SUnreclaim 40551 ± 0% +15.7% 46910 ± 0% boot-meminfo.Slabvm-kbuild-1G: qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smap
Memory: 1G
Could you send me the kernel config and full qemu command? I'll try to reproduce it.
Quoted 22 lines
boot-meminfo.KernelStack
7000 ++-------------------------------------------------------------------+
| O O O O | 6000 O+ O O O O O O O O O O O O O O O O O O | | 5000 ++ | | | 4000 ++ | | | 3000 ++ | | | 2000 ++ *.. | * *..*..*..* : * | 1000 ++ .. + : | | + . + : |0 ++-*--*--*---------------*-------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Quoted 7 lines
To reproduce:
apt-get install ruby
git clonegit://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
cd lkp-tests
bin/setup-localjob.yaml# the job file attached in this email
bin/run-localjob.yaml
I got below error.
root(a)mlin-dev:~/lkp-tests# ./bin/run-local job.yaml /usr/lib/ruby/2.1.0/psych.rb:464:in `initialize': No such file or directory @ rb_sysopen - /result/boot/1/msl-lab-afa03/debian-x86_64-2015-02-07.cgz/x86_64-lkp/gcc-4.9/9e46a1eba10500df620683e31742789ef2781e93/1/job.yaml (Errno::ENOENT) from /usr/lib/ruby/2.1.0/psych.rb:464:in `open' from /usr/lib/ruby/2.1.0/psych.rb:464:in `load_file' from /root/tmp/lkp-tests/stats/kernel-size:7:in `<main>' /root/tmp/lkp-tests/stats/kernel-size < /dev/null exit code 1, check /dev/shm/lkp-stats.70d8CxI7
root@mlin-dev:~# tree /result/
/result/
└── boot
├── 1
│ └── msl-lab-afa03
│ └── debian-x86_64-2015-02-07.cgz
│ └── x86_64-lkp
│ └── gcc-4.9
│ └── 9e46a1eba10500df620683e31742789ef2781e93
│ ├── 0
│ │ ├── boot-meminfo
│ │ ├── boot-meminfo.json
│ │ ├── boot-slabinfo
│ │ ├── boot-slabinfo.json
│ │ ├── job.sh
│ │ ├── job.yaml
│ │ ├── matrix.json.gz
│ │ ├── memmap
│ │ ├── memmap.json
│ │ ├── sleep
│ │ ├── sleep.time
│ │ ├── sleep.time.json
│ │ ├── stats.json
│ │ ├── time
│ │ ├── time.json
│ │ ├── watchdog
│ │ └── watch-oom
│ ├── 1
│ │ ├── boot-meminfo
│ │ ├── boot-meminfo.json
│ │ ├── boot-slabinfo
│ │ ├── boot-slabinfo.json
│ │ ├── job.sh
│ │ ├── job.yaml
│ │ ├── matrix.json.gz
│ │ ├── memmap
│ │ ├── memmap.json
│ │ ├── sleep
│ │ ├── sleep.time
│ │ ├── sleep.time.json
│ │ ├── stats.json
│ │ ├── time
│ │ ├── time.json
│ │ ├── watchdog
│ │ └── watch-oom
│ ├── avg.json
│ ├── matrix.json
│ └── stddev.json
└── params.yaml9 directories, 38 files
Hi, Ming,
Here it is.
qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smap -kernel /pkg/linux/x86_64-lkp/gcc-4.9/fbccf1df06574add60bdb71103cc1662e477f02d/vmlinuz-4.1.0-rc4-00001-gfbccf1d
-append 'root=/dev/ram0 user=lkp job=/lkp/scheduled/vm-kbuild-1G-15/bisect_boot-1-debian-x86_64-2015-02-07.cgz-x86_64-lkp-fbccf1df06574add60bdb71103cc1662e477f02d-1-20150529-44276-1cpbx2s.yaml ARCH=x86_64 kconfig=x86_64-lkp branch=linux-devel/devel-hourly-2015052515
commit=fbccf1df06574add60bdb71103cc1662e477f02d BOOT_IMAGE=/pkg/linux/x86_64-lkp/gcc-4.9/fbccf1df06574add60bdb71103cc1662e477f02d/vmlinuz-4.1.0-rc4-00001-gfbccf1d
max_uptime=600 RESULT_ROOT=/result/boot/1/vm-kbuild-1G/debian-x86_64-2015-02-07.cgz/x86_64-lkp/gcc-4.9/fbccf1df06574add60bdb71103cc1662e477f02d/0 LKP_SERVER=inn earlyprintk=ttyS0,115200 systemd.log_level=err debug
apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=-1
softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2
prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal rw
ip=::::vm-kbuild-1G-15::dhcp' -initrd /fs/sda1/initrd-vm-kbuild-1G-15 -m
1024 -smp 2 -device e1000,netdev=net0 -netdev user,id=net0 -boot order=nc
-no-reboot -watchdog i6300esb -rtc base=localtime -device
virtio-scsi-pci,id=scsi0 -drive
file=/fs/sda1/disk0-vm-kbuild-1G-15,if=none,id=hd0,media=disk,aio=native,cache=none
-device scsi-hd,bus=scsi0.0,drive=hd0,scsi-id=1,lun=0 -drive file=/fs/sda1/disk1-vm-kbuild-1G-15,if=none,id=hd1,media=disk,aio=native,cache=none
-device scsi-hd,bus=scsi0.0,drive=hd1,scsi-id=1,lun=1 -drive file=/fs/sda1/disk2-vm-kbuild-1G-15,if=none,id=hd2,media=disk,aio=native,cache=none
-device scsi-hd,bus=scsi0.0,drive=hd2,scsi-id=1,lun=2 -drive file=/fs/sda1/disk3-vm-kbuild-1G-15,if=none,id=hd3,media=disk,aio=native,cache=none
-device scsi-hd,bus=scsi0.0,drive=hd3,scsi-id=1,lun=3 -drive file=/fs/sda1/disk4-vm-kbuild-1G-15,if=none,id=hd4,media=disk,aio=native,cache=none
-device scsi-hd,bus=scsi0.0,drive=hd4,scsi-id=1,lun=4 -pidfile/dev/shm/kboot/pid-vm-kbuild-1G-15 -serial
file:/dev/shm/kboot/serial-vm-kbuild-1G-15 -daemonize -display none
-monitor null
Quoted 160 lines · Ming Lin
Quoted 3 lines · Huang Ying
FYI, we noticed the below changes on
git://git.kernel.org/pub/scm/linux/kernel/git/mlin/linux.gitblock-generic-req
Quoted 1 line
commit
fbccf1df06574add60bdb71103cc1662e477f02d("block: make
generic_make_requesthandle arbitrarily sized bios")Quoted 9 lines
testcase/path_params/tbox_group: boot/1/vm-kbuild-1G
v4.1-rc4 fbccf1df06574add60bdb71103 ---------------- -------------------------- %stddev %change %stddev \ | \ 1587 ± 7% +276.3% 5972 ± 1% boot-meminfo.KernelStackI tested with 5 virtual disks.
diff meminfo.v4.1-rc4 diff meminfo.fbccf1df0
(quoted diff) +5
-5-Slab: 16908 kB -SReclaimable: 8708 kB -SUnreclaim: 8200 kB -KernelStack: 1456 kB -PageTables: 1496 kB +Slab: 17492 kB +SReclaimable: 8860 kB +SUnreclaim: 8632 kB +KernelStack: 2144 kB +PageTables: 1480 kBThe KernelStack increase is because now we create bio pool for each
request_queue.Quoted 9 lines
152390 ± 0% +4.8% 159697 ± 0% boot-slabinfo.num_objs 10149 ± 0% +15.8% 11750 ± 0% boot-slabinfo.num_pages 35916 ± 5% +18.0% 42368 ± 9% boot-meminfo.DirectMap4k 478516 ± 0% -1.9% 469269 ± 0% boot-meminfo.MemFree 11477 ± 0% +49.1% 17111 ± 1% boot-meminfo.SUnreclaim 40551 ± 0% +15.7% 46910 ± 0% boot-meminfo.Slabvm-kbuild-1G: qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smap
Memory: 1GCould you send me the kernel config and full qemu command? I'll try to reproduce it.
Quoted 4 lines
boot-meminfo.KernelStack
7000
++-------------------------------------------------------------------+
Quoted 1 line
| O O O O
|
Quoted 1 line
6000 O+ O O O O O O O O O O O O O O O O
O O
Quoted 1 line
||Quoted 1 line
5000 ++
|
Quoted 1 line
||Quoted 1 line
4000 ++
|
Quoted 1 line
||Quoted 1 line
3000 ++
|
Quoted 1 line
|| > 2000 ++ *.. |Quoted 1 line
* *..*..*..* : *
|
Quoted 1 line
1000 ++ .. + :
|Quoted 1 line
| + . + :
|Quoted 1 line
0++-*--*--*---------------*-------------------------------------------+
Quoted 8 lines
[*] bisect-good sample
[O] bisect-bad sampleTo reproduce:
apt-get install ruby
git clonegit://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
Quoted 3 lines
cd lkp-tests
bin/setup-localjob.yaml# the job file attached in this email
bin/run-localjob.yamlI got below error.
root(a)mlin-dev:~/lkp-tests# ./bin/run-local job.yaml /usr/lib/ruby/2.1.0/psych.rb:464:in `initialize': No such file or directory @ rb_sysopen -/result/boot/1/msl-lab-afa03/debian-x86_64-2015-02-07.cgz/x86_64-lkp/gcc-4.9/9e46a1eba10500df620683e31742789ef2781e93/1/job.yaml (
Errno::ENOENT)
from /usr/lib/ruby/2.1.0/psych.rb:464:in `open'
from /usr/lib/ruby/2.1.0/psych.rb:464:in `load_file'
from /root/tmp/lkp-tests/stats/kernel-size:7:in `<main>'/root/tmp/lkp-tests/stats/kernel-size</dev/nullexit code 1, check/dev/shm/lkp-stats.70d8CxI7root(a)mlin-dev:~# tree /result/ /result/ └── boot ├── 1 │ └── msl-lab-afa03 │ └── debian-x86_64-2015-02-07.cgz │ └── x86_64-lkp │ └── gcc-4.9 │ └── 9e46a1eba10500df620683e31742789ef2781e93 │ ├── 0 │ │ ├── boot-meminfo │ │ ├── boot-meminfo.json │ │ ├── boot-slabinfo │ │ ├── boot-slabinfo.json │ │ ├── job.sh │ │ ├── job.yaml │ │ ├── matrix.json.gz │ │ ├── memmap │ │ ├── memmap.json │ │ ├── sleep │ │ ├── sleep.time │ │ ├── sleep.time.json │ │ ├── stats.json │ │ ├── time │ │ ├── time.json │ │ ├── watchdog │ │ └── watch-oom │ ├── 1 │ │ ├── boot-meminfo │ │ ├── boot-meminfo.json │ │ ├── boot-slabinfo │ │ ├── boot-slabinfo.json │ │ ├── job.sh │ │ ├── job.yaml │ │ ├── matrix.json.gz │ │ ├── memmap │ │ ├── memmap.json │ │ ├── sleep │ │ ├── sleep.time │ │ ├── sleep.time.json │ │ ├── stats.json │ │ ├── time │ │ ├── time.json │ │ ├── watchdog │ │ └── watch-oom │ ├── avg.json │ ├── matrix.json │ └── stddev.json └── params.yaml9 directories, 38 files
Quoted 6 lines · huang ying
Hi, Ming,
Here it is.
qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smap
-kernel /pkg/linux/x86_64-lkp/gcc-4.9/fbccf1df06574add60bdb71103cc1662e477f02d/vmlinuz-4.1.0-rc4-00001-gfbccf1d -append 'root=/dev/ram0 user=lkp job=/lkp/scheduled/vm-kbuild-1G-15/bisect_boot-1-debian-x86_64-2015-02-07.cgz-x86_64-lkp-fbccf1df06574add60bdb71103cc1662e477f02d-1-20150529-44276-1cpbx2s.yaml ARCH=x86_64 kconfig=x86_64-lkp branch=linux-devel/devel-hourly-2015052515 commit=fbccf1df06574add60bdb71103cc1662e477f02d BOOT_IMAGE=/pkg/linux/x86_64-lkp/gcc-4.9/fbccf1df06574add60bdb71103cc1662e477f02d/vmlinuz-4.1.0-rc4-00001-gfbccf1d max_uptime=600 RESULT_ROOT=/result/boot/1/vm-kbuild-1G/debian-x86_64-2015-02-07.cgz/x86_64-lkp/gcc-4.9/fbccf1df06574add60bdb71103cc1662e477f02d/0 LKP_SERVER=inn earlyprintk=ttyS0,115200 systemd.log_level=err debug apic=debugsysrq_always_enabledrcupdate.rcu_cpu_stall_timeout=100 panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal rw ip=::::vm-kbuild-1G-15::dhcp' -initrd /fs/sda1/initrd-vm-kbuild-1G-15 -m 1024 -smp 2 -device e1000,netdev=net0 -netdev user,id=net0 -boot order=nc -no-reboot -watchdog i6300esb -rtc base=localtime -device virtio-scsi-pci,id=scsi0 -drive file=/fs/sda1/disk0-vm-kbuild-1G-15,if=none,id=hd0,media=disk,aio=native,cache=none -device scsi-hd,bus=scsi0.0,drive=hd0,scsi-id=1,lun=0 -drive file=/fs/sda1/disk1-vm-kbuild-1G-15,if=none,id=hd1,media=disk,aio=native,cache=none -device scsi-hd,bus=scsi0.0,drive=hd1,scsi-id=1,lun=1 -drive file=/fs/sda1/disk2-vm-kbuild-1G-15,if=none,id=hd2,media=disk,aio=native,cache=none -device scsi-hd,bus=scsi0.0,drive=hd2,scsi-id=1,lun=2 -drive file=/fs/sda1/disk3-vm-kbuild-1G-15,if=none,id=hd3,media=disk,aio=native,cache=none -device scsi-hd,bus=scsi0.0,drive=hd3,scsi-id=1,lun=3 -drive file=/fs/sda1/disk4-vm-kbuild-1G-15,if=none,id=hd4,media=disk,aio=native,cache=none -device scsi-hd,bus=scsi0.0,drive=hd4,scsi-id=1,lun=4 -pidfile/dev/shm/kboot/pid-vm-kbuild-1G-15-serial file:/dev/shm/kboot/serial-vm-kbuild-1G-15 -daemonize -display none -monitor null
Here is the fix.
I'll merge it to patch "block: make generic_make_request handle arbitrarily sized bios".
Thanks Ying.
Quoted 1 line
block/blk-sysfs.c +2 -0
@@ -522,6 +522,8 @@ static void blk_release_queue(struct kobject *kobj)blk_trace_shutdown (q); +bioset_free (q->bio_split); +ida_simple_remove (&blk_queue_ida, q->id);call_rcu (&q->rcu_head, blk_free_queue_rcu); }